Directing attention to a behaviorally relevant visual stimulus can overcome the distracting effects of other nearby stimuli. Correspondingly, physiological studies indicate that attention serves to filter distracting stimuli from receptive fields (RFs) in several extrastriate areas. Moreover, a recent study demonstrated that lesions of extrastriate areas V4 and TEO produce impairments in attentional filtering. A critical remaining question concerns why lesions of ventral stream areas cause attentional filtering impairments. To address this question, we tested the effects of restricted area V4 and TEO lesions on both behavioral performance and the responses of downstream neurons in area TE. The lesions impaired behavioral discrimination thresholds and altered neuronal selectivity for target stimuli in the presence of distracters. With attention to the target, but in the absence of V4 and/or TEO inputs, TE neurons responded as though attentional inputs could no longer be used to filter distracters from their RFs. This presumably occurred because top-down attentional signals were no longer able to filter distracters from the RFs of the cells that provide TE with major input. Consistent with this interpretation, increasing the spatial separation between targets and distracters, such that they no longer fell within a typical V4 RF dimension, restored both behavioral performance and neuronal selectivity in the portion of TE RFs affected by the V4 lesion.
A typical scene contains many different objects, not all of which can be fully processed by the visual system at any given time. Accordingly, attentional mechanisms are required to limit processing to behaviorally relevant stimuli. Critical attentional mechanisms have been described in all areas of the ventral visual stream (Moran and Desimone, 1985; Chelazzi et al., 1993; Connor et al., 1996; Treue and Maunsell, 1996; Luck et al., 1997; Reynolds et al., 1999; Fries et al., 2001), which is necessary for normal object recognition. Without attention, when two stimuli fall within a cell's RF in extrastriate areas, responses are typically a weighted average of the response to each stimulus alone, with the weight determined in part by relative contrast (Reynolds et al., 1999). Thus, without attention, the information communicated by a cell about any given stimulus is degraded (Reynolds and Desimone, 2003).
By contrast, when attention is directed to one of the two competing stimuli within the RF, inputs from the attended stimulus tend to dominate the cell's response, overriding the effects of relative contrast (Moran and Desimone, 1985; Chelazzi et al., 1993; Motter, 1993; Connor et al., 1996; Luck et al., 1997; Reynolds et al., 1999). This is thought to come about by virtue of top-down inputs to ventral stream areas, from areas involved in the control of attentional selection, such as prefrontal and posterior parietal cortex (for reviews, see Kastner and Ungerleider, 2000; Desimone and Duncan, 1995). In this way, ignored stimuli are effectively filtered out of the RF. This attentional filtering appears to be a multi-stage process, operating over larger and larger RFs as one proceeds from one area to the next.
De Weerd et al. (1999) directly tested whether V4 and TEO neurons are actually necessary for attentional filtering by examining the behavioral effects of removing these areas. Monkeys received ‘mosaic’ lesions of V4 and TEO such that three visual field quadrants were differentially affected by the lesions and one quadrant served as a control. Orientation discrimination thresholds for isolated target stimuli were relatively normal in the lesion quadrants, consistent with the results of other studies that have found normal processing of several different stimulus features in the absence of area V4 (Heywood and Cowey, 1987; Schiller and Lee, 1991; Heywood et al., 1992; Walsh et al., 1992, 1993; Schiller, 1993; Merigan, 1996). However, orientation discrimination thresholds were impaired in the lesion-affected quadrants, but not the normal quadrant, when attended target stimuli were surrounded by high-contrast distracters. Likewise, in a visual search task, Schiller and Lee (1991) found that V4 lesions impair the ability to find a target object in the presence of high contrast distractors. Thus, for normal object discrimination, V4 and TEO appear to be necessary for top-down attentional inputs (presumably deriving from areas outside of the ventral stream) to overcome competition from nearby strong distracters. Similar results have been found in a human subject with a lesion in V4 (Gallant et al., 2000).
These data suggest that in the absence of V4 and TEO, information may continue to reach downstream area TE, but with information about the attended and unattended stimuli averaged together, or unfiltered. However, lesions in one part of a complex, highly interconnected, and adaptable circuit can lead to impaired behavior for reasons that are not the expected ones, e.g. TE might simply be ‘deafferented’ by the V4 and TEO lesions. We therefore directly tested these ideas by recording TE neuronal responses in monkeys discriminating complex target objects. With inputs from V4 and TEO intact, we predicted that distracters would have little influence on the selectivity of TE cells for attended target stimuli, since the distracters should be filtered out. By contrast, with V4 or TEO inputs removed, we predicted that TE neurons would respond to visual stimuli (reaching TE through alternative visual pathways) but that distracters would alter their selectivity for target stimuli, since the distracters would not have been filtered out of the inputs to TE. Furthermore, if V4 and TEO are sites where top-down signals filter distracters from RFs, rather than the ‘source’ of the top-down attentional signals, we predicted that the effects of the lesions would be limited to target–distracter configurations that fit within the RF sizes of these areas.
Materials and Methods
Two monkeys (De Weerd et al., 1999) were used. Surgical and behavioral procedures followed NIH guidelines. Implant surgeries involved the placement of a post to immobilize the head, a recording chamber positioned over area TE in the right hemisphere, and the introduction of an eye-coil in the sclera to monitor eye movements (Robinson, 1963). Comparable lesions were made in the two monkeys by aspiration of the gray matter.
The retinotopic ‘mosaic’ lesions in V4 and TEO have been described previously (De Weerd et al., 1999). Briefly, both monkeys were prepared with a unilateral right hemisphere lesion of area TEO, which affected the left half of the visual field, and with a bilateral lesion of the dorsal portion of V4 (on the prelunate convexity) that affected the lower half of the visual field (Fig. 1a). In this way, we could compare the effects of a lesion restricted to V4 (lower right quadrant), a lesion restricted to TEO (upper left quadrant), and a combined V4 and TEO lesion (lower left quadrant), with a control that received normal visual input (upper right quadrant, Fig. 1b).
Many TE neurons respond better to images of complex, colored objects than to stimuli with simple features that can be varied parametrically, such as the orientation of a bar (Desimone et al., 1984). Thus, we used complex stimuli that could be varied at least monotonically along a morphed stimulus dimension. The target stimuli used were colorful, complex stimuli drawn from a large library of digitally cropped and modified images (photographs, artwork, etc.). Some of the images were clearly recognizable as objects (e.g. human faces, animals, fruits, etc.), whereas others appeared as abstract pictures and patterns, images that have previously been shown to elicit responses from a large proportion of TE neurons (Erickson and Desimone, 1999). No attempt was made to objectively control for the physical qualities of the stimuli (e.g. chromatic spectrum, spatial frequencies, etc.) or for their perceptual similarity. The monkeys were first trained to perform a Go/No-Go discrimination with 10 pairs of complex objects, and were then tested with intermediate morphed images. To construct the intermediate morphed images, we computed the difference between the original stimuli in the red–green–blue (RGB) color space by subtracting, pixel by pixel, each of the R, G and B intensity values of the go stimulus from the no-go stimulus. We could then produce intermediate images by assigning to each pixel RGB values corresponding to an arbitrary proportion of that difference, where 0% is the pixel color of the go stimulus and 100% is the pixel color of the original no-go stimulus. The target stimuli were presented at an eccentricity of 5.8°, within a circular aperture of 2.2°; trials were aborted for eye movements outside a 1.6° square window centered on the fixation point.
The distracter stimuli consisted of white disks of the same diameter as the target stimuli with a contrast of 66% relative to the display background. Distracters were shown in triplets, chosen randomly in each trial from eight predefined distracter configurations. The position of the target stimulus remained fixed within each quadrant. In Experiment 1, distracters were positioned close to the target stimulus (2.5°, center–center from the target stimulus), within the same quadrant of the visual field (Inside Condition, Fig. 1c). In Experiment 2, the distracters were positioned further away from the target (11.6°, center–center from the target stimulus), within the other three visual field quadrants (Outside Condition, Fig. 1d), in Experiment 2.
The monkeys were first trained to discriminate 10 pairs of stimuli. Briefly, the monkeys initiated trials by grabbing a bar, which was followed by the presentation of a fixation spot at the center of the screen. After the monkeys maintained fixation for 300 ms, a target stimulus was presented for 600 ms. The monkeys received a juice reward for maintaining hold on the bar for at least 800 ms after the onset of the stimulus for one of the stimuli in each pair (the ‘no-go’ stimulus) and for releasing the bar within 600 ms after the stimulus presentation for the other stimulus (the ‘go’ stimulus). After reaching 85% correct performance on all 10 pairs of stimuli, the monkeys were presented with the original go and no-go stimuli along with morphed images between the two original stimuli of each pair. Monkeys were rewarded for releasing the bar when the original go stimulus of each pair was presented, and for holding the bar when all other stimuli were presented. Monkeys were tested with these stimuli in each of the visual field quadrants with the targets presented by themselves and in the presence of distracters.
Discrimination thresholds were determined using a staircase procedure (Wetherill and Levitt, 1965). For each trial, a stimulus was chosen by multiplying the percentage of difference from the go stimulus by 0.75 after four consecutive correct responses and dividing this percentage by 0.75 after a single incorrect one. That is, the task becomes easier when the monkey makes an incorrect response, and it becomes harder when the monkey makes four consecutive correct responses. Using this procedure, stimulus differences converge around a level corresponding to 84% correct performance. The measurement ended after a maximum of 120 trials or after 14 reversals (movements up or down in difficulty). The discrimination threshold was calculated as the geometric mean of the morph percentage at all reversal points except the first four, such that each threshold was based on ∼100 trials. Thus, the discrimination threshold describes the percentage of change required in the go stimulus for the monkey to detect a difference.
A typical behavioral testing session consisted of 10 consecutive threshold measurements in each quadrant, with each distracter condition. The order of quadrants and distracter conditions tested was randomized over sessions. Both Experiments 1 and 2 were preceded by training to achieve stable thresholds. The discrimination thresholds reported are averaged over at least 60 independent threshold determinations for each quadrant and distracter condition.
Neurophysiological Recording in area TE
Area TE is the main recipient of the feedforward projections from TEO and from V4 (Distler et al., 1993) and represents the next functional stage in the object-processing hierarchy of the ventral pathway (Ungerleider and Mishkin, 1982). Area TE lacks the retinotopic organization observed in the earlier stages of the visual cortex. The RFs of area TE neurons are often very large, typically including the center of gaze and often extending into both the contralateral and ipsilateral visual fields (Gross et al., 1972; Desimone and Gross, 1979; Schwartz et al., 1983; Desimone et al., 1984; Tanaka et al., 1991; Fujita et al., 1992). These properties allowed us to compare directly neuronal responses to stimuli presented in different quadrants of the visual field, i.e. quadrants with normal input and quadrants that lacked input from areas V4 and/or TEO.
Neurons were recorded either from a single or an array of four sharpened tungsten electrodes. Neurons recorded from single electrodes were isolated on-line, while neurons recorded from the array of electrodes were isolated off-line using commercial spike-sorting software (Plexon Inc., TX). The behavioral task was similar during the recording sessions and during the sessions used to determine discrimination thresholds, the only difference being the number of stimuli presented. In order to accumulate enough trials with each stimulus to determine neuronal responses, we presented only five target stimuli during the recording sessions. After isolating a neuron, we determined which one of the 10 pairs of original (un-morphed) stimuli elicited the most selective neuronal responses (i.e. the largest response differences between the two stimuli in the pair) when these stimuli were presented at fixation. We then presented the go stimulus of that pair, along with the original no-go stimulus (100% change from the go stimulus), and three intervening morphed stimuli (5, 17 and 55% change from the go stimulus, Fig. 3a) that spanned the monkeys' discrimination ability. These five stimuli were presented, in randomized order, in the different visual field quadrants. Within a recording session for a single neuron, blocks of trials in the normal quadrant were randomly interleaved with blocks of trials in a lesion-affected quadrant. Distracter conditions were randomly intermixed within each block. We included in the analysis neurons for which we were able to obtain at least 20 trials with each stimulus, in each quadrant tested, and each distracter condition. We analyzed responsiveness by comparing pre-stimulus firing rates to firing rates during the stimulus presentation (75–275 ms after stimulus onset. We analyzed firing latencies by fitting a Poisson distribution to the average pre-stimulus firing rate and noting the time during stimulus presentation when the firing rate surpassed a 99% confidence interval of the pre-stimulus firing rate for three consecutive bins of 5 ms each.
Although we constructed the morphed stimuli so that they would span the range between a good stimulus and a poor stimulus for the cell, we did not assume that TE cells would necessarily show conventional, single-peaked, tuning curves along such a dimension. The number of target stimuli tested (five) was also small, precluding standard tuning curve measurements such as tuning width, etc. Therefore, we used several measures to assess stimulus selectivity that did not assume conventional tuning curves. These included ANOVA, to test for significant variations in response across the five stimuli, omega squared, which is an estimate of the proportion of variance in the responses that can be explained by the different stimuli and is calculated according to the following formula: ω2 = (regression sum of squares − (degrees of freedom × mean squared error))/total sum of squares + mean squared error, and bits of information which gives a measure of the amount of information carried by a cell about a stimulus set (Kjaer et al., 1994).
To assess the similarity between the pattern of selectivity for the targets presented alone and the responses to the targets with distracters, we calculated the correlation coefficient (r-value) of the mean firing rates to all of the target stimuli across each of the distracter conditions (response window = 75–275 ms after stimulus onset). If a given neuron responded similarly to the target stimuli presented alone and to the same targets surrounded by distracters, the correlation coefficient would be close to one. However, if the neuron showed different selectivity for the targets when presented alone compared to when they were presented with distracters, then the correlation coefficient would be lower.
Experiment 1: Effects of Nearby Distracters
As described above, a previous study found that V4 and TEO lesions impair the ability of monkeys to discriminate a target grating orientation in the presence of distracters (De Weerd et al., 1999, 2003). In the previous study, the discriminanda were relatively simple stimuli, such as oriented gratings. In Experiment 1, we first attempted to replicate the general finding that V4 and TEO lesions impair attentional filtering, using more complex objects as the discriminanda.
With the target stimuli presented alone, there was no significant difference in discrimination thresholds between the normal and lesion quadrants [F(3,346) = 1.72, P = 0.16]. This confirmed the results of the earlier study (De Weerd et al., 1999), but with complex objects as discriminanda. When distracters were added to the targets, thresholds in the normal quadrant were not significantly different (t-tests, P = 0.175). That is, in the normal quadrant, attention to the target was apparently sufficient to filter out the influence of the distracters, so that the sensory information about the target was preserved. However, in all of the lesion-affected quadrants, discrimination thresholds for target images were significantly larger in the presence of distracters than they were when presented alone (t-tests: TEO, P < 0.01; V4, P < 0.001; V4 + TEO, P < 0.001; Fig. 2). There was a significant interaction between quadrant and effect of distracter [ANOVA, F(3,671) = 8.55, P < 0.001], i.e. the distracters differentially impaired performance in the lesion-affected quadrants compared to the control. These data indicate that the lesions did not produce a general visual discrimination deficit, but that the deficits were specific to the case when targets and distracters were both present in the lesion-affected quadrant. Distracter-induced threshold increases were comparable in the visual field quadrant affected by both V4 and TEO lesions and in the visual field quadrant affected by a V4 lesion alone.
We recorded from TE neurons while the monkeys discriminated a set of five complex visual images presented in the quadrants that caused the largest behavioral deficits (V4 alone and combined V4 + TEO). The five target stimuli consisted of an original go stimulus, an original no-go stimulus, and three morphed stimuli that spanned the range from the go to the no-go stimulus (Fig. 3a). To assess whether attention could reduce or eliminate the effects of distracters on the neuronal responses, the targets were presented in two conditions: either alone or surrounded by high-contrast distracter stimuli. During the recording sessions, the distracter conditions were randomly intermixed (Materials and Methods).
We recorded from 127 neurons in two monkeys, and tested for significant responses above baseline firing rates (paired t-test, P < 0.05) in the normal and lesion-affected quadrants (Fig. 2b). Sixteen cells gave significant responses to stimuli only in the normal quadrant, 12 cells responded to stimuli only in a lesion quadrant but not the normal quadrant, and 25 cells were responsive at the fovea, but were not significantly responsive to extrafoveal stimuli in any of the quadrants. The remaining 74 visually responsive neurons gave significant responses to stimuli in the normal quadrant and at least one of the lesion quadrants. Thus, in the absence of inputs from V4 and/or TEO, the large majority of TE neurons remain visually responsive in at least one lesion quadrant. Of these 74 visually responsive neurons, 37 were recorded with stimuli presented in the normal and the V4 + TEO-affected quadrants (monkey M1), and the other 37 neurons were recorded with stimuli presented in the normal and the V4-affected quadrants (monkeys M1 and M2). All of the remaining analyses are based on these 74 cells, using comparisons of responses across quadrants.
For these 74 visually responsive cells, there was very little overall difference in general response properties of neurons for stimuli presented in the different visual field quadrants. We found no significant differences among the mean firing rates (75–275 ms after stimulus onset) for target stimuli presented in the normal, V4 + TEO-affected, and V4-affected quadrants (11.78, 9.73 and 9.59 s/s, respectively, paired t-tests, for all t-tests, P > 0.05). There was also no significant quadrant difference in the firing rates for target stimuli presented with distracters (normal, 10.44 s/s; V4 + TEO, 9.50 s/s; V4, 8.80 s/s; paired t-tests, for all t-tests, P > 0.05). We also found that firing latencies were not significantly different across the quadrants either for target stimuli presented alone (normal, 125.71 ms; V4 + TEO, 127.00 ms; V4, 136.32 ms) or for target stimuli presented with distracters (normal, 132.63 ms; V4 + TEO, 130.66 ms; V4, 127.77 ms; paired t-tests, for all t-tests, P > 0.05).
We also examined the range of responses to the best and worst target stimulus presented alone in each quadrant, using an index of the response to the best stimulus (BS) normalized against the responses to the worst stimulus (WS), according to the formula: (BS − WS)/(BS + WS). As with the average firing rate, we found no significant differences in the range of responses to the best and worst stimulus in the normal, V4 + TEO-affected, or V4-affected quadrants (0.51, 0.46, 0.47, respectively; paired t-tests, for all t-tests, P > 0.05). There were also no differences across quadrants when the stimuli were presented with distracters (normal, 0.47; V4 + TEO, 0.43; V4, 0.36; paired t-tests, for all t-tests, P > 0.05). In sum, the general response parameters in the lesion quadrants were remarkably normal.
Most cells were stimulus-selective, in that they responded better to some stimuli than to others. For some cells, the stimuli eliciting the best and worst responses were the two originally selected as the endpoints of the morphed-stimulus continuum, whereas other cells responded better to one or more of the morphed stimuli between the two endpoints. This was not surprising, as some complex features appeared to be ‘emergent’ in some of the morphed patterns. We do not know which object features were the critical ones for any given cell, nor do we know if the animal attended selectivity to some features over others for a given stimulus, which would likely alter the neuronal responses. Furthermore, we do not know which features or components of these complex stimuli were processed or attended in the lesion versus normal quadrants. The critical feature in the normal quadrant might have been on the left side of a given complex stimulus, for example, whereas the critical feature in a lesion quadrant may have been on the right side of the same stimulus. Likewise, the shape, color, or texture of any given stimulus for any given cell may have been processed differently in the different quadrants, all of which could alter the pattern of selectivity. Therefore, rather than considering the details of stimulus tuning in the different quadrants for individual cells, we focused on the overall magnitude of stimulus selectivity in the population of cells, and how this tuning was affected by distracters, on the average.
We quantified stimulus selectivity in each of the studied quadrants by applying a one-way ANOVA to the mean firing rates to the five target stimuli presented alone. There was a difference between the two monkeys in the proportion of cells that were significantly selective in the lesion-affected quadrants. For monkey M1, 37 recordings with stimuli presented in the normal and V4 + TEO-affected quadrants showed only a small drop in the percentage of selective cells in the V4 + TEO-affected quadrant (81 versus 60% selective, respectively), and the same was true for 22 recordings with stimuli presented in the normal and V4-affected quadrants (68 versus 64% selective, respectively). The proportion of selective cells was not significantly different between the normal and lesion-affected quadrants according to a chi-square test [normal versus V4 + TEO, χ2(1) = 0.951, P = 0.330; normal versus V4, χ2(1) = 0.119, P = 0.730]. For monkey M2, 15 recordings with stimuli presented in the normal and V4-affected quadrants showed a larger decrease, from 93% of cells with selectivity in the normal quadrant compared to only 33% in the V4-affected quadrant, but this did not reach significance [χ2(1) = 0.105, P = 0.746].
We also computed average selectivity values in both monkeys, using average omega-squared values and bits of information (see Table 1). With each measure, monkey M1 showed a modest, but non-significant decrease in selectivity in the V4-affected quadrants considered separately, which reached significance if we pooled the data from both monkeys and both the V4 and V4 + TEO quadrants together.
|Monkey||Quadrant||Distracter condition||% Selectivea||FR indexb||w2c||Informationd||r-valuee||Fisher's zf|
|V4 + TEO||Without||0.595||0.462||0.094||0.098||0.453||0.797|
|Monkey||Quadrant||Distracter condition||% Selectivea||FR indexb||w2c||Informationd||r-valuee||Fisher's zf|
|V4 + TEO||Without||0.595||0.462||0.094||0.098||0.453||0.797|
Bold numbers indicate a significant decrease (paired t-test, P < 0.05) compared to the condition with target stimuli presented alone in the normal quadrant.
The percentage of cells for which a one-way ANOVA revealed a significant effect of stimulus, P < 0.05.
The firing rate (FR) index was computed with the following formula: (best stimulus − worst stimulus)/(best stimulus + worst stimulus).
The omega-squared value describes how much of the variance in the spike rate can be accounted for by the different stimuli.
Information was calculated using the neural network described in Kjaer et al. (1994).
Correlation coefficient between responses to stimuli presented alone (Without) and responses to stimuli surrounded by distracters (With) in each of the tested quadrants.
Correlation coefficients were transformed to normalized Fisher z-scores for statistical analyses.
For both monkeys, the addition of distracters to the targets did not significantly reduce the magnitude of stimulus selectivity in either the normal or the lesion-affected quadrants (Table 1). In the normal quadrant, the tuning was not only similar in magnitude but also seemed to remain relatively invariant in the shape of the curve when distracters were added, i.e. the cells appeared to retain their preference for particular target stimuli when distracters were added to the quadrant, consistent with the idea that the distracters were filtered out of the RFs. However, we noticed that in the lesion-affected quadrants, the stimulus preferences appeared to change when distracters were added, suggesting that the distracters were influencing the cells responses in the absence of V4 or V4 and TEO. Figure 4 shows responses from an example TE neuron to the set of stimuli in the normal and the V4-affected quadrant, with and without distracters. In the normal quadrant, the cell responded selectively, i.e. showed tuning, across the morphed stimulus dimension, and both the pattern of selectivity and the magnitude of responses appeared to be altered very little by the presence of distracters (Fig. 4a,c,e), as expected. In the V4-affected quadrant, the cell also showed stimulus-selective responses to both the target alone and the target plus distracters. However, it appeared that the pattern of stimulus selectivity was altered by the presence of distracters (Fig. 4b,d,f).
To quantify the relationship between the patterns of selectivity across distracter conditions, we calculated the correlation coefficient (r-value) of the mean firing rates to all of the target stimuli presented alone, compared to the same stimuli presented with distracters. For example, the cell illustrated in Figure 4 showed a response correlation between targets alone and targets plus distracter conditions of 0.95 in the normal quadrant (Fig. 4e), which declined to −0.01 for the same comparison in the lesion-affected quadrant (Fig. 4f).
Across the population, the mean correlation between responses for targets versus targets plus distracters was 0.74 in the normal quadrant, indicating that the distracters had only a minimal effect on the pattern of responses to the targets. By contrast, the correlation coefficients in the V4 and V4 + TEO lesion quadrants dropped to 0.46 in both quadrants, indicating that the distracters were more disruptive of the pattern of selectivity (Fig. 5). Paired t-tests revealed that the drop in correlation values was significant in both lesion quadrants (for both t-tests, P < 0.05; r-values were transformed into Fisher z-scores before statistical analyses). The cells recorded from both monkeys showed the same effect, i.e. a significant decrease in correlation was found when the data from each individual monkey were analyzed separately (V4 and V4 + TEO for M1 and V4 for M2). Thus, distracters altered the neuronal tuning to target stimulus features in the lesion-affected quadrants. There was no significant difference in the correlation values between the two lesion-affected quadrants (P = 0.41). Figure 6 shows this effect across the population of recorded neurons. In both monkeys, and in both of the lesion quadrants tested, there was a significant drop in correlation values compared to the normal quadrant. The higher correlation between responses observed when stimuli were placed in the normal quadrant suggests that when inputs to TE from V4 and/or TEO are intact, TE neurons are better able to effectively filter out distracters and respond to the behaviorally relevant stimulus.
Relationship between Selectivity for Targets Alone and Targets plus Distracters
As described above, there was a modest reduction in target stimulus selectivity in the lesion quadrants compared to the normal. A modest decrease in selectivity for the target alone in the lesion-affected quadrant was not surprising considering the loss of V4 and/or TEO inputs to TE. However, it raised the question of whether a reduction in selectivity accounted for the decrease in correlation between responses to the target alone and the target with distracters in the lesion quadrants. Accordingly, we reanalyzed the effect of distracters on neuronal responses for both monkeys and included in the analysis only those cells with significant selectivity for the targets presented alone, in both the normal and lesion-affected quadrants. For these 37 cells, there was still a significant reduction in the correlation between responses to the target alone and the target with distracters in the lesion quadrants (normal quadrant: r = 0.79, Fisher z-score = 1.65; lesion quadrants: r = 0.69, Fisher z-score = 1.30; paired t-test, P < 0.05).
As a further test of the influence of stimulus selectivity, we examined a subgroup of 19 cells with significant stimulus selectivity, with and without distracters, and with similar magnitude of selectivity across quadrants. The omega-squared selectivity values for the target stimuli for this group of neurons were not significantly different across quadrant and distracter conditions (normal quadrant, target alone = 0.27; normal quadrant, with distracters = 0.24; lesion-affected quadrants, target alone = 0.21; lesion-affected quadrants, with distracters = 0.18; paired t-tests, for all t-tests, P > 0.30). However, as was found for the complete set of recordings, this analysis showed a significant drop in correlation values for responses to targets and responses to targets with distracters in the lesion-affected quadrants (normal quadrant, r = 0.94, Fisher z-score = 2.23; lesion-affected quadrants, r = 0.88, Fisher z-score = 1.80; paired t-test, P < 0.05). There was also no evidence that a higher selectivity value, as measured by omega-squared predicted a higher correlation value (Fig. 7). Thus, while the modest drop in stimulus selectivity in the lesion quadrants might contribute to the effects of distracters on selectivity in these quadrants, they cannot fully account for the effect of distracters on tuning. Further support for this conclusion comes from the neuronal recordings in Experiment 2, described below, in which we moved the distracters farther from the target in the lesion quadrant (to a distance that could not be contained within a V4 RF). In this case, the correlations between responses to the target alone versus target plus distracters were restored to normal values, consistent with the idea that the effects of distracters on the pattern of selectivity in the lesion quadrant in Experiment 1 could not be completely accounted for by a simple loss of stimulus selectivity in the lesion quadrants.
Experiment 2: Effects of Increased Target–Distracter Spacing
The stimulus placement and configuration used in Experiment 1 was such that the target and distracter stimuli were contained within a typical V4 RF (∼4.7°) at an eccentricity of 5–6° (Gattass et al., 1988). If attention serves to resolve the competition between stimuli within the same RF, and because different areas along the ventral stream have different RF sizes, then one would predict that a lesion along this pathway would primarily impair attentional filtering at the scale of the missing area's average RF. According to this hypothesis, the V4 lesion impaired attentional filtering in Experiment 1 because the stimuli were spaced so that V4 neurons would have been necessary to mediate the competition between target and distracter stimuli. Therefore, in Experiment 2, we tested this prediction by comparing discrimination thresholds for stimuli with distracters placed inside the same quadrant as the target stimulus (‘inside’ condition, Fig. 1b) to thresholds with distracters placed outside of the quadrant that contained the target (‘outside’ condition, Fig. 1c). If the above interpretation were correct, then both the behavior of the monkeys and the responses of area TE neurons should be normal for stimuli presented in the V4-affected quadrant, whenever the distracters were placed outside of the V4 RF that contained the target.
For both the normal and V4-affected quadrants, we measured distracter-induced increases in the behavioral discrimination threshold. As in Experiment 1, in the quadrant affected by the V4 lesion, discrimination thresholds for target images were significantly higher with inside distracters than when the target was presented alone (t-test, P < 0.001, Fig. 8a). However, this significant increase in threshold was eliminated by presenting the distracters in the outside condition (t-test, P = 0.50), i.e. when the target–distracter spacing was larger than a V4 RF size. In the normal quadrant, there were no significant differences between thresholds with the targets presented alone compared to either the inside or outside distracter condition (t-tests, for both t-tests, P > 0.05), indicating that the animals were very efficient at filtering out distracters at any stimulus spacing. There was a significant interaction between lesion-normal quadrants and inside-outside distracter conditions [ANOVA, F(1,454) = 14.17, P < 0.001]. In contrast to Experiment 1, discrimination thresholds in the V4-affected quadrant were significantly higher than those in the normal quadrant (P < 0.01), even for the target alone condition; however, the increased threshold in the target alone condition in the lesion quadrant was much smaller (37%) than the increase in threshold caused by adding distracters in the inside condition (109%).
The fact that distracters affected discrimination thresholds in the V4-affected quadrant much less when they were presented in the outside condition supports the idea that area V4 is not critical for mediating competition between objects separated by a distance that exceeds the size of V4 RFs. Neurons in area TE, where RFs often extend into all quadrants, are more likely to mediate competition between widely spaced stimuli, such as in the outside condition (Chelazzi et al., 1993). This behavioral effect is consistent with previous findings (De Weerd et al., 1999) that distracter induced deficits caused by V4 and TEO lesions were maximal for stimulus arrays that were roughly the size of their typical RFs and decreased for larger target–distracter spacing. Taken together, these data suggest that neuronal RF size limits the spatial extent over which that visual area is important for resolving the competition between stimuli.
To test whether increasing the target–distracter spacing allows TE neurons to filter distracters from their RFs, we recorded TE responses when the stimuli were presented alone, in the presence of inside distracters, and in the presence of outside distracters. During the recording sessions, these three distracter conditions were randomly intermixed.
We recorded from 56 neurons from the two monkeys, and restricted our analysis to the 41 neurons that gave significant responses above baseline to at least one stimulus presented by itself in both the normal and the quadrant affected by the V4 lesion (t-test, P < 0.05). As in Experiment 1, when stimuli were presented in the lesion-affected quadrant, the addition of inside distracters had a substantial effect on the responses of TE neurons. By contrast, outside distracters did not significantly modify the response of these same neurons.
We quantified the relationship between responses of TE neurons to the targets presented alone and with distracters the same way as in Experiment 1, i.e. by calculating the correlation coefficient (r-value) from the mean firing rate within a 200 ms window for each stimulus across the distracter conditions. Across the population, we compared the correlation coefficients computed between target alone and inside distracter conditions and between target alone and outside distracter conditions when stimuli were presented in the normal quadrant to those correlation coefficients computed when stimuli were presented in the quadrant affected by the V4 lesion. Overall, the correlation values were somewhat smaller in this experiment than in Experiment 1, possibly reflecting the smaller number of cells. Consistent with the results in Experiment 1, the mean correlation between the responses in the target alone and inside distracter conditions was higher in the normal quadrant than in the V4 lesion quadrant (normal, r = 0.61; V4, r = 0.13), which was a significant difference according to a paired t-test (P < 0.001; note that r-values were transformed into Fisher z-scores before statistical analyses). However, in the outside distracter condition, the correlations for stimuli presented in the normal and V4-affected quadrant were identical (normal, r = 0.47; V4, r = 0.47; P = 0.37). Inside distracters altered the selectivity of TE neurons in the lesion quadrants far more than in the outside condition (Fig. 8b). Moreover, a two-way analysis of variance revealed a significant quadrant by distracter condition interaction [F(1,160) = 5.4, P = 0.02]. Additionally, there was no significance difference in correlation values in the normal quadrant between inside and outside distracter conditions (P = 0.19). Moving the distracters farther from the target in the lesion quadrant restored the correlations in responses to normal values. Consistent with the behavior of the monkeys, TE neurons were better able to filter out the distracter stimuli in this condition.
Location of Recording Site
In both monkeys, the recording site was located in the right hemisphere, ipsilateral to the normal and the V4-affected quadrants, and contralateral to the TEO-affected and V4 + TEO-affected quadrants. Accordingly, any preference of TE neurons for contralateral stimuli would presumably give a preference to stimuli presented in the V4 + TEO-affected quadrant, compared to the normal quadrant, working against our hypothesis. Consequently, it is unlikely that the reduction in correlation with inside distracters in the lesion-affected quadrants was due to the location of the recording site.
Location and Extent of Lesions
The lesions were intended to include the lower field representation of area V4 bilaterally (Gattass et al., 1988) and all of area TEO in the right hemisphere (Boussaoud et al., 1991), based on previously published visuotopic maps of these areas relative to sulcal landmarks. This resulted in a lower right quadrant of the visual field affected by the V4 lesion, an upper left quadrant of the visual field affected by the TEO lesion, and a lower left quadrant of the visual field affected by both the V4 and TEO lesion. The upper right visual field quadrant was unaffected by any lesion and served as the control (normal) quadrant. This particular combination of lesions was chosen because the upper and lower field representations of area V4 are well separated (beyond the fovea) and relatively easy to remove independently, but the map of area TEO is compact and, thus, it is easier to remove its upper and lower field representations together in a single hemisphere.
Lesion reconstruction was based on coronal slices obtained with MRI (GE 1.5 T, 1 mm thick, 256 × 160 or 265 × 192 matrix, FOV = 10–11 cm). The lesions of V4 and TEO in these monkeys have been described previously (De Weerd et al., 1999). Briefly, the TEO lesion in monkey M1 was as intended and included the tissue on the posterior-lateral surface of the inferior temporal gyrus of the right hemisphere. The caudal border was 3 mm posterior to the tip of the inferior occipital sulcus (IOS) and the lesion extended rostrally for 11 mm to about the rostro-caudal midpoint of the occipito-temporal sulcus (OTS). While tissue in the ventral bank of the superior temporal sulcus (STS) was largely spared, the lesion included the lateral bank of the OTS for 3 mm in the rostro-caudal plane. The V4 lesion began at the anterior tip of the IOS bilaterally and extended caudally up the prelunate gyrus for ∼9 mm. The tissue removed included the prelunate gyrus and adjacent cortex. Tissue in the lateral bank of the IOS and in the ventral bank of the STS was largely spared. The lesion in monkey M2 was slightly larger and included some encroachment by the TEO lesion into the posterior portion of area TE. This encroachment was not significant enough to alter discrimination performance in the normal quadrant of monkey M2. Discrimination thresholds without distracters in the normal quadrant were not significantly different between the two monkeys (M1, 7.8; M2, 10.4; unpaired t-test, P > 0.05).
In the behavioral testing, object discrimination thresholds for isolated targets were largely unaffected by the lesions; however, in the lesion-affected quadrants, but not the normal quadrant, these same thresholds were elevated in the presence of nearby, high-contrast distracters. Discrimination thresholds with distracters were doubled, on average, compared to those in the normal quadrant. These results with complex objects extend those of an earlier study, which had tested behavioral thresholds with simple target stimuli, such as oriented gratings, colored disks, and moving dots (De Weerd et al., 1999, 2003).
The neuronal recordings revealed, first, that TE neurons continue to respond to stimuli in the lesion quadrants, indicating that lesions of V4 and TEO do not disconnect TE from visual inputs. Indeed, the visual response properties of the cells in the lesion-affected quadrants tested with the target stimuli presented alone were remarkably similar to those in the normal quadrant, consistent with our behavioral results in this condition. Second, the results suggest that the behavioral impairments found with distracter stimuli in the lesion quadrants could be attributed to the loss of attentional filtering by downstream neurons in area TE. TE neurons showed a high correlation between responses to the targets presented alone compared to the responses to targets plus distracters in the normal quadrant, consistent with effective attentional filtering in the normal quadrant. However, consistent with the behavioral deficit, the same cells showed a disrupted pattern of selectivity for target stimuli in the presence of distracters in the lesion-affected quadrants. Furthermore, this altered pattern of selectivity was found only when the target–distracter spacing in the V4 lesion quadrant was similar to V4 RF size at the same eccentricity; attentional filtering with large target–distracter spacing was intact. This suggests that top-down attentional inputs to TE itself remain intact following V4 and TEO lesions, consistent with our own behavioral results and those from a previous study (De Weerd et al., 1999). These top-down attentional inputs to TE are thought to arise from areas outside of the ventral stream, such as prefrontal and/or posterior parietal cortex (see Desimone and Duncan, 1995). Together, the results suggest that, in the absence of V4 and/or TEO, TE neurons do not effectively filter out distracter stimuli from their RFs when the target–distracter spacing is on the scale of V4 or TEO RFs. Thus, one way of viewing the behavioral and neuronal losses following V4 and TEO lesions is that while ‘attention’ per se is intact, the attentional resolution, or acuity, is degraded.
Accordingly, these data suggest that, while visual information can still reach area TE via alternative pathways following V4 and TEO lesions, these pathways are unable to carry out the critical filtering of distracters at high resolution normally carried out by V4 and TEO. These pathways might be from V1 or V2 through the pulvinar, or through areas in the superior temporal sulcus, the parahippocampal gyrus, or the dorsal stream (Boussaoud et al., 1990; Webster et al., 1991, 1993, 1994; Baleydier and Morel, 1992; Martin-Elkins and Horel, 1992; Gattass et al., 1997; Saleem et al., 2000).
How can monkeys discriminate isolated stimuli so well in the absence of V4 and/or TEO? One possibility is that monkeys discriminate the objects based on local features or components of the target stimulus, which might be sufficiently processed by neurons in areas such as V2. Lesions of V4 are known to preferentially impair discrimination of objects based on global or multiple features, rather than local cues (Schiller, 1995; Merigan, 1996, 2000a; Merigan and Pham, 1998; De Weerd et al., 2003). We did not test object discriminations that required complex grouping operations, complex texture discriminations, or recognition across size transformations, which are known to be affected by V4 lesions (Schiller and Lee, 1991; Merigan, 2000b). Accordingly, information from early visual areas such as V2, which represent only a small piece of the stimulus within individual RFs, might be sufficient to allow for apparently normal behavioral performance in the absence of distracters. Similarly, indirect input from these early areas (e.g. through the pulvinar) might be sufficient to drive visually responsive and selective responses in area TE neurons. Why top-down attentional inputs to neurons in these alternative pathways between V1/V2 and TE are apparently not able to compensate for the attentional filtering functions of V4 and TEO is unknown. Possibly, in the absence of V4 or TEO, neurons along the remaining pathways to TE have RFs that are too small to resolve the competition between targets and distracters at the same scale as V4 and TEO, or they may not receive the necessary types of top-down inputs.
We considered the possibility that the reduced correlation between responses to the targets alone and targets plus distracters in the lesion quadrants might be caused by reduced stimulus selectivity for the targets in the lesion quadrants. For this analysis, we focused on the average magnitude of stimulus selectivity in the lesion-affected and normal quadrants rather than the details of stimulus tuning for each cell and each stimulus in the different quadrants, because the stimuli used were highly complex and we did not know which features or components of the stimuli were critical for any given TE cell. We therefore cannot say whether a given cell was more selective to the shape or texture of a given complex stimulus in the normal quadrant or more selective to the color in a lesion quadrant, for example. On average, however, the overall magnitude of stimulus selectivity for the targets alone seemed only modestly affected in the lesion quadrants. There was a significant loss of selectivity in only one quadrant of one monkey (the V4-affected quadrant of monkey M2) or if we pooled the data across the remaining lesion-affected quadrants. When we compared only those cells with significant target selectivitiy in both the normal and lesion-affected quadrants, we continued to find a significant drop in correlation values with distracters in the lesion quadrants. Furthermore, the smaller correlation values with nearby distracters in the lesion quadrants were restored to normal values when the distracters were moved farther from the target. Taken together, the results suggest that while there may be some loss of selectivity in the lesion-affected quadrants, this loss cannot fully account for the decrease in correlation values with nearby distracters, i.e. the loss of attentional filtering. Although V4 and TEO lesions are known to impair some types of high-order visual discriminations (Cowey and Gross, 1970; Merigan, 1996; Huxlin et al., 2000) with an isolated target stimulus, their effects on attentional filtering appear to affect a wider range of object features.
Likewise, we also considered whether the behavioral and neuronal impairments caused by adding distracters in the lesion quadrants were due to impaired attentional filtering per se or whether they reflected impaired sensory processing of the target–distracter configurations. An inability to discriminate a target among distracters is sometimes referred to as ‘crowding’, which tends to occur with very small target–distracter spacing, with peripheral stimuli, and with similar target and distracter features (Nazir, 1992; Toet and Levi, 1992; Kooi et al., 1994). Crowding appears to reflect sensory and/or attentional resolution limits, depending on the stimulus configuration, and it is made worse by amblyopia (Levi and Klein, 1985; Intriligator and Cavanagh, 2001). Because it is conceptually difficult to measure sensory discrimination abilities in the absence of attention to the target stimulus except under dual task conditions (Braun, 1994), we were not able to measure the monkeys' ability to discriminate the target among distracters in the absence of attention, as a comparison. Likewise, we were not able to measure the sensory responses of TE to the targets and distracters in an ‘unattended’ condition (e.g. with attention directed at a distant point in the visual field) because TE neurons typically respond very poorly to unattended stimuli (Moran and Desimone, 1985). Indeed, our present results with large target–distracter spacing suggest that attentional filtering of responses to distracters at distant locations is largely intact in TE following V4 and TEO lesions, presumably because the top-down inputs to TE remained intact. In the absence of such ‘unattended’ comparison data, we cannot rule out a sensory explanation. However, in a recent extensive behavioral study, we tested several different hypotheses about complex sensory impairments in the lesion quadrants of the same two monkeys (De Weerd et al., 2003) that might account for the effect of distracters, and we ruled out all but the attentional filtering account. Here, we can only say that both the behavioral and neural impairments in the lesion quadrants occurred under conditions where attention to the target was very effective in eliminating interference from the distracters in the normal quadrant. The present neuronal results are consistent with the attentional filtering account but do not rule out purely sensory explanations based on complex, possibly non-linear, interactions between targets and distracter features in the lesion quadrants.
Likewise, we do not believe that the impairments caused by the distracters were caused by a loss of top-down attentional mechanisms per se. In the absence of V4 and TEO, the animals were presumably able to ‘attend’ to the stimuli, and these top-down attentional signals presumably modulated processing in both TE and other visual areas. The present data suggest that what was lost in the lesion quadrants were the sites where the top-down attentional signals modulated object feature processing on the scale of a V4 RF. If the lesions were at the ‘source’ of the top-down signals, this should have had a much broader impact on attentive visual processing and would not be limited to a given RF size. Consistent with this, we found similar attentional filtering impairments, but with targets and distracters spaced anywhere throughout the visual field, in a patient with a large, bilateral lesion of parietal cortex, which is thought to be a source of top-down attentional control (Friedman-Hill et al., 2003).
Previous physiological studies give some insight into the nature of sensory processing in the absence of attentional filtering (Chelazzi et al., 1998; Reynolds et al., 1999). When attention is directed elsewhere in the visual field, neuronal responses to a pair of stimuli in a V4 RF are approximately a weighted average of the response given to each stimulus presented alone. One possible interpretation of this averaging is that in the absence of attention, V4 neurons treat multiple independent stimuli as a single compound stimulus with multiple features. However, with attention directed to just one stimulus in the RF, the neuronal response is similar to the response to that same stimulus presented alone.
Here we find that in the absence of this attentional mechanism in V4 and TEO, e.g. when stimuli were presented in the lesion-affected quadrants in the presence of nearby distracters, the stimulus selectivity of most TE neurons was altered. There was not a substantial loss of stimulus selective responses; rather, the preferences of the cells for the different target stimuli were changed. Specifically, there were decreased correlations between firing rates to targets presented alone and firing rates to targets plus nearby distracters. This would be expected if, when distracters were added to the target, the cells responded as though the target and distracters formed a new, compound stimulus. Because for most neurons, the responses to some effective target stimuli increased and the responses to other effective targets deceased by the presence of the same distracters, TE neurons were probably not performing a simple averaging of responses to the targets and distracters presented alone. If the cells were treating the target plus distracters as a single, compound stimulus with multiple features, then the feature-interaction effects on responses in area TE must be nonlinear. This is not surprising, as the perception of complex objects with multiple features cannot easily be described as the linear combination of percepts of individual features (Kobatake and Tanaka, 1994). In sum, the results indicate that in the absence of V4 or V4 + TEO, the effect of distracters is not to suppress the TE response to relevant target stimuli but to degrade the response in complex ways.
We are grateful to Barry Richmond for assistance with the information analysis. We are grateful to K. Montgomery and H. Thakar for help with the behavioral testing of the monkeys and P. DeWeerd, B. Jagadeesh and C. Erickson for discussions of the data.
1Laboratory of Neuropsychology, NIMH, 49 Convent Drive, Bethesda, MD 20892, USA and 2Laboratory of Brain and Cognition, NIMH, 10 Center Drive, Bethesda, MD 20892, USA 3Current address: University of Verona, Verona, Italy