It is often stated that color is selectively processed in cortical area V4, in both macaques and humans. However most recent data suggests that color is instead processed in region(s) antero-ventral to V4. Here we tested these two hypotheses in macaque visual cortex, where ‘V4’ was originally defined, and first described as color selective. Activity produced by equiluminant color-varying (versus luminance-varying) gratings was measured using double-label deoxyglucose in awake fixating macaques, in multiple areas of flattened visual cortex. Much of cortex was activated near-equally by both color- and luminance-varying stimuli. In remaining cortical regions, discrete color-biased columns were found in many cortical visual areas, whereas luminance-biased activity was found in only a few specific regions (V1 layer 4B and area MT). Consistent with a recent hypothesis, V4 was not uniquely specialized for color processing, but areas located antero-ventral to V4 (in/near TEO and anterior TE) showed more color-biased activity.
Some aspects of primate vision are ‘modular’: processed in (partially) segregated channels within the brain. For instance in cerebral cortex, visual motion cues are processed selectively (though not exclusively) in specific layers (e.g. layer 4B of V1), areas (e.g. MT, MST) and columns (Albright et al., 1984; Malonek et al., 1994; Geesaman et al., 1997).
However the hypothesis of modularity is more controversial in other stimulus dimensions, such as color. For instance, in primary visual cortex (V1) it has been reported that color is processed selectively in specialized anatomical compartments (the ‘blobs’) (Livingstone and Hubel, 1984; Tootell et al., 1988; Landisman and Ts’o, 2002) but this claim has been disputed (Lennie et al., 1990; Leventhal et al., 1995) and qualified (Yoshioka and Dow, 1996). A similar controversy persists in area V2, where color may (Hubel and Livingstone, 1987; Roe and Ts’o, 1999; Tootell and Hamilton, 1989; Moutoussis and Zeki, 2002; DeYoe et al., 1994; Xiao et al., 2003) or may not (Levitt et al., 1994; Kiper et al., 1997) be selectively processed in the thin dark cytochrome oxidase (CO) stripes.
Further anterior in extrastriate cortex, the main color-related controversy has centered on area V4, for more than three decades. During this time, several sub-issues emerged (for a review, see Tootell et al., 2003). For instance, are single units in macaque V4 color-selective (Zeki, 1973, 1977) or not (Kruger and Gouras, 1980; Van Essen et al., 1981; Fischer et al., 1981; Schein et al., 1982)? Are ‘V4d’ and ‘V4v’ really retinotopic subdivisions of a common area ‘V4’ (Van Essen and Zeki, 1978; Gattass et al., 1988), or not (Tootell and Hadjikhanim, 2001; Fize et al., 2003)? Is the main color-selective region in human visual cortex homologous with macaque area V4 (Lueck et al., 1989; McKeefry and Zeki, 1997; Bartels and Zeki, 2000), or a different area further anterior (Hadjikhani et al., 1998; Beauchamp et al., 1999; Van Essen et al., 2001)? In macaque, is there one main color ‘center’ as originally proposed (Zeki, 1973, 1977), or more as suggested in human fMRI studies (Beauchamp et al., 1999; Bartels and Zeki, 2000)?
In the macaque (where the relevant cortical areas were originally defined), the original proposal was that color is processed selectively in V4 (Zeki, 1973). A more recent hypothesis predicts that color is not processed selectively in V4, but it is processed in discrete regions(s) anteriorly, in/near TEO (Hadjikhani et al., 1998; Tootell and Hadjikhanim, 2001; Van Essen et al., 2001) and perhaps in related areas further anterior (Komatsu et al., 1992; Buckley et al., 1997). Here we used neuroimaging to test these two proposals about color processing in macaque. This approach allowed us to conduct a common color-luminance test simultaneously in many areas of visual cortex.
Functional activity was labeled in matched conditions, during presentation of an experimental (color-varying) and a control (luminance-varying) stimulus, using a recent version of the double-label deoxyglucose (2L-DG) technique (Geesaman et al., 1997; Vanduffel et al., 2000, 2002a). This technique has excellent spatial resolution (to resolve cortical columns, layers, and small visual areas), high functional sensitivity, and the two resultant activity maps are essentially independent of each other (see Methods). Unlike alternative functional labeling techniques, this deoxyglucose approach is not susceptible to vascular artifacts, and it yields data throughout the brain. Moreover, the 2L-DG technique can be used in awake behaving (rather than anesthetized) macaques.
Studies of color sensitivity often disambiguate color variations from the luminance variations that normally accompany them, by equating the luminance across all stimuli. Recently it has been recognized that macaque and human sensitivity curves differ significantly (Dobkins et al., 2000), so one cannot extrapolate human equiluminance values to macaques. Here we empirically measured the actual ‘equiluminance’ values in each monkey’s cortex using visual evoked potentials (VEPs). This ensured that our color-varying test stimuli were equal in luminance, for both the species and the individuals tested.
Materials and Methods
Surgery and Training
The procedures were similar to those described previously (Geesaman et al., 1997; Vanduffel et al., 2000, 2002a). Three male rhesus monkeys (Macaca mulatta) were tested. First, each was trained to restraint in a primate chair. The head of each monkey was fixed in place using a previously implanted stainless steel device. Eye positions were measured using the scleral eye-coil technique. All surgical procedures conformed to the American and European Guidelines for the care and use of laboratory animals (for details, see Vanduffel et al., 2002a). Muscle relaxants were not used.
Access to water was restricted during the period of testing, and behavioral control was achieved using operant conditioning. The monkeys were rewarded with drops of apple juice for maintaining fixation within a square-shaped central fixation window (1–2° on a side for training, 3° during the DG experiment). Multiple rewards were given at gradually decreased intervals within a given fixation period (maximally 64 s duration), to encourage progressively longer fixation by the monkey. The health status of the monkeys was closely monitored throughout the period of water restriction.
Stimuli were presented only when the monkey was steadily fixating the center of the screen. Each animal was presented with a color-varying grating (experimental stimulus) and an achromatic luminance-varying grating (control stimulus). The color-varying gratings varied along an axis between the purest ‘red’ and ‘blue’ values of the CRT screen. The achromatic luminance-varying grating varied around a white point calculated for the macaque. All gratings had a trapezoidal-shaped waveform, designed to incorporate the advantages of a square wave (which gives optimal contrast at each stimulated location) and a sine wave (which minimizes chromatic aberration at the ‘edges’ of each grating cycle) (see Fig. 1). All stimuli within a given experiment had a common mean luminance (5.9 cd/m2 for white), which was luminance-equated for the macaque. Stimuli were presented on a Brilliance 21A Phillips monitor (70 Hz refresh rate), positioned 44 cm from the monkey’s eyes. The stimuli subtended the full extent of the rectangular screen (50 × 36°).
In previous studies of color selectivity in macaque cortex, it has been common to test with CRT-based red–green color combinations. Here we instead tested with red–blue color combinations, for several reasons. First, the ‘red’ and ‘blue’ are actually closer to complementary colors than the ‘red’ and ‘green’, when comparing dominant wavelengths of the three CRT-based phosphors (∼470, 543, 612 nm), in the human visual system. Second, the ‘fundamental’ human color axes (e.g. ‘red–green’) cannot be safely generalized to macaques, for the reasons described above (e.g. Dobkins et al., 2000). Third, the blue phosphor is spectrally more pure, and probably more saturated, compared with the green phosphor. This is significant because an earlier DG study (Tootell et al., 1988) showed that more saturated stimuli (e.g. blue compared with green) produce higher DG uptake in macaque V1.
In two cases (nos 1 and 3; see below) the color-varying grating was presented when [3H]DG was available in the bloodstream, and [14C]DG was injected into the bloodstream immediately after the luminance-varying presentation began. This order of presentation was reversed in the remaining case (no. 2), to ensure that the color-luminance differences were not confounded by order-of-presentation (or isotope-linked) differences. No such artifactual differences were found here, nor in previous studies with analogous controls (Vanduffel et al., 2002a).
Within each of the three cases, all stimulus variables were equated, except for the experimental manipulation of color versus luminance. All gratings (both color- and luminance-varying) were of low spatial frequency (0.45 cycles/degree), because sensitivity to color is higher than to luminance in that spatial range, in human psychophysics (Van der Horst and Bouman, 1969; Granger and Heurtley, 1973; Watanabe et al., 1976; Mullen, 1985) and macaque physiology (De Valois et al., 1977; Hicks et al., 1983; Thorell et al., 1984; Kruger and Gouras, 1980).
Between cases, we varied several additional parameters, to encompass a range of typical grating configurations. This approach extended the generality of our basic color-luminance results. For instance in experiment 1, the grating was concentric, presented in continuous motion at 0.52 cycles/s. The direction of motion (centrifugal/centripetal) reversed every 32 s. Thus for a given location in visual space, this grating was effectively orientation-specific.
In experiment 2, we used a conventional grating presented at a single (vertical) orientation, across the whole stimulus. Thus this grating was also orientation-specific. Technically, this grating did not move. Instead, to ‘refresh’ the activation, the phase of the grating was randomly changed at 4.4 Hz.
The stimulus for experiment 3 was equivalent to that in case 2, except that the grating orientation was systematically changed every 32 s, between four values spanning the orientation range (45, 90, 135 and 0/180°).
In all three cases, we were able to quantitatively compare the activation due to color, compared to that due to luminance. In case 3, that was the main variable we labeled, since orientation was systematically varied. In cases 1 and 2, the (constant-local-orientation) stimuli also produced corresponding orientation-selective columns in the individual-isotope maps, in areas V1, V2, V3 and VP (e.g. Vanduffel et al., 2002a). However, in the difference images from these latter two cases, the orientation columns subtracted out, nearly completely (see Figs 3–5).
Functional Labeling and Histology
The double-label deoxyglucose (2L-DG) procedures are described elsewhere (Vanduffel et al., 2000, 2002a; Geesaman et al., 1997). Each isotope was injected intravenously (and remotely) while monkeys were awake and fixating to either experimental and control stimulus. A perfusion step removed unbound deoxyglucose. Before freezing, cortical tissue was physically flattened. Later the tissue block was cut parallel with the flattened cortical surface, to reveal images emphasizing the cortical topography. Selected sections were stained for cytochrome oxidase after autoradiography.
Visual area borders were estimated based on gyral/sulcal landmarks, DG-based functional and metabolic differences, histology (e.g. cytochrome oxidase), and retinotopy extrapolated from other individuals (e.g. Vanduffel et al., 2002a,b; Fize et al., 2003; R. Tootell, unpublished data). Residual uncertainties in area localization was estimated to be 1–2 mm in most cases; such uncertainty will be more significant in very thin areas such as V3/VP, but less important in larger areas such as V3A and V4d/v.
Imaging and Analysis
The autoradiographs corresponding to the 3H- and 14C-dominated images were digitized, registered, normalized and processed as described elsewhere (e.g. Vanduffel et al., 2002a). The two isotope-linked activity maps were ∼96% independent of each other, prior to any difference imaging. This near-independence was achieved by: (i) using a much higher 3H:14C ratio than in previous 2L-DG studies, by (ii) greatly truncating the period of second-isotope accumulation followed quickly by a perfusion step, and by (iii) using color film and filters for isotope separation in subsequent autoradiography and analysis.
Here additional statistical analysis was done, similar to that used in human neuroimaging studies (e.g. Worsley et al., 1996). To restrict analysis to the tissue of interest, a digital mask was first calculated for the relevant portion of each section. Then a difference image was calculated for the aligned, normalized, 3H- and 14C-based images. The resultant image was smoothed (kernel size = 2 mm).
Detrending (removing the best fit linear trend from the data) was imposed in order to standardize the data, but this step did not change the ultimate result, except in a single tissue section. The standard error was calculated by subtracting the smoothed difference image from the difference image. This error map was smoothed with a kernel 10 times the width of the initial smoothing kernel. Next, the t-values were computed by dividing the smoothed difference image by the smoothed error. The minimum t-value was calculated for P < 0.05, corrected for multiple comparisons. For each pseudocolor scale, the maximum t-value was set to the highest value in the relevant difference image (usually, color minus luminance).
In humans, VEP reversal potentials have been used as a physiological reflection of individual subjects’ equiluminance values (Previc, 1986). Here we used this technique to define the equiluminance value of each monkey subject, in response to the exact colors used later in the test stimuli.
During the headset implantation, stainless steel skull screws were implanted directly over foveal striate cortex (3.5 cm along the skull from the posterior medial border, immediately dorsal to the nuccal ridge). Reference skull screws were implanted over frontal cortex. These skull screws were not fully covered with dental acrylic, so that later they could be accessed as VEP leads.
During a subsequent recording session, monkeys performed their fixation task while the amplitudes of these VEP signals were measured, in response to sinusoidal variations in the intensity of red and/or blue phosphor combinations, during stimulus counterphase at 4.4 Hz (see Fig. 1E–G). In the macaque, the maximum intensity of the red phosphor imposed the upper limit on the mean luminance used. To define equiluminance for the color-varying (red–blue) grating, the intensity of the blue phosphor was systematically varied until a ‘reversal’ point was found, at which the red and blue phosphors produced equal amplitude responses. The equiluminance value for white was set in the same way.
The VEP-based equiluminance values from different animals were quite similar to each other (red to blue ratio in cd/m2 = 5.98, 6.24 and 5.33, in cases 1, 2 and 3, respectively). However, these macaque equiluminance ratios were also significantly different from analogous measurements in humans. This is consistent with a sensitivity shift to lower wavelengths in macaque as described previously (Dobkins et al., 2000). However the luminance difference in our study was larger than in the previous report, because we used a color pair taken from the spectral extremes (red–blue), whereas Dobkins et al. (2000) tested a color pair spanning a smaller portion of the visible spectrum (red–green).
The percentage of time that each monkey spent fixating was monitored continuously, in real time. The monkey’s gaze remained within the central fixation window either 90, 85 and 95% of the time (average values, for experiments 1, 2 and 3, respectively). There were no significant differences in the average time the monkey spent fixating between the two isotope uptake periods.
Striate Cortex (Area V1)
Consistent with most previous evidence (see above), the color-varying gratings produced higher uptake in the upper layer blobs of layers 2 + 3 and adjacent layer 4A, compared to the control (luminance-varying) grating (see Fig. 2A–C). In the same animals, we found a strong luminance bias in magnocellular-dominated layer 4B (see Fig. 2D–G). For instance, in Figure 2E the (single-orientation) luminance-varying grating produced obvious orientation columns in layer 4B but analogous columns were almost negligible when the animal viewed the equiluminant color-varying grating (Fig. 2D,F,G). This 2L-DG-based ‘color null’ (minima) in layer 4B confirmed that our VEP-based equiluminance values selectively activated the chromatic pathway in macaque. Similar equiluminant color null responses have been reported in area MT (Dobkins et al., 2000), to which layer 4B of V1 projects.
The elaborate laminar variation seen in V1 was largely absent in other areas. Throughout extrastriate cortex, the column contrast was highest in layer 4, tapering off gradually through both supra- and sub-granular layers.
In V2, our 2L-DG results largely confirmed the ‘classical’ model of anatomically segregated color processing (see Fig. 3). In all three cases, color-driven DG activity was distinctively higher than luminance-driven activity within the thin dark CO stripes. More specifically, the DG color columns were localized to the small ‘islands’ of dark CO staining which comprise the so-called thin ‘stripes’.
Within other regions of V2 (the pale ‘interstripe’ region and ‘thick dark’ CO stripes), luminance-driven activity was marginally higher than color activity. Consistent with prior reports (Hubel et al., 1978; Tootell and Hamilton, 1989; Ts’o et al., 1990; Roe and Ts’o, 1995; Vanduffel et al., 2002a,b), the gratings including a single local orientation (cases 1 and 2) produced orientation columns in the thick dark CO stripes, extending into the adjacent pale interstripes (see Fig. 3). Thus in V2, the regions of orientation selectivity correlated inversely with regions of high color sensitivity, as in macaque V1.
Areas VP and V4v
Beyond V2 in the macaque, there is little previous imaging data on color sensitivity. Figure 4 shows representative maps of color selectivity in VP (also known as V3v) and V4v. In both areas, we found discrete color-biased columns. These VP/V4v columns were rather wide and sparsely spaced (as in V2), but they were arranged in a less regular topography (compared with the systematic array in V2). In VP, the color-biased columns tended to be higher in contrast, and more numerous, compared with those in V4v.
Intriguingly, we also found isolated patches of higher (darker) CO activity in VP, V4v and V4d (see Figs 4 and 5). These isolated CO patches often showed higher color selectivity, when compared with the DG results from the same tissue (green arrows, Figs 4 and 5) — just as in the V2 thin stripes (Fig. 3). However, other CO and color-biased columns were not coincident, beyond V2.
Areas V3, V3A and V4d
Single units in macaque V3 (a.k.a. V3d) are reportedly direction-selective, with little overt color selectivity (Burkhalter et al., 1986; Felleman and Van Essen, 1987; but see Gegenfurtner et al., 1997). In fact, this relatively diminished color sensitivity was one of the defining functional properties distinguishing V3 from area VP (V3v) (Burkhalter et al., 1986). Consistent with this, we found comparatively little color bias in V3 (e.g. Figs 5 and 6), and V3 appeared less color-biased than VP (cf. Figs 4, 5 and 6). Relatively higher color bias was found in adjoining areas V3A anteriorly (e.g. Figs 5 and 6) and V2 posterio-laterally (Figs 3 and 4).
In V4d, where there have been so many conflicting reports about color selectivity, we found little overall color bias — even at the most forgiving statistical thresholds, across all six hemispheres tested. However a few large color-biased columns were found in V4d (e.g. yellow regions in Figs 5 and 6, and dark regions in Fig. 10). Interestingly, these few columns were located approximately where high percentages of color cells were reported many years ago by Zeki (1977) — in the anterior bank of the lunate sulcus (e.g. Figs 5 and 6), and the posterior bank of the superior temporal sulcus (Fig. 10). However because such color columns were so rare in V4, a larger sample would be necessary to confirm this observation.
TEO and TE
A recent hypothesis (see above) predicts that color-selective regions are located anterior to area V4, in/near macaque TEO, and perhaps further anterior in TE. Remarkably, we did find higher color selectivity, in both these predicted regions (Figs 7, 8, 9). Although there was some variability in the size and amplitude of these color-biased patches, the color selectivity was statistically significant in both regions, in all six hemispheres tested. One patch of color-biased activity (∼TEO) lay between the posterior middle temporal sulcus (PMTS) and the anterior fundus of the inferior occipital sulcus. A more anterior patch of color-biased activity lay just posterior to the posterior terminus of the anterior middle temporal sulcus (AMTS); for convenience we will refer to it as ‘aTE’ (without assuming it is a genuine cortical area).
Near threshold (at ‘red’ pseudocolor values), the patches of significant color selectivity in ∼TEO and aTE often merged together (see Figs 7–9). However, closer inspection of the raw autoradiographs, and the regions of highest color bias (yellow pseudocolor) in the same figures, showed that the most robust color-selective activity was segregated into multiple distinct columns.
Areas MT and STS
Previous single unit reports (e.g. Thiele et al., 2001) and the results in layer 4B of V1 (e.g. Fig. 2) predict a significant luminance bias in the MT response, at least in response to moving stimuli. This hypothesis was confirmed in case 1, in which the stimuli were moving (see Fig. 10).
In the other two cases we used stationary stimuli, which produced low uptake to both color and luminance stimuli. Both the luminance bias, and the greater response to moving stimuli, were entirely consistent with previous studies of MT in macaque, and MT(+) in humans. In this study, we saw few luminance biases elsewhere in extrastriate cortex.
A Color ‘Center’ in Primate Visual Cortex?
In primate visual cortex, it has been attractive to think in terms of a specific color center (one main area, plus perhaps related areas), in the way that area MT (and its satellites) are considered a primary center for visual motion processing. However prior evidence for such a color center has been difficult to evaluate, because it has been based on different techniques, from different visual cortical areas, in different (human and non-human) species.
In some ways, the human data are the most straightforward to interpret. Clinical damage to a cortical region in/near the fusiform/lingual gyri in occipitotemporal cortex causes an intriguing deficit called ‘achromatopsia’ in which the visual world appears permanently colorless, like ‘various shades of gray’, or ‘dirty snow’ (Pearlman et al., 1979; Damasio et al., 1980). This intriguing perceptual deficit was attributed to damage in a color-processing ‘center’ in that brain location.
Subsequent neuroimaging studies in normal subjects confirmed the presence of color-related activity in the same cortical location, named variously ‘V4’ (Lueck et al., 1989; Zeki, 1990; McKeefry and Zeki, 1997; Bartels and Zeki, 2000), ‘V8’ (Hadjikhani et al., 1998; Tootell and Hadjikhanim, 2001) or ‘VO’ (Wandell, 1999). There is no disagreement about the presence and location of this small color-selective area ‘V4’ (Lueck et al., 1989; McKeefry and Zeki, 1997; Hadjikhani et al., 1998; Beauchamp et al., 1999; Bartels and Zeki, 2000) or related anterior regions (Beauchamp et al., 1999; Bartels and Zeki, 2000). However the retinotopy in and around this region (which helps define the macaque-human homology) is currently under spirited discussion.
Direct comparisons of the human and macaque maps (Hadjikhani et al., 1998; Van Essen et al., 2001) predicts that any hypothetical (color-selective) homologues of the human color area (and satellites) should be located in/near macaque TEO and anterior TE, respectively. This prediction was confirmed here (Figs 7–9). One color-selective patch was in/near TEO, as mapped electrophysiologically (Boussaoud et al., 1991) — just posterior to the PMTS, and anterior to V4 (Figs 7 and 8). The second patch was located ∼2 cm further anterior on the inferior temporal gyrus (Fig. 9).
Previous results support the color selectivity we found in these regions. For instance, experimental lesions in macaque TE/TEO can produce behavioral evidence for a macaque ‘achromatopsia’ (Heywood et al., 1995; Huxlin et al., 2000). Furthermore, discrete patches of color-selective cells were reported in/near ‘aTE’, well before we found such regions here (Komatsu et al., 1992). Our findings here are also consistent with a PET study of attention to color in monkeys (Takechi et al., 1997), although color sensitivity per se was not tested in that study.
Early single unit studies from one investigator (Zeki, 1973, 1977) reported that macaque V4(d) is color-selective. However subsequent quantitative studies concluded that macaque V4d units are not especially color-selective (Kruger and Gouras, 1980; Fischer et al., 1981; Van Essen et al., 1981; Schein et al., 1982). A recent study agreed in general, but allowed for the possibility of ‘hidden’ color selectivity in V4 (Schein and Desimone, 1990). However, subsequent studies found similar ‘hidden’ color selectivity in V1 (Wachtler et al., 2001) — which only re-emphasized the lack of unique color specificity in V4d.
Moreover, lesions of V4d did not produce fundamental deficits in color perception (Heywood and Cowey, 1987; Heywood et al., 1992; Schiller, 1993; Walsh et al., 1993). Thus we were not surprised to find only low-to-moderate color-bias in the present 2L-DG results from V4d (Figs 4–7).
Color Architecture in Lower-tier Areas
The present results strongly support the ‘classical’ blob-and-thin-stripe model of color-processing architecture in V1 and V2 (Livingstone and Hubel, 1984; Hubel and Livingstone, 1987; Ts’o and Gilbert, 1988; Tootell and Hamilton, 1989; DeYoe et al., 1994; Roe and Ts’o, 1999; Landisman and Ts’o, 2002; Moutoussis and Zeki, 2002). However, some single unit reports have not completely confirmed this classical organization (Lennie et al., 1990; Peterhans and von der Heydt, 1993; Levitt et al., 1994; Leventhal et al., 1995; Yoshioka and Dow, 1996; Kiper et al., 1997). How can these discrepant single unit reports be reconciled with the profound and unambiguous organization revealed in the functional anatomy (e.g. Figs 2 and 3)?
One obvious consideration is that single unit sampling is statistically more noisy, compared with functional maps which reflect the activity of thousands or millions of neurons at one time. For instance, one electrophysiological study of color in macaque V1 described only eight histologically confirmed single units within CO blobs (Lennie et al., 1990). Another possibility is that the dendrites (which contribute more to the DG and optical signals) have a more orderly functional organization, compared with the neuronal somas (recorded by the extracellular microelectrodes) which are embedded within this matrix.
Architecture of Color Columns Beyond V1/V2
Where it was present, the limited color-selective activity in V4 was segregated into discrete, relatively large columns, in both V4d (Figs 5, 6 and 10) and V4v (Fig. 4). Other studies (Zeki, 1973; DeYoe et al., 1994; Xiao et al., 1999) also support our current DG evidence for large and isolated color-selective columns in V4. With hindsight, one can imagine how a misleading impression of high color selectivity could have arisen from a series of fortuitous (or unfortunate) single unit probes into such large column(s) in V4d.
In area(s) VP, V4v and V4d, the color columns were often coincident with dark patches of cytochrome oxidase staining (e.g. Figs 4 and 5). This is strikingly reminiscent of analogous relationships in the V1 blobs and the V2 thin stripes (e.g. Figs 2, 3 and 10). However, in V1 and V2 the CO-color patches are essentially always coincident, whereas only some of the CO and color patches were aligned in V4d, V4v and VP. In V4, those coincident regions may receive specific cortical connections from the V2 thin stripes (DeYoe et al., 1994; Xiao et al., 1999).
We had no preconceptions about possible columnar architectures underlying color or luminance sensitivity, but several generalities emerged from the data. Typically, the color-biased activity took the form of discrete patches or isolated columns. Outside these color-biased regions, cortex had a more balanced sensitivity to luminance and color, slightly biased towards luminance in our stimulus configuration. These ‘non-color-bias’ regions were topographically diffuse and widespread, rather than segregated in discrete, isolated columns (e.g. Fig. 3). We found few examples of alternative possibilities (e.g. strongly luminance-biased columns, or topographically diffuse color biases). Simplistically, the general pattern was of isolated color-selective ‘islands’ in a sea of cortex that was responsive to both luminance and color. Magnocellular-influenced area MT (Fig. 10) and layer 4B of V1 (Fig. 2) were the notable exceptions to this rule.
The topography of the color columns showed an interesting trend, when revealed like this across different levels of the visual cortical hierarchy. Recall that in V1, the color-biased ‘blobs’ form a regular, tightly packed, two-compartment (blob/interblob) array (Fig. 2). In V2, the color-biased columns (basically, the thin stripes) become relatively larger and more widely spaced, in a more complex three-compartment (thin/thick/interstripe) matrix (e.g. Fig. 3). In higher-tier areas (especially VP, V3A and V4d/v), we found that this general tendency continued: the DG-labeled color columns were typically spaced sparsely and/or irregularly relative to each other (Figs 4–7 and 10), and the color columns were sometimes quite large. Anterior to V4, some of the columns were ‘re-grouped’ into functional domains (patches of columns) (Figs 7–9) but the columnar topography never returned to a repeating, regularly spaced array (as in V1).
What would produce such topographic changes? One possibility is that more stimulus dimensions (or more complicated dimensions) are coded in correspondingly more columnar systems, in progressively higher-tier areas (e.g. Fujita et al., 1992). Thus in higher-tier areas, the label for a given single type of column (e.g. color) will be ‘crowded out’ by the additional columnar systems, hence appearing more sparsely spaced, and/or irregularly arrayed. This model is consistent with the well-known transformation of simpler retinotopic information (e.g. V1, V2) into more complex information coding (e.g. IT cortex) along the same hierarchy.
This idea also rationalizes an otherwise-puzzling aspect of the color topography. Even in those areas which are demonstrably retinotopic (e.g. V1, V2, V3, V3A, etc.), regions of high color selectivity occupy only a subset of the cortical surface area (e.g. ∼30% of the surface area in the V1 upper layers, ∼23% of the surface area in V2, etc.). With normal trichomatic vision, there is a much wider range of discriminable variation in color than in luminance, at each point in the visual field: shouldn’t this lead to relatively more color columns in these retinotopic areas? However, since the spatial resolution of the color system is so much lower than that of the luminance-based system (e.g. Van der Horst and Bouman, 1969; Granger and Heurtley, 1973; Watanabe et al., 1976; Mullen, 1985), presumably many fewer neurons are required to ‘cover’ equivalent regions of the visual field for color, compared with the number of neurons required for the high-resolution luminance system. This difference in spatial resolution would thus rationalize the relatively sparse distribution of color columns, even in retinotopic areas.
Our stimuli spanned luminance contrasts and color contrasts that were both as large as possible, given the physical constraints involved. Due to the different nature of color and luminance processing, this meant that our effective cone contrast was much (∼1 log unit) larger for the luminance-varying stimulus, compared to that of the color-varying stimulus. Despite this luminance bias in the cone contrast, the columns we uncovered were almost all selective for color rather than luminance. Presumably this color bias would have been even greater if we had equated for cone contrast, rather than for luminance.
The grating stimuli used here probably did not optimally activate all cells in inferotemporal (IT) cortex. It is well accepted that IT single units often respond best to more complicated stimuli. However, most IT cells respond best to different complex stimuli. Thus it was not clear how to optimize our stimuli for all IT cells, nor how to avoid chromatic aberration in such complicated stimuli, etc. In IT cortex, it is gratifying that we were able to reveal as much color- and luminance-biased activity as we did (e.g. Figs 7–9).
Which ‘Primate’ Visual Cortex?
In this study, we found a reassuring correspondence in the cortical maps of color selectivity, in humans compared with macaques. Many previously confusing details were resolved here by testing two simple hypotheses: macaque color maps are (i) different than proposed previously, but (ii) equivalent to those in humans. In other comparisons between macaque and human cortical maps, correspondence has also been the rule rather than the exception. However, specific human–macaque differences are also becoming more well-documented (e.g. Vanduffel et al., 2002b; Fize et al., 2003; Tootell et al., 2003).
Current assumptions about ‘primate’ vision are based on an uncalibrated mix of data from both humans and macaques. To specify exactly how primate visual cortex works, it will be crucial to sort out exactly how visual cortex differs in these two families, and which features remain constant, across ∼30 million years of independent evolution.
We thank Keith Worsley and Denis Fize for statistical expertise and assistance. We also thank C. Fransen, M. Depaep, G. Vanparrys, G. Meulemans, P. Kayenberg and Y. Celis for additional help.
Address correspondence to: Roger B. H. Tootell, Nuclear Magnetic Resonance Center, Massachusetts General Hospital, and Department of Radiology, Harvard Medical School, Boston, MA 02115, USA. Email: email@example.com.