Stereoscopic vision is based on small differences in both retinal images known as retinal disparities. We investigated the cortical responses to retinal disparities in a patient suffering from occipital epilepsy by recording evoked potentials to random dot stereograms (RDS) from subdural electrodes placed in the parieto-occipito-temporal junction, medial surface of the occipital lobe (pericalcarine cortex) and basal surface of the occipital and temporal lobes (fusiform gyrus). Clear responses to disparity present in RDS were found in the fusiform cortex. We observed that the fusiform responses discriminate the onset from the offset of the stimulus, correlation from uncorrelation, and they show a longer latency than responses found in the pericalcarine cortex. Our findings indicate that the fusiform area is involved in the processing of the stereoscopic information and shows responses that suggest a high level of stereoscopic processing.
Stereopsis is traditionally defined as the perception of depth based on small positional differences, known as retinal disparities. Neurophysiological studies in monkeys showed that there is a widespread distribution of retinal disparity sensitive cells throughout many cortical areas of nonhuman primates. Sensitivity to retinal disparity has been recently found in cells from the inferotemporal (IT) cortex (Janssen et al., 1999, 2003; Uka et al., 2000), which indicates that the ventral visual pathway is involved in this modality of visual perception. It has also been observed that, in monkeys, removal of the IT impaired global stereopsis (Cowey and Porter, 1979).
Cortical activation associated with stereopsis in humans was mostly studied by using functional magnetic resonance imaging (fMRI). These studies basically show that stereoscopic stimuli produces weak or variable activity in areas V1, V2, V3 and MT, strong activity in area V3A, V7 and posterior IPS, and stronger activity in V4d-topo, whereas more complex objects produce activity in the lateral occipital cortex (LO) and the fusiform gyrus (Kwee et al., 1999; Mendola et al., 1999; Backus et al., 2001; Gilaie-Dotan et al., 2001; Nishida et al., 2001; Iwami et al., 2002; Tsao et al., 2003). Scalp visual evoked potentials have also been used to study stereopsis (Braddick and Atkinson, 1983; Norcia and Tyler, 1984), but this technique has limited spatial resolution and therefore it is not appropriate to identify cortical visual areas. On the contrary, subdural electrodes have high spatial and temporal resolution. The implant of subdural electrodes in patients undergoing evaluation for surgery therefore offers a unique opportunity to examine how sensory information is processed throughout the various sensory pathways and how it reaches conscious perception. This technique combined with evoked potential recordings has been used to study several aspects of sensory cortical functions in humans (Ray et al., 1999; Grunwald et al., 2003; Mundel et al., 2003).
The goal of the study reported here was to determine whether some cortical regions from the occipital and temporal lobes are involved in disparity processing, a requirement needed to achieve stereopsis. For this, we studied the cortical visually evoked responses to RDS recorded from subdural electrodes in a patient who suffered from occipital epilepsy. To our knowledge this is the first report to address this question that uses this recording technique combined with RDS in humans.
Materials and Methods
A 47-year-old right-handed woman suffering from medically intractable right occipital epilepsy who referred complex partial seizures with visual aura was scheduled for surgery in order to study her epileptic seizures and evaluate possible further surgical treatment. For this, subdural electrode arrays covering the mesial occipital (MO, 8 × 1 electrodes), lateral occipito-temporo-parietal (LOTP, 8 × 4 electrodes), and basal occipito-temporal (BOT, 6 × 1 electrodes) cortices of the right hemisphere were implanted (Fig. 1). The electrode arrays were made of platinum–iridium (Ad-Tech, Racine, WI) with a 1 mm diameter exposure of each electrode and the electrode centers separated by 10 mm. They were inlaid as fixed rectangular or linear arrays in a thin transparent silastic plate. Shielded cables transmitted the signals to a customized patch panel that allowed the pairing of any two electrodes for recordings. Subdural electrodes were placed over the cortical surface between the dura and the pia-aracnhoid, followed by closure of the dura.
The visual stimuli were projected by a computer-controlled standard multimedia projector onto a frontoparallel flat surface facing the subject at a distance of 115 cm. RDS were used as stimulus to study disparity sensitivity. They were generated by using a conventional personal computer running software developed in our own laboratory (Gonzalez and Krause, 1994). The patient wore red/green eyeglasses and viewed the frontal surface where the RDS were projected. To obtain a separate viewing for each eye red and green dot patterns were used. The picture viewed by each eye was a rectangular area made up of a matrix of 320 × 200 pixels subtending 38.8 × 24° of visual field (‘Background’ in Fig. 2). To generate the RDS, 10% of pixels were bright and 90% were dark. The patient had to fixate a small target with both eyes (0.36 × 0.36°, ‘Fixation target’ in Fig. 2) located in the center of the screen. For evoking cortical potentials, all dots within an area of 1.2 × 1.2° centered within the left hemifield (x = −1.7°, y = 0°; ‘Figure’ in Fig. 2) were shifted in opposite directions producing various negative (crossed) and positive (uncrossed) horizontal disparities. The disparity was maintained for a period of 275 ms. Repetitive disparity presentations were made every 2 s. When the disparity was present a small square was perceived either in front (negative disparity) or behind (positive disparity) the background, whereas with zero disparity no figure was perceived. Disparities ranging from +0.75 to −0.75° were used. Both dynamic (frame change every 1/60th s) and static RDS were used to assess disparity sensitivity. The luminance was 18.0 cd/m2 for the random dot fields (10% bright pixel density). After filtered by the eyeglasses, the luminance was 2.2 cd/m2 for the red pattern and 1.43 cd/m2 for the green pattern.
Recordings started 2 days after the electrodes were implanted and were made for three consecutive days. Direct inspection of the patient was continuously made to ensure the reliability of the procedure. The patient reported to perceive the central ‘Figure’ (Fig. 2) in front or behind the ‘Background’ whenever negative or positive disparity was present in both static and dynamic RDS.
Evoked potentials were recorded from pairs of subdural electrodes by using standard equipment (Viking IV IOM; Nicolet Biomedical Ltd, Wisconsin) from electrodes 2–3, 3–4 and 5–6 of the BOT strip, 3–4 and 4–8 of the MO strip, and 1–2, 3–4, 5–6 and 7–8 of the LOTP grid (Fig. 1). The remaining electrodes did not allow reliable recordings. The recording of the electrode potential started 250 ms before the onset of the stimulus and lasted for 1 s. The trigger for starting each recording was produced by the same computer that generated the visual stimulus. The responses evoked by 100 consecutive stimulus presentations were averaged to obtain a final evoked potential for each type of stimulation. We used a 1 kHz high pass filter and 100 Hz low pass filter (American Electroencephalographic Society, 1986). Electrode impedance was <3 kΩ. The resulting averaged evoked potentials were then analyzed for the presence of deflections related to stimulus responses.
The experiments were conducted in compliance with the relevant laws and guidelines of the Bioethical Committee of our institution. The patient was fully informed about the objectives, details, and risks of the experiment and a written consent was obtained before the recordings documented in this report were made.
The LOTP grid was placed under direct visual inspection covering the postero-lateral surface of the temporal cortex and the junction of the occipito-parieto-temporal cortex. Because of the limitations in the surgical technique, it was not possible to know the exact location of the MO and BOT electrode strips by direct visual inspection. However, the image studies (X-ray, NMR and CT) indicated that the MO strip was covering the pericalcarine cortex with the electrode 1 antero-superior and 8 infero-posterior, whereas the BOT strip was lying along the fusiform gyrus with electrode 1 anterior and 6 posterior. Figure 1 shows the electrode locations, indicating those from which recordings were made.
Responses to RDS were found in those recordings made from electrodes 2–3 of the BOT strip and 4–8 of the MO strip. Talairach (Talairach and Tournoux, 1988) coordinates (x, y, z) were (34, −50, −14) for electrode BOT-2, (49, −55, −16) for electrode BOT-3, (3, −65, 14) for electrode MO-4 and (3, −87, −2) for electrode MO-8. We shall refer to these areas as fusiform and pericalcarine areas, respectively. The recordings made from the remaining electrodes including those from the LOTP array (Fig. 1) did not show detectable responses to disparity.
As Figure 3 shows, recordings made in the fusiform area clearly show a prominent response for a disparity of +0.25°, whereas the remaining disparities evoked either a weaker response or no response at all. The evoked potential peaked at ∼210 ms after the stimulus onset, but there was no response to the stimulus offset. In the pericalcarine area every disparity produced both onset and offset potentials (Fig. 4). In this case the peak of the response occurred at ∼150 ms after the stimulus onset and offset respectively.
To make sure the recorded responses from the fusiform area were due to disparity and not to a change in the dot pattern, we used dynamic RDS that changed the pattern every 1/60th of a second. This stimulus produced similar responses as static RDS (Fig. 3). Additionally, we recorded the responses to two disparities (+0.25 and −0.25°) with and without red/green eyeglasses. In our RDS, when no eyeglasses are used and disparity is present instead of a stereofigure an area of mixed yellow, red and green dots is perceived. When eyeglasses are used and disparity is present, a figure in front or behind the background is perceived. Figure 5A clearly shows how, when eyeglasses are used, a prominent response to +0.25° of disparity is present, whereas there is no response to a disparity of −0.25°. On the contrary, when the eyeglasses are removed there is a similar response to both disparities as shown in Figure 5B.
In order to know whether correlation and uncorrelation had the same significance for both areas, we used uncorrelated RDS as stimulus. Figure 6 shows the responses to uncorrelated RDS in the fusiform and pericalcarine areas. It can be observed that whereas in the pericalcarine area there is a clear response to the onset and offset of the uncorrelated RDS, the response in the fusiform area is absent.
To test hemifield dominance we used a RDS with a stereofigure that was 1.2 × 1.2° in size centred at −1.7,0° for contralateral stimulation and +1.7,0° for ipsilateral stimulation. Figure 7 shows a clear response when the stimulus is presented on the contralateral hemifield whereas a very mild response with longer latency was recorded when the same stimulus was delivered on the ipsilateral hemifield.
Human studies aimed to find cortical areas involved in stereopsis have been made mostly using fMRI (Kwee et al., 1999; Mendola et al., 1999; Backus et al., 2001; Gilaie-Dotan et al., 2001; Nishida et al., 2001; Iwami et al., 2002), PET (Ptito et al., 1993; Gulyás and Roland, 1994; Fortin et al., 2002) or in patients with brain lesions (Ptito and Zatorre, 1988; Ptito et al., 1991). Functional MRI studies showed that areas located in the lateral occipital and parieto-occipital cortices are related to stereoscopic processing. However, by using subdural recordings, we failed to record responses to RDS from the LOPT array. The reason may be that the electrodes of this array did not intersect any of the areas shown to be activated by stereoscopic stimuli in fMRI studies. On the contrary, in our study we provide evidence that the fusiform area is involved in disparity processing. As shown in Figure 3, responses in this area to static and dynamic RDS are highly dependent of the stimulus disparity. We obtained the strongest response for positive disparities which are generated by objects behind the fixation point. This is surprising because near objects are perceptually more salient. Indeed, Gilaie-Dotan et al. (2001) by using fMRI found preferential activation to ‘front’ compared to ‘back’ objects. This discrepancy may be due to the spatial resolution of the subdural technique. Our electrodes had 1 mm diameter exposure separated by 10 mm and therefore they had a restricted area or recording which may have included only fusiform regions with more prominent responses to positive disparities.
We also found responses to static RDS in the pericalcarine area. These responses may be due to the pattern change caused by the onset and offset of the disparity and not by the disparity itself. We cannot determine this because we were not able to measure responses to dynamic RDS from this array.
The responses to RDS we found in the fusiform area were disparity dependent whereas in the pericalcarine area were not (see Figs 3 and 4). One explanation for this finding is that the responses in the pericalcarine area were due to pattern change and not to disparity. Another possible explanation is that subdural electrodes averaged population activity across several disparity columns. In monkeys, at single-cell level, there is disparity sensitivity in areas V1, V2, V3 and V3A (Poggio et al., 1985, 1988; Gonzalez et al., 1993; Durand et al., 2002; Prince et al., 2002); however, disparity sensitivity changes within a few hundred microns. Subdural electrodes do not have enough spatial resolution to detect the disparity selectivity of small cellular clusters and therefore record the activity of a large population of cells simultaneously responding to several disparities and to pattern change. It may be possible that in the fusiform area cells related to positive and negative disparity are arranged in larger patches which can cause the electrodes to record responses selective to a given disparity.
Response latencies to the stimulus onset in the fusiform area peak at ∼210 ms, whereas in the pericalcarine area they peak at ∼150 ms. This longer response delay found in the fusiform area is indicative of a higher level in the processing of the visual information. Whereas responses to RDS in the pericalcarine area may be partially caused by pattern changes, this is not the case for the fusiform area. As shown in Figure 3, there is no response to ‘zero’ disparity nor to most of the tested disparities. This indicates that in this area, the responses we found are caused by the disparity present in the stimulus and not by pattern changes. Moreover, when we used dynamic RDS we recorded similar responses as when we used static RDS (Fig. 3). Further evidence is shown in Figure 5. The upper part of the figure shows the responses to a static RDS with crossed and uncrossed disparities when the patient wore red/green glasses. Note that the elicited response is present only when the figure was perceived behind the background. The lower part of Figure 5 shows the response to the same stimuli when the patient did not wear the eyeglasses and therefore no depth was perceived. In this case the potential was similar in amplitude and shape for both disparities. It is interesting to observe that the morphology of the evoked potential changed, suggesting that disparity triggers a specific pattern of activation of the cell population in the fusiform area.
Correlated and uncorrelated RDS evoke similar responses in the pericalcarine area, while in the fusiform area only correlated RDS produce responses. This finding suggests that this later area represents a high level of the processing of retinal disparities. As Figure 6 shows, while uncorrelated RDS produce clear on- and offset responses in the pericalcarine area, they do not produce any response in the fusiform area. Single-cell responses to uncorrelated stereograms have been reported in early stages of the visual pathways such as areas V1, V2 and V3–V3A of the monkey (Poggio et al., 1988; Gonzalez et al., 1993). It is assumed that at some stage in the hierarchy of the visual areas, the neural mechanisms for disparity selectivity must achieve stereo correspondence by exhibiting selectivity only for correlated random dot stereograms. Although it is likely that the stereo correspondence problem is not achieved solely in the fusiform area, in our study we provide evidence for an area in which this problem may have been solved. This is in agreement with the finding that in monkeys the end-stage of the ventral visual pathway for stereopsis may be the lower bank of the rostral superotemporal sulcus, known as area TEs (Janssen et al., 2003).
We observed that for RDS, while there was a strong response from the contralateral hemifield only a weak and delayed response was obtained from the ipsilateral hemifield (Fig. 7). Single unit recording studies in monkeys indicate that cells from the IT cortex have receptive fields that are larger than in more posterior areas of the ventral visual pathway, that respond stronger at the foveal position and that prefer the contralateral hemifield above the ipsilateral hemifield (Schwartz et al., 1983; Komatsu and Ideura, 1993; Tovee et al., 1994; Logothetis et al., 1995; Missal et al., 1999; Op De Beeck and Vogels, 2000). The preference of IT cells for contralateral stimulation agrees with our observation that the stronger response was obtained from the contralateral hemifield. Thus, at this stage, although most of the stereoscopic processing may already be solved, both hemifields are not yet fully combined. We believe that this may happen at hierarchically higher cortical areas.
It has been suggested that the 3D features of objects are processed in the parietal areas, which belong to the dorsal visual system (Shikata et al., 1996; Sakata et al., 1997). Recently, neurons and regions related to binocular disparity have been found in the IT cortex, which is part of the ventral visual pathway, of monkeys (Janssen et al., 1999, 2003; Uka et al., 2000). These neurons are selective for disparity-defined 3D shapes and in the vast majority of them the selectivity depends on the global binocular disparity gradient and not on the local disparity, indicating a high level of stereoprocessing (Janssen et al., 1999). Our findings support the involvement of the ventral pathway in processing stereoscopic information.
This work was partially supported by grants BFI2001-3206 from the Spanish Ministerio de Ciencia y Tecnologia, 02PXIC-20803PN from the Xunta de Galicia and CIEN-Galicia (FIS-Instituto de Salud Carlos III). We are grateful to Marisol Justo for her help in preparing this manuscript.
1Department of Physiology, School of Medicine, University of Santiago de Compostela, Santiago de Compostela, Spain, 2Service of Ophthalmology, Complejo Hospitalario Universitario de Santiago de Compostela, Santiago de Compostela, Spain, 3Service of Neurosurgery, Complejo Hospitalario Universitario de Santiago de Compostela, Santiago de Compostela, Spain and 4Service of Clinical Neurophysiology, Complejo Hospitalario Universitario de Santiago de Compostela, Santiago de Compostela, Spain