Oscillatory synchrony has been proposed to dynamically coordinate distributed neural ensembles, but whether this mechanism is effectively used in neural processing remains controversial. We trained two monkeys to perform a delayed matching-to-sample task using new visual shapes at each trial. Measures of population-activity patterns (cortical field potentials) were obtained from a chronically implanted array of electrodes placed over area V4 and posterior infero-temporal cortex. In correct trials, oscillatory phase synchrony in the beta range (15–20 Hz) was observed between two focal sites in the inferior temporal cortex while holding the sample in short-term memory. Error trials were characterized by an absence of oscillatory synchrony during memory maintenance. Errors did not seem to be due to an impaired stimulus encoding, since various parameters of neural activity in sensory area V4 did not differ in correct and incorrect trials during sample presentation. Our findings suggest that the successful performance of a visual short-term memory task depends on the strength of oscillatory synchrony during the maintenance of the object in short-term memory. The strength of oscillatory synchrony thus seems to be a relevant parameter of the neural population dynamics that matches behavioral performance.
Oscillatory synchrony is one of the mechanisms that have been proposed to coordinate the functional interactions between multiple neural ensembles (Singer and Gray, 1995). In animals, it has been observed in a number of perceptual (Kreiter and Singer, 1996; Fries et al., 1997; Friedman-Hill et al., 2000; Gail et al., 2000), visuo-motor (Bressler et al., 1993; Roelfsema et al., 1997; von Stein et al., 2000) and attentive (Murthy and Fetz, 1996; Steinmetz et al., 2000; Fries et al., 2001) states of cats and monkeys. Short-term memory, the fundamental ability of the brain to maintain representations of sensory stimuli over a few seconds, is a cognitive process for which oscillatory synchrony could be of particular relevance: oscillatory synchrony could (i) coordinate activity in the distributed memory network (Goldman-Rakic, 1995; Fuster, 1997) and (ii) establish reverberatory loops enabling persistent activity in the system in the absence of sensory input, as postulated by Hebb (1949). Holding visual information in short-term memory is indeed accompanied by oscillatory synchrony in the beta range (15–20 Hz) between distinct extrastriate ventral visual areas in human intra-cranial data (Tallon-Baudry et al., 2001) and by local, within-area synchrony at a higher frequency in the monkey parietal cortex (Pesaran et al., 2002). But does neural processing effectively use this temporal pattern? Whether oscillatory synchrony plays a significant functional role or should be considered as an epiphenomenon remains a highly controversial issue (Shadlen and Movshon, 1999).
A particularly convincing approach in establishing the behavioral relevance of a neural mechanism is to show that its presence is associated with the correct performance of the task, while its absence coincides with behavioral errors. For instance, the close match between mean firing rates in area MT and performance in a visual motion discrimination task (Newsome et al., 1989) shows that a rate code carries behaviorally relevant information. We adopted this approach to test whether oscillatory synchrony is likely to play a functional role in maintaining information in short-term memory. In two monkeys trained to perform a short-term memory task (Fig. 1), we test the prediction that errors are associated with reduced synchrony during memory maintenance as compared to trials in which the monkey responds correctly. Since the location of the cortical sites engaged in oscillatory synchrony in this task is not known a priori, we recorded cortical field potentials using a chronically implanted grid of epidural electrodes covering extended regions of the early ventral visual pathway. With this technique population activity can be monitored in distinct cortical regions simultaneously. It is particularly well suited to detect a collective network behavior such as oscillatory synchrony, because it provides an estimate of coordinated synaptic activity in large but localized neural populations (Hughes, 1964; Bullock and McClune, 1989; Barth and MacDonald, 1996; Freeman and Barrie, 2000; Rols et al., 2001).
In support of a functional role of oscillatory synchrony in short-term memory, we found in both monkeys that two sites located over the posterior infero-temporal cortex (IT) were synchronized in the beta range during memory maintenance in correct trials, but that synchrony failed to develop in incorrect trials.
Material and Methods
Two male macaque monkeys (Macaca mulatta) were trained to perform a delayed matching-to-sample task (Fig. 1). The monkeys sat in a primate chair with the eyes 83 cm in front of a CRT screen (refresh rate 75.3 Hz). To begin a trial, the monkeys pressed a lever at the appearance of a bright 0.14° fixation spot on a gray background. After 800–1020 ms, the sample stimulus was presented for 400 ms, followed by a delay of 800, 1000 or 1200 ms. The three delay durations were randomized to avoid the anticipatory activity that accompanies a temporally predictable sequence of events (Tallon-Baudry et al., 1999; Moody and Wise, 2000). The test stimulus was presented for 820 ms. If it matched the sample, the monkey had to release the lever within the 820 ms of the test stimulus presentation to obtain a drop of fruit juice. In non-match trials, the monkey had to withhold the lever release until 800 ms after test offset to be rewarded. Fixation had to be maintained throughout the trial within a rectangular window of with a total width of 1–1.3° and total height of 1–1.2° of visual angle centered on the fixation spot, as measured by an infrared oculometer. If the monkey deviated from fixation or released the lever either too early or too late, the trial was automatically stopped and the inter-trial waiting period of 3–4 s started without reward. Only errors corresponding to misses (withholding the response in match trials) or false alarms (responding during test presentation in non-match trials) were included in the data analysis of incorrect trials. Errors corresponding to a lever release during the delay or to a loss of fixation were excluded from further analysis.
Stimuli were smooth black shapes (Fig. 1) subtending between 1.8 and 2.5°. A new pair of stimuli was computed at each trial by randomly selecting the radial position of 12 anchoring points and using an interpolation based on Lagrange’s polynomial, as described earlier (Tallon-Baudry et al., 1998), with a coefficient of modulation (difference between stimuli in non-match trials) of 23.5% in monkey No. 1 and 22% in monkey No. 2. To avoid any bias due to the stimulus construction algorithm, the sample stimulus was chosen randomly as the first or the second stimulus of the pair. The remaining stimulus was used as the test stimulus in non-match trials.
Anesthesia was induced with an injection of ketamine (10 mg/kg i.m.) and continued with 1–3% isoflurane in oxygen/nitrous oxide (30/70) after tracheal intubation. In a first surgery, the headpost was fixed with bone screws, bone cement and dental acrylic over frontal parts of the skull. During a second surgery, a cranial window was opened above the left prelunate gyrus, extending from 10 mm posterior to 8 mm anterior to the inter-aural line. The electrode array (see below) was inserted between dura and bone through this window. The trepanation was closed with hydroxyapatite cement and covered with dental acrylic cement. After surgery, monkeys were treated with antibiotics and allowed 6 and 2 weeks recovery after the first and second surgery, respectively. All surgical procedures were performed in accordance with the guidelines for the welfare of experimental animals issued by the federal government of Germany.
Electrode grid location was estimated from structural magnetic resonance images obtained on a 4.7 T MRI scanner (Bruker, Ettlingen, Germany). In addition, electrophysiological retinotopic mapping was performed during a simple fixation task using white flashed squares (0.7 × 0.7°) to functionally localize area V4. The combination of stereotaxic coordinates during surgery, MRIs and electrophysiological retinotopic mapping allowed us to localize the pair of electrodes of interest in the vicinity of the superior temporal sulcus, close to the inter-aural line (Fig. 2a,b). If further data improving anatomical localization become available, they will be presented at the authors’ website.
Recordings and Data Analysis
The electrode array consisted of a sheet of silicone 0.1 mm thick (Goodfellow), in which platinum–iridium (90% Pt–10% Ir) wires (diameter 50 µm, Teflon coated; Science Products), were inserted with a regular spacing of 3 mm. The electrode contact was an uninsulated loop (diameter 250 µm) lying parallel to the dura. The ground and reference electrodes (150 µm diameter platinum–iridium wires) were inserted epidurally over the central and anterior frontal regions respectively. The signals were amplified (×40 000, 1–150 Hz bandwidth) and continuously recorded at a sampling rate of 1 kHz.
To quantitatively assess the relationship between neural activity and performance, a sufficient number of error trials must be obtained, which requires a very large number of trials in animals working at a high level of performance. Because the electrode array is chronically implanted, data from several recording sessions (two in monkey No. 1, three in monkey No. 2) could be concatenated after checking the stability of both the neural responses and behavioral performance for exactly identical stimulus parameters from one day to the other. We obtained 312 and 318 incorrect trials in monkey No. 1 and monkey No. 2, respectively. To balance the number of correct and incorrect trials, we first restricted our analysis of correct trials to a selected subset of 312 (respectively 318) correct trials embedded into long periods of high performance. Practically, a criterion on performance was adjusted until the requested number of correct trials was reached (performance >90% on 30 consecutive trials in monkey No. 1, >95% on 33 consecutive trials in monkey No. 2). To further generalize our results to any subset of correct trials, synchrony was computed on 10 000 subsets of 312 or 318 correct trials randomly chosen among the ∼1100 correct trials available. Having thus estimated the distribution of synchrony by this randomization method, we could test whether the null hypothesis (i.e. synchrony in error trials comes from the same population) could be rejected or not at the corresponding level of significance, using Fisher’s one-sample randomization test (Manly, 1991). The principle of such a randomization is to estimate the distribution of the population by selecting randomly subsets of data. If the value to be tested comes from the same population, it should appear as a typical value from the randomization distribution.
To suppress the effect of the common reference and minimize spatial smearing (Nunez et al., 1997), the second spatial derivative (Laplacian operator) was computed on the high-pass filtered data (Butterworth IIR filter, cutting frequency 6 Hz at 3 dB; forward and backward filtering to avoid phase-shifts) using third order spline functions (Perrin et al., 1987). For each single trial, data were analyzed in the time-frequency domain by convolution with complex gaussian Morlet’s wavelets with a ratio f/σf of 10, with f the central frequency of the wavelet and σf its standard deviation in frequency, the frequency ranging from 8 to 100 Hz in 1 Hz steps (Tallon-Baudry and Bertrand, 1999). The wavelet duration at 18 Hz is 2σt = 177 ms. At each time t and frequency f the result of the convolution for trial j is a complex number:
where A represents the amplitude of the signal and φ its phase. To identify electrode pairs showing a difference between correct and incorrect trials, we first computed a synchrony factor between electrodes k and l across n trials at each latency and frequency as:
This synchrony factor (Lachaux et al., 1999) varies between 0 (independent signals) and 1 (constant phase-lag between the two signals across trials). To further characterize the phase-lag distribution, we computed for each single trial j a mean between-electrode synchrony vector sj across time t (between –400 and 0 ms prior test onset):
The circular distributions of the mean synchrony in correct and incorrect trials were compared using the Watson U2-test for circular data (Zar, 1999). Similarly, a mean power value in the same 400 ms time-window was computed for each single trial and distributions of power for correct and incorrect trials were compared using the Mann–Whitney U-test.
Because we performed multiple statistical tests, the threshold for significance has to be corrected. However, these multiple tests are not independent: synchrony between electrodes A and B is not independent from synchrony between A and C, both having the signal from A in common. The classical Bonferroni correction (dividing the significance level by the number of tests performed) would therefore be inappropriate in our case, since it leads to incorrect acceptance of the null hypothesis when applied on correlated measures (Wright, 1992; Shaffer, 1995). Elegant solutions for estimating the number of independent samples have been described for imaging data (Worsley et al., 1992), but cannot be readily applied to synchrony measures. We used the number of electrodes as an estimate of the number of independent variables, and thus a priori corrected all P values (Watson U2-test, Mann–Whitney U-test) by the number of electrodes (27 in monkey No. 1 and 36 in monkey No. 2). An adjusted P-value of 0.03 corresponds to an uncorrected α level of ∼0.001 (0.03/27= 0.00111 in monkey No. 1; 0.03/36 = 0.00084 in monkey No. 2).
The hypothesis that oscillatory phase synchrony in the beta band is necessary to successfully perform this short-term memory task (Fig. 1) predicts that synchrony in correct and incorrect trials should differ during the delay period: impaired short-term memory maintenance should be accompanied by a reduced oscillatory synchrony during the delay in error trials.
As a first approach to test this prediction we compared the strength of synchrony in error trials to the strength of synchrony in an equal number of correct trials chosen among streams of high performance. The difference in phase synchrony between correct and incorrect trials during the last 400 ms of the delay was computed for all possible electrode pairs and all frequency bands (Fig. 2c). For all but one electrode pair, this difference is scattered around zero. In both animals one electrode pair stands out from the others, showing a larger phase synchrony in correct than incorrect trials during memory maintenance. This effect occurred in the beta range (15–20 Hz) in both monkeys during the end of the delay, peaking at 18 Hz in monkey No. 1 and 16 Hz in monkey No. 2 (Fig. 2d). The electrode pair showing more synchrony in correct trials was located close to the superior temporal sulcus (STS) in both animals (Fig. 2a,b) and were separated by 4.2 mm in monkey No. 1 and by 6.7 mm in monkey No. 2.
For that electrode pair, the mean synchrony factor in the beta band during the last 400 ms of the delay was significantly larger in correct than incorrect trials in both monkeys (Watson U2-test for circular data, adjusted P < 0.03), reflecting the fact that in incorrect trials the precise temporal relationship of neural activity between the two recording sites was strongly reduced. In correct trials, the temporal lag between the signals of the two electrodes was 8 ms in monkey No. 1 and 23 ms in monkey No. 2. In monkey No. 2, two additional recording sessions providing 298 error trials confirmed this result: beta synchrony was increased in the delay period of correct trials and was markedly reduced in error trials during the last 400 ms of the delay (peak at 19 Hz, synchrony factor in correct trials: 0.184; in incorrect trials: 0.031; difference significant with an adjusted P < 0.02, Watson U2-test, temporal lag between the signals of the two electrodes in correct trials: 20 ms).
The difference in synchrony between correct and incorrect trials was highly specific to the electrode pair considered. We systematically tested the significance of the difference in synchrony for all possible electrode pairs in the beta range. Although some electrode pairs could show a certain amount of synchrony, synchrony did not differ statistically between correct and incorrect trials for any pair other than the pair of interest (adjusted P > 0.1).
The significant difference in beta synchrony between correct and incorrect trials was only observed during memory maintenance in both monkeys. Figure 3a shows the amount of synchrony at the beginning of the trial, when the monkey was fixating a blank screen and waiting for the first stimulus, and at the end of the delay when the monkey was fixating the same blank screen and waiting for the second stimulus, but in addition maintained information in short-term memory. At the beginning of the trial, synchrony was weak and did not show any significant difference between correct and incorrect trials (adjusted P > 0.4, Watson U2-test). During the delay, synchrony increased in correct trials with a similar time-course at all delay durations, but failed either to develop or to be maintained in incorrect trials (adjusted P < 0.03). Thus, the error-related differences in synchrony could be observed best during the last 400 ms of the delay for all trial lengths (Fig. 3a).
This existence of a larger synchrony during memory maintenance in correct trials was not specific to the particular selection of correct trials analyzed here. We estimated the synchrony factor in 104 different subsets of correct trials. The distribution of the synchrony factors obtained by this randomization technique is shown in Figure 3b. The value of the synchrony factor in error trials falls clearly outside the distribution of synchrony factors for correct trials. Because the observed value of synchrony for incorrect trials was smaller than any of the 104 values of synchrony for correct trials, synchrony in error trials is significantly different from synchrony in correct trials with a P-value <0.0001 in this Fisher’s one-sample randomization test. Our results therefore suggest that oscillatory synchrony in posterior IT during the delay is a consistent feature of neural activity when successfully performing this short-term memory task.
No other component of the electrophysiological response we studied (power and between-electrode synchrony in the 8–100 Hz range, evoked potentials) showed any reliable differences between correct and incorrect trials during the delay at any recording site. No difference of power in any frequency band was consistently observed in both monkeys. In particular, the reduced synchrony in incorrect trials we describe was not due to a smaller amplitude of the local neural activity in the beta range (Fig. 4). Indeed, there was no statistically significant difference in beta power at any electrode between correct and incorrect trials during the last 400 ms of the delay. Only a non-significant trend (Mann–Whitney U-test, adjusted P = 0.09) for higher power in correct trials could be observed at the far apart electrode A5 in monkey No. 2. In monkey No. 1 only, within the same regions as those engaged in beta synchrony, oscillatory synchrony in the alpha range (centered on 10 Hz) was larger in correct than in incorrect trials (Fig. 2d). This difference was significant throughout the entire trial, (Watson U2-test, adjusted P < 0.03), beginning even before sample onset. No such effect in the alpha range could be observed in monkey No. 2.
No evidence for impaired stimulus processing was found by comparing the different response components during sample presentation between correct and incorrect trials. Over area V4, evoked responses to the sample were followed by induced oscillations in the high gamma range (60–100 Hz). Neither the transient evoked components (Fig. 5a) nor the more sustained induced oscillations (Fig. 5b) showed any difference related to the monkey’s behavioral performance. Furthermore, the sites engaged in the sustained induced oscillations did not show any consistent error-related difference in synchrony. The electrodes showing the error-related reduced synchrony during the delay were only weakly responsive during stimulus encoding compared to those electrodes located over area V4.
We show that actively holding an item in visual short-term memory is associated with an elevated synchrony in the beta range between distinct sites in the posterior inferotemporal cortex when the monkey’s response to the test item is correct. In contrast, error trials are characterized by the absence of synchrony during memory maintenance. The close match between oscillatory synchrony and behavioral performance suggests that this temporal pattern cannot be considered as a meaningless ringing epiphenomenon. Rather, oscillatory synchrony seems to be a relevant feature of the neural mechanism required to successfully perform a short-term memory task.
The difference in beta synchrony between correct and incorrect trials was found between two sites located in posterior IT. Although more anterior areas were most often investigated in short-term memory tasks (Fuster and Jervey, 1981; Chelazzi et al., 1993; Miller et al., 1993), this region is also known to contain neurons showing enhanced firing during the delay (Fuster and Jervey, 1982). However, variations of synchrony can occur without changes in firing rates, as described for the monkey primary auditory cortex (deCharms and Merzenich, 1996). In any case, the temporal correlation we observe in posterior IT is likely to enhance the impact of the synchronous discharges from this region on subsequent stages of processing (Niebur et al., 1993; Engel and Singer, 2001) and hence to contribute to the pronounced delay activity observed in more anterior parts of IT.
Behavioral errors may in principle be related to failures of various cognitive processes, i.e. impaired stimulus encoding, ineffective preparatory processes, deficient memory maintenance, or a combination of any of these factors. The neural processes underlying each of these cognitive processes differ in their time-course: signs of impaired encoding should appear during stimulus presentation, signs of reduced vigilance throughout the whole trial and those indicating a deficient memory maintenance during the delay period.
No evidence was found for an impaired stimulus encoding in incorrect trials due to attentional deficits: there was no difference between correct and incorrect trials during sample presentation over sensory area V4, neither in the evoked potentials nor in the induced gamma oscillations, while in area V4 attentive versus unattentive stimulus processing is known to produce modulations of discharge rates (Desimone and Duncan, 1995; Maunsell, 1995), of visual evoked potentials (Mehta et al., 2000) and of gamma oscillations (Fries et al., 2001). Furthermore, we did not observe error related differences in synchrony between correct and incorrect trials for any pair of electrodes during stimulus presentation, whereas ineffective stimulus encoding has been associated with a reduced 40 Hz synchrony between medial temporal lobe structures in humans (Fell et al., 2001).
Similarly, there was no consistent relationship between an ineffective preparatory process or a decreased vigilance and behavioral errors. Such a non-specific deficit is likely to affect the whole trial. The only component with an appropriate time-course was a reduced synchrony in the alpha range in error trials in monkey No. 1 that started already before stimulus onset and remained present throughout the whole trial. This may suggest that in this monkey a reduced level of vigilance could have accompanied errors. However, no evidence for a reduced synchrony in the alpha range could be observed in monkey No. 2 (Fig. 2c,d).
In conclusion behavioral errors are most likely related to a deficient memory maintenance in this task designed to strongly challenge short-term memory. Synchrony in the beta band was the only component of the neural response systematically related to error production in both animals and the decrease of synchrony in error trials was restricted to the delay period. This interpretation is in keeping with findings obtained in humans in the same delayed-matching-to-sample paradigm showing that beta oscillatory synchrony between extra-striate visual areas develops during memory maintenance, but is absent in a control task matched in expectancy and difficulty to the memory task (Tallon-Baudry et al., 1998, 1999, 2001).
Oscillatory synchrony of neural activity may be particularly relevant in the context of short-term memory tasks: it could reflect the reverberating activity underlying sustained memory-related activity as postulated by Hebb (1949) and its temporal structure could be most effective to induce synaptic plasticity (Markram et al., 1997; Froemke and Dan, 2002) that may in turn promote long-term memory storage. Temporal patterns have been searched for in single-unit recordings in the temporal lobe of monkeys performing short-term memory tasks but were found to be scarce or absent (Nakamura et al., 1992; Villa and Fuster, 1992; Miller et al., 1993). Several reasons may account for the difference between these single-unit studies and the present findings, as follows. (i) Our recordings were done in posterior IT cortex, contrary to other studies that investigated more anterior regions. (ii) The recording level we used (cortical field potentials) is known to facilitate the detection of collective, synchronized events (Murthy and Fetz, 1996; Gail et al., 2000; Pesaran et al., 2002). It is therefore often easier to observe oscillatory activity in field potentials than in single unit recordings, although the two recording levels are closely related (Singer et al., 1990; Fries et al., 2001). (iii) Perhaps the most relevant explanation is that in our study a new stimulus was generated at each trial, to prevent the development of neurons responding specifically to individual stimuli of a limited training set (Sakai and Miyashita, 1991; Sigala and Logothetis, 2002). The detailed and complete representation of the stimulus required to perform the task thus probably depends on a distributed code in a population of neurons. Such population coding is likely to rely on temporal mechanisms of coordination in a distributed neuronal assembly.
Stimulus-specific sustained firing is classically observed in IT neurons in animals well trained with a limited set of stimuli, suggesting the existence of a sparse representation of an over-learned stimulus set (Fuster, 1995). This stimulus-specificity is acquired in the course of training (Sakai and Miyashita, 1991; Sigala and Logothetis, 2002), probably by strengthening connections and modulating synaptic weights in the network. New stimuli, never seen before by the animal, elicit only a weak firing during the delay compared to those that have been well learned in the course of training (Miyashita, 1988). It is possible that the oscillatory synchrony we observe here in response to new stimuli reveals synaptic reverberation in an assembly not yet stabilized by time-dependent synaptic plasticity (Markram et al., 1997; Froemke and Dan, 2002) — in Hebb’s view (Hebb, 1949; Seung, 2000), the very first step in memory formation.
We thank Sigrun Wicker and Katrin Thoß for excellent technical assistance. We are very grateful to Dieter Leibfritz and Kurt Bockhorst (Institute for Organic Chemistry, University of Bremen) for doing the MRI scans and to Olivier Bertrand for his help in computing the Laplacian transform. This project was supported by DFG SFB517 ‘Neurocognition’ and by an EC Marie-Curie fellowship to C.T.B.