Abstract

We have used positron emission tomography to map the mnemonic components of two tasks at the extremes of the visual short-term/ working memory spectrum. The successive discrimination task requires only storage of a single item for very short time (ultra-short- term memory), while the 2back task requires both maintenance (i.e. storage and rehearsal) and manipulation of several items (working memory). We tested whether or not the storage component, common to the two tasks, engaged the same cerebral regions. To remove unnecessary confounds, we reduced the cues available to the subjects to a single elementary attribute, the orientation of a grating presented in central vision. This prevented subjects from using verbal strategies or vestibular cues and allowed equating of difficulty among tasks. Ultra-short-term memory for orientation engaged a large expanse of occipito-temporal cortex with a rate-dependent antero-posterior gradient: a fast trial rate engaged posterior regions, a slow trial rate anterior regions. On the other hand, working memory for orientation involved the left inferior parietal cortex, left dorsolateral prefrontal cortex and a left superior frontal sulcus region, and to a lesser degree the symmetrical right superior frontal region and a left superior parietal region. Direct comparison of the two orientation memory networks confirmed their functional segregation. We conclude that at least the storage of orientation information engages distinct regions depending on whether or not short-term memory/working memory involves rehearsal and/or manipulative processes.

Introduction

Short-term memory is defined as a limited capacity system in which information is actively maintained during a time period of seconds [for a review see Baddeley (Baddeley, 1990)]. Originally, the concept of working memory comprised a central executive and two slave systems, the phonological loop and the visuo- spatial sketchpad (Baddeley and Hitch, 1974). Subsequent workers have blended the two terms and separate subsystems for spatial and object short-term memory/working memory have been proposed (Goldman-Rakic, 1987; Wilson et al., 1993; Courtney et al., 1996), building upon a similar distinction in the visual system. Another functional segregation is based on the type of executive processing (Petrides, 1995): mid-ventrolateral frontal cortex is engaged in active retrieval, while mid-dorsolateral frontal cortex is engaged in monitoring and manipulation. A similar distinction is that between tasks requiring only maintenance across a non-distracted delay period and tasks requiring maintenance plus higher order processes (D'Esposito et al., 1998b; Smith and Jonides, 1999). To keep information on-line during maintenance-only short-term memory/working memory tasks, information needs to be rehearsed. Such tasks, however, may not involve a rehearsal component when the delay period is short. Indeed, in successive discrimination tasks, subjects also briefly store a trace of the preceding grating (Orban et al., 1997) or random dot pattern (Cornette et al., 1998) to perform a temporal comparison. Given the 300 ms interval between two successive stimuli used in those studies, it is unlikely that this retention involves a rehearsal component. Although this interval is close to the range typical for iconic memory [for a review see Long (Long, 1980)], we believe that the memory process involved is different. First, duration of stimulus presentation was 300 ms (Orban et al., 1997; Cornette et al., 1998) or 500 ms (current experiment), which exceeds the stimulus duration (<200 ms) used in classical iconic memory experiments (Haber, 1970; Long and Sakitt, 1980). In addition, a longer stimulus duration challenges the definition of iconic memory as being pre-categorical or ‘not yet identified’ (Coltheart, 1983). Second, the phase of the grating was randomized in the orientation discrimination study and iconic memory is sensitive to stimulus position (Philips, 1974). Third, the delay period used in successive discrimination (300 ms) exceeds that of iconic memory, which is typically <100 ms (Di Lollo, 1980).

We therefore refer to the mnemonic component of successive discrimination as ultra-short-term memory, which represents one extreme of the short-term memory/working memory spectrum (Fig. 1). At the opposite end lies working memory or operant memory (Fuster, 1995), which includes the maintenance (i.e. storage and rehearsal) of multiple items, whether or not accompanied by manipulation, with storage and rehearsal of a single item in short-term or immediate memory (Fuster, 1995) in between.

There is general agreement that the underlying neural mechanism of maintenance during short-term memory/working memory is the sustained neuronal activity during delay intervals, typical of monkey prefrontal, premotor and parietal neurons [for a review see Fuster (Fuster, 1995)]. Delay activity has also been observed in the inferotemporal (IT) cortex (Fuster and Jervey, 1981), where it may underlie storage in ultra-short-term memory (Miller et al., 1994; Vogels and Orban, 1994). Thus, the entire short-term memory/working memory spectrum seems to rely on the same type of neuronal mechanism. For a given kind of information, a broad range of anatomical organizations are compatible with this single physiological mechanism. At one end, all short-term memory/working memory tasks engage the same cortical network with a single cerebral store and additional regions are recruited as tasks require more cognitive processes. At the other extreme of the range, a given information is stored in distinct regions depending on whether or not short-term memory/working memory involves only storage or also includes rehearsal or manipulation.

To decide between these two hypotheses, we have mapped the memory-related processes at the extremes of the visual short-term memory/working memory spectrum. By definition, its lower end is engaged by successive discrimination tasks. To investigate its upper end, we chose the N-back paradigm (Kirchner, 1958). We used the attribute orientation, the coding of which is well known (Hubel and Wiesel, 1968), to avoid the intrusion of non-essential processes. Its elementary nature and presentation in the central visual field avoided the necessity of processing object identity and spatial location, respectively. The presentation of oblique orientations prevented the use of vestibular cues (Brandt et al., 1994) or verbal codes that could be rehearsed using articulatory mechanisms. Finally, this attribute enabled us to equate performance levels among tasks.

Materials and Methods

Subjects

Fourteen male volunteers (mean age 24.4 years, range 19–27 years) participated in the study. They were strictly right-handed as judged by the Edinburgh inventory and had no evidence of neurological/psychiatric history, pathology or drug abuse. They all had normal or corrected-to-normal vision and a normal brain structure as visualized by MRI. Scanning procedures were undertaken with the understanding and written consent of each subject, in accordance with the Declaration of Human Rights, Helsinki, 1974. The study was approved by the Ethical Committee of the Medical School, Katholieke Universiteit Leuven. Subjects were trained in two sessions before scanning.

Stimulus Characteristics

Physical stimulus characteristics were chosen in accordance with the study of Orban and co-workers (Orban et al., 1997) and were identical among all tasks: a small static square wave grating (4° diameter, mean luminance 23.1 cd/m2, contrast 90%, cycle width 1.3°) was presented in the central visual field. In all conditions subjects were instructed to fixate a central red fixation point. In both the experimental and control conditions phase was randomized between trials and noise was superimposed on the edges of the bars, preventing subjects from using cues other than orientation. Half of the subjects were presented only with orientations between 20 and 70°, the other half with orientations between 110 and 160°. No cardinal (vertical or horizontal) orientations were shown, to avoid verbal and semantic encoding. The stimuli were displayed on a high resolution color screen (Philips Brilliance 2120, horizontal width 380 mm, vertical height 285 mm, resolution 800 × 600 pixels, refresh rate 78 Hz non-interlaced), hosted by a 486 TIGA work- station. The monitor was mounted above the scanner bed at an angle of 52° relative to the horizontal. Subjects viewed the stimuli binocularly in a dimly lit room (0.07 cd/m2) from a fixed distance of 114 cm.

Task Configuration

1back and 2back were used as working memory tasks, with 0back as the control task. The N-back paradigm refers to a serial recognition task in which the target is continuously changing. In the 0back task subjects were instructed to identify the orientation of a single initially specified target grating. In the 1back task they compared each grating to the one immediately before in the sequence. In the 2back task each grating was compared with the one presented two trials earlier (Fig. 2). Time settings were chosen in accordance with Jonides and co-workers (Jonides et al., 1997): stimuli were presented for 500 ms at a rate of 20 stimuli/min in all three conditions (interstimulus interval 2500 ms). Motor output was conditional-associative: subjects pressed a right-hand key for an identical orientation and a left-hand key for a different orientation. Motor responses were allowed at any point during the interstimulus interval, with the instruction to press as soon as a decision was made. All three conditions were matched for the number of left and right motor responses prepared and executed. Mnemonic as well as non-mnemonic demands gradually increase from 0back through 1back to 2back. Indeed, the number of items to be stored gradually increases. With regard to the executive processes, the 1back task involves an updating or reshuffling operation, i.e. the operation of entering a new item in the memory store and removing an old one upon each new stimulus presentation. The 2back task involves at least three executive operations. In addition to updating, the task involves temporal coding, which is the need to assign a temporal order to the current memory set, in order to facilitate item-by-item updating and matching with the correct item. The 2back task also involves resistance to the influence of interfering stimuli or distracters, a term which refers to the need to maintain items in store despite processing the subsequent stimulus.

In all three tasks, the orientations were constrained to lie within the interval 20–70 or 110–160°. The task difficulty was adjusted by changing the orientation difference δ and adapted individually to the results of the second training session. The following steps were taken to ensure that in all N-back tasks the gratings to be compared differed in orientation by an angle δ. In the 2back task, two series of gratings were interleaved. Each series started with a pseudo-randomly chosen seed orientation and consisted of gratings differing in orientation by δ degrees. The orientation difference between the two seed gratings was set at δ, in order to minimize possible interference between storage and manipulation of the two series. In each 150 s run of this task, two new seed orientations were used, further preventing any use of verbal labeling. In the 1back task, a series started with a randomly chosen orientation and the orientation of the subsequent gratings was either identical to the preceding one or differed from it by an angle +δ or –δ. In the 0back task, the seed grating had a randomly chosen orientation. All subsequent gratings either had the same orientation as the seed orientation (shown in half of the trials) or were randomly chosen among a set of gratings that differed in orientation from the first one by a multiple of the value +α or –α, used to adapt the difficulty of the task. In this task the average (i.e. among trials) orientation difference δ differed depending on the seed orientation and equaled approximately (50 + α)/6 for the range of seed orientations used. Subjects were admitted for training only if their performance during a relatively easy 2back task (δ = 20°) exceeded 80%. Load during the 2back task using oblique orientations was at ceiling, since subjects in pilot studies performed at chance level in a 3back task unless δ exceeded 20°. This task was not included since the small number of orientations (n = 3) available in the range enabled subjects to use a verbal strategy. In this case, subjects labeled the orientations as high, middle and low. As soon as four or five orientations were presented, this labeling strategy could no longer be used. This was clearly indicated by a substantial decrease in performance, which recovered only after training. Within the two 50° orientation intervals we presented an average of 5.48, 5.76 and 4.82 different orientations during the 0back, 1back and 2back tasks, respectively.

The successive orientation discrimination or temporal same different (TSD) task of Orban et al. (1997) was used as the ultra-short-term memory task, with orientation identification (ID) as the control task. Two different presentation rates were used: 70 stimuli/min in the TSDfast task, corresponding to 35 trials/min, with IDfast as the control task, to replicate the rate of the previous study (Orban et al., 1997), and 20 stimuli/min in the TSDslow task (10 trials/min), with IDslow as the control task, to use the same stimulus rate as used in the working memory tasks (Fig. 2). Inter-trial intervals lasted 413 and 357 ms during the TSDfast and IDfast tasks, respectively, whereas they occupied 4700 and 2500 ms during the TSDslow and IDslow tasks, respectively. Note that to keep visual input constant among all seven tasks, we used a stimulus presentation time of 500 ms, which is different from the 300 ms duration used in the study of Orban and co-workers (Orban et al., 1997). In the TSD task each trial involved the successive presentation of two gratings, bridging a non-distracted delay interval of 300 ms. This delay was identical in the TSDslow and TSDfast trials. However, over a 60 s data acquisition period (see below) active storage took 3.5 times longer in TSDfast compared with TSDslow. Whereas all stimuli had orientations within 5° of the vertical meridian in our previous study, all orientations presented here belonged to the 20–70° or 110–160° range. The orientation difference δ between two successive gratings was adapted individually for the two TSD tasks, using the results from the second training session as a guide. Subjects were asked to press the left-hand key for two different orientations and the right-hand key for two identical orientations, within 600 ms after the onset of each second stimulus presentation. The ID control task was adapted from the study of Orban and co-workers (Orban et al., 1997): subjects were presented with oblique orientations that were tilted either –δ/2° (press right) or +δ/2° (press left) from a reference orientation. They were asked to press the keys within 600 ms after onset of each stimulus presentation. Half of the subjects were trained for two reference orientations (one for IDslow and one for IDfast) between 20 and 70° and the other half for two reference orientations between 110 and 160°. Across all subjects, 28 different reference orientations were presented during scanning. The ID task matches the TSD task for visual processing, attention demands and type of motor decision. Differential activity between the two tasks reflects ultra-short-term storage of orientation and temporal comparison.

In order to equate the visual input among all tasks and all subjects, stimulus presentation during the N-back and TSD tasks was programmed to include as many orientations in the 20–70° or 110–160° range as possible and to distribute them equally among all subjects. Also, all 28 reference orientations used in the two ID tasks were chosen to cover the two orientation ranges as homogeneously as possible, taking into account each subject's orientation threshold. A Kolmogorov–Smirnov test for distribution fitting (CSS software) was performed to test whether or not during acquisition the visual input was matched among tasks.

Training Sessions

Prior to the scanning session, subjects were trained in two 1.5 h sessions. As soon as performance in a task reached a steady level of 80–85% correct, the orientation difference δ was gradually decreased for that task. The 2back task required more training than the other tasks to reach stable performance. The δ for which the subjects reached 80% correct at the end of the second training session was then used in the corresponding condition of the PET study. No auditory feedback was provided during training or PET acquisition.

We attempted to equate the level of performance among tasks by adjusting the orientation differences. Although some of the manipulations of difficulty level, such as image degradation, both decrease accuracy and increase reaction times (Grady et al., 1996), manipulating the orientation difference or the presentation rate within one of the present tasks changes only the performance level, while reaction times only slightly increased (Cornette et al., 1999) (also A. Rosier, unpublished observations). Thus, increases in reaction times among tasks largely reflect the increased computational demands.

It is noteworthy that the application of phase randomization prevented subjects from using alternative strategies based on apparent motion detection rather than successive orientation discrimination. Additional psychophysical testing in which both phase randomization and delay interval between successive stimuli were omitted resulted in extremely low orientation differences (δ ~ 1–2°) for a performance level of 80–85% correct. These thresholds differ significantly from δ commonly reached at the end of the second training session for the standard TSDfast task (δ ~ 8–10°), indicating that subjects did not use apparent motion cues in the standard TSDfast task.

Statistical Analysis of Behavioral Data

Performance, expressed as the percent correct responses, was normalized to Z scores (CSS software) and analyzed by analysis of variance (ANOVA) and post hoc comparisons for significance (Scheffé test). In all analyses, the number of memory tasks was included as a repetitive factor within each subject. The mean reaction times were calculated as the average of all latencies corresponding to trials in which subjects responded within the response window.

Data Acquisition

Brain activity was monitored as the relative change in regional cerebral blood flow (rCBF) using the H215O method (Fox et al., 1986). All measurements were performed in 3D mode with a Siemens CTI Ecat Exact HR+ (Brix et al., 1997). Subjects were not allowed to speak during the procedure and had been instructed not to think of anything in particular, apart from concentrating on the stimulus and the task. The room was kept as quiet as possible. The head was immobilized with a foam headholder (Smither Medical Products, Akron, OH). Each subject had a catheter inserted into the left brachial vein for tracer administration. Accuracy of fixation was monitored with electro-oculographical recordings (EOG), using contact electrodes placed on the outer ocular canthi and a reference electrode placed between the eyes. Before each experiment, this EOG was calibrated for fixation and for horizontally visually guided saccades of 2 and 4° amplitude. A transmission scan was taken (68Ge rod sources) to correct for attenuation. The start of each task coincided with the i.v. injection of 300 MBq H215O (half-life 123 s) over 12 s. Each subject performed all seven tasks twice, with a 10 min interval between two successive injections. Different orientations were used in these replications. This yielded 14 emission scans per subject and 28 emission scans per task. The order in which tasks were presented was randomized both within and between subjects. Data acquisition (60 s) began as soon as the intracranial radioactivity count rate rose sharply, i.e. usually ~40 s after injection. Total duration of each task performed was 150 s. The attenuation-corrected data were reconstructed using the reprojection algorithm (Kinahan and Rogers, 1989) resulting in 63 planes (plane separation 2.425 mm). The integrated radioactivity counts were used as a measure of rCBF.

Data Analysis

Analysis was done on Sun SPARC computers (Sun Microsystems, Mountain View, CA) with the Statistical Parametric Mapping software (Wellcome Department of Cognitive Neurology, London, UK), version SPM96, implemented in MATLAB (Mathworks Inc., Sherborn, MA).

Realignment and Spatial Normalization

The scans from each subject were realigned using the first scan as a reference. The six parameters of this rigid body transformation were estimated using a least squares approach. Images were subsequently stereotactically transformed to a standard template in the Talairach space (Talairach and Tournoux, 1988). This normalizing spatial transformation matches each scan (in a least squares sense) to a reference or template image that already conforms to the standard space. The procedure involves a 12 parameter affine (linear) and quadratic (non-linear) 3-dimensional transformation, followed by a 2-dimensional piece-wise (transverse slices) non-linear matching, using a set of smooth basic functions that allows for normalization at a finer anatomical scale (Friston et al., 1995a). Finally, images were smoothed with an isotropic Gaussian kernel of 16 mm full width at half-maximal (FWHM). The final image smoothness estimates (FWHM) were x = 22.0, y = 24.4, z = 18.8 mm.

Statistical Analysis

Statistical parametric maps (SPMs) are spatially extended statistical processes used to characterize regionally specific effects in imaging data, combining the general linear model (to create the statistical map of SPM) and the theory of Gaussian fields (to make statistical inferences about regional effects) (Friston et al., 1991, 1994; Worsley et al., 1992). The statistical analysis can be regarded as an ANCOVA, as the design matrix includes global brain activity as a covariate of no interest fixed at 50 ml/dl min (Friston et al., 1995b). The condition, subject and covariate effects are estimated according to the general linear model at each voxel. To test hypotheses about regionally specific condition effects, the estimates were compared using linear contrasts. The resulting set of voxel values for each contrast constitute a statistical parametric map of the t-statistic SPM{t}. These SPM{t} values were then transformed to the unit normal distribution (SPM{Z}). Activations reaching Pcorr < 0.05 (corrected for multiple comparisons) for peak height (Z ≥ 4.40) were considered significant. Regions significant at Puncorr < 0.001 (uncorrected for multiple comparisons) for height (Z ≥ 3.09) were considered equally significant only if based upon an a priori hypothesis. Other activation sites significant at Puncorr < 0.001 for height were added for descriptive purpose.

MRI Template

Each subject also underwent a high resolution MRI scan of the brain, using a 3D Magnetization Prepared Rapid Gradient Echo (MPRAGE) sequence (Mugler and Brookeman, 1990). Acquisition parameters were: repetition time 10 ms, echo time 4 ms, flip angle 8°, field of view 256 mm, acquisition matrix 256 × 256. The 3D volume had a thickness of 160 mm, partitioned into 128 sagittal slices. MRI images of each subject were registered to the corresponding PET images using the Multi-modality Image Registration algorithm based on Information Theory (MIRIT) (Maes et al., 1997). The same transformations into the standard space as those that were used for the PET images were applied to the resampled (and registered) MRI images. An average (n = 14) MRI image in the standard space was constructed and the thresholded parametric maps were projected onto these images for visualization.

Planned Analysis

In keeping with other N-back working memory studies, we used the subtraction [2back – 0back] to characterize the network involved in orientation working memory. The 1back task was used only to describe the activity in the regions obtained by this subtraction. The regions involved in ultra-short-term memory during orientation discrimination were characterized for the two rates by the subtractions [TSDslow – IDslow] and [TSDfast – IDfast].

To test whether the mnemonic components of the 2back and the successive discrimination tasks engaged separate cortical regions, we first tested all regions resulting from the subtraction [2back – 0back] for their significance in [2back – 0back] – [TSDslow – IDslow] as well as in [2back – 0back] – [TSDfast – IDfast]. Similarly, all regions resulting from the subtraction [TSDfast – IDfast] were tested for their significance in [TSDfast – IDfast] – [2back – 0back] and all regions resulting from the subtraction [TSDslow – IDslow] for their significance in [TSDslow – IDslow] – [2back – 0back]. Regions were considered specifically related to either ultra-short-term or working memory mnemonic components if they reached Puncorr < 0.001 in both the simple subtraction isolating a memory type and the contrasts directly comparing the two types of short-term memory/working memory. This procedure (Haxby et al., 1994) avoids considering regions which reach significance in the direct comparisons of ultra-short-term and working memory mnemonic components because of deactivation in the second element of these comparisons.

Alternatively, we applied a conjunction analysis (Price and Friston, 1996) to investigate whether regions resulting from a series of independent (i.e. orthogonal) task pairs were jointly significant in ultra-short-term and working memory. We first investigated which voxels reached a non-stringent criterion of Puncorr < 0.05 for height in the orthogonal contrasts [2back – 0back] and [TSDslow – IDslow]. Since storage activity in ultra-short-term memory at the fast rate lasts 3.5 times longer than at the slow rate, we then performed the same analysis between the three orthogonal contrasts [2back – 0back], [TSDslow – IDslow] and [TSDfast – IDfast].

Results

Visual Stimulation during PET Scanning

Group analysis (n = 14) revealed that the distributions of the orientations presented in both the 20–70° and 110–160° ranges were similar among the various tasks (Fig. 3). In this regard, no significant differences were found when comparing the distributions of 2back to 0back, TSDslow to IDslow, TSDfast to IDfast or 2back to TSDslow tasks (Kolmogorov–Smirnov, P > 0.5). Hence, the visual input was matched among all tasks that were compared in the subtractive design.

Task Performance during PET Scanning

During scanning, mean performance in the different tasks ranged between 83 and 87% correct (Fig. 4A). This was similar to the performance at the end of the second training session, indicating that little additional learning occurred during the scanning. More importantly, accuracy did not differ significantly between conditions (repeated measures ANOVA, F[6,78] = 1.31, P > 0.2). To achieve this equal performance, the orientation difference δ was systematically varied between tasks (repeated measures ANOVA, F[6,78] = 26.25, P < 10–6), being larger in the computationally more demanding tasks (Fig. 4B). The value of δ was significantly higher in the 2back task than in the 0back (Scheffé test, P < 10–5) and 1back tasks (Scheffé test, P < 10–6), as well as in the two TSD tasks compared with their respective ID tasks (Scheffé tests P < 0.05 and P < 10–3 for slow and fast tasks, respectively). The mean δ was 9.6° for the TSDfast task and 8.1° for the IDfast task, respectively, which is clearly higher than the mean values used in the study of Orban and co-workers (Orban et al., 1997), where the average δ was 4.6° in the TSDfast task and 3° in the IDfast task. This difference in all likelihood reflects the oblique effect in orientation discrimination (Orban et al., 1984; Regan and Beverley, 1985; Heeley et al., 1997). Indeed, in the present study only oblique orientations were presented, whereas in the previous study all stimuli had orientations within 5° of the vertical.

The differences in computational demands between the tasks are reflected not only in the orientation differences, but also in the reaction times. These reaction times differed significantly among tasks (repeated measures ANOVA, F[6,78] = 66.25, P < 10–6) (Fig. 4C). Post hoc analysis showed that both the 1back and the 2back tasks resulted in significantly longer reaction times than either the 0back task (Scheffé tests, P < 0.05 and P < 10–4, respectively) or any of the TSD and ID conditions (Scheffé tests, P < 10–6 in all comparisons). Reaction times during the TSDslow and IDslow tasks did not significantly differ from the reaction times during the TSDfast and IDfast tasks, in agreement with Cornette and co-workers (Cornette et al., 1999).

Subjects maintained fixation well during the different tasks, as witnessed by the electro-oculographical recordings. Two saccades were found in one subject, whereas all other traces were virtually identical to the fixation trace of the calibration.

Working Memory: Subtraction [2back – 0back]

The aim of subtracting the 0back task from the 2back task was to determine which areas are involved in working memory for orientation, either in storage or in executive processes. As shown in Table 1 (columns 3 and 4) and Figure 5, three left hemispheric regions reached the Pcorr < 0.05 level for height: the left middle frontal gyrus (dorsolateral prefrontal cortex, DLPFC, BA 9/46), the left superior frontal sulcus (SFS, straddling BA 6 and dorsal area 8A [8Ad]) (Petrides and Pandya, 1999) and the left supramarginal gyrus (BA 40). Their activity profiles, i.e. the adjusted rCBF plotted for all tasks, demonstrate a similar monotonous, linear increase of rCBF going from the 0back through 1back to 2back tasks (Fig. 6A). This load-related effect suggests that activity in these areas indexes executive working memory operations for orientation. Equating visual input across experimental and baseline tasks controls for the fact that prefrontal neurons are responsive to visual stimuli (Ó Scalaidhe et al., 1999). Hence, the recruitment of prefrontal regions during working memory can only be related to mnemonic and/or executive processes. Moreover, little or no change in activity was observed in any of these three areas during the four ultra-short-term memory conditions. An exception is the left SFS, in which both ID tasks evoked slightly more differential activity than their corresponding TSD tasks. The precise topography of the local maxima of this fronto-parietal circuit is shown for three representative subjects in Figure 5. The most significant voxel of the DLPFC region is located on the superior bank of the inferior frontal sulcus. The SFS region is located on the lateral bank of the superior frontal sulcus, anterior to its junction with the precentral sulcus, corresponding to the most cranial part of the middle frontal gyrus. This SFS region (Talairach coordinates [–26 4 62]) is situated more than 20 mm anterior and superior to the frontal eye fields, the coordinates of which are [–39 –15 38] and [33 –15 48] according to Petit and co-workers (Petit et al., 1997). The most active voxel of the supramarginal gyrus region is located on the upper bank of the posterior part of the Sylvian fissure. Although the three significant regions are located in the left hemisphere, it is worth noting that a right-sided SFS region symmetrical to the left BA 6/8Ad was significant at Puncorr < 0.001. Other regions significant at Puncorr < 0.001 (Fig. 5 and Table 1) were the left superior parietal lobe (BA 7), a left and right premotor region (BA 6/44, both extending posteriorly from the SFS activation), a left middle frontal region (BA 46, extending anteriorly from left DLPFC), as well as right inferior and right middle cerebellum. Only one region in the left middle temporal gyrus (BA 21) reached Puncorr < 0.001 in the opposite subtraction [0back – 2back] (Fig. 5).

The subtraction [2back – 0back] enabled us to compare the working memory network for orientation with the results of other neuroimaging studies using the N-back paradigm (i.e. studies on verbal, spatial and object working memory domains). However, one could argue that the 0back task is not an adequate control, since even if it lacks the executive components of the 2back task, it may involve a different mnemonic component because subjects are required to remember the first orientation for the entire task duration (150 s). Both the orientation difference used to achieve 85% correct (Fig. 4B) and the reaction times (Fig. 4C) suggest that the computational demand of the 0back task is similar to that of the 1back task and higher than that of orientation identification (IDslow task). To explicitly control for the possibility that a storage component was subtracted out in the subtraction [2back – 0back], we investigated the subtraction [0back – IDslow], both tasks sharing an identical visual input and number of motor acts. This subtraction yielded differential activity in four regions (Puncorr < 0.001), all located in the ventral part of the frontal cortex: right inferior frontal gyrus (BA 47 [50, 30, –14] and BA 10 [46, 54, –6]), left gyrus rectus (BA 11 [–20, 30, –28]) and right superior frontal gyrus (BA 10 [36, 58, 8]). Most importantly, no significant differential activation was observed in any of the regions reaching Puncorr < 0.001 or higher in the subtraction [2back – 0back]. Furthermore, the subtraction [2back – IDslow] yielded no additional significant regions, confirming that no region was missed in [2back – 0back] because of equally high activity in the 0back and 2back tasks.

Ultra-short-term Memory: Subtraction [TSDfast – IDfast]

We determined the regions specifically involved in ultra-short- term memory for orientation by subtracting the ID from the TSD tasks, starting with the rate of 70 stimuli/min, since this rate was used in our previous orientation discrimination study (Orban et al., 1997). In that study the activation of the right middle fusiform gyrus [38, –50, –12] was attributed to storage and temporal comparison of orientations. Given this a priori knowledge, the activation observed here in the right middle fusiform gyrus at the Puncorr < 0.001 level (Table 2, columns 3 and 4) is considered significant. The three right fusiform activation sites, listed in Table 2, covered a large part of the middle fusiform gyrus, extending into the occipito-temporal sulcus (Talairach coordinates range from x = 38 to 52, y = –50 to –64 and z = –4 to –14). The activity profile (Fig. 6B) demonstrates that this region is not differentially active during the working memory conditions and that its activity increases with the rate of visual stimulation, since activity increases in the IDfast task compared with IDslow. We also observed a significant activation (Pcorr < 0.05) of the left posterior part of the lingual gyrus (BA 18) (Table 2), which we had not previously observed (Orban et al., 1997; Dupont et al., 1998). Its activity profile is highly comparable with that of the right middle fusiform gyrus (Fig. 6B). Several other occipito-temporal regions were significant at Puncorr < 0.001, including the right transverse occipital sulcus (BA 18/19, Fig. 6B) and right parahippocampal (BA 28/36), left middle fusiform (BA 19/37) and left superior temporal gyri (BA 38). Finally, a weak left parieto-occipital sulcus (BA 31) activation was observed.

Ultra-short-term Memory: Subtraction [TSDslow – IDslow]

To exclude any rate effect, we next subtracted ID from TSD for the presentation rate of 20 stimuli/min, the same as that applied during the working memory tasks. Interestingly, the regions involved during successive discrimination at a slow rate differed from those involved at a fast rate. A significant (Pcorr < 0.05) activation was observed in the left superior frontal gyrus (BA 9), anterior to DLPFC (Table 3, columns 3 and 4). Its activity profile (Fig. 6C) indicates that this region was nearly as active in the 2back task as in the TSDslow task. Regions significant at Puncorr < 0.001 were located mainly in the temporal cortex, involving the right inferior temporal gyrus (BA 20, Fig. 6C) and right middle temporal gyrus (BA 21/37). Activity remained unchanged during working memory conditions in these two regions. In the left hemisphere the middle temporal gyrus (BA 21), the middle occipital gyrus (BA 19) and the parahippocampal gyrus (BA 28/36) were weakly activated (Puncorr < 0.001). Finally, both the posterior (BA 31, Fig. 6C) and anterior (BA 24/32) cingulate also exhibited weak differential activity.

It is noteworthy that the TSD task, compared with the ID task, engages the right temporal cortex at both the fast and slow rates. Direct comparison of the mnemonic components of the TSDslow and TSDfast tasks show that the middle fusiform region, engaged at the fast rate, is co-activated with the occipito-parietal cortex (Table 4). These parietal regions, however, are distinct from those involved in orientation working memory (see Table 1). On the other hand, the more anterior middle temporal region, engaged at the slow rate, is co-active with frontal regions (Table 4).

Functional Dissociation between the Mnemonic Components of 2back and TSD Tasks

If the mnemonic components of the working memory tasks are really distinct from those active in ultra-short-term memory, one predicts that the regions revealed by the subtraction [2back – 0back] would still reach significance in the direct comparison of [2back – 0back] with [TSDslow – IDslow] or [TSDfast – IDfast] (Table 1, columns 5 and 6). These comparisons uniquely test for differences in mnemonic components, excluding differences in the number of stimuli to be stored, decisions and motor responses. All three regions significant at Pcorr < 0.05 in [2back – 0back] remained significant at Puncorr < 0.001 in the two direct mnemonic comparisons (Table 1), as could be expected from their activity profiles (Fig. 6A). Moreover, with the exception of two cerebellar foci, all regions significant at Puncorr < 0.001 in [2back – 0back] remained significant at this level when subtracting [TSDslow – IDslow] or [TSDfast – IDfast] from [2back – 0back].

We also tested the regions involved in ultra-short-term memory for their residual activity level after subtracting [2back – 0back]. Both posterior lingual and middle fusiform gyri that were significant in the subtraction [TSDfast – IDfast] remained significant at Puncorr < 0.001 (Table 2, column 5). All other regions retained Puncorr < 0.01. Similarly, most regions yielded by [TSDslow – IDslow], including the right middle temporal cortex, remained significant at Puncorr < 0.001, with three regions reaching Puncorr < 0.01 (Table 3, column 5).

To complete the demonstration of a functional dissociation, we investigated the conjunction between the subtractions [2back – 0back] and [TSDslow – IDslow], yielding a significant activation in the left anterior superior frontal gyrus ([–28 40 30], Pcorr = 0.019) and a non-significant activation in the right cerebellum ([18 –56 –38], Pcorr = 0.159). If the left superior frontal gyrus region reflected storage, it should still remain significant when the subtraction [TSDfast – IDfast] was also included in the conjunction, since total storage lasted 3.5 times longer in the TSDfast task compared with the TSDslow task. However, no voxels remained jointly significant in the latter conjunction.

The functional dissociation between orientation working memory and ultra-short-term orientation memory is illustrated in Figure 7, in which regions significant in the direct comparisons of the mnemonic components are displayed on a rendered brain. Clearly, orientation working memory engages fronto-parietal regions, while ultra-short-term memory for orientation engages occipito-temporal regions. Apparently, the occipito-temporal cortex of the two hemispheres seems engaged by ultra-short- term memory. The left temporal activation sites of Figure 7, however, correspond to a deactivation in the 2-back task (see Fig. 5). Thus only the right temporal sites correspond to activation in the TSD task. At first glance, the TSDslow and TSDfast tasks engaged disjunct regions in the right occipito-temporal cortex. This only reflects the stringent statistical criteria. Indeed, to investigate the impact of trial rate on successive discrimination in more detail, we calculated the percent change in differential activity between the TSD and ID tasks at several positions across the occipito-temporal cortex, i.e. from the right middle fusiform site, yielded by [TSDfast – IDfast] (A in Fig. 7), to the right middle temporal site, yielded by [TSDslow – IDslow] (B in Fig. 7), extending the probing into more anterior portions of temporal cortex. The percent change in rCBF in TSD compared with ID is plotted as a function of position for the fast and slow rates (Fig. 7). This spatial profile of neuronal activity shows that a large expense (~6 cm) of right occipito-temporal cortex is engaged in ultra-short-term memory for orientation. Posterior occipito-temporal regions are involved in ultra-short-term memory at a fast rate, while more anterior temporal regions are recruited during ultra-short-term memory at a slow rate. The reversal point is located between –50 and –40 mm in y Talairach coordinates.

Discussion

Our results demonstrate a functional dissociation between the mnemonic components of the 2back and successive discrimination tasks. Working memory for orientation engaged the left middle frontal gyrus (DLPFC, BA 9/46), left SFS (BA 6/8Ad) and left supramarginal gyrus (BA 40). Ultra-short-term- memory for orientation engaged a large expanse of the right occipito-temporal cortex, but the exact antero-posterior position depended on the trial rate.

Design Issues

The current tasks were equated not only for visual input, but also for level of performance by adapting the orientation difference δ between successive stimuli per subject and per task. In the current tasks difficulty is reflected mainly in performance level rather than in reaction times (see Materials and Methods). Although psychometric matching of tasks does not affect the mnemonic processes involved, the procedure ensures that differences in rCBF between tasks are not confounded by differences in task difficulty. Indeed, the prefrontal regions engaged by difficulty (Grady et al., 1996; Barch et al., 1997; Elliott et al., 1999; Sunaert et al., 1999) are located in front and below those observed in the present study.

The use of multiple (4), closely spaced orientations, which in addition changed in each 150 s run, made it impossible for subjects to use verbal labels. This was indicated by the sudden decrease in performance as soon as the number of orientations exceeded two or three, as well as by debriefing of the subjects. It may be that this use of a visual strategy enhanced the rehearsal and executive components of the 2back task. The use of oblique orientations and phase randomization excluded the contribution of vestibular, position or rotation cues.

The null hypothesis tested required us to compare the neural substrate at the extremes of the short-term memory/working memory spectrum. We therefore carefully matched the experimental tasks (TSD and 2back) with specific control tasks (ID and 0back, respectively). For instance, the 0back control task was equated with the 2back task for visual processing, attention demands, number and type of decisions and number of motor responses. Since subjects had to remember a predetermined orientation throughout the 0back task, it could be argued that this additional mnemonic component reduces differential activity between the experimental task and this control (D'Esposito et al., 1998a). The activity profiles of the left-sided parietal and prefrontal regions indicate that such a mnemonic component was not involved (Fig. 6A). Although we also performed the subtraction [2back – IDs], we preferred to use the subtraction [2back – 0back] because in these two tasks the type of decision (same or different) is the same.

Any subtraction directly involving the intermediate 1back task (i.e. [2back – 1back] and [1back – 0back]) was redundant in the testing of our null hypothesis. Other studies have performed such direct comparisons using letters (Jonides et al., 1997) and have provided evidence that as N increases, brain activation monotonically increases in a large number of regions related to working memory processes, as was the case in the three fronto-parietal regions engaged by orientation working memory (Fig. 6).

Orientation Ultra-short-term Memory: Impact of Trial Rate in Occipito-temporal Cortex

A large portion of the occipito-temporal cortex was involved in successive discrimination of orientation. This is in agreement with monkey lesion studies. Vogels and co-workers observed that only a complete infero-temporal (IT) lesion, but not a lesion restricted to the posterior or anterior part of IT, impaired successive orientation discrimination (Vogels et al., 1997). This study and its physiological counterpart (Vogels and Orban, 1994) were performed at 11 trials/min. Higher trial rates have not been tested in the monkey. Yet, the current results suggest a rate- dependent segregation within the occipito-temporal cortex during ultra-short-term memory (Fig. 7). To investigate the impact of trial rate in more detail, we performed a control PET study (Cornette et al., 1999), engaging 12 subjects in an identical successive orientation discrimination task, but now compared to a simple fixation (FIX) task. This baseline matches better the single cell studies, in which neuronal activity during a short-term memory/working memory task is generally compared with spontaneous activity (Miller et al., 1993). TSD task trial rate was parametrically varied among six experimental conditions. Percent change in activity in TSD compared with FIX (Fig. 8) was investigated in voxels closely matching the trajectory outlined in Figure 7 (see the figure legend for further details). This control study clarifies the rate-sensitive antero-posterior gradient across the occipito-temporal cortex. Indeed, compared with FIX, successive orientation discrimination at slow trial rates involved a large expanse of the occipito-temporal cortex, in agreement with the study of Vogels and co-workers (Vogels et al., 1997). In fact, for slow rates, activity increased from the posterior to anterior occipito-temporal cortex, which most likely reflects the larger number of delay cells in anterior compared with posterior regions (Fuster, 1990; Mikami, 1995). High trial rates preferentially drove more posterior parts of the occipito-temporal cortex, in agreement with single cell studies showing that more posterior cells in IT react better to simple stimuli than anterior cells (Miyashita and Chang, 1988; Tanaka et al., 1991). Animal experiments are unhelpful in interpreting the suppression of neuronal activity in the anterior parts of the occipito-temporal cortex at fast trial rates, since only slow trial rates have been tested. However, inhibition of neuronal activity has been demonstrated in the lateral temporal cortex of patients undergoing awake craniotomy while engaged in visuo-spatial short-term memory tasks (Holmes et al., 1996). Finally, it is worth noting that the rate gradient in the right occipito-temporal cortex fits the recent report of Elliott and Dolan on delayed matching tasks performed at two delay intervals (Elliott and Dolan, 1999). In that study, in which rate co-varied with delay interval, the right middle fusiform cortex ([32, –58, –18]) was more active for a 5 s than a 15 s delay, while the right middle temporal cortex ([58, –22, –12]) showed the opposite behavior.

One could argue that differential activity within the occipito-temporal cortex might underlie perceptual discrimination or temporal comparison, rather than the mnemonic storage component. This view is supported by our earlier report (Dupont et al., 1998) that simultaneous orientation discrimination, which shares a comparison component with the TSD task, also engages the right fusiform cortex. The two components, storage and comparison, are, however, not mutually exclusive. On the contrary, the right fusiform cortex may well be involved in both storage and the ensuing comparison, as single cell studies have shown for IT neurons during successive orientation discrimination [for a review see Orban and Vogels (Orban and Vogels, 1998)]. In accordance with this view, Haxby and co-workers have reported differential activity in the fusiform cortex for both simultaneous (i.e. no storage component present) and delayed comparison of faces for a range of relatively short delays (Haxby et al., 1995).

It is important to note that the functional gradient for the TSD task in occipito-temporal cortex closely parallels an anatomical dissociation within IT of macaque monkeys. Webster and co-workers reported that the posterior IT is scarcely connected with frontal areas, but strongly projects to parietal visual areas and to V3A (Webster et al., 1994). Similarly, ultra-short-term memory at a fast rate involved an occipito-temporal-parietal network (Table 4). Conversely, the anterior IT is predominantly connected with dorsal and ventral prefrontal areas, matching the occipito-temporal-frontal activity pattern observed at the slow trial rate (Table 4).

Orientation Working Memory: Comparison with Previous N-Back Studies

So far, several functional imaging studies have made use of the N-back paradigm to investigate the neural correlate of verbal working memory (Awh et al., 1995; Schumacher et al., 1996; Braver et al., 1997; Cohen et al., 1997; Jonides et al., 1997; D'Esposito et al., 1998a) and visuo-spatial working memory (Awh et al., 1995; Carlson et al., 1998; D'Esposito et al., 1998a; Owen et al., 1999). The present study is clearly unique, since no verbal or semantic strategies could be adopted due to the use of an elementary attribute, orientation. In addition, gratings were presented in the central position and steady fixation was required. Yet, our results are in excellent agreement with these previous N-back studies with respect to the three main regions involved. First, activation of both BA 9/46 (DLPFC) and 46 has consistently been observed in all verbal and spatial N-back studies. Second, SFS (BA 6/8Ad) activation has been observed in all spatial studies and in three (Awh et al., 1995; Braver et al., 1997; Cohen et al., 1997) of the six verbal N-back studies. Third, all verbal N-back studies consistently reported differential activation in left BA 40. The involvement of area BA 40 is less consistently seen in the spatial N-back studies (Awh et al., 1995; Carlson et al., 1998). Thus, the functional architecture of orientation working memory is similar to that of verbal and visuo-spatial working memory. Interestingly, a recent N-back study using numbers as stimuli (Callicott et al., 1999) also reported activation of DLPFC, SFS and the inferior parietal cortex. The fronto-parietal involvement in orientation working memory fits with the known anatomical connections, since the inferior parietal lobe strongly projects to BA 46/9, 46, 6 and 8Ad in the monkey (Petrides and Pandya, 1999). It also fits the co-activation of prefrontal cortex (PFC) and inferior parietal cortex in working memory tasks as revealed by 2DG functional mapping in the monkey (Friedman and Goldman-Rakic, 1994).

Separate Cortical Stores for Orientation Working Memory and Ultra-short-term Memory

The current study provides evidence for a functional dissociation between orientation working memory, underlain by a fronto- parietal network, and ultra-short-term memory, underlain by the occipito-temporal cortex. Hence, our results do not support the hypothesis that orientation short-term memory/working memory tasks involve the same anatomical region(s) for the mnemonic storage component. Although several brain regions yielded by the subtractions [TSD – ID] and [2back – 0back] may underlie non-mnemonic cognitive task components, the observed functional dissociation implies that storage during orientation ultra-short-term memory and working memory involves different neuronal populations. Functional imaging, however, cannot distinguish between two alternatives: either a neuronal population may be more active in one type of memory than in the other or a population may be active in one type of memory and not active at all in the other.

This dissociation closely parallels the neural mechanisms of visual short-term memory/working memory as demonstrated in the monkey. When using a delayed match to a sample (DMS) task with multiple intervening non-match stimuli, delay activity in the PFC was unaffected by these intervening stimuli, in contrast to IT neuronal activity (Miller et al., 1994, 1996). This indicates that delay activity in the PFC represents the sample stimulus, while in IT cortex it represents the last stimulus shown. Hence, it is not surprising that orientation ultra-short-term memory, which only represents the last stimulus shown, exclusively relies on occipito-temporal cortex activity. That the PFC is not engaged in ultra-short-term memory might reflect a task dependency of prefrontal activity, not unlike that documented for the auditory (Colombo et al., 1990) and visual (Orban et al., 1997) systems. Earlier studies have indeed indicated that during psychophysical tasks, such as simple discriminations, the PFC is deactivated (Orban et al., 1997). However, one may raise the question whether the small percentage of total time taken by the memory processes in the successive discrimination tasks could account for the absence of PFC activation. This is unlikely for two reasons. First, the dissociation was still present for fast successive discrimination in which storage takes at least 18% of the total time, rather than 5% at the slow rate. Second, if percentage of total time was the explanation, it would be impossible to see activation in the temporal cortex since both single unit (Miller et al., 1994) and functional imaging data (Courtney et al., 1997) indicate that delay activity in the occipito-temporal cortex is, if anything, smaller than that in the PFC.

Overall, our results indicate, in agreement with primate single cell studies, that if delay activity represents the neuronal correlate of storage in short-term memory/working memory tasks, delay activity in different brain regions reflects storage in ultra-short-term memory and working memory.

Notes

We thank M. Bex, D. Crombez, T. De Groot, P. Pitschon, K. Stessel, L. Verhaegen, V. Van den Maegdenbergh and S. Vleugels for the assistance during PET scanning, E. Beatse for the MRI scanning, P. Falleyn for help with the software programming, P. Kayenbergh and G. Meulemans for technical help, A. Rosier for help with training the subjects and E. Salmon and S. Raiguel for invaluable comments on the manuscript. We are much indebted to Prof. R. Frackowiak for making available the SPM software and to Prof. Suetens for providing us with the MIRIT software. This work was supported by a grant from the Queen Elisabeth Medical Foundation to L. Mortelmans and grant 0358.98 from the Fund for Scientific Research–Flanders (Belgium). L.C. is a research assistant of the Fund for Scientific Research–Flanders (Belgium). P.D. is a post-doctoral fellow of the Fund for Scientific Research–Flanders (Belgium).

Address correspondence to Prof. Dr G.A. Orban, Laboratorium voor Neuro- en Psychofysiologie, Katholieke Universiteit Leuven, Campus Gasthuisberg, Herestraat 49, B-3000 Leuven, Belgium. Email: guy.orban@med.kuleuven.ac.be.

Table 1

[2back – 0back]

Brain region Coordinates (x, y, z[2b – 0b] [2b – 0b] – [TSDs – IDs] [2b – 0b] – (TSDf – IDf) 
  Z score % change Z score Z score 
x, y, z are the Talairach coordinates of the local maxima (in mm); x = 0 at the midline (+/– , right/left-sided); y = 0 at the anterior commissure (+/–, anterior/posterior); z = 0 at the AC-PC level (+/–, superior/inferior). BA, Brodmann area. Z score designates the level of significance: bold, activation significant at Pcorr < 0.05 (Z score > 4.40) for peak height; regular, activation significant at Puncorr < 0.001 (Z score > 3.09) for peak height. Percent change in rCBF in brackets. Underlined Z scores refer to regions significant at Puncorr < 0.001 in both [2back – 0back] and the direct comparison of the working memory network with ultra-short-term memory at slow or at fast trial rate. 0b, 0back task; 1b, 1back task; 2b, 2back task; TSDs, TSDslow task; IDs, IDslow task; TSDf, TSDfast task; IDf, IDfast task. 
Frontal      
 L middle frontal gyrus BA 9/46 –32 24 30 4.58 [2.25] 3.10 4.31 
 L middle frontal gyrus BA 46 –34 38 22 3.84 [1.92] 3.33 2.80 
 L superior frontal sulcus BA 6/8Ad –26 4 62 4.56 [2.77] 4.74 4.28 
 R superior frontal sulcus BA 6/8Ad  32 10 58 3.31 [2.32] 3.84 2.49 
 L precentral sulcus BA 6/44 –42 –4 46 3.61 [1.89] 3.97 2.94 
 R precentral sulcus BA 6/44  32 –2 62 3.30 [2.14] 4.04 2.52 
Parietal      
 L supramarginal gyrus BA 40 –46 –44 30 5.22 [2.67] 4.03 3.91 
 L superior parietal lobe BA 7 –20 –62 60 3.82 [2.40] 4.19 2.51 
Cerebellum      
 R inferior cerebellum  18 –58 –42 3.53 [2.42] 1.71 3.61 
 R middle cerebellum  14 –62 –24 3.38 [1.87] 2.17 2.96 
Brain region Coordinates (x, y, z[2b – 0b] [2b – 0b] – [TSDs – IDs] [2b – 0b] – (TSDf – IDf) 
  Z score % change Z score Z score 
x, y, z are the Talairach coordinates of the local maxima (in mm); x = 0 at the midline (+/– , right/left-sided); y = 0 at the anterior commissure (+/–, anterior/posterior); z = 0 at the AC-PC level (+/–, superior/inferior). BA, Brodmann area. Z score designates the level of significance: bold, activation significant at Pcorr < 0.05 (Z score > 4.40) for peak height; regular, activation significant at Puncorr < 0.001 (Z score > 3.09) for peak height. Percent change in rCBF in brackets. Underlined Z scores refer to regions significant at Puncorr < 0.001 in both [2back – 0back] and the direct comparison of the working memory network with ultra-short-term memory at slow or at fast trial rate. 0b, 0back task; 1b, 1back task; 2b, 2back task; TSDs, TSDslow task; IDs, IDslow task; TSDf, TSDfast task; IDf, IDfast task. 
Frontal      
 L middle frontal gyrus BA 9/46 –32 24 30 4.58 [2.25] 3.10 4.31 
 L middle frontal gyrus BA 46 –34 38 22 3.84 [1.92] 3.33 2.80 
 L superior frontal sulcus BA 6/8Ad –26 4 62 4.56 [2.77] 4.74 4.28 
 R superior frontal sulcus BA 6/8Ad  32 10 58 3.31 [2.32] 3.84 2.49 
 L precentral sulcus BA 6/44 –42 –4 46 3.61 [1.89] 3.97 2.94 
 R precentral sulcus BA 6/44  32 –2 62 3.30 [2.14] 4.04 2.52 
Parietal      
 L supramarginal gyrus BA 40 –46 –44 30 5.22 [2.67] 4.03 3.91 
 L superior parietal lobe BA 7 –20 –62 60 3.82 [2.40] 4.19 2.51 
Cerebellum      
 R inferior cerebellum  18 –58 –42 3.53 [2.42] 1.71 3.61 
 R middle cerebellum  14 –62 –24 3.38 [1.87] 2.17 2.96 
Table 2

[TSDfast – IDfast]

Brain region Coordinates (x, y, z[TSDf – IDf] [TSDf – Idf] – [2b – 0b] 
  Z score % change Z score 
See Table 1 for abbreviations and conventions. 
Parietal     
 L parieto-occipital sulcus BA 31 –20 –66 22 3.59 [1.78] 2.44 
Occipital     
 L posterior lingual gyrus BA 18 –14 –96 –20 4.63 [3.19] 4.12 
 R transverse occipital sulcus BA 18/19  26 –86 28 3.57 [1.74] 2.64 
Temporal     
 R middle fusiform gyrus BA 19/37  46 –64 –4 3.76 [2.06] 3.38 
  52 –56 –14 3.61 [2.10] 3.06 
  38 –50 –12 3.31 [2.18] 2.79 
 R parahippocampal gyrus BA 28/36  30 –16 –32 3.52 [2.56] 2.53 
  24 –24 –12 3.20 [1.91] 2.18 
 L middle fusiform gyrus BA 19/37 –44 –64 4 3.28 [1.69] 2.77 
 L superior temporal gyrus BA 38 –40 12 –16 3.19 [1.78] 3.07 
Brain region Coordinates (x, y, z[TSDf – IDf] [TSDf – Idf] – [2b – 0b] 
  Z score % change Z score 
See Table 1 for abbreviations and conventions. 
Parietal     
 L parieto-occipital sulcus BA 31 –20 –66 22 3.59 [1.78] 2.44 
Occipital     
 L posterior lingual gyrus BA 18 –14 –96 –20 4.63 [3.19] 4.12 
 R transverse occipital sulcus BA 18/19  26 –86 28 3.57 [1.74] 2.64 
Temporal     
 R middle fusiform gyrus BA 19/37  46 –64 –4 3.76 [2.06] 3.38 
  52 –56 –14 3.61 [2.10] 3.06 
  38 –50 –12 3.31 [2.18] 2.79 
 R parahippocampal gyrus BA 28/36  30 –16 –32 3.52 [2.56] 2.53 
  24 –24 –12 3.20 [1.91] 2.18 
 L middle fusiform gyrus BA 19/37 –44 –64 4 3.28 [1.69] 2.77 
 L superior temporal gyrus BA 38 –40 12 –16 3.19 [1.78] 3.07 
Table 3

[TSDslow – IDslow]

Brain region Coordinates (x, y, z[TSDs – IDs] [TSDs – Ids] – [2b – 0b] 
  Z score % change Z score 
See Table 1 for abbreviations and conventions. 
Frontal     
 L superior frontal gyrus BA 9  –28 46 42 4.53 [2.86] 2.34 
  –6 54 40 3.31 [2.35] 3.67 
 L superior frontal gyrus BA 10  –22 62 26 4.12 [3.23] 3.10 
Cingulate     
 Posterior cingulate gyrus BA 31  –8 –32 34 3.67 [2.09] 3.71 
 Anterior cingulate gyrus BA 24/32  0 22 –8 3.44 [2.32] 3.31 
Occipital     
 L middle occipital gyrus BA 19  –50 –72 30 3.40 [2.05] 2.67 
Temporal     
 R inferior temporal gyrus BA 20  58 –42 –34 4.39 [3.92] 4.05 
 R middle temporal gyrus BA 21/37  32 –46 6 3.73 [2.37] 2.51 
 L parahippocampal gyrus BA 28/36  –18 –8 –24 3.55 [2.52] 3.27 
 L middle temporal gyrus BA 21  –54 8 –28 3.31 [2.64] 4.52 
Brain region Coordinates (x, y, z[TSDs – IDs] [TSDs – Ids] – [2b – 0b] 
  Z score % change Z score 
See Table 1 for abbreviations and conventions. 
Frontal     
 L superior frontal gyrus BA 9  –28 46 42 4.53 [2.86] 2.34 
  –6 54 40 3.31 [2.35] 3.67 
 L superior frontal gyrus BA 10  –22 62 26 4.12 [3.23] 3.10 
Cingulate     
 Posterior cingulate gyrus BA 31  –8 –32 34 3.67 [2.09] 3.71 
 Anterior cingulate gyrus BA 24/32  0 22 –8 3.44 [2.32] 3.31 
Occipital     
 L middle occipital gyrus BA 19  –50 –72 30 3.40 [2.05] 2.67 
Temporal     
 R inferior temporal gyrus BA 20  58 –42 –34 4.39 [3.92] 4.05 
 R middle temporal gyrus BA 21/37  32 –46 6 3.73 [2.37] 2.51 
 L parahippocampal gyrus BA 28/36  –18 –8 –24 3.55 [2.52] 3.27 
 L middle temporal gyrus BA 21  –54 8 –28 3.31 [2.64] 4.52 
Table 4

[TSD – ID]

Brain region Coordinates (x, y, z[TSDf – IDf] – [TSDs – IDs] [TSDs – IDs] –[TSDf – IDf] 
  Z score  
See Table 1 for abbreviations and conventions. 
Frontal    
 R precentral sulcus BA 6/44  42 4 38 3.76  
 L superior frontal gyrus BA 10  –24 64 22  4.12 
  –24 58 32  3.51 
 L superior frontal gyrus BA 9  –28 48 44  3.70 
 L medial frontal gyrus BA 6  –2 –12 66  3.22 
Parietal    
 R superior parietal lobe BA 7  30 –58 54 4.44  
 L inferior parietal lobe BA 40  –36 –36 48 3.20  
 L superior parietal lobe BA 7  –40 –56 58 3.12  
Occipital    
 L transverse occipital sulcus BA 18/19  –24 –70 28 4.23  
 R transverse occipital sulcus BA 18/19  26 –86 24 4.22  
 L posterior lingual gyrus BA 18 –10 –92 –18 3.85  
Temporal    
 R middle fusiform gyrus BA 19/37  48 –62 –4 3.66  
 R inferior temporal gyrus BA 20  58 –42 –34  4.15 
Brain region Coordinates (x, y, z[TSDf – IDf] – [TSDs – IDs] [TSDs – IDs] –[TSDf – IDf] 
  Z score  
See Table 1 for abbreviations and conventions. 
Frontal    
 R precentral sulcus BA 6/44  42 4 38 3.76  
 L superior frontal gyrus BA 10  –24 64 22  4.12 
  –24 58 32  3.51 
 L superior frontal gyrus BA 9  –28 48 44  3.70 
 L medial frontal gyrus BA 6  –2 –12 66  3.22 
Parietal    
 R superior parietal lobe BA 7  30 –58 54 4.44  
 L inferior parietal lobe BA 40  –36 –36 48 3.20  
 L superior parietal lobe BA 7  –40 –56 58 3.12  
Occipital    
 L transverse occipital sulcus BA 18/19  –24 –70 28 4.23  
 R transverse occipital sulcus BA 18/19  26 –86 24 4.22  
 L posterior lingual gyrus BA 18 –10 –92 –18 3.85  
Temporal    
 R middle fusiform gyrus BA 19/37  48 –62 –4 3.66  
 R inferior temporal gyrus BA 20  58 –42 –34  4.15 
Figure 1.

A possible taxonomy of short-term/working memory tasks. STM/WM tasks are plotted along three axes: (i) the x-axis, indicating the duration of item storage; (ii) the y-axis, indicating the presence or absence of executive operations; (iii) the z-axis, indicating the number of items to be stored. Ultra-short-term memory indicates the capacity to retain only one item over a delay interval of 300 ms, without executive control. Short-term or immediate memory covers a non-distracted delay interval, generally a couple of seconds, again without executive control. Working or operant memory involves maintenance, whether or not accompanied by executive operations. TSD, temporal same different or successive discrimination task.

Figure 1.

A possible taxonomy of short-term/working memory tasks. STM/WM tasks are plotted along three axes: (i) the x-axis, indicating the duration of item storage; (ii) the y-axis, indicating the presence or absence of executive operations; (iii) the z-axis, indicating the number of items to be stored. Ultra-short-term memory indicates the capacity to retain only one item over a delay interval of 300 ms, without executive control. Short-term or immediate memory covers a non-distracted delay interval, generally a couple of seconds, again without executive control. Working or operant memory involves maintenance, whether or not accompanied by executive operations. TSD, temporal same different or successive discrimination task.

Figure 2.

Schematic representation of stimulus timing in five tasks. (A) 2back task; (B) 1back task; (C) 0back task; (D) IDslow task; (E) TSDslow task. Duration of stimulus presentation is 500 ms in all conditions. Inter-trial interval (ITI) is 2500 ms in (A)–(D), 4700 ms in (E), with an inter-stimulus interval of 300 ms. Orientations were chosen within the 20–70° range. Arrows point at the stimulus with which a subsequent grating has to be compared. The grating orientations shown in the upper row refer to the 2back task, requiring two series of gratings. Seed orientation of series 1 = 65°; seed orientation of series 2 = 40°; subsequent orientations are 53, 40, 65 and 28° (δ = 12°). The gratings shown in the middle row refer to the 0back task. The orientation of the ‘0 stimulus’ = 60°; subsequent orientations are 42, 33, 60, 51 and 24° (δ = 9°). The gratings shown in the lower row refer to the TSD task. Orientations shown are 35–43, 55–55 and 25–33° (δ = 8°). Notice that although the temporal spacing of stimuli in the TSDslow task is different, the number of stimuli per minute equals that of the other tasks.

Figure 2.

Schematic representation of stimulus timing in five tasks. (A) 2back task; (B) 1back task; (C) 0back task; (D) IDslow task; (E) TSDslow task. Duration of stimulus presentation is 500 ms in all conditions. Inter-trial interval (ITI) is 2500 ms in (A)–(D), 4700 ms in (E), with an inter-stimulus interval of 300 ms. Orientations were chosen within the 20–70° range. Arrows point at the stimulus with which a subsequent grating has to be compared. The grating orientations shown in the upper row refer to the 2back task, requiring two series of gratings. Seed orientation of series 1 = 65°; seed orientation of series 2 = 40°; subsequent orientations are 53, 40, 65 and 28° (δ = 12°). The gratings shown in the middle row refer to the 0back task. The orientation of the ‘0 stimulus’ = 60°; subsequent orientations are 42, 33, 60, 51 and 24° (δ = 9°). The gratings shown in the lower row refer to the TSD task. Orientations shown are 35–43, 55–55 and 25–33° (δ = 8°). Notice that although the temporal spacing of stimuli in the TSDslow task is different, the number of stimuli per minute equals that of the other tasks.

Figure 3.

Polar plots of presented orientations. Distributions of all orientations presented to the 14 subjects are plotted for each task. Only oblique orientations between both the 20–70 and 110–160° ranges (oblique dashed lines) were presented. Orientations were first transformed into radians. The proportion of each orientation presented during a specific task was then calculated and multiplied by the corresponding cosine (yielding the x-axis values) and sine (yielding the y-axis values) of each orientation. The plots were smoothed using a bin width of 10°.

Figure 3.

Polar plots of presented orientations. Distributions of all orientations presented to the 14 subjects are plotted for each task. Only oblique orientations between both the 20–70 and 110–160° ranges (oblique dashed lines) were presented. Orientations were first transformed into radians. The proportion of each orientation presented during a specific task was then calculated and multiplied by the corresponding cosine (yielding the x-axis values) and sine (yielding the y-axis values) of each orientation. The plots were smoothed using a bin width of 10°.

Figure 4.

Psychophysical data. Histograms showing, for each task, performance expressed as: (A) mean percent correct responses; (B) mean orientation difference; (C) mean reaction time. Vertical bars indicate standard error of the mean (SEM).

Figure 4.

Psychophysical data. Histograms showing, for each task, performance expressed as: (A) mean percent correct responses; (B) mean orientation difference; (C) mean reaction time. Vertical bars indicate standard error of the mean (SEM).

Figure 5.

Anatomical localization of regions significant in the subtractions [2back – 0back] and [0back – 2back]. (Upper) SPMs showing the regions differentially active in the subtractions [2back – 0back] and [0back – 2back], superimposed on a rendered brain. Red indicates voxels significant at Puncorr < 0.001; pink indicates voxels significant at Pcorr < 0.05 in the subtraction [2back – 0back]; green indicates voxels significant at Puncorr < 0.001 in [0back – 2back], i.e. left middle temporal gyrus (BA 21, [–58 –2 –24] and [–54 8 –30]). (Middle) Local maxima (red crosses) of the SPMs corresponding to left DLPFC, SFS and supramarginal gyrus superimposed on coronal sections through the averaged MRI of all 14 subjects. The significance threshold for display is set at Puncorr < 0.001. The y Talairach coordinate for each coronal section and color scale of significance are indicated. L, left hemisphere. (Lower) Local maxima superimposed on the individual MRIs of three subjects (I–III). The following anatomical landmarks are labeled: (1), inferior frontal sulcus; (2), middle frontal sulcus; (3), precentral sulcus; (4), superior frontal sulcus (caudal part); (5), Sylvian fissure; (6), postcentral sulcus.

Figure 5.

Anatomical localization of regions significant in the subtractions [2back – 0back] and [0back – 2back]. (Upper) SPMs showing the regions differentially active in the subtractions [2back – 0back] and [0back – 2back], superimposed on a rendered brain. Red indicates voxels significant at Puncorr < 0.001; pink indicates voxels significant at Pcorr < 0.05 in the subtraction [2back – 0back]; green indicates voxels significant at Puncorr < 0.001 in [0back – 2back], i.e. left middle temporal gyrus (BA 21, [–58 –2 –24] and [–54 8 –30]). (Middle) Local maxima (red crosses) of the SPMs corresponding to left DLPFC, SFS and supramarginal gyrus superimposed on coronal sections through the averaged MRI of all 14 subjects. The significance threshold for display is set at Puncorr < 0.001. The y Talairach coordinate for each coronal section and color scale of significance are indicated. L, left hemisphere. (Lower) Local maxima superimposed on the individual MRIs of three subjects (I–III). The following anatomical landmarks are labeled: (1), inferior frontal sulcus; (2), middle frontal sulcus; (3), precentral sulcus; (4), superior frontal sulcus (caudal part); (5), Sylvian fissure; (6), postcentral sulcus.

Figure 6.

Functional profiles. (A) Functional profiles of three regions significant at Pcorr < 0.05 in the subtraction [2back – 0back]: left DLPFC (BA 9/46), left SFS (BA 6/8Ad) and left supramarginal gyrus (BA 40). The adjusted rCBF (y-axis) is plotted for the seven different conditions (x-axis, from left to right: 0b, 0back task; 1b, 1back task; 2b, 2back task; TSDs, TSDslow task; IDs, IDslow task; TSDf, TSDfast task; IDf, IDfast task). Vertical bars indicate SEM. (B) Functional profiles corresponding to the subtraction [TSDfast – IDfast]: right middle fusiform gyrus (BA 19/37), left posterior lingual gyrus (BA 18) and right transverse occipital sulcus (BA 18). (C) Functional profiles corresponding to the subtraction [TSDslow – IDslow]: left superior frontal gyrus (BA 9), right inferior temporal gyrus and posterior cingulate (BA 31).

Figure 6.

Functional profiles. (A) Functional profiles of three regions significant at Pcorr < 0.05 in the subtraction [2back – 0back]: left DLPFC (BA 9/46), left SFS (BA 6/8Ad) and left supramarginal gyrus (BA 40). The adjusted rCBF (y-axis) is plotted for the seven different conditions (x-axis, from left to right: 0b, 0back task; 1b, 1back task; 2b, 2back task; TSDs, TSDslow task; IDs, IDslow task; TSDf, TSDfast task; IDf, IDfast task). Vertical bars indicate SEM. (B) Functional profiles corresponding to the subtraction [TSDfast – IDfast]: right middle fusiform gyrus (BA 19/37), left posterior lingual gyrus (BA 18) and right transverse occipital sulcus (BA 18). (C) Functional profiles corresponding to the subtraction [TSDslow – IDslow]: left superior frontal gyrus (BA 9), right inferior temporal gyrus and posterior cingulate (BA 31).

Figure 7.

Rendered images of the direct comparisons between working memory and ultra-short-term memory for slow and fast rates. (Upper) Red and pink colors (conventions as in Fig. 5) indicate voxels differentially active in the subtractions [2back – 0back] – [TSDfast – IDfast] (left column) and [2back – 0back] – [TSDslow – IDslow] (right column). Blue color (all voxels significant at Puncorr < 0.001) indicate voxels differentially active in the subtractions [TSDfast – IDfast] – [2back – 0back] (left column) and [TSDslow – IDslow] – [2back – 0back] (right column). Apart from two left cerebellar regions ([–24 –42 –50] and [–42 –50 –36]), all regions significant at Puncorr < 0.001 in [2back – 0back] – [TSDfast – IDfast] correspond to those indicated in Table 1 (column 3, underlined Z scores). Apart from a left cerebellar focus ([–42 –50 –38]), all regions significant at Puncorr < 0.001 in [2back – 0back] – [TSDslow – IDslow] correspond to those listed in Table 1 (column 5, underlined Z scores). Regions significant at Puncorr < 0.001 in [TSDfast – IDfast] – [2back – 0back] are indicated in Table 2 (column 5, underlined Z scores). Additional activations, shown in the SPMs, were observed in right medial frontal gyrus (BA 9, [6 50 12]), right inferior frontal gyrus (BA 47, [48 32 –10]), right superior frontal gyrus (BA 9, [6 46 46]), left middle temporal gyrus (BA 21, [–54 4 –28]), right middle temporal gyrus (BA 21, [36 –10 –32]), left inferior temporal gyrus (BA 20, [–64 –24 –30]) and right cerebellum ([30 –86 –34]). Regions significant at Puncorr < 0.001 in [TSDslow – IDslow] – [2back – 0back] include those listed in Table 3 (column 5, underlined Z scores). Additional activations were seen in left inferior occipital gyrus (BA 19, [–42 –88 –12]), left middle fusiform gyrus (BA 19/37, [–32 –48 –8]), left superior temporal gyrus (BA 38, [-28 10 -38]) and right middle temporal gyrus (BA 21, [48 6 -32]). This figure also serves to illustrate the activity patterns involved in ultra-short-term memory, i.e. [TSDfast – IDfast] (Table 2) and [TSDslow – IDslow] (Table 3), since the blue labeled activity pattern resulting from a direct comparison between ultra-short-term and working memory largely includes the neural substrate yielded by the original ultra-short-term memory subtractions. Note the overlap between left middle temporal regions for slow and fast ultra-short-term memory, due to deactivation in working memory (see the subtraction [0back – 2back], green voxels in Fig. 5) rather than to increased activation in ultra-short-term memory. (Lower) Spatial profile of neuronal activity in occipito-temporal cortex during the ultra-short-term memory task at fast and slow trial rates. (Left) Percent change in adjusted rCBF (y-axis), plotted for 10 voxels across the occipito-temporal cortex (y Talairach coordinate on x-axis). The full line indicates [TSDfast – IDfast], the stippled line [TSDslow – IDslow]. Voxel A corresponds to [46 –64 –4] resulting from [TSDfast – IDfast], voxel B corresponds to [58 –42 –34] resulting from [TSDslow – IDslow]. Arrows indicate their position on the above SPMs. Percent change in adjusted rCBF was calculated for additional voxels in between A and B ([48 –60 –10], [50 –56 –14], [52 –52 –20] and [56 –44 –30]) and also in the anteroventral portion of the occipito-temporal cortex ([56 –38 –26], [56 –30 –26], [56 –20 –30] and [56 –10 –34]). (Right) A sagittal section (x = 52 mm in Talairach space) through the mean MRI of all participating subjects with the 10 voxels analyzed superimposed. Voxels A and B are indicated.

Rendered images of the direct comparisons between working memory and ultra-short-term memory for slow and fast rates. (Upper) Red and pink colors (conventions as in Fig. 5) indicate voxels differentially active in the subtractions [2back – 0back] – [TSDfast – IDfast] (left column) and [2back – 0back] – [TSDslow – IDslow] (right column). Blue color (all voxels significant at Puncorr < 0.001) indicate voxels differentially active in the subtractions [TSDfast – IDfast] – [2back – 0back] (left column) and [TSDslow – IDslow] – [2back – 0back] (right column). Apart from two left cerebellar regions ([–24 –42 –50] and [–42 –50 –36]), all regions significant at Puncorr < 0.001 in [2back – 0back] – [TSDfast – IDfast] correspond to those indicated in Table 1 (column 3, underlined Z scores). Apart from a left cerebellar focus ([–42 –50 –38]), all regions significant at Puncorr < 0.001 in [2back – 0back] – [TSDslow – IDslow] correspond to those listed in Table 1 (column 5, underlined Z scores). Regions significant at Puncorr < 0.001 in [TSDfast – IDfast] – [2back – 0back] are indicated in Table 2 (column 5, underlined Z scores). Additional activations, shown in the SPMs, were observed in right medial frontal gyrus (BA 9, [6 50 12]), right inferior frontal gyrus (BA 47, [48 32 –10]), right superior frontal gyrus (BA 9, [6 46 46]), left middle temporal gyrus (BA 21, [–54 4 –28]), right middle temporal gyrus (BA 21, [36 –10 –32]), left inferior temporal gyrus (BA 20, [–64 –24 –30]) and right cerebellum ([30 –86 –34]). Regions significant at Puncorr < 0.001 in [TSDslow – IDslow] – [2back – 0back] include those listed in Table 3 (column 5, underlined Z scores). Additional activations were seen in left inferior occipital gyrus (BA 19, [–42 –88 –12]), left middle fusiform gyrus (BA 19/37, [–32 –48 –8]), left superior temporal gyrus (BA 38, [-28 10 -38]) and right middle temporal gyrus (BA 21, [48 6 -32]). This figure also serves to illustrate the activity patterns involved in ultra-short-term memory, i.e. [TSDfast – IDfast] (Table 2) and [TSDslow – IDslow] (Table 3), since the blue labeled activity pattern resulting from a direct comparison between ultra-short-term and working memory largely includes the neural substrate yielded by the original ultra-short-term memory subtractions. Note the overlap between left middle temporal regions for slow and fast ultra-short-term memory, due to deactivation in working memory (see the subtraction [0back – 2back], green voxels in Fig. 5) rather than to increased activation in ultra-short-term memory. (Lower) Spatial profile of neuronal activity in occipito-temporal cortex during the ultra-short-term memory task at fast and slow trial rates. (Left) Percent change in adjusted rCBF (y-axis), plotted for 10 voxels across the occipito-temporal cortex (y Talairach coordinate on x-axis). The full line indicates [TSDfast – IDfast], the stippled line [TSDslow – IDslow]. Voxel A corresponds to [46 –64 –4] resulting from [TSDfast – IDfast], voxel B corresponds to [58 –42 –34] resulting from [TSDslow – IDslow]. Arrows indicate their position on the above SPMs. Percent change in adjusted rCBF was calculated for additional voxels in between A and B ([48 –60 –10], [50 –56 –14], [52 –52 –20] and [56 –44 –30]) and also in the anteroventral portion of the occipito-temporal cortex ([56 –38 –26], [56 –30 –26], [56 –20 –30] and [56 –10 –34]). (Right) A sagittal section (x = 52 mm in Talairach space) through the mean MRI of all participating subjects with the 10 voxels analyzed superimposed. Voxels A and B are indicated.

Figure 8.

Control experiment: spatial profile of neuronal activity in occipito-temporal cortex during [TSD-FIX]. (Upper) Percent change in adjusted rCBF compared with FIX (y-axis), plotted for 12 voxels across occipito-temporal cortex (y Talairach coordinate on x-axis). To simplify visualization (subjects were engaged in successive discrimination at six different trial rates), neuronal activity is plotted for the mean of slow trial rates (5 and 11 trials/min, stippled line), the mean of intermediate trial rates (17 and 23 trials/min, dashed line) and the mean of fast trial rates (29 and 35 trials/min, full line). Two additional voxels were included in the probing trajectory, i.e. voxels C ([48 –72 –6]) and D ([60 –2 –30]), constituting the most posterior and anterior parts of the occipito-temporal cortex, respectively, as directly yielded by comparing all six TSD tasks to their control. All other 10 voxels, including voxels A and B, were selected from the SPM{F} map under the constraint of matching as closely as possible the trajectory outlined in Figure 7. (Lower) Probing locations superimposed on a sagittal section (x = 52 mm in Talairach space) through the mean MRI of all participating subjects. Voxels A–D are indicated.

Control experiment: spatial profile of neuronal activity in occipito-temporal cortex during [TSD-FIX]. (Upper) Percent change in adjusted rCBF compared with FIX (y-axis), plotted for 12 voxels across occipito-temporal cortex (y Talairach coordinate on x-axis). To simplify visualization (subjects were engaged in successive discrimination at six different trial rates), neuronal activity is plotted for the mean of slow trial rates (5 and 11 trials/min, stippled line), the mean of intermediate trial rates (17 and 23 trials/min, dashed line) and the mean of fast trial rates (29 and 35 trials/min, full line). Two additional voxels were included in the probing trajectory, i.e. voxels C ([48 –72 –6]) and D ([60 –2 –30]), constituting the most posterior and anterior parts of the occipito-temporal cortex, respectively, as directly yielded by comparing all six TSD tasks to their control. All other 10 voxels, including voxels A and B, were selected from the SPM{F} map under the constraint of matching as closely as possible the trajectory outlined in Figure 7. (Lower) Probing locations superimposed on a sagittal section (x = 52 mm in Talairach space) through the mean MRI of all participating subjects. Voxels A–D are indicated.

References

Awh E, Smith EE, Jonides J (
1995
) Human rehearsal processes and the frontal lobes: PET evidence.
Ann NY Acad Sci
 
769
:
97
–118.
Baddeley AD (1990) Human memory, theory and practice. Hove: Lawrence Erlbaum Associates.
Baddeley AD, Hitch G (1974) Working memory. In: Recent advances in learning and motivation, Vol. 8 (Bower GA, ed.), pp. 47–90. New York: Academic Press.
Barch DM, Braver TS, Nystrom LE, Forman SD, Noll DC, Cohen JD (
1997
) Dissociating working memory from task difficulty in human prefrontal cortex.
Neuropsychologia
 
35
:
1373
–1380.
Brandt T, Dieterich M, Danek A (
1994
) Vestibular cortex lesions affect the perception of verticality.
Ann Neurol
 
35
:
403
–412.
Braver TS, Cohen JD, Nystrom LE, Jonides J, Smith EE, Noll DC (
1997
) A parametric study of prefrontal cortex involvement in human working memory.
NeuroImage
 
5
:
49
–62.
Brix G, Zaers J, Adam LE, Belleman ME, Ostertag H, Trojan H, Haberkorn U, Doll J, Oderdorfer F, Lorenz WJ (
1997
) Performance evaluation of a whole-body PET scanner using the NEMA protocol.
J Nucl Med
 
38
:
1614
–1623.
Callicott JH, Mattay VS, Bertolino A, Finn K, Coppola R, Frank JA, Goldberg TE, Weinberger DR (
1999
) Physiological characteristics of capacity constraints in working memory as revealed by functional MRI.
Cereb Cortex
 
9
:
20
–26.
Carlson S, Martinkauppi S, Rama P, Salli E, Korvenoja A, Aronen HJ (
1998
) Distribution of cortical activation during visuo-spatial N-back tasks as revealed by functional magnetic resonance imaging.
Cereb Cortex
 
8
:
743
–752.
Cohen JD, Perlstein WM, Braver TS, Nystrom LE, Noll DC, Jonides J, Smith EE (
1997
) Temporal dynamics of brain activation during a working memory task.
Nature
 
386
:
604
–608.
Colombo M, D'Amato MR, Rodman HR, Gross CG (
1990
) Auditory association cortex lesions impair auditory short-term memory in monkeys.
Science
 
247
:
336
–338.
Coltheart M (
1983
) Iconic memory.
Phil Trans R Soc Lond
 
302
:
283
–294.
Cornette L, Dupont P, Rosier A, Sunaert S, Van Hecke P, Michiels J, Mortelmans L, Orban GA (
1998
) Human brain regions involved in direction discrimination.
J Neurophysiol
 
79
:
2749
–2765.
Cornette L, Dupont P, Peuskens H, Bormans G, Claeys K, De Schutter E, Mortelmans L, Orban GA (
1999
) Rate dependence of task-related cerebral activations: a PET-study.
NeuroImage
 
9
:
S856
.
Courtney SM, Ungerleider LG, Keil K, Haxby JV (
1996
) Object and spatial visual working memory activate separate neural systems in human cortex.
Cereb Cortex
 
6
:
39
–49.
Courtney SM, Ungerleider L, Keil K, Haxby JV (
1997
) Transient and sustained activity in a distributed neural system for human working memory.
Nature
 
386
:
608
–611.
D'Esposito M, Ballard D, Aguirre GK, Zarahn E (
1998
) Human prefrontal cortex is not specific for working memory: a functional MRI study.
NeuroImage
 
8
:
274
–282.
D'Esposito M, Aguirre GK, Zarahn E, Ballard D, Shin RK, Lease J (
1998
) Functional MRI studies of spatial and nonspatial working memory.
Cogn Brain Res
 
7
:
1
–13.
Di Lollo V (
1980
) Temporal integration in visual memory.
J Exp Psychol
 
109
:
75
–97.
Dupont P, Vogels R, Vandenberghe R, Rosier A, Cornette L, Bormans G, Mortelmans L, Orban GA. (
1998
) Regions in the human brain activated by simultaneous orientation discrimination: a study with positron emission tomography.
Eur J Neurosci
 
10
:
3689
–3699.
Elliott R, Dolan RJ (
1999
) Differential neural responses during performance of matching and nonmatching to sample tasks at two delay intervals.
J Neurosci
 
19
:
5066
–5073.
Elliott R, Rees G, Dolan RJ (
1999
) Ventromedial prefrontal cortex mediates guessing.
Neuropsychologia
 
37
:
403
–411.
Fox P, Mintun M, Raichle M, Miezin FM, Allman JM, Van Essen DC (
1986
) Mapping human visual cortex with positron emission tomography.
Nature
 
323
:
806
–809.
Friedman HR, Goldman-Rakic PS (
1994
) Coactivation of prefrontal cortex and inferior parietal cortex in working memory tasks revealed by 2DG functional mapping in the rhesus monkey.
J Neurosci
 
14
:
2775
–2788.
Friston KJ, Frith CD, Liddle PF, Frackowiak RSJ (
1991
) Comparing functional (PET) images: the assessment of significant change.
J Cereb Blood Flow Metab
 
11
:
690
–699.
Friston KJ, Worsley KJ, Frackowiak RSJ, Mazziotta JC, Evans AC (
1994
) Assessing the significance of focal activations using their spatial extent.
Hum Brain Map
 
1
:
214
–220.
Friston KJ, Ashburner J, Poline JB, Frith CD, Heather JD, Frackowiak RSJ (
1995
) Spatial realignment and normalization of images.
Hum Brain Map
 
3
:
165
–189.
Friston KJ, Holmes AP, Worsley KJ, Poline JB, Frith CD (
1995
) Statistical parametric maps in human functional imaging: a general linear approach.
Hum Brain Map
 
23
:
189
–210.
Fuster JM (
1990
) Inferotemporal units in selective visual attention and short-term memory.
J Neurophysiol
 
64
:
681
–697.
Fuster JM (1995) Memory in the cerebral cortex: an empirical approach to neural networks in the human and nonhuman primate brain. Cambridge, MA: MIT Press.
Fuster JM, Jervey JP (
1981
) Inferotemporal neurons distinguish and retain behaviorally relevant features of visual stimuli.
Science
 
212
:
952
–955.
Goldman–Rakic PS (1987) Circuitry of primate prefrontal cortex and regulation of behaviour by representational memory. In: Handbook of physiology, Vol. 5 (Plum F, Mountcastle U, eds), pp. 373–417. Washington, DC: The American Physiological Society.
Grady CL, Horwitz B, Pietrini P, Mentis MJ, Ungerleider LG, Rapoport SI, Haxby JV (
1996
) Effect of task difficulty on cerebral blood flow during perceptual matching of faces.
Hum Brain Map
 
4
:
227
–239.
Haber RN (
1970
) How we remember what we see.
Scient Am
 
222
:
104
–112.
Haxby JV, Horwitz B, Ungerleider LG, Maisog JM, Pietrini P, Grady CL (
1994
) The functional organization of human extratriate cortex: a PET-rCBF study of selective attention to faces and locations.
J Neurosci
 
14
:
6336
–6353.
Haxby JV, Ungerleider LG, Horwitz B, Rapoport SI, Grady CL (
1995
) Hemispheric differences in neural systems for face working memory: a PET-rCBF study.
Hum Brain Map
 
3
:
68
–86.
Heeley DW, Buchanan-Smith HM, Cromwell JA, Wright JS (
1997
) The oblique effect in orientation acuity.
Vision Res
 
37
:
235
–242.
Holmes MD, Ojemann GA, Lettich E (
1996
) Neuronal activity in human right lateral temporal cortex related to visuospatial memory and perception.
Brain Res
 
711
:
44
–49.
Hubel DH, Wiesel TN (
1968
) Receptive fields and functional architecture of monkey extrastriate cortex.
J Physiol (Lond)
 
195
:
215
–243.
Jonides J, Schumacher EH, Smith EE, Lauber EJ, Awh E, Minoshima S, Koeppe RA (
1997
) Verbal working memory load affects regional brain activation as measured by PET.
J Cogn Neurosci
 
9
:
462
–475.
Kinahan PE, Rogers JG (
1989
) Analytic three-dimensional image reconstruction using all detected events.
IEEE Trans Nucl Sci
 
36
:
964
–968.
Kirchner WK (
1958
) Age differences in short-term retention of rapidly changing information.
J Exp Psychol
 
55
:
352
–358.
Long GM (
1980
) Iconic memory: a review and critique of the study of short-term visual storage.
Psychol Bull
 
88
:
785
–820.
Long GM, Sakitt B (
1980
) The retinal basis of iconic memory: Eriksen and Collins revisited.
Am J Psychol
 
93
:
195
–206.
Maes F, Collignon A, Vandermeulen D, Marchal G, Suetens P (
1997
) Multi-modality image registration by maximization of mutual information.
IEEE Trans Med Imaging
 
16
:
187
–198.
Mikami A (
1995
) Visual neurons with higher selectivity can retain memory in the monkey temporal cortex.
Neurosci Lett
 
192
:
157
–160.
Miller EK, Li L, Desimone R (
1993
) Activity of neurons in anterior inferior temporal cortex during a short-term memory task.
J Neurosci
 
13
:
1460
–1478.
Miller EK, Li L, Desimone R (
1994
) A neural mechanism for working and recognition memory in inferior temporal cortex.
Science
 
254
:
1377
–1379.
Miller EK, Erickson CA, Desimone R (
1996
) Neural mechanisms of visual working memory in prefrontal cortex of the macaque.
J Neurosci
 
16
:
5154
–5167.
Mugler JP, Brookeman JR (
1990
) Three dimensional Magnetization- Prepared Rapid Echo Gradient Echo Imaging (3D MPRAGE).
Magn Reson Med
 
15
:
152
–157.
Miyashita Y, Chang HS (
1988
) Neuronal correlate of pictorial short-term memory in the primate temporal cortex.
Nature
 
331
:
68
–70.
Orban GA, Vogels R (
1998
) The neuronal machinery involved in successive orientation discrimination.
Prog Neurobiol
 
55
:
117
–147.
Orban GA, Vandenbussche E, Vogels R (
1984
) Human orientation discrimination tested with long stimuli.
Vision Res
 
24
:
121
–128.
Orban G, Dupont P, Vogels R, Bormans G, Mortelmans L (
1997
) Human brain activity related to orientation discrimination tasks.
Eur J Neurosci
 
9
:
246
–259.
Ó Scalaidhe SP, Wilson FAW, Goldman-Rakic PS (
1999
) Face-selective neurons during passive viewing and working memory performance of rhesus monkeys: evidence for intrinsic specialization of neural coding.
Cereb Cortex
 
9
:
459
–475.
Owen AM, Herrod NJ, Menon DK, Clark JC, Downey SP, Carpenter TA, Minhas PS, Turkheimer FE, Williams EJ, Robbins TW, Sahakian BJ, Petrides M, Pickard JD (
1999
) Redefining the functional organization of working memory processes within human lateral prefrontal cortex.
Eur J Neurosci
 
11
:
567
–574.
Petit L, Clark VP, Ingeholm J, Haxby JV (
1997
) Dissociation of saccade- related and pursuit-related activation in human frontal eye fields as revealed by fMRI.
J Neurophysiol
 
77
:
3386
–3390.
Petrides M (
1995
) Impairments on nonspatial self-ordered and externally ordered working memory tasks after lesions of the mid-dorsal part of the lateral frontal cortex in the monkey.
J Neurosci
 
15
:
359
–375.
Petrides M, Pandya D (
1999
) Dorsolateral prefrontal cortex: comparative cytoarchitectonic analysis in the human and the macaque brain and corticocortical connection patterns.
Eur J Neurosci
 
11
:
1011
–1036.
Philips WA (
1974
) On the distinction between sensory storage and short-term visual memory.
Percept Psychophys
 
16
:
283
–290.
Price CJ, Friston K (
1996
) Cognitive conjunction: a new approach to brain activation experiments.
NeuroImage
 
5
:
261
–270.
Regan D, Beverley KI (
1985
) Postadaptation orientation discrimination.
J Opt Soc Am
 
A2
:
147
–153.
Schumacher EH, Lauber E, Awh E, Jonides J, Smith EE, Koeppe RA (
1996
) PET evidence for an amodal verbal working memory system.
NeuroImage
 
3
:
79
–88.
Smith EE, Jonides J (
1999
) Storage and executive processes in the frontal lobes.
Science
 
283
:
1657
–1661.
Sunaert S, Van Hecke P, Marchal G, Orban GA (
1999
) Human brain activity during speed identification in well trained subjects: a parametric fMRI study of task difficulty.
NeuroImage
 
9
:
S890
.
Talairach J, Tournoux P (1988) Co-planar stereotaxic atlas of the human brain. New York: Thieme.
Tanaka K, Saito H, Fukada Y, Moriya M (
1991
) Coding visual images of objects in the inferotemporal cortex of the macaque monkey.
J Neurophysiol
 
66
:
170
–189.
Vogels R, Orban G (
1994
) Activity of inferior temporal neurons during orientation discrimination with successively presented gratings.
J Neurophysiol
 
71
:
1428
–1451.
Vogels R, Saunders RC, Orban GA (
1997
) Effects of inferior temporal lesions on two types of orientation discrimination in the macaque monkey.
Eur J Neurosci
 
9
:
229
–245.
Webster MJ, Bachevalier J, Ungerleider LG (
1994
) Connections of inferior temporal areas TEO and TE with parietal and frontal cortex in macaque monkeys.
Cereb Cortex
 
4
:
470
–483.
Wilson FA, Ó Scalaidhe SP, Goldman-Rakic PS (
1993
) Dissociation of object and spatial processing domains in primate prefrontal cortex.
Science
 
260
:
1955
–1958.
Worsley KJ, Evans AC, Marrett S, Neelin P (
1992
) A three-dimensional statistical analysis for rCBF activation studies in human brain.
J Cereb Blood Flow Metab
 
12
:
900
–918.