Abstract

The spatio-temporal distribution of brain activity as revealed by non-invasive functional imaging helps to elucidate the neuronal encoding and processing strategies required by complex cognitive tasks. We investigated visual short-term memory for objects, places and conjunctions in humans using event-related time-resolved functional magnetic resonance imaging that permitted segregation of encoding, retention and retrieval phases. All conditions were accompanied by the activation of a widespread network of parietal and prefrontal areas during the retention phase, but this retention-related activity showed additional modulations depending on task instructions. These modulations confirmed a posterior — anterior and right — left dissociation for spatial versus non-spatial memory and revealed that conjunction memory does not rely on a linear addition of the component processes.

Introduction

The neuronal mechanisms subserving the integration of multiple aspects of stimuli in visual STM (see Abbreviations for all definitions) (Fuster and Alexander, 1971; Mishkin and Delacour, 1975; Cohen et al., 1997; Courtney et al., 1997; Fuster, 1998; Prabhakaran et al., 2000), have been investigated in both humans and non-human primates. These studies all suggest a pivotal role of the prefrontal cortex (Miller, 2000), the activation of which has also been shown to reflect individual performance levels and objective memory load in working memory tasks (Callicott et al., 1999; Rypma and D’Esposito, 1999; Prabhakaran et al., 2000). In laboratory settings, STM is frequently studied using DMS tasks (Elliott and Dolan, 1999). Single-unit recordings in behaving monkeys have revealed neurons around the principal sulcus of the lateral prefrontal cortex that increase their firing during the delay between the presentation of sample and test stimuli (Fuster and Alexander, 1971; Funahashi et al., 1989; Miller et al., 1996). fMRI studies in humans have revealed neuronal activation in prefrontal areas during STM for faces (Courtney et al., 1997) and locations (Courtney et al., 1998a). The results of these studies suggested a domain specific dissociation of areas involved in STM: retention of objects engaging more ventrally and retention of spatial relations engaging more dorsally located regions (Courtney et al., 1998b). An alternative interpretation is that the differential engagement of ventral and dorsal subdivisions of lateral prefrontal cortex reflects different processing modes, such as maintenance on the one hand and manipulation of retained information on the other, rather than the nature of the remembered cues (D’Esposito et al., 1998, 1999; Owen et al., 1998, 1999; Nystrom et al., 2000; Postle et al., 2000).

In addition to prefrontal cortex, IT and PP cortex have also been assigned functions in STM. In IT, neurons exhibit delay activity when monkeys perform DMS tasks preferentially for the retention of object-specific features (Miller et al., 1993), while PP neurons seem to be activated more during the retention of spatial relations (Constantinidis and Steinmetz, 1996). Relatively little is known about how these areas cooperate with the prefrontal cortex in STM. In order to address this issue, one requires information about the spatial and temporal distribution of activity associated with encoding, retention and retrieval of information in both domains.

Despite the rather limited temporal resolution of fMRI, evaluation of single trial responses (event-related fMRI) can provide some information about the temporal sequence of processing (Zarahn et al., 1999) and about the coherence of processes occurring simultaneously in different areas (Goebel et al., 1998a). We therefore applied event-related fMRI to investigate visual STM in a design that allowed us to separate in time the encoding, retention, retrieval and response phases. We used a DDT rather than a conventional DMS task, because the latter is not balanced with respect to attention and response preparation for matching and non-matching trials. In experiment 1, subjects performed DDT tasks on series of different objects (Postle and D’Esposito, 1999) or identical objects in different places (‘where’, see Fig. 1A). Functional images were acquired at high rate (TR = 1 s) in order to allow for a separation of activity that is evoked by the presentation of the stimuli from the sustained activity that is related to retention. In experiment 2, visual sample stimuli consisted of four natural objects that were sequentially presented in an imaginary two-dimensional grid (Fig. 2A). After the delay period, subjects had to decide whether one object presented as test stimulus at one of the positions of the imaginary grid matched one of the objects (Postle and D’Esposito, 1999), locations (Postle and D’Esposito, 1999), or both (‘what & where’) of the preceding sample stimulus. This design permitted the comparison of cortical activation patterns associated with retention of conjunctions and single features (Rypma and D’Esposito, 1999), respectively. As most human subjects attempt to use verbal descriptions in order to retain information about natural objects, we added a control experiment using abstract stimuli.

Materials and Methods

Subjects, Stimulation and Behavioral Task

We recruited five right-handed healthy volunteers (four male, one female; mean age 30.8 years, range 27–36 years) for experiment 1, 10 (eight male, two female; mean age 29.2 years, range 24–39 years) for experiment 2 and eight (six male, two female; mean age 27.2 years, range 21–35 years) for the non-verbal control experiment, who gave their informed consent to participate in the study. The reported experiments were undertaken with the understanding and written consent of each subject and in accordance with the Declaration of Helsinki. Three volunteers participated in all experiments. Experiment 1 was preceded by a training session which allowed subjects to undertake as many trials as necessary to familiarize themselves with the structure and timing of the task. Visual stimuli (for details about stimulus content and sequence see legends to Figs 1 and 2) were delivered under PC control to an LCD projector (EIKI LC-6000). The image was back-projected onto a frosted screen positioned at the foot end of the scanner.

In experiment 1, four randomly drawn (of 24 possible) sample stimuli (fruit drawings) were presented in rapid sequence (250 ms per item) during the first second of each trial. During ‘what’ trials, a sequence of sample stimuli appeared at the center of the screen. During ‘where’ trials, a single object appeared at four out of eight possible parafoveal positions. Thus, during the sample presentation period, the ‘what’ and ‘where’ tasks differed only by the number of different objects and the respective positions of the objects on the display. After a delay of 11 s, a test stimulus was presented for 4 s at the center of the screen in the ‘what’ trials, or at one of the eight possible positions in the ‘where’ trials. Subjects had to respond by left or right button presses (L/R in the lower traces of Fig. 1) if they detected a match or non-match, respectively. Following the test and response period, the fixation cross turned green or red for correct and incorrect responses, respectively. The design of the control experiment matched that of the ‘what’ condition of experiment 1, except that non-natural objects (BORTS: blurred outlines of random tetris shapes) were used as visual stimuli.

In experiment 2, three different instructions (‘what’, ‘where’ and ‘what and where’) were presented at the beginning of each trial in a pseudo-randomized sequence. The structure of all trials was identical. Four out of 24 different fruit drawings were presented in 4 s (1 s per item) in 1 out of 8, 12 or 16 spatial positions, depending on subjective performance. After a delay of 12 s, a test stimulus was presented for 4 s, after which the subjects had to respond as above. Prior to experiment 2, each subject had to perform a training session in front of a PC screen with at least 240 trials, for which reaction times were recorded. The training sessions were balanced for match and non-match trials for each task type. Depending on the task instruction, which was randomized for each trial, the task-relevant and the task-irrelevant (spatial versus object) information was also balanced. The scanner sessions were then designed for each individual subject by selecting 3 × 12 trials from the training session that had yielded reaction times within one standard deviation of the individual mean, again balanced for task-relevant and task-irrelevant information.

fMRI Measurements and Analysis

fMRI data were acquired with a 1.5 T Magnetom Vision MRI scanner (Siemens, Erlangen, Germany) using a gradient echo EPI sequence [1 volume = 6 (experiment 1)/16 (experiment 2, control) axial slices; TR = 1000 ms (experiment 1)/2000 ms (experiment 2, control); TE = 60 (experiment 2, control)/69 ms (experiment 1); FA = 90°; FOV = 210 × 210 mm2; voxel size = 1.6 × 1.6 × 5.0 (experiment 1) or 3.2 × 3.2 × 5 (experiment 2, control) mm3] for fMRI. Each scan comprised the acquisition of 128 (experiment 1) or 256 (experiment 2, control) volumes. In experiment 1, the slices covered large parts of the occipital, temporal and frontal lobes (z-coordinate range from –5 to 25 at y = –50 and from 15 to 45 at y = 20, Talairach coordinates), whereas in experiment 2 and the control they covered the whole cerebrum. A T1-weighted 3-D MP RAGE scan was recorded in each session (magnetization-prepared rapid acquisition gradient echo, TR = 9.7 ms, TE = 4 ms, FA = 12°, matrix = 256 × 256, voxel size 1.0 × 1.0 × 1.0 mm3).

In experiment 1, subjects underwent four scans of each condition, yielding an overall of 16 ‘what’ and 16 ‘where’ trials. Experiment 2 consisted of three functional scans with a pseudo-random sequence of task types, yielding 12 trials of every task type (‘what’, ‘where’, ‘what and where’).

The statistical analysis was based on the application of the multiple regression analysis to time-series of task-related functional activation (Friston et al., 1995). These analytical tools were implemented in BrainVoyager 4.4 (Goebel et al., 1998a,b; Dierks et al., 1999).

Talairach transformation (Trojano et al., 2000) was performed for the complete set of functional data of each subject, yielding a 4-D data representation (volume time-course: 3 × space, 1 × time). Prior to statistical analysis, the time-series of functional images was aligned in order to minimize the effects of head movements. The central volume of the time-series was used as a reference volume to which all other volumes were registered, using a 3-D motion correction that estimates the three translation and three rotation parameters of rigid body transformation. Data pre-processing furthermore comprised spatial smoothing with a Gaussian kernel (FWHM = 8 mm), the removal of linear trends and (in experiment 2 and the control) temporal lowpass filtering (lowpass: 48 per functional run of 256 volumes).

The high resolution T1-weighted anatomical 3-D data set of a template brain (courtesy of the Montreal Neurological Institute) was used for the surface reconstruction and flatmap representation of both hemispheres.

The GLMs of the ‘what’ and ‘where’ sessions of experiment 1 were computed from the 20 (five subjects, four scans per subject) z-normalized volume time-courses. The signal values during the encoding, delay and retrieval phases were considered effects of interest. The GLMs of experiment 2 were computed from 30 volume time-courses (10 subjects, three scans per subject). GLMs were computed for the encoding (two volumes), early delay (two volumes), delay (six volumes) and retrieval (two volumes) phases, and the task type (‘what’, ‘where’, ‘what and where’) was considered the effect of interest. The corresponding predictors, obtained by convolution of an ideal box-car response (assuming a value of 1 for the volumes of task presentation and a value of 0 for the remaining time points) with a linear model of the hemodynamic response (Boynton et al., 1996), were used to build the design matrix of the experiment. The global level of the signal time-courses in each session was considered to be a confounding effect and a fixed effects analysis was employed. To analyze the effects of conditions compared to baseline and contrasts between conditions, 3-D individual and group statistical maps were generated by associating each voxel with the F-value corresponding to the specified set of predictors and calculated on the basis of the least mean squares solution of the GLM. Statistical results were then visualized through projecting 3-D statistical maps on the flattened surface reconstruction of the MNI template. Effects were only shown if, considering an F distribution with n1 and n2 degrees of freedom (n1 = number of orthogonal predictors and n2 = number of time samples – n1 –1), the associated P-value yielded P′ < 10−2, corrected for multiple comparisons (RC maps), or P < 10−3, uncorrected (superposition maps) and if a minimum cluster size of 100 mm3 was reached.

RC Maps

For significantly activated voxels, the relative contributions, RC, between two selected sets of conditions in explaining the variance of a voxel time-course were computed as 

\[\mathit{RC}\ {=}\ (\mathit{b}_{1}\ {\mbox{--}}\ \mathit{b}_{2})/(\mathit{b}_{1}\ {+}\ \mathit{b}_{2})\]
where bi is the sum of the estimates of the standardized regression coefficients of all conditions included in set i (Trojano et al., 2000). The RC index was visualized with the pseudo-color scales shown in the respective figures. In experiment 1, the first four time points (convolved with the hemodynamic function) were taken to represent the encoding predictor. This was contrasted with the late delay predictor (time points 9–12 convolved with the hemodynamic function) in order to minimize the influence of encoding-related signal modulation on the delaypredictor (Fig. 1). In experiment 2, the encoding, early delay, delay and retrieval predictors were defined as outlined above and separated by condition, yielding 12 different predictors. The RCs of the encoding and delay predictors of each condition were presented in six separate maps (Fig. 3). All predictors were used for the statistical analysis of significant differences between conditions in the four phases, based on the t-test of differences between the individual (for each subject) beta weights of the three respective predictors (‘what’, ‘where’, ‘what and where’) after removal of serial correlation (Bullmore et al., 1996) (Table 2).

Superposition Maps

In experiment 2, each of the effects of interest (the three task types) was given a color of the RGB system. In order to visualize all three effects on a single flatmap, colors were superimposed and areas of overlap (cortical regions showing an activation during more than one condition) received the appropriate mixed color (superposition maps). Time-courses of experiment 1 and experiment 2 were computed by event-related averaging of the mean time-courses of indicated clusters over all 20 volume time-courses, using the same voxels (in Talairach space) for all subjects and all repetitions.

Results

Behavioral Data

Subjects performed at high accuracy (>85%) in all experiments. In experiment 1, reaction times of correct responses did not differ significantly between the ‘what’ and ‘where’ conditions (P = 0.68, Mann–Whitney U-test). In experiment 2, where the response had to be delayed until the disappearance of the stimulus, accuracy rather than reaction time was used as measure of task performance. Accuracy rates (‘what and where’, 86%; ‘where’, 87%; ‘what’, 89%) did not differ significantly between conditions (χ2, P = 0.78).

Experiment 1

In experiment 1, data were acquired from the occipital, temporal and frontal lobes only in order to achieve high temporal resolution. The results confirm that stimulus and retention-related activity can be separated by event-related fMRI. In IT cortex of both hemispheres, the presentation of the target and the test stimuli evoked temporally well-segregated activities that peaked ~4 s after the onset of the respective stimuli (Fig. 1C,D). In prefrontal cortex, by contrast, the same stimulus constellation was followed by a sustained activation that rose more slowly, remained high during the delay period, peaked ~5 s after the presentation of the test stimulus and then returned to baseline (Fig. 1C,D). We will call activity occurring immediately after the presentation of the target and test stimuli ‘encoding’ and ‘retrieval’ activation, respectively, and activity present during the delay period ‘retention’ activation. RC maps (see Materials and Methods) between encoding and retention-related brain activation in experiment 1 yielded prominent bilateral clusters in the temporal lobes during encoding and in the frontal lobes during retention (maps in Fig. 1C,D). The parietal lobe was not included in the sampling volume for experiment 1. During encoding, temporal lobe activation occurred in similar regions in the ‘what’ and ‘where’ conditions. In the ‘what’ condition, the size of activated clusters was larger and their position, as estimated by their center of mass, was more posterior than in the ‘where’ condition (Fig. 1 and Table 1). During retention, frontal activation occurred more anteriorly in the ‘what’ condition, particularly in the right hemisphere, than in the ‘where’ condition and showed a clear asymmetry in favor of the left hemisphere, while it was fairly symmetrical during the ‘where’ condition (Table 1). In experiment 1, subjects were instructed to respond immediately after the presentation of the probe, whereas in experiment 2 they had to hold off their button press response for 4 s. This element of the task design allowed for a separation of activation related to the execution of the button press response from retrieval-related activation.

Experiment 2

Superposition Maps

In experiment 2, slices covered the entire cortex. Encoding activity covered bilateral occipito-temporal and parietal cortex and was also prominent in the DLPFC, particularly for the ‘where’ and ‘what and where’ conditions, in the left VLPFC and bilateral INS, particularly for the ‘what’ and ‘what and where’ conditions, and in frontal midline structures (SMA/anterior cingulate; Fig. 2). Retention activity was observed mainly in the frontal and parietal lobes (Fig. 2D). While the overlap between conditions in the superposition maps was large, it was not as widespread as in the encoding maps. In particular, the parietal lobes showed very little contribution of the ‘what’ predictor, and activation in the right IPL showed a clear preponderance for the ‘where’ predictor. In the frontal lobes, large areas in the anterior middle and inferior frontal gyri and INS (particularly in the left hemisphere) showed a predominance of ‘what’ and ‘what and where’ over ‘where’, while the posterior middle and superior frontal gyri of both hemispheres showed little contribution of the ‘what’ as compared to the ‘where’ and ‘what and where’ predictors.

In the non-verbal control experiment (which only consisted of ‘what’ trials) we could replicate the finding of experiment 1 and experiment 2 of predominantly left hemispheric retention activity in prefrontal cortex (Table 1; Figs 1C and 2), but not the activation of mesial superior frontal cortex shown in Figure 2.

RC Maps

The RC maps (Fig. 3), which had a more conservative threshold than the superposition maps, again showed much larger overlap during encoding than during retention. For encoding, the contrast ‘what’ versus ‘where’ revealed a higher contribution of the ‘what’ predictor in some occipito-temporal and inferior frontal areas bilaterally and of the ‘where’ predictor in the right IPL and DLPFC bilaterally. The contrast ‘what and where’ versus ‘where’ again revealed a higher contribution of the ‘what and where’ predictor in occipito-temporal areas bilaterally and the left INS and of the ‘where’ predictor in the right IPL, while the contribution of both conditions to the signal in DLPFC was approximately equal. The contrast ‘what and where’ versus ‘what’ mainly yielded a higher contribution of the ‘what and where’ predictor to right DLPFC activation.

The RC maps for the delay period tended to show a higher degree of separation of the predictors. The ‘what’ versus ‘where’ contrast yielded distinct clusters of ‘where’-related activation in the parietal lobes and DLPFC bilaterally and of ‘what’-related activation in left VLPFC and INS. The ‘what and where’ versus ‘where’ map showed preponderance of ‘where’ activation in the right SPL and IPL and of ‘what and where’ activation in the left VLPFC and bilateral INS. The ‘what and where’ versus ‘what’ map showed that bilateral superior parietal and frontal midline activity had a higher contribution from ‘what and where’ trials.

Time-courses

The detailed documentation of the BOLD signal change time-courses of the areas whose activation was found to be accounted for differently by the task condition predictors during encoding and/or retention contributed important additional information. The color coded statistical maps revealed the areas whose activation is explained by predictors at a determined threshold. The time-course plots, however, reveal the temporal dynamics of activation changes in the task phases of a particular condition in a particular area in relation to the other conditions and phases (Fig. 4). They can thus help to determine if differential effects observed during retention are mere carry-over effects of encoding-related activation and if significant differences between conditions during a phase of the task (e.g. encoding) are specific to that phase (Table 2). The time-courses revealed that most retention areas had also shown a response in the encoding phase. Yet, while occipito-temporal areas showed an early response (peaking 6 s after onset of stimulus presentation) and returned to baseline after another 8 s (and then showed a second response to the probe stimulus), activity in the frontal and parietal areas peaked later (8–10 s), remained above baseline during the delay period and showed a second peak in response to the probe stimulus. In order to account for this first transient response evoked by the sample stimulus (that peaked 3–5 s later than would have been expected from a pure hemodynamic shift of stimulus onset), we introduced the ‘early delay’ predictor to the statistical analysis of differences between conditions (Table 2). While significant differences in the ‘early delay’ predictor might be carry-over effects from encoding, this is rather unlikely for differences that are present in the time-courses (and significant) during the entire delay. For a full account of the statistical analysis of differences between conditions see Table 2.

Discussion

Separation of Task Phases

The time-courses and maps presented in Figures 1 and 2 confirm that the design of the study permitted a separation of encoding-, retention- and retrieval-related brain activity. Experiment 1 revealed distinct time-courses in IT and prefrontal cortex during visual STM. While activity in IT peaked at ~5 s — the commonly assumed time-to-peak of the BOLD signal (Boynton et al., 1996) — after the onset of a visual stimulus and returned to baseline immediately afterwards, activity in prefrontal cortex rose more slowly after the presentation of the target stimulus, remained high during retention and peaked after the test stimulus. Experiment 2, which used a lower sampling rate than experiment 1 (2 instead of 1 s) but covered the entire brain, revealed a similar pattern of time-courses (Fig. 4) and additional retention-related activation in the parietal lobe, predominantly for the ‘where’ and conjunction conditions. Retention-related activity can thus be regarded as being largely confined to the frontal and parietal lobes (Fig. 2D), while the first response to a new target stimulus, which can be seen as the neural correlate of stimulus encoding, was observed in IT (Figs 1C,D and 2C). Retrieval-related activity was also present in the temporal, frontal and parietal lobes (Fig. 4).

Comparison of Spatial, Non-spatial and Conjunction Memory

Frontal activation

Beyond the segregation of the phases of a typical STM task, our design permitted the comparison of cortical activation patterns associated with spatial, non-spatial and conjunction STM. Particularly during delay (and most clearly in the left hemisphere), the RC maps (Fig. 3) show a separation of more ventral prefrontal areas (anterior IFG and MFG) involved in the ‘what’ and more dorsal prefrontal areas (posterior MFG and SFG) involved in the ‘where’ condition, while ‘what and where’ recruits parts of both regions. From this perspective, our results might seem to confirm a clear-cut segregation of dorsal ‘where’ and ventral ‘what’ areas in prefrontal cortex. Yet the superposition maps (Fig. 2) and the time-courses of the prefrontal areas (Fig. 4) even more so, reveal that the issue is more complex than this. Even areas with high RC values in favor of one condition still show a considerable and very stable departure from baseline during the other conditions. For example, the left INS, albeit displaying a significantly higher activation for ‘what’ and ‘what and where’ compared to ‘where’ during delay, still shows a clear difference from baseline for the ‘where’ condition. Conversely, the ‘what’ condition was accompanied by a consistent activation of right SFG in all phases, although this area clearly showed a more prominent modulation for the spatial conditions. This shows that if only activations that survive a very stringent threshold are considered, some aspects of the distributed cortical activity subserving complex cognitive processes might be lost, as has also been observed for categorical visual processing (Ishai et al., 1999). It thus seems that a wide range of prefrontal areas is recruited during visual STM, regardless of the characteristics to be remembered and that the additional processing required by the precise nature of the task leads to the differential modulation of subsets of this network. Moreover, the role of the prefrontal cortex in STM is clearly not only confined to functions during the delay period. Most areas of PFC that showed task-related activity in the present study responded even more strongly in the encoding phase (although they differed from pure encoding areas in that their activity remained significantly higher than baseline during the entire delay period) and showed a second peak for retrieval (Fig. 4). Our data show that PFC is active during all phases of visual STM. Furthermore they confirm a non-exclusive DLPFC/ VLPFC dissociation for spatial and non-spatial memory. This is consistent with the claim that while there is considerable overlap of delay activity in lateral prefrontal areas, the level of participation is generally higher for the SFS region bilaterally in spatial and for left inferior and mid-frontal cortex in non-spatial tasks (Courtney et al., 1998a; Haxby et al., 2000).

Most identified frontal areas were also active during the different phases of the conjunction task (‘what and where’). Yet conjunction-related activation was clearly not an addition of the activations related to the component processes. Some ventro-lateral prefrontal areas showed higher activity for ‘what’ than conjunction and some dorsolateral areas for ‘where’ than conjunction. However, in areas where one of the component tasks evoked the highest activation, conjunction always took the second place. This would be compatible with a theory that regards not the addition, but the recruitment of parts of the networks for the components as the likely neuronal mechanism for the solution of conjunction tasks. The only area that consistently displayed the highest BOLD signal change for conjunction versus the component tasks was found in the mesial superior frontal cortex bilaterally (extending from the SMA to the anterior cingulate). This region has been identified as being a central element of the network for feature integration in working memory in a number of previous studies (Mitchell et al., 2000; Prabhakaran et al., 2000).

The superposition map of experiment 2 (Fig. 2) shows that the dissociation of lateral PFC into more dorsal areas that participated more in the spatial conditions and more ventral areas that participated more in the non-spatial conditions tended to be present in both hemispheres. Yet significantly higher activation for ‘what’ versus ‘where’ during delay was only found in the left inferior and mid-frontal cortex, whereas the SFG and parietal activation was significantly higher for ‘where’ than ‘what’ in both hemispheres (Fig. 3A and Table 2). Predominantly left hemispheric ‘what’ activation during maintenance has recently been described by Postle and D’Esposito (Postle and D’Esposito, 2000) who proposed that the difference between maintaining spatial and non-spatial information might be hemispheric. However, of the previous studies that included a direct comparison between spatial and non-spatial working memory, only some have reported a left lateralized prefrontal activation for the non-spatial task (Courtney et al., 1998a), while a number of studies have found bilateral activation in mid-frontal cortex (McCarthy et al., 1996; Belger et al., 1998). Prefrontal activation for the spatial task was either bilateral (Courtney et al., 1998a) or predominantly on the right (McCarthy et al., 1996; Belger et al., 1998). In terms of lateralization, the most consistent finding of both the previous and the present studies seems to be the predominantly left-hemispheric IFG activation for the non-spatial task. The fact that the activation of left IFG could be confirmed in our control experiment suggests that it is not exclusively associated with the verbal components of working memory.

Parietal Activation

The parietal retention activation seems to be linked to the spatial component of STM, because it was mainly observed in the ‘where’ and conjunction conditions of experiment 2 and much less prominent in the ‘what’ task that was based on the same stimulus material. Thus, our results confirm the view that spatial STM involves coactivation of PP and prefrontal cortical areas (Chafee and Goldman-Rakic, 1998). Primate PP is known to play a key role in visuomotor integration (Sakata et al., 1997; Goodale, 1998; Quintana and Fuster, 1999), the spatial analysis of the visual scene (Colby and Goldberg, 1999) and the integration of spatial information from different sensory modalities (Andersen, 1997). Posterior parietal areas LIP, 7a and 7ip of non-human primates have been shown to be active during delayed saccade tasks (Andersen et al., 1990; Chafee and Goldman-Rakic, 1998) and DMS paradigms (Constantinidis and Steinmetz, 1996). A preponderance of parietal over DLPFC activity during visuospatial STM in humans has recently been described by Pochon et al. (Pochon et al., 2001) who found a prominent DLPFC activation only when the preparation of a sequential movement was required. While our data suggest that the STM-related DLPFC activation also occurs in the delay phase of simple response tasks, we can confirm their finding of the important role of the parietal-premotor network in visuospatial STM. The observation of a hemispheric difference of parietal activation is consistent with most of the imaging and neuro-psychological literature on the spatial functions of the parietal lobe. However, the finding that the right SPL showed a higher response for the ‘where’ than the conjunction condition might seem surprising, because the spatial attention load and need to rehearse the positions mentally would have been the same in both conditions. Yet, in the present experiment, the ‘where’ condition actually involved a higher demand on visuospatial attention because the number of possible locations was higher (based on individual performance in the test trials) in order to match the two conditions for difficulty. Furthermore, there is evidence that the presence of a second feature on which the match–non-match judgement can be based (in this case the identity of the object) leads to a reduced recruitment of the parietal lobes in conjunction as opposed to pure visuospatial tasks (Sack et al., 2002).

Infero-temporal Activation

The typical time-course of the BOLD signal in IT cortex showed a prompt response to sample stimuli, returned to baseline during the delay and peaked again in response to the probe stimulus. Thus we could observe the expected stimulus responses, but not the delay activity described for IT in a number of studies (Fuster and Jervey, 1981; Miller et al., 1993, 1996). A possible explanation might be provided by the particular nature of our DDT task. Our sample stimulus always consisted of four sequentially presented items. Based on the finding that intervening stimuli cancel out delay activity in IT but not in prefrontal neurons of macaque monkeys (Miller et al., 1996), we would expect delay activity only in IT neurons responding to the fourth item of the samples. Considerably fewer neurons in IT cortex than PFC would thus be active during the delay phase of our task and the population of active IT neurons might have been too small to evoke a BOLD response.

Conclusion

Activation patterns in the ‘what’, ‘where’, and ‘conjunction’ conditions showed consistent and significant differences during the encoding and delay intervals of the delayed discrimination task (Figs 2 and 3). This suggests that the retention of spatial and non-spatial cues from identical visual stimuli and the retention of conjunctions between these cues engages not only different encoding strategies during stimulus presentation, but also different processes during the delay interval. While encoding activity was observed in the occipito-temporal, parietal and prefrontal cortex, activity in most occipito-temporal areas returned to baseline after a transient stimulus-related response. Prefrontal and parietal areas, however, showed a sustained activation during the entire delay period. While all conditions evoked a significant change of activation from baseline in these prefrontal and parietal areas, an additional task-dependent modulation was observed in most of these areas, leading to an anterior–posterior (parietal and posterior dorsal prefrontal, more ‘where’; ventral prefrontal, more ‘what’) dissociation between ‘spatial’ and ‘non-spatial’ conditions. For the conjunction task, the amplitude of BOLD signal change was not an addition of the activation related to the component processes. In most areas it was even surpassed by one of the component tasks. This suggests that the retention of conjunctions of features is not based on a linear addition of the feature memories, but recruits the cortical areas subserving the component processes in a manner that reflects their contribution to the solution of the conjunction task. In the group analysis, this network involved the SPL and mesial superior frontal cortex bilaterally and dorso-lateral and ventro-lateral prefrontal areas, particularly in the left hemisphere. There might, however, be considerable inter-individual variability brought about by the different strategies employed to solve the conjunction task. The group analysis chosen for the present study (and the inherent necessity of some spatial smoothing) might also have obscured some of the more fine-grained segregation of spatial and non-spatial functions in prefrontal cortex that can be found in individual activation maps but, because of the small extent and spatial variability, does not survive in group maps. Another caveat regards the fixed effects analysis used in the general linear model of the present experiments. It leaves the possibility that some small but significant differences in cortical activation that are present in the population could not be detected in our sample.

In conclusion, our data suggest that retention of different aspects of visual stimuli (‘what’, ‘where’ and conjunctions) depends on processes that recruit, in a task-specific manner, partly overlapping combinations of prefrontal and parietal areas.

Abbreviations

BOLD blood oxygen level-dependent

CaS calcarine sulcus

CiS cingulate sulcus

CoS collateral sulcus

CU cuneus

DDT delayed discrimination task

DLPFC dorsolateral prefrontal cortex

DMS delayed matching-to-sample (task)

FA flip angle

fMRI functional magnetic resonance imaging

FOV field of view

GF gyrus fusiformis

GL gyrus lingualis

GLM general linear model

GPrC precentral gyrus

IFG/IFS inferior frontal gyrus/sulcus

INS insula

IPL inferior parietal lobule

IPS intraparietal sulcus

IS insular sulcus (sulcus circularis insulae)

IT inferior temporal cortex

LS lateral sulcus

MFG/MFS middle frontal gyrus/sulcus

MNI Montreal Neurological Institute

MTS middle temporal sulcus

OF orbito-frontal sulci

OTS occipito-temporal sulcus

PCS postcentral sulcus

PFC prefrontal cortex

POS parieto-occipital sulcus

PP posterior parietal cortex

RC relative contribution

RGB red–green–blue

RS Rolandic (central) sulcus

SFG superior frontal gyrus

SFS superior frontal sulcus

SMA supplementary motor area

SPL superior parietal lobule

STM short-term memory

STS superior temporal sulcus

TE echo time

TR repetition time

VLPFC ventrolateral prefrontal cortex

Table 1

Talairach coordinates for centers of mass of activation clusters shown in Figure 1

 ‘What’ ‘Where’ 
 x y z Voxels x y z Voxels 
Temporal lobe, left hemisphere –37 –75 –9 2621 –50 –62 –5 1194 
Temporal lobe, right hemisphere 31 –67 –9 4088 43 –55 –2 1587 
Frontal lobe, left hemisphere –44 29 7860 –46 29 2270 
Frontal lobe, right hemisphere 35 38 30 380 45 31 1857 
 ‘What’ ‘Where’ 
 x y z Voxels x y z Voxels 
Temporal lobe, left hemisphere –37 –75 –9 2621 –50 –62 –5 1194 
Temporal lobe, right hemisphere 31 –67 –9 4088 43 –55 –2 1587 
Frontal lobe, left hemisphere –44 29 7860 –46 29 2270 
Frontal lobe, right hemisphere 35 38 30 380 45 31 1857 
Table 2

Centers of mass and significance levels of contrasts between conditions (for each task phase) for regions of interest in relative contribution maps

Area Hem. RC > 0.2 x y z No. of voxels Encoding Delay Delay 1 Retrieval 
See list of abbreviations for areas. Hem., hemisphere (LH, left; RH, right); w&w, ‘what and where’; x, y, z, Talairach coordinates. Conditions listed in column ‘RC > 0.2’ denote the condition with the stronger BOLD response in the respective map (see headlines in table). Cluster size (No. of voxels) is provided in mm3. Most footnotes refer to additional contrasts, all remaining contrasts did not reach significance. 
aFor contrast w&w versus where significant at P < 10−6. bHere, RC for where > what. cFor w&w versus where significant at P < 10−3. dFor w&w versus what significant at P < 10−4. eFor w&w versus what significant at P < 0.05. fFor w&w versus what significant at P < 10−2. gFor where versus w&w significant at P < 0.05. hFor where versus w&w significant at P < 10−5. iFor where versus w&w significant at P < 0.05. j For what versus where significant at P < 10−4. kFor what versus where significant at P < 2 × 10−4. lFor what versus where significant at P < 0.01. mFor where versus what significant at P < 10−6. nFor where versus what (where > what) significant at P < 2 × 10−5. oFor w&w versus what significant at P < 10−3. pFor what versus where significant at P < 0.02. qFor where versus what (where > what) significant at P < 10−6. rFor where versus what significant at P < 2 × 10−4. sFor w&w versus what P < 0.06. tHere, RC for where > w&w. 
Encoding, ‘where’ versus ‘what’ 
MFG/SFG LH where –26  –7 46 144  0.6  2 × 10−4 10−5  0.4 
GL LH what –14 –62 –3 404 10−6a  0.3  0.4 0.01b 
CU RH what –85 21 2128 10−6c  0.06  0.02  0.04b 
SPL RH where 32 –49 43 1999  0.6 10−6d  10−4 >0.9 
SFG RH where 20 –7 62 974  0.3  0.05 e  0.05f  0.5 
GPrC RH where 46 23 1197  0.05g 10−4  10−6h  0.05i 
Encoding, ‘w&w’ versus ‘where’ 
GF LH w&w –31 –39 –14 1204 10−6j  0.004 >0.9  0.7 
POS LH w&w –5 –78 20 850  0.01k  0.3  0.5  0.004 
INS LH w&w –32 10 10 354  0.2  0.05l  0.2  0.07 
IPL RH where 35 –34 43 994  0.6  0.01m 10−4m  0.9 
GL RH w&w 15 –39 986 10−4n  0.2  0.8  0.07n 
Delay, ‘where’ versus ‘what’ 
MFG/IFG LH what –43 24 25 6434  0.006  0.003  0.3  0.002 
IFG/INS LH what –38 14 2742  0.9  0.001  0.5  0.03 
SPL LH where –26 –60 47 10907  0.2  2 × 10−6  0.0002  0.2 
MFG/SFG LH where –27 –7 47 391  0.5 10−3 10−6  0.3 
Delay, ‘w&w’ versus ‘what’ 
SFG RH w&w 27 –9 59 1768  0.4  0.01  0.01  0.6 
IFG LH what –49 11 12 299  0.4  0.7  0.5  0.1 
SPL LH w&w –22 –66 46 8977  0.8  3 × 10−4  0.01  0.6 
SPL RH w&w 12 –72 46 2897  0.8 10−6  0.05  0.6 
SMA LH w&w –4 43 383  0.2  0.009  0.3  0.5 
SMA RH w&w 49 774  0.6  0.02  0.6  0.8 
Delay, ‘w&w’ versus ‘where’ 
IPL LH where –47 –41 36 569  0.4  0.03o  0.0007 >0.9 
IFG LH w&w –44 20 23 4723 10−3  0.003  0.6  0.005 
INS LH w&w –36 13 2255  0.2  0.005  0.2  0.004 
MFG LH w&w –43 36 2584  0.002  0.05p  0.4 >0.9 
SMA LH w&w –4 11 45 835  0.4  0.02  0.7  0.01 
SPL RH where 25 –57 49 7022  0.3 10−4q 10−3q  0.2 
SFG RH where 25 –8 47 462  0.2  0.01r  0.01q >0.9 
SMA RH w&w 51 263  0.8  0.01s >0.9  0.01t 
INS RH w&w 32 14 1585  0.6  0.05  0.2 >0.9 
Area Hem. RC > 0.2 x y z No. of voxels Encoding Delay Delay 1 Retrieval 
See list of abbreviations for areas. Hem., hemisphere (LH, left; RH, right); w&w, ‘what and where’; x, y, z, Talairach coordinates. Conditions listed in column ‘RC > 0.2’ denote the condition with the stronger BOLD response in the respective map (see headlines in table). Cluster size (No. of voxels) is provided in mm3. Most footnotes refer to additional contrasts, all remaining contrasts did not reach significance. 
aFor contrast w&w versus where significant at P < 10−6. bHere, RC for where > what. cFor w&w versus where significant at P < 10−3. dFor w&w versus what significant at P < 10−4. eFor w&w versus what significant at P < 0.05. fFor w&w versus what significant at P < 10−2. gFor where versus w&w significant at P < 0.05. hFor where versus w&w significant at P < 10−5. iFor where versus w&w significant at P < 0.05. j For what versus where significant at P < 10−4. kFor what versus where significant at P < 2 × 10−4. lFor what versus where significant at P < 0.01. mFor where versus what significant at P < 10−6. nFor where versus what (where > what) significant at P < 2 × 10−5. oFor w&w versus what significant at P < 10−3. pFor what versus where significant at P < 0.02. qFor where versus what (where > what) significant at P < 10−6. rFor where versus what significant at P < 2 × 10−4. sFor w&w versus what P < 0.06. tHere, RC for where > w&w. 
Encoding, ‘where’ versus ‘what’ 
MFG/SFG LH where –26  –7 46 144  0.6  2 × 10−4 10−5  0.4 
GL LH what –14 –62 –3 404 10−6a  0.3  0.4 0.01b 
CU RH what –85 21 2128 10−6c  0.06  0.02  0.04b 
SPL RH where 32 –49 43 1999  0.6 10−6d  10−4 >0.9 
SFG RH where 20 –7 62 974  0.3  0.05 e  0.05f  0.5 
GPrC RH where 46 23 1197  0.05g 10−4  10−6h  0.05i 
Encoding, ‘w&w’ versus ‘where’ 
GF LH w&w –31 –39 –14 1204 10−6j  0.004 >0.9  0.7 
POS LH w&w –5 –78 20 850  0.01k  0.3  0.5  0.004 
INS LH w&w –32 10 10 354  0.2  0.05l  0.2  0.07 
IPL RH where 35 –34 43 994  0.6  0.01m 10−4m  0.9 
GL RH w&w 15 –39 986 10−4n  0.2  0.8  0.07n 
Delay, ‘where’ versus ‘what’ 
MFG/IFG LH what –43 24 25 6434  0.006  0.003  0.3  0.002 
IFG/INS LH what –38 14 2742  0.9  0.001  0.5  0.03 
SPL LH where –26 –60 47 10907  0.2  2 × 10−6  0.0002  0.2 
MFG/SFG LH where –27 –7 47 391  0.5 10−3 10−6  0.3 
Delay, ‘w&w’ versus ‘what’ 
SFG RH w&w 27 –9 59 1768  0.4  0.01  0.01  0.6 
IFG LH what –49 11 12 299  0.4  0.7  0.5  0.1 
SPL LH w&w –22 –66 46 8977  0.8  3 × 10−4  0.01  0.6 
SPL RH w&w 12 –72 46 2897  0.8 10−6  0.05  0.6 
SMA LH w&w –4 43 383  0.2  0.009  0.3  0.5 
SMA RH w&w 49 774  0.6  0.02  0.6  0.8 
Delay, ‘w&w’ versus ‘where’ 
IPL LH where –47 –41 36 569  0.4  0.03o  0.0007 >0.9 
IFG LH w&w –44 20 23 4723 10−3  0.003  0.6  0.005 
INS LH w&w –36 13 2255  0.2  0.005  0.2  0.004 
MFG LH w&w –43 36 2584  0.002  0.05p  0.4 >0.9 
SMA LH w&w –4 11 45 835  0.4  0.02  0.7  0.01 
SPL RH where 25 –57 49 7022  0.3 10−4q 10−3q  0.2 
SFG RH where 25 –8 47 462  0.2  0.01r  0.01q >0.9 
SMA RH w&w 51 263  0.8  0.01s >0.9  0.01t 
INS RH w&w 32 14 1585  0.6  0.05  0.2 >0.9 
Figure 1.

Experiment 1. (A) Structure and timing of the STM task (for details see Materials and Methods). The lines labeled ‘L’ and ‘R’ schematically represent the voltage of the two response buttons, which increases after the reaction time (RT) to signal a correct ‘match’-response and which was fed back to the subject by a green fixation cross at the end of the trial. (B) Color maps for the major sulci of the MNI template brain flattened with BrainVoyager software, see list of abbreviations. (C, D) Averaged BOLD time-courses (percentage signal change) and activation maps for the left and right hemispheres of five subjects. Error bars denote mean ± SEM. The onset of the target stimulus was at 4 s. Note that sampling of functional data was restricted to six axial slices covering most of the frontal, temporal and occipital cortex and excluding the entire parietal lobe. Green lines and green/blue clusters represent activity in areas that respond to visual stimulation during sample and test presentation, whereas yellow lines and yellow/blue clusters represent significant activity during the delay. (C) ‘What’ task. (D) ‘Where’ task.

Figure 1.

Experiment 1. (A) Structure and timing of the STM task (for details see Materials and Methods). The lines labeled ‘L’ and ‘R’ schematically represent the voltage of the two response buttons, which increases after the reaction time (RT) to signal a correct ‘match’-response and which was fed back to the subject by a green fixation cross at the end of the trial. (B) Color maps for the major sulci of the MNI template brain flattened with BrainVoyager software, see list of abbreviations. (C, D) Averaged BOLD time-courses (percentage signal change) and activation maps for the left and right hemispheres of five subjects. Error bars denote mean ± SEM. The onset of the target stimulus was at 4 s. Note that sampling of functional data was restricted to six axial slices covering most of the frontal, temporal and occipital cortex and excluding the entire parietal lobe. Green lines and green/blue clusters represent activity in areas that respond to visual stimulation during sample and test presentation, whereas yellow lines and yellow/blue clusters represent significant activity during the delay. (C) ‘What’ task. (D) ‘Where’ task.

Figure 2.

Experiment 2. (A) Structure and timing of the task (for details see Materials and Methods). (B) Color legend for activation maps in (C, D). Mixing of the three basic colors is performed in RGB-space. Intersections of the maps computed for the different tasks appear as natural color combinations, e.g. red overlapping with green appears as yellow, overlap of all three colors appears as white. (C, D) Superimposed semi-transparent activation maps (superposition maps) for the three task types during encoding (C) and delay (D) epochs.

Figure 2.

Experiment 2. (A) Structure and timing of the task (for details see Materials and Methods). (B) Color legend for activation maps in (C, D). Mixing of the three basic colors is performed in RGB-space. Intersections of the maps computed for the different tasks appear as natural color combinations, e.g. red overlapping with green appears as yellow, overlap of all three colors appears as white. (C, D) Superimposed semi-transparent activation maps (superposition maps) for the three task types during encoding (C) and delay (D) epochs.

Figure 3.

Relative contribution maps for the three pairs of conditions. Left and right pairs of maps in each row represent activation during encoding and delay, respectively. Conventions as in Figure 1C,D. (A) ‘What’ versus ‘where’. (B) Conjunction versus ‘what’. (C) Conjunction versus ‘where’.

Relative contribution maps for the three pairs of conditions. Left and right pairs of maps in each row represent activation during encoding and delay, respectively. Conventions as in Figure 1C,D. (A) ‘What’ versus ‘where’. (B) Conjunction versus ‘what’. (C) Conjunction versus ‘where’.

We thank N. Kriegeskorte for designing the stimuli of the control experiment, S. Kemble for her contribution to data analysis, J. A. Waltz for comments on the manuscript, A. Wiatrowski for help with the training sessions and R. Ruhl for help with the figures.

References

Andersen RA (
1997
) Multimodal integration for the representation of space in the posterior parietal cortex.
Phil Trans R Soc Lond B Biol Sci
 
352
:
1421
–1428.
Andersen RA, Bracewell RM, Barash S, Gnadt JW, Fogassi L (
1990
) Eye position effects on visual, memory, and saccade-related activity in areas LIP and 7a of macaque.
J Neurosci
 
10
:
1176
–1196.
Belger A, Puce A, Krystal JH, Gore JC, Goldman-Rakic P, McCarthy G (
1998
) Dissociation of mnemonic and perceptual processes during spatial and nonspatial working memory using fMRI.
Hum Brain Mapp
 
6
:
14
–32.
Boynton GM, Engel SA, Glover GH, Heeger DJ (
1996
) Linear systems analysis of functional magnetic resonance imaging in human V1.
J Neurosci
 
16
:
4207
–4211.
Bullmore E, Brammer M, Williams SC, Rabe-Hesketh S, Janot N, David A, Mellers J, Howard R, Sham P (
1996
) Statistical methods of estimation and inference for functional MR image analysis.
Magn Reson Med
 
35
:
261
–277.
Callicott JH, Mattay VS, Bertolino A, Finn K, Coppola R, Frank JA, Goldberg TE, Weinberger DR (
1999
) Physiological characteristics of capacity constraints in working memory as revealed by functional MRI.
Cereb Cortex
 
9
:
20
–26.
Chafee MV, Goldman-Rakic PS (
1998
) Matching patterns of activity in primate prefrontal area 8a and parietal area 7ip neurons during a spatial working memory task.
J Neurophysiol
 
79
:
2919
–2940.
Cohen JD, Perlstein WM, Braver TS, Nystrom LE, Noll DC, Jonides J, Smith EE (
1997
) Temporal dynamics of brain activation during a working memory task.
Nature
 
386
:
604
–608.
Colby CL, Goldberg ME (
1999
) Space and attention in parietal cortex.
Annu Rev Neurosci
 
22
:
319
–349.
Constantinidis C, Steinmetz MA (
1996
) Neuronal activity in posterior parietal area 7a during the delay periods of a spatial memory task.
J Neurophysiol
 
76
:
1352
–1355.
Courtney SM, Ungerleider LG, Keil K, Haxby JV (
1997
) Transient and sustained activity in a distributed neural system for human working memory.
Nature
 
386
:
608
–611.
Courtney SM, Petit L, Maisog JM, Ungerleider LG, Haxby, JV (
1998
) An area specialized for spatial working memory in human frontal cortex.
Science
 
279
:
1347
–1351.
Courtney SM, Petit L, Haxby JV, Ungerleider LG (
1998
) The role of prefrontal cortex in working memory: examining the contents of consciousness.
Philos Trans R Soc Lond B Biol Sci
 
353
:
1819
–1828.
D’Esposito M, Aguirre GK, Zarahn E, Ballard D, Shin RK, Lease J (
1998
) Functional MRI studies of spatial and nonspatial working memory.
Brain Res Cogn Brain Res
 
7
:
1
–13.
D’Esposito M, Postle BR, Ballard D, Lease J (
1999
) Maintenance versus manipulation of information held in working memory: an event-related fMRI study.
Brain Cogn
 
41
:
66
–86.
Dierks T, Linden DEJ, Jandl M, Formisano E, Goebel R, Lanfermann H, Singer W (
1999
) Activation of Heschl’s gyrus during auditory hallucinations.
Neuron
 
22
:
615
–621.
Elliott R, Dolan RJ (
1999
) Differential neural responses during performance of matching and nonmatching to sample tasks at two delay intervals.
J Neurosci
 
19
:
5066
–5073.
Friston KJ, Frith CD, Frackowiak RS, Turner R (
1995
) Characterizing dynamic brain responses with fMRI: a multivariate approach.
Neuroimage
 
2
:
166
–172.
Funahashi S, Bruce CJ, Goldman-Rakic PS (
1989
) Mnemonic coding of visual space in the monkey’s dorsolateral prefrontal cortex.
J Neurophysiol
 
61
:
331
–349.
Fuster JM (
1998
) Cellular dynamics of network memory.
Z Naturforsch [C]
 
53
:
670
–676.
Fuster JM, Alexander GE (
1971
) Neuron activity related to short-term memory.
Science
 
173
:
652
–654.
Fuster JM, Jervey JP (
1981
) Inferotemporal neurons distinguish and retain behaviorally relevant features of visual stimuli.
Science
 
212
:
952
–955.
Goebel R, Linden DEJ, Lanfermann H, Zanella FE, Singer W (
1998
) Functional imaging of mirror and inverse reading reveals separate coactivated networks for oculomotion and spatial transformations.
Neuroreport
 
9
:
713
–719.
Goebel R, Khorram-Sefat D, Muckli L, Hacker H, Singer W (
1998
) The constructive nature of vision: direct evidence from functional magnetic resonance imaging studies of apparent motion and motion imagery.
Eur J Neurosci
 
10
:
1563
–1573.
Goodale MA (
1998
) Vision for perception and vision for action in the primate brain.
Novartis Found Symp
 
218
:
21
–34.
Haxby JV, Petit L, Ungerleider LG, Courtney SM (
2000
) Distinguishing the functional roles of multiple regions in distributed neural systems for visual working memory.
Neuroimage
 
11
:
380
–391.
Ishai A, Ungerleider LG, Martin A, Schouten JL, Haxby JV (
1999
) Distributed representation of objects in the human ventral visual pathway.
Proc Natl Acad Sci USA
 
96
:
9379
–9384.
McCarthy G, Puce A, Constable RT, Krystal JH, Gore JC, Goldman-Rakic P (
1996
) Activation of human prefrontal cortex during spatial and nonspatial working memory tasks measured by functional MRI.
Cereb Cortex
 
6
:
600
–611.
Miller EK (
2000
) The prefrontal cortex: no simple matter.
Neuroimage
 
11
:
447
–450.
Miller EK, Li L, Desimone R (
1993
) Activity of neurons in anterior inferior temporal cortex during a short-term memory task.
J Neurosci
 
13
:
1460
–1478.
Miller EK, Erickson CA, Desimone R (
1996
) Neural mechanisms of visual working memory in prefrontal cortex of the macaque.
J Neurosci
 
16
:
5154
–5167.
Mishkin M, Delacour J (
1975
) An analysis of short-term visual memory in the monkey.
J Exp Psychol: Anim Behav Process
 
1
:
326
–334.
Mitchell KJ, Johnson MK, Raye CL, D’Esposito M (
2000
) fMRI evidence of age-related hippocampal dysfunction in feature binding in working memory.
Brain Res Cogn Brain Res
 
10
:
197
–206.
Nystrom LE, Braver TS, Sabb FW, Delgado MR, Noll DC, Cohen JD (
2000
) Working memory for letters, shapes, and locations: fMRI evidence against stimulus-based regional organization in human prefrontal cortex.
Neuroimage
 
11
:
424
–446.
Owen AM, Stern CE, Look RB, Tracey I, Rosen BR, Petrides M (
1998
) Functional organization of spatial and nonspatial working memory processing within the human lateral frontal cortex.
Proc Natl Acad Sci USA
 
95
:
7721
–7726.
Owen AM, Herrod NJ, Menon DK, Clark JC, Downey SP, Carpenter TA, Minhas PS, Turkheimer FE, Williams EJ, Robbins TW, Sahakian BJ, Petrides M, Pickard JD (
1999
) Redefining the functional organization of working memory processes within human lateral prefrontal cortex.
Eur J Neurosci
 
11
:
567
–574.
Pochon JB, Levy R, Poline JB, Crozier S, Lehericy S, Pillon B, Deweer B, Le Bihan D, Dubois B (
2001
) The role of dorsolateral prefrontal cortex in the preparation of forthcoming actions: an fMRI study.
Cereb Cortex
 
11
:
260
–266.
Postle BR, D’Esposito M (
1999
) ‘What’–then–‘where’ in visual working memory: an event-related fMRI study.
J Cogn Neurosci
 
11
:
585
–597.
Postle BR, D’Esposito M (
2000
) Evaluating models of the topographical organization of working memory function in frontal cortex with event-related fMRI.
Psychobiology
 
28
:
132
–145.
Postle BR, Berger JS, Taich AM, D’Esposito M (
2000
) Activity in human frontal cortex associated with spatial working memory and saccadic behavior.
J Cogn Neurosci
 
12
(Suppl. 2):
2
–14.
Prabhakaran V, Narayanan K, Zhao Z, Gabrieli JD (
2000
) Integration of diverse information in working memory within the frontal lobe.
Nat Neurosci
 
3
:
85
–90.
Quintana J, Fuster JM (
1999
) From perception to action: temporal integrative functions of prefrontal and parietal neurons.
Cereb Cortex
 
9
:
213
–221.
Rypma B, D’Esposito M (
1999
) The roles of prefrontal brain regions in components of working memory: effects of memory load and individual differences.
Proc Natl Acad Sci USA
 
96
:
6558
–6563.
Sack AT, Hubl D, Prvulovic D, Formisano E, Jandl M, Zanella FE, Maurer K, Goebel R, Dierks T, Linden DEJ (
2002
) The experimental combination of rTMS and fMRI reveals the functional relevance of parietal cortex for visuospatial functions.
Cogn Brain Res
 
13
:
85
–93.
Sakata H, Taira M, Kusunoki M, Murata A, Tanaka Y (
1997
) The TINS Lecture. The parietal association cortex in depth perception and visual control of hand action.
Trends Neurosci
 
20
:
350
–357.
Trojano L, Grossi D, Linden DEJ, Formisano E, Hacker H, Zanella FE, Goebel R, Di Salle F (
2000
) Matching two imagined clocks: the functional anatomy of spatial analysis in the absence of visual stimulation.
Cereb Cortex
 
10
:
473
–481.
Zarahn E, Aguirre GK, D’Esposito M (
1999
) Temporal isolation of the neural correlates of spatial mnemonic processing with fMRI.
Brain Res
 
7
:
255
–268.