Abstract

Systems models hold working memory to depend on specialized, domain-specific storage buffers. Here, however, we demonstrate that short-term retention of the identity or location of visually presented stimuli is disrupted by nonvisual secondary tasks performed in the dark—passive listening to nouns or endogenous generation of saccades, respectively. This indicates that the short-term retention of visual information relies on multiple mental codes, some of them nonvisual. Event-related functional magnetic resonance imaging (fMRI) reveals the neural correlates of these interference effects to be more complex and more regionally specific than previously described. Although nonspecific dual-task effects produce a generalized decrease of task-evoked fMRI response across many brain regions, the interference-specific effect is a relative increase of activity localized to regions associated with the secondary task in question: left hemisphere perisylvian cortex in the case of passive listening distraction and frontal oculomotor regions in the case of saccadic distraction. Within these regions, the neural interference effects are specific to voxels that show delay-period activity on unfilled memory trials. They also predict individual differences in the magnitude of the behavioral interference effect. These results indicate that nonvisual processes supported by nonvisual brain areas contribute importantly to “visual” working memory performance.

Introduction

Working memory refers to the ability to retain information in an active state when it is not present in the environment, to transform it when necessary, and to use it to guide behavior. The multicomponent model of working memory, first proposed by Baddeley and Hitch (1974), has enjoyed remarkable influence over the past 3 decades in guiding experimental psychological, neurophysiological, neuroimaging, and computational studies of working memory (Postle 2006; Repovs and Baddeley 2006). This model depicts working memory as supported by a system of storage buffers that are specialized for retaining information in a domain-specific manner, often preserving the domain in which the to-be-remembered stimulus information was presented. Much of the foundational research that preceded, validated, and extended this model has relied on patterns of selective interference presumed to reflect competition between the primary and secondary tasks for a common mental code (Baddeley 1986). For example, the idea that human visual working memory for what an object is depends on ventral stream mental codes draws support from the fact that it is disrupted by secondary tasks also known to recruit the ventral visual stream, such as color discrimination (Tresch et al. 1993) or viewing abstract paintings (Della Sala et al. 1999). Conversely, that visual working memory for where an object is located depends on dorsal stream mental codes is demonstrated when it is disrupted by secondary tasks that also recruit the dorsal visual stream, such as motion discrimination (Tresch et al. 1993) or haptically guided sequential hand movement (Della Sala et al. 1999). At the neural level, neuroimaging data have suggested that a neural correlate of such behavioral interference is overlapping cortical recruitment by the primary and secondary task (Klingberg and Roland 1997; Klingberg 1998), sometimes resulting in a decrease in signal in the region of overlap (Gisselgard et al. 2003; Gruber and von Cramon 2003). The present study used this venerable experimental logic, but with a novel procedural twist and with event-related functional magnetic resonance imaging (fMRI), to refine and/or reevaluate these theoretical and neurophysiological assumptions.

In the majority of selective interference research to date, secondary tasks and stimuli have been selected precisely because they featured the same domain and modality, and thus were expected to recruit the same mental processes, as did the primary memory tasks. (Two notable exceptions are nonvisual spatial secondary tasks that have demonstrated an important spatial component of visuospatial working memory [reviewed in Postle et al. 2006] and phonological secondary tasks that demonstrate fact that visually presented digits, letters, and words are recoded into phonologically based representations [reviewed in Repovs and Baddeley 2006].) In the present study, in contrast, we evaluated the effects of secondary distracting tasks that did not entail any visual stimulation and that did not overlap with the domains from which the memoranda were drawn. These secondary tasks, therefore, presented no obvious source of interference. Our study was motivated by the principle of multiple encoding (Wickens 1973; Postle 2006), which posits that humans will recode information and represent it in parallel in as many mental codes as are afforded by the stimulus. This principle accounts for the fact that the representational bases of working memory are not limited to the domain in which stimuli are perceived (Postle 2006). The behavioral and neural correlates of 2 examples of this principle were tested in the experiments presented here.

One example that we studied was spatial working memory. It is well-established that one way that we represent visually perceived locations is with attention-based rehearsal, the covert allocation of attention to the location at which the stimulus appeared (Awh and Jonides 2001). A second mechanism for the short-term retention of a location, however, is to prepare a motor plan for the acquisition of the target (e.g., with a saccade or a grasp) and to retain this motor plan as a “prospective memory” (e.g., Fuster 1995; Postle and D'Esposito 2003; Curtis et al. 2004). Thus, the present experiments were designed, in part, to replicate the finding that spatial delayed recognition would be selectively sensitive to concurrent performance of endogenously guided saccades (Postle et al. 2006).

The second example of multiple encoding that we studied here was working memory for visually presented objects, for which the well-established representational codes are those used by the perceptual apparatus of the ventral visual stream (e.g., Tresch et al. 1993; Smith et al. 1995; Della Sala et al. 1999; Ranganath 2006). In addition to visual codes, however, we have postulated that the short-term retention of object information also recruits verbal (and, perhaps, semantic) representations (Postle et al. 2005). Specifically, because object recognition entails the association of visually perceived information with preexisting knowledge, we believe that the short-term retention of information about a visually presented object entails the retention of this semantic information, in addition to the retention of visual information. (We have hypothesized that this multiple encoding occurs automatically and obligatorily [Postle et al. 2005], although we do not test this particular claim here). As a result, we predicted that object delayed recognition in the present study would be selectively sensitive to concurrent passive listening to concrete nouns. This predicted outcome would represent a replication and extension of previous results, in that a previous demonstration of selective disruption of object delayed recognition performance with a verbal secondary task used visually presented words for the verbal secondary task (Postle et al. 2006). This procedure left open the possibility that the interference was visual, not verbal. For the present study, we did not have strong predictions, a priori, about whether the object–verbal interference would occur at the level of phonological representations (i.e., the “sounds” of the secondary words interfering with phonological representations of the object memoranda) or at the level of semantic representations (i.e., the “ideas” evoked by the secondary words interfering with semantic representations of the object memoranda). It would turn out, however, that the fMRI results suggested a post hoc account of the locus of the interference.

For both of these domains of memory, we used fMRI to examine the neural bases of secondary task interference produced by secondary tasks employing nonoverlapping modalities and drawing on nonoverlapping domains.

Overview of the Design

The logic of the study was to test the prediction that the data from 4 cells of a 2 (memory domain: location, object) × 2 (distraction: saccades, passive listening) fully crossed design would produce a crossover interaction. Such a “double dissociation” would permit strong inference that the predicted disruptive effects of saccades on location memory and of listening to nouns on object memory were selective. That is, it would rule out concerns that either of these instances of interference might be a nonspecific effect due merely to dividing attention between 2 concurrent tasks. This critical 2 × 2 arrangement was embedded in an overall design that had 3 levels per factor, because the factor of memory domain also had a “no memory” level, and the factor of distraction also had a “no-distraction” level. This was important for the fMRI study in particular because our underlying hypothesis for spatial working memory, for example, was that oculomotor codes contribute even to the simple, undistracted short-term retention of location information. Additionally, although the 2 × 2 contrast was necessary for demonstrating the specificity of the predicted effects, an interference effects itself must be defined and measured in relation to a no-distraction baseline. Thus, we used a multistep procedure, the essence of which was to first identify voxels that were active during the delay period of no-distraction trials, then to evaluate how the activity in these same “memory-delay” voxels varied as a function of concurrent distraction.

Materials and Methods

Subjects

Twenty-two healthy young adults (3 males, mean age = 22.6, standard deviation [SD] = 3.5) participated in a preliminary behavioral study, and 12 healthy young adults (5 males, mean age = 22.8, SD = 4.2) participated in the fMRI study. In both cases, subjects gave informed consent and reported neither history of neurological or psychiatric conditions nor recent use of psychoactive drugs. The methods described here were approved by the relevant Institutional Review Boards of the University of Wisconsin-Madison.

Apparatuses

A PC running Eprime software (Psychology Software Tools, Pittsburgh, PA) was used for all stimulus presentation and response collection. For the preliminary behavioral study, testing took place in a sound-attenuated booth in which all sources of illumination, except the monitor screen on which stimuli were presented, were occluded. The “brightness” setting on the monitor (Mitsubishi Diamond Plus 200) was set to 0 in order to prevent the gray–black glow that typically emanates from a computer monitor that is displaying a uniformly blank screen. The effect for the subject was to be sitting in a room in which nothing was visible except for the periodic appearance of instructions and stimuli. The subject sat with chin and forehead resting in a chin rest that was equipped with optics for infrared-based eye tracking (ASL Model 501, Bedford, MA). The chin rest was oriented such that, when positioned in it, the subject's natural “straight ahead” gaze would fall on the center of the display screen, with the eye approximately 63.5 cm from the screen. During the fMRI session, subjects viewed stimuli through eyepieces that displayed a rectangular screen subtending approximately 30 horizontal by 23 vertical degrees of visual angle (Avotec Silent Vision, Stuart, FL) and fitted with infrared-based eye-tracking cameras (SMI iView X, Teltow, Germany). Auditory cues were presented via nonpneumatic magnetic resonance–compatible headphones with 30 dB external noise attenuation (Resonance Technology Commander XG, Northridge, CA). Responses were recorded via a fiber optic button box connected to the PC running Eprime.

Behavioral Methods

Trials were drawn from each cell of the fully crossed 3 (memory domain: location, object, no memory) × 3 (secondary task: endogenously guided saccades, passive listening to concrete nouns, none) design, with the exception of the “no memory/no secondary task” cell. Each trial began with a 2-s presentation of instructions, indicating the memory domain. This was followed by a 2-s blank period, after which the target stimulus was presented for 1 s. Following this was a delay period, during which one of the 3 types of secondary tasks occurred. This was followed by a 1-s presentation of a memory probe stimulus, to which the subject was instructed to make a match/nonmatch response (right thumb/left thumb, respectively). For the preliminary behavioral study, the delay period lasted 11 s, 500 ms into which a luminance mask was flashed for a duration of 500 ms. Three seconds after the mask offset the secondary task period began, lasting until 500 ms prior to the onset of the memory probe. The procedure for the fMRI study was similar, except there was no mask, and the delay period was 7 s long. The intertrial interval (ITI) was 3 s for the preliminary behavioral study and 10 s for the fMRI study. With the exception of the saccadic secondary task and the ITI, subjects were trained to always keep their eyes in their natural straight ahead position whenever a stimulus was not present on the screen. (Although this corresponded to central fixation, there were never fixation cues, and care was taken not to suggest that subjects imagine a fixation cue).

The preliminary behavioral experiment was administered in 4 24-trial blocks. The fMRI experiment was administered in 8 16-trial blocks, with 2 trials of each type occurring in a randomly determined order in each block.

Memory Tasks

The 64 memory stimuli were organized around 16 abstract shapes (Attneave and Arnoult 1956) normatively determined to be difficult to verbalize (Vanderplas and Garvin 1959), subtending approximately 2 degrees of visual angle in their largest dimension. Each of these served as a prototype for 3 variants that each differed from the prototype by one salient feature (Postle and D'Esposito 1999). On each trial, the target was pulled at random (without replacement) from the pool of 64. On “location memory” trials, the target position for each trial was determined pseudorandomly with 2 constraints: 1) across an experimental session, an equal number of location targets appeared in each quadrant of the screen and 2) no 2 targets were centered on precisely the same coordinates. Location probes were always drawn at random from one of the other 15 prototype groups, and the central coordinate of nonmatching probe locations was an average of 4 degrees of visual angle (SD = 1 degree, min = 2 degrees, max = 10.7 degrees) distant from the central coordinate of the corresponding target positions. That is, the average distance between the closest edges of a target and a nonmatching trial-final probe was roughly equivalent to the diameter of a stimulus. Instructions and training emphasized that a location-match probe stimulus would always have precisely the same center of mass as the corresponding target, whereas a location nonmatch would not overlap any area occupied by the target stimulus. On “object memory” trials, the stimuli always appeared in the center of the display, and nonmatching probes were always drawn from the same prototype group as the target. On location and object trials, the probe matched the target with P = 0.5. The stimulus on “no memory” trials was a white circle of 2 degrees of visual angle that was presented at the center of the screen as both target and probe. On these trials, subjects were instructed to respond with either thumb.

Secondary Tasks

Secondary task cues and stimuli were delivered via a loudspeaker positioned directly below the computer monitor for the preliminary behavioral study and via the headphones for the fMRI study. The onset and offset of saccadic secondary task periods were cued by a tone of 500 ms duration, the onset of which lagged the offset of the target by 500 ms and the offset of which preceded the onset of the probe by 500 ms. Subjects had been instructed that, upon hearing the first tone, they were to move their eyes in any manner that they chose until the second tone signaled the end of this secondary task period. The intent of this procedure was to minimize the likelihood that nonmotoric factors could explain the selective disruption that saccadic distraction was expected to produce. Because participants were seated in absolute darkness during the distraction portion of the delay period, and because the source of the auditory “start” and “stop” signals was 0 degree with respect to trunk and head position, there were no possible sources of perceptual distraction. Saccades were endogenously generated, and participants had never been trained to guide them with, for example, a visual array of targets. Therefore, the likelihood that participants guided their saccades with internally generated mental images was minimized to the greatest extent possible.

The “passive listening” secondary task began 500 ms into the delay, consisted of the serial presentation of 6 concrete nouns, one per second, spoken by a male voice, and ended 500 ms prior to the onset of the memory probe. Subjects were told that they did not have to “do anything” with the stimuli, but simply to maintain straight ahead eye position throughout the delay period.

No secondary task trials amounted to conventional delayed recognition with unfilled delay periods.

Tracking Eye Position

Great care was taken to rule out the possibility that endogenously guided eye movements made in the dark may have been targeting specific locations in space, as well as to ensure that fixation was maintained during all other trial types. Following the experiment, eye-tracking data were aligned to each block's starting time and segmented into the target, delay, probe, and ITI periods. Eye position during the delay period was analyzed with ILAB software (Gitelman 2002) to evaluate compliance with the instructions for each type of secondary task. Although there was no fixation cross for the passive listening and “no secondary task” conditions, subjects were instructed to maintain straight ahead eye position during these delay periods. On these trials, eye position 500 ms after the offset of the target was designated to be the initial center of gaze for that trial's delay period, and trials for which delay-period center of gaze moved more than 2 degrees of visual angle from this initial center of gaze were designated as “fixation failure” trials. (How this influenced analyses is described below, in fMRI Methods, Data Analyses). “Saccade” trials, in contrast, were scored as “unacceptable” if delay-period eye position failed to cover each of the quadrants of the screen, and/or center of gaze was stationary for greater than 50% of the 6-s secondary task period. Additionally, eye movement patterns were analyzed to ensure that subjects did not make saccades in an orderly pattern, such as by tracing out a shape, or repeatedly moving to a specific location. This ensured that subjects were not using eye movements to represent properties of the target.

fMRI Acquisition and Preprocessing

Whole-brain images were acquired with a 3-T scanner (GE Signa VH/I). For all volunteers, we acquired high-resolution T1-weighted images (30 axial slices, 0.9375 × 0.9375 × 4 mm). We used a gradient-echo, echo-planar sequence (time repetition = 2000 ms, echo time = 50 ms) to acquire data sensitive to the blood oxygen level–dependent (BOLD) signal (Kwong et al. 1992; Ogawa et al. 1992) within a 64 × 64 matrix (30 axial slices coplanar with the T1 acquisition, 3.75 × 3.75 × 4 mm). Eight scans of the experimental task were obtained for each subject, each lasting 7 min 16 s (6 min 56 s of task preceded by 20 s of dummy pulses to achieve a steady state of tissue magnetization). Prior to scans of the experimental tasks, for each participant we obtained a scan from which we derived an estimate of the hemodynamic response function (HRF). During this scan, each participant performed a simple reaction-time task that required a bimanual button press once every 20 s in response to a brief change in shape of the fixation stimulus.

Raw BOLD image files went through the following preprocessing steps, in order: reconstruction (in-house software incorporating GE's EPIrecon code); sync interpolation in time to correct for the slice acquisition sequence (VoxBo—www.voxbo.org); correction for magnetic field inhomogeneities (in-house software); and rigid-body realignment to the first volume from the experimental task (VoxBo). Note that neither was spatial smoothing imposed nor were the data spatially transformed into a common atlas space prior to hypothesis testing. Rather, the data from each subject were analyzed in that subject's unsmoothed, native space (Postle, Zarahn et al. 2000; Postle et al. 2003). (Warping of data to Montreal Neurological Institute [MNI] space for display [Fig. 2] and tabulation [Table 1] of group-averaged results was done after the fact, for descriptive purposes only.) For the HRF-derivation run, a partial F-test associated with a Fourier basis covariate set was used to evaluate the significance of task-correlated activity in each voxel of the primary somatosensory and motor cortical regions of interest (ROIs). We obtained an HRF estimate by spatially averaging the time series of the suprathreshold voxels of these ROIs, filtering the resultant averaged fMRI time series to remove high (>0.244 Hz) and low (<0.05 Hz) frequencies, adjusting it to remove the effects of nuisance covariates (Friston et al. 1995) and trial averaging. The HRF describes the fMRI response resulting from a brief impulse of neural activity (Boynton et al. 1996) and can vary markedly across participants (Aguirre et al. 1998; Handwerker et al. 2004). Each subject's HRF was used to convolve independent variables entered into the modified general linear model (GLM) (Worsley and Friston 1995) that we used to analyze the data from the scans of the experimental task.

Table 1

Tabulation of cluster sizes and loci of suprathreshold activity (P < .001, uncorrected) from spatially normalized group-averaged analyses

Region (BA) Number of voxel Talairach coordinates 
  x y z 
Delaylocation, no distraction versus baseline 
    R superior frontal (6/8) 29 14 48 
    L post cingulate (31) −20 −28 36 
    R post cingulate (31) 13 −29 30 
    R anterior cingulate (32) 13 36 22 
    Thalamus −9 −17 11 
    L occipital/lingual (18) −22 −57 
    L fusiform (37) 42 −52 −4 
    R middle temporal (21) 39 −49 −8 
    L middle temporal (21) −43 −49 −8 
    L cerebellum −44 −44 −36 
Delayobject, no distraction versus baseline 
    L middle frontal (6) −36 10 42 
    L cingulate (24) −5 −3 38 
    R precentral (6) 47 −7 38 
    L precentral (6) −38 −10 38 
    R supramarginal (40) 31 −48 34 
    L cingulate (24) −9 26 
    R anterior cingulate (32) 13 18 26 
    R inferior frontal (9) 42 26 
    R caudate 23 −23 26 
    L insula (13) −31 −23 23 
    L inferior frontal (9) −47 11 24 
    R middle frontal (9) 38 22 23 
    R inferior frontal (44) 54 14 19 
    L inferior frontal (45) −43 27 15 
    R inferior frontal (46) 43 34 12 
    R caudate 16 14 12 
    R insula 38 −3 
    L superior temporal (22) −58 −26 
    L insula −36 −4 
    R inferior frontal (47) 46 24 −4 
    L cerebellum −9 −61 −8 
    Midbrain −3 −7 −11 
    R inferior frontal (34) 13 −12 
    L inferior frontal (47) −31 18 −12 
    L cerebellum −21 −56 −21 
    L fusiform (20) −43 −28 −21 
    R superior temporal (38) 42 14 −21 
    R parahippocampal 31 −11 −24 
    Pons −21 −26 
    R cerebellum 38 −26 −31 
Delayno memory, saccades versus baseline 
    Precuneus (7) −49 53 
    R middle frontal (6) 21 53 
    R precentral (4) 39 −21 53 
    L middle frontal (6) −20 −1 49 
    L paracentral (5) −2 −34 49 
    L medial frontal (32) −2 46 
    L precentral (6) −38 −11 42 
    R posterior cingulate (31) −38 41 
    R precentral (6) 10 45 −9 39 
    R precuneus (7) 12 −52 37 
    L cingulate (24) −6 −5 37 
    L precuneus (31) −15 −54 34 
    R cingulate (24) 13 31 
    L cingulate (23) −8 −12 31 
    L posterior cingulate (32) −12 −50 28 
    R precuneus (31) 12 −69 24 
    R anterior cingulate (32) 25 24 
    L inferior frontal (44) −51 10 18 
    L cuneus (18) 13 −4 −72 17 
    R inferior frontal (44) 56 16 
    R anterior cingulate (24) 23 16 
    R posterior cingulate (24/30) 18 −60 12 
    L middle occipital (19) −31 −61 
    L insula (13) −44 
    L parahippocampal 11 −13 −49 
    R putamen 11 20 
    R lingual (19) 24 10 −60 
    R superior temporal (21) 42 −29 
    L lingual (19) 10 −13 −52 
    L caudate −20 15 
    L insula −29 14 
    Thalamus −6 −8 −1 
    R inferior frontal (47) 34 28 −4 
    R putamen 16 −9 −4 
    L putamen −14 −7 −4 
    L superior temporal (38) −51 12 −8 
    R parahippocampal 21 −29 −12 
    L inferior frontal (47) −28 27 −12 
    R inferior frontal (47) 20 20 −17 
    L fusiform −55 −11 −23 
    R cerebellum 20 −44 −26 
    L cerebellum 15 −22 −48 −26 
Delayno memory, passive listening versus baseline 
    R middle frontal (6) 34 −7 53 
    L middle frontal (6) −12 −6 49 
    R precentral (4) 36 −19 48 
    L medial frontal (8) −1 20 46 
    R anterior cingulate (24) 11 44 
    R middle frontal (6) 40 43 
    R precentral (6) 42 −7 40 
    L postcentral (3) −47 −23 40 
    R cingulate (31) −33 37 
    L cingulate (23) −5 −7 34 
    R supramarginal (40) 35 −52 31 
    L caudate −20 −22 28 
    R cingulate (23) 16 −30 27 
    R caudate 21 −16 27 
    L precentral (6) −39 27 
    L anterior cingulate (24) −24 26 22 
    R precentral (40) 60 −19 19 
    R middle frontal (46) 47 26 18 
    L inferior frontal (45) −44 22 18 
    L insula (13) 28 −45 −13 10 
    Putamen −16 10 
    R insula (13) 19 39 −14 
    L caudate −16 19 
    L inferior frontal (47) −43 28 −1 
    Thalamus 14 −12 −1 
    R superior temporal (22) 48 54 −10 −5 
    L superior temporal (22) 31 −52 −11 −6 
    L lingual (19) −21 −64 −6 
    R middle frontal (11) 37 38 −9 
    R amygdala 26 −7 −9 
    L parahippocampal −9 −6 −12 
    Brainstem −15 −15 
    R fusiform 45 −23 −18 
    R middle temporal (21) 54 −21 
    R parahippocampal 38 −33 −24 
    R cerebellum 12 −29 −27 
    L cerebellum −21 −41 −32 
Region (BA) Number of voxel Talairach coordinates 
  x y z 
Delaylocation, no distraction versus baseline 
    R superior frontal (6/8) 29 14 48 
    L post cingulate (31) −20 −28 36 
    R post cingulate (31) 13 −29 30 
    R anterior cingulate (32) 13 36 22 
    Thalamus −9 −17 11 
    L occipital/lingual (18) −22 −57 
    L fusiform (37) 42 −52 −4 
    R middle temporal (21) 39 −49 −8 
    L middle temporal (21) −43 −49 −8 
    L cerebellum −44 −44 −36 
Delayobject, no distraction versus baseline 
    L middle frontal (6) −36 10 42 
    L cingulate (24) −5 −3 38 
    R precentral (6) 47 −7 38 
    L precentral (6) −38 −10 38 
    R supramarginal (40) 31 −48 34 
    L cingulate (24) −9 26 
    R anterior cingulate (32) 13 18 26 
    R inferior frontal (9) 42 26 
    R caudate 23 −23 26 
    L insula (13) −31 −23 23 
    L inferior frontal (9) −47 11 24 
    R middle frontal (9) 38 22 23 
    R inferior frontal (44) 54 14 19 
    L inferior frontal (45) −43 27 15 
    R inferior frontal (46) 43 34 12 
    R caudate 16 14 12 
    R insula 38 −3 
    L superior temporal (22) −58 −26 
    L insula −36 −4 
    R inferior frontal (47) 46 24 −4 
    L cerebellum −9 −61 −8 
    Midbrain −3 −7 −11 
    R inferior frontal (34) 13 −12 
    L inferior frontal (47) −31 18 −12 
    L cerebellum −21 −56 −21 
    L fusiform (20) −43 −28 −21 
    R superior temporal (38) 42 14 −21 
    R parahippocampal 31 −11 −24 
    Pons −21 −26 
    R cerebellum 38 −26 −31 
Delayno memory, saccades versus baseline 
    Precuneus (7) −49 53 
    R middle frontal (6) 21 53 
    R precentral (4) 39 −21 53 
    L middle frontal (6) −20 −1 49 
    L paracentral (5) −2 −34 49 
    L medial frontal (32) −2 46 
    L precentral (6) −38 −11 42 
    R posterior cingulate (31) −38 41 
    R precentral (6) 10 45 −9 39 
    R precuneus (7) 12 −52 37 
    L cingulate (24) −6 −5 37 
    L precuneus (31) −15 −54 34 
    R cingulate (24) 13 31 
    L cingulate (23) −8 −12 31 
    L posterior cingulate (32) −12 −50 28 
    R precuneus (31) 12 −69 24 
    R anterior cingulate (32) 25 24 
    L inferior frontal (44) −51 10 18 
    L cuneus (18) 13 −4 −72 17 
    R inferior frontal (44) 56 16 
    R anterior cingulate (24) 23 16 
    R posterior cingulate (24/30) 18 −60 12 
    L middle occipital (19) −31 −61 
    L insula (13) −44 
    L parahippocampal 11 −13 −49 
    R putamen 11 20 
    R lingual (19) 24 10 −60 
    R superior temporal (21) 42 −29 
    L lingual (19) 10 −13 −52 
    L caudate −20 15 
    L insula −29 14 
    Thalamus −6 −8 −1 
    R inferior frontal (47) 34 28 −4 
    R putamen 16 −9 −4 
    L putamen −14 −7 −4 
    L superior temporal (38) −51 12 −8 
    R parahippocampal 21 −29 −12 
    L inferior frontal (47) −28 27 −12 
    R inferior frontal (47) 20 20 −17 
    L fusiform −55 −11 −23 
    R cerebellum 20 −44 −26 
    L cerebellum 15 −22 −48 −26 
Delayno memory, passive listening versus baseline 
    R middle frontal (6) 34 −7 53 
    L middle frontal (6) −12 −6 49 
    R precentral (4) 36 −19 48 
    L medial frontal (8) −1 20 46 
    R anterior cingulate (24) 11 44 
    R middle frontal (6) 40 43 
    R precentral (6) 42 −7 40 
    L postcentral (3) −47 −23 40 
    R cingulate (31) −33 37 
    L cingulate (23) −5 −7 34 
    R supramarginal (40) 35 −52 31 
    L caudate −20 −22 28 
    R cingulate (23) 16 −30 27 
    R caudate 21 −16 27 
    L precentral (6) −39 27 
    L anterior cingulate (24) −24 26 22 
    R precentral (40) 60 −19 19 
    R middle frontal (46) 47 26 18 
    L inferior frontal (45) −44 22 18 
    L insula (13) 28 −45 −13 10 
    Putamen −16 10 
    R insula (13) 19 39 −14 
    L caudate −16 19 
    L inferior frontal (47) −43 28 −1 
    Thalamus 14 −12 −1 
    R superior temporal (22) 48 54 −10 −5 
    L superior temporal (22) 31 −52 −11 −6 
    L lingual (19) −21 −64 −6 
    R middle frontal (11) 37 38 −9 
    R amygdala 26 −7 −9 
    L parahippocampal −9 −6 −12 
    Brainstem −15 −15 
    R fusiform 45 −23 −18 
    R middle temporal (21) 54 −21 
    R parahippocampal 38 −33 −24 
    R cerebellum 12 −29 −27 
    L cerebellum −21 −41 −32 

Note: R = right and L = left.

Data Analyses

The analyses derived from GLMs that modeled task-related variance in the fMRI time series data with covariates consisting of delta functions positioned along the time line of the task to represent trial epochs of interest—instructions (time 0 s), target presentation (time 4 s), delay (time 8 s), and probe (time 12 s)—and convolved with the empirically derived HRF (Zarahn et al. 1997; Postle, Zarahn et al. 2000). Parameter estimates associated with each covariate of interest were extracted from the least-squares solution of the GLM of the fMRI time series. This approach yielded estimates of delay-period activity that were not contaminated by variance attributable to earlier portions of the trial (Zarahn et al. 1997; Postle, Zarahn et al. 2000). One GLM, the “full” GLM, modeled the data from all trials in this manner, and a second, the “eye position–corrected” GLM, excluded trials that were flagged by the eye position analysis.

Memory-Delay Analysis

This addressed the question how is memory-delay activity (i.e., delay-period activity during no secondary task trials) affected by concurrent distraction? It used the logic of a “functionally constrained search within a factorial design” (Friston et al. 2006) that proceeded in 3 steps. First, whole-brain maps were generated from the full GLM for each single-subject data set with the contrasts (Delaylocation/no secondary task – baseline) and (Delayobject/no secondary task – baseline) and thresholded at the mapwise Bonferroni-corrected α of P < 0.05. (The full GLM was used for these contrasts to maximize sensitivity to identify critical voxels because it modeled all the trials in the condition of interest. This was important to do because each subject only performed 16 trials of each of these types, a relatively small number from which to generate a statistical map in an event-related analysis. The use of the full, rather than the eye position–corrected, GLM did not risk contaminating our results, because it was only used to identify voxels in which our hypotheses would be tested, but not to generate the quantitative estimates that were the bases of the hypothesis tests.) Second, maps were compared across subjects to identify anatomical regions in common (across subjects) that contained memory-delay activity (see Identifying Anatomical Regions, below). Regions identified in the majority (i.e., 5 or more) of the subjects were selected for subsequent analysis, and data from all suprathreshold voxels within each region were pooled to create functional ROIs. For single-subject data sets for which no suprathreshold voxels were identified in a selected region, the threshold was dropped to the regionwise Bonferroni-corrected α of P < 0.05 (for each [Delaydomain/none – baseline] contrast, there were a few single-subject data sets that did not show suprathreshold voxels at the regionwise level for one or more regions. This is not surprising when one considers that each subject performed only 16 trials of any one type. For the [Delaylocation/none – baseline] contrasts, there were 2 single-subject data sets that did not show suprathreshold voxels at the regionwise level for either right or left frontal eye fields [FEF], and so analyses for these ROIs only included 7 subjects. For the [Delayobject/none – baseline] contrast, there were 2 single-subject data sets that did not show suprathreshold voxels at the regionwise level for the left sylvian and 4 for the left superior temporal gyrus [STG] ROIs. Thus, to be able to include observations from all subjects for these 2 ROIs, the threshold for them was dropped to a critical t of 0 [one-tailed]). Third, from these memory-delay voxels the magnitudes of the delay-evoked responses across the 3 levels of secondary task were estimated with the contrasts (Delaydomain/no secondary task – baseline), (Delaydomain/saccades – baseline), and (Delaydomain/passive listening – baseline). These estimates were obtained with the eye position–corrected GLM, to ensure that quantitative estimates of delay-period activity were not contaminated by data from trials on which subjects did not comply with instructions.

Secondary Task Analysis

This addressed the question how are “secondary task–sensitive” voxels (i.e., voxels showing delay-period activity during no memory trials) affected by the demands of a concurrent memory task? It was performed following the same logic as the first analysis, with the exception that the first step used the contrasts (Delayno memory/saccades – baseline) and (Delayno memory/passive listening – baseline) to identify critical voxels.

Identifying Anatomical Regions

Anatomical regions were defined to encompass regions that are known to support different functions. In some cases, these spanned multiple Brodmann areas (BAs) if the functional distinctions between them was judged to be ill defined (as for, e.g., BAs 9 and 46 of middle frontal gyrus [MFG]) or too fine grained to be relevant for the purposes of our study (as for, e.g., BAs 18, 19, and 37 for inferior occipitotemporal cortex [IOTC]). Defining functional ROIs in this way allowed us to aggregate effects from functionally comparable regions across subjects and to perform inferential statistical tests at the group level. Anatomical regions were defined on the native-space anatomical scans of each subject in the following way: MFG, bounded by the inferior frontal sulcus, superior frontal sulcus (SFS), and precentral sulcus (PCS); FEF, beginning from the intersection of the PCS and SFS, the anterior bank of the PCS extending 6 mm lateral to this intersection, and the SFS extending 6 mm anterior to this intersection; supplementary eye fields (SEF), in the medial wall in the upper part of the PCS; STG, bounded by the sylvian fissure and superior temporal sulcus; sylvian, the cortex of the sylvian fissure extending from the frontal operculum to ventral supramarginal gyrus; IOTC, comprising the lingual, fusiform, and inferior temporal gyri and extending to the caudal terminus of the transverse occipital sulcus. Inferior parietal lobule (IPL), inferior and lateral to the intraparietal sulcus (IPS) and superior to the lateral occipital sulcus and sylvian fissure; IPS, the cortex in the banks of the IPS; superior parietal lobule (SPL), dorsal and medial to the IPS, posterior to the postcentral sulcus, and superior to the parietooccipital fissure; V1, the cortex in the banks of the calcarine sulcus; extrastriate cortex, corresponding to BAs 18 and 19 of the occipital lobe. ROIs that spanned the midline (SEF and SPL) were not divided by hemisphere.

Conjunction Analysis

This was performed to determine whether there may be additional areas, not engaged by single-task performance, but recruited in response to interfering secondary task performance. Such activity would be characteristic of some type of executive control, such as a “central executive” (D'Esposito et al. 1995) or an inhibitory control process (Hester et al. 2004), being recruited to manage the coordination of the primary and interfering secondary tasks. The conjunction analysis was designed to identify evidence for domain-general control, that is, voxels recruited by the concurrent performance of location memory and endogenously guided saccades that was dissociable from the nonspecific secondary task effect of concurrent location memory and passive listening, as well as by the concurrent performance of object memory with passive listening that was dissociable from the nonspecific secondary task effect of concurrent object memory with saccades. The analysis was performed as a spatial normalization-based group analysis. First, for each single-subject data set, the contrasts that would isolate the hypothesized “control” voxels were computed: For control of location memory, ([Delaylocation memory/saccades – Delayno memory/saccades] – [Delaylocation memory/passive listening – Delayno memory/passive listening]) and for control of object memory, ([Delayobject memory/passive listening – Delayno memory/passive listening] – [Delayobject memory/saccades – Delayno memory/saccades]). Second, the resultant unthresholded volumes of parameter estimates were warped to a template in MNI space and smoothed to a full-width half-maximum of 8 mm. Third, 2 group-average statistical maps were generated, for control of location memory and for control of object memory, and thresholded at P < 0.005 (uncorrected; degree of freedom = 8). The final step, which constituted the test for conjunction, was to evaluate whether there was any anatomical overlap in these 2 control maps.

Results

Behavioral Results

From the preliminary behavioral study, a 2 × 2 analysis of variance (ANOVA) indicated no main effects of memory domain or of distraction type, but a significant interaction (F1,21 = 24.31, P < 0.0001). Pairwise tests confirmed that this crossover interaction reflected code-specific interference: “Location/saccades” performance was significantly lower than that of location/no secondary task performance (t21 = 2.79, P = 0.01) and that “object/passive listening” performance was significantly lower that “object/no secondary task” performance (t21 = 2.18, P < 0.05) (Fig. 1a,b). Behavioral performance during the fMRI session followed the same trend (Fig. 1c). A 2 × 2 ANOVA of the fMRI behavioral performance also found no main effects of memory or distraction type but did find a significant interaction (F2,16 = 17.20, P < 0.0001). Pairwise tests showed the same trends as seen in the behavioral experiment: location/saccades performance was significantly lower than location/no secondary task performance (t8 = 2.60, P < 0.05), and object/passive listening performance trended toward being lower than object/no secondary task performance (t8 = 1.27, not significant [n.s.]).

Figure 1.

(a) The experimental design. (b) Behavioral results from 22 subjects, revealing a significant code-specific interference. (c) Behavioral results from the fMRI study (n = 9), replicating the interaction.

Figure 1.

(a) The experimental design. (b) Behavioral results from 22 subjects, revealing a significant code-specific interference. (c) Behavioral results from the fMRI study (n = 9), replicating the interaction.

fMRI Results

Data from 2 subjects were excluded from the analyses due to excessive noncompliance with eye movement/eye position instructions, and data from a third subject were excluded because of an error in recording eye position data.

Memory-Delay

Delay-period activity from no-distraction location memory trials was identified reliably across single-subject data sets in FEF, bilaterally, SEF, IPS, bilaterally, SPL, extrastriate cortex, bilaterally, primary visual cortex, and caudate nucleus, bilaterally. The analogous activity from object memory trials was found in MFG, bilaterally, left sylvian cortex, left STG, IPL, bilaterally, IOTC, bilaterally, and primary visual cortex. These patterns of activity, for both stimulus domains, are consistent with prior reports in the literature. The identification of object delay-period activity in the left hemisphere sylvian and STG regions, for example, has precedence in previous studies of working memory for unfamiliar faces (Rama et al. 2001; Rama and Courtney 2005; Roth et al. 2006), colored disks (Imaruoka et al. 2005), and abstract geometric shapes (Gibbs and D'Esposito 2006), to name just a few object stimulus categories. (See Fig. 2 and Table 1 for results from spatially normalized group-average analyses.) The general trend, across both memory domains, was that a concurrent secondary task suppressed the delay-evoked response and that the magnitude of this suppression tended to be greater for concurrent passive listening than for concurrent saccades (Fig. 3). There were marked deviations from this trend, however, in regions commonly associated with the secondary tasks themselves.

Figure 2.

Spatially normalized group-averaged (i.e., “2nd level”) contrasts, using eye position–corrected GLM and displayed on an inflated brain corresponding to the atlas of Talairach and Tournoux (1988).

Figure 2.

Spatially normalized group-averaged (i.e., “2nd level”) contrasts, using eye position–corrected GLM and displayed on an inflated brain corresponding to the atlas of Talairach and Tournoux (1988).

Figure 3.

(a) Location delay-evoked responses, by ROI and by secondary task type: white = none, red = saccades, blue = passive listening. The effect of (Delaylocation/saccades – Delaylocation/none), pooled across all location memory-delay voxels in the 3 frontal oculomotor regions, differed significantly from this effect pooled across location memory-delay voxels from all other ROIs. W.B. = whole brain, C.N. = caudate nucleus, SPL = superior parietal lobule, ES = extrastriate, V1 = primary visual cortex. (b) Object delay-evoked responses. Note that the Delayobject/passive listening is greater than the Delayobject/saccades effect only in left sylvian, left STG, and right IPL ROIs. Furthermore, the effect of (Delayobject/passive listening – Delayobject/none), pooled across the left sylvian and left STG ROIs differed significantly from this effect pooled across object delay-sensitive voxels from all other ROIs (including right IPL). Syl. = sylvian fissure. (c) Saccade-evoked responses, by ROI and by domain of concurrent memory task: white = no concurrent task, orange = location, green = object. (d) Passive listening–evoked responses. (For clarity, only regions that also appear in panel (b) are displayed.)

Figure 3.

(a) Location delay-evoked responses, by ROI and by secondary task type: white = none, red = saccades, blue = passive listening. The effect of (Delaylocation/saccades – Delaylocation/none), pooled across all location memory-delay voxels in the 3 frontal oculomotor regions, differed significantly from this effect pooled across location memory-delay voxels from all other ROIs. W.B. = whole brain, C.N. = caudate nucleus, SPL = superior parietal lobule, ES = extrastriate, V1 = primary visual cortex. (b) Object delay-evoked responses. Note that the Delayobject/passive listening is greater than the Delayobject/saccades effect only in left sylvian, left STG, and right IPL ROIs. Furthermore, the effect of (Delayobject/passive listening – Delayobject/none), pooled across the left sylvian and left STG ROIs differed significantly from this effect pooled across object delay-sensitive voxels from all other ROIs (including right IPL). Syl. = sylvian fissure. (c) Saccade-evoked responses, by ROI and by domain of concurrent memory task: white = no concurrent task, orange = location, green = object. (d) Passive listening–evoked responses. (For clarity, only regions that also appear in panel (b) are displayed.)

On location memory trials, the only regions in which delay-evoked responses were numerically greater with concurrent saccades than with no secondary task were frontal areas associated with oculomotor control: the FEF, bilaterally, and the SEF (Figs 3a and 4a). The delay-evoked response in memory-delay voxels in each of these regions was significantly greater with concurrent saccades than with concurrent passive listening (right FEF t6 = 2.53, P < 0.04; left FEF t6 = 5.71, P < 0.001; and marginally so in SEF t8 = 2.11, P = 0.07). To further investigate this effect, we created an “anterior oculomotor” ROI by combining the FEF and SEF ROIs and compared it against the pooled location memory-delay voxels from all other brain regions. A comparison of the effect of the contrast (Delaylocation/no secondary task – Delaylocation/saccades) in the anterior oculomotor ROI versus the rest of the brain revealed a significant difference (t8 = 4.41, P < 0.005). Finally, we evaluated whether individual differences in the neural interference effect (i.e., [Delaylocation/no secondary task – Delaylocation/saccades]) in these anterior oculomotor regions would predict the magnitude of the behavioral interference effect (location/no secondary task – location/saccades). They did so in right FEF (r = −0.75, P = 0.05) and trended in this direction in left FEF (r = −0.56, n.s.) and SEF (r = −0.44, n.s.).

Figure 4.

(a) Loci of memory-delay responses in the “no-distraction” conditions for location (orange) and object (green) memoranda, in a single subject (number 9). The top left cutout features activity in the FEF and the SEF; the bottom right cutout in the left sylvian fissure. (b) Trial averaged time series from the location memory-delay voxels in panel (a). Gray line along horizontal axis indicates duration of delay period; black line indicates duration of secondary task period. (c) Trial averaged time series from the object memory-delay voxels in panel (a). (d) Group data from frontal oculomotor location memory-delay and left sylvian shape memory-delay ROIs, illustrating a region by secondary task interaction. This neural double dissociation mirrors the behavioral effects and illustrates that interference-specific neural effects are anatomically specific.

Figure 4.

(a) Loci of memory-delay responses in the “no-distraction” conditions for location (orange) and object (green) memoranda, in a single subject (number 9). The top left cutout features activity in the FEF and the SEF; the bottom right cutout in the left sylvian fissure. (b) Trial averaged time series from the location memory-delay voxels in panel (a). Gray line along horizontal axis indicates duration of delay period; black line indicates duration of secondary task period. (c) Trial averaged time series from the object memory-delay voxels in panel (a). (d) Group data from frontal oculomotor location memory-delay and left sylvian shape memory-delay ROIs, illustrating a region by secondary task interaction. This neural double dissociation mirrors the behavioral effects and illustrates that interference-specific neural effects are anatomically specific.

On object memory trials, in general, concurrent passive listening led to lower delay-evoked responses than did concurrent saccades. However, this pattern was reversed in the cortex of the left sylvian fissure and the left STG, both areas associated with auditory/phonological processing of language-based information in working memory (Buchsbaum et al. 2005), as well as in the right IPL, (Figs 3b and 4a). Paired t-tests did not show differential effects of secondary task on object memory delay–evoked activity. We reasoned that such null results may be due, in part, to the limited number of object memory trials remaining after discarding trials in which eye position instructions were not followed. Therefore, to increase our ability to detect more subtle changes, we repeated the analyses within each anatomical ROI, by collapsing across all voxels within the ROI whose (Delayobject/no secondary task – baseline) effect was greater than 0. This analysis showed that object delay-period activity in the left STG was significantly greater with concurrent passive listening than with the saccades secondary task (t8 = 2.75, P < 0.05) and that it trended in this direction in the left sylvian fissure (t8 = 2.08, P = 0.07). Analogous to the analysis of spatial memory-delay activity, we next created an ROI representing these 2 auditory/phonology processing areas by combining the left sylvian and left STG ROIs and comparing it against object memory-delay voxels across the rest of the brain (including right IPL). This indicated that the effects of (Delayobject/no secondary task – Delayobject/passive listening) was reliably different for the auditory/phonological ROI than for object memory-delay voxels in the rest of the brain (t8 = 4.07, P < 0.005). Did individual differences in the neural interference effect (i.e., [Delayobject/no secondary task – Delayobject/passive listening]) in the auditory/phonological ROIs predict the magnitude of the behavioral interference effect of (object/no secondary task – object/passive listening)? They failed to do so for either ROI, although the trends were in the opposite directions for the 2—left STG (r = 0.41, n.s.); left sylvian cortex (r = −0.48, n.s.)—a difference that was, itself, statistically reliable (t8 = 2.56, P < 0.05). This difference fits with recent demonstration that delay-period activity of left STG is associated with acoustic (i.e., echoic) codes, whereas delay-period activity of left sylvian cortex is associated with the retention of phonological codes (Buchsbaum et al. 2005). This brain-behavior trend in left sylvian cortex was also significantly different from that seen in right IOTC (r = 0.19, n.s.; difference of correlations: t8 = −2.02, P < 0.05), a region strongly associated with the object perception component of working memory.

The similarity of the neural-selective interference effects to the behavioral-selective interference effects in the behavioral data was further illustrated with a 2 × 2 ANOVA of these neural effects, which compared location memory in the frontal oculomotor ROI with object memory in the left auditory/phonological ROI. It revealed a main effect of ROI (F1,8 = 10.02, P < 0.05), no main effect of secondary task (F1,8 = 0.34, n.s.), and a crossover interaction of the 2 (F1,8 = 7.67, P < 0.05) (Fig. 4).

“Distraction-Sensitive” Activity

We also evaluated the effects of the retention of different types of memory on voxels associated with processing of each distraction task (i.e., those identified with the contrasts [Delayno memory/saccades – baseline] and [Delayno memory/passive listening – baseline]). Although the distraction tasks activated voxels in many of the same regions as did the memory tasks (see Fig. 3), there was typically considerable nonoverlap at the voxel level. Thus, for example, many of the saccade-sensitive voxels of the FEF were different from the memory-delay voxels of the FEF. This is reflected in the fact that saccade-sensitive voxels in all brain areas decreased their activity with the addition of memory for either stimulus domain, and in no regions was the saccade-evoked effect more sensitive to one memory domain than the other. The same was true of passive listening–sensitive voxels (Fig. 3c,d). Thus, there was no evidence of a neural correlate of code-specific interference in distraction-sensitive ROIs.

Conjunction Analysis

Contrasts were computed to isolate dual-task effects attributable to the pairing of object memory with passive listening and of location memory with saccades. Their results yielded no regions of overlap, thereby failing to find evidence for domain-general control of dual-task performance (an effect that would have been consistent with engagement of a “central executive” [D'Esposito et al. 1995] or an inhibitory control process [Hester et al. 2004]). This null result is consistent with the results of previous studies employing similar experimental procedures (Klingberg 1998; Adcock et al. 2000; Bunge et al. 2000, 2001).

Discussion

The results from this study confirmed that endogenously guided saccades selectively disrupt the short-term retention of stimulus locations and that passive listening to nouns selectively disrupts the short-term retention of stimulus identity. These results are consistent with the multiple-encoding prediction that location working memory is supported by prospective motor codes and that object working memory is supported by verbal codes. The neural correlates of the behavioral double dissociation were also consistent with this interpretation: The neural interference effect—a relative increase of activity localized to regions associated with the secondary task in question—was observed in frontal oculomotor regions in the case of saccadic distraction of location memory; and it was observed in left hemisphere superior temporal and sylvian cortex in the case of passive listening distraction of object memory. We will first consider the logical bases for these interpretations and then consider their implications.

One challenge with secondary task interference studies is to discriminate patterns of interference that identify specific mental codes or processes from those that are nonspecific and that may simply result from divided attention or increased difficulty. This can be particularly troublesome in situations in which the secondary tasks are not carefully matched for difficulty and other factors. For example, is it problematic that one of the secondary tasks in the present study was active (making endogenously generated eye movements), whereas the other was passive? The answer is “no” because of the logic of the double dissociation. To argue that performing the secondary saccadic task was attention-demanding in a general sense, for example, one would need to show that this secondary task had disruptive effects on both memory tasks. Not only was this not the case but also the selective effect of passive listening on the object memory task means that the insensitivity of this latter task to concurrent saccades was not indicative of a general insensitivity to any kind of concurrent task. Rather, one is compelled to conclude that the selective patterns of interference were due to resources shared by the 2 tasks that interfered with one another. In the case of delayed recognition of locations, the fact that the interfering saccade task made no overt demands on sensory processing makes it most likely that the common resource was oculomotor control. The localization of the neural interference effect to frontal regions associated with oculomotor control is also consistent with this interpretation. In the case of delayed recognition of objects, the common resource might be verbal (our preferred interpretation) or it might be visual. According to the visual account, listening to concrete nouns might have evoked visual imagery, and these mental images would be expected to interfere with remembered object representations. There are 4 pieces of evidence, however, that make this unlikely. First, in a different study we have demonstrated an analogous pattern of selective disruption of object working memory with a secondary task requiring the concurrent syntactic analysis of abstract nouns and adjectives (Postle et al. 2005). It is highly unlikely that this secondary task engaged visual imagery. Second, individual differences in the neural interference effect in left sylvian cortex, a region associated with phonological working memory (Buchsbaum et al. 2005), showed a trend toward predicting the behavioral interference effect. The brain-behavior relation in right IOTC, however, was in the opposite direction, and these effects in these 2 ROIs differed significantly from one another. We interpret these results as evidence that although both of these regions were active during the object working memory task, their delay-period activity likely supported different functions, in that they were (significantly) differentially sensitive to concurrent passive listening. Third, although voxels in the right IOTC were strongly active during the object delay periods, they were not differentially sensitive to concurrent passive listening as compared with concurrent saccades. If passive listening engaged imagery, one would expect to see neural correlates of imagery in visual regions such as the right IOTC. Finally, the locus of the Delayobject memory, passive listening effect corresponds to a neural locus of the irrelevant speech effect (ISE) (Gisselgard et al. 2003, 2004), an unambiguously “verbal” effect that will be reviewed in detail further along in this discussion.

Turning now to the implications of these results, we believe that they require extension and/or refinement of current conceptions about working memory at 2 levels. Neurally, they specify a physiological and anatomical specificity of code-specific interference effects: they manifest as increases in delay-evoked signal that are restricted anatomically to memory-delay voxels in regions associated with the secondary tasks. Our results suggest that these code-specific neural interference effects are superimposed on general nonspecific decreases in evoked signal that are more generally characteristic of dual-task versus single-task conditions. Theoretically, they specify processes of recoding and parallel representation (i.e., of multiple encoding) that need to be explicitly accommodated in models of working memory. We will now consider these implications in more detail, first focusing on the neural implications.

The extant literature on the neural correlates of behavioral interference indicates that they are characterized by overlap of the cortical territories associated with performance of the 2 tasks (Klingberg and Roland 1997; Klingberg 1998; Gruber and von Cramon 2003) and/or b) recruitment of “new” control areas (D'Esposito et al. 1995; Hester et al. 2004). Within the “overlap” literature, the experiments of Klingberg and Roland featured nonspecific interference, in that they examined the deleterious effects of having to perform 2 tasks concurrently, but they did not seek to produce code-specific interference of the kind featured in the present study. Their results indicated that one could predict behavioral interference with concurrent task performance if each of the 2 tasks recruited overlapping cortical territories (Klingberg and Roland 1997; Klingberg 1998) and that signal intensity within the region of overlap was unchanged (it was elevated for dual- vs. single-task performance, but not significantly so [Klingberg, 1998]) during dual-task performance with respect to single-task performance (Klingberg 1998). These are broadly consistent with what we report here.

The study of Gruber and von Cramon (2003), in contrast to those of Klingberg but in common with the present study, did employ the logic of domain-specific interference. It differed from the present study, however, in that its secondary tasks drew on the same sensory modalities and the same informational domains as did the primary tasks with which they were paired—articulatory suppression with delayed letter recognition and visuospatial suppression with delayed visuospatial recognition. The results of this study also differ from what we report here. Gruber and von Cramon (2003) reported the neural correlates of domain-specific interference to be marked reductions in activity in the regions of overlap between primary and secondary task, with no appreciable effect of the pairing of the noninterfering secondary task on primary task performance (e.g., no effect of concurrent articulatory suppression on visuospatial delayed recognition). The present results, in contrast, show code-specific interference effects to be increases in the evoked response that are restricted to memory-delay voxels in regions associated with the secondary task, whereas nonspecific effects are in the opposite direction—a decrease in evoked signal in every brain area engaged by the task, regardless of overlap or nonoverlap of the 2 tasks in isolation. (Note that the code-specific increase is presumably superimposed on the nonspecific decrease, and so the value of the dual-task–evoked response might fall above [as it does for Delaylocation/saccades in frontal oculomotor control regions] or below [as it does for Delayobject/passive listening in left perisylvian regions] the single-task–evoked response.) At a general level, the 2 studies are compatible in that they each implicate cortical overlap and, by inference, competition for neural resources, as the physiological basis for domain-specific interference. At a more detailed level of description, however, they differ, in that the Gruber and von Cramon (2003) results suggest that this competition produces a decrease in activity, whereas the present results suggest that it produces an increase. It may be that this discrepancy reflects the fact that the competition produced by the Gruber and von Cramon task could have occurred at multiple levels—transduction from physical energy into a neural code, perceptual processing, and short-term retention—whereas the competition produced by our task could only occur at the level of the short-term retention of target-related information. Indeed, we interpret the anatomical specificity of the neural interference effects in the present study as further evidence for multiple encoding, in that it reinforces the behavioral evidence for a contribution of oculomotor control to the short-term retention of visuospatial information and of verbal processing to the short-term retention of visuoobject information.

Another relevant comparison for the present study is neuroimaging studies of the ISE, which occurs when irrelevant sounds (including speech) are presented concurrent with or subsequent to presentation of to-be-remembered linguistic information (e.g., digits, letters, words). Although the precise factors underlying the ISE remain a matter of debate, there is general consensus that it is brought about by interference between mnemonic representations and distractors at the level of the phonological representation of the memoranda (Repovs and Baddeley 2006). Thus, our interpretation of the interfering effect of passive listening on object delayed recognition can be cast as an ISE account. A previous study of the ISE using positron emission tomography showed it to be associated with a general decrease in all regions recruited by a verbal working memory task, an effect that was particularly salient in a region spanning STG and sylvian cortex, bilaterally (Gisselgard et al. 2003). A follow-up study by the same group added memory load as an experimental factor, and found that the neural correlate of the ISE in left STG varied with memory load, with a relative increase in signal in this region in the high load condition (Gisselgard et al. 2004). The centrality of the perisylvian cortex in these studies of the ISE in verbal working memory provides an independent corroboration of the face validity of our interpretation of the code-specific passive listening effecting our study: listening to words interfered with phonological representations of the object memoranda.

Theoretically, the present results highlight the importance of the principle of multiple encoding in working memory. For the short-term retention of location information, they provide independent behavioral evidence that the location delay-period activity observed in many studies (and, indeed, in many species) in the FEF (e.g., Postle, Berger et al. 2000; Rowe et al. 2000; Takeda and Funahashi 2002, 2004; Balan and Ferrera 2003a, 2003b; Curtis et al. 2004) reflects, at least in part, a motoric code. For the short-term retention of object information, they offer 2 novel insights. The first is that the behavioral results confirm that the interference produced by a secondary verbal task occurs at a nonvisual level of representation. That is, the interference produced by the secondary verbal task was likely verbal, not visual, because the nouns were presented auditorily. (And we have already reviewed reasons to discount an imagery explanation of the passive listening effect.) Second, the fact that the critical anatomical locus of code-specific interference was left sylvian cortex, a region previously implicated in phonological representation in working memory (Buchsbaum et al. 2005), further suggests that one representational locus of this interference is articulatory/phonological. (Whether there may also be a semantic component to this interference effect will require additional research.) We have also seen that this analysis corresponds well with neuroimaging correlates of the ISE in verbal working memory (Gisselgard et al. 2003, 2004).

How do these results and, more broadly, the principle of multiple encoding relate to current models of working memory? We will consider this question via the multiple-component model, unquestionably the most influential in contemporary cognitive psychology and cognitive neuroscience. The multiple-component model is a memory systems model, in that it posits specialized mechanisms whose principle function is to support a kind of memory. For example, it posits the existence of an “inner scribe” that represents locations and the paths between locations, a “visual cache” that represents visual object identity, and a “phonological loop” comprising a “phonological store” and an articulatory rehearsal process (Logie 1995, 2003; Baddeley and Logie 1999; Repovs and Baddeley 2006). From this perspective, one could portray the present results as adding mechanistic and neural specificity to the multiple-component model. To the inner scribe portion of the model, it specifies a mechanism—prospective oculomotor coding—and a neural substrate—the FEF. It can also be noted that a mechanism of prospective motor coding recruited in the service of spatial working memory can be portrayed as a process that is analogous to the manner in which visually presented verbal material is recoded into a phonological/articulatory code and retained in memory as a speech-based motor code (e.g., Page and Norris 1998). To pursue the analogy, our saccadic distraction task might be viewed as analogous to articulatory suppression as an effective distractor of verbal working memory. Turning now to the short-term retention of object information, the present results can be interpreted to suggest that the function of the visual cache can be supplemented by the phonological loop, provided that the memoranda afford verbal recoding. This verbal recoding is hypothesized to be mediated via the association of visual stimuli with long-term knowledge, which, in turn, activates a lexical representation.

An alternative to this memory systems analysis, however, is to view the short-term retention of information as a property of each of the systems engaged by this task—oculomotor control, visual perception, and speech production. From this “emergent processes” perspective (Postle 2006), what the present study has illustrated is ways in which nominally “nonmemory” systems (e.g., perceptual systems, motor systems, language systems) accomplish the short-term retention information in the service of behavioral goals.

We thank Olufunsho Faseyitan, Jared Hotaling, Christopher Jordan, Jamie Lapin, and George McPhail for assistance with programming, data collection, and analyses. Supported by National Institutes of Health MH064498 to B.R.P. Conflict of Interest: None declared.

References

Adcock
RA
Constable
RT
Gore
JC
Goldman-Rakic
P
Functional neuroanatomy of executive processes involved in dual-task performance
Proc Natl Acad Sci USA
 , 
2000
, vol. 
97
 (pg. 
3567
-
3572
)
Aguirre
GK
Zarahn
E
D'Esposito
M
The variability of human, BOLD hemodynamic responses
Neuroimage
 , 
1998
, vol. 
8
 (pg. 
360
-
369
)
Attneave
F
Arnoult
MD
Methodological considerations in the quantitative study of shape and pattern perception
Psychol Bull
 , 
1956
, vol. 
53
 (pg. 
221
-
227
)
Awh
E
Jonides
J
Overlapping mechanisms of attention and spatial working memory
Trends Cogn Sci
 , 
2001
, vol. 
5
 (pg. 
119
-
126
)
Baddeley
AD
Working memory
1986
London
Oxford University Press
Baddeley
AD
Hitch
GJ
Bower
GH
Working memory
The psychology of learning and motivation
 , 
1974
New York
Academic Press
(pg. 
47
-
89
)
Baddeley
AD
Logie
RH
Miyake
A
Shah
P
Working memory: the multiple-component model
Models of working memory
 , 
1999
Cambridge (UK)
Cambridge University Press
(pg. 
28
-
61
)
Balan
PF
Ferrera
VP
Effects of gaze shifts on maintenance of spatial memory in macaque frontal eye field
J Neurosci
 , 
2003
, vol. 
23
 (pg. 
5446
-
5454
)
Balan
PF
Ferrera
VP
Effects of spontaneous eye movements on spatial memory in macaque periarcuate cortex
J Neurosci
 , 
2003
, vol. 
23
 (pg. 
11392
-
11401
)
Boynton
GM
Engel
SA
Glover
GH
Heeger
DJ
Linear systems analysis of functional magnetic resonance imaging in human V1
J Neurosci
 , 
1996
, vol. 
16
 (pg. 
4207
-
4221
)
Buchsbaum
BR
Olsen
RK
Koch
P
Berman
KF
Human dorsal and ventral auditory streams subserve rehearsal-based and echoic processes during verbal working memory
Neuron
 , 
2005
, vol. 
48
 (pg. 
687
-
697
)
Bunge
SA
Klingberg
T
Jacobsen
RB
Gabrieli
JDE
A resource model of the neural basis of executive working memory
Proc Natl Acad Sci USA
 , 
2000
, vol. 
97
 (pg. 
3573
-
3578
)
Bunge
SA
Ochsner
KN
Desmond
JE
Glover
GH
Gabrieli
JDE
Prefrontal regions involved in keeping information in and out of mind
Brain
 , 
2001
, vol. 
124
 (pg. 
2074
-
2086
)
Curtis
CE
Rao
VY
D'Esposito
M
Maintenance of spatial and motor codes during oculomotor delayed response tasks
J Neurosci
 , 
2004
, vol. 
24
 (pg. 
3944
-
3952
)
D'Esposito
M
Detre
JA
Alsop
DC
Shin
RK
Atlas
S
Grossman
M
The neural basis of the central executive system of working memory
Nature
 , 
1995
, vol. 
378
 (pg. 
279
-
281
)
Della Sala
S
Gray
C
Baddeley
A
Allamano
N
Wilson
L
Pattern span: a tool for unwelding visuo-spatial memory
Neuropsychologia
 , 
1999
, vol. 
37
 (pg. 
1189
-
1199
)
Friston
KJ
Holmes
AP
Worsley
KJ
Poline
JP
Frith
CD
Frackowiak
RSJ
Statistical parametric maps in functional imaging: a general linear approach
Human Brain Mapp
 , 
1995
, vol. 
2
 (pg. 
189
-
210
)
Friston
KJ
Rotshtein
P
Geng
JJ
Sterzer
P
Henson
RN
A critique of functional localisers
Neuroimage
 , 
2006
, vol. 
30
 (pg. 
1077
-
1087
)
Fuster
JM
Memory in the cerebral cortex
1995
Cambridge (MA)
MIT Press
Gibbs
SEB
D'Esposito
M
A functional magnetic resonance imaging study of the effects of pergolide, a dopamine receptor agonist, on component processes of working memory
Neuroscience
 , 
2006
, vol. 
139
 (pg. 
359
-
371
)
Gisselgard
J
Petersson
KM
Baddeley
AD
Ingvar
M
The irrelevant speech effect: a PET study
Neuropsychologia
 , 
2003
, vol. 
41
 (pg. 
1899
-
1911
)
Gisselgard
J
Petersson
KM
Ingvar
M
The irrelevant speech effect and working memory load
Neuroimage
 , 
2004
, vol. 
22
 (pg. 
1107
-
1116
)
Gitelman
DR
ILAB: a program for post experimental eye movement analysis
Behav Res Methods Instrum Comput
 , 
2002
, vol. 
34
 (pg. 
605
-
612
)
Gruber
O
von Cramon
DY
The functional neuroanatomy of human working memory revisited: Evidence from 3-T fMRI studies using classical domain-specific interference tasks
Neuroimage
 , 
2003
, vol. 
19
 (pg. 
797
-
809
)
Handwerker
DA
Ollinger
JM
D'Esposito
M
Variation of BOLD hemodynamic responses across subjects and brain regions and their effects on statistical analyses
Neuroimage
 , 
2004
, vol. 
21
 (pg. 
1639
-
1651
)
Hester
R
Murphy
K
Garavan
H
Beyond common resources: the cortical basis for resolving task interference
Neuroimage
 , 
2004
, vol. 
23
 (pg. 
202
-
212
)
Imaruoka
T
Saiki
J
Miyauchi
S
Maintaining coherence of dynamic objects requires coordination of neural systems extended from anterior frontal to posterior parietal brain cortices
Neuroimage
 , 
2005
, vol. 
26
 (pg. 
227
-
284
)
Klingberg
T
Concurrent performance of two working memory tasks: potential mechanisms of interference
Cereb Cortex
 , 
1998
, vol. 
8
 (pg. 
593
-
601
)
Klingberg
T
Roland
PE
Interference between two concurrent tasks is associated with activation of overlapping fields in the cortex
Cogn Brain Res
 , 
1997
, vol. 
6
 (pg. 
1
-
8
)
Kwong
KK
Belliveau
JW
Chesler
DA
Goldberg
IE
Weisskoff
RM
Poncelet
BP
Kennedy
DN
Hoppel
BE
Cohen
MS
Turner
R
, et al.  . 
Dynamic magnetic resonance imaging of human brain activity during primary sensory stimulation
Proc Natl Acad Sci USA
 , 
1992
, vol. 
89
 (pg. 
5675
-
5679
)
Logie
RH
Visuo-spatial working memory
1995
Hove (UK)
Erlbaum
Logie
RH
Irwin
D
Ross
B
Spatial and visual working memory: a mental workspace
Cognitive vision: the psychology of learning and motivation
 , 
2003
San Diego (CA)
Elsevier Science
(pg. 
37
-
78
)
Ogawa
S
Tank
DW
Menon
R
Ellermann
JM
Kim
SG
Merkle
H
Ugurbil
K
Intrinsic signal changes accompanying sensory stimulation: functional brain mapping using MRI
Proc Natl Acad Sci USA
 , 
1992
, vol. 
89
 (pg. 
5951
-
5955
)
Page
MPA
Norris
D
The primacy model: a new model of immediate serial recall
Psychol Rev
 , 
1998
, vol. 
105
 (pg. 
761
-
781
)
Postle
BR
Working memory as an emergent property of the mind and brain
Neuroscience
 , 
2006
, vol. 
139
 (pg. 
23
-
38
)
Postle
BR
Berger
JS
Taich
AM
D'Esposito
M
Activity in human frontal cortex associated with spatial working memory and saccadic behavior
J Cogn Neurosci
 , 
2000
, vol. 
12
 
Suppl 2
(pg. 
2
-
14
)
Postle
BR
D'Esposito
M
“What”—then—“where” in visual working memory: an event-related fMRI study
J Cogn Neurosci
 , 
1999
, vol. 
11
 (pg. 
585
-
597
)
Postle
BR
D'Esposito
M
Spatial working memory activity of the caudate nucleus is sensitive to frame of reference
Cogn Affect Behav Neurosci
 , 
2003
, vol. 
3
 (pg. 
133
-
144
)
Postle
BR
D'Esposito
M
Corkin
S
Effects of verbal and nonverbal interference on spatial and object visual working memory
Mem Cognit
 , 
2005
, vol. 
33
 (pg. 
203
-
212
)
Postle
BR
Druzgal
TJ
D'Esposito
M
Seeking the neural substrates of working memory storage
Cortex
 , 
2003
, vol. 
39
 (pg. 
927
-
946
)
Postle
BR
Idzikowski
C
Della Salla
S
Logie
RH
Baddeley
AD
The selective disruption of spatial working memory by eye movements
Q J Exp Psychol
 , 
2006
, vol. 
59
 (pg. 
100
-
120
)
Postle
BR
Zarahn
E
D'Esposito
M
Using event-related fMRI to assess delay-period activity during performance of spatial and nonspatial working memory tasks
Brain Res Protoc
 , 
2000
, vol. 
5
 (pg. 
57
-
66
)
Rama
P
Courtney
SM
Functional topography of working memory for face or voice identity
Neuroimage
 , 
2005
, vol. 
24
 (pg. 
224
-
234
)
Rama
P
Sala
JB
Gillen
JS
Pek
JJ
Courtney
SM
Dissociation of the neural systems for working memory maintenance of verbal and nonverbal visual information
Cogn Affect Behav Neurosci
 , 
2001
, vol. 
1
 (pg. 
161
-
171
)
Ranganath
C
Working memory for visual objects: complementary roles of inferior temporal, medial temporal, and prefrontal cortex
Neuroscience
 , 
2006
, vol. 
139
 (pg. 
277
-
289
)
Repovs
G
Baddeley
AD
The multi-component model of working memory: explorations in experimental cognitive psychology
Neuroscience
 , 
2006
, vol. 
139
 (pg. 
5
-
21
)
Roth
JK
Serences
JT
Courtney
SM
Neural system for controlling the contents of object working memory in humans
Cereb Cortex
 , 
2006
, vol. 
16
 (pg. 
1595
-
1603
)
Rowe
JB
Toni
I
Josephs
O
Frackowiak
RSJ
Passingham
RE
The prefrontal cortex: response selection or maintenance within working memory?
Science
 , 
2000
, vol. 
288
 (pg. 
1656
-
1660
)
Smith
EE
Jonides
J
Koeppe
RA
Awh
E
Schumacher
EH
Minoshima
S
Spatial vs. object working memory: PET investigations
J Cogn Neurosci
 , 
1995
, vol. 
7
 (pg. 
337
-
356
)
Takeda
K
Funahashi
S
Prefrontal task-related activity representing visual cue location or saccade direction in spatial working memory tasks
J Neurophysiol
 , 
2002
, vol. 
87
 (pg. 
567
-
588
)
Takeda
K
Funahashi
S
Population vector analysis of primate prefrontal activity during spatial working memory
Cereb Cortex
 , 
2004
, vol. 
14
 (pg. 
1328
-
1339
)
Talairach
J
Tournoux
P
Co-planer stereotaxic atlas of the human brain
1988
New York
Thieme Medical Publishers
Tresch
MC
Sinnamon
HM
Seamon
JG
Double dissociation of spatial and object visual memory: evidence from selective interference in intact human subjects
Neuropsychologia
 , 
1993
, vol. 
31
 (pg. 
211
-
219
)
Vanderplas
JM
Garvin
EA
The association value of random shapes
J Exp Psychol
 , 
1959
, vol. 
57
 (pg. 
147
-
163
)
Wickens
DD
Some characteristics of word encoding
Mem Cognit
 , 
1973
, vol. 
1
 (pg. 
485
-
490
)
Worsley
KJ
Friston
KJ
Analysis of fMRI time-series revisited–again
Neuroimage
 , 
1995
, vol. 
2
 (pg. 
173
-
182
)
Zarahn
E
Aguirre
GK
D'Esposito
M
A trial-based experimental design for fMRI
Neuroimage
 , 
1997
, vol. 
6
 (pg. 
122
-
138
)