Abstract

It has recently been shown, using functional magnetic resonance imaging with a change detection paradigm, that activity in posterior parietal cortex (PPC) correlates with the limited number of objects held in visual short-term memory (VSTM). We replicate this finding and extend it to tasks that use similar stimuli, but without explicit memory requirements. As well as a perceptual task used previously (detecting an item at fixation), 2 additional tasks were designed to increase attentional demands across space (searching for a red item anywhere in the array) and across both space and time (detecting a staggered offset after prolonged viewing of the array). During the VSTM task, a capacity-limited set-size effect was seen in PPC as well as occipital and frontal regions. However, the PPC showed similar activity during 2 of the tasks not requiring VSTM. These findings cannot easily be explained by behavioral performance measures or memory demands alone, suggesting a role of the PPC in processing a limited number of discrete object representations, whether in the current perceptual scene or working memory. The differential influence of item load across perceptual tasks is consistent with task requirements affecting the form of these representations.

Introduction

Visual short-term memory (VSTM) is the ability to maintain a small amount of visual information for a period of seconds, in contrast to sensory or “iconic” memory (IM) which stores a richer, but more fragile, representation that decays within half a second and is more sensitive to eye movements and intervening distracters (Phillips 1974). VSTM has a capacity limit of approximately 4 items (Cowan 2001), although this varies somewhat between individuals (Vogel and Machizawa 2004; Todd and Marois 2005), and depends on the particular task and objects being remembered (Alvarez and Cavanagh 2004; Xu and Chun 2006; Awh et al. 2007). The number of items held in working memory (K) can be estimated in a change detection task using a formula introduced by Pashler (1988) and modified by Cowan (2001) (see Methods). Although this measure is typically interpreted as K items being perfectly encoded, with no encoding of any other items, it is possible that a larger number of lower fidelity representations are being stored (Wilken and Ma 2004). It might therefore be useful to consider K as the number of “object's-worth” of information that can be maintained (Vogel et al. 2006).

There is still debate as to the nature of the representations that are stored in working memory. It has been demonstrated that whole, multifeature objects can be successfully encoded almost as well as single feature objects, but that performance declines if the same number of features are distributed amongst multiple objects (Luck and Vogel 1997). This suggests that VSTM is limited by the number of perceptual objects rather than by simple features, and is consistent with object effects seen in numerous other situations of which a small selection include selective attention to overlapping stimuli (Duncan 1984; O'Craven et al. 1999), multiple-object tracking (MOT; Yantis 1992; Scholl et al. 2001), inhibition of return (Paul and Tipper 2003), saccade planning (Vergilino-Perez and Findlay 2006), and target detection in patients with unilateral neglect (Tipper and Behrmann 1996; Humphreys 1998). Such object effects may be mediated by constructs such as object files, which store or index information about discrete spatiotemporally continuous entities (Kahneman et al. 1992). However, it has been variously suggested that VSTM also stores relational information (Jiang et al. 2000; Jiang et al. 2004), is limited by both the number of objects and their feature complexity (Olson and Jiang 2002; Alvarez and Cavanagh 2004; Xu and Chun 2006; but see Awh et al. 2007), or is limited by object parts and feature conjunctions rather than the number of objects per se (Davis and Holmes 2005). Because most working memory tasks involve multiple component process, both object-based and feature-based processing may occur, but for different components of the task. For example, an electroencephalography study which varied both object load and feature load suggested that encoding and retention are object based, whereas retrieval of the object to compare with a probe must be done on a feature-by-feature basis (Busch and Herrmann 2003).

Although the frontal lobe has long been associated with working memory processes, a number of studies suggest that frontal regions contribute more to strategic or goal-based components, whereas the parietal lobe may be more important for maintaining the limited contents of VSTM (Ferreira et al. 1998; Pochon et al. 2001; Bor et al. 2003; Linden et al. 2003; Owen 2004; Todd and Marois 2004; Vogel and Machizawa 2004; Xu and Chun 2006). It was recently demonstrated using functional magnetic resonance imaging (fMRI) that the activity of a bilateral region in human posterior parietal cortex (PPC) correlates with the number of objects encoded in VSTM (Todd and Marois 2004). The authors took advantage of the capacity limit by estimating K over a range of set-sizes (from 1 to 8 items) during a delayed visual match-to-sample task, and identifying brain regions whose activity correlated with this behavioral measure. A bilateral PPC region near the intersection of the intraparietal sulci (IPS) and intraoccipital sulci (IOS) showed such an effect, with the blood-oxygen-level–dependent (BOLD) signal increasing from set-sizes 1 to 4, but leveling off thereafter. In a series of experiments, it was shown that this effect generalized to other types of visual stimuli, was caused by neither hemodynamic saturation nor insufficient time to encode the items, and was unlikely to be explained by task difficulty (task difficulty increased linearly with set-size, but the BOLD activity was explained significantly better by a quadratic function than by a linear function). It thus seems that PPC is involved in the storage and/or processing of the limited number of mental objects that can be represented in VSTM. This suggestion is bolstered by observations that differences in the BOLD activity of this region across different subjects also correlate with individual differences in their short-term memory capacity (Todd and Marois 2005). A subsequent study (Xu and Chun 2006) extended this finding to show that a superior part of the IPS and the lateral occipital complex track the number of objects stored, largely independent of spatial location, but also limited by the complexity of the objects. In contrast, a more inferior region of the IPS reached asymptote around set-size 4 regardless of object complexity, but only when the objects occupied different spatial locations.

Electrophysiological measures provide converging evidence of limited capacity neural activity during VSTM tasks. For example, interhemispheric difference waveforms, such as the “contralateral delay activity” to an attended memory array, asymptote as VSTM capacity is reached, and also correlate with individual differences in VSTM capacity. This effect was found to predominate over posterior parietal and lateral occipital electrodes, consistent with the fMRI data (Vogel and Machizawa 2004).

Todd and Marois also conducted a control iconic memory (IM) experiment which was identical to their VSTM experiment, except that only the sample display was presented and rather than performing a memory task, the subjects simply had to report the presence or absence of an item in the center of the display. It was found that the BOLD signal in IPS/IOS was attenuated and was no longer significantly affected by set-size. It was therefore concluded that “the IPS/IOS is insensitive to the perceptual load of the visual scene” (Todd and Marois 2004). However, various other studies have explicitly or implicitly suggested that activity in similar posterior parietal regions is modulated by the number of perceived objects in the absence of any obvious working memory requirements beyond the time span of sensory memory. Examples can be found from across a range of different paradigms and even sensory modalities, including judgments of numerosity (Piazza et al. 2004), the perception of ambiguous auditory stimuli (Cusack 2005), and the perception of visual objects defined by illusory contours (Ritzl et al. 2003). It has been argued that “the short-term retention of information is supported by sustained activity in the same brain regions that are responsible for the processing and representation of this information in nonworking memory situations, such as perception” (Postle 2006). If this view is correct, then it might be expected that areas sensitive to the number of items held in working memory would also be sensitive to the number of items processed in nonworking memory situations and vice versa. Furthermore, the capacity limitations, which are so evident in working memory paradigms, may reflect constraints on the processing of underlying perceptual representations.

Whether the IPS shows sensitivity to perceptual load may depend on whether the items concerned are being attended. For example, the appearance of new objects leads to an increase in IPS activity, even if they are task irrelevant, but only when they are being attended (Hon et al. 2006). Several studies have shown increased IPS activity as the number of attended objects increases, using a multiple-object tracking task (Culham et al. 2001; Jovicich et al. 2001; Tomasi et al. 2004). Thus, no effect of perceptual load may be seen in the IM task if only the central location is attended. This requirement for attention would be consistent with proposals that VSTM capacity is at least partly limited by how much information can reside at one time in the focus of attention (Cowan 2001; Fougnie and Marois 2006) and that the role of the IPS in VSTM is likely to be attention-based (Marois and Ivanoff 2005). On the other hand, parietal sensitivity to presented objects does not seem to require conscious awareness of their presence (Fang and He 2005).

Although many of the imaging studies mentioned above have not focused on capacity limits, it has been frequently demonstrated that our capacity for perception and parallel deployment of attention can be severely limited in tasks where working memory, at least as commonly conceived, does not play an obvious role. For example, the number of orientation-defined transparent-motion dot surfaces that can be simultaneously perceived is approximately 2. This limit can be raised by increasing the salience of the motion signals, however, there still appears to be a higher-order upper limit of 3 surfaces which is insensitive to manipulations of signal strength (Greenwood and Edwards 2006). The number of independently moving objects that can be simultaneously tracked is typically reported as being limited to approximately 3–5 in a single plane (Pylyshyn and Storm 1988; Culham et al. 2001; Jovicich et al. 2001; Tomasi et al. 2004), although this may be increased to as many as 7 when the targets and distracters are distributed across multiple depth planes (Viswanathan and Mingolla 2002). When people are asked to report the numerosity of a set of briefly presented objects, they are accurate and confident for sets of below about 3–6 items and reaction times increase gradually with set-size. As set-sizes increase beyond this range, all 3 functions show apparent discontinuities, with accuracy and confidence dropping and an accompanying sudden increase in reaction times. The different methods of judgment either side of this discontinuity are termed “subitizing” and “estimating,” respectively (Kaufman and Lord 1949; Trick and Pylyshyn 1994). Even at only 22 weeks of age, a similar limit on the sensitivity to object number is apparent, with infants being able to discriminate 2 from 3, but not 4 from 6 (Starkey and Cooper 1980). Illusory motion of a line away from a location, which has been cued immediately before (Hikosaka et al. 1993), has also been used to show that multiple items can be precued in parallel, but only up to a limit of approximately 3–7 (Schmidt et al. 1998). The origin of the capacity limits in these tasks is often viewed as attentional or preattentive rather than memory based and may share a common mechanism (Trick and Pylyshyn 1994; Pylyshyn 2001). In summary, therefore, there is evidence for a parallel, capacity-limited process in early visual perception and the capacity limit is found to be in a similar range across various paradigms and similar to the capacity limit observed in working memory tasks.

The purpose of the current study was to explore how the modulation of BOLD activity by visual object load depends on the specific task requirements, and whether such sensitivity in perceptual tasks reflects the same capacity limitations apparent in working memory situations. To this end, we aimed to replicate the findings of Todd and Marois (2004) described above, and to add 2 further perceptual controls, one which forced the subjects to attend to all of the items presented, rather than just the central position (responding “yes” or “no” depending on whether or not a red disc was present), and another which encouraged the subjects to attend to the items for a period of time comparable with the memory delay in the original condition (responding “yes” or “no” at the offset of the display, depending on whether one of the discs faded away over 100 ms rather than vanishing instantaneously). It was hypothesized that if the regions whose activity correlates with the limited number of items stored in VSTM are insensitive to the perceptual load of the visual scene per se, then their activity should be unrelated to the number of presented items in any of the perceptual tasks. On the other hand, if they are sensitive to the number of objects being attended, then set-size–related activity might be revealed in the additional control tasks by increasing the attentional demands across space and time. If any such activity has no capacity limit (or a limit higher than the maximum set-size of 8 items used here), then it might be expected to increase linearly with set-size, whereas if it has a capacity limit reflecting that of VSTM then it should be preferentially revealed by an asymptotic regressor derived from the behavioral K function.

Materials and Methods

Participants

Sixteen healthy subjects took part (aged 18–36 years, mean 22.8; 10 female; the first author and 15 naïve volunteers). All reported normal color vision and visual acuity, gave their informed, written consent, and were reimbursed for their time. The behavioral data of all subjects were analyzed, although 2 subjects were discarded prior to statistical analysis of the fMRI data, due to excessive movement during the acquisitions (frequently more than 3 mm). The behavioral data presented are from the remaining 14 subjects only.

Procedure

The task and stimulus presentation (Fig. 1) for the VSTM and IM conditions were as close as possible to those used by Todd and Marois (2004), although to reduce scanning time only 3 set-sizes (1, 4, and 8) were used and the null events were replaced by variable duration intertrial intervals (ITI). In all conditions, 2 spoken digits were initially presented serially over headphones, for 250 ms each, followed by a silent period and auditory mask of similar durations. These were to be retained for the duration of the trial, in order to occupy auditory/verbal working memory systems and so prevent verbal recoding/rehearsal of the visually presented discs (Baddeley 1986; Todd and Marois 2004). A fixation point was then presented alone for 1500 ms and then replaced by an array of 1, 4, or 8 different colored discs (each 0.38° in diameter). The discs were selected from a palette of 10 colors (red, orange, yellow, light green, dark green, light blue, dark blue, pink, white, and black), randomly positioned in a 3-by-3 matrix (1.38° × 1.38°) and centered at fixation. In the VSTM task, this sample display lasted for 150 ms, then, after a retention interval of 1200 ms, a single probe disc reappeared and subjects were asked to judge whether or not a disc of this color had occupied the same position in the original display. Probes were always presented at an occupied position in the sample display and matched the sample color on a random 50% of trials. For nonmatch trials at the larger set-sizes, the probe color had a 50% chance of having occurred elsewhere in the sample display. The sequence was repeated 96 times, 32 for each set-size, with the set-sizes being randomly intermixed. Working memory load was estimated for each set-size using Cowan's formula: K = N(HF), where N is the set-size, H is the hit rate, and F is the false alarm rate (Cowan 2001). Sensitivity (d′) was estimated using signal detection theory, as a measure of performance independent of any response bias that might vary across the different set-sizes.

Figure 1.

Schematic of the stimulus presentation sequence. Each trial began with a visual “ready” prompt, then 2 auditory digits followed by a fixation point on a mid-gray background. A sample display of 1, 4, or 8 differently colored discs appeared for 150 ms in the VSTM, IM, and ESA tasks and for 750–1350 ms in the ESTA task. In the VSTM task the sample was followed by a 1200-ms retention interval, probe disc and response interval, whereas in the other tasks the response interval followed directly from the offset of the sample display. All trials ended with 2 visually presented digits, a second response and a variable duration ITI.

Figure 1.

Schematic of the stimulus presentation sequence. Each trial began with a visual “ready” prompt, then 2 auditory digits followed by a fixation point on a mid-gray background. A sample display of 1, 4, or 8 differently colored discs appeared for 150 ms in the VSTM, IM, and ESA tasks and for 750–1350 ms in the ESTA task. In the VSTM task the sample was followed by a 1200-ms retention interval, probe disc and response interval, whereas in the other tasks the response interval followed directly from the offset of the sample display. All trials ended with 2 visually presented digits, a second response and a variable duration ITI.

The IM condition was identical to the VSTM condition except that the probe disc did not appear and the subjects responded immediately to the sample display according to whether or not a disc was present at the fixation position.

The “Extended Spatial Attention” (ESA) condition was identical to the IM condition except that subjects had to indicate whether a red disc was present anywhere in the display.

The “Extended Spatial and Temporal Attention” (ESTA) condition was identical to the IM and ESA conditions except that the sample display was presented for between 750 and 1350 ms and the subjects had to decide whether one of the discs faded away over 100 ms prior to the offset of the whole array. The exact offset time was jittered within the 600 ms range to encourage subjects to maintain attention throughout.

Finally, at the end of each trial, in all conditions, a pair of digits was presented visually and the subjects had to indicate whether or not this was the same pair as presented at the beginning of the trial. The 4 conditions were run as separate blocks, the order counterbalanced across subjects, with the constraint that the VSTM and IM tasks were adjacent, as were the ESA and ESTA tasks. Subjects received practice at all 4 tasks prior to entering the scanner, until they and the experimenter were satisfied that they understood the instructions. Subjects were asked to respond as quickly and as accurately as possible in all conditions, although accuracy was emphasized. Yes/no responses were made with the index fingers of each hand using a button box. Response mapping was counterbalanced across subjects.

fMRI Data Acquisition and Analysis

Volunteers were scanned at the MRC Cognition and Brain Sciences Unit, Cambridge, UK, on a Siemens (Munich, Germany) TIM Trio 3T machine. The tasks were performed during 4 separate sessions of echoplanar imaging with 439 volumes acquired in each (including 12 s of initial dummy scans which were discarded to allow for equilibration of the signal). Acquisition parameters were as follows: time repetition = time acquisition = 2 s; time echo = 30 ms; flip angle = 78°; 32, 3-mm thick, interleaved slices of matrix size 64 × 64 with a 25% gap; approximate voxel size 3 × 3 × 3.75 mm. Visual Basic 6, running under Windows XP, was used to create and present the stimuli as well as to collect the responses. Visual stimuli were projected onto a screen using a Christie video projector with a 60-Hz refresh rate, and viewed using an angled mirror positioned approximately 90 mm above the subjects’ eyes. Auditory stimuli were presented binaurally over Siemens tube headphones at a volume comfortable for the volunteer.

Functional data were preprocessed and analyzed in a standard manner using the SPM5 software package (Wellcome Department of Imaging Neuroscience, London, UK; http://www.fil.ion.ucl.ac.uk/spm). Preprocessing included slice-time correction, realignment to a reference image, nonlinear normalization to the Montreal Neurological Institute (MNI) template brain, and spatial filtering with a 10-mm full-width half-maximum Gaussian kernel. This was automated using in-house software (http://imaging.mrc-cbu.cam.ac.uk/imaging/AutomaticAnalysisManual).

The measured signal time-series from each voxel was fitted to a combination of regressors under a general linear model. The first 2 regressors modeled the neural response to left and right hand motor responses, whereas the third regressor modeled any set-size independent effect of each visual sample-display event. In each case, the events of interest were convolved with a canonical hemodynamic response function to generate a time-course of predicted neural activity associated with each event type. The next 2 regressors, of primary interest, were similar to the sample-display regressor except that each event was weighted, prior to convolution, either by the number of items in the display, or by the estimated number of items stored in memory (K) at that set-size. This analysis allowed the BOLD signal to be partitioned at the first-level model into components that varied linearly with set-size or plateaued with the number of items in VSTM. A second model was run to determine the response to individual set-sizes, in which the 3 regressors modeling different weightings of all sample displays were replaced by 3 new regressors each modeling displays of a single set-size. In a supplementary analysis, an extra regressor was included to model those trials where the probe disc was taken from the sample color set, but occurred at a different position. All analyses also included regressors to model any mean effect of each session, the realignment parameters as unconvolved regressors to model out residual effects of subject movement, and binary regressors to model out occasional scans where a striping artifact was detected. (These were caused by a combination of subject movement with a sequence that used thin, interleaved slices.) The data and the model were high-pass filtered with a cut-off at 200 s to remove low frequency drift in the signal. Contrasts were run to estimate the signal change associated with these regressors at each voxel, and the resulting maps from each subject were entered into a second level analysis treating subjects as a random effect. Statistical parametric maps of t-statistics were thresholded at a significance level of P < 0.05, corrected for multiple comparisons using the false discovery rate (FDR). All experimental sessions (i.e., the VSTM, IM, ESA, and ESTA tasks) were treated similarly.

In addition to this whole-brain approach, regions of interest (ROIs) were defined as spheres of 10 mm radius, centered on the Talairach coordinates reported by Todd and Marois (2004): left IPS: −22, −65, 42; right IPS: 23, −59, 45; left VO: −34, −68, 0; right VO: 32, −77, −9; ACC: 8, 19, 30. MarsBar (http://marsbar.sourceforge.net/) was used to extract mean BOLD signal time-courses from these regions, and to estimate the coefficients associated with the weighted regressors in the first analysis, and the signal change associated with the sample display at each set-size in the second analysis.

Results

Behavioral

In all comparisons of performance, the d′ and percentage correct measures led to identical conclusions. d′ is reported except where percentage correct is useful for comparison with previous results.

Behavioral performance on the VSTM task (Fig. 2) was comparable with that reported by Todd and Marois (2004):

  1. Performance on the flanking verbal memory test was high (mean 99% correct across all set-sizes) and independent of set-size (percent correct: F2,26 = 0.124, P > 0.05; d′: F2,26 = 0.228, P > 0.05; reaction time: F2,26 = 0.102, P > 0.05).

  2. The number of items stored in VSTM (K) increased from set-size 1 to set-size 4 (t(13) = 10.0, P < 0.01), but remained constant (at about 3 items) from set-size 4 to set-size 8 (t(13) = 0.168, P > 0.05).

  3. Task performance (d′) decreased from set-size 1 to set-size 4 (t(13) = 5.30, P < 0.01) and again from set-size 4 to set-size 8 (t(13) = 6.62, P < 0.01).

  4. Reaction time (RT) increased from set-size 1 to set-size 4 (t(13) = 11.6, P < 0.01) and again from set-size 4 to set-size 8 (t(13) = 4.82, P < 0.01).

Figure 2.

Mean behavioral performance across the group. In the upper panel, the number of items-worth of information encoded in short-term memory (K) is plotted as a function of set-size in the VSTM task. Performance (d′) and reaction time (ms) are plotted against set-size for all tasks, in the middle and lower panels, respectively. Error bars indicate ±1 standard error of the mean across subjects.

Figure 2.

Mean behavioral performance across the group. In the upper panel, the number of items-worth of information encoded in short-term memory (K) is plotted as a function of set-size in the VSTM task. Performance (d′) and reaction time (ms) are plotted against set-size for all tasks, in the middle and lower panels, respectively. Error bars indicate ±1 standard error of the mean across subjects.

Performance in the IM task was high (95–99%) and comparable with that reported by Todd and Marois (2004; 96–98%). Performance in the ESA and ESTA tasks was also consistently high (99% and 94–96%, respectively).

In a repeated measures analysis of variance (ANOVA), with factors of task (VSTM, IM, ESA, ESTA) and set-size (1, 4, 8), performance (d′) on the flanking verbal memory test was independent of task (F3,39 = 1.14, P > 0.05) and set-size (F2,26 = 1.69, P > 0.05), with no interaction (F6,78 = 0.888, P > 0.05). RT for the verbal memory response was affected by task (F3,39 = 4.43, P < 0.01), probably reflecting the fact that in the VSTM and ESTA tasks there was a shorter interval between the 2 required responses leading to a slight increase in the second reaction time. However, as with d′, reaction time for the verbal memory response was unaffected by set-size (F2,26 = 0.377, P > 0.05) or the interaction of set-size and task (F6,78 = 0.719, P > 0.05).

With respect to the primary response to the visual stimuli, a similar ANOVA with factors of task and set-size showed significant effects of both manipulations on both d′ and RT (all P < 0.01). The effect of set-size was examined in more detail for each task with one-way ANOVAs and planned t-tests: In the VSTM task, d′ decreased and RT increased with both increases in set-size as described above. In the IM task, d′ remained constant from set-size 1 to set-size 4 (t(13) = 0.865, P > 0.05) but increased from set-size 4 to set-size 8 (t(13) = 2.58, P < 0.05), whereas RT remained constant from set-size 1 to set-size 4 (t(13) = 1.65, P > 0.05) and decreased from set-size 4 to set-size 8 (t(13) = 5.98, P < 0.01). Neither d′ nor RT was significantly affected by set-size in the ESA task (F2,26 = 0.846, P > 0.05 and F2,26 = 0.226, P > 0.05, respectively) or in the ESTA task (F2,26 = 0.993, P > 0.05 and F2,26 = 0.109, P > 0.05, respectively).

Given that the stimuli in the IM and ESA tasks were identical, it is appropriate to compare performance between them. Such comparisons between the other tasks were not performed because the stimulus differences would complicate their interpretation. RT did not differ significantly between the IM and the ESA tasks (F1,13 = 3.86, P > 0.05), whereas the ESA task was found to be slightly, but significantly, easier than the IM task, as measured either by d′ (F1,13 = 11.0, P < 0.05) or percent correct (F1,13 = 10.6, P < 0.05).

Neuroimaging

In a whole-brain analysis, with subjects as a random effect, a contrast of the K-weighted regressor during the VSTM task revealed significant activity (P < 0.05 FDR-corrected) bilaterally along the IPS/IOS and in ventral and lateral occipital cortex, as well as in numerous other regions including the central and precentral sulci, cerebelli, cingulate cortex extending into the presupplementary motor areas, the hippocampi, insulae, inferior frontal regions, and subcortical structures (Fig. 3A left panel, Fig. 3B left column, and Table 1). This activity represents the component of the BOLD response which correlates with the number of items stored in VSTM, after accounting for any components which vary linearly with set-size or occur on each trial independently of set-size. A similar, though weaker, pattern of activity was revealed in the occipital and parietal lobes by the same regressor in the ESTA task (Fig. 3A right panel, Fig. 3B right column, and Table 1). No voxels exceeded a whole-brain corrected threshold for the contrast of the K-weighted regressor in the IM or ESA tasks. However, examination of the K-weighted regression coefficients from the IM task revealed a pattern showing some similarity to that in the VSTM and ESTA tasks, especially along the posterior IPS (Fig. 3B, second column). A significant linear effect of set-size was only seen in the ESTA task. This was localized to occipital cortex, medial and posterior to the K-weighted activity.

Table 1

Talairach coordinates of 10 largest local maxima for each whole-brain contrast

Contrast Location x y z t (df = 13) 
VSTM task FEF −28 −3 57 4.5 
K-weighted regressor FEF 30 55 3.9 
 IOG/ITG −44 −74 −3 3.9 
 aIPS −40 −37 46 6.0 
 pIPS −22 −62 51 5.5 
 pIPS 28 −60 44 5.4 
 MOG 50 −60 −4 4.5 
 MOG −26 −78 30 4.3 
 MOG 38 −79 13 3.4 
 preSMA 10 46 4.6 
ESTA task Cerebellum −30 −67 −12 4.0 
K-weighted regressor IOG/ITG 46 −70 8.6 
 IOG/ITG −44 −74 −1 4.9 
 aIPS 32 −32 50 4.1 
 pIPS 30 −50 52 5.4 
 pIPS −26 −54 51 4.1 
 pIPS 28 −62 40 4.1 
 MOG 36 −79 6.5 
 MOG −28 −72 31 5.7 
 MOG −34 −81 5.2 
ESTA task FG 32 −54 −1 5.0 
Set-size–weighted regressor FG 34 −66 −7 4.5 
 LG −14 −94 −5 6.2 
 LG −30 −84 −9 6.0 
 LG 28 −86 −7 5.4 
 MOG −24 −95 5.3 
 MOG −34 −95 4.6 
 MOG 32 −87 14 4.5 
 Pons −27 −27 4.3 
 Thalamus −20 −19 −1 4.7 
Contrast Location x y z t (df = 13) 
VSTM task FEF −28 −3 57 4.5 
K-weighted regressor FEF 30 55 3.9 
 IOG/ITG −44 −74 −3 3.9 
 aIPS −40 −37 46 6.0 
 pIPS −22 −62 51 5.5 
 pIPS 28 −60 44 5.4 
 MOG 50 −60 −4 4.5 
 MOG −26 −78 30 4.3 
 MOG 38 −79 13 3.4 
 preSMA 10 46 4.6 
ESTA task Cerebellum −30 −67 −12 4.0 
K-weighted regressor IOG/ITG 46 −70 8.6 
 IOG/ITG −44 −74 −1 4.9 
 aIPS 32 −32 50 4.1 
 pIPS 30 −50 52 5.4 
 pIPS −26 −54 51 4.1 
 pIPS 28 −62 40 4.1 
 MOG 36 −79 6.5 
 MOG −28 −72 31 5.7 
 MOG −34 −81 5.2 
ESTA task FG 32 −54 −1 5.0 
Set-size–weighted regressor FG 34 −66 −7 4.5 
 LG −14 −94 −5 6.2 
 LG −30 −84 −9 6.0 
 LG 28 −86 −7 5.4 
 MOG −24 −95 5.3 
 MOG −34 −95 4.6 
 MOG 32 −87 14 4.5 
 Pons −27 −27 4.3 
 Thalamus −20 −19 −1 4.7 

Note: Maxima are at least 15 mm apart and significant at P < 0.05 after FDR correction for multiple comparisons. FEF, frontal eye fields; IOG/ITG, inferior occipital/temporal gyri; aIPS, anterior intraparietal sulcus; pIPS, posterior intraparietal sulcus; MOG, middle occipital gyrus; preSMA, presupplementary motor area; FG, fusiform gyrus; LG, lingual gyrus.

Figure 3.

fMRI whole-brain group results. (A) t-statistics representing K-weighted (red) and linearly weighted (green) BOLD activity during the VSTM task (left) and the ESTA task (right). Illustrated activations are significant at P < 0.05 after whole-brain FDR correction, and are rendered onto the Colin template brain. The IM and ESA tasks revealed no suprathreshold voxels. Dashed blue lines indicate the positions of the cross-sections shown below. (B) Regression coefficients for the regressor modeling K-weighted BOLD signal changes, overlaid on selected axial slices. z-coordinates are given in MNI space; black and green contours, respectively, indicate the significant K-weighted and set-size–weighted activations rendered in (A); white contours indicate ROIs centered on coordinates taken from Todd and Marois (2004).

Figure 3.

fMRI whole-brain group results. (A) t-statistics representing K-weighted (red) and linearly weighted (green) BOLD activity during the VSTM task (left) and the ESTA task (right). Illustrated activations are significant at P < 0.05 after whole-brain FDR correction, and are rendered onto the Colin template brain. The IM and ESA tasks revealed no suprathreshold voxels. Dashed blue lines indicate the positions of the cross-sections shown below. (B) Regression coefficients for the regressor modeling K-weighted BOLD signal changes, overlaid on selected axial slices. z-coordinates are given in MNI space; black and green contours, respectively, indicate the significant K-weighted and set-size–weighted activations rendered in (A); white contours indicate ROIs centered on coordinates taken from Todd and Marois (2004).

In order to probe in more detail the activity in some of these regions, the mean BOLD signal change was extracted from ROIs centered on the coordinates reported by Todd and Marois (2004) and illustrated in Figure 3B. Figure 4A shows the best fitting canonical hemodynamic response function to the sample display at each set-size, across the 3 ROIs and 4 tasks. Because the design was optimized to examine differential responses across set-sizes rather than to estimate the absolute evoked response to a specific set-size, plotted responses are normalized to a set-size of 1 by subtracting the response when only a single item was presented. The absolute signal change at each set-size (provided in Table 2) shows a negative offset in some conditions, especially in the occipital ROI. The significance of this is difficult to interpret due to little power to separate responses to the sample display from other subprocesses within each trial. Incorporating temporal jitter between the auditory stimuli, sample display and probe display would allow this to be resolved in future experiments. In Figure 4B, the same data are replotted showing the peak estimated BOLD signal change increase as a function of set-size. The behavioral measure of the estimated number of objects maintained in the VSTM task (K) is included on each axis for comparison. Figure 4C shows the coefficients associated with the regressors modeling the components of the BOLD signal that vary across set-size in a linear, and K-weighted manner. The data from this analysis are provided in Table 3.

Table 2

Results of the ROI analysis in which each set-size was modeled separately

ROI Task Regressor % Signal change (from ROI session mean) Increase in % signal change relative to set-size 1 Pairwise test of increase from lower set-size (t(13)) 
IPS VSTM Set-size 1 0.032   
Left: −22, −65, 42  Set-size 4 0.19 0.16 5.9*** 
Right: 23, −59, 45  Set-size 8 0.16 0.13 −2.3 
 IM Set-size 1 −0.028   
  Set-size 4 0.056 0.083 3.4** 
  Set-size 8 0.040 0.068 −0.55 
 ESA Set-size 1 0.0067   
  Set-size 4 0.012 0.0049 0.41 
  Set-size 8 0.028 0.022 0.89 
 ESTA Set-size 1 −0.062   
  Set-size 4 0.030 0.092 4.9*** 
  Set-size 8 0.034 0.096 −0.35 
ACC VSTM Set-size 1 −0.068   
Right: 8, 19, 30  Set-size 4 0.036 0.10 5.6*** 
  Set-size 8 0.039 0.11 0.32 
 IM Set-size 1 −0.052   
  Set-size 4 −0.039 0.013 0.63 
  Set-size 8 −0.038 0.014 0.076 
 ESA Set-size 1 −0.015   
  Set-size 4 −0.017 −0.0024 −0.14 
  Set-size 8 −0.021 −0.0061 0.29 
 ESTA Set-size 1 −0.068   
  Set-size 4 −0.048 0.020 1.1 
  Set-size 8 −0.049 0.019 −0.081 
VO VSTM Set-size 1 −0.25   
Left: −34, −68, 0  Set-size 4 −0.15 0.094 4.2*** 
Right: 32, −77, −9  Set-size 8 −0.15 0.094 −0.0063 
 IM Set-size 1 −0.20   
  Set-size 4 −0.16 0.038 1.7 
  Set-size 8 −0.12 0.075 1.5 
 ESA Set-size 1 −0.20   
  Set-size 4 −0.16 0.032 2.4* 
  Set-size 8 −0.12 0.073 2.7* 
 ESTA Set-size 1 −0.25   
  Set-size 4 −0.14 0.11 7.3*** 
  Set-size 8 −0.098 0.16 4.1*** 
ROI Task Regressor % Signal change (from ROI session mean) Increase in % signal change relative to set-size 1 Pairwise test of increase from lower set-size (t(13)) 
IPS VSTM Set-size 1 0.032   
Left: −22, −65, 42  Set-size 4 0.19 0.16 5.9*** 
Right: 23, −59, 45  Set-size 8 0.16 0.13 −2.3 
 IM Set-size 1 −0.028   
  Set-size 4 0.056 0.083 3.4** 
  Set-size 8 0.040 0.068 −0.55 
 ESA Set-size 1 0.0067   
  Set-size 4 0.012 0.0049 0.41 
  Set-size 8 0.028 0.022 0.89 
 ESTA Set-size 1 −0.062   
  Set-size 4 0.030 0.092 4.9*** 
  Set-size 8 0.034 0.096 −0.35 
ACC VSTM Set-size 1 −0.068   
Right: 8, 19, 30  Set-size 4 0.036 0.10 5.6*** 
  Set-size 8 0.039 0.11 0.32 
 IM Set-size 1 −0.052   
  Set-size 4 −0.039 0.013 0.63 
  Set-size 8 −0.038 0.014 0.076 
 ESA Set-size 1 −0.015   
  Set-size 4 −0.017 −0.0024 −0.14 
  Set-size 8 −0.021 −0.0061 0.29 
 ESTA Set-size 1 −0.068   
  Set-size 4 −0.048 0.020 1.1 
  Set-size 8 −0.049 0.019 −0.081 
VO VSTM Set-size 1 −0.25   
Left: −34, −68, 0  Set-size 4 −0.15 0.094 4.2*** 
Right: 32, −77, −9  Set-size 8 −0.15 0.094 −0.0063 
 IM Set-size 1 −0.20   
  Set-size 4 −0.16 0.038 1.7 
  Set-size 8 −0.12 0.075 1.5 
 ESA Set-size 1 −0.20   
  Set-size 4 −0.16 0.032 2.4* 
  Set-size 8 −0.12 0.073 2.7* 
 ESTA Set-size 1 −0.25   
  Set-size 4 −0.14 0.11 7.3*** 
  Set-size 8 −0.098 0.16 4.1*** 

Note: Coordinates given in Talairach space. 1-tailed significance: *P ≤ 0.05; **P ≤ 0.01; ***P ≤ 0.001.

Table 3

Results of the ROI analysis in which the K-weighted and linear-set-size–weighted components of the BOLD response were modeled

ROI Task Regressor Mean regression coefficient t (df = 13) 
IPS VSTM K-weighted 0.82 5.0*** 
Left: −22, −65, 42  Set-size weighted −0.22 −2.8 
Right: 23, −59, 45 IM K-weighted 0.48 2.3* 
  Set-size weighted −0.15 −0.82 
 ESA K-weighted −0.061 −0.68 
  Set-size weighted 0.13 1.2 
 ESTA K-weighted 0.40 3.7** 
  Set-size weighted 0.016 0.20 
ACC VSTM K-weighted 0.36 3.8** 
Right: 8, 19, 30  Set-size weighted 0.012 0.22 
 IM K-weighted 0.047 0.37 
  Set-size weighted −0.017 −0.15 
 ESA K-weighted 0.0089 0.098 
  Set-size weighted −0.028 −0.45 
 ESTA K-weighted 0.065 0.73 
  Set-size weighted 0.0055 0.090 
VO VSTM K-weighted 0.35 2.9** 
Left: −34, −68, 0  Set-size weighted −0.027 −0.32 
Right: 32, −77, −9 IM K-weighted 0.063 0.44 
  Set-size weighted 0.16 1.2 
 ESA K-weighted −0.0056 −0.073 
  Set-size weighted 0.19 2.6* 
 ESTA K-weighted 0.27 4.0** 
  Set-size weighted 0.19 4.1** 
ROI Task Regressor Mean regression coefficient t (df = 13) 
IPS VSTM K-weighted 0.82 5.0*** 
Left: −22, −65, 42  Set-size weighted −0.22 −2.8 
Right: 23, −59, 45 IM K-weighted 0.48 2.3* 
  Set-size weighted −0.15 −0.82 
 ESA K-weighted −0.061 −0.68 
  Set-size weighted 0.13 1.2 
 ESTA K-weighted 0.40 3.7** 
  Set-size weighted 0.016 0.20 
ACC VSTM K-weighted 0.36 3.8** 
Right: 8, 19, 30  Set-size weighted 0.012 0.22 
 IM K-weighted 0.047 0.37 
  Set-size weighted −0.017 −0.15 
 ESA K-weighted 0.0089 0.098 
  Set-size weighted −0.028 −0.45 
 ESTA K-weighted 0.065 0.73 
  Set-size weighted 0.0055 0.090 
VO VSTM K-weighted 0.35 2.9** 
Left: −34, −68, 0  Set-size weighted −0.027 −0.32 
Right: 32, −77, −9 IM K-weighted 0.063 0.44 
  Set-size weighted 0.16 1.2 
 ESA K-weighted −0.0056 −0.073 
  Set-size weighted 0.19 2.6* 
 ESTA K-weighted 0.27 4.0** 
  Set-size weighted 0.19 4.1** 

Note: Coordinates given in Talairach space. 1-tailed significance: *P ≤ 0.05; **P ≤ 0.01; ***P ≤ 0.001.

Figure 4.

BOLD activity extracted from the ROIs illustrated in Figure 3B. The ACC ROI was a single locus, whereas the posterior parietal (IPS) and ventral occipital (VO) ROIs pooled data across both hemispheres. (A) Estimated BOLD time-courses following onset of the sample display at each set-size. Event-related amplitude fits of the canonical hemodynamic response function are normalized relative to set-size 1 within each task and ROI. Dotted lines indicate ±1 standard error. (B) Peak, normalized, percent signal change, plotted as a function of set-size. The estimated number of objects maintained in the VSTM task (K) is plotted on each axis for comparison. (C) Blue bars represent mean regression coefficients for the regressor modeling K-weighted BOLD signal changes, whereas red bars represent coefficients for the regressor modeling a linear effect of set-size. Different task conditions are presented along the y-axes for each ROI. Error bars indicate ±1 standard error of the mean across subjects.

Figure 4.

BOLD activity extracted from the ROIs illustrated in Figure 3B. The ACC ROI was a single locus, whereas the posterior parietal (IPS) and ventral occipital (VO) ROIs pooled data across both hemispheres. (A) Estimated BOLD time-courses following onset of the sample display at each set-size. Event-related amplitude fits of the canonical hemodynamic response function are normalized relative to set-size 1 within each task and ROI. Dotted lines indicate ±1 standard error. (B) Peak, normalized, percent signal change, plotted as a function of set-size. The estimated number of objects maintained in the VSTM task (K) is plotted on each axis for comparison. (C) Blue bars represent mean regression coefficients for the regressor modeling K-weighted BOLD signal changes, whereas red bars represent coefficients for the regressor modeling a linear effect of set-size. Different task conditions are presented along the y-axes for each ROI. Error bars indicate ±1 standard error of the mean across subjects.

Primary interest was in the posterior parietal (IPS/IOS) ROI, where the signal change associated with the K-weighted regressor was significantly greater than zero in the VSTM, IM and ESTA tasks (all P < 0.05, 1 tailed; Fig. 4C, upper panel). The K-weighted activity in the IPS was significantly greater during the IM task than the ESA task (t(13) = 2.65, P < 0.05, 2 tailed). Because these tasks were identical in terms of stimuli, the difference can only be attributed to the different task instructions. For the IPS, in no task was the signal change associated with the linear set-size–weighted regressor significantly greater than zero. In the anterior cingulate (ACC) ROI, the only significant effect was for the K-weighted regressor during the VSTM task (Fig. 4C, middle panel). In the ventrolateral occipital (VO) ROI, a significant effect of the K-weighted regressor was seen during the VSTM and ESTA tasks, along with a significant effect of the linear set-size–weighted regressor during the ESA and ESTA tasks (Fig. 4C, lower panel).

In a supplementary analysis, an extra regressor was included in the design matrix to model those memory trials in which the probe disc was taken from the sample color set, but occurred at a different position. The regressor modeling these events detected significant group activity in both the ACC and IPS ROIs (both P < 0.05, 2 tailed), but importantly, in all ROIs, its inclusion had a negligible influence on the other regressors.

Discussion

The main findings of the current study are that 1) activity in the posterior IPS increases with the number of presented objects in simple perceptual tasks as well as in a visual working memory task; 2) this perceptual load–related activity has a similar capacity limit to that observed during the VSTM task; and 3) the observation of load-related activity in perceptual tasks is sensitive to the particular task demands imposed.

Many aspects of these results are consistent with previous studies, especially those of Todd and Marois (2004), both in terms of the behavioral measures and, importantly, in terms of the replication that activity in the posterior IPS is better explained by a function that asymptotes when the number of display items exceeds working memory capacity, than by a linear function of set-size. However, there are also interesting differences. In the VSTM task, we found significant K-weighted activity across a wider brain network including the ventral occipital and ACC ROIs, where we found a significant positive effect for the K-weighted regressor but not for the linear set-size–weighted regressor. During the IM task, we observed a significant correlation of IPS activity with the K-weighted regressor, and we failed to find a significant linear effect of set-size in the ventral occipital region.

There are several possible reasons why the present study found parietal K-weighted activity in perceptual tasks, in contrast to the findings of Todd and Marois (2004). Perhaps apparently perceptual tasks that activate this region involve automatic encoding of object representations into short-term memory, despite no obvious requirement to do so. This leaves the question of why such automatic encoding would have occurred in the current IM task, but not in that of Todd and Marois. It is conceivable that the current multitask context in which half the subjects performed the IM task directly after the VSTM task could have encouraged such a task-set. Alternatively, there may have been some sensitivity to perceptual load in Todd's and Marois’ control experiment, which failed to reach significance due to a lower effect size or reduced power for other task-related or methodological reasons. This would be consistent with the more extensive activations observed during the VSTM task in the current study. Indeed, amongst the additional activated areas found in the whole-brain analysis were those regions identified by Todd and Marois when they relaxed their statistical threshold. Another region strongly activated in the VSTM task was around the junction of the precentral sulcus and the superior frontal sulcus, near to the frontal eye fields (FEF). Signal change in this region has previously been noted to asymptote around set-size 2 (Culham et al. 2001; Todd and Marois 2005), and may reflect planning of upcoming saccades or attention shifts (Corbetta et al. 1998). In the current study, the brief, foveal presentation of stimuli was designed to encourage fixation and render saccades unhelpful, however, covert attention shifts might constitute an important rehearsal mechanism in working memory (Smyth 1996; Awh and Jonides 2001; Awh et al. 2006).

A potential confound in the current VSTM task was that for change trials with a single sample item, the probe item was always a novel color, however, for change trials at the higher set-sizes, the probe had an equal probability of being a novel color or being a color used elsewhere in the sample display, possibly causing response conflict. To rule out this explanation for the BOLD set-size effect, the data were reanalyzed with the latter events explicitly modeled. Although these events produced significant group activity in both the ACC and IPS ROIs, which may reflect an additional element of response conflict, modeling them separately had negligible influence on the other regressors. This suggests that although such probes increased activity within the ACC and IPS, this is insufficient to explain the activity that correlates with perceptual and working memory load.

The current study failed to find significant positive correlations between the BOLD signal and the linear set-size regressor in either the whole-brain analysis or any of the ROIs during the VSTM or IM tasks. It would be inappropriate to draw strong conclusions from such a null result, especially for the occipital ROI, where such an effect was observed during the ESA and ESTA tasks. It remains possible that, with greater statistical power, a linear effect of set-size would have been detected in the occipital ROI during the IM task. The observation that, during the VSTM task, activity in the occipital ROI is better explained by the K-weighted regressor than the linear set-size–weighted regressor is consistent with a recent study (Xu and Chun 2006), although their ROI lay approximately 10 mm more lateral than the current occipital ROI. The reduced linear effect of set-size in VO during the VSTM task relative to the perceptual tasks (P < 0.05, 2 tailed) is also apparent in the plots of signal change against set-size (Fig. 4B, lower panel). One might speculate that representations in earlier visual areas reflect an interaction between incoming sensory input and feedback biasing signals from higher levels (Friston 2005). During perceptual tasks, this balance may favor the visual input, whereas during the memory delay, earlier visual areas may be recruited by feedback connections from PPC, to help maintain its limited number of higher level representations.

It should be noted that the use of relatively short trials in these experiments means that it is not possible here to separate the sluggish hemodynamic response into components reflecting subprocesses of each task, for example encoding, maintenance, and retrieval in the VSTM task. Studies which have used slow event-related fMRI designs (Todd and Marois 2004; Xu and Chun 2006), or electrophysiology (Busch and Herrmann 2003), suggest, however, that the activity reflecting object-based capacity limits is likely to correspond to processes occurring during encoding and memory maintenance more than memory retrieval.

What Aspects of a Perceptual Task Might Determine Whether it Evokes a Capacity-Limited Response in PPC?

Although we have demonstrated that the IPS has a capacity-limited response to perceptual load as well as to working memory load, there was a noticeable lack of such an effect in the ESA task, despite its being visually identical to the IM task. This sensitivity of IPS perceptual load–related activity to the specific perceptual task could potentially be explained by a number of factors, which include the extent of spatial attention, the duration of attention, the dependence of the task on spatial information, task difficulty, and the binding of properties to spatiotemporal object tokens. These will be discussed in turn.

The hypothesis motivating this study was that even in the absence of an explicit working memory requirement, as attentional demands were increased in the perceptual tasks, significant set-size–related BOLD signal changes would be observed in PPC. The IM and ESA tasks were designed to be physically identical, but with the required spatial scope of attention being increased in the latter. We therefore expected to see more reliable K-weighted activity in the ESA task if the spatial extent of attention were important (at least at the scale employed in the current manipulation), but no difference if it were not. The reverse finding that the K-weighted activity in the IPS ROI was significantly greater during the IM task than the ESA task therefore suggests that perceptual load–related activity is not dependent on increasing the extent of spatial attention at this scale.

It was also considered that temporally sustained attention to the stimulus representations, whether in working memory or constantly visible, may be required for distinct object representations to build up in PPC, and so set-size–weighted BOLD activity would not be measured if attention were disengaged from the stimuli too quickly. That sustained attention is required for segregated object representations to build up is suggested from studies of visual apparent motion (Anstis et al. 1985) and auditory streaming (Cusack et al. 2004), where the percept of multiple, separable objects/streams increases in likelihood as attention to an ambiguous stimulus is maintained. This motivated the design of the ESTA task, where attention to the sample display was prolonged. If perceptual load–related activity were dependent on attentional duration, we expected to see a greater set-size effect in the VSTM and ESTA tasks, where the items (or their representations in memory) were being attended for over a second, compared with the IM and ESA tasks, where a response was made immediately and attention could thus be disengaged much earlier. Given that the stimuli in the latter 2 tasks were identical, and they showed no significant difference in reaction times, the significant difference in K-weighted activity between the IM and ESA tasks makes this hypothesis unlikely.

Considering the role of PPC in the processing of spatial information (Owen et al. 1996; Sereno et al. 2001; Silver et al. 2005; Schluppeck et al. 2006), and the fact that different objects typically occupy unique spatial locations, one might surmise that the observed load sensitivity corresponds specifically to the processing of object locations rather than object representations more generally. In this case it should be seen in those conditions where location is relevant to the task. Although this would explain the observed pattern in the VSTM, IM, and ESA tasks (with the ESA task being the only one where location was irrelevant and the only one where IPS activity did not vary with set-size), it would predict no significant set-size effect in the ESTA task where position was equally irrelevant. This hypothesis was therefore also rejected. That distinct spatial locations are not necessary to observe an effect of object number on IPS activity is also suggested by paradigms using sequential presentation at a single location (Linden et al. 2003; Xu and Chun 2006) as well as experiments from our lab using simultaneous presentation, where “objects” were either transparent overlapping dot surfaces or auditory streams coming from the same part of space (Cusack 2005).

Although there was no significant difference in reaction times between the IM and ESA tasks, the ESA task was found to be slightly, but significantly, easier than the IM task, as measured either by d′ or percent correct. So, although the K-weighted activity does not seem to index task difficulty directly (difficulty increased linearly with set-size in the VSTM task and did not increase with set-size in any of the perceptual tasks), it remains possible that this activity is modulated by the overall difficulty of the task, becoming more pronounced as the task becomes more challenging.

In retrospect, it seems plausible that although the whole stimulus array had to be assessed in the ESA task, because only a single feature (redness) had to be detected, this could have been accomplished in an efficient, parallel manner, without needing to bind this color property to an individual object. In contrast, in the IM task, the central disc must be bound to its location, which may require the items to be individuated rather than processed as a group. (Of course, it may seem optimal in the IM task to just encode the central position, while completely ignoring any surrounding discs. The fact that subjects did not do this, however, is indicated by the significant difference in performance and reaction time in this task between set-sizes of 4 and 8, which were identical as far as the central array position was concerned. It is worth noting that for set-size 8 and some configurations of 4 items, although processing of the surrounding items is not necessary to complete the task, it is sufficient, because the global configuration of the array determines whether an item is present in the central position.) Following this logic, it may be that detecting the staggered offset in the ESTA task requires binding of each object to its time of offset. Along similar lines, segregating the discs may be required to enhance the signal to noise ratio by binding properties to individual items rather than binding a mean property to the group as a whole. This would be consistent with classical models of visual search, where search for targets defined by features coded automatically in early vision (e.g., redness in the IM task) can occur in parallel across the array, whereas search for the absence of a primitive feature (e.g., contrast in the ESTA task) benefits from segmenting the array into individual objects (Treisman and Gormican 1988). There is also evidence that elements can be selectively grouped into higher-order objects depending on the requirements of a visual search task (Treisman 1982; Duncan and Humphreys 1989) or a MOT task (Yantis 1992). It is therefore possible that the observation of perceptual load–related activity in the IPS is dependent on flexible grouping/segmentation of hierarchical object representations and that the optimal degree of segmentation depends on the salience of the discrimination required or the necessity to bind the relevant features to particular spatiotemporal tokens. This idea may not be unrelated to the explanation in terms of task difficulty, that is, as the required discriminations become more demanding, task difficulty increases and a greater level of segmentation may be required to compensate. Further work will be required to disentangle these possibilities.

The current observation that posterior parietal activity increases with the number of objects that are presented, is consistent with previous studies in which storage of objects in working memory has not been emphasized (Jovicich et al. 2001; Ritzl et al. 2003; Tomasi et al. 2004; Cusack 2005; Fang and He 2005; Hon et al. 2006). That this activity asymptotes by the time the number of objects has risen to around 4 is also consistent with neuroimaging studies of MOT (Culham et al. 2001; Jovicich et al. 2001; Tomasi et al. 2004), a paradigm in which the capacity limit is often thought to reflect preattentive or attentional selection processes rather than working memory processes (Pylyshyn and Storm 1988). The visual indexing mechanism suggested by Pylyshyn et al. to account for performance in MOT and other tasks that show evidence of limited capacity parallel processes in early vision (e.g., subitizing and the detection of illusory line motion; Pylyshyn 1989; Trick and Pylyshyn 1994; Schmidt et al. 1998) is proposed to consist of a limited number of demonstrative pointers which are linked to objects prior to any assignment of properties. They serve to individuate the objects, to provide rapid access to their properties, and to maintain the identities of spatiotemporally continuous entities despite changes in spatial location, sensory properties, or classification label. They may be further elaborated into temporary object files (Kahneman et al. 1992). It is possible that the capacity-limited posterior parietal activity observed here in simple perceptual tasks reflects this process of individuating a small number of relevant objects, so that properties such as color can then be bound to them. This would also be consistent with a recent study which demonstrated that a similar region was better activated by discrete visual stimuli than by continuous, analog stimuli, from which it is presumably more difficult to individuate distinct items (Castelli et al. 2006).

Conclusion

This study demonstrates that the region of the IPS where BOLD activity reflects the limited number of objects held in working memory also shows a similar capacity-limited response under certain perceptual conditions that involve no explicit memory requirements. This is consistent with proposals that the IPS activity seen during VSTM tasks is a product of attentional selection processes rather than representing the contents of working memory per se (Cowan 2001; Marois and Ivanoff 2005; Majerus et al. 2007). Furthermore, this effect is sensitive to the particular perceptual task imposed, even when the stimuli are identical, perhaps reflecting a task requirement for the binding of properties to spatiotemporally individuated object tokens. Thus, the physiological markers of the number of internal object representations developed by Todd and Marois (2004) and by Vogel and Machizawa (2004) provide a useful tool to investigate how stimulus properties and task demands constrain the visual system's individuation of objects from a scene in perception and working memory. Given the highly limited capacity of VSTM and of visuospatial attention, the ability to manage these resources flexibly is crucial. Dynamic grouping and segregation of perceptual object representations according to current task demands may be one mechanism of achieving such flexibility, and this computation may occur within PPC.

Funding

D.J.M. and R.C. were supported by the Medical Research Council (UK).

We thank John Duncan for helpful comments on the manuscript. Conflict of interest: None declared.

References

Alvarez
GA
Cavanagh
P
The capacity of visual short-term memory is set both by visual information load and by number of objects
Psychol Sci.
 , 
2004
, vol. 
15
 (pg. 
106
-
111
)
Anstis
S
Giaschi
D
Cogan
AI
Adaptation to apparent motion
Vision Res.
 , 
1985
, vol. 
25
 (pg. 
1051
-
1062
)
Awh
E
Barton
B
Vogel
EK
Visual working memory represents a fixed number of items regardless of complexity
Psychol Sci.
 , 
2007
, vol. 
18
 (pg. 
622
-
628
)
Awh
E
Jonides
J
Overlapping mechanisms of attention and spatial working memory
Trends Cogn Sci.
 , 
2001
, vol. 
5
 (pg. 
119
-
126
)
Awh
E
Vogel
EK
Oh
SH
Interactions between attention and working memory
Neuroscience
 , 
2006
, vol. 
139
 (pg. 
201
-
208
)
Baddeley
A
Working memory
 , 
1986
New York
Oxford University Press
Bor
D
Duncan
J
Wiseman
RJ
Owen
AM
Encoding strategies dissociate prefrontal activity from working memory demand
Neuron
 , 
2003
, vol. 
37
 (pg. 
361
-
367
)
Busch
NA
Herrmann
CS
Object-load and feature-load modulate EEG in a short-term memory task
Neuroreport
 , 
2003
, vol. 
14
 (pg. 
1721
-
1724
)
Castelli
F
Glaser
DE
Butterworth
B
Discrete and analogue quantity processing in the parietal lobe: a functional MRI study
Proc Natl Acad Sci USA.
 , 
2006
, vol. 
103
 (pg. 
4693
-
4698
)
Corbetta
M
Akbudak
E
Conturo
TE
Snyder
AZ
Ollinger
JM
Drury
HA
Linenweber
MR
Petersen
SE
Raichle
ME
Van Essen
DC
, et al.  . 
A common network of functional areas for attention and eye movements
Neuron
 , 
1998
, vol. 
21
 (pg. 
761
-
773
)
Cowan
N
The magical number 4 in short-term memory: a reconsideration of mental storage capacity
Behav Brain Sci.
 , 
2001
, vol. 
24
 (pg. 
87
-
114
discussion 114–185
Culham
JC
Cavanagh
P
Kanwisher
NG
Attention response functions: characterizing brain areas using fMRI activation during parametric variations of attentional load
Neuron
 , 
2001
, vol. 
32
 (pg. 
737
-
745
)
Cusack
R
The intraparietal sulcus and perceptual organization
J Cogn Neurosci.
 , 
2005
, vol. 
17
 (pg. 
641
-
651
)
Cusack
R
Deeks
J
Aikman
G
Carlyon
RP
Effects of location, frequency region, and time course of selective attention on auditory scene analysis
J Exp Psychol Hum Percept Perform.
 , 
2004
, vol. 
30
 (pg. 
643
-
656
)
Davis
G
Holmes
A
The capacity of visual short-term memory is not a fixed number of objects
Mem Cognit.
 , 
2005
, vol. 
33
 (pg. 
185
-
195
)
Duncan
J
Selective attention and the organization of visual information
J Exp Psychol Gen.
 , 
1984
, vol. 
113
 (pg. 
501
-
517
)
Duncan
J
Humphreys
GW
Visual search and stimulus similarity
Psychol Rev.
 , 
1989
, vol. 
96
 (pg. 
433
-
458
)
Fang
F
He
S
Cortical responses to invisible objects in the human dorsal and ventral pathways
Nat Neurosci.
 , 
2005
, vol. 
8
 (pg. 
1380
-
1385
)
Ferreira
CT
Verin
M
Pillon
B
Levy
R
Dubois
B
Agid
Y
Spatio-temporal working memory and frontal lesions in man
Cortex.
 , 
1998
, vol. 
34
 (pg. 
83
-
98
)
Fougnie
D
Marois
R
Distinct capacity limits for attention and working memory: evidence from attentive tracking and visual working memory paradigms
Psychol Sci.
 , 
2006
, vol. 
17
 (pg. 
526
-
534
)
Friston
K
A theory of cortical responses
Philos Trans R Soc Lond B Biol Sci.
 , 
2005
, vol. 
360
 (pg. 
815
-
836
)
Greenwood
JA
Edwards
M
Pushing the limits of transparent-motion detection with binocular disparity
Vision Res.
 , 
2006
, vol. 
46
 (pg. 
2615
-
2624
)
Hikosaka
O
Miyauchi
S
Shimojo
S
Focal visual attention produces illusory temporal order and motion sensation
Vision Res.
 , 
1993
, vol. 
33
 (pg. 
1219
-
1240
)
Hon
N
Epstein
RA
Owen
AM
Duncan
J
Frontoparietal activity with minimal decision and control
J Neurosci.
 , 
2006
, vol. 
26
 (pg. 
9805
-
9809
)
Humphreys
GW
Neural representation of objects in space: a dual coding account
Philos Trans R Soc Lond B Biol Sci.
 , 
1998
, vol. 
353
 (pg. 
1341
-
1351
)
Jiang
Y
Chun
MM
Olson
IR
Perceptual grouping in change detection
Percept Psychophys.
 , 
2004
, vol. 
66
 (pg. 
446
-
453
)
Jiang
Y
Olson
IR
Chun
MM
Organization of visual short-term memory
J Exp Psychol Learn Mem Cogn.
 , 
2000
, vol. 
26
 (pg. 
683
-
702
)
Jovicich
J
Peters
RJ
Koch
C
Braun
J
Chang
L
Ernst
T
Brain areas specific for attentional load in a motion-tracking task
J Cogn Neurosci.
 , 
2001
, vol. 
13
 (pg. 
1048
-
1058
)
Kahneman
D
Treisman
A
Gibbs
BJ
The reviewing of object files: object-specific integration of information
Cognit Psychol.
 , 
1992
, vol. 
24
 (pg. 
175
-
219
)
Kaufman
EL
Lord
MW
The discrimination of visual number
Am J Psychol.
 , 
1949
, vol. 
62
 (pg. 
498
-
525
)
Linden
DE
Bittner
RA
Muckli
L
Waltz
JA
Kriegeskorte
N
Goebel
R
Singer
W
Munk
MH
Cortical capacity constraints for visual working memory: dissociation of fMRI load effects in a fronto-parietal network
Neuroimage
 , 
2003
, vol. 
20
 (pg. 
1518
-
1530
)
Luck
SJ
Vogel
EK
The capacity of visual working memory for features and conjunctions
Nature
 , 
1997
, vol. 
390
 (pg. 
279
-
281
)
Majerus
S
Bastin
C
Poncelet
M
Van der Linden
M
Salmon
E
Collette
F
Maquet
P
Short-term memory and the left intraparietal sulcus: focus of attention? Further evidence from a face short-term memory paradigm
Neuroimage
 , 
2007
, vol. 
35
 (pg. 
353
-
367
)
Marois
R
Ivanoff
J
Capacity limits of information processing in the brain
Trends Cogn Sci.
 , 
2005
, vol. 
9
 (pg. 
296
-
305
)
O'Craven
KM
Downing
PE
Kanwisher
N
fMRI evidence for objects as the units of attentional selection
Nature
 , 
1999
, vol. 
401
 (pg. 
584
-
587
)
Olson
IR
Jiang
Y
Is visual short-term memory object based? Rejection of the “strong-object” hypothesis
Percept Psychophys
 , 
2002
, vol. 
64
 (pg. 
1055
-
1067
)
Owen
AM
Working memory: imaging the magic number four
Curr Biol.
 , 
2004
, vol. 
14
 (pg. 
R573
-
R574
)
Owen
AM
Milner
B
Petrides
M
Evans
AC
Memory for object features versus memory for object location: a positron-emission tomography study of encoding and retrieval processes
Proc Natl Acad Sci USA.
 , 
1996
, vol. 
93
 (pg. 
9212
-
9217
)
Pashler
H
Familiarity and visual change detection
Percept Psychophys.
 , 
1988
, vol. 
44
 (pg. 
369
-
378
)
Paul
MA
Tipper
SP
Object-based representations facilitate memory for inhibitory processes
Exp Brain Res.
 , 
2003
, vol. 
148
 (pg. 
283
-
289
)
Phillips
WA
On the distinction between sensory storage and short-term visual memory
Percept Psychophys.
 , 
1974
, vol. 
16
 (pg. 
283
-
290
)
Piazza
M
Izard
V
Pinel
P
Le Bihan
D
Dehaene
S
Tuning curves for approximate numerosity in the human intraparietal sulcus
Neuron
 , 
2004
, vol. 
44
 (pg. 
547
-
555
)
Pochon
JB
Levy
R
Poline
JB
Crozier
S
Lehericy
S
Pillon
B
Deweer
B
Le Bihan
D
Dubois
B
The role of dorsolateral prefrontal cortex in the preparation of forthcoming actions: an fMRI study
Cereb Cortex.
 , 
2001
, vol. 
11
 (pg. 
260
-
266
)
Postle
BR
Working memory as an emergent property of the mind and brain
Neuroscience
 , 
2006
, vol. 
139
 (pg. 
23
-
38
)
Pylyshyn
Z
The role of location indexes in spatial perception: a sketch of the FINST spatial-index model
Cognition
 , 
1989
, vol. 
32
 (pg. 
65
-
97
)
Pylyshyn
ZW
Visual indexes, preconceptual objects, and situated vision
Cognition.
 , 
2001
, vol. 
80
 (pg. 
127
-
158
)
Pylyshyn
ZW
Storm
RW
Tracking multiple independent targets: evidence for a parallel tracking mechanism
Spat Vis.
 , 
1988
, vol. 
3
 (pg. 
179
-
197
)
Ritzl
A
Marshall
JC
Weiss
PH
Zafiris
O
Shah
NJ
Zilles
K
Fink
GR
Functional anatomy and differential time courses of neural processing for explicit, inferred, and illusory contours. An event-related fMRI study
Neuroimage
 , 
2003
, vol. 
19
 (pg. 
1567
-
1577
)
Schluppeck
D
Curtis
CE
Glimcher
PW
Heeger
DJ
Sustained activity in topographic areas of human posterior parietal cortex during memory-guided saccades
J Neurosci.
 , 
2006
, vol. 
26
 (pg. 
5098
-
5108
)
Schmidt
WC
Fisher
BD
Pylyshyn
ZW
Multiple-location access in vision: evidence from illusory line motion
J Exp Psychol Hum Percept Perform.
 , 
1998
, vol. 
24
 (pg. 
505
-
525
)
Scholl
BJ
Pylyshyn
ZW
Feldman
J
What is a visual object? Evidence from target merging in multiple object tracking
Cognition
 , 
2001
, vol. 
80
 (pg. 
159
-
177
)
Sereno
MI
Pitzalis
S
Martinez
A
Mapping of contralateral space in retinotopic coordinates by a parietal cortical area in humans
Science
 , 
2001
, vol. 
294
 (pg. 
1350
-
1354
)
Silver
MA
Ress
D
Heeger
DJ
Topographic maps of visual spatial attention in human parietal cortex
J Neurophysiol.
 , 
2005
, vol. 
94
 (pg. 
1358
-
1371
)
Smyth
MM
Interference with rehearsal in spatial working memory in the absence of eye movements
Q J Exp Psychol A.
 , 
1996
, vol. 
49
 (pg. 
940
-
949
)
Starkey
P
Cooper
RG
Jr
Perception of numbers by human infants
Science
 , 
1980
, vol. 
210
 (pg. 
1033
-
1035
)
Tipper
SP
Behrmann
M
Object-centered not scene-based visual neglect
J Exp Psychol Hum Percept Perform
 , 
1996
, vol. 
22
 (pg. 
1261
-
1278
)
Todd
JJ
Marois
R
Capacity limit of visual short-term memory in human posterior parietal cortex
Nature
 , 
2004
, vol. 
428
 (pg. 
751
-
754
)
Todd
JJ
Marois
R
Posterior parietal cortex activity predicts individual differences in visual short-term memory capacity
Cogn Affect Behav Neurosci.
 , 
2005
, vol. 
5
 (pg. 
144
-
155
)
Tomasi
D
Ernst
T
Caparelli
EC
Chang
L
Practice-induced changes of brain function during visual attention: a parametric fMRI study at 4 Tesla
Neuroimage
 , 
2004
, vol. 
23
 (pg. 
1414
-
1421
)
Treisman
A
Perceptual grouping and attention in visual search for features and for objects
J Exp Psychol Hum Percept Perform
 , 
1982
, vol. 
8
 (pg. 
194
-
214
)
Treisman
A
Gormican
S
Feature analysis in early vision: evidence from search asymmetries
Psychol Rev.
 , 
1988
, vol. 
95
 (pg. 
15
-
48
)
Trick
LM
Pylyshyn
ZW
Why are small and large numbers enumerated differently? A limited-capacity preattentive stage in vision
Psychol Rev.
 , 
1994
, vol. 
101
 (pg. 
80
-
102
)
Vergilino-Perez
D
Findlay
JM
Between-object and within-object saccade programming in a visual search task
Vision Res.
 , 
2006
, vol. 
46
 (pg. 
2204
-
2216
)
Viswanathan
L
Mingolla
E
Dynamics of attention in depth: evidence from multi-element tracking
Perception
 , 
2002
, vol. 
31
 (pg. 
1415
-
1437
)
Vogel
EK
Machizawa
MG
Neural activity predicts individual differences in visual working memory capacity
Nature
 , 
2004
, vol. 
428
 (pg. 
748
-
751
)
Vogel
EK
Woodman
GF
Luck
SJ
The time course of consolidation in visual working memory
J Exp Psychol Hum Percept Perform
 , 
2006
, vol. 
32
 (pg. 
1436
-
1451
)
Wilken
P
Ma
WJ
A detection theory account of change detection
J Vis.
 , 
2004
, vol. 
4
 (pg. 
1120
-
1135
)
Xu
Y
Chun
MM
Dissociable neural mechanisms supporting visual short-term memory for objects
Nature
 , 
2006
, vol. 
440
 (pg. 
91
-
95
)
Yantis
S
Multielement visual tracking: attention and perceptual organization
Cognit Psychol.
 , 
1992
, vol. 
24
 (pg. 
295
-
340
)