High Quality but Limited Quantity Perceptual Evidence Produces Neural Accumulation in Frontal and Parietal Cortex

Abstract

Goal-directed perceptual decisions involve the analysis of sensory inputs, the extraction and accumulation of evidence, and the commitment to a choice. Previous neuroimaging studies of perceptual decision making have identified activity related to accumulation in parietal, inferior temporal, and frontal regions. However, such effects may be related to factors other than the integration of evidence over time, such as changes in the quantity of stimulus input and in attentional demands leading up to a decision. The current study tested an accumulation account using 2 manipulations. First, to test whether patterns of accumulation can be explained by changes in the quantity of sensory information, objects were revealed with a high quality but consistent quantity of evidence throughout the trial. Imaging analysis revealed patterns of accumulation in frontal and parietal regions but not in inferior temporal regions. This result supports a framework in which evidence is processed in sensory cortex and integrated over time in higher order cortical areas. Second, to test whether accumulation signals are driven by attentional demands, task difficulty was increased on some trials. This manipulation did not affect the nature of accumulating functional magnetic resonance imaging signals, indicating that accumulating signals are not necessarily driven by changes in attentional demand.

evidence accumulation, fMRI, object recognition, perceptual decision making

Introduction

Understanding how sensory information is transformed into choice by a biological system is a fundamental challenge in science. Goal-directed perceptual decisions require the processing of sensory input, the evaluation of alternative solutions, and the commitment to a choice (Gold and Shadlen 2007) (Fig. 1). Research at neuronal (Shadlen and Newsome 2001; Romo and Salinas 2003; Wang 2008) and system (Heekeren et al. 2006; James and Gauthier 2006; Philiastides and Sajda 2007; Ploran et al. 2007; Heekeren et al. 2008; Wheeler et al. 2008; Ho et al. 2009; Kayser et al. 2010; Noppeney et al. 2010) levels indicates that perceptual choices are derived from information processing in brain areas that integrate sensory evidence over time. Activity in these neural “accumulators” has been found to increase at a rate that is predictive of response timing and, in neurophysiological data, response choice (Gold and Shadlen 2007).

Figure 1.

The outline depicts feed-forward flow of information in the decision-making process. Time courses on the right illustrate BOLD responses from prior work (Ploran et al. 2007) that may correspond to the theoretical decision processes on the left (e.g., Ratcliff 2002).

Recent functional magnetic resonance imaging (fMRI) studies of object recognition using a gradual unmasking approach have found patterns of accumulating signals in regions located in frontal, parietal, and temporal cortex (Carlson et al. 2006; James and Gauthier 2006; Ploran et al. 2007). In a study by Ploran et al. (2007), a pattern of accumulation in temporal, parietal, and frontal regions was signified by increased fMRI activity when the stimulus appeared, followed by a rising edge of activity that decreased in slope as decision time increased (Fig. 1). This pattern differed from a more transient pattern found in medial frontal and frontal opercular regions, in which activity did not change from baseline until the time of decision. Given that activity onset began at the time of decision, this pattern may reflect neural processing related to the commitment to a choice.

Given the conformity with random-walk and diffusion models (Link and Heath 1975; Ratcliff and Rouder 1998; Usher and McClelland 2001; Ratcliff et al. 2007) and consistency with fMRI studies using standard paradigms (Pleger et al. 2006; Tosoni et al. 2008; Ho et al. 2009; Kayser et al. 2010), the “accumulation” pattern of activity may reflect a neural integration-to-bound mechanism (e.g., Hanes and Schall 1996) in which the flow of information is gated at certain stages of processing. Decisions about object identity may be derived from signals accruing in those areas. However, it is necessary to consider factors other than evidence accumulation. For example, the gradual unmasking paradigm used in some studies (Carlson et al. 2006; James and Gauthier 2006; Ploran et al. 2007; Wheeler et al. 2008) increased the quantity of available perceptual information over time, possibly causing an accumulation of fMRI activity that reflects a gradual increase in the quantity of stimulus information rather than accumulated evidence. Another factor is the role of attention. Across paradigms, increases in fMRI activity as a decision becomes imminent may reflect other cognitive functions such as increased demands on attentional control.

To test the extent to which sensory processing contributes to accumulation effects in an extended object recognition paradigm, objects were masked with semirandomly spaced openings. By changing the location (but not amount) of the openings over time, subjects saw more of the object as the trial progressed. Critically, this technique maintained a consistent quantity of stimulus information throughout the trial, while it increased the overall perceptual evidence. If the observations of accumulation were a result of the gradual revelation procedure, and accompanying increase in available perceptual information over time, this new mask technique should eliminate accumulating fMRI signals. Alternatively, a pattern of accumulation should occur if it is a representation of increasing evidence supporting the impending decision. As a further test of an accumulation account, we also examined the pattern of activity on trials in which the item was not identified during the masking procedure. An accumulation, or integration-to-bound, account would predict a slower rate of accumulation on these trials than on trials in which an item is correctly identified during the masking procedure.

To investigate the extent to which attention or cognitive effort modulated the pattern of accumulating blood oxygen level–dependent (BOLD) signal, 2 objects were shuffled beneath the mask on some trials. On these “shuffle” trials, object information was inconsistent; each step alternated between revealing information for the same object or information for a similarly shaped object of a different identity. The consequence of this manipulation is that identifying the object is intended to be more difficult on shuffle than static presentation. If accumulation is a reflection of effort, shuffle trials should require greater effort and result in greater activity than trials in which the object remains the same throughout (static trials).

Materials and Methods

Subjects

Subjects were 18 healthy right-handed native English speakers with normal or corrected-to-normal vision. Two subjects were excluded from analyses due to excessive movement. The remaining 16 subjects (8 females) ranged in age from 19 to 31 years (mean, 23). Informed consent was obtained from all subjects according to procedures approved by the Institutional Review Board of the University of Pittsburgh. Subjects received $50 for participation.

Stimuli

Stimuli consisted of 102 grayscale images of common objects that were reformatted into a standard 284 × 284-pixel image with a white background (Rossion and Pourtois 2004). On most trials (“static” trials), one object was shown beneath a series of masks that revealed different parts of the object over time. Ninety objects were randomly selected for use on these trials. Twenty-four other objects were reserved for practice. For additional practice with 3 subjects, 20 new stimuli were added to better familiarize the task and ensure satisfactory behavioral performance. Two initial subjects viewed 102 stimuli, but the number of trials was reduced thereafter due to time constraints. Each subject received a randomly selected list of stimuli, ensuring that each viewed a unique set and order of objects.

On some trials (shuffle trials), the stimulus beneath the masks alternated between 2 similar objects. For these trials, there were 34 sets of objects, each composed of 2 grayscale object images (68 objects total) measuring 284 × 284 pixels. The 2 objects in a set were matched and edited to be of similar appearance by visual inspection (i.e., brightness, contrast, position, shape, size), despite clear membership in different object categories (e.g., whale and cucumber).

A set of masks was used to hide parts of the objects. Objects were presented within a 10.0 × 10.0 cm grid at the center of the screen with a white background. The remaining screen had a black background. Ten sets of 7 masks were created. Each mask was a 21 × 21 grid measuring 10.0 × 10.0 cm, the perimeter of which was black, leaving a grid of 19 × 19 squares. In each mask, 45 squares in the 19 × 19 grid were removed (∼12.5% of the total area), while the remaining squares were opaque (Fig. 2a). To prevent redundancy in stimulus presentation, square removal within a set was random without replacement. Over the course of a trial, a total of 315 squares in a mask were at some point transparent, while the remaining 46 were always opaque, allowing a total of 87.25% of the image to be seen. This arrangement was determined in a sequence of behavioral pilot experiments to produce a broad distribution of recognition times. Object–mask pairings and the order of masks within a set were randomized so that each object–mask combination was unique for each subject. The masks were placed in the foreground, the objects in the background.

Figure 2.

Behavioral task and results. (a) This diagram illustrates the task design. Every 2 s, the mask covering the object changed (T_R1–7), revealing different parts of the object underneath but without changing the quantity of visual information. (b) Percentages of trials with a response before VoA are plotted as a function T_R step, separately by trial type (static, shuffle). (c) The proportions of trials with a response before VoA for static and shuffle trials are plotted as a function of T_R and fit with logistic functions. Note that T_R8 trials were not included due to a scaling disparity (see text for results).

Behavioral Paradigm

Testing consisted of 4 runs of 31 trials (124 total trials per subject). Two initial subjects had 4 runs of 34 trials (136 total trials). On each trial, an object, or set of objects, was displayed for 16 s in 8 discrete 2-s steps. At trial onset, the object appeared under one of the 7 masks from a set. On static trials, the object remained the same while the mask changed at 2-s intervals until all 7 masks in a set were exhausted (Fig. 2a). On the eighth and final step, the mask was removed and the object was revealed. On shuffle trials, 2 objects alternated pseudorandomly behind the mask, with a maximum of 2 appearances in succession. On each shuffle trial, each object was viewed a total of 4 times. The order of presentation was randomized, ensuring each subject had a unique random order of shuffling for each set. On the eighth and final step of shuffle trials, the mask was removed and one of the 2 objects was revealed. For both trials types, each of the 8 steps corresponded with a whole-brain acquisition. Between each step, a 100-ms black screen was inserted to help disguise the object switch on shuffle trials. Thus, each mask was on the screen for 1900 ms. An exponential distribution of between-trial jitter of 2, 4, or 6 s (mean interstimulus interval = 3.16 s) was included to allow event-related analysis (Dale 1999). To obtain a consistent measure of the evolving task-related BOLD response, there was no within-trial jittering of step onsets.

Subjects were instructed to press a button when they could identify the object with a reasonable degree of confidence. Neither speed nor accuracy was emphasized in the instructions, and subjects were not specifically instructed to respond before the final step. When an object was fully revealed on the final (eighth) step, subjects pressed the same button again only if their earlier recognition had been correct. This response served as the verification of accuracy (VoA) of earlier recognition. If subjects were unable to identify an object during the masked portion of a trial, they were instructed to make a button press on the eighth step (hereafter referred to as “T_R8” trials). Thus, subjects generated 2 motor responses on correct trials, one at T_R and one at VoA. Subjects generated one response during incorrect trials (at T_R) and during trials in which recognition did not occur before VoA (T_R8). Assessing accuracy on shuffle trials was not straightforward because only one of the 2 shuffled objects was revealed at VoA. Correct shuffle trials were defined as those in which a recognition response was followed by a VoA response. Shuffle trials with a recognition response but no VoA response may have been correct, but because it was not possible to determine their accuracy, these trials were discarded from the main analyses.

Response hand was counterbalanced across subjects to help factor out lateralized motor signals in group analyses. E-Prime software was used for stimulus presentation and data collection (Psychology Software Tools). The cumulative distributions of behavioral responses were fit with a sigmoid function (Fig. 2c).

Image Acquisition

Images were obtained using a Siemens Trio 3-T scanner at the University of Pittsburgh Medical Center's Magnetic Resonance Research Center. The paradigm, presented by a PC running E-Prime, was projected onto a screen at the head of the magnet bore using a BrainLogics MRI Digital Projection System. The subject viewed the stimuli via a mirror attached to the RF coil. Earplugs were provided to minimize scanner noise. A fiber optic glove response pad connected to the PC via an interface box recorded responses (BrainLogics, Psychology Software Tools).

Anatomic images were obtained using a high signal-to-noise magnetization-prepared rapid-acquisition gradient echo sequence (repetition time [TR] = 2200 ms, echo time [TE] = 3.29 ms, flip angle [FA] = 9°, inversion time [TI] = 1000 ms). BOLD-sensitive functional images were acquired during performance of the task using a whole-brain spin-echo echo-planar T₂^*-weighted series (TR = 2000 ms, TE = 30 ms, FA = 79°, 3.2 × 3.2-mm in-plane resolution with 3.2-mm slice thickness). The first 3 image acquisitions of each run were discarded to allow net magnetization to reach steady state.

Data Analysis

Imaging data from each subject were preprocessed to remove noise and artifacts. Motion was corrected across and within runs using a rigid-body rotation and translational algorithm (Snyder 1996). Whole-brain functional data were then normalized to a mode of 1000 to facilitate intersubject comparisons (Ojemann et al. 1997). To account for differences in single slice acquisition times, slices were temporally realigned to the temporal midpoint of the first slice using sinc interpolation. BOLD data were then resampled into 2-mm isotropic voxels and transformed into stereotaxic atlas space by aligning an individual subject's T₁-weighted image to an atlas-transformed target T₁-weighted template using a series of affine transformations (Talairach and Tournoux 1988; Lancaster et al. 1995; Michelon et al. 2003; Fox et al. 2005).

After preprocessing, data were analyzed voxel-by-voxel using the general linear model approach for each subject (Friston et al. 1994; Miezin et al. 2000; Ollinger et al. 2001). BOLD data in each voxel at each time point were modeled as the sum of coded effects, produced by modeled events and by error. Event regressors were coded into each model at trial onset according to time of recognition (T_R1–8), accuracy (correct, incorrect), and trial type (static, shuffle), for a total of 32 possible events. Though trials were 16 s in duration, events were modeled over 32 s (16 time points) from trial onset to account for the slow hemodynamic response. Within each run, signal drift was modeled by a linear trend parameter, while baseline signal was modeled by a constant term. A series of delta functions described event-related effects as estimates of the percent of BOLD signal change from the baseline term. It is important to note that this approach makes no assumptions about the shape of the BOLD response. Software developed at Washington University (FIDL) was used for image processing and analyses (Ollinger et al. 2001).

To identify task-related activity during accurate performance on correct masked trials, a voxelwise repeated-measures mixed-effects analysis of variance (ANOVA) was computed on the correct static trial data, with subject treated as a random effect. In this analysis, recognition time across steps 3–7 (T_R3–7) was treated as a within-subject factor with 5 levels, and time was treated as a repeated factor with 16 levels of time point (beginning at trial onset). T_R1–2 trials were not included in the analysis due to insufficient numbers. T_R8 trials were also not included because, while subjects recognized the object on the last step, they made only one button press during the trial and it was not possible to score recognition accuracy on these trials. This analysis produced a separate image for each main effect and interaction term. To minimize bias effects in ROI selection, the main effect of time image was used to define regions of interest (ROIs). This image reflects the degree to which the temporal profile over the 16 time points differs from the baseline term (i.e., there is a change in signal magnitude over time), independently of T_R. The uncorrected z-transformed F-statistical image for the main effect of time was smoothed with a 4-mm full-width at half-maximum Gaussian filter and a threshold of z = 5.0 was applied. This map was then corrected for sphericity and multiple comparisons using a Monte Carlo method with a threshold of P < 0.05 at a 100-voxel extent (Wheeler et al. 2006).

To define ROIs, the uncorrected main effect of time image was further smoothed by convolving with a 4-mm hard sphere kernel, and an automated algorithm searched for the location of peaks exceeding P < 0.001 significance. Peaks separated by <10 mm were consolidated by averaging their coordinates. ROI volumes were grown up to a maximum 10-mm radius of contiguous voxels around the peak coordinates. Voxels failing sphericity and multiple comparisons corrections were then excluded from the ROIs. For practical purposes, ROIs comprising fewer than 100 voxels were excluded from analysis. This procedure defined 53 ROIs that were significantly active during correct trials. Time courses for correct T_R3–7 and T_R8 trials in the static condition, and T_R8 trials in the shuffle condition, were extracted from each ROI and used in second order analyses.

Hierarchical Clustering

To objectively separate and group ROIs according to the shape of their T_R-dependent responses, a hierarchical cluster analysis (Salvador et al. 2005) was used to classify region time courses. Five time courses of 16 time points each, corresponding to correct static trials at T_R3–7, were concatenated for each region and then entered into a single matrix for all 53 ROIs, resulting in a 53 × 80 matrix of time points. The Euclidean distances were computed between pairs of the objects in the matrix, and a hierarchical cluster tree was then generated from these distance vectors using Ward's linkage algorithm (Ward 1963) in the Matlab software package (Mathworks). This linkage method uses the inner squared distance of each pair in a minimum variance algorithm. The algorithm computes the increase in the within-cluster error sum of squares due to joining a pair of clusters. When this increase is minimized, those 2 clusters are joined. Because Ward's linkage uses an ANOVA approach, this method tends to yield a more reliable cluster tree than other methods (Dimitriadou et al. 2004). The distance vectors were then used to construct a dendrogram, a chart illustrating the correlation between regions and clusters of regions by length of the connecting line. Regions connected with shorter lines have greater similarity in their time course patterns and are thus more highly correlated than regions connected with longer lines.

After the initial dendrogram was generated, it was clear that the signal magnitude in some regions influenced how they were clustered. Because we wanted to cluster regions according to the timing and shape of their time courses rather than activity level (i.e., magnitude of signal change), the magnitudes were scaled across regions so that they would cluster according to their relative shapes. Signal change estimates at each time point were scaled to a minimum value of 0 and a maximum of 1. A scaling procedure was chosen in favor of a standardizing technique (e.g., z-scores) because standard scores would still exhibit region-specific differences in magnitude and therefore fail to address the issue. It is important to note that scaling was applied across regions and did not affect within-region time course differences. This preserved the relative shape and magnitude of each time course within a region, thereby still allowing within-region comparisons across conditions. The magnitude-scaled data were only used for the cluster analysis and not for analyses discussed in later sections.

Characterization of Waveform Patterns

In order to quantify specific properties of time courses pertaining to our hypotheses about accumulation rate, the time of activity onset and the slope of the leading edge of each time course were computed. Note that the cluster analysis does not provide a parametric test of the reliability of differences in these properties, so we also performed a series of analyses to test their reliablity. T_R8 trials were excluded from this analysis because, unlike T_R3–7 trials, they included only a single decision point.

Time of signal onset was computed in 2 stages. In the first, time courses were interpolated linearly. This procedure simply connected the 16 time points of a time course with a straight line of interpolated values. Along each line, 1000 points were generated, extending each initial 16-point time course to 15 000 points. By interpolating the data, the effective temporal resolution can be increased under the assumption that the values between any 2 time points can be estimated along a straight line. In the second stage, the goal was to determine the time at which activity increased reliably. Pairwise statistical comparisons between the interpolated time point value and a time course-specific estimate of baseline activity were computed. An estimate of baseline activity was computed for each T_R condition within each ROI as the mean of the first and last time points of its time course (i.e., not necessarily zero). A t-test was then performed at each interpolated time point along the time course, comparing the baseline estimate for that time course with the signal change at that time point, starting at time 0.0 s and advancing at steps of 0.001 s until the values differed at the P < 0.05 level. This point provided an estimate of the onset time of activity.

The slope of the leading edge of each time course was calculated within specific time windows for each cluster. Except for the fusiform/IT ROIs, the time windows were informed by observations from a prior study (Ploran et al. 2007). Slope was calculated as the rise (signal change) divided by the run (change in time across the window) in each ROI, for each T_R, and for each subject. Because the pattern of time courses differed markedly across clusters, it was necessary to establish cluster-specific time windows. When the pattern of time course changed as a function of T_R (e.g., commitment and accumulator ROIs), it was also necessary to use a shifting time window within a cluster to capture the leading edge of the BOLD response. The time window parameters are listed in Table 1. For sensory and fusiform/IT ROIs, slope was calculated between 2 and 6 s for each T_R. In accumulator ROIs, windows started at the 2-s mark and shifted endpoints according to T_R, with endpoints of 8, 10, 12, 14, and 16 s for T_R3–7, respectively. Constant window widths of 4 s were applied to commitment ROIs, with start points of 4, 6, 8, 10, and 12 s for T_R3–7, respectively.

Table 1

Time windows (in seconds) used to calculate the slope of the leading edge of each time course, broken down by T_R and by cluster

Cluster	TR3	TR4	TR5	TR6	TR7
Sensory	2–6	2–6	2–6	2–6	2–6
Fusiform/IT	2–6	2–6	2–6	2–6	2–6
Accumulator	2–8	2–10	2–12	2–14	2–16
Commitment	4–8	6–10	8–12	10–14	12–16

To test the reliability of these observations within and between the 4 clusters, each region's onset and slope estimates were entered into a repeated-measures ANOVA using 5 levels of the repeated-measure T_R (3–7). First, for onset times, a repeated-measures ANOVA tested for differences across the between factor of cluster (sensory, fusiform/IT, accumulator, commitment regions). Similarly, differences in slope measurements were tested across clusters using a repeated-measures ANOVA, with 4 levels of the cluster factor and 5 levels of the T_R factor. Separate 1 × 5 repeated-measures ANOVAs with polynomial trend analyses tested for reliable trends within the onset and slope values within each cluster. A main effect of recognition time would verify the reliability of mean differences across the 5 T_R levels (3–7), and the trend analysis determines whether the change in values across T_R is linear.

Slope Estimation Procedures

To test whether accumulation was faster on T_R3–7 than on T_R8 trials in the static condition, initial slope (in units of percent signal change divided by time) for each condition and region was estimated in each subject. Slope estimates were then averaged across all ROIs in each of the 4 positive clusters for each subject. A single factor ANOVA, with 6 levels of T_R was used to test differences between T_R3–8 trials within each cluster. Significant main effects of T_R were further investigated in pairwise analyses using the Tukey Honestly Significant Difference method.

To test whether the shuffle manipulation affected the rate of accumulation, slope estimates for correct static and shuffle T_R3–7 trials were compared for each cluster. Due to the low number of trials in the shuffle condition when data were divided into 2-s bins, data were averaged over all time bins prior to T_R8. Slope estimates were then calculated for each condition from the cluster averages within a 2–12 s window of activity. The slope estimates were entered into paired t-tests for each cluster to compare the initial slope depending on static or shuffle trial status.

Results

Behavioral Data

On static trials (n = 1474), subjects made recognition responses to 56.3% (standard deviation [SD] 13.4%) of the objects, only verification responses to 42.5% (13.4%) of objects (indicating no recognition until the object was fully revealed on these static Tr8 trials), and no response to 1.2% (3.0%) of objects. Given an initial recognition response, 73.5% (21.4%) received a verification response (VoA) to indicate accurate object identification. Correct responses were distributed widely throughout the trial period, with a relatively even spread over T_R2–7 (2–16 s) and a peak at T_R4 (Fig. 2b).

On shuffle trials (n = 534), subjects made fewer recognition responses overall, 44.4% (SD 18.3%). Given an initial recognition response, 52.7% (20.3%) received a verification response. Of the remaining total shuffle trials, 52.1% (16.2%) received only a verification response (shuffle T_R8 trials) and 3.5% (4.4%) received no response at all. Most correct responses were made at T_R4–7 (Fig. 2b). Compared with static trials, the shuffle trial response time distribution was shifted toward the end of the trial (Fig. 2c), reflecting an overall increase in response times on shuffle trials.

Subjects were not informed of the shuffling prior to the experiment. Postexperiment debriefing confirmed that 10 of the 16 subjects did not notice the shuffling. Six subjects reported noticing the shuffling, but none changed their approach to the task.

Task-Related ROIs

The ROI analysis described in the Materials and Methods identified 53 regions with significant changes in signal magnitude over time. Activations from the main effect of time map are shown in Figure 3, middle panels. ROIs defined from this map were distributed throughout the brain, with both positive and negative waveforms and a variety of temporal profiles. Regions with positive waveforms (Table 2) included lingual gyrus, precuneus, middle occipital gyrus (mOG), anterior insula/frontal operculum (aI/fO), intraparietal sulcus (IPS), and the medial portion of the superior frontal gyrus (mFG). Figure 3a–d shows static trial time courses on T_R3–7 trials from single ROIs, including right cuneus, fusiform gyrus, superior occipital gyrus, and aI/fO, respectively. The static T_R8 condition is plotted for comparison. Regions with negative waveforms were found in or near cuneus, precuneus, lingual gyrus, superior temporal gyrus, and posterior and anterior cingulate cortex (Supplementary Table 1). Both positive and negative regions showed a high degree of overlap with those found in our study using a similar perceptual decision paradigm (Ploran et al. 2007). Table 2 lists the region coordinates from both studies for comparison, along with a label denoting the cluster assignment (described in the following sections) in the 2 studies.

Table 2

ROIs with reliable increases in correct trial-related activity

ROI (#)	Anatomic location	Current study					Ploran et al. (2007)
		x	y	z	BA	Cluster	x	y	z	BA	Cluster
1	L cuneus	−28	−91	−2	18	Sens	−19	−99	−2	18	Sens
2	R inf occipital G	29	−88	−5	18	Sens	1	−93	−6	17	Sens
3	R mid occipital G	31	−87	7	19	Sens	—	—	—	—	—
4	L lingual G	−19	−92	−10	18	Sens	−10	−99	−5	18	Sens
5	R cuneus	22	−94	2	17	Sens	16	−99	−1	18	Sens
6	R fusiform G	44	−63	−6	37	IT	49	−61	−9	37	Accum
7	R inf occipital G	40	−76	−5	19	IT	—	—	—	—	—
8	R intraparietal S	27	−55	45	7	Accum	34	−57	47	7	Comm
9	L fusiform G	−40	−65	−11	19	IT	−42	−63	−9	37	Accum
10	R mid occipital G	33	−80	19	19	Accum	30	−78	17	19	Sens
11	R intraparietal S	25	−68	40	7	Accum	—	—	—	—	—
12	R sup occipital G	30	−73	28	19	Accum	31	−71	29	19	Accum
13	L mid occipital G	−32	−85	11	19	Sens	—	—	—	—	—
14	L fusiform G	−29	−70	−11	19	IT	−32	−89	−9	18	Accum
15	L precuneus	−24	−71	34	19	Accum	−30	−78	21	19	Accum
16	L intraparietal S	−24	−57	45	7	Accum	−26	−68	38	7	Accum
17	R inf parietal	39	−40	45	40	Comm	49	−48	47	40	Comm
18	L fusiform G	−30	−55	−15	37	IT	−31	−39	−14	20	Accum
19	L ant insula	−30	19	2	13	Comm	−32	22	01	13	Comm
20	R ant insula	30	21	4	13	Comm	33	22	−2	13	Comm
21	R fusiform G	31	−68	−9	19	IT	—	—	—	—	—
22	L med frontal G	−7	13	48	6	Comm	−1	14	51	6	Comm
23	R med frontal G	5	15	47	6	Comm	1	26	42	8	Comm
24	R fusiform G	28	−46	−16	36	IT	—	—	—	—	—
25	R fusiform G	33	−55	−17	37	IT	—	—	—	—	—
26	L inf frontal G	−43	2	32	9	Accum	−46	0	32	9	Accum
27	L inf parietal	−42	−42	44	40	Comm	—	—	—	—	—
28	R mid frontal G	30	−8	47	6	Accum	—	—	—	—	—
29	R inf frontal G	39	5	28	9	Accum	44	6	33	6	Accum
30	R postcentral G	55	−18	34	2/40	Comm	—	—	—	—	—
31	L ant cingulate G	−5	25	39	32	Comm	—	—	—	—	—
32	R ant cingulate G	9	21	35	32	Accum	6	24	31	32	Comm
33	R inf frontal G	44	16	1	47	Comm	45	14	−3	47	Comm
34	R postcentral G	50	−23	42	2	Comm

Notes: Regions are from the current study (left half) and comparable regions found in prior work (Ploran et al. 2007). L, left; R, right; Ant, anterior; Sup, superior; Inf, inferior; Med, medial; Mid, middle; G, gyrus; S, sulcus; x,y,z, Talairach atlas coordinate dimensions; BA, approximate Brodmann's area; Sens, sensory; Accum, accumulator; IT, Fusiform/IT; Comm, commitment. Cluster assignment determined by hierarchical cluster analysis. Anatomic locations are approximate.

Figure 3.

Task-related activation and T_R-dependent time courses from representative ROIs. In the middle panels, horizontal anatomic slices are overlaid with above-threshold voxels in the main effect of time map (see text for details). Reliability of activation is denoted by the scale bar in standardized units. ROIs were derived from this map. Time courses from conditions T_R3–8 are plotted from right cuneus (a), right fusiform gyrus (b), right superior occipital gyrus (c), and right anterior insula (d) ROIs.

Hierarchical Cluster Analysis of Temporal Profiles

A hierarchical cluster analysis was used to objectively sort the 53 regions according to correlations among the temporal profiles on correct static trials. This initial clustering resulted in 34 regions with overall positive activity and 19 regions with overall negative activity (Supplementary Fig. 1). While it will ultimately be of interest to evaluate the role of regions with negative time courses in the formation of a decision (Donaldson et al. 2010), we focus here on regions with positive time courses to test hypotheses derived from our previous research (Ploran et al. 2007). A second cluster analysis was then performed using only the 34 positive regions (Fig. 4a). Note that this procedure produced the same ordering of regions as the first cluster analysis and was done for illustration purposes only. Pruning the cluster tree at a Euclidean distance of 2.5 yielded 4 distinct clusters of regions (Fig. 4a). The pattern of activity across clusters was evaluated on T_R3–7 trials in the static condition by averaging across all ROIs in the cluster. These data are presented in Figure 4b–e. Again, the static T_R8 condition is plotted for comparison.

Figure 4.

Hierarchical cluster analysis and averaged T_R-dependent time courses. (a) The dendrogram graphically depicts the similarity of time courses for positive-going ROIs (y-axis) in terms of Euclidean distance (x-axis) as determined by the hierarchical cluster analysis. Linkages at a small distance value depict similar time courses. The 4 ROI groups were defined by pruning the tree at a distance of 2.5, indicated by a dotted line. Numbers in parentheses refer to the region numbers listed in Table 2. (b–e) BOLD time courses of static trials for T_R3–7 are averaged across all ROIs in each color-coded cluster group. Time courses indicate percentage signal change from baseline (0%, shown as horizontal dotted line) over time (in seconds). Color-coded squares on the x-axis depict the subjects' actual time of response.

In one cluster (Fig. 4, red), an early onset of activity was followed by a rapid increase in activity for all conditions, after which activity persisted at a plateau for the duration of the trial. Activity returned to baseline at stimulus offset, and there were no obvious effects of T_R on the pattern of time course (Fig. 4e). Due to the absence of T_R-dependent activity and the location of these regions in the occipital lobe (Fig. 5, red regions), we attribute this pattern of activity to low-level sensory processing (labeled “Sens” in Table 2).

Figure 5.

Anatomic distribution of cluster groups. ROIs classified in sensory (red), fusiform/IT (purple), accumulator (blue), and commitment (green) clusters are shown projected onto inflated cortical surfaces using Caret software (Van Essen et al. 2001). ROI numbers in parentheses correspond to those in Table 2. L, left hemisphere; R, right hemisphere.

In a second cluster of regions (Fig. 4, green), there was a shift in both onset and peak times according to T_R, such that earlier recognition decisions were associated with earlier onset and peak than were later recognition decisions (Fig. 4b). Regions in this cluster were located in or near medial portions of the superior frontal gyrus near the pre-SMA, the anterior cingulate cortex (ACC), bilateral aI/fO, bilateral inferior parietal lobule (iPL), and the right postcentral gyrus (PoCG; Fig. 5, green regions). Based on the high sensitivity of these regions to response timing in both onset and peak measures, it is plausible that their function is associated with the commitment to a recognition decision (labeled “Comm” in Table 2). One interesting finding was that, whereas in the past study activity in these regions remained at baseline until T_R, in this study, there was a clear increase above baseline prior to T_R.

In a third cluster (Fig. 4, blue), activity onset was early in the trial (similar to the sensory group), but the slope of the leading edge of activity appeared to decrease across levels of T_R (Fig. 4c). The peak in activity occurred approximately 4 s after the recognition button press. These regions were found in bilateral parietal lobes along the length of the IPS, middle (mOG) and superior (sOG) occipital gyri, inferior frontal gyrus (IFG), right ACC, and middle frontal gyrus (mFG; Fig. 5, blue regions). The combination of an increase in activity early in the trial and a T_R-dependent rate of increase are consistent with an accumulation account (labeled “Accum” in Table 2).

A fourth cluster of regions (Fig. 4, purple) was located almost exclusively in bilateral IT, including the fusiform gyrus (Fig. 5, purple regions). This group showed an early activity onset (similar to the sensory regions), with a separation in rate of increase occurring at ∼6 s. The time of peak activity correlated with response timing (Fig. 4d). Regions in this cluster are labeled “IT” in Table 2.

Because some temporal profiles from different clusters appeared similar upon visual inspection, it is worth noting the distance values displayed in the dendrogram in Figure 4. The IT and accumulator clusters were the closest in Euclidean distance (2.83). Time courses in both clusters had early onset times, and their peak times increased with T_R. These 2 clusters were closest in Euclidean units (3.42) to the sensory cluster. Activity in this cluster had an early onset and extended throughout the trial. In contrast, the “commitment” cluster was farthest away from the other 3, at a distance of 4.78 units. Both onset and peak times in this cluster appeared to increase monotonically with T_R.

Analysis of Waveform Properties

Onset Times

To statistically analyze the reliability of differences in temporal profiles across the clusters, time of signal onset and a measure of initial slope were computed (see Materials and Methods). The signal onset measure was the time point at which signal significantly rose above the baseline estimate in each region (Fig. 6a). In the sensory cluster, onset occurred early and showed little difference across T_R3–7 (mean = 2.09, 1.51, 1.85, 2.00, 1.90 s, respectively). The fusiform/IT cluster also had relatively early onset times with little difference across T_Rs (mean = 2.85, 1.95, 2.68, 2.91, 2.80 s). The accumulator cluster had slightly later onset times, but there was little change in values across T_Rs (mean = 2.50, 2.67, 3.14, 3.12, 3.20 s). The commitment cluster had the longest onset times, and in contrast to the other cluster the onsets increased across levels of T_R (mean = 3.97, 3.90, 5.10, 4.6, 6.0 s). An ANOVA of onset times testing for differences between the 4 clusters (see Materials and Methods) displayed a significant main effect of T_R (F_4,120 = 10.53, P < 0.0001), indicating that onset times differed across levels of T_R. This effect was modulated by cluster, as revealed by a significant interaction of T_R with cluster (F_12,120 = 3.68, P < 0.0001), indicating that the pattern of onset times differed reliably across clusters. The main effect of the between-factor cluster was also significant (F_3,30 = 25.03, P < 0.0001). Subsequent pairwise t-tests comparing each pair of clusters displayed significant differences in onset times between commitment ROIs and all other groups (all P < 0.0001), sensory and accumulator groups (P = 0.006), and sensory and fusiform/IT groups (P = 0.05), but no differences between the fusiform/IT and accumulator groups (P = 0.39).

Figure 6.

Analysis of waveform properties. Mean onset times (a) and slope values (b) for T_R-dependent time courses are displayed as a color-coded line for sensory (red), fusiform/IT (purple), accumulator (blue), and commitment (green) ROI clusters. Error bars indicate standard error of the mean.

To test whether onset times increased linearly with T_R, 1 × 5 repeated-measures ANOVAs were computed for each cluster. Onset times in sensory ROIs differed across T_R (F_4,20 = 4.05, P = 0.014) but did not follow a linear trend (F_1,5 = 0.71, P = 0.71). Fusiform/IT ROIs showed a significant main effect across recognition times (F_4,28 = 10.85, P < 0.0001) and a marginal linear trend (F_1,7 = 5.38, P = 0.053). Subsequent pairwise comparisons in this cluster revealed that this effect and linear trend were due to the onset times of T_R4, which were significantly lower than the values for the other T_Rs (all P < 0.01). The accumulator group did not show a significant main effect (F_4,36 = 2.01, P = 0.12) but showed a reliable linear trend (F_1,9 = 8.42, P = 0.018). Commitment ROIs, conversely, showed both a significant main effect (F_4,36 = 9.97, P < 0.0001) and upward linear trend (F_1,9 = 29.66, P < 0.0001), indicating that onset times in regions of this group showed a strong linear increase as a function of T_R.

Slope Measurements

The slope of the leading edge of the time course was computed within predefined time windows (Table 1) to test for a decreasing rate of accumulating activity across T_R (Fig. 6b). Slope measurements did not change across T_R for sensory (mean = 0.34, 0.29, 0.33, 0.31, 0.30% signal change/s), fusiform/IT (mean = 0.21, 0.17, 0.18, 0.16, 0.16% signal change/s), and commitment time windows (mean = 0.10, 0.09, 0.07, 0.11, 0.10% signal change/s). In contrast, mean slope measurements decreased across T_R for the accumulator time windows (mean = 0.14, 0.14, 0.11, 0.09, 0.07% signal change/s).

An ANOVA testing mean differences across the 5 T_Rs and 4 clusters (see Materials and Methods) revealed a significant main effect of cluster (F_4,60 = 39.49, P < 0.0001), a main effect of T_R (F_4,60 = 5.67, P = 0.001), as well as a cluster by T_R interaction (F_16,240 = 2.47, P = 0.002), indicating that slope measurements differed across T_R and cluster. Subsequent pairwise comparisons of cluster revealed differences between the sensory cluster and all others (all P < 0.0001) and between the fusiform/IT slope and all other clusters (all P < 0.01). The mean slope comparison between the accumulator and commitment groups was not statistically significant (P > 0.11).

Separate 1 × 5 ANOVAs were computed to examine within-cluster effects of T_R and identify whether there was a decrease in slope across T_R. There was no main effect of T_R in the sensory cluster (F_2.56,38.34 = 0.66, P = 0.56), and there was no linear trend (F_1,15 = 0.44, P = 0.51), indicating that the mean slope measurements in this cluster did not change across T_R. Similarly, the fusiform/IT cluster showed neither a main effect of T_R (F_2.18,32.66 = 0.62, P = 0.56) nor linear trend (F_1,15 = 2.16, P = 0.16). Accumulator ROIs exhibited a main effect of T_R (F_2.36,35.35 = 6.11, P = 0.004) and decreasing linear trend across T_R (F_1,15 = 12.86, P = 0.003), indicating that the slope in these ROIs was steep for early response time courses and became progressively shallower for later T_Rs. Conversely, commitment ROIs displayed neither a main effect (F_2.17,32.60 = 0.54, P = 0.602) nor a linear trend (F_1,15 = 0.07, P = 0.792), indicating that the slope did not change across T_R for ROIs in this cluster. In sum, the statistical analyses support the qualitative observations noted in the section describing the cluster analysis.

Comparing Slopes from T_R3–7 and T_R8 Static Trials

Next we compared the slopes of the initial increase in activity for correct T_R3–7 trials and T_R8 trials in each cluster (see Materials and Methods). As shown in Figure 4b–e, the initial slope on T_R8 trials was similar to the slope on T_R3–7 trials in the sensory and fusiform/IT clusters but less in the accumulation cluster. Visual inspection of activity in the commitment cluster (Fig. 4b) suggests that the slope on T_R8 trials was also less than T_R3–7 trials, though we predicted the opposite. However, an ANOVA revealed a significant main effect of T_R only for the accumulator cluster (F_2.43,34.04 = 8.60, P < 0.01). Subsequent pairwise comparisons revealed that slopes in this cluster were less on T_R8 trials than T_R3, T_R4, and T_R5 trials (all P < 0.02) and less on T_R7 than T_R4 trials (P < 0.03). No other cluster, including the commitment cluster, demonstrated a significant change in the initial slope (all P > 0.25).

Comparing the Slope of Increasing Activity on Shuffle and Static Trial

An analysis of the slopes on static and shuffle trials (see Materials and Methods) revealed no effect of trial type within the accumulator cluster (t₁₅ = 0.57, P = 0.57). The left panel of Figure 7 shows the slopes for each ROI in the accumulator cluster, plotted as a function of condition. The right panel displays the cluster-average time courses for each condition. Paired t-tests for the other clusters also failed to reach significance. The results were the same when both correct and incorrect shuffle trials were included.

Figure 7.

Accumulator cluster slopes on shuffle and static T_R3–7 trials. Left, plotted are the static and shuffle slope values for each ROI, connected with a line. Mean values for the accumulator cluster are shown in black. Right, the gray box shows the analyzed time window (2–14 s) for the time courses for static and shuffle VoA trials averaged across all ROIs in the accumulator cluster.

Discussion

In this study, we used an extended object recognition task to test 2 hypotheses about the nature of accumulating BOLD fMRI signals. Critically, we found patterns of dynamically accumulating signals, prior to a decision, in frontal and parietal regions when the quantity of stimulus inputs was held constant. Lending further support to an accumulation account, activity in these regions was significantly greater when objects were recognized during the masking procedure than when they were not. We also found that shuffling objects increased response times but did not affect the rate of accumulation of activity in accumulator regions, suggesting that the accumulation effects were not driven by task difficulty. In contrast to past findings (Ploran et al. 2007; Wheeler et al. 2008), regions in the inferior temporal lobes, including the fusiform gyrus, failed to accumulate activity as defined by an early rise in activity accompanied by a T_R-dependent decrease in the slope of the leading edge of activity. Combined, the current findings support an accumulator account in which parietal and frontal regions integrate sensory inputs over time. These novel findings provide empirical support for models of perceptual decision making in which evidence is extracted from sensory areas, such as MT (middle temporal area) and IT, and integrated over time in higher order processing centers (Kiani et al. 2006; Lo and Wang 2006; Gold and Shadlen 2007). In the next sections, we discuss the findings and their relevance to behavioral choice.

Dissociating Accumulation and Commitment Signals

Holding stimulus quantity constant during revelation, we found a pattern of accumulating signals in frontal and parietal regions that were located in close proximity to the accumulator regions described in our previous study (Fig. 4c and Table 2). In these regions, activity onset occurred shortly after trial onset and increased at a T_R-dependent rate such that the leading edge of activity followed a sharper leading slope for shorter than for longer T_Rs (Fig. 6). This pattern of activity is consistent with an integration-to-bound framework in which sensory choice occurs after activity in neural “accumulators” reaches threshold. By this account, activity should be greater (i.e., above threshold) when items are recognized than when they are not (i.e., below threshold). This prediction is supported by 2 observations in the current study. The first is related to the inverse relationship between the rate of accumulating activity and T_R (as illustrated in Fig. 6b). On correct recognition trials (T_R3–7), activity was greater for items identified at a given point in time than for items not yet identified. This can be illustrated in Figure 4c by placing an arbitrary signal change boundary at a relatively high level, such as 0.3% or 0.4%, and noting that activity for a given T_R condition is greater at that point in time than for subsequent T_R conditions. Second, activity was significantly slower to accumulate during T_R8 trials compared with T_R3–5. Thus, the pattern of data indicates that a failure to recognize an object was related to insufficient levels of activity in regions in the accumulator cluster.

The pattern of activity in the dorsal ACC, pre-SMA, iPL, and aI/fO was broadly consistent with the previously reported commitment pattern (Ploran et al. 2007), with increasing onset and peak times that correlated significantly with T_R. There was also an interesting difference. Whereas in a previous study, activity remained remarkably unchanged from baseline until T_R, in the current study, activity increased above baseline before T_R, on average (across T_R) ∼6 s earlier than in the previous study. Given the earlier onset of activity, it may be difficult, upon visual inspection, to appreciate differences in the patterns of activity in the 2 clusters (Fig. 4b,c). However, several findings support a dissociation. First, the commitment ROIs clustered the furthest (in Euclidean distance) from any of the other positive-going clusters, including the accumulator ROIs (Fig. 4a). This outcome indicates that the pattern of activity in the commitment cluster was less correlated with the pattern of activity in the accumulator cluster than either the fusiform/IT or sensory clusters. Second, the slope of the leading edge of activity in commitment regions did not change reliably as a function of T_R, as it did in the accumulator regions (Fig. 6b). The absence of a difference in slope indicates that, despite earlier onset times, the greatest demand for neural processing in the commitment cluster occurred near the time of recognition.

One explanation for the earlier onset of activity in regions in the commitment cluster is that the current task required a greater degree of effort (e.g., stimulus maintenance in working memory across steps, hypothesis testing, etc.) than the gradual revelation task. This explanation is plausible because regions in the commitment cluster have often been implicated in various aspects of cognitive control. For example, the ACC and aI/fO have been implicated in task-level control (Dosenbach et al. 2006; Dosenbach et al. 2008; Velanova et al. 2008; Nelson et al. 2010), awareness (Craig 2009), and performance monitoring (Carter et al. 1998; Ito et al. 2003; Ridderinkhof et al. 2004; Oliveira et al. 2007; Eichele et al. 2008; Wheeler et al. 2008), all of which were in greater demand in the current task.

Domain-General and Domain-Specific Accounts

An open question about the role of accumulator and commitment regions is their specificity of function. One possibility is that their function generalizes across domains of information. For example, by this account the operation of accumulators is to integrate information over time, regardless of source of that information. An alternative explanation is that accumulating signals occur in domain-specific regions in which information processing relevant to the decision occurs. The selection of relevant information can be influenced by the organism and the available choices and may be more useful in some circumstances than in others. It is possible to assess the degree of domain-specificity by observing the consistency with which a region accumulates activity across tasks. A domain-general account would be supported by the recruitment of a region across different tasks involving different types of information and response options. A specificity account would be supported by a strict mapping between regions showing accumulation effects and type of task and response options. In this section, we consider domain-general and domain-specific accounts by comparing this task with our previous task (Ploran et al. 2007).

To begin, there were some task-related differences in the location of regions showing commitment and accumulator patterns of activity (Table 2). The thalamus and striatum showed reliable commitment patterns of activity in the past study. However, theses regions were not observed in the present study. An inspection of the uncorrected statistical map revealed no significant clusters of activity in thalamus and dorsal striatum, indicating that those regions were not overlooked by the ROI selection procedure. These regions are commonly reported in studies of decision making, so it is surprising that they were not active in the present study. Another difference was that, in the current study, regions in both right and left iPL were included in the commitment cluster (Fig. 4, ROI #17, 27), whereas only the right iPL was included in this cluster the previous study. The iPL ROIs were located immediately posterior to the precentral gyrus, in proximity to regions that have been associated with motor behaviors directed toward a target, such as pointing and grasping (Binkofski et al. 1998; Connolly et al. 2000; Culham et al. 2003; Castiello 2005; Frey et al. 2005). Anterior parietal regions are thought to be involved in computing spatial coordinates subserving these behaviors (Culham et al. 2003), so activity in these regions may be related to the generation of a motor response following the commitment to a decision.

In both studies, we found an accumulator pattern of activity in bilateral mOG, IFG, and the IPS. The left IFG region (Table 1, #26) is close to a region reported by Noppeney et al. (2010) that accumulated audiovisual evidence in a multimodal sensory choice task. Two regions previously classified as commitment regions were instead classified as accumulator regions in the current study. These regions were located in the right superior parietal lobe and the right dorsal ACC (Table 2, ROI #8 and #32). The change in classification may represent between-study error variance (i.e., ROI selection procedures) or may represent real differences in the nature of processing between the 2 tasks. However, these differences are minor considering that most cluster classifications were stable across the 2 studies.

Finally, the hierarchical analysis identified a cluster of regions with a markedly different pattern of activity than those observed previously. Regions in this cluster were located exclusively in or near IT and the fusiform gyrus (Fig. 4). Some of these ROIs were near previously described IT accumulators (Wheeler et al. 2008) but others were novel to the current task (Table 2). Notably, the slope of the leading edge of activity in fusiform/IT was the same across all T_Rs. Due to the location of these regions in the ventral visual processing stream (Ungerleider and Mishkin 1982; Felleman and Van Essen 1991), and correlation of peak activity with T_R, it is plausible that their initial response was related to the onset of visual stimulation, and their later response was related to the presence (or absence) of relevant object features. We speculate that the between-task differences in the pattern of BOLD activity in these regions are most likely due to strategic factors arising from changes in stimulus presentation. In the previous study, the overall object shape could be perceived gradually and monotonically, providing a sense of object identity before individual features were perceptible. This type of presentation may have produced an increasing BOLD signal in visual regions that respond to domain-specific requirements of object recognition (e.g., Gauthier et al. 1997). The new masking technique instead revealed disjointed individual features and imparted a lesser degree of overall object shape, thus possibly delaying activity in fusiform/IT regions until readily identifiable features (e.g., geons) appeared (Grill-Spector et al. 2001; Haxby et al. 2001; Kanwisher 2003). The absence of an accumulator pattern in the fusiform/IT regions in the current study is consistent with proposals that sensory processing areas do not integrate information over time (Gold and Shadlen 2007).

It should be emphasized that, despite modifications in the masking procedures, significant accumulation effects were observed in many higher order regions, such as IFG and IPS, suggesting a domain-general mechanism that operates across object recognition tasks. Whereas the previous gradual unmasking approach encouraged a passive mode of evaluation as objects resolved on screen, the current task required an increased demand to maintain perceptual evidence throughout the trial. This demand may have been met by the same domain-general neural resources used in visual working memory tasks, represented by increased recruitment of frontal and parietal regions. In particular, regions in or near the occipital lobes, right superior and left iPL, the fusiform gyrus, and left middle frontal gyrus are near those that modulate according to information load in visual working memory tasks (Todd and Marois 2004; Ranganath et al. 2005).

Accumulation Reflects Impending Decision Rather than Task Difficulty

To test whether accumulation effects were modulated by changes in attentional control, we manipulated the consistency of object identity. Behaviorally, subjects took longer to respond on shuffle trials, indicating that shuffle trials were more difficult than static trials. If accumulation reflects the degree of effort exerted during the task, we should observe greater activity on shuffle than static trials (Heekeren et al. 2004). However, the absence of a steeper initial slope on shuffle trials compared with static trials (Fig. 7) indicates that the accumulating signals do not reflect increasing attentional demands.

While the absence of slope differences supports an accumulation account instead of an attention/effort account, one might predict, in contrast, that the slope of activity on shuffle trials would be less than on static trials. On shuffle trials, subjects saw 2 different pictures and as a consequence had fewer opportunities to gather evidence about either picture than they did on static trials. This finding raises the possibility that the slope of accumulating activity was not related to the amount of evidence integrated over time. However, there are several reasons to doubt this possibility. First, because most subjects were unaware that the pictures were changing, they were just as likely to attempt to integrate the disjointed features into a single-object solution on shuffle trials as they were on static trials. Second, because correct trials from both conditions were analyzed according to their T_R, and an accumulator model would predicate that activity reach threshold prior to the commitment to a decision, then there should be no difference in the rate of accumulation. In both trial types, recognition was accurate, reflecting an integration to boundary of evidence about object identity, regardless of the number of times the object was presented.

Extended Task Paradigm Complements Standard Event-Related Approaches

The results obtained here overlap considerably with those from other research groups using different approaches. Regions near the anterior insula and medial frontal lobe, including the dorsal ACC and pre-SMA, have been reported to be more active for difficult than easy decisions in studies using line length judgments (Grinband et al. 2006), fear-disgust discrimination (Thielscher and Pessoa 2007), and motion discrimination (Ho et al. 2009). In addition, the IPS has been found to be more active for difficult decisions in motion discrimination (Ho et al. 2009) and shape prediction (Huettel et al. 2005). While our results were consistent with the results of these rapid event-related approaches, the extended paradigm allowed us to examine dynamic changes in the time course despite the low temporal resolution of the BOLD signal. The time course analysis dissociated commitment patterns of activity in the anterior insula and medial frontal lobe from accumulator patterns in the IPS and prefrontal cortex, suggesting different roles in the decision process.

It is worth noting that predictions about the relationship between accumulating neural activity and the BOLD signal can depend on the structure of the fMRI paradigm. In some studies using rapid event-related designs, the magnitude of BOLD signal has been hypothesized to reflect the rate of neural accumulation or the amount of perceptual evidence (e.g., Tosoni et al. 2008; Ho et al. 2009; Kayser et al. 2010; Noppeney et al. 2010). This prediction about the relationship between accumulation and the BOLD signal is derived from the fact that, in these designs, the neural events occur rapidly and the sum is reflected in the BOLD response. Using this approach, it can be difficult to determine whether changes in activity are due to changes in sensory evidence or to other factors such as arousal, task difficulty, or effort, which have been associated with increased activity across a range of cognitive tasks (Garavan et al. 2002; Velanova et al. 2003; Wheeler and Buckner 2003; Grinband et al. 2006; Thielscher and Pessoa 2007). In the gradual paradigm, because the underlying neural events are more extended in time, different rates of accumulation can instead be measured by changes in the slope of the leading edge of the BOLD response. Changes in magnitude can thus be evaluated orthogonally to slope changes and provide leverage to address confounds such as effort and attention.

Conclusions and Future Directions

This study tested an accumulation hypothesis of perceptual decision making. Because stimulus inputs were consistent throughout the trial, the observation of patterns of accumulating activity in parietal and frontal regions cannot be explained by a systematic increase in stimulus quantity. This approach identified a new pattern of activity in fusiform/IT regions that was not consistent with an accumulation account. Instead, our findings are consistent with the framework posited by Gold and Shadlen (2007) in which evidence is extracted from sensory processing areas and integrated over time in higher order regions. It is worth noting here that while the current work assumes a feed-forward mechanism of evidence accumulation, it will be important to also consider the implications of alternative accounts, such as recurrent feedback and attractor states in neuronal circuits (Wang 2008). Such models can account for the reiterative type of processing that likely occurred in the extended paradigm.

A next step toward testing whether different patterns of activity represent discrete stages of a decision hierarchy is to use converging evidence from other tasks and approaches. As noted above, if the data represent both domain-specific (e.g., fusiform/IT regions) and domain-general (e.g., frontal/parietal accumulator regions) processing of evidence, we should find consistent recruitment of domain-general areas while the domain-specific areas change with task demands. For example, processing of evidence in motion discrimination (e.g., Shadlen and Newsome 2001) should involve the middle temporal area MT instead of IT, while the integration of that evidence should occur in parietal and frontal accumulator regions. Recent findings from an fMRI study of motion discrimination already offer support for this view. Kayser et al. (2010) found that activity in/near the IPS, but not area MT, was consistent with an accumulation hypothesis. Using a gradual approach, it would be possible to assess accumulation dynamically as a decision forms and determine how the amount of available evidence is related to decision outcome.

Supplementary Material

Supplementary material can be found at: http://www.cercor.oxfordjournals.org/

Funding

National Institute of Mental Health at the National Institutes of Health (MH086492 to M.E.W.).

Steve Petersen and Gordon Shulman provided helpful discussion. Abraham Snyder and Mark McAvoy provided imaging analysis software and support and Kate Fissel and Max Novelli assisted with image processing and data management. Some stimulus images courtesy of Michael J. Tarr, Center for the Neural Basis of Cognition and Department of Psychology, Carnegie Mellon University, http://www.tarrlab.org/. Conflict of Interest: None declared.