Abstract

Objects seen from unusual relative to more canonical views require more time to categorize and recognize, and, according to object model verification theories, additionally recruit prefrontal processes for cognitive control that interact with parietal processes for mental rotation. To test this using functional magnetic resonance imaging, people categorized and recognized known objects from unusual and canonical views. Canonical views activated some components of a default network more on categorization than recognition. Activation to unusual views showed that both ventral and dorsal visual pathways, and prefrontal cortex, have key roles in visual object constancy. Unusual views activated object-sensitive and mental rotation (and not saccade) regions in ventrocaudal intraparietal, transverse occipital, and inferotemporal sulci, and ventral premotor cortex for verification processes of model testing on any task. A collateral–lingual sulci “place” area activated for mental rotation, working memory, and unusual views on correct recognition and categorization trials to accomplish detailed spatial matching. Ventrolateral prefrontal cortex and object-sensitive lateral occipital sulcus activated for mental rotation and unusual views on categorization more than recognition, supporting verification processes of model prediction. This visual knowledge framework integrates vision and memory theories to explain how distinct prefrontal–posterior networks enable meaningful interactions with objects in diverse situations.

Introduction

People can quickly and accurately categorize objects (e.g., horse, plane, or pen) and recognize them as familiar, despite changing viewing conditions, an ability known as object constancy. Yet, people do worse when visual input is relatively impoverished, as when objects are occluded or viewed at unusual angles (Palmer et al. 1981; Tarr et al. 1998). Of the few vision frameworks explaining object cognition under impoverished input, perceptual hypothesis testing theories propose that top-down processes in a frontoparietal network drive iterative processes of object model verification that determine the stored structural model in posterior object-sensitive cortex that best accounts for the visual percept (Lowe 1985, 2000; Kosslyn et al. 1994; Grossberg 1999; Lee and Mumford 2003; Ganis et al. 2007). One class of such theories, multiple views plus transformation (MVPT) (Bülthoff et al. 1995; Ullman 1996), would predict that the role of the parietal part of the network is a transformation process, such as mental rotation (Tarr and Pinker 1989), that is required for unusual views of objects to bring the visual input and stored representation(s) into sufficient visuospatial correspondence to achieve a match. To test key predictions of model verification and MVPT, brain activity was compared between canonical (best) and unusual views of real objects during categorization and recognition using functional magnetic resonance imaging (fMRI). Although vision frameworks tend to posit generalization across tasks, memory frameworks predict task dissociations because distinct brain systems support perceptual and semantic, generic memory for categorization versus episodic memory for recognition (Hintzman 1978; Schacter and Tulving 1994).

Feedforward computational models explain rapid detection of a target category (i.e., animal or not), but model verification theories can better explain object cognition under impoverished viewing conditions (Serre et al. 2007). According to the latter, impoverished images of objects recruit model verification processes implemented in ventrolateral prefrontal cortex (VLPFC) and parietal networks that are involved in top-down control of long-term memory retrieval, working memory, and covert attention processes and modulate activity in ventral object-sensitive regions, with greater engagement as image impoverishment increases (Ganis et al. 2007). Verification processes are recruited after the initial bottom-up pass through the visual system provides only weak candidate object models for the match with the more impoverished input. Similarly, MVPT and related psychophysical and computational accounts also include iterative model testing, but posit processes and representations that enable more specific hypotheses about the role of posterior cortex, despite being largely agnostic about neural correlates (Bülthoff et al. 1995; Peissig and Tarr 2006). Object constancy is posited to be achieved by storing multiple views of an object, consistent with findings of view-dependent performance on diverse tasks (Tarr et al. 1998). A familiar or canonical view can be matched directly to stored views. When an object is instead encountered from a noncanonical, unusual, or unfamiliar viewpoint, an extra process of mental rotation (or transformation, normalization, or alignment) is needed to compensate for any shape differences between the perceived input and stored views to enable them to be matched to each other (Palmer et al. 1981; Tarr and Pinker 1989; Ullman 1989).

Here, we focus on testing the MVPT variant that implicates the core process underlying the well-studied spatial ability of mental rotation on object cognition (Tarr and Pinker 1989). On mental rotation tasks (Shepard and Cooper 1982), 2 objects appear at different orientations and are either identical or left–right mirror images of each other. Response times (RTs) and error rates to judge whether the 2 objects are the “same” or “different” increase linearly as the orientation disparity between the 2 objects increases. This linear relation has been taken as evidence that observers imagine one of the objects moving through space along the same continuous trajectory as if being physically rotated to match the orientation of the other object (Shepard and Cooper 1982). Many tasks of object memory also show linear changes in performance with increasing rotations away from a target view, suggesting a common process is responsible for the curves in both mental rotation and object memory tasks, and motivating the MVPT account (Jolicoeur 1985; Tarr and Pinker 1989; Tarr 1995). A key finding is that performance on categorization and recognition is monotonically slower and less accurate as objects rotate farther from a canonical view (Palmer et al. 1981; Edelman and Bulthoff 1992; Srinivas 1993, 1995; Cutzu and Edelman 1994; Schendan and Kutas 2003).

MVPT variants are consistent with evidence that the dorsal pathway has a key role in both mental rotation and object cognition. Although more superior intraparietal sulcus (IPS) regions are also involved in saccades, the ventral–caudal IPS (vcIPS) and adjacent transverse occipital sulcus (TOS) are not, and show a linear parametric increase as the degree of rotation increases, implicating these object-sensitive dorsal foci (Schendan and Stern 2007) specifically in the scaling of mental rotation task performance with angular disparity (Carpenter et al. 1999; Harris et al. 2000; Podzebenko et al. 2002; Ecker et al. 2006). Occipitoparietal damage impairs mental rotation and object categorization. Patients with the “perceptual categorization” or “unusual views deficit” type of visual apperceptive agnosia (Farah 1990) have right inferior parietal damage, including vcIPS and TOS (Warrington 1982; Turnbull and McCarthy 1996; Turnbull 1997; Turnbull, Beschin, et al. 1997). They have problems with impoverished images, in general, and are impaired with unusual, but not canonical, views of objects. Although some neuropsychological work suggests dissociations between mental rotation and object cognition, the patients performed somewhat abnormally on both tasks (Farah and Hammond 1988; Turnbull and McCarthy 1996).

Few tests of visual object cognition theories have used more than one memory task in the same study, and none with neuroimaging. One behavioral study found that view dependence differs across 3 memory tests: categorization, recognition, and working memory (Srinivas 1995). Although another behavioral test of MVPT used multiple tasks and found generalization (Tarr et al. 1998), all were tests of episodic explicit memory for personally experienced events (Schacter and Tulving 1994). Of neuroimaging studies, 2 found occipitoparietal activation for unusual relative to canonical views during categorization (Kosslyn et al. 1994; Sugio et al. 1999). Another study found substantial overlap in vcIPS and TOS during object categorization and mental rotation (and not saccades) (Schendan and Stern 2007). Of 2 other studies that also included a mental rotation task, one compared mental rotation of letters and real objects with object naming (Wilson and Farah 2006), whereas the other compared mental rotation with immediate recognition of cube objects rotated from 15° to 90° (Gauthier et al. 2002), and both found some occipitoparietal overlap. Still, some anatomical dissociations were found, suggesting a greater role for ventral posterior regions (e.g., lateral occipital sulcus [LOS]) in view dependence of object cognition and for right rostrodorsal IPS in mental rotation (Gauthier et al. 2002; Wilson and Farah 2006). However, these superior IPS regions are those implicated in saccades, working memory, and attention (Courtney 2004; Postle 2006; Schendan and Stern 2007), as opposed to the inferior IPS areas considered critical for scaling effects on mental rotation (Podzebenko et al. 2002). Also, interpreting task dissociations in one study (Gauthier et al. 2002) was complicated by stimulus (i.e., different objects for each task), repetition, and working memory factors that differed between tasks, which could explain the direction of some IPS task dissociations and dorsal–ventral differences.

Throughout the present study, participants saw objects from unusual or canonical views. First, each object was categorized as belonging indoors or outdoors. Next, participants discriminated new objects from old objects recognized as seen before during categorization. Object model verification theories predict greater activation for unusual than canonical views in VLPFC and ventral posterior cortex (Ganis et al. 2007). Both model verification and MVPT also predict such view atypicality effects in the vcIPS and TOS, with MVPT predicting these regions are involved also in mental rotation. Although view atypicality and image impoverishment effects in these cortical areas have been found on categorization (Kosslyn et al. 1994; Sugio et al. 1999; Ganis et al. 2007), we tested the predictions in regions of interest (ROIs) defined independently as object-sensitive and/or mental rotation– (and not saccade-) related in the same subjects (Schendan and Stern 2007) and assessed whether they held also for recognition.

Materials and Methods

Materials

Ten lists of 35 grayscale objects were matched for name frequency (Kucera and Francis 1967) and category (20 tools [57.1%] and 15 were animals [12.9%], furniture [12.3%], musical instruments [3.1%], vehicles [8.0%], food/plants [5.1%], buildings [2.2%]). Canonical and unusual views of 85 objects were taken from our prior work (Verfaillie and Boutsen 1995; Schendan and Kutas 2003). Views of 265 other objects were chosen based on pilot results for computer models from copyright free sources or real objects on a turntable photographed with a digital camera on a tripod. Views were taken at 30° and 45° increments from 0° (most symmetric), and the depth rotation axis was chosen to maximize view differences; for object databank models, top and bottom, but no 45°, views existed (Tarr and Yu 1996). In pilot work, participants performed 2 tasks on all views of each object (N = 24 for digital photographs and object databank; N = 7 for other models). First, up to 16 views of each object were shown all at once arranged in a 4 column × 4 row grid (∼6° × 4° visual angle) in random order on a 17″ cathode ray tube monitor (Samsung, South Korea; 1024 × 768 resolution). Subjects typed the object's name into the computer. Next, they rank ordered the goodness of each view from best (rank 1) to worst, as elsewhere (Palmer et al. 1981; Verfaillie and Boutsen 1995; Schendan and Kutas 2003). Second, each view was shown sequentially at the size used in the experiment centered on the monitor. Subjects rated how hard each view was to identify on a 7-point Likert scale (1 = definitely identify easily). Results were used to choose canonical views as those with the lowest mean goodness rank (i.e., best view) and a median identifiability rating of 1–2. Unusual views were those with the highest mean goodness rank (i.e., far from best), and a median identifiability rating of 1–3 to ensure high accuracy and valid RTs. A repeated measures ANOVA showed the mean goodness rank was significantly lower (i.e., better) for canonical (M = 2.9, SE = 0.06) than unusual views (M = 10.6, SE = 0.14), F1,264 = 3428, P < 0.0001.

Design and Procedure

We used a 2 view (canonical, unusual) × 2 task (categorization, recognition) repeated measures, fast event-related design (Fig. 1). Object lists were assigned to conditions in a Latin square design counterbalanced across participants. Conditions were randomly intermixed. Object order avoided semantic associates. Participants performed each task by pressing 1 of 3 keys (right hand) fast and accurately, receiving instructions and practice shortly before doing each. In runs 1–3, they categorized objects from canonical or unusual views as indoor or outdoor objects, or unidentified. In runs 4–6, participants saw objects from canonical or unusual views, and recognized each object as remembered from the categorization task with high or low confidence or as a new object. Because recognition requires old and new objects, the categorization runs included old objects categorized before fMRI, counterbalanced across conditions; for old objects in both tasks, half were shown in the same view and half in the other view (i.e., canonical then unusual, or vice versa), but view was task-irrelevant, similar to related work on mental rotation and recognition (Gauthier et al. 2002); only new objects from the categorization task were later shown as old objects on recognition. Finally, 3 localizer tasks used standard methods and blocked designs (for details see Schendan and Stern 2007). The first 2 localizers defined object-sensitive areas and regions activated during mental rotation, with task order randomized; note, this design ensured that all subjects were naïve about mental rotation until after the categorization and recognition runs. The last run was a saccade area localizer.

Figure 1.

Procedures for categorization and recognition tasks. On each trial of both tasks, a canonical or unusual view of an object was shown for 3750 ms followed by fixation for 250 ms using PsyScope (Cohen et al. 1993) triggered by the scanner. Both views were scaled proportionately to 11° of visual angle of height or width, whichever was longer, and centered on a white background. The fixation image was black dot centered on a gray background of the mean luminance of all object images. Both fixation and object pictures were framed by a black background (not shown). Optseq software (MGH Martinos Center) set the sequence of extra fixation trials and their duration in multiples of the 2 s TR intermixed with the objects per scanning run to optimize signal discriminability between conditions: fixation between pictures lasted from 250 ms up to 8250 ms (i.e., 250 ms + multiples of the TR; most 2250 ms). (A) First, in scanning runs 1–3, participants pressed a key to categorize objects as belonging “indoors” (index finger) or “outdoors” (middle finger), or uncategorized (ring finger), if they had no idea what the object was. (B) Second, in scanning runs 4–6, methods were as in the categorization runs, except for the task. Participants pressed a key to report whether objects were recognized as “old” with high (index finger) or low confidence (middle finger), or were “new” objects not seen before in the experiment (ring finger); new objects were randomly intermixed with old objects that had been seen before as new objects on the categorization task.

Figure 1.

Procedures for categorization and recognition tasks. On each trial of both tasks, a canonical or unusual view of an object was shown for 3750 ms followed by fixation for 250 ms using PsyScope (Cohen et al. 1993) triggered by the scanner. Both views were scaled proportionately to 11° of visual angle of height or width, whichever was longer, and centered on a white background. The fixation image was black dot centered on a gray background of the mean luminance of all object images. Both fixation and object pictures were framed by a black background (not shown). Optseq software (MGH Martinos Center) set the sequence of extra fixation trials and their duration in multiples of the 2 s TR intermixed with the objects per scanning run to optimize signal discriminability between conditions: fixation between pictures lasted from 250 ms up to 8250 ms (i.e., 250 ms + multiples of the TR; most 2250 ms). (A) First, in scanning runs 1–3, participants pressed a key to categorize objects as belonging “indoors” (index finger) or “outdoors” (middle finger), or uncategorized (ring finger), if they had no idea what the object was. (B) Second, in scanning runs 4–6, methods were as in the categorization runs, except for the task. Participants pressed a key to report whether objects were recognized as “old” with high (index finger) or low confidence (middle finger), or were “new” objects not seen before in the experiment (ring finger); new objects were randomly intermixed with old objects that had been seen before as new objects on the categorization task.

Magnetic Resonance Imaging

At 3 T (Siemens Allegra, Munich, Germany), we acquired high-resolution T1-weighted scans (magnetization prepared rapid gradient echo [MP–RAGE]; 192 × 256), a T1-weighted echo-planar image (EPI) image, and T2*-weighted functional blood oxygen level–dependent (BOLD) scans using a gradient-echo EPI pulse sequence (21 anterior-posterior commissure line slices, 5 mm thick, 1 mm skip, time repetition [TR] = 2 s, time echo [TE] = 30 ms; flip angle = 90°, 64 × 64). If time allowed, a second MP–RAGE scan was acquired during practice for the mental rotation localizer. Using SPM99 software (Wellcome Department of Cognitive Neurology), anatomical and functional scans were coregistered using the T1-EPI image. Coregistered anatomical images were then normalized to MNI305 (Montreal Neurological Institute [MNI]) stereotactic space (interpolating to 3-mm3 voxels; neurological convention). Preprocessing of fMRI data included slice timing correction (for event-related data), motion correction, normalization (using the anatomical parameters), and spatial smoothing (8-mm3 Gaussian kernel). Statistical analyses used the general linear model. High-pass filtering was applied, but global signal scaling was not to avoid spurious deactivations. Each event in the design matrices was modeled using 2 gamma functions (positive, negative) shifted 2 s apart convolved with a canonical hemodynamic response function. Activation was assessed in linear contrasts between unusual and canonical views in each task; new objects were analyzed, because old objects confound view atypicality and repetition factors. Contrast images for each subject were used in second-level analyses treating subjects as a random effect. One-sample t tests assessed contrasts for each task. Paired t tests assessed contrasts between tasks. Extent threshold was 5 voxels. Anatomical structures were labeled using the human brain atlas of Duvernoy et al. (1999).

Participants

Data from the categorization task were available from 18 young, neurologically normal people with normal or corrected vision (age M = 22 years; education M = 15 years; 8 males). Some of these completed other tasks during the allotted 2-h session: 14 completed recognition (age M = 22 years; education M = 15 years; 7 males), but 4 others did not due to technical problems. The localizer tasks were run last: 16 stayed to complete the object-sensitive localizer, 13 the mental rotation localizer, and 12 the saccade localizer; for details see Schendan and Stern (2007). All but one were right-hand dominant (Oldfield 1971).

Results

Performance

To assess view effects, RT was the main measure (Bülthoff et al. 1995), and the study was designed for high accuracy. Anticipatory RTs under 300 ms and RTs over ±2.5 SD of the mean per subject and condition were not analyzed; RT was not used to exclude trials for fMRI analyses. Categorization and recognition accuracy was high, and RTs and errors for new objects were higher for unusual than canonical views on both tasks, confirming the classic behavioral finding (Palmer et al. 1981; Cutzu and Edelman 1994). A 2 × 2 omnibus repeated measures ANOVA (SPSS 14.0, SPSS Inc., Chicago, IL) showed significant main effects of view (RT: F1,12 = 33.72, P < 0.001; accuracy: F1,13 = 12.26, P = 0.004), and task (RT: F = 36.23, P < 0.001; accuracy: F = 23.1, P < 0.001), that did not interact (RT: F < 0.8, P > 0.4; accuracy: F < 0.2, P > 0.6). Separate ANOVAs for each task showed view effects on both tasks (Fig. 2). Discrimination of old and new objects was high, as shown by the discrimination index (d′) of signal detection theory with logistic distributions using a corrected matrix (Snodgrass and Corwin 1988), which was 3.69 for canonical, and 3.62 for unusual views, and showed no view effect, F1,13 = 0.06, P = 0.81; correct recognition of old objects (“hits” for old objects from the same view collapsed across confidence) was 89.6% for canonical and 91.5% for unusual, with no view effect, F = 2.31, P = 0.15. Note, for recognition RTs on new objects, data from one participant were unavailable due to a broken key. Localizer results were reported elsewhere (Schendan and Stern 2007).

Figure 2.

Performance results. View atypicality effects on performance with novel real objects were found on both tasks and were comparable between tasks. (A) On both tasks, mean RTs were significantly slower for unusual than canonical views. The view effect was significant for both categorization, F1,17 = 20.116, and recognition, F1,12 = 10.93. (B) On both tasks, mean error rates were higher for unusual than canonical views. The view effect was significant for categorization, F1,17 = 19.536, and showed a trend for recognition, F1,13 = 4.07; for recognition, errors were false alarms (i.e., responding “old” to the new objects). **P < 0.0001, *P = 0.006, +P = 0.065.

Figure 2.

Performance results. View atypicality effects on performance with novel real objects were found on both tasks and were comparable between tasks. (A) On both tasks, mean RTs were significantly slower for unusual than canonical views. The view effect was significant for both categorization, F1,17 = 20.116, and recognition, F1,12 = 10.93. (B) On both tasks, mean error rates were higher for unusual than canonical views. The view effect was significant for categorization, F1,17 = 19.536, and showed a trend for recognition, F1,13 = 4.07; for recognition, errors were false alarms (i.e., responding “old” to the new objects). **P < 0.0001, *P = 0.006, +P = 0.065.

Functional MRI

Analyses of ROIs Using Small-Volume Correction

Analyses focused on a priori ROIs. Analysis methods and results for the saccade, object-sensitive, and mental rotation localizers, and methods for using them to define the ROIs and volume masks (ROI Analysis Toolbox v. 1.7, SPM99), and the resultant ROIs, have been described elsewhere (Schendan and Stern 2007). For the most focused ROI analysis, the Gaussian-field small-volume correction (SVC) for multiple voxel-wise comparisons was applied in ROI volume masks defined from mental rotation and object-sensitive localizer results, separately, with saccade voxels in frontal and posterior eye fields exclusively masked out (Supplemental Fig. 1). In addition, 2 VLPFC ROIs were based on published coordinates for model verification and memory effects (Kirchhoff et al. 2000; Ganis et al. 2007): anterior VLPFC (aVLPFC; Brodmann area [BA] 45/47/12 ± 37 26 5) and posterior VLPFC (pVLPFC; BA 44/6 ± 48 9 34). For these ROIs, SVC was applied to a sphere of 16-mm radius around the coordinate (Worsley et al. 1996); if results were nonsignificant but some voxels were suprathreshold (uncorrected P value [Pu] < 0.05), the nearest cluster SVC algorithm was applied, and significant clusters in the ROI reported (Friston 1997). To apply SVC, each categorization or recognition SPM was threshold at Pu < 0.05 to exclude nonsignificant single voxels. SVC results are reported if significant at corrected P value (Pc) ≤ 0.05 based on the lower of random field theory with Bonferroni family-wise error (FWE) or false discovery rate (FDR) correction (Genovese et al. 2002).

Saccade ROIs.

Saccade task results (saccade > fixation) were used to create a bilateral volume mask of frontal eye fields and posterior eye fields of the IPS in anterior (aIPS) and posterior (pIPS) parts in order to exclude all saccade-related voxels (Pu <0.05) from other task results using SPM99 exclusive masking.

Object-sensitive ROIs.

Six ROIs based on object-decision task results (intact > scrambled) were combined into 4 volume masks: 1) of 2 ventral occipitotemporal (VOT) volumes, one mask was in posterior fusiform gyrus (pFG), also known as “LOa” (Malach et al. 2002), “mid-FG” (Denys et al. 2004), or “vTO” (James et al. 2002). 2) Another VOT mask was in anterior collateral sulcus (CoS) (Hasson et al. 2003). 3) For a dorsal occipitotemporal (DOT) area mask, one part of the ROI encompassed occipitotemporal and inferotemporal sulci (ITS), also known as “LO” (Malach et al. 2002) or “post-ITG” (Denys et al. 2004), and another part of the ROI was around LOS in the most posterior part of the lateral occipital complex (Hasson et al. 2003), with the more ventral part perhaps corresponding to retinotopic area LO1 (Larsson and Heeger 2006). 4) For a dorsal foci mask (Grill-Spector et al. 2000), one ROI was around the TOS (TOS = “DF1”) in a building-selective area (Hasson et al. 2003; Epstein et al. 2005), and another ROI was in the most vcIPS region (vcIPS= “DF2”) (Grill-Spector et al. 2000), also known as ventral IPS, which is also motion-sensitive (Denys et al. 2004). (Other dorsal shape-sensitive areas, Denys et al. 2004, located superior to vcIPS were almost entirely masked out of our object-sensitive localizer results by the saccade localizer. Supplemental Fig. 2 shows these areas.)

Mental rotation ROIs.

Mental rotation task results (rotation > no rotation control) were used to create 4 ROI masks. 1) The main ROI volume covered occipitoparietal areas that were not saccade-related but overlapped object-sensitive TOS and vcIPS (Schendan and Stern 2007). 2) An ROI volume for a caudal IPS region (cIPS) just superior to vcIPS corresponds to retinotopic area IPS2 (Silver et al. 2005) or shape-selective parietoccipital IPS (POIPS) (Denys et al. 2004) and was tested for completeness, and showed effects similar to vcIPS, but we do not focus on cIPS effects because saccade-related activity was found there (Schendan and Stern 2007); note, more rostrodorsal parietal regions have not shown parametric modulation with mental rotation and so were not of interest, but we observed these areas to show no view effects. 3) One ROI volume covered parts of object-sensitive left ITS and left LOS. This more dorsal LOS region may correspond to retinotopic area LO2, which lies just posterior to motion area MT+ (Larsson and Heeger 2006). 4) Another ROI in the right hemisphere covered an area centered at the junction where the lingual sulcus (LS) intersects the object-sensitive CoS area between fusiform and posterior parahippocampal gyri (x 30, y −42, z −9; Fig. 5C), sometimes referred to as a CoS “building,” lingual “landmark” (Aguirre et al. 1998; Hasson et al. 2003), or parahippocampal “place” area (Epstein et al. 1999); we use the anatomical label of CoS/LS.

Analyses of ROIs Using SPM Overlays

Another ROI approach was used because the volume masks for SVC analyses are large and may include structures outside the cortex of interest. To examine 4 bilateral ROIs of VOT (pFG, CoS, CoS/LS), DOT (ITS, LOS), dorsal foci (vcIPS, TOS), and VLPFC, the contrast SPMs of categorization and recognition results were superimposed on the mental rotation and object-sensitive localizer SPM contrasts at PFDR < 0.05, excluding saccade voxels; only left pVLPFC was active in the localizers, but the corresponding right pVLPFC was assessed because picture novelty effects are bilateral (Kirchhoff et al. 2000). For categorization, the threshold for the SPMs was Pu < 0.005, and significant voxels were defined as those located in a ROI. For recognition and task comparisons, which involved fewer subjects, the threshold for the SPMs was Pu <0.01 (≅0.05 ÷ 4 ROIs).

View Atypicality (Unusual > Canonical) Results

Data from all trials and correct trials were analyzed, separately, to facilitate interpretation of task effects: because the 2 tasks depend upon distinct memory processes, different object trials would be correct for each task so results over all trials are considered more valid (Tulving 1972; Curran et al. 2002). To analyze objects eliciting correct responses, the SPM99 design matrix further sorted trials for each condition according to subject response: for categorization, correctly categorized new objects and effects of no interest (i.e., correctly categorized old objects, unidentified, or no responses); for recognition, correct rejections (new objects correctly reported as new), and effects of no interest (i.e., hits [old objects correctly remembered], false alarms [new incorrectly reported as remembered], misses [old incorrectly reported as new, and no responses]). Participants were included in analyses of correct recognition trials if correct rejection rates were over 50%, which excluded 1 person (n = 13). Too few incorrect trials were available to assess these. Because most trials were correct, results should be similar for both analyses, as was found.

Categorization.

Overlaying results from all categorization trials on the localizers showed that activation was greater for unusual than canonical views of new instances of common objects (Fig. 3B,C,E) in both mental rotation ROIs and object-sensitive ROIs of bilateral vcIPS (and right cIPS), TOS, ITS, and LOS. Activation was also found in ROIs that were only object sensitive in right TOS caudomedially and a mid-ITS region bilaterally (in-between mental rotation parts of ITS and LOS), extending ventromedially a little into pFG, and in the mental rotation defined ROIs of pVLPFC and aVLPFC bilaterally (Fig. 4; Supplemental Fig. 3); note, the aVLPFC region is located in the caudal part of BA 47/12 in the horizontal ramus of the lateral fissure, which is just anterior to insular cortex (Kostopoulos et al. 2007). Overlaying results for correct trials on the localizers showed a similar regional pattern (Supplemental Fig. 4B,C,E). SVC results for all and correct trials supported this pattern, except an effect in the mental rotation ROI of right CoS/LS was revealed (Table 1). Assessing results also without the saccade mask uncovered view effects for all and correct trials in saccade areas of right aIPS, a small part of left aIPS, and right dorsal shape-selective areas of POIPSs and medial dorsal IPS (Denys et al. 2004).

Table 1

View atypicality effects (unusual > canonical) for categorization: Z scores and corrected P values of most significantly activated volume cluster (>8 mm apart) for each ROI

  All trials Correct trials 
    MNI (mm)   MNI (mm) 
Region Hemisphere Z P x y z Z P x y z 
Object sensitive 
    vcIPS Left 3.68 0.0235 −33 −84 27 — —    
 Right 3.37 0.0033 33 −72 30 3.27 0.0110 33 −72 30 
    TOS Left 3.29 0.0308 −33 −81 12 — —    
 Right 4.72 0.0005 33 −81 12 4.14 0.0060 27 −84 12 
    LOS Left 3.30 0.0104 −36 −87 3.81 0.0230 −36 −72 −12 
 Right 3.29 0.0090 33 −90 −6 — —    
    ITS Left 3.99 0.0104 −36 −69 −15 3.61 0.0230 −42 −63 −12 
 Right 3.81 0.0090 30 −81 −12 —     
    pFG Left — —    — —    
 Right 2.55 0.0377 36 −39 −21 — —    
Mental rotation 
    cIPS Left — —    — —    
 Right 3.17 0.0410 33 −63 48 (3.06) (0.059) 33 −63 48 
    vcIPS Left 3.74 0.0160 −36 −84 24 3.38 0.0470 −36 −84 21 
 Right 3.37 0.0060 33 −72 30 3.27 0.0150 33 −72 30 
    TOS Left 3.29 0.0260 −33 −81 12 3.39 0.0470 −39 −81 18 
 Right 4.27 0.0020 36 −84 12 3.49 0.0150 33 −78 18 
    LOS Left 3.20 0.0060 −27 −90 −6 2.69 0.0140 −27 −90 −6 
    ITS Left 3.66 0.0040 −42 −63 −12 3.61 0.0070 −42 −63 −12 
    CoS/LS Right — —    2.55 0.0330 33 −39 −9 
Object model verification 
    aVLPFC Left 4.10 0.0110 −33 27 −3 3.51 0.0360 −33 33 12 
 Right 3.80 0.0100 36 30 −3 2.42 0.0230 39 33 
    pVLPFC Left 2.78 0.0499 −48 33 3.36 0.0370 −54 21 27 
 Right 3.51 0.0280 54 15 30 3.92 0.0090 57 15 30 
  All trials Correct trials 
    MNI (mm)   MNI (mm) 
Region Hemisphere Z P x y z Z P x y z 
Object sensitive 
    vcIPS Left 3.68 0.0235 −33 −84 27 — —    
 Right 3.37 0.0033 33 −72 30 3.27 0.0110 33 −72 30 
    TOS Left 3.29 0.0308 −33 −81 12 — —    
 Right 4.72 0.0005 33 −81 12 4.14 0.0060 27 −84 12 
    LOS Left 3.30 0.0104 −36 −87 3.81 0.0230 −36 −72 −12 
 Right 3.29 0.0090 33 −90 −6 — —    
    ITS Left 3.99 0.0104 −36 −69 −15 3.61 0.0230 −42 −63 −12 
 Right 3.81 0.0090 30 −81 −12 —     
    pFG Left — —    — —    
 Right 2.55 0.0377 36 −39 −21 — —    
Mental rotation 
    cIPS Left — —    — —    
 Right 3.17 0.0410 33 −63 48 (3.06) (0.059) 33 −63 48 
    vcIPS Left 3.74 0.0160 −36 −84 24 3.38 0.0470 −36 −84 21 
 Right 3.37 0.0060 33 −72 30 3.27 0.0150 33 −72 30 
    TOS Left 3.29 0.0260 −33 −81 12 3.39 0.0470 −39 −81 18 
 Right 4.27 0.0020 36 −84 12 3.49 0.0150 33 −78 18 
    LOS Left 3.20 0.0060 −27 −90 −6 2.69 0.0140 −27 −90 −6 
    ITS Left 3.66 0.0040 −42 −63 −12 3.61 0.0070 −42 −63 −12 
    CoS/LS Right — —    2.55 0.0330 33 −39 −9 
Object model verification 
    aVLPFC Left 4.10 0.0110 −33 27 −3 3.51 0.0360 −33 33 12 
 Right 3.80 0.0100 36 30 −3 2.42 0.0230 39 33 
    pVLPFC Left 2.78 0.0499 −48 33 3.36 0.0370 −54 21 27 
 Right 3.51 0.0280 54 15 30 3.92 0.0090 57 15 30 

Note: Reported P values reflect either a Bonferroni FWE or FDR correction for multiple voxel-wise comparisons (marginal effects in parentheses); italics are statistics from nearest cluster SVC in ventrolateral prefrontal cortex (VLPFC); aVLPFC = anterior VLPFC at +/− 37 26 5 (BA 45/47/12); pVLPFC = posterior VLPFC at +/− 48 9 34 (BA 44/6); brain structures labeled based on Duvernoy et al. (1999).

Figure 3.

View atypicality effects (unusual > canonical) for all new objects on categorization and recognition tasks. Note: Activation in statistical parametric maps (SPM) is shown with saccade voxels masked out exclusively and superimposed on axial slices from an individual canonical brain (MNI, SPM99); MNI z coordinate labeled in center. The ROIs, which were defined based on the mental rotation and object-sensitive localizer results (corrected PFDR < 0.05 with saccade voxels at Pu < 0.05 masked out exclusively), are superimposed on each slice in green and blue outlines, respectively. pVLPFC = posterior ventrolateral prefrontal cortex (BA 44/6); aVLPFC = anterior VLPFC (BA 45/47/12); cIPS = caudal intraparietal sulcus; vcIPS = ventral caudal intraparietal sulcus; TOS = transverse occipital sulcus; LOS = lateral occipital sulcus; ITS = inferior temporal and occipitotemporal sulci; pFG = posterior fusiform gyrus; CoS = collateral sulcus; CoS/LS = junction of collateral and lingual sulci sometimes referred to as a `place area', which, for clarity, the map at z −6 shows the location of the right CoS/LS by copying the mirror reverse of the left CoS/LS ROI from mental rotation into the right hemisphere. (A) Categorization showed view atypicality effects in aVLPFC and LOS that were larger than on recognition. The contrast of categorization task > recognition task is shown for view atypicality activation (unusual > canonical). View atypicality effects were significantly (Pu < 0.01) greater during categorization than recognition in the mental rotation ROIs of LOS and aVLPFC (BA 47/12). (B) For ROIs in the left hemisphere, each bar plots the average, adjusted, BOLD response for all trials in unusual and canonical conditions in the categorization task (Cat) and recognition task (Rec), separately. For each bar, the signal extracted from the ROI was averaged between 4- and 8-s poststimulus. To obtain the signal in each ROI, the selective averaging function of the ROI Analysis Toolbox (SPM99) was used to extract the mean signal intensity peristimulus time series for each subject from the 16-mm ROI volumes from analyses with SVC; right pVLPFC was not active in the localizers so the ROI Analysis Toolbox automatically extracted clusters of view atypicality activation from categorization data with saccade voxels exclusively masked out, yielding a 16-mm volume mask for right pVLPFC. Signal was preprocessed to filter and adjust for effects of no interest and other trial conditions in the design matrix. Selective averaging involved reconstructing the time series using only the regressors for each specific condition and a finite impulse response formulation of the general linear model, modeling each time point as a separate regressor of stick functions. For clarity, ROIs shown are for the critical mental rotation volumes (green bold labels), object-sensitive regions not activated in the mental rotation localizer (blue labels), and right pVLPFC (black italic label). As found in the activation maps (panels A, C, D, F), signal was higher for unusual than canonical views on both categorization and recognition in the left hemisphere ROIs, except in LOS, which showed the largest view atypicality effects during categorization, and object-sensitive CoS and pFG, which showed minimal effects. *Significant view atypicality effect (unusual > canonical) based on corresponding SPM (all Pu < 0.01) in panels C and D or SVC analyses in Tables 1 and 2. (C) Categorization task SPM maps showed significant (Pu < 0.005) view a typicality effects in bilateral vcIPS (and right cIPS), TOS, ITS, and LOS in both mental rotation and object-sensitive ROIs. The mental rotation part of LOS dorsally (see z 0 or −6) may correspond to retinotopic area LO2, whereas the more ventral object sensitive part of LOS (see z −12 or −18) may correspond to retinotopic area LO1. In ROIs that were only object sensitive, bilateral ITS and right TOS caudomedially were activated. Effects were also found in the mental rotation region of bilateral aVLPFC in the caudal part of BA 47/12 in the horizontal ramus of the lateral fissure and pVLPFC, especially in the right hemisphere, but not prefrontal parts active in the object-sensitive localizer (BA 45 in aVLPFC; medial part of left pVLPFC, even at Pu < 0.05). (D) Recognition task SPM maps (Pu < 0.01) showed view atypicality effects in the mental rotation and object-sensitive ROIs in bilateral vcIPS (and right cIPS), right TOS, bilateral ITS, right CoS/LS, and right pVLPFC. The CoS/LS area, sometimes referred to as a “place area,” was centered around the junction where the lingual and collateral sulci meet, extending anteriorly from the parahippocampal gyrus to the lingual cortex posteriorly, abutted ventrolaterally by fusiform gyrus. (E) Same as (B), except for ROIs in the right hemisphere. As found in the activation maps (panels A, C, D, F), signal was higher for unusual than canonical views on both categorization and recognition in the right hemisphere ROIs, except in aVLPFC, which showed the largest view atypicality effects during categorization, and CoS/LS, which showed the largest view atypicality effect during recognition. (F) Recognition showed larger view atypicality effects than categorization in right CoS/LS. The contrast of recognition task > categorization task is shown for view atypicality activation (unusual > canonical). View atypicality effects (Pu < 0.01) were greater during recognition than categorization in the mental rotation ROI of right CoS/LS; for clarity, the map at z −9, where CoS/LS activity is maximal, is shown instead of z −12.

Figure 3.

View atypicality effects (unusual > canonical) for all new objects on categorization and recognition tasks. Note: Activation in statistical parametric maps (SPM) is shown with saccade voxels masked out exclusively and superimposed on axial slices from an individual canonical brain (MNI, SPM99); MNI z coordinate labeled in center. The ROIs, which were defined based on the mental rotation and object-sensitive localizer results (corrected PFDR < 0.05 with saccade voxels at Pu < 0.05 masked out exclusively), are superimposed on each slice in green and blue outlines, respectively. pVLPFC = posterior ventrolateral prefrontal cortex (BA 44/6); aVLPFC = anterior VLPFC (BA 45/47/12); cIPS = caudal intraparietal sulcus; vcIPS = ventral caudal intraparietal sulcus; TOS = transverse occipital sulcus; LOS = lateral occipital sulcus; ITS = inferior temporal and occipitotemporal sulci; pFG = posterior fusiform gyrus; CoS = collateral sulcus; CoS/LS = junction of collateral and lingual sulci sometimes referred to as a `place area', which, for clarity, the map at z −6 shows the location of the right CoS/LS by copying the mirror reverse of the left CoS/LS ROI from mental rotation into the right hemisphere. (A) Categorization showed view atypicality effects in aVLPFC and LOS that were larger than on recognition. The contrast of categorization task > recognition task is shown for view atypicality activation (unusual > canonical). View atypicality effects were significantly (Pu < 0.01) greater during categorization than recognition in the mental rotation ROIs of LOS and aVLPFC (BA 47/12). (B) For ROIs in the left hemisphere, each bar plots the average, adjusted, BOLD response for all trials in unusual and canonical conditions in the categorization task (Cat) and recognition task (Rec), separately. For each bar, the signal extracted from the ROI was averaged between 4- and 8-s poststimulus. To obtain the signal in each ROI, the selective averaging function of the ROI Analysis Toolbox (SPM99) was used to extract the mean signal intensity peristimulus time series for each subject from the 16-mm ROI volumes from analyses with SVC; right pVLPFC was not active in the localizers so the ROI Analysis Toolbox automatically extracted clusters of view atypicality activation from categorization data with saccade voxels exclusively masked out, yielding a 16-mm volume mask for right pVLPFC. Signal was preprocessed to filter and adjust for effects of no interest and other trial conditions in the design matrix. Selective averaging involved reconstructing the time series using only the regressors for each specific condition and a finite impulse response formulation of the general linear model, modeling each time point as a separate regressor of stick functions. For clarity, ROIs shown are for the critical mental rotation volumes (green bold labels), object-sensitive regions not activated in the mental rotation localizer (blue labels), and right pVLPFC (black italic label). As found in the activation maps (panels A, C, D, F), signal was higher for unusual than canonical views on both categorization and recognition in the left hemisphere ROIs, except in LOS, which showed the largest view atypicality effects during categorization, and object-sensitive CoS and pFG, which showed minimal effects. *Significant view atypicality effect (unusual > canonical) based on corresponding SPM (all Pu < 0.01) in panels C and D or SVC analyses in Tables 1 and 2. (C) Categorization task SPM maps showed significant (Pu < 0.005) view a typicality effects in bilateral vcIPS (and right cIPS), TOS, ITS, and LOS in both mental rotation and object-sensitive ROIs. The mental rotation part of LOS dorsally (see z 0 or −6) may correspond to retinotopic area LO2, whereas the more ventral object sensitive part of LOS (see z −12 or −18) may correspond to retinotopic area LO1. In ROIs that were only object sensitive, bilateral ITS and right TOS caudomedially were activated. Effects were also found in the mental rotation region of bilateral aVLPFC in the caudal part of BA 47/12 in the horizontal ramus of the lateral fissure and pVLPFC, especially in the right hemisphere, but not prefrontal parts active in the object-sensitive localizer (BA 45 in aVLPFC; medial part of left pVLPFC, even at Pu < 0.05). (D) Recognition task SPM maps (Pu < 0.01) showed view atypicality effects in the mental rotation and object-sensitive ROIs in bilateral vcIPS (and right cIPS), right TOS, bilateral ITS, right CoS/LS, and right pVLPFC. The CoS/LS area, sometimes referred to as a “place area,” was centered around the junction where the lingual and collateral sulci meet, extending anteriorly from the parahippocampal gyrus to the lingual cortex posteriorly, abutted ventrolaterally by fusiform gyrus. (E) Same as (B), except for ROIs in the right hemisphere. As found in the activation maps (panels A, C, D, F), signal was higher for unusual than canonical views on both categorization and recognition in the right hemisphere ROIs, except in aVLPFC, which showed the largest view atypicality effects during categorization, and CoS/LS, which showed the largest view atypicality effect during recognition. (F) Recognition showed larger view atypicality effects than categorization in right CoS/LS. The contrast of recognition task > categorization task is shown for view atypicality activation (unusual > canonical). View atypicality effects (Pu < 0.01) were greater during recognition than categorization in the mental rotation ROI of right CoS/LS; for clarity, the map at z −9, where CoS/LS activity is maximal, is shown instead of z −12.

Figure 4.

aVLPFC showed view atypicality effects (unusual > canonical) for all new objects on the categorization task. The aVLPFC region is located in the caudal part of BA 47/12 in the horizontal ramus of the lateral fissure, which is just anterior to insular cortex (Kostopoulos et al. 2007). The view atypicality effect on categorization was located bilaterally in the same aVLPFC ROI as found during mental rotation of objects (Schendan and Stern 2007). Note: Activation in SPM is shown with saccade voxels masked out exclusively and superimposed on an individual canonical brain (MNI, SPM99) in coronal slices at the location of the maximum activated voxel for aVLPFC in left (−30 24 −3) and right (39 30 −3) hemispheres. MNI y coordinate labeled in center. The ROIs, which were defined based on the mental rotation and object-sensitive localizer results (corrected PFDR < 0.05 with saccade voxels at Pu < 0.05 masked out exclusively), are superimposed on each slice in green and blue outlines, respectively. HR = horizontal ramus of lateral fissure; AR = ascending ramus of lateral fissure; IFS = inferior frontal sulcus; SFS = superior frontal sulcus.

Figure 4.

aVLPFC showed view atypicality effects (unusual > canonical) for all new objects on the categorization task. The aVLPFC region is located in the caudal part of BA 47/12 in the horizontal ramus of the lateral fissure, which is just anterior to insular cortex (Kostopoulos et al. 2007). The view atypicality effect on categorization was located bilaterally in the same aVLPFC ROI as found during mental rotation of objects (Schendan and Stern 2007). Note: Activation in SPM is shown with saccade voxels masked out exclusively and superimposed on an individual canonical brain (MNI, SPM99) in coronal slices at the location of the maximum activated voxel for aVLPFC in left (−30 24 −3) and right (39 30 −3) hemispheres. MNI y coordinate labeled in center. The ROIs, which were defined based on the mental rotation and object-sensitive localizer results (corrected PFDR < 0.05 with saccade voxels at Pu < 0.05 masked out exclusively), are superimposed on each slice in green and blue outlines, respectively. HR = horizontal ramus of lateral fissure; AR = ascending ramus of lateral fissure; IFS = inferior frontal sulcus; SFS = superior frontal sulcus.

Recognition.

Overlaying results for all recognition trials on the localizers showed greater activation for unusual than canonical views (Fig. 3B,D,E) in both mental rotation ROIs and object-sensitive ROIs of bilateral vcIPS (and right cIPS), right TOS, and bilateral ITS. Activation was found in ROIs that were only object sensitive in right TOS anterolaterally, and the mental rotation ROI of right CoS/LS, right pVLPFC, and a small part of left pVLPFC (z 23; inferior precentral sulcus; Supplemental Fig. 3). SVC results further supported the regional pattern (Table 2). For correct trials, a similar regional pattern was found, except for no effect in right CoS/LS (Supplemental Fig. 4B,D,E). SVC results were significant in right pVLPFC (nearest cluster Z = 3.06, Pc = 0.038; x 60, y 18, z 33). Assessing results without the saccade mask, revealed the same additional regions over all trials as found for categorization but, for correct trials, only a few voxels near right pIPS.

Table 2

View atypicality effects (unusual > canonical) for all recognition trials: Z scores and corrected P values of most significantly activated volume cluster (>8 mm apart) for each ROI

    MNI (mm) 
Region Hemisphere Z P x y z 
Object sensitive    
    vcIPS Right 3.64 0.0140 39 −81 30 
    TOS Right 3.65 0.0140 42 −84 18 
Mental rotation 
    vcIPS Right 3.64 0.0110 39 −81 30 
    TOS Right 3.65 0.0110 42 −84 18 
    DOT–ITS Left 2.80 0.0450 −48 −63 −12 
    CoS/LS Right 2.69 0.0140 30 −45 −9 
    MNI (mm) 
Region Hemisphere Z P x y z 
Object sensitive    
    vcIPS Right 3.64 0.0140 39 −81 30 
    TOS Right 3.65 0.0140 42 −84 18 
Mental rotation 
    vcIPS Right 3.64 0.0110 39 −81 30 
    TOS Right 3.65 0.0110 42 −84 18 
    DOT–ITS Left 2.80 0.0450 −48 −63 −12 
    CoS/LS Right 2.69 0.0140 30 −45 −9 

Note: Same legend as in Table 1.

Task dependence: categorization versus recognition.

For categorization relative to recognition, view atypicality effects for all trials were greater in the mental rotation ROI and object-sensitive ROI of left LOS and the corresponding object-sensitive ROI of right LOS, a few voxels bordering the object-sensitive ROI of right ITS, and the mental rotation ROIs of bilateral aVLPFC (Fig. 3A). SVC results confirmed significant effects in the mental rotation ROI of left LOS (Z = 3.11, Pc = 0.043; x −30, y −90, z −6), the object-sensitive ROI of right LOS (Z = 3.77, Pc = 0.031; x 30, y −87, z −6), and the nearest cluster in left aVLPFC (Z = 2.82, Pc = 0.049; x −33, y 30, z −6). For recognition relative to categorization, task differences in view atypicality effects across all trials were found in the mental rotation ROI of right CoS/LS (Fig. 5A; Supplemental Fig. 6A), which was also significant in SVC results (Z = 2.42, Pc = 0.027; x 30, y −45, z −9). For correct trials, we were concerned that, because subject's criteria for making categorization versus recognition decisions must differ, trials sorted by response would have uncontrolled stimulus differences between tasks (i.e., unmatched subsets of object trials for each task), which may reduce task effects by increasing noise or induce spurious stimulus-induced effects. These concerns did not apply to results over all trials because stimuli were counterbalanced between tasks. Results over all trials are thus the primary basis for our conclusions (for a similar approach, see Curran et al. 2002), and used to further constrain analyses of correct trials. For view atypicality effects on categorization relative to recognition, results over all trials showed task differences in 2 mental rotation ROIs (LOS, VLPFC), so we assessed whether, for correct trials, the overlay results would show task effects there at Pu <0.025 (= 0.05 ÷ 2 ROIs), which were found (Supplemental Fig. 4A); SVC results showed nearest clusters were significant in left aVLPFC (Z = 2.43, P = 0.05) and right aVLPFC (Z = 1.98, P = 0.049). In contrast, for view atypicality effects on recognition relative to categorization, results over all trials suggested task differences in the ROI of right CoS/LS, so we assessed whether, for correct trials, the overlay results would show a task effect there at Pu <0.05, which was not found (Supplemental Fig. 5D); SVC results showed no effect. In sum, view atypicality effects in LOS and aVLPFC were generally larger on categorization than recognition. Also, view atypicality effects in right CoS/LS were larger on recognition than categorization over all trials but indistinguishable between tasks on correct trials.

Figure 5.

Activity in the CoS/LS region varies across different tasks in this and other experiments. Activation at Pu < 0.01 is shown on right hemisphere sagittal (left column), axial (center), and coronal (right column) slices from an individual canonical brain (MNI). Neurological convention has right hemisphere on the right. Crosshairs centered at coordinate 30, −45, −9, the maximum voxel for view atypicality effects on recognition, and, for axial and coronal slices, the corresponding region in the left hemisphere. Results in A–C and E–F from the present experiment shown with the same t value scale and saccade voxels at Pu < 0.05 masked out exclusively in SPM99. See the papers cited for other activations. Mid-FG = fusiform gyrus region anteroventral to CoS/LS and adjacent to parahippocampal gyrus in the area referred to as mid-FG/PHG in Schon et al. (2004). ITS = ITS part of DOT object-sensitive cortex (Supplemental Fig. 1). (A) View atypicality effects in CoS/LS are greater bilaterally during recognition than categorization over all trials (recognition > categorization for contrast of unusual > canonical). (B) View atypicality effects in CoS/LS bilaterally over all recognition trials (unusual > canonical). (C) Mental rotation activation (rotation > control) in CoS/LS bilaterally using data from Schendan and Stern (2007). (D) Delay activity during a delayed matching to sample type of working memory task with scenes was found in CoS/LS bilaterally and a region including mid-fusiform and adjacent parahippocampal gyri (delay period on working memory > control tasks; SPM2). Data were from Schon et al. (2004). (E) View atypicality effects during all categorization trials (unusual > canonical) were minimal in CoS/LS but significant in other areas, such as ITS. (F) Object-sensitive activation (intact > scrambled) was minimal or absent in CoS/LS but was high in other areas, including surrounding fusiform and parahippocampal gyri, using data from Schendan and Stern (2007); axial slice shown at z −6 for clarity. (G) Subsequent memory modulation during the delay period of a delayed matching to sample task with scenes (recognition confidence of 5 (maximum) > 1–4; SPM2) was minimal in CoS/LS but significant in surrounding mid-FG and parahippocampal gyrus. Data were from Schon et al. (2004).

Figure 5.

Activity in the CoS/LS region varies across different tasks in this and other experiments. Activation at Pu < 0.01 is shown on right hemisphere sagittal (left column), axial (center), and coronal (right column) slices from an individual canonical brain (MNI). Neurological convention has right hemisphere on the right. Crosshairs centered at coordinate 30, −45, −9, the maximum voxel for view atypicality effects on recognition, and, for axial and coronal slices, the corresponding region in the left hemisphere. Results in A–C and E–F from the present experiment shown with the same t value scale and saccade voxels at Pu < 0.05 masked out exclusively in SPM99. See the papers cited for other activations. Mid-FG = fusiform gyrus region anteroventral to CoS/LS and adjacent to parahippocampal gyrus in the area referred to as mid-FG/PHG in Schon et al. (2004). ITS = ITS part of DOT object-sensitive cortex (Supplemental Fig. 1). (A) View atypicality effects in CoS/LS are greater bilaterally during recognition than categorization over all trials (recognition > categorization for contrast of unusual > canonical). (B) View atypicality effects in CoS/LS bilaterally over all recognition trials (unusual > canonical). (C) Mental rotation activation (rotation > control) in CoS/LS bilaterally using data from Schendan and Stern (2007). (D) Delay activity during a delayed matching to sample type of working memory task with scenes was found in CoS/LS bilaterally and a region including mid-fusiform and adjacent parahippocampal gyri (delay period on working memory > control tasks; SPM2). Data were from Schon et al. (2004). (E) View atypicality effects during all categorization trials (unusual > canonical) were minimal in CoS/LS but significant in other areas, such as ITS. (F) Object-sensitive activation (intact > scrambled) was minimal or absent in CoS/LS but was high in other areas, including surrounding fusiform and parahippocampal gyri, using data from Schendan and Stern (2007); axial slice shown at z −6 for clarity. (G) Subsequent memory modulation during the delay period of a delayed matching to sample task with scenes (recognition confidence of 5 (maximum) > 1–4; SPM2) was minimal in CoS/LS but significant in surrounding mid-FG and parahippocampal gyrus. Data were from Schon et al. (2004).

Further analyses of CoS/LS.

The anatomical landmarks, coordinates, shape, and spatial extent of the right CoS/LS activations fit those of a “building,” “landmark,” or “place” area (Aguirre et al. 1998; Hasson et al. 2003; Epstein et al. 2005). To further explore this important region for vision and memory theories, first, we compared the categorization, recognition, and between task results for view atypicality effects with results from our prior studies using the same threshold of Pu <0.01. We found activation bilaterally in the same CoS/LS region not only during mental rotation and recognition, and more so for recognition than categorization, but also during the delay period of a working memory task (Fig. 5AD) (Schon et al. 2004). In contrast, right CoS/LS showed minimal effects for our indoor/outdoor categorization task, another categorization task, that is, for object-sensitivity during an object-decision task from our localizer data, and processes related to long-term memory encoding (Schon et al. 2004) (Fig. 5EG). Overall, the CoS/LS showed its known right laterality (Epstein et al. 2007). Even so, left CoS/LS also showed effects of mental rotation and working memory delay (Fig. 5C,D) (Schon et al. 2004; Schendan and Stern 2007), and marginal view atypicality effects during recognition and a marginal task difference in view atypicality effects on recognition relative to categorization (Supplemental Fig. 6A,B). Second, we asked whether view atypicality effects in right CoS/LS were merely below the SPM threshold over all categorization trials and for correct trials on both tasks by assessing results in the single ROI of right CoS/LS at Pu <0.05, and found significant effects. For all trials, effects were significant on categorization in an anteroventral part of the right CoS/LS region (Supplemental Fig. 6E), which had also shown effects on recognition (Fig. 5B, Supplemental Fig. 6B), and did not show a larger view atypicality effect on recognition than categorization, in contrast to a posterior part of right CoS/LS, which did show this task effect (Fig. 5A; Supplemental Fig. 6A). For correct trials, effects were also significant in the right CoS/LS on both categorization and recognition, but this ROI did not show a task effect (Supplemental Fig. 5). In sum, right CoS/LS showed view atypicality effects for all stimulus–response combinations, except incorrect categorizations; over all trials, view effects for correct rejections plus false alarms on recognition were greater than view effects for correct plus incorrect categorizations, and correct trial results showed comparable view effects for correct rejections on recognition and correct categorizations; thus, the task effect over all trials reflected smaller view effects for incorrect categorizations than false alarms on recognition.

View Canonicality (Canonical > Unusual) Results

Model verification theory makes no predictions for this contrast, but recent findings suggest that the “active task” network showing view atypicality effects is anticorrelated with activity in a “default resting state” network (e.g., Fox et al. 2005; Vincent et al. 2006) implicated in memory and mental imagery (Schacter et al. 2007), and we found that a no rotation control condition relative to mental rotation activates the default network (Schendan and Stern 2007). An MVPT variant of model verification theory is compatible with the idea that processing canonical views is similar to that in a no rotation condition, predicting canonical more than unusual views will activate the same default state network. To test this, in addition to the active task ROI analyses, which were not expected to show effects, we assessed a default state network by overlaying the categorization and recognition results at Pu <0.01 (=0.05 ÷ 5 default network regions of medial posterior [i.e., retrosplenium, precuneus], posterior cingulate, medial prefrontal [including anterior cingulate], left lateral temporal, and left lateral parietal cortex) on the default network defined using the reverse contrast of mental rotation (no rotation control > rotation, Pu <0.05).

As expected, analyses on the active task (mental rotation and object sensitive) ROIs showed no view effects for all and correct trials on categorization and recognition. In contrast, all regions in the default network ROIs were activated for canonical relative to unusual views over all categorization trials: precuneus, retrosplenial, posterior cingulate, anterior cingulate, medial superior frontal gyrus, and left lateral temporal and inferior parietal cortex (Fig. 6A, Table 3). Results for correct trials also showed these regions (Supplemental Fig. 7A; Table 3). For recognition, results revealed activation in left lateral parietal cortex over all (x −42, y −45, z 21; T = 3.98; Fig. 6B) and on correct trials (x −57, y −48, z 18; T = 2.98; Supplemental Fig. 7B).

Table 3

Default State Network shows view canonicality effects (canonical > unusual) during categorization: Z scores and coordinates of most significantly activated volume cluster (>8 mm apart) within each anatomical structure at uncorrected P < 0.001.

  All trials Correct trials 
   MNI (mm)  MNI (mm) 
Region Hemisphere Z x y z Z x y z 
Retrosplenial cortex  4.32 −51 33 4.05 −51 33 
Posterior cingulate Left 3.94 −3 −18 57 3.47 12 −24 42 
 Right 3.99 −18 51 3.36 −12 36 
Precuneus  4.04 −63 39 4.65 −78 36 
Cuneus Right —    4.38 −78 33 
STS/lateral parietal Left 4.18 −48 −48 21 3.70 −45 −48 21 
  3.73 −60 −54 15 3.70 −45 −48 21 
 Right 3.63 60 −57 21 4.05 60 −57 21 
Middle temporal gyrus Left 3.52 −54 −3 −27 —    
 Left 3.41 −60 −21 −9 3.37 −60 −21 −9 
Medial prefrontal Left 3.63 −6 66 24 3.77 −3 66 18 
 Right 3.85 15 48 −6 4.10 15 51 −6 
  3.24 39 3.63 45 
  All trials Correct trials 
   MNI (mm)  MNI (mm) 
Region Hemisphere Z x y z Z x y z 
Retrosplenial cortex  4.32 −51 33 4.05 −51 33 
Posterior cingulate Left 3.94 −3 −18 57 3.47 12 −24 42 
 Right 3.99 −18 51 3.36 −12 36 
Precuneus  4.04 −63 39 4.65 −78 36 
Cuneus Right —    4.38 −78 33 
STS/lateral parietal Left 4.18 −48 −48 21 3.70 −45 −48 21 
  3.73 −60 −54 15 3.70 −45 −48 21 
 Right 3.63 60 −57 21 4.05 60 −57 21 
Middle temporal gyrus Left 3.52 −54 −3 −27 —    
 Left 3.41 −60 −21 −9 3.37 −60 −21 −9 
Medial prefrontal Left 3.63 −6 66 24 3.77 −3 66 18 
 Right 3.85 15 48 −6 4.10 15 51 −6 
  3.24 39 3.63 45 

Note: STS = superior temporal sulcus (ascending or horizontal posterior segment), which corresponds to the lateral inferior parietal part of the network around the angular gyrus. Same conventions as in other tables.

Figure 6.

Activation for canonical relative to unusual views (Pu < 0.01) superimposed on ROIs in a “default state network” superimposed on a rendered left hemisphere, and sagittal and coronal slices of an individual canonical brain (MNI from SPM99). The default state ROIs (red outline) were independently defined based on the mental rotation results (Pu < 0.05 with saccade voxels at Pu < 0.05 masked out exclusively). Scale shows the t values. Neurological convention: left hemisphere is left (L), right hemisphere is right (R). PCC = posterior cingulate cortex (extends posteriorly into precuneus and dorsally into subparietal sulcus); ACC = anterior cingulate cortex; MPF = medial prefrontal cortex around superior frontal gyrus; LP = left lateral inferior parietal cortex near the occipitotemporoparietal junction and the angular gyrus; LT = left lateral temporal cortex. Anatomical structures labeled based on Duvernoy et al. (1999). (A) Categorization: activation for all categorization trials in all regions of the default network: medial regions in the precuneus, subparietal sulcus, retrosplenial cortex, and posterior cingulate gyrus, posterior and anterior cingulate cortex, and medial superior frontal gyrus, and bilateral lateral inferior parietal cortex (encompassing posterior superior temporal cortex [horizontal posterior segment] a left anterior middle temporal gyrus region was also activated. (B) Recognition: activation for all recognition trials in left lateral parietal cortex. (C) Task Differences: activation for all categorization relative to recognition trials in precuneus, retrosplenial, and medial prefrontal cortex.

Figure 6.

Activation for canonical relative to unusual views (Pu < 0.01) superimposed on ROIs in a “default state network” superimposed on a rendered left hemisphere, and sagittal and coronal slices of an individual canonical brain (MNI from SPM99). The default state ROIs (red outline) were independently defined based on the mental rotation results (Pu < 0.05 with saccade voxels at Pu < 0.05 masked out exclusively). Scale shows the t values. Neurological convention: left hemisphere is left (L), right hemisphere is right (R). PCC = posterior cingulate cortex (extends posteriorly into precuneus and dorsally into subparietal sulcus); ACC = anterior cingulate cortex; MPF = medial prefrontal cortex around superior frontal gyrus; LP = left lateral inferior parietal cortex near the occipitotemporoparietal junction and the angular gyrus; LT = left lateral temporal cortex. Anatomical structures labeled based on Duvernoy et al. (1999). (A) Categorization: activation for all categorization trials in all regions of the default network: medial regions in the precuneus, subparietal sulcus, retrosplenial cortex, and posterior cingulate gyrus, posterior and anterior cingulate cortex, and medial superior frontal gyrus, and bilateral lateral inferior parietal cortex (encompassing posterior superior temporal cortex [horizontal posterior segment] a left anterior middle temporal gyrus region was also activated. (B) Recognition: activation for all recognition trials in left lateral parietal cortex. (C) Task Differences: activation for all categorization relative to recognition trials in precuneus, retrosplenial, and medial prefrontal cortex.

Task dependence: categorization versus recognition.

The contrast of canonical > unusual for categorization relative to recognition is mathematically equivalent to unusual > canonical for recognition relative to categorization (for the latter results in mental rotation and object-sensitive ROIs, see section above), and reveals both sets of task effects altogether. For all trials, assessing 5 default network ROIs, view canonicality effects were larger for categorization than recognition in precuneus, retrosplenial, medial prefrontal, and left lateral temporal cortex (Fig. 6C); a task effect in left lateral parietal cortex was located posterior to the ROI. For correct trials, task effects were found in default state ROIs of precuneus, retrosplenial, and medial prefrontal cortex (Supplemental Fig. 7C). The contrast of canonical > unusual for recognition relative to categorization is mathematically equivalent to unusual > canonical for categorization relative to recognition (for the latter results of mental rotation and object-sensitive ROIs, see section above), and reveals both sets of task effects altogether. Assessing 5 default network ROIs, no task effect was found for all and correct trials. In sum, together with results for each task separately, the default network showed view canonicality effects, but more so for categorization than recognition, except in the lateral parietal region, which showed comparable effects on both tasks.

Discussion

Brain activity related to categorization, recognition, mental rotation, object-sensitivity, and saccades were compared in order to examine the role of interactions between prefrontal–parietal and ventral posterior cortex in visual object cognition and memory. The fMRI results support key predictions of object model verification theory and MVPT. We found that the vcIPS and TOS are dorsal visual regions that unusual views of real objects recruit more strongly than canonical views during “nonspatial tasks” of object categorization and recognition. These areas are also object sensitive and involved in the cardinal spatial task of mental rotation (and not saccades) (Schendan and Stern 2007). (The same effects were found in a more anterodorsal region of cIPS, but most of this overlaps aIPS and pIPS saccade areas, Schendan and Stern 2007, so it will not be discussed in detail.) Although view atypicality effects in prefrontal–posterior networks have been reported for categorization (Kosslyn et al. 1994; Sugio et al. 1999), we find that activity in this same network, involving the pVLPFC (BA 44/6), and object-sensitive areas in vcIPS, TOS, and ITS, shows view sensitivity not only for mental rotation and categorization but also for recognition, comprising a task-general prefrontal–posterior network (Schendan and Stern 2007). A right CoS/LS “place” area is also view-sensitive on diverse tasks, showing view sensitivity on categorization, recognition, and mental rotation (Schendan and Stern 2007), but less for uncategorized objects, and is activated during the delay period of a matching to sample task, regardless of subsequent memory confidence (Schon et al. 2004). In contrast, the aVLPFC (caudal part of BA 47/12) and part of object-sensitive LOS is view-sensitive on mental rotation (Schendan and Stern 2007) and more so on categorization than recognition, comprising a task-dependent prefrontal–posterior network. In addition, the reverse contrast shows view canonicality effects in a “default” state network, which has been shown to be anticorrelated with the “active” task networks of lateral prefrontal–IPS regions (Fox et al. 2005; Vincent et al. 2006). Canonical relative to unusual views activate medial parts of prefrontal and posterior cortex, and lateral temporal cortex on categorization more than recognition, and lateral inferior parietal parts on both tasks, and these regions lie within the same “default” network activated for no rotation (control) relative to a mental rotation task (Schendan and Stern 2007), and for small than large rotations on object cognition tasks (Gauthier et al. 2002). We aimed to test object model verification theory and MVPT, which makes the strongest predictions about view atypicality effects in active task VLPFC–posterior networks and so, next, we introduce an emerging account of the brain basis of visual object knowledge, and then detail the implications of the active task network findings for this account and the role of each network component in achieving object constancy.

Two-State Interactive Account of Visual Object Knowledge

We propose a Two-State Interactive (2SI) account of the cortical dynamics of visual object knowledge activation (Schendan and Kutas 2007) that integrates model verification and MVPT accounts with each other and with memory theory (Fig. 7). The 2SI account combines timing ideas from event-related potential (ERP) studies on object constancy (Schendan and Kutas 2002, 2003, 2007) and emerging ideas that complex cognition, attention, feature binding, and fine discriminations depend upon later recurrent, feedback, and interareal contributions to visual information processing (Ullman 1995; Grossberg 1999; Lamme and Roelfsema 2000; David et al. 2005; Treisman 2006). The central idea of the 2SI account is that ventral object-sensitive cortex is activated in 2 functionally distinct states. First, a fast feedforward pass through ventral object-sensitive cortex takes place during an “initial classification” or structural encoding state (Bruce and Young 1986) between 125 and 200 ms, sufficient for perceptual categorization and global discriminations (e.g., face or not) (Schendan et al. 1998). We propose that more precise categorization and memory discriminations (i.e., determining a specific object class) are related to later neural computations supporting the critical matching processes of Object Model Selection (Schendan and Kutas 2002, 2003, 2007): between 200 and 500 ms, a second more sustained state of iterative activation of ventral object-sensitive cortex involves both feedforward and feedback interactions among these areas and with VLPFC and/or the mediotemporal lobe, with engagement of these other regions varying with memory task requirements. The purpose of the second interactive state is to disambiguate the input by integrating bottom-up and top-down information. At this later time, Object Model Verification processes in prefrontal–parietal networks modulate ventral posterior cortex (Ganis et al. 2007), facilitating Object Model Selection, to achieve object constancy despite impoverished viewing conditions. Accordingly, ERPs differ between unusual and canonical views after 200 ms (Schendan and Kutas 2003), and computational models suggest bottom-up processing dominates ERPs before 200 ms, whereas feedback connections along the hierarchical visual pathways dominate later ERPs between 200 and 500 ms (David et al. 2005). Our emphasis on later processes differs from other top-down accounts proposing earlier rapid feedback modulation in which extrastriate cortex projects via the magnocellular pathway into VLPFC and into orbitofrontal cortex within 130 ms, which then uses information about low spatial frequency candidate object models to send top-down feedback into ventral object-sensitive cortex by as early as 180 ms (Bar 2003; Bar et al. 2006).

Figure 7.

2SI theory of visual knowledge and summary of view atypicality effects. Superimposed on the right hemisphere of a rendered, individual canonical brain (SPM99) are the view atypicality effects (unusual > canonical, with saccade voxels masked out exclusively) on the (A) categorization task (Pu < 0.005) and (B) the recognition task (Pu < 0.01). Green text denotes brain regions important for interactive states of object model testing. Blue text denotes brain regions important for interactive states of model prediction. Same abbreviations and results as Figure 3C,D. We propose that the view atypicality effects define a set of brain regions supporting processes of model selection to categorize and recognize objects and verification to facilitate object constancy under more impoverished viewing conditions. First, a fast feedforward pass through ventral object-sensitive cortex (LOS to ITS) mediates an “Initial Classification” state sufficient for perceptual categorization and global discrimination (e.g., animal or not). In parallel, a feedforward pass through dorsal object-sensitive cortex (TOS to vcIPS) analyzes the spatial configuration of parts and the global shape of objects. These bottom-up inputs ultimately reach VLPFC within 200 ms. Second, between 200 and 500 ms, a more sustained state of iterative activation disambiguates the input by integrating bottom-up and top-down information. This state involves feedforward and feedback interactions among the ventral posterior object-sensitive areas and with other brain regions, especially occipitoparietal cortex and VLPFC, and, depending upon memory task requirements, the mediotemporal lobe. Interactions in the pVLPFC–posterior network contribute to the testing component of model verification for both categorization and recognition. The vcIPS and TOS compute the relative spatial relations between features and parts of an object, and transform the visuospatial structure, as needed, to facilitate matching the perceived input with other internal representations, whereas pVLPFC is responsible for top-down control of covert attention shifting, attribute biasing, and working memory for the spatial relations. The pVLPFC–vcIPS–TOS components modulate the ventral object-sensitive areas (Ganis et al. 2007). The aVLPFC and pVLPFC can interact in a predictiontesting loop. The caudal BA 47/12 part of aVLPFC controls interactions with posterior parts of a network supporting 2 model prediction processes of model verification during categorization: 1) object model selection processes evaluate and select the candidate model in memory that best matches the visual percept (Schendan and Kutas 2002, 2003, 2007); 2) working memory processes retrieve and maintain salient visual features of the selected candidate model (Ganis et al. 2007). The ventral posterior areas of LOS, ITS, and CoS/LS are critical for model representation, storing the view-specific object models and their salient features in long-term memory, thereby providing the view atypicality signal driving network interactions. For both categorization and recognition, ITS interacts with the right pVLPFC–vcIPS–TOS network for model testing. For recognizing an object as familiar and correct categorizations, the CoS/LS is important for detailed spatial matching, presumably interacting with ITS, to accomplish model testing. For categorization more than recognition, the LOS is important for the model prediction functions of the aVLPFC (task-dependent), but LOS may also contribute to model testing.

Figure 7.

2SI theory of visual knowledge and summary of view atypicality effects. Superimposed on the right hemisphere of a rendered, individual canonical brain (SPM99) are the view atypicality effects (unusual > canonical, with saccade voxels masked out exclusively) on the (A) categorization task (Pu < 0.005) and (B) the recognition task (Pu < 0.01). Green text denotes brain regions important for interactive states of object model testing. Blue text denotes brain regions important for interactive states of model prediction. Same abbreviations and results as Figure 3C,D. We propose that the view atypicality effects define a set of brain regions supporting processes of model selection to categorize and recognize objects and verification to facilitate object constancy under more impoverished viewing conditions. First, a fast feedforward pass through ventral object-sensitive cortex (LOS to ITS) mediates an “Initial Classification” state sufficient for perceptual categorization and global discrimination (e.g., animal or not). In parallel, a feedforward pass through dorsal object-sensitive cortex (TOS to vcIPS) analyzes the spatial configuration of parts and the global shape of objects. These bottom-up inputs ultimately reach VLPFC within 200 ms. Second, between 200 and 500 ms, a more sustained state of iterative activation disambiguates the input by integrating bottom-up and top-down information. This state involves feedforward and feedback interactions among the ventral posterior object-sensitive areas and with other brain regions, especially occipitoparietal cortex and VLPFC, and, depending upon memory task requirements, the mediotemporal lobe. Interactions in the pVLPFC–posterior network contribute to the testing component of model verification for both categorization and recognition. The vcIPS and TOS compute the relative spatial relations between features and parts of an object, and transform the visuospatial structure, as needed, to facilitate matching the perceived input with other internal representations, whereas pVLPFC is responsible for top-down control of covert attention shifting, attribute biasing, and working memory for the spatial relations. The pVLPFC–vcIPS–TOS components modulate the ventral object-sensitive areas (Ganis et al. 2007). The aVLPFC and pVLPFC can interact in a predictiontesting loop. The caudal BA 47/12 part of aVLPFC controls interactions with posterior parts of a network supporting 2 model prediction processes of model verification during categorization: 1) object model selection processes evaluate and select the candidate model in memory that best matches the visual percept (Schendan and Kutas 2002, 2003, 2007); 2) working memory processes retrieve and maintain salient visual features of the selected candidate model (Ganis et al. 2007). The ventral posterior areas of LOS, ITS, and CoS/LS are critical for model representation, storing the view-specific object models and their salient features in long-term memory, thereby providing the view atypicality signal driving network interactions. For both categorization and recognition, ITS interacts with the right pVLPFC–vcIPS–TOS network for model testing. For recognizing an object as familiar and correct categorizations, the CoS/LS is important for detailed spatial matching, presumably interacting with ITS, to accomplish model testing. For categorization more than recognition, the LOS is important for the model prediction functions of the aVLPFC (task-dependent), but LOS may also contribute to model testing.

Two VLPFC–Posterior Networks for Visual Object Constancy

The aVLPFC (BA 45/47/12) has been proposed to control network interactions during model verification in order to determine the object's category, modulating more ventral posterior areas from which visual knowledge is retrieved (Ganis et al. 2007). We propose that the aVLPFC–posterior network is primarily important for 2 prediction processes of model verification: 1) object model selection to evaluate whether the visual percept matches candidate models of known object categories in memory and select the best match (Schendan and Kutas 2002, 2003, 2007); and 2) retrieval and working memory for salient visual features of the selected candidate model. Accordingly, the caudal part of BA 47/12 in aVLPFC, where we found view atypicality effects, has been implicated in actively selecting and comparing information in memory to disambiguate the interpretation of perceived inputs (Kostopoulos et al. 2007). The BA 45 part of left aVLPFC has been proposed to select from among competing alternatives stored in left ITS (Kan and Thompson-Schill 2004; Kan et al. 2006). Our aVLPFC effects are bilateral, consistent with prior studies of categorization and recognition (James et al. 2002) and view-specific knowledge in right posterior cortex (Burgund and Marsolek 2000). Our findings suggest an aVLPFC (BA 47/12)–LOS network for model prediction is more important for categorization than recognition. After all, categorization entails a many-instance-to-one-class mapping, enabling a pool of previously learned, instances of a category (i.e., object models) to be available for the verification processes of prediction that can then be tested for their similarity with the visual input (Rosch et al. 1976; Sigala et al. 2002).

The pVLPFC–posterior network primarily contributes to the testing component of model verification for both categorization and recognition. Testing involves top-down control processes of 3) covert attention shifting and 4) attribute biasing to search for and switch attention between distinctive attributes in the input that are predicted by the candidate model in memory (Ganis et al. 2007), and 5) working memory for the relative spatial relations among parts of an object. The pVLPFC corresponds to the frontal part (ventral premotor area in precentral sulcus at the junction of BA 44/6; a human homolog to monkey F5) of a “canonical neuron” circuit preferentially recruited for perceiving objects, especially manipulable ones, executing a grasp toward an object, and motor knowledge retrieval (Grezes et al. 2003; Kan et al. 2006). Our findings suggest that, across tasks, the occipitoparietal region (vcIPS, TOS) is most consistently coactivated with right pVLPFC (and ITS) to achieve visual constancy with impoverished images of objects. This is consistent with evidence that the pVLPFC is connected reciprocally with the aIPS and pIPS saccade areas, and vcIPS is an earlier process in an IPS cascade from inferior to superior parts for actions with objects (Chao and Martin 2000; Grezes et al. 2003; Kan et al. 2006), a role for pVLPFC, vcIPS, and TOS (and cIPS) in modulating ventral posterior activity during object cognition (Wojciulik and Kanwisher 1999; Courtney 2004; Silver et al. 2005; Noppeney et al. 2006; Postle 2006; Xu and Chun 2006), and the bias competition model of selective attention in which VLPFC and ventral temporal cortex interact to generate object and feature templates (Desimone and Duncan 1995). In contrast to a common finding of left lateralization (Grezes et al. 2003; Kan et al. 2006; Schendan and Stern 2007), view atypicality effects in our pVLPFC–posterior network are more extensive in the right hemisphere. During model verification, the pVLPFC–vcIPS–TOS network modulates activity in ventral posterior cortex (LOS, ITS, CoS/LS) as a function of image impoverishment (Ganis et al. 2007), including view atypicality. The pVLPFC may interact with aVLPFC in a predictiontesting loop. Both are interconnected bidirectionally with each other and with the ventral stream (Ungerleider et al. 1989; Petrides and Pandya 2002; Petrides 2005), and have been implicated in visual working memory, in which parietal cortex is typically also involved (Passingham et al. 2000; Stern et al. 2000; Postle 2006). Considering our performance results, as well, the pVLPFC–posterior network may have a relatively greater role in performance speed, whereas the aVLPFC has a greater role in accuracy, though more for categorization than recognition.

Dorsal Visual Pathway for Object Constancy

We propose that the role of vcIPS and TOS is to compute the relative spatial relations between features and parts of an object, and, if necessary, transform the visuospatial structure to facilitate matching the perceived input with object models in memory, another perceived object, or a grasping action. Visual input has a multilevel hierarchical space: spatial arrangements either of local parts define an object or of multiple objects define a scene at more global hierarchical levels (Cave and Kosslyn 1993; Robertson 1996; Newell et al. 2005). We suggest occipitoparietal cortex will be recruited as a function of stimulus and task requirements for additional analysis and/or retrieval of visuospatial structure, as opposed to abstract, semantic, or lexical knowledge, for which pVLPFC and ITS interactions are enough (Kan and Thompson-Schill 2004; Kan et al. 2006). The vcIPS and/or TOS have been implicated in perceiving and matching object structure in depth, stereopsis, symmetry, spatial cognition and coordinate transformation, grasping objects, and tasks demanding greater spatial analysis of object structure (Milner and Goodale 1993; Kanwisher and Wojciulik 2000; Backus et al. 2001; Faillenot et al. 2001; Beauchamp et al. 2002; Gilaie-Dotan et al. 2001; Assad 2003; Shikata et al. 2003; Tsao et al. 2003; Altmann et al. 2005; Arbib 2005; Jeannerod and Jacob 2005; Noppeney et al. 2006; Sasaki et al. 2005; Schluppeck et al. 2005). The underlying function is not merely spatial but also object sensitive (Malach et al. 1995; Grill-Spector et al. 1998, 1999, 2001; Schendan and Stern 2007), and these dorsal regions are modulated routinely during object perception, categorization, and memory tasks in a way not directly attributable to saccade or attention processes (Grill-Spector et al. 2000, 2001; James et al. 2002; Vuilleumier et al. 2002; Hasson et al. 2003; Denys et al. 2004; Altmann et al. 2005; Ganis et al. 2007; Schendan and Stern 2007). Bilateral TOS has been implicated in Balint's syndrome patients who perceive only one object or feature at a time without knowing where it is located (Friedman-Hill et al. 1995, 2003; Turnbull, Carey, et al. 1997). The occipitoparietal region has been implicated in the explicit dynamic binding into a coherent structural representation of the hierarchical spatial configuration of perceptual features, which can be bound together with information about object identity into a “temporary token (object file)” that enables appropriate actions and cognitive decisions about objects (Treisman 2006; Treisman and Kanwisher 1998). Similarly, monkey posterior parietal area 7A is involved in constructing the spatial configuration among the parts of objects (Chafee et al. 2005) and automatically provides a map of visual salience required to allocate attention (Constantinidis and Steinmetz 2005).

Mental rotation may not be a direct mental analog of physical rotation, as originally suggested (Shepard and Cooper 1982). Mental rotation performance curves vary with object complexity and dimensionality (two [2D] vs. three [3D]) (Jolicoeur and Landau 1984; Folk and Luce 1987; Bauer and Jolicoeur 1996), and 3D rotations that alter visuospatial structure widely yield more variable rotation curves and view canonicality effects than 2D rotations of known objects (Edelman and Bulthoff 1992; Cutzu and Edelman 1994; Blanz et al. 1999; Gauthier et al. 2002). Neuroimaging has shown violations from a simple linear relationship on cortical activity during object mental rotation and recognition (Gauthier et al. 2002), motivating multiple views accounts to dismiss mental rotation or any other process beyond the ventral stream as contributing to object constancy. Instead, generalization to novel or unusual views is achieved via time-consuming, evidence accumulation, or interpolation across multiple views within a neural population entirely in the ventral stream (Edelman and Bulthoff 1992; Bülthoff et al. 1995; Ullman 1996; Perrett et al. 1998; Tarr and Bulthoff 1998; Peissig and Tarr 2006). Our findings in VLPFC and occipitoparietal cortex, however, demonstrate view-sensitive neural activity outside the ventral stream, indicating additional processes do underlie scaling of performance on categorization (Kosslyn et al. 1994; Sugio et al. 1999; Ganis et al. 2007) and recognition with increasing rotations away from target, familiar, or canonical view(s), or greater image impoverishment.

Ventral Visual Pathway for Categorization, Memory, and View Representation

View atypicality effects reflect perceptual and memory factors (Palmer et al. 1981; Edelman and Bulthoff 1992; Perrett et al. 1992). The unique geometric properties of each view, such as the set of diagnostic 2D shapes visible, may differentially recruit perceptual and visuospatial processes, but this seems more important for non-sense, wire-frame objects with uniform parts visible from any angle (Cutzu and Edelman 1994), including stimuli used in seminal mental rotation studies (Shepard and Cooper 1982). For meaningful known objects, learning and memory are more important. Which views are stored depends upon the views learned, their exposure frequency, object structure, context, and the tasks performed on the object during encoding and retrieval (Edelman and Bulthoff 1992; Cutzu and Edelman 1994; Srinivas 1995; Blanz et al. 1999).

Dual visual stream frameworks posit that, whereas the dorsal visual pathway is responsible for action and spatial vision, the ventral pathway is responsible for perception and memory (Ungerleider and Mishkin 1982; Goodale and Milner 1992; Perrett et al. 1998), and object- and category-sensitive areas in human ventral posterior cortex have well-established roles in visual object cognition, learning, and memory (Kourtzi and Kanwisher 2000; Bar et al. 2001; Grill-Spector et al. 2001; Hasson et al. 2003; Courtney 2004; Kan and Thompson-Schill 2004; Price and Devlin 2003; van Turennout et al. 2003). Our findings of view sensitivity in LOS, ITS, and CoS/LS regions implicate these as sites from which view-specific object models and their features are retrieved and a source of a view atypicality signal driving VLPFC–posterior network interactions for model verification. Multiple memory systems theories posit generic explicit memory stores semantic knowledge of concepts and facts, such as a function or name, and nonsemantic knowledge (Tulving 1972; Hintzman 1978). The candidate object models evaluated during the object model selection process supporting categorization are stored as nonsemantic knowledge about visual structure (Warrington 1982; Humphreys et al. 1999). DOT cortex (LOS, ITS) has been implicated in computing spatial aspects of objects and their configuration of parts (e.g., stereoscopic structure, symmetry, depth), and relations between objects and space (e.g., position, motion, kinematics, scale) (Van Oostende et al. 1997; Grill-Spector et al. 1999; Gilaie-Dotan et al. 2001; Moore and Engel 2001; Kourtzi et al. 2002; Hasson et al. 2003; Sasaki et al. 2005; Behrmann et al. 2006). Monkey inferotemporal neurons code object parts in 2D but with some depth structure from qualitative information about disparity gradient, depth curvature, or horizontal disparity (Tanaka 2003). Multiple views accounts suggest that the 2D views have some depth or stereo information (Edelman and Bulthoff 1992). For categorization more than recognition, the LOS seems particularly important for the model prediction functions of the aVLPFC, as both regions show similar patterns of view and task effects. We propose that the ITS has the most task-general role in object cognition and in achieving visual constancy: ITS is modulated on categorization, recognition, mental rotation, and working memory tasks (Schon et al. 2004; Schendan and Stern 2007), and interacts with the right pVLPFC and occipitoparietal regions for model testing. Successful model selection can subsequently activate associated information in semantic memory, and semantic retrieval recruits the ITS (Tulving 1972; Kan and Thompson-Schill 2004), which may provide convergent information about both nonsemantic and semantic knowledge (Damasio et al. 1996).

For model testing, the CoS/LS area can be additionally recruited. We found evidence that right CoS/LS is view-sensitive, except for uncategorized objects, which by definition, have failed to match to any nonsemantic visuo-(structural) or semantic memory. The right CoS/LS has been found to be activated on N-back and recognition tasks for scenes, landmarks, and buildings relative to other images, including objects, and shows view-specificity (Aguirre et al. 1998; Epstein and Kanwisher 1998; Epstein et al. 2003, 2005, 2007; Hasson et al. 2003; Tsivilis et al. 2003; Wheeler and Buckner 2003). CoS/LS may be analogous to a part of higher-order visual area TE in monkeys, providing key input to medial temporal structures for explicit episodic memory, including parahippocampal gyrus, activation of which it can be associated (Fig. 5) (Suzuki and Amaral 1994; Stern et al. 1996; Buffalo et al. 1999; Kirchhoff et al. 2000; Hasson et al. 2003; Rosenbaum et al. 2004; Yonelinas et al. 2005; Wais et al. 2006). CoS/LS effects often co-occur also with effects in TOS/vcIPS and retrosplenial cortex (Epstein et al. 2007), consistent with our results, and we find co-occurrence also with ITS on recognition, mental rotation (Schendan and Stern 2007), and working memory tasks (Schon et al. 2004), suggesting CoS/LS integrates input from the ITS, TOS, and vcIPS. We suggest that the CoS/LS area is recruited for perceiving and representing a more unitary spatial configuration of the features or parts of scenes, landmarks, or other objects as needed by the task to evaluate the detailed visuospatial match between an internal representation and a current percept or another internal representation(s). Detailed spatial information can help individuate between perceptually similar instances of a place or an object.

Conclusions

Visual object categorization and recognition tasks are where vision meets memory. The 2SI account is an initial attempt to develop a cognitive framework that integrates vision and memory theories. This account is compatible with psychophysical and computational models, including multiple views theories (perhaps facilitating processes of interpolation, evidence accumulation, or pooling across a neural population), and bottom-up architectures that could provide the initial object model(s) for the predictiontesting loops (Serre et al. 2007). By a 2SI account, after 200 ms, distinct VLPFC–posterior networks are engaged to facilitate object constancy. An aVLPFC–LOS network supports prediction verification processes on categorization more than recognition. A pVLFPC–vcIPS–TOS–ITS network supports testing verification processes on diverse tasks. Right CoS/LS is also involved for not only view representation but also detailed spatial matching. Most important, the present findings demonstrate that not only the ventral stream but also object-sensitive occipitoparietal cortex (vcIPS, TOS), which is critical for mental rotation, also plays a key role in object constancy, computing the relative spatial relations among object parts, conscious spatial binding, and transforming hierarchical visuospatial structure. This framework may apply to other perceptual and memory situations in other sensory modalities when stimulus interpretation is ambiguous.

Supplementary Material

Supplementary material can be found at: http://www.cercor.oxfordjournals.org/.

Funding

National Institute on Aging grant (F32 AG05914) and Tufts University Faculty start-up funds to H.E.S.; National Institute of Mental Health grant (P50 MH071702) to C.E.S. and H.E.S.; National Center for Research Resources grant (P41RR14075); and the Mental Illness and Neuroscience Discovery (MIND) Institute.

Brain scanning was conducted at the MGH/MIT/HMS Athinoula A. Martinos Center for Biomedical Imaging. The Optseq software used to optimize fMRI signal discriminability was developed and made freely available at the Martinos Center. This study was conducted in the Vision and Memory Neuroimaging Lab, Department of Psychology at Tufts University and the Vision and Cognition Lab and Cognitive Neuroimaging Lab at the Center for Memory and Brain, Department of Psychology, Boston University. We are grateful to Courtney Horwitz, Mimi Castelo, Sule Tinaz, MD, PhD, Michael J. Tavantzis, and Reyyan A. Bilge for assistance with data acquisition and/ or analyses, and Dr Alice Cronin-Golomb for computer hardware support. Many thanks to Dr. Karin Schon for examining results from Schon et al. (2004) for delay and subsequent memory effects in the subregion of CoS/LS, and providing the results included in Figure 5 and Supplemental Figure 6 (panels D and G in both). We thank Dr Giorgio Ganis for helpful discussion. Conflict of Interest: None declared.

References

Aguirre
GK
Zarahn
E
D'Esposito
M
An area within human ventral cortex sensitive to “building” stimuli: evidence and implications
Neuron
 , 
1998
, vol. 
21
 (pg. 
373
-
383
)
Altmann
CF
Grodd
W
Kourtzi
Z
Bulthoff
HH
Karnath
HO
Similar cortical correlates underlie visual object identification and orientation judgment
Neuropsychologia
 , 
2005
, vol. 
43
 (pg. 
2101
-
2108
)
Arbib
MA
From monkey-like action recognition to human language: an evolutionary framework for neurolinguistics
Behav Brain Sci.
 , 
2005
, vol. 
28
 (pg. 
105
-
167
)
Assad
JA
Neural coding of behavioral relevance in parietal cortex
Curr Opin Neurobiol.
 , 
2003
, vol. 
13
 (pg. 
194
-
197
)
Backus
BT
Fleet
DJ
Parker
AJ
Heeger
DJ
Human cortical activity correlates with stereoscopic depth perception
J Neurophysiol
 , 
2001
, vol. 
86
 (pg. 
2054
-
2068
)
Bar
M
A cortical mechanism for triggering top-down facilitation in visual object recognition
J Cogn Neurosci
 , 
2003
, vol. 
15
 (pg. 
600
-
609
)
Bar
M
Kassam
KS
Ghuman
AS
Boshyan
J
Schmid
AM
Dale
AM
Hamalainen
MS
Marinkovic
K
Schacter
DL
Rosen
BR
, et al.  . 
Top-down facilitation of visual recognition
Proc Natl Acad Sci USA
 , 
2006
, vol. 
103
 (pg. 
449
-
454
)
Bar
M
Tootell
RB
Schacter
DL
Greve
DN
Fischl
B
Mendola
JD
Rosen
BR
Dale
AM
Cortical mechanisms specific to explicit visual object recognition
Neuron
 , 
2001
, vol. 
29
 (pg. 
529
-
535
)
Bauer
B
Jolicoeur
P
Stimulus dimensionality effects in mental rotation
J Exp Psychol Hum Percept Perform
 , 
1996
, vol. 
22
 (pg. 
82
-
94
)
Beauchamp
MS
Lee
KE
Haxby
JV
Martin
A
Parallel visual motion processing streams for manipulable objects and human movements
Neuron
 , 
2002
, vol. 
34
 (pg. 
149
-
159
)
Behrmann
M
Peterson
MA
Moscovitch
M
Suzuki
S
Independent representation of parts and the relations between them: evidence from integrative agnosia
J Exp Psychol Hum Percept Perform
 , 
2006
, vol. 
32
 (pg. 
1169
-
1184
)
Blanz
V
Tarr
MJ
Bulthoff
HH
What object attributes determine canonical views?
Perception
 , 
1999
, vol. 
28
 (pg. 
575
-
599
)
Bruce
V
Young
A
Understanding face recognition
Br J Psychol
 , 
1986
, vol. 
77
 
3
(pg. 
305
-
327
)
Buffalo
EA
Ramus
SJ
Clark
RE
Teng
E
Squire
LR
Zola
SM
Dissociation between the effects of damage to perirhinal cortex and area TE
Learn Mem
 , 
1999
, vol. 
6
 (pg. 
572
-
599
)
Bülthoff
HH
Edelman
SY
Tarr
MJ
How are three-dimensional objects represented in the brain?
Cereb Cortex
 , 
1995
, vol. 
5
 (pg. 
247
-
260
)
Burgund
ED
Marsolek
CJ
Viewpoint-invariant and viewpoint-dependent object recognition in dissociable neural subsystems
Psychon Bull Rev.
 , 
2000
, vol. 
7
 (pg. 
480
-
489
)
Carpenter
PA
Just
MA
Keller
TA
Eddy
W
Thulborn
K
Graded functional activation in the visuospatial system with the amount of task demand
J Cogn Neurosci
 , 
1999
, vol. 
11
 (pg. 
9
-
24
)
Cave
CB
Kosslyn
SM
The role of parts and spatial relations in object identification
Perception
 , 
1993
, vol. 
22
 (pg. 
229
-
248
)
Chafee
MV
Crowe
DA
Averbeck
BB
Georgopoulos
AP
Neural correlates of spatial judgement during object construction in parietal cortex
Cereb Cortex
 , 
2005
, vol. 
15
 (pg. 
1393
-
1413
)
Chao
LL
Martin
A
Representation of manipulable man-made objects in the dorsal stream
NeuroImage
 , 
2000
, vol. 
12
 (pg. 
478
-
484
)
Cohen
JD
MacWhinney
B
Flatt
M
Provost
J
PsyScope: a new graphic interactive environment for designing psychology experiments
Behav Res Methods Instrum Comput.
 , 
1993
, vol. 
25
 (pg. 
257
-
271
)
Constantinidis
C
Steinmetz
MA
Posterior parietal cortex automatically encodes the location of salient stimuli
J Neurosci
 , 
2005
, vol. 
25
 (pg. 
233
-
238
)
Courtney
SM
Attention and cognitive control as emergent properties of information representation in working memory
Cogn Affect Behav Neurosci
 , 
2004
, vol. 
4
 (pg. 
501
-
516
)
Curran
T
Tanaka
JW
Weiskopf
DM
An electrophysiological comparison of visual categorization and recognition memory
Cogn Affect Behav Neurosci
 , 
2002
, vol. 
2
 (pg. 
1
-
18
)
Cutzu
F
Edelman
S
Canonical views in object representation and recognition
Vision Res.
 , 
1994
, vol. 
34
 (pg. 
3037
-
3056
)
Damasio
H
Grabowski
TJ
Tranel
D
Hichwa
RD
Damasio
AR
A neural basis for lexical retrieval
Nature
 , 
1996
, vol. 
380
 (pg. 
499
-
505
)
David
O
Harrison
L
Friston
KJ
Modeling event-related responses in the brain
NeuroImage
 , 
2005
, vol. 
25
 (pg. 
756
-
770
)
Denys
K
Vanduffel
W
Fize
D
Nelissen
K
Peuskens
H
Van Essen
D
Orban
GA
The processing of visual shape in the cerebral cortex of human and nonhuman primates: a functional magnetic resonance imaging study
J Neurosci
 , 
2004
, vol. 
24
 (pg. 
2551
-
2565
)
Desimone
R
Duncan
J
Neural mechanisms of selective visual attention
Annu Rev Neurosci
 , 
1995
, vol. 
18
 (pg. 
193
-
222
)
Duvernoy
HM
Bourqouin
P
Cabanis
EA
Cattin
F
The human brain: surface, three-dimensional sectional anatomy with MRI, and blood supply
 , 
1999
New York
Springer
Ecker
C
Brammer
MJ
David
AS
Williams
SC
Time-resolved fMRI of mental rotation revisited—dissociating visual perception from mental rotation in female subjects
NeuroImage
 , 
2006
, vol. 
32
 (pg. 
432
-
444
)
Edelman
S
Bulthoff
HH
Orientation dependence in the recognition of familiar and novel views of three-dimensional objects
Vision Res.
 , 
1992
, vol. 
32
 (pg. 
2385
-
2400
)
Epstein
RA
Graham
KS
Downing
PE
Viewpoint-specific scene representations in human parahippocampal cortex
Neuron
 , 
2003
, vol. 
37
 (pg. 
865
-
876
)
Epstein
RA
Harris
A
Stanley
D
Kanwisher
N
The parahippocampal place area: recognition, navigation, or encoding?
Neuron
 , 
1999
, vol. 
23
 (pg. 
115
-
125
)
Epstein
RA
Higgins
JS
Jablonski
K
Feiler
AM
Visual scene processing in familiar and unfamiliar environments
J Neurophysiol
 , 
2007
, vol. 
97
 (pg. 
3670
-
3683
)
Epstein
RA
Higgins
JS
Thompson-Schill
SL
Learning places from views: variation in scene processing as a function of experience and navigational ability
J Cogn Neurosci
 , 
2005
, vol. 
17
 (pg. 
73
-
83
)
Epstein
RA
Kanwisher
N
A cortical representation of the local visual environment
Nature
 , 
1998
, vol. 
392
 (pg. 
598
-
601
)
Faillenot
I
Sunaert
S
Van Hecke
P
Orban
GA
Orientation discrimination of objects and gratings compared: an fMRI study
Eur J Neurosci
 , 
2001
, vol. 
13
 (pg. 
585
-
596
)
Farah
MJ
Visual agnosia: disorders of object recognition and what they tell us about normal vision
1990
 
Cambridge (MA): MIT Press
Farah
MJ
Hammond
KM
Mental rotation and orientation-invariant object recognition: dissociable processes
Cognition
 , 
1988
, vol. 
29
 (pg. 
29
-
46
)
Folk
MD
Luce
RD
Effects of stimulus complexity on mental rotation rate of polygons
J Exp Psychol Hum Percept Perform
 , 
1987
, vol. 
13
 (pg. 
395
-
404
)
Fox
MD
Snyder
AZ
Vincent
JL
Corbetta
M
Van Essen
DC
Raichle
ME
The human brain is intrinsically organized into dynamic, anticorrelated functional networks
Proc Natl Acad Sci USA
 , 
2005
, vol. 
102
 (pg. 
9673
-
9678
)
Friedman-Hill
SR
Robertson
LC
Desimone
R
Ungerleider
LG
Posterior parietal cortex and the filtering of distractors
Proc Natl Acad Sci USA
 , 
2003
, vol. 
100
 (pg. 
4263
-
4268
)
Friedman-Hill
SR
Robertson
LC
Treisman
AM
Parietal contributions to visual feature binding: evidence from a patient with bilateral lesions
Science
 , 
1995
, vol. 
269
 (pg. 
853
-
855
)
Friston
KJ
Testing for anatomically specified regional effects
Hum Brain Mapp
 , 
1997
, vol. 
5
 (pg. 
133
-
136
)
Ganis
G
Schendan
HE
Kosslyn
SM
Neuroimaging evidence for object model verification theory: role of prefrontal control in visual object categorization
NeuroImage
 , 
2007
, vol. 
34
 (pg. 
384
-
398
)
Gauthier
I
Hayward
WG
Tarr
MJ
Anderson
AW
Skudlarski
P
Gore
JC
BOLD activity during mental rotation and viewpoint-dependent object recognition
Neuron
 , 
2002
, vol. 
34
 (pg. 
161
-
171
)
Genovese
CR
Lazar
NA
Nichols
T
Thresholding of statistical maps in functional neuroimaging using the false discovery rate
NeuroImage
 , 
2002
, vol. 
15
 (pg. 
870
-
878
)
Gilaie-Dotan
S
Ullman
S
Kushnir
T
Malach
R
Shape-selective stereo processing in human object-related visual areas
Hum Brain Mapp
 , 
2001
, vol. 
15
 (pg. 
67
-
79
)
Goodale
MA
Milner
AD
Separate visual pathways for perception and action
Trends Neurosci
 , 
1992
, vol. 
15
 (pg. 
20
-
25
)
Grezes
J
Armony
JL
Rowe
J
Passingham
RE
Activations related to “mirror” and “canonical” neurones in the human brain: an fMRI study
NeuroImage
 , 
2003
, vol. 
18
 (pg. 
928
-
937
)
Grill-Spector
K
Kourtzi
Z
Kanwisher
N
The lateral occipital complex and its role in object recognition
Vision Res.
 , 
2001
, vol. 
41
 (pg. 
1409
-
1422
)
Grill-Spector
K
Kushnir
T
Edelman
S
Avidan
G
Itzchak
Y
Malach
R
Differential processing of objects under various viewing conditions in the human lateral occipital complex
Neuron
 , 
1999
, vol. 
24
 (pg. 
187
-
203
)
Grill-Spector
K
Kushnir
T
Edelman
S
Itzchak
Y
Malach
R
Cue-invariant activation in object-related areas of the human occipital lobe
Neuron
 , 
1998
, vol. 
21
 (pg. 
191
-
202
)
Grill-Spector
K
Kushnir
T
Hendler
T
Malach
R
The dynamics of object-selective activation correlate with recognition performance in humans
Nat Neurosci
 , 
2000
, vol. 
3
 (pg. 
837
-
843
)
Grossberg
S
The link between brain learning, attention, and consciousness
Conscious Cogn
 , 
1999
, vol. 
8
 (pg. 
1
-
44
)
Harris
IM
Egan
GF
Sonkkila
C
Tochon-Danguy
HJ
Paxinos
G
Watson
JDG
Selective right parietal lobe activation during mental rotation: a parametric PET study
Brain
 , 
2000
, vol. 
123
 (pg. 
65
-
73
)
Hasson
U
Harel
M
Levy
I
Malach
R
Large-scale mirror-symmetry organization of human occipito-temporal object areas
Neuron
 , 
2003
, vol. 
37
 (pg. 
1027
-
1041
)
Hintzman
DL
The psychology of learning and memory
 , 
1978
San Francisco
Freeman
Humphreys
GW
Price
CJ
Riddoch
MJ
From objects to names: a cognitive neuroscience approach
Psychol Res.
 , 
1999
, vol. 
62
 (pg. 
118
-
130
)
James
TW
Humphrey
GK
Gati
JS
Menon
RS
Goodale
MA
Differential effects of viewpoint on object-driven activation in dorsal and ventral streams
Neuron
 , 
2002
, vol. 
35
 (pg. 
793
-
801
)
Jeannerod
M
Jacob
P
Visual cognition: a new look at the two-visual systems model
Neuropsychologia
 , 
2005
, vol. 
43
 (pg. 
301
-
312
)
Jolicoeur
P
The time to name disoriented natural objects
Mem Cogn
 , 
1985
, vol. 
13
 (pg. 
289
-
303
)
Jolicoeur
P
Landau
MJ
Effects of orientation on the identification of simple visual patterns
Can J Psychol
 , 
1984
, vol. 
38
 (pg. 
80
-
93
)
Kan
IP
Kable
JW
Van Scoyoc
A
Chatterjee
A
Thompson-Schill
SL
Fractionating the left frontal response to tools: dissociable effects of motor experience and lexical competition
J Cogn Neurosci
 , 
2006
, vol. 
18
 (pg. 
267
-
277
)
Kan
IP
Thompson-Schill
SL
Effect of name agreement on prefrontal activity during overt and covert picture naming
Cogn Affect Behav Neurosci
 , 
2004
, vol. 
4
 (pg. 
43
-
57
)
Kanwisher
N
Wojciulik
E
Visual attention: insights from brain imaging
Nat Rev Neurosci
 , 
2000
, vol. 
1
 (pg. 
91
-
100
)
Kirchhoff
BA
Wagner
AD
Maril
A
Stern
CE
Prefrontal-temporal circuitry for episodic encoding and subsequent memory
J Neurosci
 , 
2000
, vol. 
20
 (pg. 
6173
-
6180
)
Kosslyn
SM
Alpert
NM
Thompson
WL
Chabris
CF
Rauch
SL
Anderson
AK
Identifying objects seen from different viewpoints: a PET investigation
Brain
 , 
1994
, vol. 
117
 (pg. 
1055
-
1071
)
Kostopoulos
P
Albanese
MC
Petrides
M
Ventrolateral prefrontal cortex and tactile memory disambiguation in the human brain
Proc Natl Acad Sci USA
 , 
2007
, vol. 
104
 (pg. 
10223
-
10228
)
Kourtzi
Z
Bulthoff
HH
Erb
M
Grodd
W
Object-selective responses in the human motion area MT/MST
Nat Neurosci
 , 
2002
, vol. 
5
 (pg. 
17
-
18
)
Kourtzi
Z
Kanwisher
N
Cortical regions involved in perceiving object shape
J Neurosci
 , 
2000
, vol. 
20
 (pg. 
3310
-
3318
)
Kucera
H
Francis
WN
Computational analysis of present-day American English
 , 
1967
Providence (RI)
Brown University Press
Lamme
VA
Roelfsema
PR
The distinct modes of vision offered by feedforward and recurrent processing
Trends Neurosci
 , 
2000
, vol. 
23
 (pg. 
571
-
579
)
Larsson
J
Heeger
DJ
Two retinotopic visual areas in human lateral occipital cortex
J Neurosci
 , 
2006
, vol. 
26
 (pg. 
13128
-
13142
)
Lee
TS
Mumford
D
Hierarchical Bayesian inference in the visual cortex
J Opt Soc Am A Opt Image Sci Vis
 , 
2003
, vol. 
20
 (pg. 
1434
-
1448
)
Lowe
DG
Perceptual organisation and visual recognition
 , 
1985
Boston
Kluwer Academic
Lowe
DG
Towards a computational model for object recognition in IT cortex
First IEEE International Workshop on Biologically Motivated Computer Vision
 , 
2000
 
London: Springer. p. 20-31
Malach
R
Levy
I
Hasson
U
The topography of high-order human object areas
Trends Cogn Sci.
 , 
2002
, vol. 
6
 (pg. 
176
-
184
)
Malach
R
Reppas
JB
Benson
RR
Kwong
KK
Jiang
H
Kennedy
WA
Ledden
PJ
Brady
TJ
Rosen
BR
Tootell
RB
Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex
Proc Natil Acad Sci USA
 , 
1995
, vol. 
92
 (pg. 
8135
-
8139
)
Milner
AD
Goodale
MA
Visual pathways to perception and action
Prog Brain Res.
 , 
1993
, vol. 
95
 (pg. 
317
-
337
)
Moore
C
Engel
SA
Neural response to perception of volume in the lateral occipital complex
Neuron
 , 
2001
, vol. 
29
 (pg. 
277
-
286
)
Newell
FN
Sheppard
DM
Edelman
S
Shapiro
KL
The interaction of shape- and location-based priming in object categorisation: evidence for a hybrid “what + where” representation stage
Vision Res.
 , 
2005
, vol. 
45
 (pg. 
2065
-
2080
)
Noppeney
U
Price
CJ
Penny
WD
Friston
KJ
Two distinct neural mechanisms for category-selective responses
Cereb Cortex
 , 
2006
, vol. 
16
 (pg. 
437
-
445
)
Oldfield
R
The assessment and analysis of handedness: The Edinburgh inventory
Neuropsychologia
 , 
1971
, vol. 
9
 (pg. 
97
-
113
)
Palmer
SE
Rosch
E
Chase
P
Long
J
Baddeley
A
Canonical perspective and the perception of objects. Attention and performance
1981
Hillsdale (NJ)
Erlbaum
(pg. 
135
-
151
)
Passingham
RE
Toni
I
Rushworth
MF
Specialisation within the prefrontal cortex: the ventral prefrontal cortex and associative learning
Exp Brain Res.
 , 
2000
, vol. 
133
 (pg. 
103
-
113
)
Peissig
JJ
Tarr
MJ
Visual object recognition: do we know more now than we did 20 years ago?
Annu Rev Psychol
 , 
2007
, vol. 
58
 (pg. 
75
-
96
)
Perrett
DI
Harries
MH
Looker
S
Use of preferential inspection to define the viewing sphere and characteristic views of an arbitrary machined tool part
Perception
 , 
1992
, vol. 
21
 (pg. 
497
-
515
)
Perrett
DI
Oram
MW
Ashbridge
E
Evidence accumulation in cell populations responsive to faces: an account of generalisation of recognition without mental transformations
Cognition
 , 
1998
, vol. 
67
 (pg. 
111
-
145
)
Petrides
M
Lateral prefrontal cortex: architectonic and functional organization
Philos Trans R Soc Lond B Biol Sci.
 , 
2005
, vol. 
360
 (pg. 
781
-
795
)
Petrides
M
Pandya
DN
Comparative cytoarchitectonic analysis of the human and the macaque ventrolateral prefrontal cortex and corticocortical connection patterns in the monkey
Eur J Neurosci
 , 
2002
, vol. 
16
 (pg. 
291
-
310
)
Podzebenko
K
Egan
GF
Watson
JD
Widespread dorsal stream activation during a parametric mental rotation task, revealed with functional magnetic resonance imaging
NeuroImage
 , 
2002
, vol. 
15
 (pg. 
547
-
558
)
Postle
BR
Working memory as an emergent property of the mind and brain
Neuroscience
 , 
2006
, vol. 
139
 (pg. 
23
-
38
)
Price
CJ
Devlin
JT
The myth of the visual word form area
NeuroImage
 , 
2003
, vol. 
19
 (pg. 
473
-
481
)
Robertson
LC
Attentional persistence for features of hierarchical patterns
J Exp Psychol Gen
 , 
1996
, vol. 
125
 (pg. 
227
-
249
)
Rosch
E
Mervis
CB
Gray
W
Johnson
D
Boyes-Braem
P
Basic objects in natural categories
Cognit Psychol
 , 
1976
, vol. 
8
 (pg. 
382
-
439
)
Rosenbaum
RS
Ziegler
M
Winocur
G
Grady
CL
Moscovitch
M
“I have often walked down this street before”: fMRI studies on the hippocampus and other structures during mental navigation of an old environment
Hippocampus
 , 
2004
, vol. 
14
 (pg. 
826
-
835
)
Sasaki
Y
Vanduffel
W
Knutsen
T
Tyler
C
Tootell
R
Symmetry activates extrastriate visual cortex in human and nonhuman primates
Proc Natl Acad Sci USA
 , 
2005
, vol. 
102
 (pg. 
3159
-
3163
)
Schacter
DL
Addis
DR
Buckner
RL
Remembering the past to imagine the future: the prospective brain
Nat Rev Neurosci
 , 
2007
, vol. 
8
 (pg. 
657
-
661
)
Schacter
DL
Tulving
E
Schacter
DL
Tulving
E
What are the memory systems of 1994?. In: Memory systems 1994
1994
Cambridge (MA)
MIT Press
(pg. 
1
-
38
)
Schendan
HE
Ganis
G
Kutas
M
Neurophysiological evidence for visual perceptual categorization of words and faces within 150 ms
Psychophysiology
 , 
1998
, vol. 
35
 (pg. 
240
-
251
)
Schendan
HE
Kutas
M
Neurophysiological evidence for two processing times for visual object identification
Neuropsychologia
 , 
2002
, vol. 
40
 (pg. 
931
-
945
)
Schendan
HE
Kutas
M
Time course of processes and representations supporting visual object identification and memory
J Cogn Neurosci
 , 
2003
, vol. 
15
 (pg. 
111
-
135
)
Schendan
HE
Kutas
M
Neurophysiological evidence for the time course of activation of global shape, part, and local contour representations during visual object categorization and memory
J Cogn Neurosci
 , 
2007
, vol. 
19
 (pg. 
734
-
749
)
Schendan
HE
Stern
CE
Mental rotation and object categorization share a common network of prefrontal and dorsal and ventral regions of posterior cortex
NeuroImage
 , 
2007
, vol. 
35
 (pg. 
1264
-
1277
)
Schluppeck
D
Glimcher
P
Heeger
DJ
Topographic organization for delayed saccades in human posterior parietal cortex
J Neurophysiol
 , 
2005
, vol. 
94
 (pg. 
1372
-
1384
)
Schon
K
Hasselmo
ME
LoPresti
ML
Tricarico
MD
Stern
CE
Persistence of parahippocampal representation in the absence of stimulus input enhances long-term encoding: a functional magnetic resonance imaging study of subsequent memory after a delayed match-to-sample task
J Neurosci
 , 
2004
, vol. 
24
 (pg. 
11088
-
11097
)
Serre
T
Wolf
L
Bileschi
S
Riesenhuber
M
Poggio
T
Robust object recognition with cortex-like mechanisms
IEEE Trans Pattern Anal Mach Intell
 , 
2007
, vol. 
29
 (pg. 
411
-
426
)
Shepard
RN
Cooper
LA
Mental images and their transformations
 , 
1982
Cambridge (MA)
MIT Press
Shikata
E
Hamzei
F
Glauche
V
Koch
M
Weiller
C
Binkofski
F
Buchel
C
Functional properties and interaction of the anterior and posterior intraparietal areas in humans
Eur J Neurosci
 , 
2003
, vol. 
17
 (pg. 
1105
-
1110
)
Sigala
N
Gabbiani
F
Logothetis
NK
Visual categorization and object representation in monkeys and humans
J Cogn Neurosci
 , 
2002
, vol. 
14
 (pg. 
187
-
198
)
Silver
MA
Ress
D
Heeger
DJ
Topographic maps of visual spatial attention in human parietal cortex
J Neurophysiol
 , 
2005
, vol. 
94
 (pg. 
1358
-
1371
)
Snodgrass
JG
Corwin
J
Pragmatics of measuring recognition memory: applications to dementia and amnesia
J Exp Psychol Gen
 , 
1988
, vol. 
117
 (pg. 
34
-
50
)
Srinivas
K
Perceptual specificity in nonverbal priming
J Exp Psychol Learn Mem Cogn
 , 
1993
, vol. 
19
 (pg. 
582
-
602
)
Srinivas
K
Representation of rotated objects in explicit and implicit memory
J Exp Psychol Learn Mem Cogn
 , 
1995
, vol. 
21
 (pg. 
1019
-
1036
)
Stern
CE
Corkin
S
Gonzalez
RG
Guimaraes
AR
Baker
JR
Jennings
PJ
Carr
CA
Sugiura
RM
Vedantham
V
Rosen
BR
The hippocampal formation participates in novel picture encoding: evidence from functional magnetic resonance imaging
Proc Natl Acad Sci USA
 , 
1996
, vol. 
93
 (pg. 
8660
-
8665
)
Stern
CE
Owen
AM
Look
RB
Tracey
I
Rosen
BR
Petrides
M
Activity in ventrolateral and mid-dorsolateral prefrontal cortex during non-spatial visual working memory processing: evidence from functional magnetic resonance imaging
NeuroImage
 , 
2000
, vol. 
11
 (pg. 
392
-
399
)
Sugio
T
Inui
T
Matsuo
K
Matsuzawa
M
Glover
GH
Nakai
T
The role of the posterior parietal cortex in human object recognition: a functional magnetic resonance imaging study
Neurosci Lett.
 , 
1999
, vol. 
276
 (pg. 
45
-
48
)
Suzuki
WA
Amaral
DG
Perirhinal and parahippocampal cortices of the macaque monkey: cortical afferents
J Comp Neurol
 , 
1994
, vol. 
350
 (pg. 
497
-
533
)
Tanaka
K
Columns for complex visual object features in the inferotemporal cortex: clustering of cells with similar but slightly different stimulus selectivities
Cereb Cortex
 , 
2003
, vol. 
13
 (pg. 
90
-
99
)
Tarr
MJ
Rotating objects to recognize them: a case study on the role of viewpoint dependency in the recognition of three-dimensional objects
Psychon Bull Rev.
 , 
1995
, vol. 
2
 (pg. 
55
-
82
)
Tarr
MJ
Bulthoff
HH
Image-based object recognition in man, monkey and machine
Cognition
 , 
1998
, vol. 
67
 (pg. 
1
-
20
)
Tarr
MJ
Pinker
S
Mental rotation and orientation-dependence in shape recognition
Cognit Psychol
 , 
1989
, vol. 
21
 (pg. 
233
-
282
)
Tarr
MJ
Williams
P
Hayward
WG
Gauthier
I
Three-dimensional object recognition is viewpoint dependent
Nat Neurosci
 , 
1998
, vol. 
1
 (pg. 
275
-
277
)
Tarr
MJ
Yu
S
Object databank [CD-ROM]. Version 1.0. Providence (RI)
1996
 
Brown University
Treisman
AM
How the deployment of attention determines what we see
Vis Cogn
 , 
2006
, vol. 
14
 (pg. 
411
-
443
)
Treisman
AM
Kanwisher
NG
Perceiving visually presented objects: recognition, awareness, and modularity
Curr Opin Neurobiol.
 , 
1998
, vol. 
8
 (pg. 
218
-
226
)
Tsao
DY
Vanduffel
W
Sasaki
Y
Fize
D
Knutsen
TA
Mandeville
JB
Wald
LL
Dale
AM
Rosen
BR
Van Essen
DC
, et al.  . 
Stereopsis activates V3A and caudal intraparietal areas in macaques and humans
Neuron
 , 
2003
, vol. 
39
 (pg. 
555
-
568
)
Tsivilis
D
Otten
LJ
Rugg
MD
Repetition effects elicited by objects and their contexts: an fMRI study
Hum Brain Mapp
 , 
2003
, vol. 
19
 (pg. 
145
-
154
)
Tulving
E
Tulving
E
Donaldson
W
Episodic and semantic memory
Organization of memory
 , 
1972
New York
Academic Press
 
p. 381–403
Turnbull
OH
A double dissociation between knowledge of object identity and object orientation
Neuropsychologia
 , 
1997
, vol. 
35
 (pg. 
567
-
570
)
Turnbull
OH
Beschin
N
DellaSala
S
Agnosia for object orientation: Implications for theories of object recognition
Neuropsychologia
 , 
1997
, vol. 
35
 (pg. 
153
-
163
)
Turnbull
OH
Carey
DP
McCarthy
RA
The neuropsychology of object constancy
J Int Neuropsychol Soc.
 , 
1997
, vol. 
3
 (pg. 
288
-
298
)
Turnbull
OH
McCarthy
RA
When is a view unusual? A single case study of orientation-dependent visual agnosia
Brain Res Bull.
 , 
1996
, vol. 
40
 (pg. 
497
-
502
)
Ullman
S
Aligning pictorial descriptions: an approach to object recognition
Cognition
 , 
1989
, vol. 
32
 (pg. 
193
-
254
)
Ullman
S
Sequence seeking and counter streams: a computational model for bidirectional information flow in the visual cortex
Cereb Cortex
 , 
1995
, vol. 
5
 (pg. 
1
-
11
)
Ullman
S
High-level vision: object recognition and visual cognition
1996
 
Cambridge (MA): MIT Press
Ungerleider
LG
Gaffan
D
Pelak
VS
Projections from inferior temporal cortex to prefrontal cortex via the uncinate fascicle in rhesus monkeys
Exp Brain Res.
 , 
1989
, vol. 
76
 (pg. 
473
-
484
)
Ungerleider
LG
Mishkin
M
Ingle
DJ
Goodale
MA
Mansfield
RJW
Two cortical visual systems
Analysis of visual behavior
 , 
1982
Cambridge (MA)
MIT Press
(pg. 
549
-
586
)
Van Oostende
S
Sunaert
S
Van Hecke
P
Marchal
G
Orban
GA
The kinetic occipital (KO) region in man: an fMRI study
Cereb Cortex
 , 
1997
, vol. 
7
 (pg. 
690
-
701
)
van Turennout
M
Bielamowicz
L
Martin
A
Modulation of neural activity during object naming: effects of time and practice
Cereb Cortex
 , 
2003
, vol. 
13
 (pg. 
381
-
391
)
Verfaillie
K
Boutsen
L
A corpus of 714 full-color images of depth-rotated objects
Percept Psychophys
 , 
1995
, vol. 
57
 (pg. 
925
-
961
)
Vincent
JL
Snyder
AZ
Fox
MD
Shannon
BJ
Andrews
JR
Raichle
ME
Buckner
RL
Coherent spontaneous activity identifies a hippocampal-parietal memory network
J Neurophysiol
 , 
2006
, vol. 
96
 (pg. 
3517
-
3531
)
Vuilleumier
P
Henson
RN
Driver
J
Dolan
RJ
Multiple levels of visual object constancy revealed by event-related fMRI of repetition priming
Nat Neurosci
 , 
2002
, vol. 
5
 (pg. 
491
-
499
)
Wais
PE
Wixted
JT
Hopkins
RO
Squire
LR
The hippocampus supports both the recollection and the familiarity components of recognition memory
Neuron
 , 
2006
, vol. 
49
 (pg. 
459
-
466
)
Warrington
EK
Neuropsychological studies of object recognition
Philos Trans R Soc Lond B Biol Sci.
 , 
1982
, vol. 
298
 (pg. 
15
-
33
)
Wheeler
ME
Buckner
RL
Functional dissociation among components of remembering: control, perceived oldness, and content
J Neurosci
 , 
2003
, vol. 
23
 (pg. 
3869
-
3880
)
Wilson
KD
Farah
MJ
Distinct patterns of viewpoint-dependent BOLD activity during common-object recognition and mental rotation
Perception
 , 
2006
, vol. 
35
 (pg. 
1351
-
1366
)
Wojciulik
E
Kanwisher
N
The generality of parietal involvement in visual attention
Neuron
 , 
1999
, vol. 
23
 (pg. 
747
-
764
)
Worsley
KJ
Marrett
S
Neelin
P
Vandal
AC
Friston
KJ
Evans
AC
A unified statistical approach for determining significant signals in images of cerebral activation
Hum Brain Mapp
 , 
1996
, vol. 
4
 (pg. 
58
-
73
)
Xu
Y
Chun
MM
Dissociable neural mechanisms supporting visual short-term memory for objects
Nature
 , 
2006
, vol. 
440
 (pg. 
91
-
95
)
Yonelinas
AP
Otten
LJ
Shaw
KN
Rugg
MD
Separating the brain regions involved in recollection and familiarity in recognition memory
J Neurosci
 , 
2005
, vol. 
25
 (pg. 
3002
-
3008
)