Abstract

The relationship between neural activity and object perception has received considerable attention using stimulus manipulations such as masking or dichoptic presentation. Here we investigate the same problem by occluding objects with an opaque screen that acts to dissociate the direct perception of the object from the awareness of its presence. We used functional magnetic resonance imaging to measure brain activity when subjects viewed objects (faces and houses) that underwent occlusion and found that the response of the majority of the fusiform face area (FFA) and lateral occipital cortex is the same whether the object is visible or occluded. This suggests that when objects are directly viewed, activity within object-selective regions may reflect the awareness of presence, not the direct perception, of the object. Additionally, we identify a region of premotor cortex that is selectively activated by occlusion of either object type, suggesting its generic involvement with processing occluded objects.

Introduction

Objects in the world continually pass in front of one another and in doing so occlude each other from sight. Occlusion creates a problem because it means that the availability of visual information becomes erratic and unpredictable, whereas the actual presence of things such as predators, prey, and objects is independent of whether they are occluded or not. A brain that is able to process the persistent presence of objects even when they are not directly perceived would be at an obvious evolutionary advantage. Indeed, there is considerable evidence that humans, including infants, and many animal species, behave as if occluded objects are still present (Michotte 1950; Baillargeon and others 1985; Baillargeon 1986; Sekuler and others 1994; Behrmann and others 1998; Kellman and others 1998; Joseph and Nakayama 1999; Scholl and Pylyshyn 1999; Mareschal 2000; Haimson and others 2001; Rauschenberger and Yantis 2001; Churchland and others 2003; Johnson 2004; Yi and others 2003).

The capacity of humans to process objects that become occluded is especially interesting to investigations of visual perception and consciousness. When we view an occluded object, we only perceive the surface of the occluder and yet are aware not only of the identity of the occluded object but also of its location and spatial extent. Original accounts of occluded objects described this phenomenon as being “visually amodal” (Michotte 1950). This intuitively contradictory description has a rather elegant logic. On the one hand, the occluded object is “amodal” in that there is no local sensory experience of its attributes, and yet on the other hand, it is visual in modality in the sense that it is determined by contextual visual information (Palmer 1999). Perhaps the most interesting property of occlusion is that the awareness of an object's presence persists despite the absence of its direct perception. Occluded objects exemplify a clear dissociation between perception and awareness of an object's presence and therefore constitute an important opportunity for investigating the neural correlates of visual perception and of consciousness. This study uses occlusion as a means of dissociating neural activity that correlates with the awareness of an object's presence from that which correlates with the direct perception of its attributes. By virtue of the fact that the brain is functionally specialized (Zeki 1978; Zeki 1993), we have a priori hypotheses about which areas will be involved in the processing of different objects (for review, see Kanwisher 2003). Previous studies have shown that the transition from the processing to the perceptual (which we define as being conscious) state correlates with heightened activity in the relevant functionally specialized area (Zeki and ffytche 1998; Rees and others 2000; Dehaene and others 2001; Moutoussis and Zeki 2002; Ress and Heeger 2003), suggesting that the level of activity within such areas correlates with the perception of those attributes for which they are specialized. Given that occlusion dissociates direct perception from awareness of presence, this leads us to inquire how functionally specialized areas respond to the occlusion of objects.

In our study, we presented subjects with 2 types of object stimuli (images of faces or houses), which were positioned either in front of (visible) or behind (occluded) an upwardly moving opaque screen. Functional magnetic resonance imaging (fMRI) was used to measure the brain activity under these 2 conditions, allowing us to distinguish activity reflecting visible stimuli from that reflecting occluded (but previously seen) stimuli alone. If the response of an object-selective area is due to the direct perception of an object, then its response to an occluded object will be attenuated. Alternatively, if the object-selective response is due to the awareness of an object's presence independently of the direct perception of its surface, then the object-selective area will respond with equal strength whether the object is visible or occluded.

Materials and Methods

Object stimuli consisted of achromatic photographs of 10 faces (provided by the Max-Planck Institute for Biological Cybernetics, Tuebingen, Germany) and 10 houses (photographs taken specifically for the experiment) in frontal view, occupying approximately 5° of visual angle. Figure 1 illustrates the 6 different conditions for each object type. Each of the conditions could be displayed in one of 3 modes “visible (VS),” “occluded (OC),” or “segment (SG)” mode. In each mode, an opaque screen (in the image, not a real screen) moved upward for 1.5 s to cover the lower 98% of the surface area of the object, such that only the very top segment was visible for the remaining 7.5 s. In the VS mode, the screen moved upward behind the stimulus so that the stimulus was fully visible for the first 1.5 s. In the OC mode, the screen moved in front of the stimulus so that it became almost fully occluded after 1.5 s. The SG mode is different in that the stimuli are segments of the original stimulus (cropped so that only the very top 2% of surface area remains). The opaque screen moves up toward the segment such that the latter is fully visible for the first 1.5 s.

Figure 1.

Schematic depiction of the 6 conditions for faces (equivalent conditions for house stimuli not pictured). Time runs from top to bottom. There are 3 stimuli types, Occluded (OC), Visible (VS), and Segment (SG), each with their own control, Occluded Vanishing (OCv), Visible Vanishing (VSv), and Segment Vanishing (SGv). The vanishing controls are identical to their counterparts for the first 1.5 s after which the object (in this example the face or the face segment) vanishes and is absent for the remaining 7.5 s. The disappearance is depicted with cartoon vanishing lines that are for illustration purposes only.

Figure 1.

Schematic depiction of the 6 conditions for faces (equivalent conditions for house stimuli not pictured). Time runs from top to bottom. There are 3 stimuli types, Occluded (OC), Visible (VS), and Segment (SG), each with their own control, Occluded Vanishing (OCv), Visible Vanishing (VSv), and Segment Vanishing (SGv). The vanishing controls are identical to their counterparts for the first 1.5 s after which the object (in this example the face or the face segment) vanishes and is absent for the remaining 7.5 s. The disappearance is depicted with cartoon vanishing lines that are for illustration purposes only.

After 1.5 s, the stimulus can follow 1 of 2 fates according to whether the trial is a main condition or a “vanishing” control (suffixed with v e.g., OCv). In the main conditions, the object persists (i.e., its segment is still visible) for the remaining 7.5 s of the trial, and in vanishing control conditions, it disappears (as indicated by vanishing lines for illustration in Fig. 1) for the remaining 7.5 s of the trial. All conditions were presented in a pseudorandom order in the context of 3-dimensional space (as shown in Supplementary Material), whereby the regions surrounding the object and screen consisted of depth contours and a horizon in the background. This was to make the occlusion event appear more realistic. Each trial lasted 9 s and was followed by a jittered rest period with a mean duration of 3 s.

The total scanning time was divided into 2–3 sessions (3 sessions for 6 subjects), each approximately 15 min in duration. Each session consisted of 84 complete trials, thus repeating each of the conditions 7 times in pseudorandom order. All stimulus images were imported into a program constructed using Cogent 2000 Graphics (available at www.vislab.ac.uk) running in MATLAB (The MathWorks, Inc., www.mathworks.com). Visual stimuli were generated by a dedicated PC located in the control room and projected via a LCD projector onto a transparent screen positioned over the subjects head and viewed through a tilted mirror fixed to the head coil.

Subjects

A total of 13 subjects (8 males and 5 females, mean age 25.4 years, 7 right handed) with normal or corrected to normal vision were scanned. All gave informed consent in accordance with the Declaration of Helsinki, and the Ethics Committee of the National Hospital for Neurology and Neurosurgery, London, UK, granted ethics approval for the study.

Task

To ensure that subjects maintained fixation and attention, they were instructed to fixate a central cross at all times. To ensure that subjects attended to the stimuli, they were instructed to respond via button press if there was a change in the fixation cross. This occurred in only 15% of trials (randomly distributed across trials and randomly timed within a trial). If no change occurred, they did not press any button. The performance of all subjects was recorded and found to be 95.5% (mean) correct. The hand with which each button was pressed, and handedness, was counterbalanced across all 13 subjects. Subjects had to perform the same task on every trial throughout the experiment. Eye-tracking data were monitored to check correct fixation.

Scanning Details

Scanning was done in a 1.5-T Siemens Sonata fMRI scanner with a head-volume coil (Siemens, Erlangen, Germany). A gradient echo-planar imaging (EPI) sequence was used to maximize blood oxygen level–dependent (BOLD) contrast (echo time = 90 ms, repeat time 3.42 s). Each brain image was acquired in a descending sequence comprising 38 axial slices each 2 mm thick with 1 mm gaps in-between, consisting of 64 × 64 voxels, and thus covering nearly the whole brain. Each session consisted of 252 volumes with the first 5 volumes being discarded to allow for T1-equilibration effects. Each subject was also scanned after the main scanning session with a T1-weighted structural sequence, using a phased array head coil to obtain a high-resolution structural image.

Data Analysis

We analyzed the data using SPM2 (statistical parametric mapping) software (Wellcome Department of Imaging Neuroscience, London, UK). The EPI images were realigned spatially, normalized to the Montreal Neurological Institute template provided in SPM2, smoothed spatially with a 12-mm isotropic Gaussian kernel, and filtered temporally with a band-pass filter of low-frequency cutoff period of 256 s. Global changes in activity were removed by proportional scaling. The data were realigned in time with sinc interpolation and in space with spatial normalization. The experiment was event related with each event being modeled with a separate boxcar function, convolved with SPM2's canonical hemodynamic response function. The analysis was performed at the random effects level, whereby the reliability of the measurements was assessed in relation to the between-subject variance (Friston and others 1999). Each of the 12 different conditions was modeled separately in the design matrix and treated as effects of interest. The 15% of trials in which a button response was made, and the head movement parameters, were modeled independently as effects of no interest.

The resultant parameter estimates for each regressor (at each voxel) were compared using t-tests to determine whether there were significant differences in activation between conditions. The statistical results given are based on a single-voxel t-statistic corresponding to P < 0.05 corrected for multiple comparisons (unless otherwise stated). The co-ordinates of all activations are reported in Talairach space (Talairach and Tournoux 1988). The actual BOLD response of interesting voxels revealed by the above approach was assessed using an event-triggered averaging approach. The raw signal was preprocessed in an identical fashion to the data used in the SPM analysis and was mean corrected. The BOLD responses for each epoch type were then aligned to the start of each event and binned into 2.5-s bins. These values were then used to calculate the average BOLD response across subjects from the beginning of the block until 30 s after block onset.

Object-specific regions of interest (ROIs) were defined with implicit and orthogonal localizer contrasts at the random effects level, [VS face > VS house] (faces) and [VS house > VS face] (houses), as all contiguous voxels exceeding P < 0.05 corrected. ROIs are therefore defined at the group level.

Psychophysiological Interaction Analysis

We also investigated the neural interactions between the ROIs (identified with orthogonal localizers as described above) and other cortical or subcortical regions through the application of psychophysiological interaction (PPI) analysis (Friston and others 1997). PPI analysis allows the detection of regionally specific responses in terms of the interaction between the input from another brain region and a psychological variable (such as occluded vs. visible objects). PPI employs one regressor representing the deconvolved activation time course in the ROIs, which constitutes the physiological variable, a second regressor representing the psychological variable (e.g., a regressor for OC face > VS face), and a third regressor representing the cross product of the previous 2 (the PPI term). An SPM can then be computed to reveal areas where activation is predicted by the PPI term, treating the physiological and psychological regressors as effects of no interest. We therefore extracted the deconvolved time course of activity in the ROIs (5-mm radius spheres centered on hottest voxel from the face and house localizers). The product of each activation time course and the psychological variable term was calculated to create the PPI term. PPIs were carried out for each ROI in each subject and entered into a random-effects group analysis (thresholded at P < 0.001 uncorrected and a cluster size of 5 voxels).

Results

The experimental aim was to compare brain activity caused by a visual stimulus that is perceived (VS) and the same stimulus when it is not perceived directly because it is occluded (OC). We were especially interested in looking for common activations induced by the 2 conditions. The comparison is not, however, straightforward. To achieve realistic occlusion, the stimulus needs to be visible before it is occluded. OC stimuli are therefore necessarily compound stimuli consisting of visible and occluded components. Because of this, it is not valid to simply compare the OC conditions with the VS conditions. Given the slow time course of the BOLD response, it was potentially problematic to distinguish, for instance, brain activity corresponding to the period in which the stimulus is occluded from that corresponding to the initial period in which it was visible. We therefore controlled for the initial visibility of the OC stimulus by using vanishing controls. Subtracting the appropriate vanishing control conditions from the main conditions should give a measure of the neural activity corresponding to the permanence of the stimulus, whether visible or occluded, independently of the response to its initial visibility. The SG conditions were designed to control for the visual appearance of the stimuli when occluded (OC) without any stimulus being occluded. For example, in the OC face condition when the face is occluded, the very top segment (2% of the surface area) of the head is still visible. The SG face condition therefore acts as a control for any neural activity corresponding to the appearance of the top of the head (there were of course equivalent segment controls for house stimuli). All data presented are at the group level (random effects).

Activity Common to Visible and Occluded Stimuli

To investigate the first question, of which areas were commonly active in VS and OC conditions (while controlling for initial exposure), we performed a conjunction analysis using SPMs of the minimum t-statistic over 2 independent (orthogonal) contrasts. Inference was based on P < 0.05 adjusted for the search volume using random field theory (Friston and others 2005; Nichols and others 2005). The null distribution for the minimum statistic is described by Friston (Friston and others 2005). This enabled us to infer a conjunction of multiple effects at significant voxels.

To locate voxels that show significant effects for both VS and OC faces, we performed a face-specific conjunction of [OC > OCv] AND [VS> VSv] comparisons. This revealed that, bilaterally, the fusiform gyrus was significantly more active in main conditions than in vanishing controls (See Fig. 2) for both VS faces and OC faces. The face localizer contrast identified the location of the peak fusiform face area (FFA) response (left FFA=−40 −54 −18, right FFA = 46 −60 −18) to be in close proximity to the peak response revealed in the conjunction for visible and occluded faces (left FFA = 42 −66 −18, right FFA= −38 −62 −20). The same logic was applied to finding regions responding to visible and occluded houses. The lateral occipital region was significantly more active in main conditions than in vanishing controls for both VS house and OC house stimuli. To reveal areas commonly activated by OC face and OC house, we performed the occlusion-specific conjunction analysis: [OC face > OCv face] AND [OC house > OCv house]. This revealed that left ventral premotor cortex (see Fig. 5) was significantly more active during OC conditions than any other condition.

Figure 2.

Central column depicts SPM images of brain activity common to visible and occluded faces (above) and houses (below). Top brain: Red voxels show significant effects for both visible and occluded faces expressed in the conjunction [OC > OCv] AND [VS > VSv] (for face stimuli only). Blue voxels show significant effects for the face-specific localizer (VS faces > VS houses). Purple voxels show overlap between conjunction and localizer. Common activity was restricted bilaterally to the fusiform gyrus. Bottom brain: Red voxels show significant effects for both visible and occluded houses expressed in the conjunction [OC > OCv] AND [VS > VSv] (for house stimuli only). Blue voxels show house-specific localizer (VS houses > VS faces). Purple voxels show overlap between conjunction and localizer. Common activity was restricted bilaterally to the LOC. Time course data extracted from hottest voxel (marked on brain with black crosses) defined by orthogonal face and house localizers. Because this data were spatially smoothed, the hottest voxel should be considered not taken as a single voxel but as an ROI averaged across a Gaussian kernel (12-mm full width half maximum) centered on the hottest voxel (Friston and others 2006) (see Fig. 3 for data averaged across the whole ROI). Time course plots show the average BOLD response across all subjects aligned to the start of each event from the local maxima of the localized ROIs. The error bars show the random effects standard errors across subjects. The pattern of activity for FFA and LOC is qualitatively similar; both display sustained responses to the visible and occluded objects (for which they are selective) that is conditional on the object remaining present throughout the trial. For vanishing trials, the FFA and LOC show a more transient response, whereas for segment trials there is little response.

Figure 2.

Central column depicts SPM images of brain activity common to visible and occluded faces (above) and houses (below). Top brain: Red voxels show significant effects for both visible and occluded faces expressed in the conjunction [OC > OCv] AND [VS > VSv] (for face stimuli only). Blue voxels show significant effects for the face-specific localizer (VS faces > VS houses). Purple voxels show overlap between conjunction and localizer. Common activity was restricted bilaterally to the fusiform gyrus. Bottom brain: Red voxels show significant effects for both visible and occluded houses expressed in the conjunction [OC > OCv] AND [VS > VSv] (for house stimuli only). Blue voxels show house-specific localizer (VS houses > VS faces). Purple voxels show overlap between conjunction and localizer. Common activity was restricted bilaterally to the LOC. Time course data extracted from hottest voxel (marked on brain with black crosses) defined by orthogonal face and house localizers. Because this data were spatially smoothed, the hottest voxel should be considered not taken as a single voxel but as an ROI averaged across a Gaussian kernel (12-mm full width half maximum) centered on the hottest voxel (Friston and others 2006) (see Fig. 3 for data averaged across the whole ROI). Time course plots show the average BOLD response across all subjects aligned to the start of each event from the local maxima of the localized ROIs. The error bars show the random effects standard errors across subjects. The pattern of activity for FFA and LOC is qualitatively similar; both display sustained responses to the visible and occluded objects (for which they are selective) that is conditional on the object remaining present throughout the trial. For vanishing trials, the FFA and LOC show a more transient response, whereas for segment trials there is little response.

Activity Differences between Occluded and Visible Objects

To investigate regions responding more to occluded than visible objects, we located areas where the difference between main conditions and vanishing control conditions is greater for OC than VS conditions. For face stimuli, we performed the interaction: [OC > OCv] > [VS > VSv] (see Fig. 4). The converse of this interaction reveals regions more active for visible than occluded objects. The left ventral premotor region shows distinct specificity for occluded faces and houses, whereas the superior frontal gyrus shows specificity only for occluded faces. The same interaction logic was applied to the category of houses. The premotor cortex showed specificity for both OC face and OC house stimuli, whereas the left prefrontal was active for OC house but not OC face conditions. At a weaker threshold (P < 0.003 uncorrected), the right hippocampus (−28 −16 −10) was also active (not shown).

The finding that the response of early visual areas is greater for VS than OC in Figure 4 is expected because the VS conditions contain higher spatial detail (e.g., of the facial features) than OC conditions. What was not expected was that the statistical parametric maps for this contrast show significantly different locations of peak activity for faces and houses. It should be noted that the difference between the 2 is not significant because the contrast VS face versus VS house (the face localizer) and its converse yields no such activity in area 17/18 at P < 0.001 uncorrected.

A PPI analysis was employed to reveal changes in the functional interaction between brain regions involved in the processing of stimuli under conditions of visibility and occlusion. We performed 2 PPI analyses. One for faces using the FFA (defined by localizer) as the ROI, and one for houses using the lateral occipital cortex (LOC) (defined by localizer) as the ROI. The psychological parameter used was OC versus VS. As shown in Figure 5, the face-specific PPI demonstrated a region in the left premotor cortex to be the only one to increase its functional integration with the FFA (left and right) when the face was occluded. Similarly, the house-specific PPI also shows that the same region of left premotor cortex to be the only region to increase its functional integration with the LOC (left and right). The PPI analysis for VS > OC revealed no suprathreshold activity at P < 0.001 uncorrected.

Discussion

Our interpretation of the data is that activity within the FFA and LOC is invariant to whether objects are occluded or not (faces and houses, respectively). The results (Fig. 2) show that this is true for a Gaussian ROI centered on the local maxima for the FFA and LOC (as identified by the localizer). However, the average response across the whole ROI is not as clear-cut (see Fig. 3). Whereas the time course of responses is most similar for the right FFA and to a lesser extent right LOC, the response of the left FFA and left LOC shows less clear effects. We would conclude that the whole of the right FFA and a subpopulation within the left FFA show the occlusion invariant responses to faces, whereas the whole of the right LOC and a subpopulation within the left LOC show occlusion invariant responses to houses. To uphold this claim, it is first necessary to consider whether this activity can be explained in terms of other factors.

Figure 3.

Time courses shown averaged across the whole ROI as defined by face and house-specific localizers. The time courses of the whole ROI show a similar but noisier response compared with the hottest voxel analysis, particularly for right FFA and right LOC; however, the results are less clear-cut for left FFA or left LOC.

Figure 3.

Time courses shown averaged across the whole ROI as defined by face and house-specific localizers. The time courses of the whole ROI show a similar but noisier response compared with the hottest voxel analysis, particularly for right FFA and right LOC; however, the results are less clear-cut for left FFA or left LOC.

In order to achieve a sense of real occlusion, subjects needed to see the objects prior to occlusion. Thus, all stimuli presented were compound stimuli consisting of an initial viewing of the object in the first 1.5 s followed by a static period of 7.5 s in which the object persists either in front of (VS) or behind the occluder (OC). Thus, it could be argued that the sustained response could be due to the initial viewing of the object alone and have nothing to do with the fact it is occluded. This is controlled for by the vanishing condition OCv, which is identical to the OC condition for the first 1.5 s after which the object is seen to disappear. The contrast OC–OCv shows that there is a significant difference in FFA/LOC activity when controlling for initial viewing. Thus, the sustained response cannot be explained just in terms of the initial viewing of the object prior to occlusion.

The difference between the OC and OCv conditions is the visibility of the small segment of unoccluded object (approximately 2% of the object's surface area). Thus, it is possible this difference caused the sustained activity observed in the FFA/LOC. However, this is controlled for by the segment conditions (SG). The fact that [SG–SGv] showed no difference in FFA/LOC activity refutes the hypothesis that it is the visibility of the unoccluded segment that is causing the sustained activity in question.

We therefore conclude that the maintained response is a late response that cannot be explained by either the initial viewing or the segment alone. Rather, it is only the coincidence of the initial viewing and the persistence of the segment, which induces the “awareness of presence,” and only when this occurs do we observe the sustained activity in the FFA/LOC. We therefore argue that the sustained activity in FFA/LOC is a correlate of the awareness of presence and that this cannot be explained by just the initial viewing or just the visual properties of the unoccluded portion of the head.

This result is partly surprising and partly not. To a certain degree, one would expect the same set of cortical areas to be involved in processing these stimuli, regardless of whether they are directly perceived or occluded. The surprise lies rather in the fact that the areas were activated with the same magnitude and had very similar time courses, whether perceived or not. The results imply that the activity that is specific to an object is maintained in these areas even after the stimulus has disappeared from view. That there is a difference in cortical response to directly perceived and occluded visual stimuli is shown by our other result (Fig. 4), namely, that occluded stimuli selectively recruit different parts of prefrontal cortex according to the type of stimulus (face or house) and that premotor cortex is commonly recruited by the occlusion of either stimulus type (Fig. 5). Finally, we show that the premotor cortex is the only region to significantly increase its covariance with the object-selective regions (FFA and LOC) under conditions of occlusion (Fig. 5). We therefore speculate that the premotor cortex may exert a controlling influence over the retention of a previously seen visual stimulus that is subsequently occluded. Before evaluating our results in terms of perception and awareness, it is necessary to consider the experiment in light of other cognitive processes that might influence our interpretation.

Figure 4.

Upper brains show activity differences between occluded faces and visible faces. Red: Areas where the difference between main conditions and vanishing controls is greater for OC than VS conditions. Blue: Areas where the difference between main conditions and vanishing controls is greater for VS face than OC face conditions. Bar plots show difference in parameter estimates between main and vanishing controls for all conditions: Lower brains show differences between occluded houses and visible houses. Red: Areas where the difference between main conditions and vanishing controls is greater for OC house than VS house conditions. Blue: Areas where the difference between main conditions and vanishing control conditions is greater for OC house than VS house. All contrasts shown significant at P < 0.001 uncorrected.

Figure 4.

Upper brains show activity differences between occluded faces and visible faces. Red: Areas where the difference between main conditions and vanishing controls is greater for OC than VS conditions. Blue: Areas where the difference between main conditions and vanishing controls is greater for VS face than OC face conditions. Bar plots show difference in parameter estimates between main and vanishing controls for all conditions: Lower brains show differences between occluded houses and visible houses. Red: Areas where the difference between main conditions and vanishing controls is greater for OC house than VS house conditions. Blue: Areas where the difference between main conditions and vanishing control conditions is greater for OC house than VS house. All contrasts shown significant at P < 0.001 uncorrected.

Figure 5.

Upper diagram shows brain activity common to occluded faces and houses. Voxels (green) that show significant (P < 0.001 uncorrected) effects for both OC face and OC house expressed in the conjunction [OC face > OCv face] AND [OC house > OCv house]. Common activity was restricted to premotor cortex. This shows that premotor cortex response is significantly more active for occluded faces and houses than visible faces and houses. Second and third row of figure shows PPIs. Left images show the SPM computed for the PPI term. Right graphs show the extracted BOLD activity for the ROI plotted against premotor cortex activity under OC (red) and VS (blue) conditions.

Figure 5.

Upper diagram shows brain activity common to occluded faces and houses. Voxels (green) that show significant (P < 0.001 uncorrected) effects for both OC face and OC house expressed in the conjunction [OC face > OCv face] AND [OC house > OCv house]. Common activity was restricted to premotor cortex. This shows that premotor cortex response is significantly more active for occluded faces and houses than visible faces and houses. Second and third row of figure shows PPIs. Left images show the SPM computed for the PPI term. Right graphs show the extracted BOLD activity for the ROI plotted against premotor cortex activity under OC (red) and VS (blue) conditions.

Although this experiment did not explicitly require the memorization or imagery of objects, it is at least plausible that viewing a stimulus that becomes occluded automatically recruits mechanisms in common with working memory or visual imagery. Object working memory tasks, such as the delayed match to sample task, have been shown to recruit prefrontal regions (different to those in this study) and object-selective regions of inferotemporal cortex in humans and in nonhuman primates (Fuster and Jervey 1982; Chelazzi and others 1993; Nakamura and Kubota 1995; Desimone 1996; O'Scalaidhe and others 1997; Druzgal and D'Esposito 2001, 2003). Similarly, visual imagery tasks have also been shown to activate frontoparietal and object-selective regions (O'Craven and Kanwisher 2000; Ishai and others 2002).

Despite the similarities between the activities reported here and in previous working memory and imagery studies, there are important differences. During the delay period of working memory tasks, in which the face is memorized over a period of time during which it is not seen, FFA activity drops significantly (Druzgal and D'Esposito 2001, 2003). The working memory of a face therefore results in activity that is significantly lower in magnitude than the response to the perception of a face. In the same way, visual imagery of faces has been shown to activate the FFA above baseline but to a level significantly less than during perceptual conditions (O'Craven and Kanwisher 2000). In this study, we show that the time course of the FFA and LOC during the occluded (OC) conditions matches that of the visible (VS) conditions and therefore is significantly different to that reported previously under conditions of working memory and imagery.

The difference in response between occluded stimuli and those that are either imaged or held in working memory mirrors the phenomenological differences between the tasks; looking at a face that becomes occluded is very different from holding a face in working memory or imagining it in your mind's eye. The occluded face is known to be actually present, whereas the imaged face, or the working memory of a face, is known to be not actually present. One would expect such an important distinction to be reflected robustly in the response of a visual area (Pollen 1999).

One possibility is that activity common to VS and OC conditions is due to the appearance or recognition of the very top portion (2% of surface area) of the object that remained visible during OC conditions. This is controlled for with the segment (SG) conditions. The time course data for condition SG (Fig. 2) show that the FFA or the LOC do not respond to the appearance of the segment. This effectively rules out any explanation of the occluded stimulus activity in terms of spatial context or low-level visual properties.

The attentional differences between conditions should be minimal by virtue of the task, which requires fixation and detection of a change at the fixation cross. The attentional resources allocated for each condition should be equal because the task is the same for all conditions.

It is of interest that we did not observe significant parahippocampal place area (PPA) activity for the visual presentation of houses that had been previously observed (Tong and others 1998). Note that the PPA responds not to houses per se but to the spatially extended geometry of the local environment (Epstein and others 1999). Because all our stimuli were presented in the context of a 3-dimensional environment (Supplementary Material, Fig. 1), it is likely that this is the reason we detected no systematic difference in PPA activity between house conditions and face conditions. If this is the case, then the most obvious remaining difference between faces and houses is in their visual form, a hypothesis tentatively supported by the fact that houses selectively activated LOC in this experiment. Such house-selective responses have been demonstrated previously in this region (posterior fusiform) when activation caused by houses was contrasted with that caused by faces and objects (Ishai and others 2000).

The Specificity of Premotor Cortex for Occluded Stimuli

The most significant cluster of voxels for occluded stimuli was located in ventral premotor cortex. Given that premotor cortex has classically been implicated in the planning and control of movements (Rizzolatti and others 1981, 1988; Kurata and Tanji 1986), its response to occluded stimuli may seem counterintuitive. An explanation in motoric terms must therefore be considered before alternative hypotheses are entertained.

Premotor activity could plausibly indicate systematic differences in motor planning or motor actions, but we consider this to be unlikely due to the design of the paradigm. Subjects were only instructed to press a button in the 15% of all trials in which the fixation cross change appeared at a randomly selected time. The onsets of the changes were randomly distributed and unpredictable, and there could therefore be no systematic difference in motor preparation that could account for the specificity of the premotor response. Furthermore, the trials in which changes occurred (and therefore where motor responses were required) were independently modeled as effects of no interest.

The premotor activity observed in this study is perhaps better explained in terms of visual processing. Ventral premotor neurons show selectivity for stimuli independently of any motor response (Murata and others 1997) and of actions directed toward objects (Umiltà and others 2001). Additionally, the premotor cortex response to an observed object-directed action persists even when the object is occluded (Umiltà and others 2001). Such a response is contingent on the monkey knowing that an action is directed toward an occluded object and cannot be induced if the monkey knows there is no occluded object. This should stand as positive evidence that the response of the premotor cortex can be contingent on the existence of object permanence. Finally, in monkeys, the premotor response to a visible object can persist in time even when the lights are extinguished and it is rendered invisible (Graziano and others 1997a, 1997b). This has been interpreted as evidence for the role of premotor cortex in object permanence (Graziano and others 1997a) and is therefore directly relevant to the occlusion-specific premotor response we observed. In both the monkey and the human, ventral premotor cortex responds when objects are known to be present but not seen.

Our result, in light of convergent evidence in the monkey and human, prompts us to support a functional role for premotor cortex in object permanence. We speculate that for the brain to sustain knowledge of the presence of objects when deprived of local sensory information, premotor cortex forms a functional coupling with object-selective regions. We therefore predict that disruption to the premotor cortex or its connection to object-selective regions would disrupt the ability of object-selective regions to maintain heightened activity during occlusion and thus would disrupt object permanence.

FFA Does Not Correlate with Perception

Although interesting in its own right, the principal objective of this experiment was not to study occlusion per se but to study occlusion because of the perceptual dissociation it provides.

To investigate the relationship between neural activity and perceptual awareness, it is necessary to go beyond correlations and examine what neural activity is necessary and/or sufficient for perceptual awareness. One approach that follows from this strategy is to evaluate the relationship between neural activity in the FFA and the corresponding state of perceptual experience of a face by considering each of the 4 possible combinations of face perception and FFA activity (Kanwisher 2001). For this purpose alone, we will consider face perception as being either present or absent and FFA activity as being either high or low with high activity being that observed in the FFA during direct face perception and low activity being any level significantly lower than this.

The evidence that 1) high FFA activity correlates with face perception and 2) low FFA activity correlates with unconscious face processing (Moutoussis and Zeki 2002) is the basis for the correlation between the level of FFA activity and face perception. There are 2 remaining possibilities that could elucidate the relationship further. If 3) low FFA activity correlates with face perception, then the necessity of high FFA for face perception could be ruled out. There is, however, no evidence to date to support this. The results of this experiment are interesting because they satisfy the final possibility of 4) high FFA activity in the absence of face perception. High FFA activity alone, as our evidence suggests (for the whole of right FFA and a subpopulation of left FFA), cannot be sufficient for face perception. This is an important point because it motivates a re-evaluation of the contribution of the FFA to visual awareness.

If this result holds true, a theory that accounts for the FFA's contribution to visual awareness must extend beyond just the perception of faces. Whenever a face is visible, the perception of its attributes leads, under normal conditions, to the awareness that a face is present. When a face is occluded, only awareness of its presence remains and, therefore, we propose that a more parsimonious explanation is that the common FFA activity recorded in both cases correlates not with face perception but with awareness of the presence of a face. Our findings appear to confirm the theoretical speculations of Pollen who states that neural representations of an object “can achieve an independent but nonphenomenal existence apart from the sensory data from which it was derived” (Pollen 1999). All of the above arguments are equally valid for LOC and its response to houses.

In summary, our results show that the majority of the FFA responds equally to visible and occluded faces and that the majority of the LOC responds equally to visible and occluded houses. The commonality between an occluded and a visible object is the awareness of the object's presence, and therefore, we propose that high activity (as previously defined) in the FFA and LOC correlates best with the awareness of the presence of faces and houses (respectively). We therefore speculate that the activity of these areas, at the level previously observed under perceptual conditions, is due to the awareness of the object's presence. Finally, the demonstration that the premotor cortex is selectively activated and its covariation with object-selective regions increases when faces and houses are occluded suggests it is involved with processing of occluded objects.

Supplementary Material

Supplementary material can be found at: http://www.cercor.oxfordjournals.org/.

This experiment was realized using Cogent 2000 developed by the Cogent 2000 team at the Functional Imaging Laboratory and the Institute of Cognitive Neuroscience and Cogent Graphics developed by John Romaya at the Laboratory of Neurobiology at the Wellcome Department of Imaging Neuroscience. This work was supported by the Medical Research Council and the Wellcome Trust. Thanks to Barrie Roulston, Asher Mullard, Louise Whiteley, and in particular Matt Self and Karl Friston for their helpful inputs to this manuscript. Conflict of Interest: None declared.

References

Baillargeon
R
Representing the existence and the location of hidden objects: object permanence in 6- and 8-month-old infants
Cognition
 , 
1986
, vol. 
23
 (pg. 
21
-
41
)
Baillargeon
R
Spelke
ES
Wasserman
S
Object permanence in five-month-old infants
Cognition
 , 
1985
, vol. 
20
 (pg. 
191
-
208
)
Behrmann
M
Zemel
RS
Mozer
MC
Object-based attention and occlusion: evidence from normal participants and a computational model
J Exp Psychol Hum Percept Perform
 , 
1998
, vol. 
24
 (pg. 
1011
-
1036
)
Chelazzi
L
Miller
EK
Duncan
J
Desimone
R
A neural basis for visual search in inferior temporal cortex
Nature
 , 
1993
, vol. 
363
 (pg. 
345
-
347
)
Churchland
MM
Chou
I-H
Lisberger
SG
Evidence for object permanence in the smooth-pursuit eye movements of monkeys
J Neurophysiol
 , 
2003
, vol. 
90
 (pg. 
2205
-
2218
)
Dehaene
S
Naccache
L
Cohen
L
Bihan
DL
Mangin
JF
Poline
JB
Riviere
D
Cerebral mechanisms of word masking and unconscious repetition priming
Nat Neurosci
 , 
2001
, vol. 
4
 (pg. 
752
-
758
)
Desimone
R
Neural mechanisms for visual memory and their role in attention
Proc Natl Acad Sci USA
 , 
1996
, vol. 
93
 (pg. 
13494
-
13499
)
Druzgal
TJ
D'Esposito
M
Activity in fusiform face area modulated as a function of working memory load
Brain Res Cogn Brain Res
 , 
2001
, vol. 
10
 (pg. 
355
-
364
)
Druzgal
TJ
D'Esposito
M
Dissecting contributions of prefrontal cortex and fusiform face area to face working memory
J Cogn Neurosci
 , 
2003
, vol. 
15
 (pg. 
771
-
784
)
Epstein
R
Harris
A
Stanley
D
Kanwisher
N
The parahippocampal place area: recognition, navigation, or encoding?
Neuron
 , 
1999
, vol. 
23
 (pg. 
115
-
125
)
Friston
KJ
Buechel
C
Fink
GR
Morris
J
Rolls
E
Dolan
RJ
Psychophysiological and modulatory interactions in neuroimaging
Neuroimage
 , 
1997
, vol. 
6
 (pg. 
218
-
229
)
Friston
KJ
Holmes
AP
Worsley
KJ
How many subjects constitute a study?
Neuroimage
 , 
1999
, vol. 
10
 (pg. 
1
-
5
)
Friston
KJ
Penny
WD
Glaser
DE
Conjunction revisited
Neuroimage
 , 
2005
, vol. 
25
 (pg. 
661
-
667
)
Friston
KJ
Rotshtein
P
Geng
JJ
Sterzer
P
Henson
RN
A critique of functional localisers
Neuroimage
 , 
2006
, vol. 
30
 (pg. 
1077
-
1087
)
Fuster
JM
Jervey
JP
Neuronal firing in the inferotemporal cortex of the monkey in a visual memory task
J Neurosci
 , 
1982
, vol. 
2
 (pg. 
361
-
375
)
Graziano
MS
Hu
XT
Gross
CG
Coding the locations of objects in the dark
Science
 , 
1997
, vol. 
277
 (pg. 
239
-
241
)
Graziano
MS
Hu
XT
Gross
CG
Visuospatial properties of ventral premotor cortex
J Neurophysiol
 , 
1997
, vol. 
77
 (pg. 
2268
-
2292
)
Haimson
C
Behrmann
M
Cued visual attention does not distinguish between occluded and occluding objects
Psych bull rev
 , 
2001
, vol. 
8
 (pg. 
496
-
503
)
Ishai
A
Haxby
JV
Ungerleider
LG
Visual imagery of famous faces: effects of memory and attention revealed by fMRI
Neuroimage
 , 
2002
, vol. 
17
 (pg. 
1729
-
1741
)
Ishai
A
Ungerleider
LG
Martin
A
Haxby
JV
The representation of objects in the human occipital and temporal cortex
J Cogn Neurosci
 , 
2000
, vol. 
12
 
Suppl 2
(pg. 
35
-
51
)
Johnson
SP
Development of perceptual completion in infancy
Psychol Sci
 , 
2004
, vol. 
15
 (pg. 
769
-
775
)
Joseph
JS
Nakayama
K
Amodal representation depends on the object seen before partial occlusion
Vision Res
 , 
1999
, vol. 
39
 (pg. 
283
-
292
)
Kanwisher
N
Neural events and perceptual awareness
Cognition
 , 
2001
, vol. 
79
 (pg. 
89
-
113
)
Kanwisher
N
Chalupa
L
Werner
J
The ventral visual object pathway in humans: evidence from fMRI
The visual neurosciences
 , 
2003
Boston
MIT Press
(pg. 
1179
-
1189
)
Kellman
PJ
Yin
C
Shipley
TF
A common mechanism for illusory and occluded object completion
J Exp Psychol Hum Percept Perform
 , 
1998
, vol. 
24
 (pg. 
859
-
869
)
Kurata
K
Tanji
J
Premotor cortex neurons in macaques: activity before distal and proximal forelimb movements
J Neurosci
 , 
1986
, vol. 
6
 (pg. 
403
-
411
)
Mareschal
D
Object knowledge in infancy: current controversies and approaches
Trends Cogn Sci
 , 
2000
, vol. 
4
 (pg. 
408
-
416
)
Michotte
A
On phenomenal permanence: facts and theories
Acta Psychol
 , 
1950
, vol. 
7
 (pg. 
293
-
322
)
Moutoussis
K
Zeki
S
The relationship between cortical activation and perception investigated with invisible stimuli
Proc Natl Acad Sci USA
 , 
2002
, vol. 
99
 (pg. 
9527
-
9532
)
Murata
A
Fadiga
L
Fogassi
L
Gallese
V
Raos
V
Rizzolatti
G
Object representation in the ventral premotor cortex (area F5) of the monkey
J Neurophysiol
 , 
1997
, vol. 
78
 (pg. 
2226
-
2230
)
Nakamura
K
Kubota
K
Mnemonic firing of neurons in the monkey temporal pole during a visual recognition memory task
J Neurophysiol
 , 
1995
, vol. 
74
 (pg. 
162
-
178
)
Nichols
T
Brett
M
Andersson
J
Wager
T
Poline
J-B
Valid conjunction inference with the minimum statistic
Neuroimage
 , 
2005
, vol. 
25
 (pg. 
653
-
660
)
O'Craven
KM
Kanwisher
N
Mental imagery of faces and places activates corresponding stimulus-specific brain regions
J Cogn Neurosci
 , 
2000
, vol. 
12
 (pg. 
1013
-
1023
)
O'Scalaidhe
SP
Wilson
FA
Goldman-Rakic
PS
Areal segregation of face-processing neurons in prefrontal cortex
Science
 , 
1997
, vol. 
278
 (pg. 
1135
-
1138
)
Palmer
SE
An introduction to vision science
Vision science: photons to phenomenology
 , 
1999
Cambridge, MA
MIT Press
Pollen
DA
On the neural correlates of visual perception
Cereb Cortex
 , 
1999
, vol. 
9
 (pg. 
4
-
19
)
Rauschenberger
R
Yantis
S
Masking unveils pre-amodal completion representation in visual search
Nature
 , 
2001
, vol. 
410
 (pg. 
369
-
372
)
Rees
G
Wojciulik
E
Clarke
K
Husain
M
Frith
C
Driver
J
Unconscious activation of visual cortex in the damaged right hemisphere of a parietal patient with extinction
Brain
 , 
2000
, vol. 
123
 (pg. 
1624
-
1633
)
Ress
D
Heeger
DJ
Neuronal correlates of perception in early visual cortex
Nat Neurosci
 , 
2003
, vol. 
6
 (pg. 
414
-
420
)
Rizzolatti
G
Camarda
R
Fogassi
L
Gentilucci
M
Luppino
G
Matelli
M
Functional organization of inferior area 6 in the macaque monkey. II. Area F5 and the control of distal movements
Exp Brain Res
 , 
1988
, vol. 
71
 (pg. 
491
-
507
)
Rizzolatti
G
Scandolara
C
Matelli
M
Gentilucci
M
Afferent properties of periarcuate neurons in macaque monkeys. II. Visual responses
Behav Brain Res
 , 
1981
, vol. 
2
 (pg. 
147
-
163
)
Scholl
BJ
Pylyshyn
ZW
Tracking multiple items through occlusion: clues to visual objecthood
Cogn Psychol
 , 
1999
, vol. 
38
 (pg. 
259
-
290
)
Sekuler
A
Palmer
S
Flynn
C
Local and global processes in visual completion
Psychol Sci
 , 
1994
, vol. 
5
 (pg. 
260
-
267
)
Talairach
J
Tournoux
P
Co-planar stereotaxic atlas of the human brain
 , 
1988
New York
Thieme Medical
Tong
F
Nakayama
K
Vaughan
JT
Kanwisher
N
Binocular rivalry and visual awareness in human extrastriate cortex
Neuron
 , 
1998
, vol. 
21
 (pg. 
753
-
759
)
Umiltà
MA
Kohler
E
Gallese
V
Fogassi
L
Fadiga
L
Keysers
C
Rizzolatti
G
I know what you are doing. A neurophysiological study
Neuron
 , 
2001
, vol. 
31
 (pg. 
155
-
165
)
Yi
D-J
Kim
MS
Chun
MM
Inhibition of return to occluded objects
Percept Psychophys
 , 
2003
, vol. 
65
 (pg. 
1222
-
1230
)
Zeki
S
Functional specialisation in the visual cortex of the rhesus monkey
Nature
 , 
1978
, vol. 
274
 
5670
(pg. 
423
-
428
)
Zeki
S
A vision of the brain
 , 
1993
Oxford
Blackwell Scientific Publications
Zeki
S
ffytche
DH
The Riddoch syndrome: insights into the neurobiology of conscious vision
Brain
 , 
1998
, vol. 
121
 (pg. 
25
-
45
)