Symmetry is an important cue in face perception. We manipulated symmetry and other configurational variables to study their role in face processing in the human brain. We employed 2 types of symmetry: image symmetry (where one part of the image is defined as the mirrored transform of the other part about an axis) and object symmetry (where the spatial relationships among the image components are interpreted as parts of a symmetric 3-dimensional object). We compared blood oxygenation level dependent responses in healthy human observers for upright front-view faces with responses to different symmetry-controlled images. The cortical areas activated by the face images, relative to Fourier-matched scrambled images, were the fusiform (FFA) and occipital (OFA) face areas, the middle occipital gyri (MOG), and areas around the superior temporal and intraoccipital sulci (IOS). Contrasting faces and their image-symmetric scrambled versions showed a similar activation pattern except in the right OFA, suggesting an involvement in facial symmetry processing. The upright versus inverted faces (with the same image symmetry but unfamiliar object identity) showed robust differential activation in the FFA, OFA, MOG, IOS, and precuneus. The response to frontal-view versus 3/4-view faces (having the same object symmetry but disrupted image symmetry) showed little differential activation in the FFA or the OFA but strong responses in the MOG and IOS, suggesting that face processing in the FFA and the OFA is holistic and viewpoint invariant.
Face recognition and discrimination may be one of the most developed perceptual skills of visual object processing. An adult can discriminate and recognize hundreds of faces (Bharick and others 1975). Much evidence has shown that face perception cannot be achieved by analyzing facial features alone and that facial configuration, or the spatial relationships among facial features, is crucial to face perception (Fantz 1961; Valentine 1988; O'Toole and others 1994; Leder and Bruce 2000; Webster and others 2004).
There are, however, many possible spatial relations among facial features. It is not clear what types of spatial relations are essential for a human observer to analyze the facial configuration. Much attention on facial configuration has focused on the relative distance between features (e. g., Fellous 1997), such as the distance from the center of the face to several landmarks on a face (e.g., Anderson and Wilson 2005), from the eyes to the eyebrows (e.g., Brown and Perrett 1993), from the eyes to the nose and the mouth (Hosie and others 1988), and so on. On the other hand, there are reports that configuration cues such as mirror symmetry are important in face perception. There is a well-established relationship between facial symmetry and perceived attractiveness (or beauty) of potential mates in humans (Grammer and Thornhill 1994; Mealey and others 1999; Rhodes and others 2001, 2002; Jacobsen and Hofel 2002). There is also a preference for facial asymmetry that is related to emotion expressiveness (Swaddle and Cuthill 1995; Zaidel and others 1995; Kowner 1996). Recently, Rhodes and others (2005) showed that humans can detect symmetry better in upright faces than in either inverted faces or contrast reverted faces and argued that there should be a mechanism in the human visual system specific for the symmetry of faces.
Brain areas supporting face processing have been reported both in monkey (Perrett and others 1982) and in humans (Kanwisher and others 1997; McCarthy and others 1997; Halgren and others 1999; Haxby and others 2000). In particular, part of the fusiform gyrus and the inferior occipital gyrus have been reported as specific for face perception and are designated the fusiform face area (FFA, Kanwisher and others 1997) and occipital face area (OFA, Halgren and others 1999; Rossion and others 2003), respectively (although there is disagreement on the exact role of the FFA, see Gauthier and others 1999; Haxby and others 2001). It is, however, not clear whether these brain areas are involved in the analysis of local facial features, face-specific configurations, or other gestalt properties of faces. In behavioral studies, the configuration-specific system is generally isolated by contrasting performance for upright faces and inverted faces (Yin 1969; Carey and Diamond 1977; Thompson 1980). A manipulation that causes performance degradation in inverted relative to upright faces is considered to reflect processing by mechanisms specific to the absolute facial configuration because the relative configuration of the features is undisturbed by rotation.
There have been several neuroimaging studies contrasting brain activation to upright and inverted faces, but significant differential activations for upright face vs. inverted face have not been reported in either FFA or OFA (Kanwisher and others 1998; Aguirre and others 1999; Haxby and others 1999) until recently (Yovel and Kanwisher 2004). Because faces are much less recognizable when upside down, the implication of the predominantly negative result is that these areas are responding to the basic aspects of facial stimuli that are equally present when the face is inverted, such as the local features and the configuration aspects such as symmetry, rather than the higher-level perceptual processes of facial recognition, which are substantially degraded by inversion.
Our purpose here is to identify brain areas specific for the configuration properties of facial symmetry. Given recent reports on brain areas responding to image symmetry per se (Sasaki and others 2005; Tyler and others 2005a) and the behavioral evidence for facial symmetry mechanisms (Rhodes and others 2005), we are interested in the role of symmetry in brain areas responding to faces. We may conceptualize 2 types of face symmetry. The first type, “image symmetry,” occurs when one image component is a mirrored transform of another image component about some axis of transformation. This type of symmetry specifies the direct spatial relations between the parts of the 2-dimensional (2D) projections of the faces and is not concerned with the interpretation of the image as an object in space. The second type of symmetry is “object symmetry,” which specifies the spatial relationships among the components of the image interpreted as the representation of a 3D object. Hence, those components or facial features will be processed through a face gestalt or 3D template of the human head. This kind of object symmetry may not be consistent with image symmetry due to an asymmetric viewpoint of the observer or the rotation of the object's axis of symmetry away from the line of sight. Thus, a 3/4 view of a symmetric face forms an asymmetric image, even though it is the image of a symmetric object and can be understood to have accurate object symmetry. Our experiments were designed to test the role of these 2 types of symmetry in the facial representations of the FFA and the OFA. In detail, our goals were as follows:
To reveal all relevant areas supporting face processing (except those for early visual processing), we first contrasted brain activations between faces and asymmetric scrambled versions of the same images with identical Fourier energy. These images should generate the same net activation for arrays of neurons acting as local linear filters.
To identify the cortical areas responding to symmetry per se, we compared the activation for the asymmetric random images and symmetricized versions of the same images (as in Tyler and others 2005a).
To identify cortical regions involved in the processes of face recognition, we then compared brain activation of the front-view faces and their inverted versions. Inverted faces are known to disrupt the facial configuration mechanisms and degrade the function of the face recognition mechanisms. Thus, because inverted faces maintain the same local features, the same image symmetry, and the same object symmetry as the upright faces, the null hypothesis is that they should produce no activation to regions containing either local mechanisms of symmetry processing for facial recognition. Hence, any significant differences in activation for the upright versus inverted faces should reveal the operation of holistic facial configuration mechanisms.
To examine whether the differential activation patterns from (3) are due to face-specific symmetry or to early image symmetry processing, we also contrasted front-view upright faces with their phase-scrambled versions that were vertically symmetric. A contrast between faces and vertically symmetric scrambled images should reveal all relevant areas supporting face processing that are insensitive to symmetry. A comparison of the last 2 experiments should thus uncover whether areas that are attributed to face processing are actually responding to the symmetry of the faces rather than more informative cues.
Finally, to study the role of object versus image symmetry (under the assumption that faces are symmetric objects), we compared the responses to upright frontal-view faces and 3/4-view faces of the same persons. The 3/4-view faces retain the same semantic and object-centered properties as the frontal-view faces but have radically altered image symmetry. Cortical areas encoding image symmetry should be differentially activated by this comparison, but those encoding object symmetry or semantic content should show little or no differential activation with the change in viewpoint of the same faces.
Figure 1 shows examples of the stimuli used in our experiment. We had 5 types of images: upright frontal-view faces, inverted faces, 3/4-view faces, symmetric scrambled images, and asymmetric scrambled images. The face images were taken from The Facial Recognition Technology (FERET) Database (Phillips and others 2000) provided by the National Institute of Standards and Technology, USA. We computed the symmetry index (Tyler and others, 2003) of the frontal-view faces of the 1010 subjects in the FERET database and selected the 36 face images with greatest symmetry index values. The symmetry index is computed based on the power spectrum of the Fourier transform of the face images. Here we were only interested in the horizontal symmetry. Hence, we computed the difference of the power at the points (kx, ky) and (−kx, ky), where kx and ky are horizontal and vertical spatial frequencies of the images (in the upper half-plane, excluding the horizontal axis). The symmetry index is computed as a function of the root mean square difference of the power between corresponding frequencies summed over the spectrum.
The inverted faces were simply the upside-down versions of the same 36 images. The 3/4-view images were pictures of the same persons available from the database with their head rotated 67.5° either to their left (17 images) or to their right (19 images). The 2 types of scrambled images were created by manipulating the phase spectrum of the frontal-view face images. We first normalized the face images by subtracting their mean luminance values to remove the DC component and then computed their Fourier transforms. The phase spectra were scrambled randomly for the asymmetric scrambled images and set to zero phase for the symmetric scrambled images. We then generated the control images by taking inverse Fourier transforms of the original amplitude spectra combined with the new phase spectra of the 2 types. Note that this novel zero-phase manipulation introduces noticeable edge structures throughout the images similar to that of the intact faces, so it tends to equate for nonlinear edge processing in the face images in addition to the presence of bilateral symmetry. The luminance and contrast of the inverse Fourier-transformed images were normalized to match the mean luminance and contrast energy of the face images. The phase zeroed scrambled images had point symmetry, so they were converted to bilateral symmetry by inverting their left half relative to the right half.
A fourth-power Gaussian aperture (1/e scale parameter = 3.2°) was applied to all images to remove image features outside face areas (cf., Tyler and Chen forthcoming). A fixation point of 6.75′ × 6.75′ size was inserted at the center of the face image.
We used the blood oxygenation level dependent (BOLD) signal from a block-design functional Magnetic Resonance Imaging (fMRI) paradigm to measure the cortical response to the stimulus contrasts. Each of the 4 experimental runs had 18-s epochs of upright frontal-view faces alternating with 18-s epochs of one of the other image types in 6 cycles (36-s period) of the 2-epoch block alternation. Each 18-s epoch consisted of 18 presentations of the images for 850 ms followed by a 150-ms blank period. To keep the observers' attention on the presented images regardless of block content, the observers were instructed to press a response key when the currently presented image physically matched the previous one (one-back task).
Stimuli were delivered with MRVision 2000 goggles from Resonance Technology, Inc. (Northridge, CA). At 800 (H) by 600 (V) resolution, the pixel size of the display was specified as 2.25′.
Six observers participated in this study, including 2 of the authors and 4 paid observers naïve to the purpose of the study. All observers had normal or corrected-to-normal (by the focusing lenses included with the goggles) visual acuity. All observers gave informed consent for their participation.
Data Acquisition and Analysis
The images were collected with a Bruker 3T scanner located at National Taiwan University. Within each scanning session, both functional (T2* weighted, BOLD) responses and anatomical (T1 weighted) images were acquired in identical planes. The images were collected in 20 transverse planes parallel to the anterior commissure–posterior commissure line. An echo-planar imaging (EPI) sequence (Stehling and others 1991) was used to acquire the functional data (time repetition = 3000 ms, time echo = 40 ms, flip angle = 90°, voxel resolution = 2.34 × 2.34 × 3 mm). A high-resolution anatomical (T1 weighted) MRI volume scan of the entire head was run once on each observer (voxel size = 1 × 1 × 1 mm). The main experiment lasted 225 s (75 images). The first 9 s (3 images) were excluded from further analyses to avoid the start-up transient. Thus, the data analyzed for each scan spanned 216 s (72 images). To correct head-motion artifacts, we used SPM (Friston and others 1995) to realign the EPI images acquired. The realigned images, as well as the anatomic images, were then normalized to a standard template with SPM. The normalized images were fed to the mrVista software (Wandell and others 2000) for coregistration, data analysis, and 3D visualization. Statistical analysis of the BOLD activation was based on the spectral correlation between the BOLD activation time series and the experimental sequence (Engel and others 1997). For each run, the Fourier transform was calculated for each BOLD time series. The spectral correlation, or coherence, was then computed as the power at the frequency of block alternation divided by the square root of the energy of the spectrum for each voxel. The signal amplitude is given by the square root of the signal power.
There were situations where we needed to test differential contrasts across 2 or more block design runs. For this purpose, we followed the procedure proposed by Joseph and others (2002). That is, we first identified a region of interest (ROI) defined by a specific parametric contrast from coherence maps for individual runs. The signal amplitude of the averaged waveform over the voxels in this ROI from each observer was then fed to a repeated measure analysis of variance (ANOVA) that assessed the effect of conditions for the specified contrast. In addition, to get detailed information of the differential contrasts in specific ROIs defined by the localizers, we followed up significant main effects in the ANOVA with pairwise comparison t-tests of differences in activation between conditions in the specified ROIs.
In addition to the main experiments, all observers participated in localizer experiments in a separate session, in order to identify relevant brain areas reported in the literature. The lateral occipital complex (LOC) was identified by the differential BOLD activation to pictures of common objects (objects that can be seen everyday life) versus block-scrambled versions of themselves, as developed by Kourtzi and Kanwisher (2001). The human motion sensitive complex (hMT+) was identified by the differential BOLD activation to low-contrast moving dots and their stationary versions. The moving dots (4.5′ by 4.5′ square) oscillated between expansion and contraction once per second with a spatial density of 2% and a speed of 4.5°/s. The retinotopic areas were identified with checkerboard rotating wedges that spanned 180° and rotated 30° every 3 s, taking 36 s to go around the display, with 6 periods in one retinotopic run. The symmetry areas were identified by the differential BOLD activation to the symmetric versus the asymmetric scrambled images.
Result and Discussion
The symmetry localizer consisted of two types of nonsense scrambled images computed from the Fourier spectrum of the face stimuli: one vertically symmetric and the other nonsymmetric. Figure 2(a) shows the activation map averaged across observers on 3 views of the inflated brain. The gray shading denotes the gyral (light) and sulcal (dark) layout. The pseudocolor denotes the average coherence between the response waveform and the experimental sequence. The predominant symmetry response occurred in the inferior occipital gyrus, the middle occipital gyrus (MOG) and the areas around intraoccipital sulcus (IOS). The MOG activation is posterior to hMT+ (red contour) and partially overlaps with LOC (magenta contour). The dorsal predominance of the symmetry response is consistent with that observed by Tyler and others (2005a) and Sasaki and others (2005).
Figure 2(b) shows areas localized by localizers on a flatmap. The flatmaps here and in the remainder of this paper had their center near the occipital poles and extended 80 mm in radius around this point. The areas delineated by yellow borders are the first-tier retinotopic areas (V1–V3), identified with a rotating wedge. The criterion for delineating visual areas is discussed elsewhere (Tyler and others 2005b). The magenta borders denote the LOC as identified by contrasting BOLD activation to pictures of objects and their scrambled versions. The red borders denote hMT+ as identified by the motion localizer. The blue contours delimit the extent of the symmetry response. The outline of these areas will be shown for reference in the subsequent flat maps.
Brain Areas for Face Processing
Figure 3 (row 1) shows the averaged activation map across observers for a face localizer contrasting the upright frontal-view faces and the asymmetric scrambled images. We show selected slices of all conditions contrasting upright front-view faces with other images in this figure for ready comparison. The pseudocolored voxels in the figures denote voxels with coherence above 0.475 or Bonferroni corrected P-value (considering all cortical voxels) less than 0.001.
Compared with the scrambled images, the face images bilaterally activated the fusiform, the inferior occipital, and the middle occipital gyri areas around the superior temporal sulcus (STS) and areas around the IOS. The Talairach coordinates (Talairach and Tournoux, 1988) of those activated areas are given in Table 1. Notice that the first-tier retinotopic areas did not show significant differential activation between the two types of images. On average, the BOLD signal modulations in V1 of the left and the right hemispheres were 0.24% and 0.20% (with confidence intervals of 0.29% and 0.24%), respectively. These values are not statistically significant and are much (significantly) smaller than typical V1 responses to contrast energy change, which is about 1–2% modulation, with a confidence interval of about 10% of this value (Engel and others 1997; Tootell and others 1998; Boynton and others 1999). This null result demonstrates that our phase spectrum manipulation equated all the relevant activations in early vision within experimental error. This is an important result because many previous approaches to equating images with their scrambled versions have resulted in significant activation in the early retinotopic areas (Murray and others 2002, 2004; Altmann and others 2003, 2004). It is therefore a worthwhile advance to have developed a form of scrambling that reduces the residual activation in early areas below the typical level of fMRI detectability.
|Activated areas||Talairach coordinates|
|Fusiform||Left||(−32, −46, −17)|
|Right||(28, −57, −14)|
|Inferior occipital gyrus||Left||(−28, −70, −15)|
|Right||(29, −77, −15)|
|IOS||Left||(−31, −75, 16)|
|Right||(24, −86, 13)|
|MOG||Left||(−30, −84, 1)|
|Right||(27, −88, 1)|
|Activated areas||Talairach coordinates|
|Fusiform||Left||(−32, −46, −17)|
|Right||(28, −57, −14)|
|Inferior occipital gyrus||Left||(−28, −70, −15)|
|Right||(29, −77, −15)|
|IOS||Left||(−31, −75, 16)|
|Right||(24, −86, 13)|
|MOG||Left||(−30, −84, 1)|
|Right||(27, −88, 1)|
The IOS activation extended to a more dorsal region along the sulcus than did the symmetry localizer. The green symbols denote the mean locations of the FFA reported in the literature (triangles: averaged from individuals in Kanwisher and others 1997; diamonds: average reported in Rossion and others 2003; circle: reported value in Halgren and others 1999). The cyan symbols denote the OFA reported in Rossion and others (2003). The fusiform form gyrus activation is consistent with the FFA and the inferior occipital gyrus activation is consistent with the OFA reported in the literature. Regions around the STS have been reported as relating to face processing both in humans (Puce and others 1998) and in monkeys (Perrett and others 1982). The IOS activation partially overlaps with the area identified by the symmetry localizer.
For a better understanding of the spatial relationships between the activated areas, Figure 4 shows an occipital flatmap and inflated version of the same activation. As in Figure 2, the flatmap is centered at the occipital pole and displays the spatial location of the activation across the cortical surface. The first-tier retinotopic areas and the human motion area, identified in separate scans, are retained for reference. It may be seen that the face-specific activation runs in a strip lying between these two reference regions. The concentrations corresponding to the FFA and OFA from other studies are identified by triangles and diamonds, respectively. In addition, there is a pronounced dorsal activation in the IOS centered at Talairach (−37, 75, 16) for the left and (24, −86, 13) for the right hemisphere. This region has rarely been mentioned in previous studies of face-specific activation, but appears to be the strongest site of activation in the present dataset. The subsequent conditions are designed to disentangle their particular roles in the agglomeration of cues present in the face localizer.
Image Symmetry Effect
The contrast between the upright frontal-view faces and the symmetric scrambled images (Fig. 3, row 2) was designed to remove the factor of symmetry from the cortical face response in the previous condition. This contrast showed significant activation in the fusiform, the inferior occipital and the middle occipital gyri, and also the area around the IOS. Figure 5 compares the differential activation (pseudocolor patches) in relation to the symmetry (blue contour) and the face areas (FFA: green contours, OFA: cyan contours, and IOS: black contours) identified above. Qualitatively speaking, the major difference between this and the previous condition contrasting with asymmetric scrambled was in the right OFA. The number of activated voxels dropped from 82 to 10 in the right OFA when the symmetry information was available in the scrambled images. The differential contrast (Joseph and others 2002) between the face versus symmetric scrambled condition and the face versus asymmetric scrambled was also significant (F5,10 = 9.98, P = 0.0251 < 0.05) in the right OFA. The average response amplitude drops from 0.33% to 0.19% (t(5) = 3.87, P = 0.0059 < 0.05). On the other hand, the left OFA and the bilateral FFA did not show significant activation response change when symmetry was present in the scrambled images. The activation changes were only from 0.49% to 0.46% (t(5) = 0.58, P = 0.29 > 0.05, not significant [NS]) for the left OFA, from 0.49% to 0.41% (t(5) = 1.56, P = 0.09 > 0.05, NS) for the right fusiform, and 0.44% to 0.35% (t(5) = 1.21, P = 0.14 > 0.05, NS) for the left. Thus, the data for the latter 3 ROIs are consistent with the null hypothesis that the face activation in these areas was insensitive to the symmetry information in the faces, whereas the right OFA showed a significant symmetry component in its response.
Because the difference between the asymmetric and symmetric scrambled images lies in their image symmetry, the activation difference between the 2 conditions should reveal the areas processing 2D image symmetry information. Therefore, the OFA activation is apparently influenced by image symmetry. However, the OFA is not one of the areas identified by our symmetry localizer but lies just next to it. The plausible explanation is that the OFA processes face-specific symmetry information (Rhodes and others 2005). The adjacent relation between the OFA and the symmetry areas hints that the OFA may receive input from nearby symmetry area for facial processing.
The IOS activations, on the other hand, are significantly stronger for the symmetry-equated face/scrambled contrast than for the symmetric/asymmetric noise contrast (F5,10 = 17.85, P = 0.0083 < 0.05, for the right hemisphere, F5,10 = 10.39, P = 0.032 < 0.05 for the left; activation change increased from 0.89% to 1.18%, t(5) = 2.86, P = 0.017 < 0.05 for the right and from 0.46% to 0.62%, t(5) = 3.73, P = 0.007 < 0.05 for the left). Thus, although it is the major locus of activation by symmetry, the IOS is even more strongly activated by the face configuration with symmetry equated. The implication is that this is a cortical region specialized for processing configuration effects, of which symmetry and face arrangements are 2 separate forms.
Face Configuration/Inverted Face Effect
We next compared mean brain activation for the upright versus inverted faces. This is an interesting manipulation because it distinguishes between absolute and relative configuration effects. An absolute configuration effect is one that is specific to the particular orientation and scale of the configuration. For instance, the mouth is located 3° below the eyes with a horizontal position midway between the eyes. This type of configuration is also called the first-order configuration in some face perception literature (Carey and Diamond 1977). A relative configuration effect is one that maintains all the geometric relations among a set of features, regardless of their absolute orientation or scale. For instance, the mouth and the 2 eyes form a triangle regardless whether the faces are upright, tilted, or inverted. This type of configuration should not be confused with the second-order configuration specified in some face perception literature (Carey and Diamond 1977), which refers to the relative distances between facial features. The relative configuration here emphasizes the orientation of the configuration, which is not generally discussed in the second-order configuration literature. In terms of the particular property of symmetry, inverted faces are equated to the upright faces with respect to both the relative and absolute configurations of “image” symmetry. However, the absolute face configuration (or face template) response would be disrupted by the face inversion. Hence, if a brain area such as the OFA processes symmetry in faces in the same way as generic image symmetry, this area should not show differential activation for upright versus inverted face stimuli. On the other hand, if the symmetry is processed as a part of an orientation-specific face gestalt (as would be implied by our constant environmental exposure to upright faces), this area should still show differential activation for these 2 types of faces.
As shown in Figure 3 (row 3), upright versus inverted faces showed robust differential activation in the areas in the fusiform, the inferior occipital, and the middle occipital gyri, and also the areas around IOS and the precuneus. Figure 6 shows the flatmap version of the same activation. The differential activation in the FFA and the OFA was no less than that between faces and asymmetric scrambled images (F5,10 = 0.0277, P = 0.88 > 0.05). These results confirm that the symmetry processing in the OFA found in the previous condition is indeed face specific, or at least orientation specific with respect to face stimuli.
BOLD activation for face inversion has been in reported in many areas throughout the brain (Servos and others 1999; Leube and others 2003; Yovel and Kanwisher 2004, 2005). Reports of the face inversion effect in the literature for the FFA, however, are mixed. Early fMRI studies showed either no (Aguirre and others 1999; Haxby and others 1999) or a weak (Kanwisher and others 1998) face inversion effect in the FFA. More recent fMRI studies have shown a robust face inversion effect in the FFA (Yovel and Kanwisher 2004, 2005). The Yovel and Kanwisher (2005) face inversion effect extended beyond the FFA to an extensive area on the ventral and the lateral occipital surfaces, which is quite similar to ours. Although the cause of this discrepancy in the literature is not resolved, it is possible that the face stimuli used in those studies played a role. The papers reporting no face inversion effect in FFA (Aguirre and others 1999; Haxby and others 1999) used stimuli that had the external contours of the face removed and the one that reported weak face inversion effect (Kanwisher and others 1998) had part of the face covered. On the other hand, studies reporting robust face inversion effects (Yovel and Kanwisher 2004, 2005; and the current study) showed pictures of whole faces as stimuli, including the hair and the neck. It has been suggested that the external contour of a face is also an important part of the face template (Sinha and Poggio 1996; Maurer and others 2002). An incomplete face stimulus may not optimally activate the neurons responsible for the full-face template and may in turn produce insufficient differential activation between upright face and inverted face.
The precuneus activation has a temporal phase opposite to that of the other significant activations. It has been reported that the precuneus is involved in mental rotation (Bonda and others 1995; Cohen and others 1996; Jordan and others 2001). Hence, one likely explanation for the greater activation in the precuneus is that the observers were rotating the inverted face information to enhance recognition. To clarify the exact role of the precuneus in inverted face processing would, however, require further study.
The upright frontal-view faces and the 3/4-view faces share the same 3D object structure but not the same image symmetry. Hence, a brain area for image symmetry would show differential activation for these 2 types of images, whereas an area processing symmetry based on the 3D object structure inferred from the image information should respond equally to these 2 types of faces. Perceptually, we understand that the face is still symmetric although it is 3D rotated, so there must be some brain circuitry that carries this perceptual equivalence. This analysis is supported by the finding that sparsely sampled information about the positions of object features relies on an exclusively 3D coding of object properties (Likova and Tyler 2003). The fact that the face retains the same object structure when 3D rotated relative to the plane of projection can be appreciated only by a neural circuit that has encoded the 3D structure of the projected images.
The upright frontal-view faces and the 3/4-view faces showed less differential activation (Fig. 3, row 4, also see Fig. 7 for inflated and flat versions) than the face template areas identified by faces versus inverted faces. The pseudocolor patches in Figure 7(c) identify the regions showing significant differential activation (F5,10 = 8.95, P = 0.03 < 0.05) between faces and inverted faces but not between faces and 3/4-view faces. This reduction is quite pronounced in the FFA, as more than 61% of voxels showing significant differential activation for faces versus inverted faces (52% for the right and 87% for the left) did not show significant activation in this condition. The activation modulation reduces from 0.49% for faces versus inverted faces to 0.20% for faces versus 3/4-view faces in the left FFA (t(5) = 2.43, P = 0.029 < 0.05). A similar reduction was observed in the right FFA as well (from 0.43% to 0.25%) though the difference is not statistically significant (t(5) = 0.79, P = 0.23 > 0.05, NS).
The activated area in the OFA was also substantially reduced (see Figs 6 vs. 7), although there were some spots still showing activation. The activation modulation was reduced from 0.49% to 0.20% (t(5) = 2.43, P = 0.0296 < 0.05) for the left OFA and 0.47% to 0.26% for the right OFA (t(5) = 2.28, P = 0.036 < 0.05) The implication of these results is that much of the coding in the FFA and OFA is based on the 3D representation of the faces, which “look the same” to these neural circuits despite the radical change in image symmetry. The areas near the IOS and the MOG, however, still showed robust activation to the 3D rotation of the faces as 65% (right) to 90% (left) of IOS voxels showed significant activation in this condition. This strong response implies that they are coding image symmetry rather than object symmetry.
To understand this activation, we may consider the factors that change versus those that are invariant in the image of a 3D-rotated face (Fig. 1a vs. 1c). The invariant factors are low-level ones such as the local edge structure and feature properties, midlevel ones such as the 3D representation of the face structure and high-level ones such as the emotional expression, social position, and personal identity of the individual depicted. The factors that do change with 3D rotation are the local relations between local feature properties (such as angles between the edges) and 2D configurational relations among the features as a whole, including the symmetry relations that activated these regions in the first experiment. We therefore conclude that the MOG and IOS regions are more involved in 2D configural processing, including symmetry relations, than the FFA and OFA, which are mostly concerned with properties that are invariant under rotation.
Our result on viewpoint invariance is consistent with those of Grill-Spector and others (1999), which showed that the fusiform activation in both hemispheres had greater viewpoint invariance than did the dorsal occiput. Vuilleumier and others (2002) used the repetition priming technique to show that left (but not right) fusiform activation had viewpoint-invariant behavior for viewing objects, although size invariance was exhibited bilaterally. It is interesting to note that both our stimulus sets and those of Grill-Spector and others (1999) contained faces, whereas those of Vuilleumier and others (2002) contained only nonbiological objects. Hence, the discrepancy with the latter results may be due to the sampling of stimuli.
Behavioral studies of face recognition showed a viewpoint-dependent effect. When asked whether an image of a face was from the same person as a previously shown face image, the observers‘ recognition performance depends on the viewpoint of the learned face (Troje and Bulthoff 1996; Hill and others 1997) but not the test face (Troje and Bulthoff 1996). This evidence seems contradictory to our fMRI results, which show little differential activation between 3/4-view and front-view faces in the FFA and OFA. Such viewpoint invariance is compatible with models suggesting that viewpoint invariance can be achieved from the concatenation of view-based processes (for reviews, see Riesenhuber and Poggio 2000; Rolls 2000). However, it should be noted that an invariance property in a brain area as a whole does not necessary mean that cells in this area all have the same invariance property. Given the spatial resolution of the fMRI, it is likely that each voxel covers a network of view-based units. Hence, brain areas with such units would appear as viewpoint invariant in their fMRI responses, even though neural recognition responses have access to the individual components of such networks.
Ventral and Dorsal Face Processing
By this point, the roles of the ventral areas in face configuration processing are clear. The right OFA, located just next to common image symmetry areas, does process face-related image symmetry. Hence, it shows differential activation between faces and asymmetric scrambled images but not between faces and symmetric scrambled images.
The symmetry information processed in the right OFA is either orientation specific or a part of face gestalt. After all, the OFA does show differential activation between upright and inverted faces, along with the reduction of differential activation between front view and 3/4 view. The FFA (at least its anterior part) on the other hand, has little interest in image symmetry. Alternation between symmetric- and asymmetric scrambled images produces no detectable effect in this area. In terms of face-specific stimuli, the FFA shows strong differential activation between upright and inverted faces but not between front-view and 3/4-view faces. Hence, the FFA is likely to incorporate an orientation-specific face gestalt (that is, however, viewpoint invariant with respect to lateral rotation).
The roles of the dorsal areas may be understood in terms of relative versus absolute configuration analysis. The symmetry localizer activated part of the areas around the IOS. Those dorsal symmetry areas, at least in the left hemisphere, did show differential activation for the symmetry/asymmetry manipulation and for front-view versus 3/4-view faces, and also between upright and inverted faces, which are equally symmetric. Hence, the IOS area may be specialized for processing spatial configurations, either purely of the symmetry type or of the face-inversion type (with symmetry equated). Because this area responds well to configuration changes whether or not faces are present, whether or not symmetry was manipulated, and whether or not the relative configuration was varied, the implication is that the IOS region is involved in a wide variety of configuration analyses.
The only manipulation that reduced the response in the symmetry activation area was 3D face rotation, which eliminated significant response in much of the MOG region. This region has been particularly associated with the processing of 3D structure (Tyler and others 2006), so the lack of response may be understood as a similar response to the 2 forms of 3D structure carried by both the frontal and rotated faces. The strong response to faces versus scrambled textures makes sense in this context as a response to the depth structure in the faces, but the response to symmetric versus asymmetric textures and to face inversion is harder to understand. It may be that there is a stronger perception of depth structure in upright than in inverted faces due to the match to the 3D gestalt of the upright face, and a stronger sense of 3D structure from the luminance cues of the symmetric versus asymmetric noise (especially in the case of noise with the 1/f-type spectrum of the faces). These are suggestions that would require further study to pin down.
We draw 5 main conclusions from the pattern of results to the symmetry manipulations: 1) The complex of cortical areas activated by the face images, relative to Fourier-matched scrambled images, were the fusiform, inferior, and middle occipital gyri, and areas around the STS and IOS. This is a larger range of dorsal cortical areas than has been previously reported for face-specific activation. 2) The areas responding to symmetry per se were predominantly in the MOG and the IOS. Conversely, the OFA and the FFA did not show significant correlation changes for symmetry in the scrambled images. 3) To identify cortical regions involved in the processes of face recognition, we compared brain activation of the front-view faces and their inverted versions. The upright versus inverted faces showed robust differential activation in the areas in the fusiform, inferior occipital and middle occipital gyri, around the IOS, and in the precuneus. 4) Contrasting the responses to faces and to their vertically symmetric phase-scrambled versions showed a response pattern similar to that produced by contrasting the responses to faces and asymmetric scrambled images (except in the right OFA). This result suggests that the OFA may be involved in processing symmetry specific to face images (in addition to other aspects of the facial configuration). 5) Finally, to study the role of object versus image symmetry, we compare the responses to frontal and 3/4-view faces of the same individuals. This contrast showed little differential activation in the FFA or the OFA but strong responses in areas near the MOG and the IOS, indicating that viewpoint dependence is measurable in the BOLD responses of some cortical areas. Taken together, these results suggest that face processing in the FFA and the OFA is holistic and relatively invariant to the viewpoint from which the face is seen.
Supported by National Science council (Taiwan) 94-2752-H-002-007-PAE to CCC and National Institutes of Health/National Eye Institute (USA) EY 13025 to CWT.
Conflict of Interest: None declared.