Abstract

Neural models of human face perception propose parallel pathways. One pathway (including posterior superior temporal sulcus, pSTS) is responsible for processing changeable aspects of faces such as gaze and expression, and the other pathway (including the fusiform face area, FFA) is responsible for relatively invariant aspects such as identity. However, to be socially meaningful, changes in expression and gaze must be tracked across an individual face. Our aim was to investigate how this is achieved. Using functional magnetic resonance imaging, we found a region in pSTS that responded more to sequences of faces varying in gaze and expression in which the identity was constant compared with sequences in which the identity varied. To determine whether this preferential response to same identity faces was due to the processing of identity in the pSTS or was a result of interactions between pSTS and other regions thought to code face identity, we measured the functional connectivity between face-selective regions. We found increased functional connectivity between the pSTS and FFA when participants viewed same identity faces compared with different identity faces. Together, these results suggest that distinct neural pathways involved in expression and identity interact to process the changeable features of the face in a socially meaningful way.

Introduction

When someone moves about or as their gaze direction or expression changes the size and shape of the image, their face subtends on your retina also changes. To be useful, the visual system must be able to ignore these sources of variation to facilitate the recognition of who the person is, but also use this information to detect changes that enable social communication. Models of human face perception suggest that human observers deal with this problem using separate functional pathways, with the pathways involved in the visual analysis of identity being partially or fully independent of the pathway involved in processing the changeable aspects of faces (Bruce and Young 1986, 2012; Haxby et al. 2000).

Neuroimaging studies have identified a number of face-selective regions, which appear to provide support for the idea of separable visual pathways in face perception (Allison et al. 1994; Kanwisher et al. 1997): An occipital face area (OFA), an fusiform face area (FFA), and posterior superior temporal sulcus (pSTS) region. The OFA is thought to be involved in the early perception of facial features and has a feed-forward projection to both the pSTS and the FFA. The connection between the OFA and pSTS is thought to be important in processing dynamic changes in the face, such as changes in expression and gaze, which are important for social interactions (Puce et al. 1998; Pelphrey et al. 2004; Calder et al. 2007; Engell and Haxby 2007). In contrast, the connection between the OFA and FFA is considered to be involved in the representation of invariant facial characteristics that are important for recognition (Grill-Spector et al. 2004; Rotshtein et al. 2005).

Influenced by the models of face perception, studies over the past decade have concentrated on the functional roles of each of these face-selective pathways (Hoffman and Haxby 2000; Barton et al. 2002; Andrews and Ewbank 2004; Winston et al. 2004; Fox et al. 2009). Consequently, it has remained unclear whether there is any interaction between regions involved in the perception of facial identity and expression. Support for this possibility has come from recent studies that have reported functional connectivity between the pSTS and FFA (Zhang et al. 2009; Turk-Browne et al. 2010; Ethofer et al. 2011). Although the exact role of this interaction between the 2 more heavily investigated pathways is not known, one possibility is that, to process changeable aspects of faces in a socially meaningful way, it is important to track changes across the same identity. Support for this possibility has come from studies that have shown an increased response to sequences of images in with the same facial identity compared with sequences containing different facial identities (Andrews and Ewbank, 2004; Davies-Thompson et al. 2009).

The aim of this study was to probe the interaction of the pathways involved in processing identity and changeable aspects of faces. First, we asked whether the neural response to sequences of faces that change in expression and gaze direction differs according to whether the face images were of the same person or from different people. Next, we used functional connectivity to determine whether the influence of facial identity resulted from neural processes within the pSTS face-selective region itself, or whether it was dependent on interactions with other face regions. To do this, we removed the stimulus-driven activity from the fMR signal and correlated the remaining or residual time-courses between face regions (Davies-Thompson and Andrews forthcoming). This can be an extension of resting-state connectivity in which correlations between regions, independent of a response to stimuli, are examined (Biswal et al. 1995; Margulies et al. 2010). Our aim was to determine how the correlations between regions change as a function of whether the same or different identity faces were viewed (cf. Norman-Haignere et al. 2012).

Materials and Methods

Participants

Data were analyzed from functional localizer scans from 103 different participants (49 females; mean age 24) and run as a standard part of 6 different functional magnetic resonance imaging (fMRI) experiment sessions to identify face-selective regions. Taking advantage of the large number of participants run on the same paradigm allowed us to look at general properties of identity in face processing and delivered substantial statistical power to our functional connectivity analysis, which measured low-level interactions that might normally be swamped by stimulus-driven activity (main effects). All observers had normal or corrected-to-normal vision. Written consent was obtained for all participants and the study was approved by the York Neuroimaging Centre Ethics Committee.

Stimuli and Procedure

The experiment used a block design with 6 different conditions: Same identity faces, different identity faces, bodies, inanimate objects, places, and scrambled images of the former categories (Fig. 1). Face images were taken from the Psychological Image Collection at Stirling (PICS; http://pics.psych.stir.ac.uk/). These images varied in viewpoint (frontal, ¾ view, and profile) and expression (neutral, happy, and speaking) within a block. The changes in viewpoint correspond to changes in gaze direction, which is often signaled in real life by movements of both eyes and head (Bruce and Young 2012). The face images in each block therefore varied in both expression and gaze direction, but in one face condition, the face identity was constant across the images in the block and, in the other face condition, identity varied across the block (Fig. 1A). Both male and female faces were used, but gender was held constant within a block. To determine low-level differences between the image properties in the 2 face conditions, we calculated the absolute difference in gray value across successive images. We then determined the correlation between corresponding pixel values in consecutive images. Supplementary Figure 1 shows the magnitude of the low-level change between successive images in the 2 conditions. There was no significant difference in absolute pixel values across the 2 conditions (t = −1.66; P = 0.106). However, there was a small, but significantly higher correlation between successive images in the same identity faces condition when compared with the different identity faces condition (t = 3.52; P < 0.05).

Figure 1.

Examples of the stimulus conditions. (A) Face images varying in viewpoint/gaze direction and expression were presented with the same identity (first row) or different identities (second row). (B) Examples of the nonface stimulus conditions: bodies, objects, places, and scrambled images.

Figure 1.

Examples of the stimulus conditions. (A) Face images varying in viewpoint/gaze direction and expression were presented with the same identity (first row) or different identities (second row). (B) Examples of the nonface stimulus conditions: bodies, objects, places, and scrambled images.

Examples of nonface stimulus conditions are shown in Figure 1B. Body images were taken from a collection at the University of Bangor (http://www.bangor.ac.uk/~pss811/) and contained clothed male and female headless bodies in a variety of postures. Images of places consisted of a variety of unfamiliar indoor scenes, houses and buildings, city scenes, and natural landscapes. Stimuli in the object condition consisted of different inanimate objects including tools, ornaments, and furniture. Fourier-scrambled images were created by randomizing the phase of each 2-dimensional frequency component in the original image, while keeping the power of the components constant. Scrambled images were generated from the images used in the other stimulus categories.

All images (approximately 8° × 8°) were presented in gray scale and were back-projected onto a screen located inside the bore of the scanner, approximately 57 cm from participants' eyes. Each block consisted of 10 images from a single-stimulus condition; each image was presented for 700 ms and followed by a 200-ms blank screen, resulting in a total block length of 9 s. Stimulus blocks were separated by a 9-s gray screen with a central fixation cross. Each condition was repeated 4 times in a counterbalanced design, resulting in a total scan length of 7.2 min. All participants viewed the same sequence of blocks and images.

Participants were required to monitor all images for the presence of a red dot that was superimposed on 1 or 2 images in each block. Participants were required to respond, with a button press, as soon as they saw the image containing the target. The target could appear in any location on the image and was counterbalanced across conditions. We found no significant differences in the accuracy or reaction time during any of the experimental conditions. Mean detection accuracy was 96.9% overall (same identity faces: 96.6%, different identity faces: 97.5%, and non-face images: 96.9%). A 1-way analysis of variance (ANOVA) revealed no significant difference in accuracy across conditions (F2,200 = 1.01, P = 0.37). The mean reaction time was 441.4 ms overall (same identity faces: 438.7 ms, different identity faces: 441.6 ms, and nonface images: 442.1 ms). A 1-way ANOVA found no significant effect of condition on reaction times (F2,200 = 0.51, P = 0.60).

Imaging Parameters

The experiment was carried out using a 3-T GE HD Excite MRI scanner at the York Neuroimaging Centre (YNiC) at the University of York. An 8-channel, phased-array head coil (GE, Milwaukee) tuned to 127.4 MHz was used to acquire MRI data. A gradient-echo EPI (echo planar imaging) sequence was used to collect the data from 38 contiguous axial slices (time repition = 3 s, time echo = 25 ms, field of view 28 × 28 cm, matrix size = 128 × 128, slice thickness 3 mm). These were coregistered onto a T1-weighted anatomical image (1 × 1 × 1 mm) from each participant. To improve registrations, an additional T1-weighted image was taken in the same plane as the EPI slices.

Whole-Brain Analysis

Statistical analysis of the fMRI data was carried out using FEAT in the FSL toolbox (http://www.fmrib.ox.ac.uk/fsl). The first 3 volumes (9 s) of each scan were removed to minimize the effects of magnetic saturation, and slice-timing correction was applied. Motion correction was followed by spatial smoothing (Gaussian, full width at half maximum 6 mm) and temporal high-pass filtering (cut off, 0.01 Hz). Regressors for each condition in the general linear model (GLM) were convolved with a gamma hemodynamic response function. Individual participant data were then entered into a higher level group analysis using a mixed effects design (FLAME, http://www.fmrib.ox.ac.uk/fsl) whole-brain analysis. To define face-selective regions, “same identity faces” and “different identity faces” were compared with the responses from each of the nonface conditions (bodies, objects, places, and scrambled), and the average of these contrasts was taken. To determine the effect of facial identity, we compared the response from same identity faces with the response from different identity faces.

Functional Connectivity Analysis

To measure the functional connectivity between regions, face-selective regions of interest (ROIs) were identified for each participant using the averaged contrasts of face > bodies, faces > objects, faces > places, and faces > scrambled, thresholded at P < 0.001 (uncorrected). Regions identified included the left and right occipital face area (OFA), the left and right fusiform face area (FFA), and the right posterior superior temporal sulcus (pSTS) face. Because the left pSTS was found in only a small minority of participants (about 20%), this region was not included in the functional connectivity analysis. A control region, which was visually responsive but not face selective, was also defined for each participant by transforming the anatomical “occipital pole, OccP” region from the Harvard-Oxford Cortical Structural Atlas in the MNI standard brain into the participant's functional data space (Desikan et al. 2006). This region included the OccP of both left and right hemispheres.

To assess functional connectivity between regions, we first removed any stimulus-driven activity, as 2 regions will appear highly correlated if both are parallelly driven by the stimulus through a common input. As such, this analysis with stimulus-driven activity removed is orthogonal to the whole-brain GLM analysis. The stimulus-driven activity was removed through 2 steps (Supplementary Fig. 2): First, the stimulus-driven activation as modeled in the GLM analysis was removed, resulting in a residual time series response for each participant. Secondly, to capture any remaining stimulus-driven response that might not be fully accounted for the hemodynamic model, the first residual time series response was averaged across all ROIs (left and right OFA, left and right FFA, and right pSTS and OccP) and across all 103 participants. The rationale for combining across regions is that the average time-course of response was very similar across regions (Supplementary Fig. 4). The first-level analysis was then repeated with the average first residual response as an additional regressor. This gave rise to a second residual for each participant. The time-points corresponding to the same identity and different identity face blocks were then extracted for the functional connectivity analysis. Correlations between different regions were calculated using the extracted time-points for each participant for the same identity and different identity conditions. Mean correlations and standard error of the mean were calculated across participants for each condition and ROI pair. To test statistically whether the functional connectivity between regions was influenced by facial identity, correlations from each participant were converted to a normal distribution using a Fisher transformation, and then appropriate statistical tests (repeated-measures ANOVAs and post hoc t-tests) were performed.

Results

Whole-Brain Analysis

To determine the effect of facial identity, we compared the response to sequences of faces that had the same identity with sequences of faces that contained different identities (Fig. 1A). Figure 2A shows regions that had significantly different responses to the same identity faces condition compared with the different identity faces condition. Regions that responded more to the same identity faces condition are shown in red and regions that showed a greater response to different identity faces are shown in blue. The data have been thresholded to a value of P < 0.00001 (uncorrected; Z-value > 4.2) in order to highlight ROIs. A region within the right pSTS showed a significantly greater response to same identity faces compared with different identity faces. In contrast, we found a region in the fusiform gyrus that responded more to different identity faces compared with same identity faces. The coordinates of these regions are shown in Tables 1 and 2. The volume of active voxels is also shown for each region, thresholded both at Z > 4.2 (P < 0.00001, uncorrected) and at Z > 4.6 (P < 0.05, resel corrected for multiple comparisons; http://fsl.fmrib.ox.ac.uk/fsl/fsl4.0/feat5/detail.html#poststats).

Table 1

Location of regions that show a greater response to same identity faces compared with different identity faces

Region x y z Peak Z-score Volume (cm³) Z > 4.2, P < 0.000013 (uncorrected) Volume (cm³), Z > 4.6, P < 0.05 (corrected) 
STS 
 L −42 −65 −3 4.59 0.09 — 
 R 54 −49 5.16 2.55 0.70 
IFG 
 R 55 18 −3 4.72 0.30 0.02 
Region x y z Peak Z-score Volume (cm³) Z > 4.2, P < 0.000013 (uncorrected) Volume (cm³), Z > 4.6, P < 0.05 (corrected) 
STS 
 L −42 −65 −3 4.59 0.09 — 
 R 54 −49 5.16 2.55 0.70 
IFG 
 R 55 18 −3 4.72 0.30 0.02 

Coordinates refer to the center of gravity of each group of active voxels. Volume of active voxels is shown thresholded both uncorrected (P < 0.00001) and corrected (P < 0.05) for multiple comparisons.

Table 2

Location of regions that show a greater response to different identity faces compared with same identity faces

Region x y z Peak Z-score Volume (cm³), Z > 4.2, P < 0.000013 (uncorrected) Volume (cm³), Z > 4.6, P < 0.05 (corrected) 
Medial occipital 
 L −9 −93 −3 5.64 1.43 0.86 
 FFA       
 R 38 −49 −26 5.25 0.34 0.14 
Anterior temporal 
 L −36 −18 −35 4.32 0.02 — 
 R 34 −16 −36 4.93 0.08 0.02 
Region x y z Peak Z-score Volume (cm³), Z > 4.2, P < 0.000013 (uncorrected) Volume (cm³), Z > 4.6, P < 0.05 (corrected) 
Medial occipital 
 L −9 −93 −3 5.64 1.43 0.86 
 FFA       
 R 38 −49 −26 5.25 0.34 0.14 
Anterior temporal 
 L −36 −18 −35 4.32 0.02 — 
 R 34 −16 −36 4.93 0.08 0.02 

Details as in Table 1.

Figure 2.

Whole-brain analysis (n = 103). (A) Regions showing greater response to same identity faces compared with different identity faces (red), and to different identity faces compared with same identity faces (blue). (B) Activation to same identity faces compared with nonface stimuli (places, objects, bodies, and scrambled images). (C) Activation to different identity faces compared with nonface stimuli. Face-selective regions are labeled: FFA, OFA, and STS. All brain images are depicted in radiologic convention, that is, coronal and axial slices are left/right reversed. The MNI coordinates (mm) of slices shown: x = 40, y = −46, z = −26. Statistical maps were thresholded at Z > 4.2 (P < 0.00001, uncorrected).

Figure 2.

Whole-brain analysis (n = 103). (A) Regions showing greater response to same identity faces compared with different identity faces (red), and to different identity faces compared with same identity faces (blue). (B) Activation to same identity faces compared with nonface stimuli (places, objects, bodies, and scrambled images). (C) Activation to different identity faces compared with nonface stimuli. Face-selective regions are labeled: FFA, OFA, and STS. All brain images are depicted in radiologic convention, that is, coronal and axial slices are left/right reversed. The MNI coordinates (mm) of slices shown: x = 40, y = −46, z = −26. Statistical maps were thresholded at Z > 4.2 (P < 0.00001, uncorrected).

To determine the spatial relationship between the regions shown in Figure 2A and face-selective regions such as the pSTS and FFA, we compared each face condition (same and different identity faces) with all the different nonface conditions (bodies, objects, places, scrambled—Fig. 1B). Figure 2B reveals the location of face-selective regions in the occipital and temporal lobes defined by contrasting the same identity face condition with the nonface conditions (yellow). Figure 2C shows the regions defined by contrasting the different identity face condition with the nonface conditions (green). Both of these contrasts reveal a very similar pattern of face-selective regions that include the left and right OFA, the left and right FFA, and the right pSTS face area. It is also clear that the location of the face-selective right pSTS and right FFA in Figure 2B,C (Tables 3 and 4) corresponds closely with the right pSTS and fusiform gyrus regions shown in Figure 2A.

Table 3

Location of the core face-selective regions defined by the contrast of same identity faces > bodies, objects, places, and scrambled images

Region x y z Peak Z-score Volume (cm³) 
FFA 
 L −42 −56 −24 4.38 0.05 
 R 43 −53 −23 6.92 2.07 
OFA 
 L −41 −85 −16 5.41 0.93 
 R 42 −80 −16 7.14 3.06 
STS 
 L −49 −55 4.94 0.97 
 R 53 −51 8.49 19.90 
Region x y z Peak Z-score Volume (cm³) 
FFA 
 L −42 −56 −24 4.38 0.05 
 R 43 −53 −23 6.92 2.07 
OFA 
 L −41 −85 −16 5.41 0.93 
 R 42 −80 −16 7.14 3.06 
STS 
 L −49 −55 4.94 0.97 
 R 53 −51 8.49 19.90 
Table 4

Location of the core face-selective regions defined by the contrast of different identity faces > bodies, objects, places, and scrambled images

Region x y z Peak Z-score Volume (cm³) 
FFA 
 L −42 −56 −25 5.67 0.77 
 R 43 −52 −23 8.23 2.98 
OFA      
 L −41 −84 −17 5.56 0.92 
 R 40 −81 −16 7.06 2.23 
STS      
 L — — — — — 
 R 52 −53 6.86 10.44 
Region x y z Peak Z-score Volume (cm³) 
FFA 
 L −42 −56 −25 5.67 0.77 
 R 43 −52 −23 8.23 2.98 
OFA      
 L −41 −84 −17 5.56 0.92 
 R 40 −81 −16 7.06 2.23 
STS      
 L — — — — — 
 R 52 −53 6.86 10.44 

Other regions that showed a significantly greater response to same identity faces compared with different identity faces include the left pSTS and the right inferior frontal gyrus (IFG; Supplementary Fig. 3, red). The coordinates of these regions are shown in Table 1. Regions that responded more to different identity faces compared with same identity faces are shown in Supplementary Figure 3 (blue). In addition to the right FFA, significant responses were found in the anterior temporal lobe and in the medial occipital region (Table 2).

Functional Connectivity Analysis

The core face-selective regions (left and right OFA, left and right FFA, and right pSTS) were identified independently for each participant. The average time-course of response in these regions is shown in Supplementary Figure 4. Next, we determined whether the higher response to sequences of faces with the same identity in the pSTS was dependent on processing within this region or was dependent on interactions with other face-selective regions. Specifically, we compared the functional connectivity between face-selective regions when viewing same identity faces and different identity faces. To examine the functional connectivity between regions, we removed the stimulus-driven activity (Fig. 3). The residual time-courses were then correlated between pairs of face-selective regions. If the selectivity for the identity of faces in the pSTS is dependent on interactions with other face-selective regions, we would predict a change in the correlations when viewing same identity faces compared with different identity faces.

Figure 3.

Methods for calculating functional connectivity between face-selective regions. (A) Time-course of activation for 2 ROIs for a single participant (left: Region 1, right: Region 2). (B) A GLM for the 2 regions. (C) Residual time-course of activation not explained by the GLM. Colors represent the different stimulus conditions within each block (see legend). (D) Correlations of residual activity between region 1 and region 2 for time-points within same identity faces (left) or different identity faces (right) conditions.

Figure 3.

Methods for calculating functional connectivity between face-selective regions. (A) Time-course of activation for 2 ROIs for a single participant (left: Region 1, right: Region 2). (B) A GLM for the 2 regions. (C) Residual time-course of activation not explained by the GLM. Colors represent the different stimulus conditions within each block (see legend). (D) Correlations of residual activity between region 1 and region 2 for time-points within same identity faces (left) or different identity faces (right) conditions.

Figure 4 shows the average correlations between the core face-selective regions. For the bilateral ROIs (lOFA–lFFA and rOFA–rFFA), there was no significant interaction between hemisphere and condition (F1,71 = 0.4, P = 0.54). Therefore, correlations between OFA and FFA have been averaged across hemispheres for each participant. As the pSTS was found in most participants only on the right hemisphere, correlations reported between the pSTS and OFA or FFA refer to the right hemisphere only (i.e., rOFA–rSTS and rFFA–rSTS). The data show that there was an increased correlation between the STS and both the OFA and FFA when viewing same identity faces compared with different identity faces.

Figure 4.

Mean correlations (across participants) of residual activity between 3 face-selective ROIs (OFA, FFA, and STS) during the same identity faces and different identity faces conditions. Errors represent standard error of the mean (SEM) across participants. * P < 0.05.

Figure 4.

Mean correlations (across participants) of residual activity between 3 face-selective ROIs (OFA, FFA, and STS) during the same identity faces and different identity faces conditions. Errors represent standard error of the mean (SEM) across participants. * P < 0.05.

A 2 × 3 repeated-measures ANOVA with condition (same identity faces and different identity faces) and connection (OFA–FFA, OFA–pSTS, and FFA–pSTS) as factors revealed a significant effect of condition (F1,78 = 6.1, P < 0.05), connection (F2,156 = 41.7, P < 0.001) and a significant interaction between them (F2,156 = 3.2, P < 0.05). Post hoc tests were then used to determine whether there were any significant differences in the correlations between regions for same identity faces or different identity faces. Significantly greater correlations were evident between the OFA and pSTS (t(79) = 2.7, P < 0.01) and between the FFA and pSTS (t(88) = 1.9, P < 0.05) for the same identity faces condition compared with the different identity condition. These results imply that the strength of the connections between the pSTS and other face-selective regions is influenced by the identity of the face. In contrast, there was no difference in the correlations for the same identity faces condition compared with the different identity condition between the OFA and FFA (t(92) = 0.16, P = 0.98).

To validate our functional connectivity analysis, we performed a separate analysis to ensure that all stimulus-driven activity was removed from the residual time-series before calculating correlations. Rather than calculating correlations between ROIs within participants, correlations in this control analysis were calculated between random pairs of participants, for example, FFA (participant 1) – OFA (participant 2). Unlike the positive values generated by the within-participant correlations (Fig. 4), control correlations across participants were close to 0 (mean [standard error of the mean]: FFA–OFA = −0.02 [0.03], OFA–STS = −0.05 [0.02], FFA–STS = −0.05 [0.02]).

To examine whether the effect of identity on connectivity with the pSTS is specific to face-selective regions, we calculated correlations between a control region, the OccP, and each of the face-selective regions such as OFA, FFA, and STS. Supplementary Figure 4 shows the mean time-course of response in the OccP across all participants. The proportion of voxels in OccP that responded more to face compared with nonface conditions was 2.7% at P < 0.05, uncorrected or 0% at P < 0.05, corrected. This shows that the OccP was not responding selectively to faces. Figure 5 shows the average correlation values between the OccP and each face-selective region. A 2 × 3 repeated-measures ANOVA with condition (same identity faces and different identity faces) and connection (OFA–OccP, FFA–OccP, and STS-OccP) as factors revealed no significant effect of condition (F1,85 = 0.12, P = 0.73), and no significant interaction between condition and connection (F2,170 = 0.15, P = 0.86).

Figure 5.

Mean correlations (across participants) of residual activity between 3 face-selective ROIs (OFA, FFA, and STS) and the OccP region during the same identity faces and different identity faces conditions. Errors represent SEM across participants.

Figure 5.

Mean correlations (across participants) of residual activity between 3 face-selective ROIs (OFA, FFA, and STS) and the OccP region during the same identity faces and different identity faces conditions. Errors represent SEM across participants.

Finally, we examined the face selectivity of functional connectivity between regions by comparing correlation values between regions during face (same identity and different identity) and nonface blocks (bodies, >objects, places, and scrambled images). We found that correlations between the OFA and FFA were significantly higher during face blocks compared with nonface blocks in the right hemisphere (t(83) = 2.20, P = 0.03), but not in the left hemisphere (t(80) = −0.09, P = 0.93). Although correlations were not significantly higher during face blocks compared with nonface blocks for rOFA–rSTS (t(79) = 0.12, P = 0.90) or the rFFA–rSTS (t(88) = 1.46, P = 0.15), correlations were significantly higher during same identify face blocks compared with nonface blocks for both rFFA–rSTS (t(88) = 2.86, P = 0.005) and rOFA–rSTS (t(79) = 2.07, P = 0.042).

Discussion

The posterior STS is a region that is known to respond to changes in facial expression and gaze direction. Our aim was to establish whether this region is also sensitive to face identity. We found that the face-selective region within the pSTS responded preferentially to sequences of face images that had the same identity compared with sequences of face images with different identities. To determine whether this selectivity for facial identity was dependent on processing within the pSTS or involved interactions with other regions, we measured the functional connectivity between the pSTS and other face-selective regions. Our results revealed increased functional connectivity between the FFA and pSTS when changes in facial expression and gaze occur across the same identity compared with when these changes in expression and gaze occur across different identities.

Because of the considerable importance attached to different types of facial information, the most efficient way to analyze this information is thought to involve different neural subcomponents that are optimally tuned for particular types of facial signal (Bruce and Young 1986; Haxby et al. 2000). For example, models of face perception suggest that the analysis of facial identity occurs largely independently of the processing of changeable aspects such as expression. However, not all lines of evidence support a total separation of identity and expression (Calder and Young 2005; Bruce and Young 2012). For example, studies have shown that the perceptual effects of adaptation to emotional expression are more pronounced if the adapting and test expressions are from the same person (Fox and Barton 2007; Ellamil et al. 2008; Campbell and Burke 2009). These studies fit with other behavioral results that have shown that the ability to judge expression can be influenced by concomitant changes in identity (Schweinberger and Soukup 1998). A corresponding effect of identity on the processing of facial expression was also shown in later event-related potential studies (Martens et al. 2010a, 2010b). Further support for the idea that the pathways involved in the perception of identity and expression may not be completely independent can be found in the way the image statistics of the face vary with changes in expression and identity. Principal components analysis has shown that some of the principal components associated with changes to the face are associated with changes in identity or expression, but others reflect changes in both identity and expression (Calder et al. 2001).

Our results also show that there is a less than perfect separation between the neural representations used for identity and expression perception. There are 2 distinct reasons why this might be the case. One possibility might be that the response to facial identity in the face-selective pSTS could reflect that this region can represent the invariant aspects of a face that are necessary for the perception of identity. However, an equally plausible alternative is that the response to facial identity arises through interactions of pSTS with other face-selective regions that are associated with an analysis of invariant aspects of faces (such as identity). To address this issue, we investigated the functional connectivity between the pSTS and other face-selective regions. We found that there was increased connectivity between the pSTS and both the FFA and OFA for sequences of faces in which the identity was unchanged compared with sequences when the identity changed. The selectivity of this change is shown by the absence of change in connectivity between the OFA and FFA and by the lack of any difference in the correlations between the face-selective regions and an early visual region. Moreover, the increased connectivity between the pSTS and OFA or FFA for same identity faces was evident even though responses in the OFA and FFA were greater to different identity faces. This suggests that stronger connectivity is not necessarily dependent on the magnitude of the response at both ends of the connection (e.g. FFA and pSTS) and is instead due to the synergistic response between 2 regions carrying specific facial information relevant to social communication. The selectivity of this connection is further demonstrated by the fact that the correlations between the OFA/FFA and STS were not increased to the different identity faces compared with the nonface conditions (see also, Davies-Thompson and Andrews forthcoming). Indeed, these changes in patterns of functional connectivity could reflect a more general mechanism for category-selective interactions within the brain (see Norman-Haignere et al. 2012). Together, our results indicate that the response to identity in the pSTS is influenced by other face-selective regions that are involved in processing invariant aspects of faces that are important for the perception of facial identity.

To be socially meaningful, changes in expression and gaze direction must often be tracked across an individual whose invariant features (identity) remain constant. The preferential response in the pSTS to sequences of faces, which vary in expression and gaze, but not in identity, is therefore consistent with the role of this region in social communication (Allison et al. 2000; Perrett et al. 2009). However, other studies using event-related fMRI adaptation paradigms have reported an opposite pattern of results with greater responses to different identity faces compared with same identity faces (Winston et al. 2004; Fox et al. 2009). An explanation for the difference could be that the face images used in this study are presented in a sequence, which show gradual changes in expression and viewpoint over time consistent with the changes that typically occur during social interactions. Moreover, our results are consistent with previous studies that have shown an increased response in the pSTS to sequences of faces, which have the same identity (Andrews and Ewbank 2004; Davies-Thompson et al. 2009) and with studies have found that response in the pSTS is greater to dynamic sequences of faces (Lee et al. 2010; Pitcher et al. 2011). In addition to the pSTS, we also found that the right IFG was more active to same identity faces compared with different identity faces. Previous studies have shown that the right inferior frontal gyrus (rIFG) is involved in face processing and that there is a functional connection between the rSTS and rIFG (Chan and Downing 2011; Ethofer et al. 2011; Gschwind et al. 2011; Davies-Thompson and Andrews 2012). Interestingly, the response of this region to changes in facial expression has been shown to be attenuated in autism (Dapretto et al. 2006). These findings suggest that signals that are important for social communication may be relayed to the frontal lobe from the pSTS.

Models of face perception suggest that the FFA is involved in processing the invariant features of a face that are important for recognition (Haxby et al. 2000). We found a greater response to sequences of different identity faces compared with same identity faces. One explanation for the difference in response could be that the neurons in the FFA are invariant to changes in expression and gaze and are sensitive to changes in identity. Thus, the same identity faces activate an overlapping population of neurons in the FFA that adapts with repetitive presentations (Grill-Spector et al. 2006). In contrast, the different identity faces activate nonoverlapping populations of neurons that do not adapt and consequently give rise to a greater response. Indeed, if the increased functional connectivity between the FFA and pSTS is conveying important information about identity, it would be necessary for the FFA to discriminate between the same and different identity conditions.

We found greater activation to different identity faces compared with same identity faces in the anterior temporal lobes (Table 2 and Supplementary Fig. 3). These regions are known to contain image invariant representations of complex objects, including faces (Quian Quiroga et al. 2005, Freiwald and Tsao 2010). Although these results are consistent with the idea that these regions contain image invariant representations of facial identity, we also found a region in the medial occipital lobe typically associated with processing low-level visual features that were also more responsive to different than same identity faces. This suggests that a greater variability in the image statistics across successive images (Supplementary Fig. 1) could have resulted in a greater response to the different identity faces condition in early visual areas that is then relayed to face-selective regions in the inferior temporal lobe.

In conclusion, we found that pSTS responded more to sequences of faces that changed in expression and gaze, but did not change in identity, compared with similar sequences that changed in identity. We also found increases in functional connectivity between the pSTS and face-selective regions, such as the FFA, that are implicated in processing facial identity. These results are consistent with the general claim that pSTS is involved in representing changeable aspects of faces (Haxby et al. 2000), but also offer a novel perspective on the neural processing in the pSTS, in which neurons in this region are particularly interested in changeable aspects of the same face. We propose that this reflects the critical social importance of monitoring changes in a particular individual's gaze and expression and demonstrate through connectivity analyses a potential mechanism through which this can happen.

Supplementary Material

Supplementary material can be found at: http://www.cercor.oxfordjournals.org/.

Funding

This work was supported by a grant from the Wellcome Trust (WT087720MA). R.H. is supported by a studentship from the University of York. Funding to pay the Open Access publication charges for this article was provided by the Wellcome Trust.

Notes

Conflict of Interest: None declared.

References

Allison
T
McCarthy
G
Nobre
A
Puce
A
Belger
A
Human extrastriate visual cortex and the perception of faces, words, numbers, and colors
Cereb Cortex
 , 
1994
, vol. 
4
 (pg. 
544
-
554
)
Allison
T
Puce
A
McCarthy
G
Social perception from visual cues: role of the pSTS region
Trends Cogn Sci
 , 
2000
, vol. 
4
 (pg. 
267
-
278
)
Andrews
TJ
Ewbank
MP
Distinct representations for facial identity and changeable aspects of faces in the human temporal lobe
Neuroimage
 , 
2004
, vol. 
23
 (pg. 
905
-
913
)
Barton
JJS
Press
DZ
Keenan
JP
O'Connor
M
Lesions of the fusiform face area impair perception of facial configuration in prosopagnosia
Neurology
 , 
2002
, vol. 
58
 (pg. 
71
-
78
)
Biswal
B
Yetkin
FZ
Haughton
VM
Hyde
JS
Functional connectivity in the motor cortex of resting human brain using echo-planar MRI
Magn Reson Med
 , 
1995
, vol. 
34
 (pg. 
537
-
541
)
Bruce
V
Young
A
Face perception
 , 
2012
Hove, East Sussex
Psychology Press
Bruce
V
Young
A
Understanding face recognition
Br J Psychol
 , 
1986
, vol. 
77
 (pg. 
305
-
327
)
Calder
AJ
Beaver
JD
Winston
JS
Dolan
RJ
Jenkins
R
Eger
E
Henson
RN
Separate cod­ing of different gaze directions in the superior temporal sulcus and inferior parietal lobule
Curr Biol
 , 
2007
, vol. 
17
 (pg. 
20
-
25
)
Calder
AJ
Burton
AM
Miller
P
Young
AW
Akamatsu
S
A principal component analysis of facial expressions
Vis Res
 , 
2001
, vol. 
41
 
9
(pg. 
1179
-
1208
)
Calder
AJ
Young
AW
Understanding the recognition of facial identity and facial expression
Nat Rev Neurosci
 , 
2005
, vol. 
6
 (pg. 
641
-
651
)
Campbell
J
Burke
D
Evidence that identity-dependent and identity independent neural populations are recruited in the perception of five basic emotional facial expressions
Vis Res
 , 
2009
, vol. 
49
 (pg. 
1532
-
1540
)
Chan
AW
Downing
PE
Faces and eyes in human lateral prefrontal cortex
Front Hum Neurosci
 , 
2011
, vol. 
5
 pg. 
51
 
Dapretto
M
Davies
MS
Pfeifer
JH
Scott
AA
Sigman
M
Bookheimer
SY
Iacoboni
M
Understanding emotions in others: mirror neuron dysfunction in children with autism spectrum disorders
Nat Neurosci
 , 
2006
, vol. 
9
 (pg. 
28
-
30
)
Davies-Thompson
J
Andrews
TJ
Intra- and inter-hemispheric connectivity between face-selective regions in the human brain
J Neurophysiol
 , 
Forthcoming
 
.
Davies-Thompson
J
Gouws
A
Andrews
TJ
An image-dependent representation of familiar and unfamiliar faces in the human ventral stream
Neuropsychologia
 , 
2009
, vol. 
47
 
6
(pg. 
1627
-
1635
)
Desikan
RS
Ségonne
F
Fischl
B
Quinn
BT
Dickerson
BC
Blacker
D
Buckner
RL
Dale
AM
Maguire
RP
Hyman
BT
, et al.  . 
An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest
Neuroimage
 , 
2006
, vol. 
31
 (pg. 
968
-
980
)
Ellamil
M
Susskind
JM
Anderson
AK
Examinations of identity invariance in facial expression adaptation
Cogn Affect Behav Neurosci
 , 
2008
, vol. 
8
 (pg. 
273
-
281
)
Engell
AD
Haxby
JV
Facial expression and gaze-direction in human superior temporal sulcus
Neuropsychologia
 , 
2007
, vol. 
45
 (pg. 
3234
-
3241
)
Ethofer
T
Gschwind
M
Vuilleumier
P
Processing social aspects of human gaze: a combined fMRI-DTI study
Neuroimage
 , 
2011
, vol. 
55
 (pg. 
411
-
419
)
Fox
CJ
Barton
JJS
What is adapted in face adaptation? The neural representations of expression in the human visual system
Brain Res
 , 
2007
, vol. 
1127
 (pg. 
80
-
89
)
Fox
CJ
Moon
S-Y
Iaria
G
Barton
JJS
The correlates of subjective perception of identity and expression in the face network: an fMRI adaptation study
Neuroimage
 , 
2009
, vol. 
44
 (pg. 
569
-
580
)
Freiwald
WA
Tsao
DY
Functional compartmentalization and viewpoint generalization within the macaque face-processing system
Science
 , 
2010
, vol. 
330
 pg. 
845
 
Grill-Spector
K
Henson
R
Martin
A
Repetition and the brain: neural models of stimulus-specific effects
Trends Cogn Sci
 , 
2006
, vol. 
10
 (pg. 
14
-
23
)
Grill-Spector
K
Knouf
N
Kanwisher
N
The fusiform face area subserves face perception, not generic within-category identification
Nat Neurosci
 , 
2004
, vol. 
7
 (pg. 
555
-
562
)
Gschwind
M
Pourtois
G
Schwartz
S
Van De Ville
D
Vuilleumier
P
White matter connectivity between face-responsive regions in the human brain
Cereb Cortex
 , 
2011
Haxby
JV
Hoffman
EA
Gobbini
MI
The distributed human neural system for face perception
Trends Cogn Sci
 , 
2000
, vol. 
4
 (pg. 
223
-
233
)
Hoffman
EA
Haxby
JV
Distinct representation of eye gaze and identity in the distributed human neural system for face perception
Nat Neurosci
 , 
2000
, vol. 
3
 (pg. 
80
-
84
)
Kanwisher
N
McDermott
J
Chun
MM
The fusiform face area: a module in human extrastriate cor­tex specialized for face perception
J Neurosci
 , 
1997
, vol. 
17
 (pg. 
4302
-
4311
)
Lee
LC
Andrews
TJ
Johnson
SJ
Woods
W
Gouws
A
Green
GGR
Young
AW
Neural responses to rigidly moving faces displaying shifts in social attention investigated with fMRI and MEG
Neuropsychologia
 , 
2010
, vol. 
48
 (pg. 
477
-
490
)
Margulies
DS
Bottger
J
Long
X
Yating
L
Kelly
C
Schafer
A
Goldhahn
D
Abbushi
A
Milham
MP
Lohmann
G
, et al.  . 
Resting developments: a review of fMRI post-processing methodologies for spontaneous brain activity
Magn Reson Mater Phys Biol Med
 , 
2010
, vol. 
23
 (pg. 
289
-
307
)
Martens
U
Leuthold
H
Schweinberger
SR
Parallel processing in face perception
J Exp Psychol Hum Percept Perform
 , 
2010a
, vol. 
36
 (pg. 
103
-
121
)
Martens
U
Leuthold
H
Schweinberger
SR
On the temporal organization of facial identity and expression analysis: Inferences from event-related brain potentials
Cogn Affect Soc Neurosci
 , 
2010b
, vol. 
10
 (pg. 
505
-
522
)
Norman-Haignere
SV
McCarthy
G
Chun
MM
Turk-Browne
NB
Category-selective background connectivity in ventral visual cortex
Cereb Cortex
 , 
2012
, vol. 
22
 (pg. 
391
-
402
)
Pelphrey
KA
Viola
RJ
McCarthy
G
Processing of mutual and averted social gaze in the superior temporal sulcus
Psychol Sci
 , 
2004
, vol. 
15
 (pg. 
598
-
603
)
Perrett
DI
Xiao
D
Barraclough
NE
Keysers
C
Oram
MW
Seeing the future: natural image sequences produce ‘anticipatory’ neuronal activity and bias perceptual report
Q J Exp Psychol
 , 
2009
, vol. 
62
 (pg. 
2081
-
2104
)
Pitcher
D
Dilks
DD
Saxe
RR
Triantafyllou
C
Kanwisher
N
Differential selectivity for dynamic versus static information in face-selective cortical regions
Neuroimage
 , 
2011
, vol. 
56
 (pg. 
2356
-
2363
)
Puce
A
Allison
T
Bentin
S
Gore
JC
McCarthy
G
Temporal cortex activation in humans view­ing eye and mouth movements
J Neurosci
 , 
1998
, vol. 
18
 (pg. 
2188
-
2199
)
Quian Quiroga
R
Reddy
L
Kreiman
G
Koch
C
Fried
I
Invariant visual representation by single-neurons in the human brain
Nature
 , 
2005
, vol. 
435
 (pg. 
1102
-
1107
)
Rotshtein
P
Henson
RN
Treves
A
Driver
J
Dolan
RJ
Morphing Marilyn into Maggie dissociates physical and identity face representations in the brain
Nat Neurosci
 , 
2005
, vol. 
8
 (pg. 
107
-
113
)
Schweinberger
SR
Soukup
GR
Asymmetric relationships among perceptions of facial identity, emotion, and facial speech
J Exp Psychol Hum Percep Perform
 , 
1998
, vol. 
24
 (pg. 
1748
-
1765
)
Turk-Browne
NB
Norman-Haignere
SV
McCarthy
G
Face-specific resting functional connectivity between the fusiform gyrus and posterior superior temporal sulcus
Front Hum Neurosci
 , 
2010
, vol. 
4
 
176
(pg. 
1
-
15
)
Winston
JS
Henson
RN
Fine-Goulden
MR
Dolan
RJ
fMRI-adaptation reveals dissociable neural representations of identity and expression in face perception
J Neurophysiol
 , 
2004
, vol. 
92
 (pg. 
1830
-
1839
)
Zhang
H
Tian
J
Liu
J
Li
J
Lee
K
Intrinsically organized network for face perception during the resting state
Neurosci Lett
 , 
2009
, vol. 
454
 (pg. 
1
-
5
)
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by-nc/3.0/), which permits non-commercial reuse, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact journals.permissions@oup.com.