Perception of dispositions of others revealed by movement is an essential ingredient of adaptive daily-life social behavior. Brain imaging points to several brain regions involved in visual processing of social interaction represented by motion of geometric shapes. However, temporal interrelations among these regions remain unknown. Keeping in mind that successful visual social perception depends on intact communication throughout the brain, we focus here on analysis of the induced gamma neuromagnetic response to social interaction revealed by motion. A peak of induced gamma activity of 62 Hz was found at 1 s from the stimulus onset over the right parieto–temporal junction. Two further enhancements in gamma response of lower frequency of 44 Hz occurred at 1.4 s over the medial prefrontal and posterior temporal cortices in the right hemisphere. Subsequent boosts of 44 Hz were found at 1.6 s over the left temporal and right posterior temporal cortices. For the first time, the findings identify the cortical network engaged in visual processing of social interaction revealed by motion and help to better understand proper functioning of the social brain circuitry.
Visual information revealed by motion of living beings allows for veridical estimation of social properties of agents involved in these events. Seminal experiments by Heider and Simmel (1944) elegantly demonstrate that when simple geometric shapes (disks or triangles) irregularly move with different spatiotemporal characteristics (velocity and acceleration), healthy naive observers describe these events in terms of social interaction (one figure pushes, entrains, chases, or launches the other one). Moreover, particular personal traits, needs, dispositions, and emotions (aggression, fear, or rescue) are often attributed to these figures as if they were animate beings. This fascinating phenomenon has been further investigated in follow-up studies in normalcy and pathology, and interest in such issues has peaked in recent years (e.g., Heberlein and Adolphs 2004; Tremoulet and Feldman 2006; Schlottmann et al. 2009). These studies show that the impression of animacy and social interaction from dynamic displays of self-propelled moving shapes is robust, highly stimulus driven, cross-cultural and seems to be perceptually hardwired in its nature.
Functional magnetic resonance imaging (fMRI) and positron emission tomography (PET) point to several brain regions that are active during visual tasks that make use of Heider-and-Simmel (HS) animations. These areas include among others the posterior part of the right superior temporal sulcus (pSTS), parieto–temporal junction (PTJ), the fusiform face area (FFA), and the medial prefrontal cortex (mPFC) (Castelli et al. 2000, 2002; Martin and Weisberg 2003; Schultz et al. 2003; Ohnishi et al. 2004; Schultz J et al. 2004, 2005; Gobbini et al. 2007; Wheatley et al. 2007; Tavares et al. 2008). However, fMRI provides only indirect measure of neural activity, assessing the changes in cerebral metabolism that are coupled in a complex way to those in neural activity. More important, previous brain imaging findings are restricted to localization of brain regions involved in visual perception of social interaction through motion. The rationale for the present study is, therefore, to uncover the time course and dynamic topography of cortical activity associated with such perceptual phenomena.
We focused here on analysis of oscillatory gamma magnetoencephalographic (MEG) activity for the following reasons. First, it has been argued that binding of widely spread cell assemblies by synchronizing their oscillatory activity in the gamma range underlies intact brain communication (e.g., Womelsdorf et al. 2007). Gamma-band synchronization has been proposed as a putative mechanism for the functional integration of neural populations that together form a transitory, large-scale percept-specific network (Kaiser and Lutzenberger 2005). Second, intact neural communication is of immense importance for proper functioning of networks engaged in visual social perception (Pavlova 2005; Pavlova et al. 2008, 2009; Pelphrey and Carter 2008). In accord with this, the gamma MEG response to point-light body movements is altered in patients with periventricular lesions that affect brain connectivity (Pavlova et al. 2007). Autistic individuals known to be impaired in visual social cognition, including perception of HS-like animations (Abell et al. 2000; Castelli et al. 2002; Campbell et al. 2006; Klin and Jones 2006; Boraston et al. 2007), exhibit signs of abnormal brain connectivity (Castelli et al. 2002; Barnea-Goraly et al. 2004). In the present work, we recorded whole-head MEG activity in healthy adults when they observed HS animations and judged whether the salient animations of 2 moving shapes represented social interaction.
Fourteen paid right-handed volunteers (8 females and 6 males), aged between 20 and 38 years (mean age 26.62 ± 4.78 years) with normal or corrected-to-normal vision, were enrolled in the study. None had a history of neurological or psychiatric disorders, head injuries, or medication for anxiety or depression. They were naive as to the purpose of the study and did not possess previous experience with such types of stimuli. Informed written consent was obtained in accordance with the requirements of the Ethical Committee of the University of Tübingen Medical School.
Each participant was presented with a set of 180 randomized stimuli of 2 types, salient HS-like animations, and control stimuli (90 stimuli of each type: 3 movies of each type [social and control] were randomly presented 30 times in a set). HS-like animations were created by modifying the movies used in a previous fMRI study (Schultz et al. 2003) and making them suitable for MEG recording. Each movie contained 2 geometric shapes (a white triangle and a circle of the same size and luminance) that moved irregularly against a black background (Fig. 1). Our previous pilot psychophysical study indicated that healthy adult participants describe these movies as representing social interaction with a high ranking of visual impression of social interaction (such as emotionality, intentionality, and interactivity) estimated on 7-point equal-spaced unipolar scales (Guerreschi et al. 2007). Because we were interested in subjective impression of visual interaction, several independent experts in visual perception that were naive as to the aim of the study determined a culmination point of the interaction between 2 moving shapes in the HS-like animations. Then, HS-like displays were constructed in such way that each display began 1.3 s before this culmination point and ended 0.7 s after this point. In all movies, therefore, despite differences in trajectories and accelerations, culmination of visual impression of social interaction (Fig. 1, about 40th frame) occurred at the same time point in respect to stimulus onset. Control displays represented modifications of the social movies for which we used parametric transformations for changing the motion trajectories to linear form. Each shape followed its own direction opposite to that of the other shape with a mean speed of both geometric shapes of the HS animations. Control movies were judged by healthy adults as representing nonsocial events and received low ranks for visual impression of social interaction in the previous psychophysical study (Guerreschi et al. 2007). Each display appeared for 2 s on a blank screen with an interstimulus interval that varied randomly between 3.5 and 5 s. Each dynamic event was accomplished in 60 frames with frame duration of about 33 ms. Participants were required to maintain their eyes in the center of the screen that was indicated by a gray fixation cross visible during the interstimulus interval. Each display subtended a visual angle of about 6° × 6°. In a yes–no paradigm, by pressing with the dominant right hand one of 2 keys whose positions were counterbalanced between subjects, participants had to determine whether a movie represented social interaction. No immediate feedback was given regarding performance.
A participant was seated in an electromagnetically shielded chamber (Vakuum-Schmelze GmbH, Hanau, Germany). The cortical responses were recorded with the Omega 275 CTF MEG system (VSM MedTech Ltd., Coquitlam, British Columbia, Canada) composed of 275 hardware first-order magnetic gradiometers with an average distance between sensors of 2.2 cm. The signals were recorded in DC mode with an antialiazing filter set at 100 Hz and a sampling rate of 585.9 Hz. Recorded epochs lasted from 0.3 s before to 2 s after the stimulus onset. Both at the beginning and at the end of each recording session, the participant’s head position was determined with 3 localization coils fixed at the nasion and the preauricular sites. All epochs of MEG activity were first automatically and then manually inspected for artifacts. Epochs containing blinks or eye movements (greater than ±100 μV) were rejected. The detection task obligates attention to all types of stimuli and reduces possible attention effects on recorded MEG traces. Each MEG recording session (during presentation of a set of 180 stimuli) lasted 15–20 min. The entire experimental session defined by the preparatory period, instruction, familiarization, and MEG recording took about 60 min. For each participant, the artifact-free MEG data sets were analyzed separately for correct responses (hits for HS animations and correct rejections for control stimuli). On average, participants made only a few incorrect responses (misses and false alarms), and these trials were excluded from further processing. To eliminate the influence of motor activity on recorded MEG traces, participants were asked to respond after the stimulus offset and to avoid responding during the stimulus presentation. If a participant responded to the display during presentation, the trial was discarded. After data inspection, on average a total of 87 ± 3.44 trials with artifact-free correct responses to HS-like animations (hits) and 84.79 ± 4.23 trials with correct responses to control displays (correct rejections) were submitted for further data processing.
MEG Data Analysis
The data analysis is described in detail elsewhere and followed a procedure that has been applied in a series of previous studies on induced MEG gamma responses (e.g., Lutzenberger et al. 2002; Kaiser et al. 2007). In brief, first, spectral analysis was performed to identify the frequency ranges with the most robust differences between the HS-like animations and control displays. Significance of the observed spectral power values for each frequency bin and MEG sensor was tested with a statistical probability mapping including corrections for multiple comparisons. Second, after filtering in the frequency ranges with the most pronounced differences between the HS-like animations and control displays, we determined the topography (sensors) and time courses of activations. Specifically, spectral analysis was conducted on a single-trial basis for frequencies up to 90 Hz. It was applied to 2 time windows, one from 0.7 to 1.3 s after stimulus onset and the other from 1.3 to 1.9 s, because we intended to analyze gamma activity well before the culmination point of the event and after this point. To reduce the frequency leakage for the different frequency bins, the records were multiplied by Welch windows. Then, fast Fourier transforms were carried out, and square roots of the power values were computed to obtain more normally distributed spectral amplitude values. These values were then averaged across epochs to obtain measures of the total spectral activity for each condition. Spectral activity contrasts were evaluated with a statistical probability mapping procedure that included corrections both for multiple comparisons and for possible correlations between data either from neighboring frequency bins (for spectral analysis) or from time points (for time-course analysis) (for details, see Kaiser et al. 2007). Significance criteria (corrected t values, tcorr) were determined on the basis of permutation tests. To explore the time course and the topographical localization of the observed spectral amplitude differences between HS animations and the control condition, the signals across the recording interval were multiplied with cosine windows at their beginnings and ends and filtered in the frequency ranges in which the statistical probability mapping had yielded significant effects. Noncausal Gaussian curves (shaped Gabor filters with a width of ±2.5 Hz) in the frequency domain were applied to the signals on a single-epoch basis for both conditions. The filtered data were amplitude demodulated by means of a Hilbert transformation and then averaged across epochs for each condition. Differences in amplitude between conditions in the filtered frequency band were assessed with the statistical probability mapping procedure. Surface gamma-band activity patterns observed with the present method of analysis have not suggested simple dipolar source structures. Although single dipole sources would produce 2 patches with strong magnetic fields; the single patches typically found in previous work could rather be attributed to a more complex structure of combined local sources (e.g., Pavlova et al. 2006). These multiple sources would generate a relatively weak field that is maximal over the area between the dipoles. According to the model (Kaiser et al. 2000), the cortical generators would have to be localized in the vicinity of the sensor showing the strongest activation. To depict the topographical localization of the observed differential spectral amplitude enhancements, we assigned the sensor positions with significant spectral amplitude effects of each subject to common spatial coordinates (common coil system). Sensor positions with respect to the underlying cortical areas were determined using a volumetric magnetic resonance image of one subject. The localization errors introduced by employing the common coil system were within the range of spatial resolution determined by the spacing of sensors in the MEG system.
Participants exhibited a ceiling level of performance as assessed by a sensitivity index d′ (range from 4.1 to 9.9, mean 7.9 ± 2.1), a standard measure of sensitivity in signal detection theory. Although there were only a few errors, participants tended to judge the displays as representing social interaction: The false alarm rate (judging control displays as representing social interaction) was higher than the miss rate (judging HS displays as nonsocial events; 0.009 ± 0.013 and 0.002 ± 0.004, for false alarms and misses, respectively; t13 = 2.27, one tailed, P < 0.04). Response time was significantly longer for HS-like animations than for control displays (0.522 ± 0.139 vs. 0.463 ± 0.136; t13 = 2.91, one tailed, P < 0.01).
Figure 2 shows the probability time trace of induced gamma cortical neuromagnetic activity for the sensors with significant differences in response to the HS animations compared with control displays. The HS displays elicited several subsequent peaks of induced gamma activity. A first peak of induced gamma oscillatory activity at a center frequency of 62 Hz was observed at 1 s from the stimulus onset over the right PTJ. Two further enhancements in the gamma MEG response of lower frequency of 44 Hz occurred at 1.4 s over the medial prefrontal and posterior temporal cortex in the right hemisphere. Subsequent boosts of 44 Hz were found over the left temporal and again over the right posterior temporal cortices at 1.6 s from the stimulus onset. Inspection of the time–frequency plot (Fig. 3) depicting the time course of statistical strength of differences in the spectral amplitude of induced oscillatory activity between the HS animations and control displays also confirms this outcome.
Social Brain Circuitry
This study was aimed at uncovering the temporal and topographical properties of the cortical network engaged in processing of visual information about social interaction and agency revealed by motion. Peaks in the induced gamma MEG response were found over the regions that engaged in the visual processing of dynamic social signals, in particular, over the PTJ and posterior temporal cortex in the right hemisphere (Figs. 2 and 3). The topography of this activation nicely dovetails with earlier fMRI and PET studies on visual social perception that made use of HS-like animations. These studies indicate an increased activation in several brain regions (more strongly in the right hemisphere), including the posterior part of the superior temporal sulcus (STS), PTJ, and the FFA in adults (Castelli et al. 2000; Martin and Weisberg 2003; Schultz et al. 2003; Schultz et al. 2004, 2005; Gobbini et al. 2007; Wheatley et al. 2007; Tavares et al. 2008) and in 10-year-old children (Ohnishi et al. 2004). Activation in the cerebellum (Ohnishi et al. 2004; Gobbini et al. 2007) and the amygdala (Castelli et al. 2000; Martin and Weisberg 2003; Wheatley et al. 2007; Tavares et al. 2008) is also reported. This agrees with a case study of a patient S.M. with bilateral damage to the amygdala, who was impaired in spontaneous anthropomorphizing of the HS animations (Heberlein and Adolphs 2004). Analysis of activation in the amygdala and cerebellum was beyond the scope of the present work. The present findings provide evidence in favor of, and further elaborate, the notion that gamma oscillatory activity is well topographically related to fMRI response (e.g., Niessing et al. 2005; Lachaux et al. 2007; Goense and Logothetis 2008; Muthukumaraswamy and Singh 2008). Most important, the advantage of the present MEG work is that, for the first time, it identifies time properties of the network underpinning visual social cognition and, therefore, goes far beyond the previous imaging findings restricted to localization of brain regions involved in visual perception of social interaction.
The Right Posterior Temporal Cortex
The other essential aspect is that the boosts of induced gamma activity occurred over the areas that are engaged in visual perception of expressive and meaningful bodily movements, especially over the right posterior temporal cortex (Allison et al. 2000; Grossman et al. 2000; Grossman and Blake 2002; Beauchamp et al. 2003; Pelphrey et al. 2003, 2005; Puce and Perrett 2003; Pavlova et al. 2004; Morris et al. 2005; Grèzes et al. 2007; Saygin 2007; Wyk et al. 2009). Recent fMRI work reveals that in the same participants, activation during processing of bodily motions overlaps topographically (especially, in the right pSTS) with the network engaged in visual perception of agency in HS animations (Gobbini et al. 2007). The activation over the posterior temporal cortex is also greater for goal-directed and intentional motions of geometric shapes (Castelli et al. 2000) as well as when one shape (a chaser) adopts a predict strategy (predicts end position of the other shape) rather than simply follows the target’s path (Schultz et al. 2004). Moreover, the blood oxygen level dependent (BOLD) signal in the pSTS increases with the amount of interactivity between 2 moving shapes (Schultz et al. 2005). The right posterior temporal cortex appears to be a key structure for processing of visual information about animacy, agency, and intentions of others revealed by movement and actions. It seems, however, that this region is involved in different brain networks subserving social cognition tasks. Comparison of temporal characteristics of the gamma oscillatory MEG response to point-light body motion and HS-like animations provides support for this view. Previous work indicates that in response to point-light human locomotion, the peak in oscillatory activity occurs at 0.17 s from the stimulus onset (Pavlova et al. 2004). In the present work, animations representing social interaction elicit the first gamma boosts over the posterior temporal cortex at nearly the same latency after the culmination point of the event (1.3 s from the stimulus onset; Fig. 2). In response to HS animations, however, we found a subsequent boost of gamma activity at about 0.3 s from the culmination point (at 1.6 s from the stimulus onset) that may reflect more complex processing of these animations. It appears, therefore, that this cortical area is engaged in processing of biological motion and HS animations with different temporal characteristics of gamma activity. This view is in line with the outcome of recent meta-analysis of fMRI findings in the right STS that emphasizes the role of network connectivity (Hein and Knight 2008).
In the present study, the first peak of gamma activity over the right PTJ occurred at a higher frequency of 62 Hz than subsequent peaks of 44 Hz over the posterior temporal cortex. The peaks of lower frequency suggest the involvement of larger scale networks (Kopell et al. 2000; Sokolov et al. 2004). Pathological models also imply that functional and structural connectivity to the right posterior temporal cortex is of particular value for proper functioning of the neural circuitry specialized for visual social cognition. Impairments in visual social cognition in autistic individuals, for example, are associated with abnormalities in structural and functional brain connectivity. High-functioning autistic patients have difficulties in interpreting HS animations (Abell et al. 2000; Castelli et al. 2002; Campbell et al. 2006; Klin and Jones 2006; Boraston et al. 2007). In this population compared with healthy controls, the right extrastriate regions show reduced functional connectivity to the STS (Castelli et al. 2002). Reduction in the gray matter in the posterior temporal cortex and abnormalities in white matter tracts between regions implicated in social cognition are also reported in autism (Barnea-Goraly et al. 2004; Zilbovicius et al. 2006; Skranes et al. 2007). In addition, visual social cognition is impaired in patients with periventricular lesions that affect brain connectivity, and the severity of this impairment is associated with the extent of lesions in the right temporal region (Pavlova et al. 2008). Periventricular lesions also affect gamma oscillatory MEG response to point-light body motion over the right parieto-temporal cortex (Pavlova et al. 2007). Taken together, the findings highlight the role of functional brain connectivity with the right temporal cortex for the networks engaged in visual social cognition.
The Right mPFC
The essential finding of the present work is the robust peak of the induced gamma response over the right mPFC. Previous brain imaging findings are controversial. Some studies suggest bilateral or right hemispheric dominant engagement of the mPFC in tasks with HS animations (Castelli et al. 2000, 2002; Martin and Weisberg 2003; Schultz et al. 2003; Ohnishi et al. 2004; Gobbini et al. 2007; Tavares et al. 2008). However, a patient G.T. with bilateral damage to the medial frontal lobe possesses intact impression of social attribution in HS animations (Bird et al. 2004). Performance of patients with damage to the orbitofrontal cortex is also intact on the HS tasks (Heberlein and Adolphs 2004). In accord with lesional data, some fMRI findings point to the lack of activation in the prefrontal cortex in response to HS displays (Blakemore et al. 2003; Schultz et al. 2004). Most interesting, in the same sample of participants, visual tasks with HS animations and verbal mentalizing theory of mind tasks (false belief stories) activate topographically distinct brain networks with different involvement of the right prefrontal cortex (Gobbini et al. 2007). Whereas HS animations elicit activation in a small locus of the right anterior paracingulate cortex (APC), false belief stories activate the left APC. Point-light articulating bodily movements, however, do not evoke any activation in the APC, suggesting involvement of this region in representation of social meaning of actions.
To the best of our knowledge, this is the first evidence to shed light on the time course and dynamic topography of the cortical neuromagnetic oscillatory gamma response to visual agency and social interaction represented by motion. The topography of peaks in gamma activity dovetails with previous fMRI data, and therefore, the findings support the view that gamma oscillatory activity is spatially well related to the BOLD signal. The work highlights the temporal relations between several cortical areas and, in particular, the key role of the right posterior temporal cortex for the networks engaged in visual perception of social agency and interaction revealed by motion. In general, the findings provide new insights into the proper functioning of the social brain circuitry.
Else Kröner-Fresenius-Foundation (P63/2008 to M.P.); University of Tübingen Medical School (fortüne-Program Grants 1576-0-0 and 1757-0-0 to M.P.). M.G. stay was supported by the University of Tübingen Medical School (fortüne-Program Grant 1576-0-0).
This work is dedicated to the memory of Werner Lutzenberger who passed away on November 22, 2008. We thank the participants for their kind cooperation, Jürgen Dax at the MEG-Center of the University of Tübingen Medical School for substantial technical assistance, and Alexander N. Sokolov and Arseny A. Sokolov for inspirations and stimulating discussions. M.G. was a PhD student in M.P.’s laboratory. Conflict of Interest: None declared.