Abstract

Recent evidence suggests an interaction between the ventral visual-perceptual and dorsal visuo-motor brain systems during the course of object recognition. However, the precise function of the dorsal stream for perception remains to be determined. The present study specified the functional contribution of the visuo-motor system to visual object recognition using functional magnetic resonance imaging and event-related potential (ERP) during action priming. Primes were movies showing hands performing an action with an object with the object being erased, followed by a manipulable target object, which either afforded a similar or a dissimilar action (congruent vs. incongruent condition). Participants had to recognize the target object within a picture–word matching task. Priming-related reductions of brain activity were found in frontal and parietal visuo-motor areas as well as in ventral regions including inferior and anterior temporal areas. Effective connectivity analyses suggested functional influences of parietal areas on anterior temporal areas. ERPs revealed priming-related source activity in visuo-motor regions at about 120 ms and later activity in the ventral stream at about 380 ms. Hence, rapidly initiated visuo-motor processes within the dorsal stream functionally contribute to visual object recognition in interaction with ventral stream processes dedicated to visual analysis and semantic integration.

Introduction

Recognizing a visual object necessarily depends on the analysis of the visual input and on activation of stored visual object representations (Biederman 1987; Humphreys et al. 1988; Grill-Spector and Malach 2004). At a functional neuroanatomical level, visual object recognition involves the ventral visual stream, which runs from primary visual cortex to the inferior portions of occipital and temporal cortex. In contrast, the dorsal visual stream, which also originates in primary visual cortex but continues to superior parietal cortex and premotor cortex (dorso-dorsal stream), is devoted to the preparation of object-directed action (e.g., Ungerleider and Mishkin 1982; Goodale and Milner 1992). Neural connections between both pathways may provide the anatomical basis for ventral–dorsal functional interactions as recognized in refinements of the original model (Pisella et al. 2006). Influences of ventral visual-perceptual processing on dorsal visuo-motor processing are documented in healthy populations (Creem and Proffitt 2001) and particularly in brain-damaged patients (damage to the ventro-dorsal stream) exhibiting different forms of ataxia (Himmelbach and Karnath 2005; Pisella et al. 2009).

It remains to be better elucidated, however, how dorsal visuo-motor processing influences putative ventral functions such as visual object recognition. For instance, it has been proposed that the dorsal pathway serves to focus attention to the relevant object thereby improving its perception (Pisella et al. 2009). It is also possible, however, that object-related visuo-motor processing in the dorsal pathway, for example, activating action affordances of an object, influences visual-perceptual processes in the ventral stream by facilitating the activation of the appropriate visual object representation (Helbig et al. 2006; Martin 2007; Kiefer et al. 2011).

Several lines of evidence suggest that visual object recognition may not only depend on the visual-perceptual system in the ventral pathway but also on the dorsal pathway: Perception of manipulable objects like tools elicited activity in premotor and parietal motor areas (Chao et al. 1999; Rumiati et al. 2004), although task requirements did not afford to act upon them (Grafton et al. 1997; Grèzes and Decety 2002; Grèzes et al. 2003; Caspers et al. 2010; Kiefer et al. 2011). These findings are compatible with an embodied view of conceptual representations, suggesting a grounding of concepts in the sensory–motor brain systems (Gallese and Lakoff 2005; Barsalou 2008; Pulvermüller and Fadiga 2010; Kiefer and Barsalou 2013). Sensory–motor activity during recognition of manipulable objects partially overlaps with motor activity during action observation and execution: When human and nonhuman primates watched or imagined actions of others, activity in motor cortex emerged similar to self-performed actions (Rizzolatti, Fadiga, Gallese, et al. 1996; Hari et al. 1998, 2010; Grèzes and Decety 2002; Buccino et al. 2004; Pihko et al. 2010). These findings, however, do not provide evidence for a functional role of action representations in visual object recognition because the observed motor activity might not be functionally relevant to recognition (Kiefer and Barsalou 2013).

Recently, a functional contribution of action representations to visual object recognition has been suggested by a series of behavioral and event-related potential (ERP) studies with an action priming paradigm: When the action indicated by a prime was congruent to the action afforded by a subsequently presented manipulable target object, target recognition was more accurate than in an incongruent condition (Helbig et al. 2006, 2010; Kiefer et al. 2011). However, these earlier studies cannot unequivocally reveal an involvement of visuo-motor areas during the course of visual object recognition.

The present study therefore specified the functional contribution of the visuo-motor system to visual object recognition during action priming with high spatial and temporal resolution using functional magnetic resonance imaging (fMRI) and ERP recordings. This combined measurement of fMRI and ERP allowed us to determine the temporo-spatial orchestration of activation in ventral and dorsal pathways during visual object recognition. In our action priming paradigm, primes were movies showing hands performing an action with an unseen object (see Helbig et al. 2010). The objects used to record the action movies were erased, always different from the target objects and carefully matched for potential confounding variables in order to ensure that the priming effect only depends on action similarity between action shown in the movie and the target object. Subsequently to the prime movie, a manipulable target object was presented, which either afforded a similar or a dissimilar action as the prime (congruent vs. incongruent condition). The target picture was followed by a basic-level name (e.g., “hammer”). Participants had to decide whether the target picture (e.g., hammer) matched with the word label or not (picture–word matching task). After the priming experiment, participants were shown the action movies in isolation to identify brain areas involved in action observation and visuo-motor processing. As outlined earlier, action observation reliably activates premotor and motor areas (Rizzolatti, Fadiga, Matelli et al. 1996; Hari et al. 1998, 2010; Grèzes and Decety 2002; Buccino et al. 2004).

We expected an action-priming effect in parietal and frontal visuo-motor areas, which are also activated by action observation. Effective connectivity analyses of the MR signal should indicate directional functional influences from dorsal visuo-motor areas on visual object processing in the ventral stream. An early onset of the action-priming effect in the ERP recordings within the first 200 ms of object processing would suggest that visuo-motor activity emerges early during the course of visual object recognition. Such a result pattern would specify the interaction between visual (ventral pathway) and visuo-motor representations (dorsal pathway) during object recognition.

Materials and Methods

General

All participants were right-handed native German speaking (Oldfield 1971) with normal or corrected-to-normal vision and without any history for neurological or psychiatric disorders. Twenty-one subjects (12 females, 9 males; mean age: 24.1 years) participated in the fMRI experiments (Experiment 1 and 2), another 20 subjects (13 females, 7 males; mean age: 23.67 years) in the ERP experiment (Experiment 3) after signing an informed written consent. Different 20 subjects were included in the fMRI and the ERP experiments to avoid stimulus repetition effects. The study was approved by the local Ethics Committee. All subjects were paid for their participation. Presentation of visual stimuli and behavioral data acquisition was controlled in the fMRI experiment by the Presentation software package (Neurobehavioral Systems, Inc., Albany, USA) and in the EEG experiment by the ERTS software package (Berisoft, Frankfurt, Germany). In the fMRI experiments, visual stimuli were delivered through MR-compatible video goggles (Resonance Technology, Los Angeles, CA, USA). In the ERP experiment, visual stimuli were presented on a computer screen.

Stimuli and Procedure of the Action Priming Paradigm (Experiments 1, fMRI, and 3, ERP)

Each trial consisted of a prime movie, followed by a visual target object and subsequently an object name that could match the target object or not (Fig. 1). As prime movies, 8 gray-scale video clips (512 × 768 pixels) were presented, each lasting 2000 ms (25 frames/s). The video clips were drawn from our previous behavioral study (Helbig et al. 2010) and showed hands performing an action with an unseen object in front of a black background (the object itself was always removed from the video). The videos were recorded using the MPI VideoLab (Kleiner et al. 2004). The actor wore black clothing and performed the action in interaction with real objects in order to ensure that the dynamics of the action were correct. The hands, which performed the action, were segmented from the unwanted parts of the scene (actor, object, and background). The following 8 action categories were presented as prime movies: 1) screwing with a screwdriver, 2) pounding with a hammer, 3) ironing with an electric iron, 4) typing on a computer keyboard, 5) rolling out with a rolling pin, 6) sweeping with a dustpan, 7) stapling with a stapler, and 8) carrying a toolbox. These 8 actions corresponded to the dominant action affordances of the target objects. The target objects consisted of 56 gray-scale photographs of familiar man-made manipulable objects. The objects were inscribed into a square of 280 × 280 pixels in order to equate the maximal extension. Picture size on the screen was about 10.3 × 10.3 cm (visual angle about 6.5° at a viewing distance of about 90 cm). The 8 prime movies and the 56 target objects were combined to form pairs for the congruent and incongruent priming conditions (Fig. 1): In the congruent condition, the prime movies were combined with several (3 up to 10, on average 7) target objects affording actions similar to the action shown in the movie. For example, the target objects scissors, nutcracker, and pliers typically involve an action similar to the prime action “stapling with a stapler” in that they all have a typical hand movement in common: closing the hand to compress the handles. In the incongruent condition, the target objects were randomly assigned to 1 of the 7 dissimilar prime actions such that each prime movie was combined with the same number of congruent and incongruent targets. This randomization was done once and was retained unchanged for all subjects to equate congruent and incongruent prime-target pairs for semantic similarity.

Figure 1.

Examples for pairs of prime movies and target objects affording congruent and incongruent typical actions.

Figure 1.

Examples for pairs of prime movies and target objects affording congruent and incongruent typical actions.

Several measures were taken to ensure that the priming effects induced by the movies are exclusively driven by action activation and not by object-related processes. First, the objects used to perform the action during movie recording were always different from the target objects, even in the congruent condition. Second, we confirmed that participants were not able to correctly infer the object used for recording the action movies. Eleven subjects who did not participate in the main experiments viewed the action movies and were requested to infer the invisible object, which was originally involved in performing the action. Each movie was presented twice, participants had unlimited time to respond while instruction stressed accuracy. Even under these very favorable conditions, which did not impose any time pressure in contrast to the main experiments, participants inferred the correct object from viewing the action movies only in 13.6% of the trials. Hence, participants did not associate a particular object with the observed actions. It is therefore unlikely that object-related factors such as visual similarity between a possibly imagined object during movie presentation and the target contributed to the priming effects in the main experiments. Finally, according to the data of 2 norming studies (Helbig et al. 2010), in which ratings were obtained with regard to action similarity and semantic similarity, it was ensured that action similarity between prime action and the action associated with the target object was significantly higher in the congruent than in the incongruent condition. Furthermore, semantic similarity between prime movie and target object did not differ across congruent and incongruent conditions in order to rule out a potential confound by global semantic similarity. The word labels indicated the names of the target objects at the basic level of abstraction (Rosch et al. 1976). Within the congruent trials, half were trials in which the target objects were combined with the word labels such that the word label correctly named the object (matching condition), and in the other half, word labels were completely unrelated to the target (mismatching condition). In sum, the 8 action prime videos were combined with the 56 target objects twice to create the congruent condition and the incongruent condition (56 congruent and 56 incongruent pairs). All targets were presented twice but were differently combined with the word label to yield the matching and nonmatching conditions. Altogether 4 blocks of 56 different stimulus pairs of the different action congruency and picture–word matching conditions were created. In the MR experiment, each subject received only 2 blocks of 56 trials with half of the possible picture–word pairings because the inclusion of null-events together with the subsequent action observation task (see below) rendered the fMRI experiment quite long. The assignment of blocks to participants was counterbalanced so that, across participants, all 4 blocks and thus all picture–word pairings were presented with equal frequency.

In both fMRI and ERP experiments, each action priming trial started with a fixation cross followed by the prime video that had to be attentively viewed and by a blank screen for 70 ms. Subsequently, the target object was displayed for 80 ms. As visuo-motor activation in the dorsal stream presumably decays fast (Goodale and Haffenden 2003), a short interval between action movie and target object was realized. This short movie-target interval has also the advantage to discourage participants to form expectations about the identity of the unseen object involved in recording the action sequence and about the identity of the upcoming target object (Neely 1991). The target object was replaced by a blank screen for 120 ms followed by the word label (250 ms). Participants were instructed to decide whether the word label matches the previously shown target picture and to indicate their response as fast and as accurately as possible by pressing 1 of 2 buttons with their index or middle finger (button assignment counterbalanced across observers). Participants had to respond within a time window of 2000 ms before the next trial started. After the response was recorded, the fixation cross reappeared. Trials of the different experimental conditions were presented in a randomized order. Before the main experiment, participants received 10 practice trials different from the experimental trials to be familiarized with the task.

In the fMRI experiment (Experiment 1), the fixation cross was presented for a fixed duration of 1000 ms and the mean intertrial interval was 6.1 s varying randomly between 2.0 and 10.3 s. Additionally, experimental trials were intermixed with trials in which just a black screen was shown (null events). Stimuli were presented within 2 blocks of 56 trials each including 14 trials per block per condition (plus 14 null events). In the ERP experiment (Experiment 3), trials were also presented in a randomized order, but participants initiated each trial with a button press in order to reduce ocular artifacts in the EEG recordings. The ERP experiment consisted of 4 blocks with 56 stimulus pairs per block.

Stimuli and Procedure of the Action Observation Task (Experiment 2, fMRI)

The action observation task was presented in the scanner after the action priming task. Stimuli were the 8 action movies used in the main experiments as primes. Action movies were presented in 3 blocks with duration of 110 s each (a mean inter-stimulus interval of 3.5 s randomly varying between 2 and 5 s), presented in randomized order. The action observation condition alternated with a fixation baseline condition with equal duration. The experimental session began with the first fixation baseline block. Participants were instructed to attentively watch the movies or the fixation symbol during the baseline condition.

fMRI Scanning and Data Analyses (Experiments 1 and 2)

Functional and structural MR images were recorded with a 3 Tesla Allegra MRI system (Siemens, Erlangen, Germany). For the functional scans, a T2*-weighted single-shot gradient-echo EPI sequence (TE = 38 ms, TR = 2000 ms, flip angle = 90°, matrix 64 × 64 pixels, field of view (FOV) 210 × 210 mm2, voxel size 3.3 × 3.3 × 4.9 mm3) was used. Starting from the bottom of the brain, 30 transversal slices were acquired in interleaved order. Slice orientation was parallel to a line connecting the bases of the frontal lobe and the cerebellum. There were 2 imaging runs for the fMRI Experiment 1, resulting in a total of 684 functional volumes. Run duration was 8 min. For the fMRI Experiment 2 (action observation), there was 1 imaging run of 12.83-min duration (385 functional volumes). Structural images were acquired with a T1-weighted MPRAGE sequence (TR = 2300 ms, TE = 3.9 ms, flip angle = 12°, matrix 256 × 256 pixels, FOV = 256 × 256 mm2, voxel size 1 × 1 × 1 mm3). Functional data preprocessing and statistical analyses were performed with SPM5 (http://www.fil.ion.ucl.ac.uk/spm/software/spm5).

For MR activation analysis, functional images were corrected for differences in slice-timing and head motion, spatially realigned to the mean volume of each session, normalized to the MNI reference brain (resampled voxel size: 2 × 2 × 2 mm3) and smoothed with an isotropic Gaussian kernel of 8-mm FWHM. A temporal high-pass filter with cutoff frequency of 1/128 Hz was applied to the data, and temporal autocorrelation in the fMRI time series was estimated (and corrected for) using a first-order autoregressive model. Statistical analysis used a hierarchical random-effects model with 2 levels. At the first level, single-subject fMRI responses were modeled by a design matrix comprising the stimuli of the experimental conditions (Experiment 1: action congruency (congruent vs. incongruent) and type of matching (matching vs. mismatching; Experiment 2: blocks of action movies) as well as the 6 motion parameters estimated from the realignment procedure. Error trials were modeled as regressor of no interest. All these regressors were convolved with the canonical hemodynamic response function. To allow for inferences at the population level, a second-level analysis (full factorial model) considered the contrast images of all subjects and treated subjects as a random effect. In the event-related analysis of Experiment 1, 4 contrast images per subjects were submitted to a 2 × 2 ANOVA with the factors action congruency (congruent vs. incongruent) and matching (matching vs. mismatching). As our main interest rested in the action-priming effect, comparison of the congruent versus incongruent condition was restricted to trials on which picture–word label pairing was matching condition because processes underlying performance in mismatching picture–word trials are less constrained than in the matching condition. In the analysis of the block design of Experiment 2, contrast images of the action observation condition (against baseline) were subjected to a one-sample t-test. All reported comparisons for Experiment 1 were thresholded at a significance level of P < 0.001 and corrected for multiple comparisons across the entire brain (false discovery rate [FDR]). For the action localizer Experiment 2, significance level was set to P < 0.05 (FDR corrected for the whole brain) in order to capture brain activity involved in action observation to a great extent. The spatial extent threshold of clusters was 184 voxels in all reported comparisons. Anatomic labels and corresponding Brodmann areas (BAs) were determined using MRIcro (Rorden and Brett 2000). All functional group activation maps were overlaid on the MNI reference brain. Neural priming effects at the peak voxels of the clusters, which exceeded the extent threshold, were related to behavioral priming effects using Pearson's product moment correlations. As a score for behavioral priming, we used the error rate (ER) difference incongruent minus congruent in the matching trials in each subject as in the analysis of the behavioral data. Analyses were restricted to ER because behavioral priming effects were larger for ER than for reaction times (RT) (see the Results section). As a measure of neural priming, we extracted beta estimates from each peak voxel of the clusters obtained for the contrast incongruent versus congruent (see also Table 1).

Table 1

Activation peaks for main contrasts (A) incongruent versus congruent and (B) congruent versus incongruent, whole-brain analysis (significant at voxel or cluster level, P (FDR corrected) < 0.001 corrected, cluster size > 184)

Activations
 
Brain region BA MNI coordinates (mm) t P (FDR corrected) Cluster size (voxels) P (cluster corrected) 
Incongruent versus congruent 
 Temporal Pole Sup38 −56, 14, −4 7.78 <0.001 5682 <0.001 
 Temporal superior L 22 −56, −26, 12 7.66 <0.001   
 Insula L 48 −42, −12, 4 7.64 <0.001   
 Hippocampus R 37 38, −48, −2 7.73 <0.001 9096 <0.001 
 Supplementary motor area L −2, 14, 70 7.31 <0.001   
 Superior temporal R 48 44, −8, −10 6.99 <0.001   
Superior parietal−30, −44, 70 7.32 <0.001 1778 <0.001 
Precentral−28, −12, 70 6.81 <0.001   
 Postcentral L −52, −32, 56 6.42 <0.001   
 Inferior parietal R 39 52, −58, 46 6.25 <0.001 898 <0.001 
 Supramarginal R 40 54, −32, 44 4.49 <0.001   
 Parahippocampal R 35 18, −20, −20 5.64 <0.001 189 0.005 
 Cerebellum R  16, −30, −30 4.49 <0.001   
 Cerebellum R  14, −68, −50 5.30 <0.001 206 0.003 
 Cerebellum R  24, −58, −50 4.84 <0.001   
 Occipital middle 18 0, −88, 26 4.80 <0.001 290 0.001 
Occipital inferior19 4, −90, 34 4.48 <0.001   
Congruent versus incongruent 
 Occipital inferior R 19 36, −82, −6 9.90 <0.001 5929 <0.001 
 Occipital inferior R 19 28, −84, −10 9.38 <0.001   
 Occipital middle R 18 14, −96, −6 8.53 <0.001   
 Occipital inferior L 18 −28, −88, −10 8.63 <0.001 3517 <0.001 
 Occipital middle _L. 19 −46, −74, 2 7.50 <0.001   
 Occipital middle L 18 −28, −88, 14 7.12 <0.001   
 Hippocampus R 37 22, −26, −4 7.31 <0.001 204 0.001 
 Hippocampus R 37 8, −26, −8 5.56 <0.001   
 Precentral R 46, 0, 46 6.90 <0.001 548  
 Superior parietal R 30, −54, 56 6.26 <0.001 415 <0.001 
 Inferior parietal R 40 34, −34, 48 4.66 <0.001  <0.001 
Activations
 
Brain region BA MNI coordinates (mm) t P (FDR corrected) Cluster size (voxels) P (cluster corrected) 
Incongruent versus congruent 
 Temporal Pole Sup38 −56, 14, −4 7.78 <0.001 5682 <0.001 
 Temporal superior L 22 −56, −26, 12 7.66 <0.001   
 Insula L 48 −42, −12, 4 7.64 <0.001   
 Hippocampus R 37 38, −48, −2 7.73 <0.001 9096 <0.001 
 Supplementary motor area L −2, 14, 70 7.31 <0.001   
 Superior temporal R 48 44, −8, −10 6.99 <0.001   
Superior parietal−30, −44, 70 7.32 <0.001 1778 <0.001 
Precentral−28, −12, 70 6.81 <0.001   
 Postcentral L −52, −32, 56 6.42 <0.001   
 Inferior parietal R 39 52, −58, 46 6.25 <0.001 898 <0.001 
 Supramarginal R 40 54, −32, 44 4.49 <0.001   
 Parahippocampal R 35 18, −20, −20 5.64 <0.001 189 0.005 
 Cerebellum R  16, −30, −30 4.49 <0.001   
 Cerebellum R  14, −68, −50 5.30 <0.001 206 0.003 
 Cerebellum R  24, −58, −50 4.84 <0.001   
 Occipital middle 18 0, −88, 26 4.80 <0.001 290 0.001 
Occipital inferior19 4, −90, 34 4.48 <0.001   
Congruent versus incongruent 
 Occipital inferior R 19 36, −82, −6 9.90 <0.001 5929 <0.001 
 Occipital inferior R 19 28, −84, −10 9.38 <0.001   
 Occipital middle R 18 14, −96, −6 8.53 <0.001   
 Occipital inferior L 18 −28, −88, −10 8.63 <0.001 3517 <0.001 
 Occipital middle _L. 19 −46, −74, 2 7.50 <0.001   
 Occipital middle L 18 −28, −88, 14 7.12 <0.001   
 Hippocampus R 37 22, −26, −4 7.31 <0.001 204 0.001 
 Hippocampus R 37 8, −26, −8 5.56 <0.001   
 Precentral R 46, 0, 46 6.90 <0.001 548  
 Superior parietal R 30, −54, 56 6.26 <0.001 415 <0.001 
 Inferior parietal R 40 34, −34, 48 4.66 <0.001  <0.001 

Note: The 4 VOIs selected for effective connectivity analysis with DCM are written in italics.

In order to infer directed connectivity among brain regions implicated in the action-priming effect, effective connectivity analyses of Experiment 1 were performed using dynamic causal modeling (DCM) implemented in SPM 5 (Friston et al. 2003; Stephan et al. 2007). Forty-two subject-specific DCMs were constructed. Four brain regions identified in the second-level analysis of the activation data to be involved in the action congruency effect (right inferior occipital [BA 19], left superior parietal [BA 7], left precentral area [BA 6], and left temporal Pole [BA 38]) were selected to extract times series for volumes of interest (VOIs) (see Table 1). These VOIs included 2 ventral and 2 dorsal areas, 1 posterior area, and 1 anterior area. In order to keep the complexity of the DCMs at a reasonable level, we had to restrict these analyses to representative dorsal and ventral areas and could not include the full set of activated regions involved in action priming. In each subject, VOIs were centered around the individual local maximum (P < 0.05 uncorrected) closest to the peak coordinates of the group analysis (Mechelli et al. 2005) within a spherical search volume of 8-mm radius using the contrast that recombined the regressors of all conditions of interest, separately for each session. Based on the extraction of fMRI signals in the VOIs described earlier, we tested 4 competing hypotheses about this system of interest (Fig. 3A). Four DCMs were created to test whether areas of the dorsal pathway functionally interact with ventral stream areas during object recognition. In these DCMs, only different intrinsic connections were considered because it is unlikely that effective connectivity between visual areas during object recognition is modulated by action congruency. Furthermore, as DCMs had to be calculated session specific, inclusion of the factor congruency would be based on a very limited amount of data and consequently result in poor model estimation. The models differed with regard to the kind and number of dorsal–ventral influences in addition to the forward connections within both streams. Analyses were restricted to these 4 alternative models including up to 2 dorsal–ventral connections based on a pilot analysis, which suggested that inclusion of more dorsal–ventral connections resulted in a poorer model fit. In models 1–4, we systematically varied the precise locus of the possible dorsal influence on ventral stream processing. Dorsal–ventral interactions were modeled as an influence of the superior parietal gyrus on the inferior occipital cortex in model 1, or as an influence of the precentral areas on the temporal pole in model 2. In model 3, an influence of the superior parietal gyrus on the temporal pole was assumed. In model 4, both an influence of the superior parietal gyrus on the inferior occipital cortex and an influence of the superior parietal gyrus on the temporal pole were tested. After specifying and estimating these models for each subject and each session separately, Baysian model selection (BMS) (Penny et al. 2004) was used to identify the DCM that optimally explains effective connectivity between the modeled dorsal and ventral streams areas during visual object recognition. Model selection was carried out as follows (Heim et al. 2009): First, the Bayes factor (BF) from the pair-wise comparisons of the 4 models for each session and each subject were calculated. Based on these values, the group Bayes factor (GBF) was calculated by multiplying all subject-specific BFs (Smith et al. 2006; Stephan and Penny 2007). Furthermore, a positive evidence ratio (PER) (Stephan and Penny 2007) was determined, which is defined as the ratio of the number of subjects for which there is positive evidence (BF > 3) for 1 model divided by the number of subjects, in which the other model is favored. After selecting the optimal model based on these comparisons, parameters for the inputs and intrinsic connections were averaged across sessions in each subject. For statistical inferences at the group level, these individual parameter measures were tested against zero using one-sample t-tests, corrected for multiple comparisons.

ERP Recordings and Data Analysis (Experiment 3)

ERP recordings were performed in a dimly lit, sound-attenuated, electrically shielded booth. Scalp potentials were collected using an equidistant montage of 64 sintered Ag/AgCl electrodes mounted in an elastic cap (Easy Cap, Herrsching, Germany). An electrode between FPz and Fz was connected to the ground, and an electrode between Cz and FCz was used as recording reference. Eye movements were monitored with supra- and infra-orbital electrodes and with electrodes on the external canthi. Electrode impedance was kept below 5 kΩ. Electrical signals were amplified with BrainAmp (Brain Products, Gilching, Germany) amplifiers (low-pass filter: 70 Hz, 24 dB/octave attenuation; 50 Hz notch filter) and continuously recorded (digitization rate: 500 Hz), digitally bandpass filtered (high cutoff: 16 Hz, 24 dB/octave attenuation; low cutoff: 0.1 Hz, 12 dB/octave attenuation) and segmented (152 ms before to 1000 ms after the onset of the picture stimulus). Ocular artifacts were removed using independent component analysis (Makeig et al. 1997). Artifact-free EEG segments to trials with correct responses were averaged separately for each experimental condition and electrode synchronous to the onset of the stimulus. The resulting individual ERPs were referenced to average reference (BrainVision Analyzer, Brain Products, Gilching, Germany).

Statistical analysis of the ERP data focused on 2 scalp regions of interest in accordance with earlier studies: A fronto-central electrode cluster close to motor areas, in which ERP effects in response to action-related processing were frequently observed (e.g., Pulvermüller 2005; Kiefer et al. 2011), a parietal electrode cluster, in which the N400 ERP component, an electrophysiological index of semantic integration processes, is typically recorded (e.g., Kutas and Hillyard 1980; Bentin et al. 1985; Kiefer 2001; Kiefer et al. 2011). In each scalp region of interest, 4 pairs of contralateral electrodes were selected for statistical analysis: fronto-central: C3/C4, FC3/FC4, F1/F2, C1/C2; parietal: P3/P4, P1/P2, CP3/CP4, CP1/CP2. As action-priming effects previously emerged in both an early P1 time window and a later N400 time window, statistical analysis of the ERP data was performed in these 2 time windows (Kiefer et al. 2011): P1 time window, 120–150 ms after picture stimulus onset; N400 time window, 380–480 ms after picture stimulus onset. Repeated measures analyses of variance (ANOVAs) were performed on mean voltages in each time window with congruency (congruent vs. incongruent), hemisphere (left vs. right), and electrode site (4 positions within the region of interest) as within-subjects factors. Separate analyses were conducted for each time window and scalp region. In order to account for possible violations of the sphericity assumption of the repeated measures ANOVA model, degrees of freedom were adjusted according to the method of Huynh and Feldt, 1970, and the Huynh–Feldt ε as well as the corrected significance levels is reported, when appropriate. Only the corrected significance levels (P > 0.05) were reported. Similar to the fMRI experiment, ERP priming effects (ERP difference congruent minus incongruent) were related to behavioral priming effects (ER) in a correlation analysis.

We determined the neural sources for significant effects of congruency (congruent vs. incongruent) using distributed source modeling (minimum norm source estimates) (Hauk 2004) implemented in BESA 5.1 (MEGIS). This method yields the unique solution that explains the data and does not contain components that are “silent”, that is, do not produce any measurable surface signal by themselves. Sources were computed for the grand-averaged ERP difference waves between congruent and incongruent stimuli to eliminate unspecific brain activity associated with stimulus processing. Minimum norm source estimates (minimum L2 norm) were calculated using a standardized realistic head model (finite element model [FEM]). The pre-target baseline was used to estimate the noise regularization parameters. Minimum norm was computed with depth weighting, spatio-temporal weighting, and noise weighting for individual channels. Depth weighting reduces source localization bias toward superficial currents due to an attenuation of EEG lead fields with increasing source depth (Lin et al. 2006). Spatio-temporal weighting gives larger weight to sources more likely assumed to contribute to the recorded data based on the signal subspace correlation measure (Mosher and Leahy 1998). Cortical currents were determined within the time window during which significant ERP differences were obtained at the time point of maximal global field power (GFP) in the ERP difference waves to ensure optimal signal-to-noise-ratio (Kiefer et al. 2007). Talairach coordinates for the activation peaks were determined on the 2D surface covering the cortex on which the source solution was computed. We report the nearest BAs to the peak activations located by the Talairach Daemon (Lancaster et al. 2000).

Results

fMRI Experiment 1 (Action Priming)

Behavioral Data

Analysis of the behavioral data of the primed picture–word matching task revealed an action-priming effect on ER. Error rate was significantly higher in the incongruent (9.7%) than that in the congruent condition (6.5%; t(20) = 3.016, P = 0.0068, one-tailed paired-sample t test). This result shows that action observation facilitates visual recognition of objects affording similar actions and suggests a functional role of action representation in visual object recognition. Analysis of RT did not yield significant RT differences between congruent and incongruent conditions, but reactions were numerically faster in congruent (633.31 ms) than in incongruent (642. 23 ms) trials (t(20) = 1.155, P = 0.26, one-tailed paired-sample t test). Hence, as in previous studies (Helbig et al. 2006; Kiefer et al. 2011), the action-Priming effect was more pronounced for ER than for RT data.

FMRI Activation Data

As priming is typically reflected by an attenuated brain activity in the primed condition compared with the unprimed condition due to more efficient processing, we first identified brain areas with greater activity in the incongruent (unprimed) than in the congruent (primed) condition (Table 1). We found decreased activity as a function of action priming in frontal and parietal motor areas. The MR signal was reduced in the left pre- and post-central gyri encompassing the primary sensory–motor cortex (BA 1–4), in inferior frontal gyrus (BA 44) that is a part of the premotor cortex, in adjacent portions of the insula (BA 48) and in the SMA (BA 6) (Fig. 2A, left). Priming-related reduction of brain activity was also found in parietal areas including the left superior parietal gyrus (BA 7), left inferior parietal gyrus (BA 39) as well as right supramarginal gyrus (BA 40).

Figure 2.

Functional brain activation during the action priming task. (A) Activation in the incongruent versus congruent comparison (left) and activation in the congruent versus incongruent comparison (right). P (FDR-corrected) <0.001; voxel size >184 for both conditions. (B) Surface-rendered projections of statistical maps of contrasts in the whole-brain analysis. The red areas illustrate the activation in the incongruent versus congruent condition. The blue areas denote the activation in action observation. The yellow areas illustrate the overlapping activation in both tasks. For illustration purposes, statistical maps were thresholds at P (FDR-corrected) < 0.05, voxel size >184. PreCG, precentral gyrus; PoCG, postcentral gyrus; IFG, inferior frontal gyrus; STG, superior temporal gyrus; TP, temporal pole; SMG, supramarginal gyrus; MTG, middle temporal gyrus; Hippo, hippocampus; Parahippo, parahippocampus; PMC, premotor cortex; SPL, superior parietal lobule; OG, occipital gyrus; PMC, premotor cortex; FG, fusiform gyrus.

Figure 2.

Functional brain activation during the action priming task. (A) Activation in the incongruent versus congruent comparison (left) and activation in the congruent versus incongruent comparison (right). P (FDR-corrected) <0.001; voxel size >184 for both conditions. (B) Surface-rendered projections of statistical maps of contrasts in the whole-brain analysis. The red areas illustrate the activation in the incongruent versus congruent condition. The blue areas denote the activation in action observation. The yellow areas illustrate the overlapping activation in both tasks. For illustration purposes, statistical maps were thresholds at P (FDR-corrected) < 0.05, voxel size >184. PreCG, precentral gyrus; PoCG, postcentral gyrus; IFG, inferior frontal gyrus; STG, superior temporal gyrus; TP, temporal pole; SMG, supramarginal gyrus; MTG, middle temporal gyrus; Hippo, hippocampus; Parahippo, parahippocampus; PMC, premotor cortex; SPL, superior parietal lobule; OG, occipital gyrus; PMC, premotor cortex; FG, fusiform gyrus.

Outside the visuo-motor system, action priming was associated with reduced activity in lower level (middle and inferior occipital gyri, BA 18/19) and higher level visual areas (inferior temporal gyrus, BA 20). Finally, priming-related attenuation of brain activity was obtained in areas known to support semantic integration: The superior temporal gyrus (BA 22) extending to the temporal pole (BA 38) showed a MR signal decrease as a function of action priming. When relating behavioral action priming on ER to neural priming effects at the peak voxels in the above-mentioned clusters, we found a significant correlation between brain activity and behavior in the postcentral gyrus (r = 0.496, P < 0.05).

Although action priming predominantly resulted in a decrease of the MR signal, a few areas increased activity as a function of priming (Table 1). Greater activity in the congruent than in the incongruent condition was mainly observed in a large cluster encompassing inferior und middle occipital cortex (BA 18/19) as well as middle temporal cortex (BA 21) bilaterally. Additional right premotor cortex (BA6) and right superior parietal cortex (BA 7) showed a signal increase as a function of priming (Fig. 2A, right).

FMRI Effective Connectivity

After comparing adequateness of the models on the basis of GBF and PER criteria, model 3 was selected because it was superior to the 3 alternative models (Table 2). This model includes feedforward connections from inferior occipital cortex dorsally to superior parietal cortex and further on to precentral cortex and ventrally form inferior occipital cortex to the temporal pole. In addition, this model assumes influences of the dorsal stream on the ventral stream though connections between superior parietal cortex (dorsal stream) and the temporal pole (ventral stream). Statistical analysis of the model parameters showed that the driving input as well as all intrinsic connections including the influence of superior parietal cortex on temporal pole activity were significantly greater than zero (Fig. 3B, see also Table 3).

Table 2

PER and GBF for the comparison of models 1–4

Session 3 versus 1 3 versus 2 3 versus 4 2 versus 1 1 versus 4 2 versus 4 
Comparison of models 1–4 
 1. 6:1
2.2740E + 018 
5:2
7.5490E − 001 
8:1
3.4399E + 007 
4:0
3.0193 + 018 
1:3
1.8353E − 008 
5:0
7.3551E + 008 
 2. 4:1
1.9932E + 003 
1:1
4.0781E + 002 
7:0
6.3322E + 007 
1:1
2.0440E − 001 
1:1
4.8893E + 000 
3:0
5.1463E + 005 
Session 3 versus 1 3 versus 2 3 versus 4 2 versus 1 1 versus 4 2 versus 4 
Comparison of models 1–4 
 1. 6:1
2.2740E + 018 
5:2
7.5490E − 001 
8:1
3.4399E + 007 
4:0
3.0193 + 018 
1:3
1.8353E − 008 
5:0
7.3551E + 008 
 2. 4:1
1.9932E + 003 
1:1
4.0781E + 002 
7:0
6.3322E + 007 
1:1
2.0440E − 001 
1:1
4.8893E + 000 
3:0
5.1463E + 005 

PER—normal print; GBF—cursive print.

Table 3

Parameters of the selected DCM model, including inputs and intrinsic connections (statistical significance at P = 0.001, Bonferroni-corrected for multiple comparisons)

 Mean SD T20 P 
Parameters of the selected DCM model 
Intrinsic connections 
 iOcc → SPG 0.434 0.29 6.86 <0.001 
 iOcc → TP 0.257 0.16 7.34 <0.001 
 SPG → PreCe 0.526 0.20 12.26 <0.001 
 SPG → TP 0.304 0.29 4.79 <0.001 
 TP → PreCe 0.382 0.26 6.76 <0.001 
Driving inputs 
 iOcc 0.184 0.14 6.06 <0.001 
 Mean SD T20 P 
Parameters of the selected DCM model 
Intrinsic connections 
 iOcc → SPG 0.434 0.29 6.86 <0.001 
 iOcc → TP 0.257 0.16 7.34 <0.001 
 SPG → PreCe 0.526 0.20 12.26 <0.001 
 SPG → TP 0.304 0.29 4.79 <0.001 
 TP → PreCe 0.382 0.26 6.76 <0.001 
Driving inputs 
 iOcc 0.184 0.14 6.06 <0.001 

iOcc, inferior occipital cortex; SPG, superior parietal gyrus; PreCe, precentral gyrus; TP, temporal pole.

Figure 3.

Effectivity analyses with DCM. (A) Schematic representation of the 4 tested alternative models. Gray filed circles depict the brain regions (iOcc, inferior occipital cortex; SPG, superior parietal gyrus; PreCG, precentral gyrus; TP, temporal pole) involved in the model network. Arrows between the regions indicate the direction of intrinsic connections. Gray rectangles depict the inputs of stimuli into the right inferior occipital region (iOcci). (B) Average parameter estimates of the intrinsic connections in the selected model (model 3). All parameters were significantly larger than zero including the connection between superior parietal gyrus and the temporal pole.

Figure 3.

Effectivity analyses with DCM. (A) Schematic representation of the 4 tested alternative models. Gray filed circles depict the brain regions (iOcc, inferior occipital cortex; SPG, superior parietal gyrus; PreCG, precentral gyrus; TP, temporal pole) involved in the model network. Arrows between the regions indicate the direction of intrinsic connections. Gray rectangles depict the inputs of stimuli into the right inferior occipital region (iOcci). (B) Average parameter estimates of the intrinsic connections in the selected model (model 3). All parameters were significantly larger than zero including the connection between superior parietal gyrus and the temporal pole.

fMRI Experiment 2 (Action Observation Localizer)

Action observation elicited increased activity bilaterally in occipital cortex (BA 18/19), in posterior inferior temporal gyrus (BA 37) and in middle temporal gyrus (MTG) although activation peaks were located in the right hemisphere (Table 4). This large cluster also encompassed activation in the left inferior parietal gyrus (BA 40) extending to the postcentral gyrus (BA 3). Action observation was also associated with activation in a large cluster of voxels encompassing left inferior frontal gyrus (BA 44/45) as well as left middle and superior frontal gyri (BA 6/8). Increased activation was also found in the hippocampus (BA 30/37) bilaterally. The activity pattern associated with action observation partially overlapped with action priming-related activity reduction in visual and visuo-motor regions: middle occipital gyrus, inferior and middle temporal gyri, postcentral gyrus, inferior frontal gyrus and inferior parietal gyrus (see Fig. 2B).

Table 4

Activation peaks for the contrast action observation versus fixation, whole-brain analysis (significant at voxel or cluster level, P (FDR) < 0.05 corrected, cluster size > 184)

Activations
 
Brain region BA MNI coordinates (mm) T P (FDR corrected) Cluster size (voxels) P (cluster corrected) 
Temporal middle R 37 54, −68, 0 14.34 <0.001 25 573 <0.001 
Occipital inferior R 18/19 44, −82, −2 10.58 <0.001   
Temporal inferior R 37 54, −64, −12 10.47 <0.001   
Frontal inferior opercular L 48/44 −52, 16, 2 7.00 <0.001 7993 <0.001 
Frontal inferior triangular L 45 −52, 36, 14 6.20 <0.001   
Frontal inferior orbito L 47 −46, 36, −14 6.11 <0.001   
Putamen L  −28, −2, 10 4.66 <0.001 1336 0.004 
Hippocampus L  −28, −30, −8 4.15 <0.001   
Hippocampus L  −22, −22, −14 4.11 <0.001   
Activations
 
Brain region BA MNI coordinates (mm) T P (FDR corrected) Cluster size (voxels) P (cluster corrected) 
Temporal middle R 37 54, −68, 0 14.34 <0.001 25 573 <0.001 
Occipital inferior R 18/19 44, −82, −2 10.58 <0.001   
Temporal inferior R 37 54, −64, −12 10.47 <0.001   
Frontal inferior opercular L 48/44 −52, 16, 2 7.00 <0.001 7993 <0.001 
Frontal inferior triangular L 45 −52, 36, 14 6.20 <0.001   
Frontal inferior orbito L 47 −46, 36, −14 6.11 <0.001   
Putamen L  −28, −2, 10 4.66 <0.001 1336 0.004 
Hippocampus L  −28, −30, −8 4.15 <0.001   
Hippocampus L  −22, −22, −14 4.11 <0.001   

BA, Brodmann area; Coordinates (mm): MNI, Montreal Neurological Institute; P (FDR corrected), significance level, voxel-level correction; P (cluster corrected), significance level, cluster-level correction; k, cluster extent (in voxels of size 2 × 2 × 2 mm).

ERP Experiment 3 (Action Priming)

Analyses of the behavioral data did not reveal significant differences between the congruent and incongruent conditions, although direction of the mean differences for both ER (congruent: 5.89%; incongruent: 6.25%; t(19) = −0.444, P = 0.662; one-tailed paired-sample t test) and RT was comparable with the fMRI Experiment 1 (congruent: 611.82 ms; incongruent: 614.64 ms; t(19) = −0.669, P = 0.51; one-tailed paired-sample t test).

Analysis of the ERP data (Fig. 4) showed in the early time window between 120 and 150 ms after target picture onset (P1 time window) an action-priming effect over the fronto-central scalp: ERPs were more negative in the congruent than in the incongruent condition (congruent: −1.62 μV; incongruent: −1.29 μV; F1,19 = 4.50, P = 0.047). P1 amplitude over the parietal scalp region did not differ between the congruent and incongruent conditions in this time window (congruent: 0.73 μV; incongruent: 0.75 μV; F1,19 = 0.014, P = 0.91). In the later time window between 380 and 480 ms after target stimulus onset (N400 time window), significant ERP action-priming effects were observed both over fronto-central [F1,19 = 4.96, P = 0.039] and parietal scalp regions [F1,19 = 5.79, P = 0.027]: ERPs in the incongruent condition were more negative than in the congruent condition at both fronto-central (congruent: −1.30 μV; incongruent: −1.58 μV) and parietal electrodes (congruent: 3.57 μV; incongruent: 3.08 μV). The action-priming effect was more pronounced at parietal than fronto-central electrodes (Fig. 4). ERP priming effects were not significantly correlated with behavioral priming (ER), presumably due to the small effect sizes in both measures.

Figure 4.

Grand-averaged (n = 20) ERPs as a function of action congruency. (A) Shown are ERP waveforms at selected electrodes and topographic maps of the early (P1: 120–150 ms) and late (N400: 380–480 ms) action-priming effects. In this and the upcoming figures, negative potentials are plotted downward. The y-axes indicate the onset of the target. (B) Topography of the action-priming effects in the P1 and N400 time window. Arrows (ERP plots) index significant action-priming effects.

Figure 4.

Grand-averaged (n = 20) ERPs as a function of action congruency. (A) Shown are ERP waveforms at selected electrodes and topographic maps of the early (P1: 120–150 ms) and late (N400: 380–480 ms) action-priming effects. In this and the upcoming figures, negative potentials are plotted downward. The y-axes indicate the onset of the target. (B) Topography of the action-priming effects in the P1 and N400 time window. Arrows (ERP plots) index significant action-priming effects.

Source analyses of the significant action priming ERP effects using minimum norm estimation (Hauk 2004) identified for the early P1 time window (120–150 ms) generators within a large fronto-parietal visuo-motor network. Local maxima were found in motor and premotor cortex (BA 1/2/4/6) as well as in the inferior frontal (BA 44) and parietal gyri (BA 40/43), predominantly of the left hemisphere (Fig. 5). In the later N400 time window (380–480 ms), the action-priming effects were referred to generators in occipital areas (BA 17/18/19), in posterior middle and superior temporal gyri (BA 21/22), as well as in motor (BA 1/2/3) and premotor areas (BA 4/6).

Figure 5.

Sources of significant ERP action-priming effects. Maps of estimated cortical currents were calculated according to the minimum norm algorithm from grand-mean scalp ERPs. Shown are cortical sources for the difference waves of the incongruent minus congruent subtraction in the P1 time window (120–150 ms) for the pictorial prime condition and in the N400 (380–480 ms) time window for the combined pictorial and verbal prime conditions. Maps are shown for the respective maxima in GFP.

Figure 5.

Sources of significant ERP action-priming effects. Maps of estimated cortical currents were calculated according to the minimum norm algorithm from grand-mean scalp ERPs. Shown are cortical sources for the difference waves of the incongruent minus congruent subtraction in the P1 time window (120–150 ms) for the pictorial prime condition and in the N400 (380–480 ms) time window for the combined pictorial and verbal prime conditions. Maps are shown for the respective maxima in GFP.

Discussion

The present combined ERP/fMRI study elucidated the functional contribution of the visuo-motor system to visual object recognition using an action priming paradigm. We assessed whether observation of an action facilitates subsequent recognition of a manipulable artifact object and determined the spatio-temporal dynamics of brain activation underlying this action-priming effect. In particular, we wanted to specify the interaction between the dorsal visual stream dedicated to visuo-motor preparation and the ventral visual stream involved in perceptual stimulus analysis during the course of visual object recognition.

Object recognition accuracy in the picture–word matching task was higher for objects that afforded actions similar to the prime movies (congruent condition) compared with dissimilar ones (incongruent condition). In line with earlier work (Helbig et al. 2006; Helbig et al. 2010; Kiefer et al. 2011), this action-priming effect suggests a functional role of action representations in visual object recognition (Almeida et al. 2008; Sakuraba et al. 2012). The priming effect was more pronounced in the fMRI experiment than in the ERP experiment, presumably because in the latter, prime-target pairings were presented twice so that stimulus repetition may have reduced the magnitude of the priming effect (Kiefer 2005; Sim and Kiefer 2005).

Most importantly, fMRI and ERP results revealed an involvement of visuo-motor areas in the generation of the action-priming effect thereby providing the first direct evidence for a functional link between the brain circuits involved in visuo-motor processing and visual object recognition: FMRI scanning (Experiment 1) showed a reduction of brain activity as a function of action priming in various parts of the visuo-motor system including parietal dorsal stream areas and associated frontal motor regions: Reduced activity related to priming was obtained in primary motor and somatosensory areas (left pre- and post-central gyri, BA 1–4), premotor areas (inferior and superior frontal gyri, BA 6, BA 44) as well as in parietal visuo-motor areas (left superior parietal gyrus, BA 7; left inferior parietal gyrus, BA 39; right supramarginal gyrus, BA 40). Part of these frontal and parietal areas was also activated during mere passive viewing of actions in the present action observation localizer experiment (Experiment 2) demonstrating their function in visuo-motor processing in line with earlier studies (Rizzolatti, Fadiga, Gallese et al. 1996; Hari et al. 1998; Grèzes and Decety 2002; Buccino et al. 2004; Caspers et al. 2010; Hari et al. 2010; Pihko et al. 2010): Activity in the superior and inferior frontal (Kalenine et al. 2010) and parietal gyri (Caspers et al. 2010) as well as in the supramarginal gyrus (Ohgami et al. 2004) has also been shown to be critical for processing of object-related actions. Frontal and parietal visuo-motor areas also play a role in conceptual processing of action-related information as suggested by modality-specific theories of conceptual memory: Premotor and parietal areas have been found to be recruited during conceptual processing of manipulable artifact objects (Chao and Martin 2000; Gallese and Lakoff 2005; Barsalou 2008; Hoenig et al. 2008), activity in pre- and post-central gyri has been observed during both the execution of actions as well as during conceptual processing of action words (Hauk et al. 2004). Neural priming effects in the postcentral region were significantly related to behavioral priming suggesting a functional correspondence between behavioral and neural action priming, at least within a part of the visuo-motor system.

In addition to visuo-motor areas, priming-related reductions of brain activity were also obtained in visual-perceptual areas of the ventral stream including middle and inferior occipital gyri (BA 18/19) and the inferior temporal gyrus (BA 20). These areas are typically involved in visual object recognition (Goodale et al. 1991; Freedman et al. 2006; Jastorff and Orban 2009).

Action priming was also associated with reduced activity outside the visual-perceptual and the visuo-motor systems in areas known to support semantic integration (Patterson et al. 2007; Kiefer and Pulvermüller 2012). Priming-related attenuation of brain activity was obtained in the superior temporal gyrus (BA 22) including the temporal pole (BA 38). This suggests that previous activation of congruent action representations reduced the cognitive effort of semantic integration to achieve a coherent object representation as a cognitive basis for object recognition. Note that the congruent and incongruent conditions were matched for global semantic similarity ruling out semantic priming as a source of anterior temporal activity. In line with this suggested dorsal–ventral interaction, effective connectivity analyses indicated in addition to feedforward connections within the ventral and dorstal streams a significant influence of superior parietal cortex on activity in anterior temporal areas.

ERP source analyses revealed early activity (120–150 ms) in brain areas, which are typically associated with action processing (Kiefer 2001; Hauk et al. 2004; Kiefer 2005; Hoenig et al. 2008), that is, in pre- and post-central cortex as well as in inferior parietal cortex in agreement with the present fMRI experiment. This suggests that action priming depends on rapid activation of motor representations within the first 200 ms of visual object processing. In the later time window (380–480 ms), source activity was obtained in occipital and temporal areas extending to the temporal pole in addition to activity within frontal and parietal motor areas. Hence, priming-related activity in the ventral stream (occipito-temporal areas) as well in some parts of the dorsal stream (parts of premotor and parietal cortex) occurs relatively late and may reflect feedback influences from earlier motor activity. Although effective connectivity analyses of the MR signal do not provide any time course information, the observed influence of superior parietal cortex on activity in anterior temporal areas is congruent with this interpretation of the ERP data.

Taken together, our action-priming results reveal a functional role of visuo-motor representations in the dorsal stream for the visual recognition of manipulable man-made objects. They demonstrate a temporally ordered activation in both the ventral and dorsal streams during the course of visual recognition. In a first stage, action representations of the target objects are rapidly activated in the dorsal stream, whereas in a second stage, presumably due to a feedback signal derived from these initially activated action representations, priming modulates activity in various parts of the ventral stream such in areas dedicated to semantic integration in the anterior temporal lobe (Patterson et al. 2007). As action-related features are essential parts of the representations of manipulable objects (McRae and Cree 2002; Hoenig et al. 2008), visual and conceptual object representations were more accurately activated, and their visual recognition was facilitated. The present action priming study therefore demonstrates that the contribution of the dorsal visual stream to object perception is not confined to allocation of attention to the relevant stimulus as suggested previously (Pisella et al. 2009). Instead our observation of priming-related activity differences in frontal and parietal areas that overlapped with the activity pattern elicited by action observation strongly suggests a direct contribution of visuo-motor representations within in the dorsal stream to object recognition. Effective connectivity analyses of the MR signal also confirm the suggested dorsal–ventral interaction during object recognition. It has to be determined in future studies whether visuo-motor representations also facilitate the recognition of manipulable objects from other categories such as animals, plants or fruit, for which action information is less essential compared with artifact objects (McRae and Cree 2002; Hoenig et al. 2008).

While most brain areas showed the typical priming-related decrease of brain activity (Posner et al. 1997; Wagner et al. 1997; Buckner et al. 1998; Dehaene et al. 1998; Dehaene et al. 2001; Koutstaal et al. 2001), a few areas responded with increased activity as a function of priming: In a large cluster encompassing middle and inferior occipital gyri (BA 18/19) as well as in posterior inferior temporal and middle temporal gyri (BA 37, BA 21), the MR signal was higher in the congruent than in the incongruent condition. Furthermore, smaller clusters in right premotor, right superior and inferior parietal cortex also showed higher activity in response to priming. Possibly, these priming-related signal increases (Henson 2003; Price and Devlin 2011) reflect more stable and accurate object representation in the primed condition, which boost activity particularly in visual brain areas of the ventral stream.

It could be argued that the present priming effects do not arise from similarity between the prime action and the action afforded by the target object, but from object-related processes induced by the prime movie. However, this alternative explanation can be excluded for several reasons: First, the object, which was used to record the action movie, was removed from the video clip and could not be correctly inferred from observing the action as shown in a control experiment. Secondly, global semantic similarity between the unseen object from the action movie and the target object was matched for the congruent and incongruent conditions. Thirdly, the action in the prime movie and the action afforded by the target object were always different even in the congruent condition, thereby excluding effects of action repetition. Thus, our experimental design ensures that only action similarity between prime action and target object was the driving force of the priming effect and excludes alternative explanations based upon a possibly imagined object during action movie observation.

In conclusion, the present combined ERP/fMRI study specifies the interplay between visuo-motor processing in the dorsal visual stream and processes in the ventral stream dedicated to perceptual analyses and semantic integration during visual recognition of manipulable objects. Using an action priming paradigm, we found a priming-related reduction of brain activity in frontal and parietal visuo-motor areas as well as in inferior temporal and anterior temporal areas. Inferior temporal areas are known to be involved in visual shape analysis (Freedman et al. 2006; Jastorff and Orban 2009) and visual recognition whereas anterior temporal areas are thought to support semantic integration (Patterson et al. 2007; Kiefer and Pulvermüller 2012). ERP recordings suggested a rapid onset of activity in frontal and parietal visuo-motor areas at about 120 ms whereas activity in occipito-temporal and anterior temporal areas emerged later at about 380 ms. Effective connectivity analyses substantiated an influence of dorsal areas (superior parietal cortex) on processing within the ventral visual stream (temporal pole) during the course of object recognition. The present results suggest that rapidly initiated visuo-motor processes within the dorsal stream functionally contribute to the visual recognition of manipulable objects in interaction with processes within the ventral visual stream dedicated to visual analysis and semantic integration.

Notes

Conflict of Interest: None declared.

References

Almeida
J
Mahon
BZ
Nakayama
K
Caramazza
A
2008
.
Unconscious processing dissociates along categorical lines
.
Proc Natl Acad Sci USA
 .
105
:
15214
15218
.
Barsalou
LW
2008
.
Grounded cognition
.
Annu Rev Psychol
 .
59
:
617
645
.
Bentin
S
McCarthy
G
Wood
CC
1985
.
Event-related potentials, lexical decision and semantic priming
.
Electroencephalogr Clin Neurophysiol
 .
60
:
343
355
.
Biederman
I
1987
.
Recognition-by-components: a theory of human image understanding
.
Psychol Rev
 .
94
:
115
147
.
Buccino
G
Vogt
S
Ritzl
A
Fink
GR
Zilles
K
Freund
HJ
Rizzolatti
G
2004
.
Neural circuits underlying imitation learning of hand actions: an event-related fMRI study
.
Neuron
 .
42
:
323
334
.
Buckner
RL
Goodman
J
Burock
M
Rotte
M
Koutstaal
W
Schacter
D
Rosen
B
Dale
AM
1998
.
Functional-anatomic correlates of object priming in humans revealed by rapid presentation event-related fMRI
.
Neuron
 .
20
:
285
296
.
Caspers
S
Zilles
K
Laird
AR
Eickhoff
SB
2010
.
ALE meta-analysis of action observation and imitation in the human brain
.
Neuroimage
 .
50
:
1148
1167
.
Chao
LL
Haxby
JV
Martin
A
1999
.
Attribute-based neural substrates in temporal cortex for perceiving and knowing about objects
.
Nat Neurosci
 .
2
:
913
919
.
Chao
LL
Martin
A
2000
.
Representation of manipulable man-made objects in the dorsal stream
.
Neuroimage
 .
12
:
478
484
.
Creem
SH
Proffitt
DR
2001
.
Grasping objects by their handles: a necessary interaction between cognition and action
.
J Exp Psychol Hum Percept Perform
 .
27
:
218
228
.
Dehaene
S
Naccache
L
Cohen
L
Le Bihan
D
Mangin
J-F
Poline
J-B
Rivière
D
2001
.
Cerebral mechanisms of word masking and unconscious repetition priming
.
Nat Neurosci
 .
4
:
752
758
.
Dehaene
S
Naccache
L
LeClec'H
G
Koechlin
E
Mueller
M
Dehaene-Lambertz
G
van de Moortele
P-F
LeBihan
D
1998
.
Imaging unconscious priming
.
Nature
 .
395
:
597
600
.
Freedman
DJ
Riesenhuber
M
Poggio
T
Miller
EK
2006
.
Experience-dependent sharpening of visual shape selectivity in inferior temporal cortex
.
Cereb Cortex
 .
16
:
1631
1644
.
Friston
KJ
Harrison
L
Penny
W
2003
.
Dynamic causal modelling
.
Neuroimage
 .
19
:
1273
1302
.
Gallese
V
Lakoff
G
2005
.
The Brain's concepts: the role of the Sensory-motor system in conceptual knowledge
.
Cogn Neuropsychol
 .
22
:
455
479
.
Goodale
MA
Haffenden
AM
2003
.
Interactions between the dorsal and ventral streams of visual processing
.
Adv Neurol
 .
93
:
249
267
.
Goodale
MA
Milner
AD
1992
.
Separate visual pathways for perception and action
.
Trends Neurosci
 .
15
:
20
25
.
Goodale
MA
Milner
AD
Jakobson
LS
Carey
DPA
1991
.
A neurological dissociation between perceiving objects and grasping them
.
Nature
 .
349
:
154
156
.
Grafton
ST
Fadiga
L
Arbib
MA
Rizzolatti
G
1997
.
Premotor cortex activation during observation and naming of familiar tools
.
Neuroimage
 .
6
:
231
236
.
Grèzes
J
Decety
J
2002
.
Does visual perception of object afford action? Evidence from a neuroimaging study
.
Neuropsychologia
 .
40
:
212
222
.
Grèzes
J
Tucker
M
Armony
J
Ellis
R
Passingham
RE
2003
.
Objects automatically potentiate action: an fMRI study of implicit processing
.
Eur J Neurosci
 .
17
:
2735
2740
.
Grill-Spector
K
Malach
R
2004
.
The human visual cortex
.
Annu Rev Neurosci
 .
27
:
649
677
.
Hari
R
Forss
N
Avikainen
S
Kirveskari
E
Salenius
S
Rizzolatti
G
1998
.
Activation of human primary motor cortex during action observation: a neuromagnetic study
.
Proc Natl Acad Sci USA
 .
95
:
15061
15065
.
Hari
R
Parkkonen
L
Nangini
C
2010
.
The brain in time: insights from neuromagnetic recordings
.
Ann N Y Acad Sci
 .
1191
:
89
109
.
Hauk
O
2004
.
Keep it simple: a case for using classical minimum norm estimation in the analysis of EEG and MEG data
.
Neuroimage
 .
21
:
1612
1621
.
Hauk
O
Johnsrude
I
Pulvermüller
F
2004
.
Somatotopic representation of action words in human motor and premotor cortex
.
Neuron
 .
41
:
301
307
.
Heim
S
Eickhoff
SB
Amunts
K
2009
.
Different roles of cytoarchitectonic BA 44 and BA 45 in phonological and semantic verbal fluency as revealed by dynamic causal modelling
.
Neuroimage
 .
48
:
616
624
.
Helbig
HB
Graf
M
Kiefer
M
2006
.
The role of action representations in visual object recognition
.
Exp Brain Res
 .
174
:
221
228
.
Helbig
HB
Steinwender
J
Graf
M
Kiefer
M
2010
.
Action observation can prime visual object recognition
.
Exp Brain Res
 .
200
:
251
258
.
Henson
RN
2003
.
Neuroimaging studies of priming
.
Prog Neurobiol
 .
70
:
53
81
.
Himmelbach
M
Karnath
HO
2005
.
Dorsal and ventral stream interaction: contributions from optic ataxia
.
J Cogn Neurosci
 .
17
:
632
640
.
Hoenig
K
Sim
EJ
Bochev
V
Herrnberger
B
Kiefer
M
2008
.
Conceptual flexibility in the human brain: dynamic recruitment of semantic maps from visual, motor, and motion-related areas
.
J Cogn Neurosci
 .
20
:
1799
1814
.
Humphreys
GW
Riddoch
MJ
Quinlan
PT
1988
.
Cascade processes in picture identification
.
Cogn Neuropsychol
 .
5
:
67
103
.
Huynh
H
Feldt
L
1970
.
Conditions under which mean square ratios in repeated measurements design have exact Fdistribution
.
J Am Stat Assoc
 .
65
:
1582
1588
.
Jastorff
J
Orban
GA
2009
.
Human functional magnetic resonance imaging reveals separation and integration of shape and motion cues in biological motion processing
.
J Neurosci
 .
29
:
7315
7329
.
Kalenine
S
Buxbaum
LJ
Coslett
HB
2010
.
Critical brain regions for action recognition: lesion symptom mapping in left hemisphere stroke
.
Brain
 .
133
:
3269
3280
.
Kiefer
M
2001
.
Perceptual and semantic sources of category-specific effects in object categorization: event-related potentials during picture and word categorization
.
Mem Cognit
 .
29
:
100
116
.
Kiefer
M
2005
.
Repetition-priming modulates category-related effects on event-related potentials: further evidence for multiple cortical semantic systems
.
J Cogn Neurosci
 .
17
:
199
211
.
Kiefer
M
Barsalou
LW
2013
.
Grounding the Human Conceptual System in Perception, Action, and Internal States
 .
Cambridge
(
MA
):
MIT Press
.
Kiefer
M
Pulvermüller
F
2012
.
Conceptual representations in mind and brain: theoretical developments, current evidence and future directions
.
Cortex
 .
48
:
805
825
.
Kiefer
M
Sim
EJ
Helbig
H
Graf
M
2011
.
Tracking the time course of action priming on object recognition: evidence for fast and slow influences of action on perception
.
J Cogn Neurosci
 .
23
:
1864
1874
.
Kiefer
M
Sim
EJ
Liebich
S
Hauk
O
Tanaka
J
2007
.
Experience-dependent plasticity of conceptual representations in human sensory-motor areas
.
J Cogn Neurosci
 .
19
:
525
542
.
Kleiner
M
Wallraven
C
Bülthoff
HH
2004
.
The MPI Videolab—a system for high quality synchronous recording of video and audio from multiple viewpoints
.
Technical Report Nr. 123. Tn: Max Planck Institute for Biological Cybernetics
.
Koutstaal
W
Wagner
AD
Rotte
M
Maril
A
Buckner
RL
Schacter
DL
2001
.
Perceptual specificity in visual object priming: functional magnetic resonance imaging evidence for a laterality difference in fusiform cortex
.
Neuropsychologia
 .
39
:
184
199
.
Kutas
M
Hillyard
SA
1980
.
Reading senseless sentences: brain potentials reflect semantic incongruity
.
Science
 .
207
:
203
205
.
Lancaster
JL
Woldorff
MG
Parsons
LM
Liotti
M
Freitas
CS
Rainey
L
Kochunov
PV
Nickerson
D
Mikiten
SA
Fox
PT
2000
.
Automated Talairach atlas labels for functional brain mapping
.
Hum Brain Mapp
 .
10
:
120
131
.
Lin
FH
Witzel
T
Ahlfors
SP
Stufflebeam
SM
Belliveau
JW
Hamalainen
MS
2006
.
Assessing and improving the spatial accuracy in MEG source localization by depth-weighted minimum-norm estimates
.
Neuroimage
 .
31
:
160
171
.
Makeig
S
Jung
TP
Bell
AJ
Ghahremani
D
Sejnowski
TJ
1997
.
Blind separation of auditory event-related brain responses into independent components
.
Proc Natl Acad Sci USA
 .
94
:
10979
10984
.
Martin
A
2007
.
The representation of object concepts in the brain
.
Annu Rev Psychol
 .
58
:
25
45
.
McRae
K
Cree
GS
2002
.
Factors underlying category-specific semantic deficits
. In:
Forde
EME
Humphreys
GW
editors.
Category-specificity in mind and brain
 .
East Sussex, UK
:
Psychology Press
. p.
211
249
.
Mechelli
A
Crinion
JT
Long
S
Friston
KJ
Lambon Ralph
MA
Patterson
K
McClelland
JL
Price
CJ
2005
.
Dissociating reading processes on the basis of neuronal interactions
.
J Cogn Neurosci
 .
17
:
1753
1765
.
Mosher
JC
Leahy
RM
1998
.
Recursive MUSIC: a framework for EEG and MEG source localization
.
IEEE Trans Biomed Eng
 .
45
:
1342
1354
.
Neely
JH
1991
.
Semantic priming effects in visual word recognition: a selective review of current findings and theories
. In:
Besner
D
Humphreys
GW
editors.
Basic progresses in reading—visual word recognition
 .
Hillsdale
(
NJ
):
Lawrence Erlbaum Associates
. p.
264
333
.
Ohgami
Y
Matsuo
K
Uchida
N
Nakai
T
2004
.
An fMRI study of tool-use gestures: body part as object and pantomime
.
Neuroreport
 .
15
:
1903
1906
.
Oldfield
R
1971
.
The assessment and analysis of handedness: the Edinburgh Inventory
.
Neuropsychologia
 .
9
:
97
113
.
Patterson
K
Nestor
PJ
Rogers
TT
2007
.
Where do you know what you know? The representation of semantic knowledge in the human brain
.
Nat Rev Neurosci
 .
8
:
976
987
.
Penny
WD
Stephan
KE
Mechelli
A
Friston
KJ
2004
.
Comparing dynamic causal models
.
Neuroimage
 .
22
:
1157
1172
.
Pihko
E
Nangini
C
Jousmaki
V
Hari
R
2010
.
Observing touch activates human primary somatosensory cortex
.
Eur J Neurosci
 .
31
:
1836
1843
.
Pisella
L
Binkofski
F
Lasek
K
Toni
I
Rossetti
Y
2006
.
No double-dissociation between optic ataxia and visual agnosia: multiple sub-streams for multiple visuo-manual integrations
.
Neuropsychologia
 .
44
:
2734
2748
.
Pisella
L
Sergio
L
Blangero
A
Torchin
H
Vighetto
A
Rossetti
Y
2009
.
Optic ataxia and the function of the dorsal stream: contributions to perception and action
.
Neuropsychologia
 .
47
:
3033
3044
.
Posner
MI
DiGirolamo
GJ
Fernandez-Duque
D
1997
.
Brain mechanisms of cognitive skills
.
Conscious Cogn
 .
6
:
267
290
.
Price
CJ
Devlin
JT
2011
.
The interactive account of ventral occipitotemporal contributions to reading
.
Trends Cogn Sci
 .
15
:
246
253
.
Pulvermüller
F
2005
.
Brain mechanisms linking language and action
.
Nat Rev Neurosci
 .
6
:
576
582
.
Pulvermüller
F
Fadiga
L
2010
.
Active perception: sensorimotor circuits as a cortical basis for language
.
Nat Rev Neurosci
 .
11
:
351
360
.
Rizzolatti
G
Fadiga
L
Gallese
V
Fogassi
L
1996
.
Premotor cortex and the recognition of motor actions
.
Brain Res Cogn Brain Res
 .
3
:
131
141
.
Rizzolatti
G
Fadiga
L
Matelli
M
Bettinardi
V
Paulesu
E
Perani
D
Fazio
F
1996
.
Localization of grasp representations in humans by PET: 1. Observation versus execution
.
Exp Brain Res
 .
111
:
246
252
.
Rorden
C
Brett
M
2000
.
Stereotaxic display of brain lesions
.
Behav Neurol
 .
12
:
191
200
.
Rosch
E
Mervis
CB
Gray
WD
Johnson
DM
Boyes-Braem
P
1976
.
Basic objects in natural categories
.
Cognit Psychol
 .
8
:
382
439
.
Rumiati
RI
Weiss
PH
Shallice
T
Ottoboni
G
Noth
J
Zilles
K
Fink
GR
2004
.
Neural basis of pantomiming the use of visually presented objects
.
Neuroimage
 .
21
:
1224
1231
.
Sakuraba
S
Sakai
S
Yamanaka
M
Yokosawa
K
Hirayama
K
2012
.
Does the human dorsal stream really process a category for tools?
J Neurosci
 .
32
:
3949
3953
.
Sim
EJ
Kiefer
M
2005
.
Category-related brain activity to natural categories is associated with the retrieval of visual features: evidence from repetition effects during visual and functional judgments
.
Brain Res Cogn Brain Res
 .
24
:
260
273
.
Smith
AP
Stephan
KE
Rugg
MD
Dolan
RJ
2006
.
Task and content modulate amygdala-hippocampal connectivity in emotional retrieval
.
Neuron
 .
49
:
631
638
.
Stephan
KE
Penny
WD
2007
.
Dynamic causal models and Bayesian selection
. In:
Friston
KJ
editors.
Statistical parametric mapping: the analysis of functional brain image
 .
Amsterdam
:
Elsevier
. p.
577
585
.
Stephan
KE
Weiskopf
N
Drysdale
PM
Robinson
PA
Friston
KJ
2007
.
Comparing hemodynamic models with DCM
.
Neuroimage
 .
38
:
387
401
.
Ungerleider
LG
Mishkin
M
1982
.
Two cortical visual systems
. In:
Ingle
DJ
Goodale
MA
Mansfield
RJW
editors.
Analysis of visual behavior
 .
Cambridge
(
MA
):
MIT Press
. p.
549
586
.
Wagner
AD
Desmond
JE
Demb
JB
Glover
GH
Gabrieli
JDE
1997
.
Semantic repetition priming for verbal and pictorial knowledge: a functional MRI study of the left inferior prefrontal cortex
.
J Cogn Neurosci
 .
9
:
714
726
.