Abstract
Combination of visual and kinesthetic information is essential to perceive bodily movements. We conducted behavioral and functional magnetic resonance imaging experiments to investigate the neuronal correlates of visuokinesthetic combination in perception of hand movement. Participants experienced illusory flexion movement of their hand elicited by tendon vibration while they viewed video-recorded flexion (congruent: CONG) or extension (incongruent: INCONG) motions of their hand. The amount of illusory experience was graded by the visual velocities only when visual information regarding hand motion was concordant with kinesthetic information (CONG). The left posterolateral cerebellum was specifically recruited under the CONG, and this left cerebellar activation was consistent for both left and right hands. The left cerebellar activity reflected the participants' intensity of illusory hand movement under the CONG, and we further showed that coupling of activity between the left cerebellum and the “right” parietal cortex emerges during this visuokinesthetic combination/perception. The “left” cerebellum, working with the anatomically connected high-order bodily region of the “right” parietal cortex, participates in online combination of exteroceptive (vision) and interoceptive (kinesthesia) information to perceive hand movement. The cerebro–cerebellar interaction may underlie updating of one's “body image,” when perceiving bodily movement from visual and kinesthetic information.
Introduction
Perception of the continually changing spatial location of a body part during movement is required for accurate motor control of the body parts (limbs) (Rothwell et al. 1982; Bard et al. 1995; Ghez and Sainburg 1995; Sainburg et al. 1995). As visual and kinesthetic systems can both signal information about limb movement to the brain, simultaneous processing of these 2 sources of information, which are eventually combined, is particularly important in perceiving limb movement (Head and Holmes 1911; Graziano and Gross 1998). Earlier studies have suggested that activations of the premotor and parietal cortices are associated with multisensory perception of spatial location of a stationary limb (Graziano 1999; Graziano et al. 2000; Ehrsson et al. 2004). However, the neuronal substrates involved in the multisensory combinative process that subserves limb movement perception have not been investigated directly.
In the present study, right-handed participants experienced illusory flexion movements of their right hand while simultaneously viewing video-recorded flexion (congruent: CONG) or extension (incongruent: INCONG) motions of their right hand (Fig. 1a). The illusion was elicited by vibratory stimulation on the tendon of the wrist extensor (extensor carpi ulnaris: ECU) muscle (Goodwin et al. 1972a, 1972b; Naito 2004), which excites the muscle spindle afferents (Burke et al. 1976; Roll and Vedel 1982; Gandevia 1985; Roll et al. 1989). This illusion is not associated with any actual movement, intention to move, or sense of effort (Kito et al. 2006). Thus, by examining how visual information modulates the kinesthetic experience and measuring brain activity during these situations, the brain processes associated with online combination of visual and kinesthetic information for perception of hand movement can be investigated.
Conditions (a) and results (b) in the behavioral experiment. (a) Participants experienced illusory flexions of their right hands while viewing their video-recorded hand flexion (CONG) or extension motion (INCONG). Crosses on the wrist joints indicate fixation points. The open arrow indicates the direction of illusory movement, and filled arrows indicate the directions of visual hand motions. Three different velocities were used for each hand motion. (b) Filled bars represent the mean illusory angles across all participants under CONG conditions, and open bars indicate those under INCONG conditions. Error bars indicate standard errors of means across all participants. *P < 0.05.
In the behavioral experiment, 3 different velocities of visual hand motions were prepared for each flexion (CONG) and extension (INCONG) hand movement. Because directions of hand movements sensed by visual and kinesthetic systems are tightly coupled in our daily experience, we expect that visual velocity will be combined with the illusion and thus grade the kinesthetic sensation only under the CONG condition (Saunders and Knill 2004). Vision may also affect kinesthesia under the INCONG condition, but it is anticipated that the effect would differ from that under the CONG condition, given the directional discrepancy between information provided by the 2 senses, which is apparent and opposes regular experience.
We also examined brain activity to identify brain areas associated with the simultaneous processing of visual and kinesthetic information derived from hand movement (visuokinesthetic processing). Previous studies suggest that neurons in the frontoparietal cortices participate in processing of multisensory information derived from the body (Graziano 1999; Graziano et al. 2000; Ehrsson et al. 2004). Thus, frontoparietal activations are most likely to be common contributors for visuokinesthetic processing under both the CONG and INCONG conditions. Given our prediction that visuokinesthetic combination only occurs under the CONG condition (see above), we further expect activation in additional brain structures that are specifically involved in visuokinesthetic processing under the CONG condition. Several studies have shown that the human lateral cerebellum plays crucial roles in visuomotor tasks that require online combination of visual and kinesthetic information (Beppu et al. 1984; Haggard et al. 1995; Liu et al. 1999; Imamizu et al. 2000). Thus, cerebellar activation is likely to emerge under the CONG condition.
Because it is likely that the cerebellum participates in the online combination of visual and kinesthetic input, we further examined whether cerebellar activity also mediates visuokinesthetic perception. This is important because the classical view is that the cerebral (frontoparietal) cortices are predominantly engaged in multisensory processing and perception of the body (Berlucchi and Aglioti 1997; Berti et al. 2005; Haggard and Wolpert 2005; Committeri et al. 2007). Finally, prompted by the idea that the cerebro–cerebellar interaction plays an important role in processing various sensory inputs (Middleton and Strick 1998), we sought to determine whether cerebro–cerebellar interaction underlies the present visuokinesthetic combination (perception). We addressed these issues using functional magnetic resonance imaging (fMRI).
Materials and Methods
Participants
Seventeen volunteers (12 males and 5 females, 21–28 years old) participated in the behavioral experiment, 16 volunteers (11 males and 5 females, 21–28 years old) participated in the fMRI experiment 1, 7 male volunteers (23–40 years old) participated in the fMRI experiment 2, and 12 volunteers (7 males and 5 females, 18–33 years old) participated in the fMRI experiment 3. All the participants were right-handed (Oldfield 1971) and had no history of neurological or other disease. They gave written informed consent prior to the experiments, and the Ethical Committee of the National Institute of Physiological Science approved the study. The fMRI experiments were carried out following the principles and guidelines of the Declaration of Helsinki (1975).
A pretest was performed before the start of each experiment. The participants were asked to relax their vibrated hand and to check if they could experience an illusory hand movement. Most of the participants had no knowledge about kinesthetic illusion, but within a couple of trials (15-s vibration to the tendon of the wrist extensor muscle in each trial), all reported that they felt their hands flexing. After this procedure, the real experiment was performed.
Behavioral Experiment
Task
The participants received vibratory stimulation that elicited illusory flexion movement of their right hand while viewing video-recorded flexion (CONG) or extension (INCONG) hand motion of their right hand through a head-mounted display (Eye-Trek FMD-150W, Olympus, Tokyo, Japan) (Fig. 1a). The participants were seated on a comfortable chair, and their right arms were placed on a table beside them. The forearms were completely relaxed in this position, the hands were pronated, and the wrists were naturally flexed and relaxed without touching anything. The participants viewed their hand motions through the head-mounted display while they faced the front. For some participants, we also asked them to view the hand motions while they faced the actual location of their right hand.
The vibratory stimulation was given to the tendon of the ECU muscle of the right wrist at 110 Hz. The vibrator was nonmagnetic (ILLUSOR, Umihira Ltd., Kyoto, Japan) and driven by constant air pressure provided by an air compressor (AIR KING GTAC 1525, GREAT TOOL, Taipei, Taiwan) (Hagura et al. 2007). The amplitude of the vibratory stimulus was approximately ±3.5 mm. An experienced experimenter operated the vibrator manually by applying it to the skin with a constant light pressure that was the minimum required to induce the illusory movement in each participant.
For the visual hand motions, 3 different velocities (Fast, Medium, and Slow) were used for each flexion or extension hand motion. The velocities of these motions were determined as follows. First, blindfolded participants experienced an illusory flexion movement elicited by tendon vibration of the right ECU muscle for 10 s and were asked to remember the average velocity of the illusory movement. Immediately after the trial, they were required to replicate the illusory movement at the velocity they remembered by actually moving their hand. This movement was video recorded (VIDEO WALKMAN GU-D900, Sony, Tokyo, Japan) and defined as the Fast condition. Next, the participants were requested to replicate the hand movement at approximately half and approximately a quarter of the remembered velocity. These movements were also video recorded and were defined as the Medium and Slow conditions, respectively. After these procedures for flexion hand motions were completed, an illusory extension movement was elicited by vibrating the tendon of the right flexor carpi ulnaris (FCU) muscle in another trial (in this situation, the vibrated hands were semipronated without touching anything to allow the stimulation on the tendon of FCU). The same procedure was used to determine the 3 velocities for visual extension hand motion.
Replications (see above) of the flexion hand movements were started from a straight wrist position (0 degree), and replications of the extension movements were started from a wrist-flexed position (40 degrees). Both replications were performed for 10 s, and neither of the replicated movements reached the maximum rotation of the individual wrist joint. The mean velocities of these replicated flexion and extension movements for all participants were approximately 6.6 degrees per second (Fast), 4.0 degrees per second (Medium), and 2.4 degrees per second (Slow). The distance between the camera and the hand was approximately 65 cm, which ensured that the position and size of the hand image (radial view) on the display stayed constant for all conditions. The video-recorded flexion hand motions were used for the CONG condition, and the extension hand motions were used for the INCONG condition. Thus, there were a total of 6 conditions.
Three trials were performed for each of the 6 conditions (3 velocities for CONG and 3 for INCONG), and the order of the conditions was randomized. In a given trial, the tendon was vibrated for 10 s while the participants viewed 1 of the 6 video-recorded hand motions for 10 s. The start of tendon vibration was synchronized with the start of hand motion in the video. The participants were asked to be aware of illusory hand movements while they viewed their hand motions. After each trial, they were instructed to replicate the experienced maximum angle of the illusory flexion by flexing their wrists (Hagura et al. 2007), and this angle (i.e., total angular displacement) was measured with a protractor. The mean value was calculated for the 3 trials for each condition per participant. To evaluate the effect of the 3 different visual velocities on the illusory wrist angular displacement, a t-test with a Bonferroni correction was used for each pair among the 3 conditions. This test was conducted separately for the CONG and INCONG conditions.
We also recorded electromyographic (EMG) activity from the wrist FCU, which is the agonistic muscle for the illusory wrist flexion and from the vibrated ECU muscle in 16 of the 17 participants. The integrated EMG was calculated for each visual condition and compared across conditions (see detailed method in the Supplementary Materials).
fMRI Experiment 1
Scanner
A 3.0 T Siemens scanner (Magnetom Allegra) with a head coil provided T1-weighted anatomical images (3-dimensional magnetization-prepared rapid-acquisition-echo sequence) and functional T2*-weighted echoplanar images (64 × 64 matrix, 3.0 × 3.0 mm, time echo [TE] = 40 ms). One functional image volume of the brain was collected every 4 s (time repetiton [TR] = 4,000 ms) and comprised 44 slices of 3 mm in thickness, which ensured that the whole brain was within the field of view (192 × 192 mm).
Tasks
The 16 participants rested comfortably in a supine position inside the magnetic resonance scanner with the arms extended in a relaxed position parallel to the trunk. The hands were pronated and the wrists were naturally flexed and relaxed (ca., 40 degrees of flexion) without touching anything. The arms were fixed with an arm brace. The participants were instructed to relax their body completely and not to move during scanning and were also requested to be aware of the sensations from the vibrated hand as they viewed video-recorded motions of their hand.
The experiment was conducted with a 3 (somatosensory) × 3 (visual) factorial design. For the somatosensory conditions, we vibrated either the tendon of the wrist extensor muscles, which elicit illusory hand flexion movements (Tendon vibration: T), or the skin surface over the bone just beside the tendon (e.g., the processus styloideus ulnae) (Bone vibration: B) that only elicited a sensation of skin vibration with no illusion (Naito and Ehrsson 2006). The latter was used as a control condition for the former. Although the tendon vibration excites muscle spindle afferents that elicit illusory hand movement, it may simultaneously recruit vibrotactile receptors that do not directly contribute to the kinesthetic experience. Therefore, to evaluate the effect of kinesthetic input alone, the effects in the B condition were subtracted from those in the T condition. We have shown the validity of this somatosensory control condition in previous tendon vibration studies (Naito and Ehrsson 2001, 2006; Naito, Kochiyama, et al. 2002; Naito, Roland, Ehrsson 2002; Hagura et al. 2007; Naito et al. 2007, 2008). The skin contact surface was approximately 1 cm2 for both vibratory (T and B) conditions, with use of the same nonmagnetic vibrator as in the behavioral experiment. We also included a condition in which no vibratory stimuli were delivered (No vibration: N).
For the visual conditions, the participants viewed flexion (F) or extension (E) of their hands. These images were video recorded in advance, and the participants were thus completely passive during scanning, that is, they did not generate any movements. The velocities of the hand motions were set at approximately 3 degrees per second. This velocity was selected based on our previous observation that this is the average velocity for illusory hand movements when the tendon is vibrated for a relatively long time (>30 s) (e.g., Naito, Kochiyama, et al. 2002). For the flexion hand motion, the participants viewed video-recordings of flexion of their hands from a starting position of 20-degree extension to a final position of 70-degree flexion. For the extension motion, they viewed their hands extending from a starting position of 50-degree flexion to an end point of 40-degree extension. The range of hand motion was thus approximately 90 degrees under both conditions, and this range was consistent across participants. In addition, we also included a visual control condition in which the participants viewed an inanimate fixation point (Fi).
During video-recording of hand motions, a mark was put on the radial side of the wrist joint. The position of this point does not vary with hand movement (Fig. 1a). During scanning while viewing hand motion, the participants were requested to fixate on this point to restrict eye movement. In the scanner, the visual images were viewed through a mirror located just in front of the eyes, and the images were projected from outside the scanner room. Before scanning, we ensured that participants could see the whole image of the visual hand motion. An experimenter in the scanner room operated the vibrator manually by lightly applying it to the skin, as in the behavioral experiment. Instructions regarding the stimulus conditions and onset/offset timing of the vibration were given to the experimenter by computer-generated visual cues projected onto the white surface of the scanner (the participants could not see this visual information).
Each participant underwent 6 sessions, and each session included 9 conditions (F–T, F–B, F–N, E–T, E–B, E–N, Fi–T, Fi–B, and Fi–N). For the order of the 3 somatosensory conditions [tendon vibration (T), bone vibration (B), and rest (R)], we prepared 6 combinations, that is, T–B–R, T–R–B, B–T–R, B–R–T, R–T–B, and R–B–T. All 6 combinations were tested once with each participant. One combination (e.g., T–B–R) was assigned to 1 session and repeated 3 times in a session, and other combinations were used in different sessions. The repetition of 1 combination in a session (e.g., T–B–R–T–B–R–T–B–R) enables avoidance of a successive vibratory stimulation on the tendon, which may reduce the sensitivity of muscle spindle afferents (Ribot-Ciscar and Roll 1998). Similarly, for the 3 visual conditions (FLEX [F], EXT [E], FIX [Fi]), 6 combinations were prepared for each session, and all 6 combinations were tested once with each participant. One combination was assigned to 1 session and repeated 3 times in a session. As a whole, a total of 36 conditions [6 (somatosensory) × 6 (visual)] resulted from the interaction of combinations of visual and somatosensory conditions. We could not perfectly randomize these interaction patterns of combinations across participants, but we took special care to avoid the concentrating usage of particular types. Each condition lasted for 32 s (in each session, 8 functional images were collected); use of this epoch length has provided reliable results in terms of reproducibility of activation patterns related to kinesthetic illusory hand movements in our previous tendon vibration studies (Naito and Ehrsson 2001, 2006; Naito, Kochiyama, et al. 2002; Naito, Roland, Ehrsson 2002; Hagura et al. 2007; Naito et al. 2007, 2008). A period of 8 s before the start of each condition was allotted for positioning of the vibrator; this period was defined as a condition of no interest. We collected 90 functional volumes in each session, and a total of 6 × 90 volumes were collected per participant. The F–T and E–T conditions are referred to as the CONG and INCONG conditions, respectively.
Data Analysis
The fMRI data were analyzed using Statistical Parametric Mapping software (SPM99; http://www.fil.ion.ucl.ac.uk/spm; Wellcome Department of Cognitive Neurology, London). The functional images were realigned to correct for head movements, coregistered with each participant's anatomical magnetic resonance imaging, and transformed (through both linear and nonlinear transformation) to the format of the Montréal Neurological Institute (MNI) standard brain. The functional images were scaled to 100 and spatially smoothed with an 8-mm full width at half maximum (FWHM) isotropic Gaussian kernel and smoothed in time by a 4-s FWHM Gaussian kernel.
A linear regression model (general linear model) was fitted to the data for each participant. Each condition was modeled with a boxcar function delayed by 4 s and convoluted with the standard SPM99 hemodynamic response function. Because our behavioral experiment indicated that the illusion starts a few seconds after the vibration onset, we modeled the first 4 s of all experimental conditions as an effect of no interest.
First, to identify activity related to the simultaneous processing of visual and kinesthetic information (visuokinesthetic processing), we defined a linear contrast of [(CONG + INCONG) vs. (F–B + E–B)] (main effect of visuokinesthetic processing in a 2 × 2 factorial design). The rationale of this contrast is that effects related to pure visual processing, vibrotactile processing, and the interaction between them can be removed, allowing elucidation of areas related to visuokinesthetic processing.
Next, we identified activity that was exclusively related to visuokinesthetic processing under the CONG condition by examining the interaction contrast between the site of vibration and the direction of vision in a 2 × 2 factorial design [(CONG vs. F–B) vs. (INCONG vs. E–B)]. The contrast (CONG vs. F–B) was used as an inclusive mask (P < 0.05, uncorrected) to ensure that only voxels showing activation from the corresponding control condition were included. The rationale for this design is to ensure that the resultant interaction term is related only to the interaction of concordant visual and kinesthetic information (CONG) and that the activation is not confounded by effects such as simple differences in tendon vibration, cutaneous vibrotactile stimulation, or types of visual hand motions. We also searched for brain regions that were specifically active in the INCONG condition using the contrast [(INCONG vs. E–B) vs. (CONG vs. F–B)].
The results obtained from these analyses were estimated blood oxygen level–dependent (BOLD) signals for the contrasts from each of the 16 participants (contrast images). To accommodate interparticipant variability, the contrast images from all participants were subjected to random-effect group analysis (second-level analysis) (Friston et al. 1999). One-sample t-tests were used (15 degrees of freedom [df]) with a voxelwise threshold of P < 0.001, uncorrected (t > 3.73) used to generate the cluster images. For the statistical inference, we used a threshold of P < 0.05 at the cluster level after correction for multiple comparisons in the whole-brain space.
For anatomical identification of activation in the cytoarchitectonic areas 4, 6, 44, and 45, the activated regions were related to the cytoarchitectonic 30% probably maps in the MNI reference brain space using the SPM Anatomy toolbox (version 1.2) (Eickhoff et al. 2005; Naito, Roland, Ehrsson 2002; Naito et al. 2005). For identification of motor areas in the medial wall (supplementary motor area [SMA], pre-SMA, cingulate motor area [CMA]), we referred to the definitions of Roland and Zilles (1996) and Picard and Strick (2001). For identification of the areas in the cerebellum, we referred to Schmahmann et al. (2000).
Flip Analysis
Flip analysis was performed to examine whether the right frontoparietal regions related to both the CONG and INCONG conditions were predominantly active in the right hemisphere (see Naito et al. 2005) and if they corresponded to the previously identified right frontoparietal regions, which were active during the illusions for both left and right hands (nonsomatotopical region; Naito et al. 2005, 2007; Naito and Ehrsson 2006).
First, the functional images (normalized and smoothed) for all scans of each participant in fMRI experiment 1 were flipped (a left-to-right transformation on the x-axis) to make left–right reversed images (flipped images). This allowed direct comparison of the activity in a voxel in the left side of the brain with the corresponding voxel on the right side. Then, a new general linear model that included both flipped and unflipped data was defined in each participant. The right-side dominance of frontoparietal activity was tested by the contrast [[(CONG + INCONG) vs. (F–B + E–B)] vs. flip − [(CONG + INCONG) vs. (F–B + E–B)]] in each participant. After the individual analysis, the individual images were incorporated into a second-level random-effect model to evaluate population inference (Friston et al. 1999). A 1-sample t-test was used (15 df) with a voxelwise threshold of P < 0.001, uncorrected (t > 3.73).
To confirm that the right-sided regions corresponded to the frontoparietal areas active during the illusions for both left and right hands in our previous studies (see above references), a small volume correction (P < 0.05 at the cluster-level corrected; see Worsley et al. 1996) was used to restrict the search space. The search space was defined as all voxels in a sphere with a 10-mm radius around the right frontoparietal peaks obtained in our previous study (Naito et al. 2005).
fMRI Experiment 2
Tasks
In fMRI experiment 1, the left cerebellum was specifically activated under the CONG condition for the right hand (see Results and Fig. 2e). In fMRI experiment 2, we wanted to determine whether the activity of the left cerebellum during visuokinesthetic combination has nonsomatotopical characteristics, that is, common activation under the CONG condition for left and right hands. Another group of 7 volunteers participated in this study, and the same 3.0-T scanner was used.
Results from fMRI experiment 1 and 2. (a–d) Activations in the motor areas and right-dominant frontoparietal areas during visuokinesthetic processing under the CONG and INCONG conditions. (e) Left cerebellar activations exclusively under the CONG condition. The orange region indicates the active area in fMRI experiment 1 (right hand), and the blue region corresponds to that identified in fMRI experiment 2 (activated for the left and right hands), respectively. The horizontal plane (z = −27) is displayed. (f) The size of effects of left cerebellar activation across conditions in the first fMRI experiment. Bars indicate the means of contrast parameter estimates (size of effect in arbitrary units) for left cerebellar activation (−27, −69, −30) during CONG (orange bar), INCONG (black bar), and control conditions (see Materials and Methods). The size of effect was calculated by comparing each condition with the condition in which participants simply viewed the fixation point (Fi–N; see Materials and Methods). Error bars represent standard errors of means across participants.
A similar procedure to that used in fMRI experiment 1 was used in this study. The main difference was that the experimental conditions were prepared for left and right hands. In the left-hand conditions, vibratory stimulation was applied to the left hand (T and B) while the participants viewed video-recorded flexion (F) or extension (E) hand motion of their left hand (visual velocity of ca., 3 degrees per second).
Each participant underwent 8 sessions: in 4 of these sessions vibrations were applied to the right hand, and in the other 4 sessions, the vibrations were applied to the left hand. The left-hand and right-hand sessions were alternated. For each hand, we assigned 2 sessions for the F condition and 2 for the E condition, and each session included T and B conditions. Each condition lasted for 21 s (TR = 3,000 ms, TE = 30 ms, 7 functional images) and was repeated 3 times in 1 session. The order of conditions was randomized according to a balanced schedule. We collected 42 functional images per condition for each hand (CONG, INCONG, F–B, and E–B) of each participant.
Data Analysis
Image preprocessing for the fMRI data was performed similarly to that in the first fMRI experiment. A linear regression model (general linear model) was fitted to the pooled data from all participants to increase the sensitivity of the analysis (fixed-effect model, as in Hagura et al. 2007).
To test whether left cerebellar activation specific to the CONG condition is consistently observed for the left and right hands, we defined a linear contrast of [(CONG vs. F–B) vs. (INCONG vs. E–B)] for each hand and performed a conjunction analysis [left (CONG vs. F–B) vs. (INCONG vs. E–B)] ∩ [right (CONG vs. F–B) vs. (INCONG vs. E–B)] (Price and Friston 1997; Friston et al. 2005). The conjunction contrast [left (CONG vs. F–B)] ∩ [right (CONG vs. F–B)] was used as an inclusive mask (P < 0.05, uncorrected) to ensure that only voxels showing activation under CONG conditions for the left and right hands were included. Because we had an a priori anatomical hypothesis in the left cerebellum, we restricted the search space and used a small volume correction (Worsley et al. 1996). The search space was defined by the left cerebellar activation map obtained in fMRI experiment 1 [(CONG vs. F–B) vs. (INCONG vs. E–B)] (P < 0.001, uncorrected) (Fig. 2e). The definition of the search space was statistically independent because the data were obtained in another group of volunteers. In the conjunction analysis, a voxelwise threshold of P < 0.005, corrected was used, which was a conservative threshold, and we confirmed that the activation exists around the region revealed in the conjunction analysis in each of the 2 interaction contrasts (P < 0.001, uncorrected).
Single-Subject Analysis
The above statistical analysis was based on the functional data pooled across participants using a fixed-effect analysis, in which the results may be biased by a minority of participants showing strong effects. To make sure that the group results were representative of all 7 participants, we analyzed the individual data for each hand (see Hagura et al. 2007). All image-processing steps were identical to those used in the group analysis (see above). The same GLM as in the group analysis was used, with the only difference being that we considered the functional data from each participant separately. A linear contrast of [(CONG vs. F–B) vs. (INCONG vs. E–B)] was performed separately for each hand of each participant, and increases of the BOLD signal (P < 0.05, uncorrected) were probed in a volume of radius 10-mm around peaks of activation detected in the group analysis. We report the number of participants that exhibited a BOLD signal increases in the relevant areas and the results were tabulated in the Supplementary Table 1 in the Supplementary Materials.
fMRI Experiment 3
Tasks
In fMRI experiment 3, we examined if the left cerebellar activity under the CONG condition is associated with subjective visuokinesthetic perception of hand movement. An independent group of 12 volunteers participated in the study, and the same 3.0 T scanner and a similar procedure to fMRI experiment 1 were used. The main difference was that brain activity was measured during vibration of the right hand (T and B) while the participants viewed video-recorded flexion motions of their right hand at 3 different velocities (Fast, Medium, and Slow). The hand motions of each participant were video recorded before the scan, during which hand flexing was started from a 20-degree extension position for all velocity conditions. For this video-recording, each participant was asked to continuously flex the hand up to an individual maximum angle of wrist flexion for 15, 20, or 30 s. The mean velocities across participants were approximately 5.8 degrees per second (Fast), 4.3 degrees per second (Medium), and 2.9 degrees per second (Slow). In the experiment, hand motions were presented for the first 15 s only in all velocity conditions. Thus, the participants viewed their hand motions in the ranges of approximately 90, 60, and 45 degrees, respectively.
All participants underwent 12 sessions, and each session included Fast-T, Fast-B, Medium-T, Medium-B, Slow-T, and Slow-B conditions. The order of the conditions was randomized as in fMRI experiment 1. Each condition lasted for 15 s (TR = 3,000 ms, TE = 30ms, 5 functional images) and was repeated once per session. During the scanning, the participants were asked to be aware of illusory hand movements while they viewed the flexion hand motions. After each session, the participants scored the degree of perceived hand flexion for each T condition on a scale from 0 to 10 (illusion score): a score of 10 indicated illusory movements at the maximally flexed angle and a score of 0 indicated no experience of illusory movement. As shown previously, the illusion score is correlated with the replicated angular magnitude of illusory hand movement (Hagura et al. 2007) and can be used in statistical analysis.
Data Analysis
In the statistical analysis of the behavioral ratings (illusion score), the mean value of scores for each condition (Fast, Medium, or Slow) was calculated for each participant. A t-test was performed on the mean values for each pair of velocity conditions, with a Bonferroni correction for the number of comparisons.
Image preprocessing for the fMRI data was similar to that in fMRI experiment 1. To reveal brain areas with activities covarying with the subjective perception of illusory flexion experience under the T condition, a parametric modulation analysis was performed (Buchel et al. 1998). A linear regression model (general linear model) was fitted to the individual data, including all 3 somatosensory conditions (T, B, and N). A regressor for parametric modulation was added to the T condition for each session. The linear parameters were used as parametric modulators for the illusion scores.
Brain regions that showed a linear relationship with the illusion scores were tested by applying a t-contrast (1 for the parametric modulation regressor and 0 for elsewhere) in each participant. Next, the individual images were incorporated into the second-level random-effect model to evaluate population inference (Friston et al. 1999). A 1-sample t-test (11 df) was used with a voxelwise threshold of P < 0.001, uncorrected (t > 3.73).
Psychophysiological Interaction Analysis
In the series of analyses described above, we found that the posterolateral portion of the left cerebellum was specifically activated during visuokinesthetic combination (perception) under the CONG condition and that this cerebellar activation was left-side dominant (activated irrespective of left and right hands) (see Results, Fig. 2e). In contrast, we found right-side dominant frontoparietal activation during visuokinesthetic processing under both the CONG and INCONG conditions (main effect of visuokinesthetic processing; see Fig. 2a–d and 4a), in which the locations corresponded to the right-sided regions active during illusions of the left and right hands in our previous studies (Naito et al. 2005, 2007; Naito and Ehrsson 2006).
Prompted by the anatomical evidence in nonhuman primates that the cerebral cortices and the cerebellum are mainly contralaterally interconnected (Glickstein 2000; Strick 2004), we examined if the cerebro–cerebellar interaction takes place under the CONG condition by reanalyzing the data obtained from fMRI experiment 1. We expected that coupling of activity between the right frontoparietal cortices and the left cerebellum is enhanced only under the CONG condition, in which the brain is required to process concordant visuokinesthetic information. We performed a psychophysiological interaction (PPI) analysis (Friston et al. 1997) by extracting the data from the right frontal and parietal regions (see details in Supplementary Materials).
Results
Velocity of Visual Hand Motion Grades Illusory Flexion under the CONG Condition
In the behavioral experiment, no overt hand movements appeared in any trials. The visual hand motions affected the sensation of illusory hand flexion under both the CONG and INCONG conditions; however, the visual effects were clearly distinct between the conditions. This was also confirmed when some participants viewed their hand motions while they faced the actual location of the right hand.
Under the CONG condition only, the velocity of the visual hand motion graded the intensity of illusory hand movement. Identical vibratory stimuli were given to the same tendon site, but the illusions were evenly attenuated irrespective of the viewed velocity of wrist motion under the INCONG condition (Fig. 1b). Under the CONG condition, the angle of illusory, perceived hand displacement at the Fast velocity was significantly greater than that at the Slow velocity (t-test, df = 16, t = 2.8, P < 0.05, corrected), but no such difference was observed under the INCONG condition. Furthermore, under the CONG condition, some participants reported that they felt as if they were viewing their actual hands passively flexing on the display, but none reported such an experience under the INCONG condition. Thus, the results suggest that velocity of visual hand motion was incorporated into the kinesthetic experience under the CONG condition but not under the INCONG condition.
Although we found a slight increase of EMG activities from the FCU muscle (the agonistic muscle for illusory wrist flexion) and from the vibrated ECU muscle during tendon vibration, the levels of EMG activities were not graded by the velocities of visual hand motions. In addition, the activities were not correlated with the angle of illusory hand displacement in any of the visual conditions, suggesting that the muscular activities have no direct relationship with the graded illusory sensation (see details in Supplementary Materials).
Frontoparietal Activation under the CONG and INCONG Conditions
In fMRI experiment 1, all participants experienced illusory hand flexion only during tendon vibration and no illusion during the bone vibration (B). Examination of the main effect of visuokinesthetic processing using the following contrast [(CONG + INCONG) vs. (F–B + E–B)] showed activation in the hand sections of cortical motor areas and in the frontoparietal cortices (P < 0.05, corrected) (Fig. 2a–d). In the motor areas, we found peaks of activation in the left (contralateral) cytoarchitectonic area 4a, dorsal part of area 6 (dorsal premotor cortex), medial aspect of area 6 (SMA), caudal part of the CMA, and bilateral pre-SMA. In the frontal cortices, peaks of activations were located in the bilateral cytoarchitectonic areas 44/45, frontal operculum, and the anterior part of the right middle frontal gyrus. In the parietal cortices, the peaks were located in the right lateralmost part of the inferior parietal lobule (IPL) and in the right intraparietal sulcus area. A flip analysis performed to test whether activation of the right frontoparietal cortices is right-side dominant, as found in our previous studies (Naito et al. 2005, 2007; Naito and Ehrsson 2006), showed right-sided activations in the IPL [peak coordinates (x, y, z) = (66, −30, 33); t = 4.1, P < 0.05, corrected] and in areas 44/45 [peak (60, 21, 18); t = 4.0, P < 0.05, corrected] (Fig. 4a).
Exclusive Activation of the Left Lateral Cerebellum under the CONG Condition
The CONG condition [(CONG vs. F–B) vs. (INCONG vs. E–B)] exclusively activated the posterolateral portion of the left cerebellum (Crus I: peak [−12, −81, −24], t = 5.5; Lobule IV: peak [−27, −69, −30], t = 4.2; P < 0.05, corrected; Fig. 2e [orange region]). This was the only region that showed significant activation (interaction) in the entire brain and was not activated under the other 3 control conditions (F–B, INCONG, or E–B; see Fig. 2f). Furthermore, this area was not activated when the participants experienced illusions while viewing an inanimate fixation point or when they viewed their hand motion without receiving vibratory stimuli (see Materials and Methods and Fig. 2f). No specific activation for the INCONG condition was found in the entire brain [(INCONG vs. E–B) vs. (CONG vs. F–B)].
In fMRI experiment 2, we confirmed that the same left cerebellar region was activated under the CONG conditions for both left and right hands (Crus I: peak [−21, −78, −30], P < 0.005, corrected after small volume correction; Fig. 2e [blue region]). The left cerebellum was the only region that was activated in the entire brain (cluster size > 10 voxels), and no significant activation was observed in the right corresponding cerebellar region (P > 0.05, uncorrected). Finally, single-subject analysis revealed that all 7 participants had increased activity in this left cerebellar region for both left and right hands (see Supplementary Table 1).
Left Lateral Cerebellar Activity Is Related to Visuokinesthetic Perception of Hand Movement
In fMRI experiment 3, we examined whether activity in the posterolateral portion of the left cerebellum is associated with the subjective visuokinesthetic perception of hand movement. During fMRI sessions, the visual velocity of hand motion graded illusory hand movement, as in the behavioral experiment (t-test, Fast-T vs. Medium-T: df = 11, t = 3.8, P < 0.01, corrected; Medium-T vs. Slow-T: t = 5.9, P < 0.001, corrected) (Fig. 3a).
Results from fMRI experiment 3. (a) Mean illusion scores in fMRI experiment 3. **P < 0.01, ***P < 0.001. Error bars indicate the standard errors of means across participants. (b) The left cerebellar region (red region; P < 0.005, uncorrected for display purpose) in which activity was correlated with the intensity of visuokinesthetic perception. The activation is superimposed on the same plane as that in Figure 2e. (c) Significant correlation between behavioral ratings (illusion scores) and left cerebellar activity (size of effect) in a representative participant (r = 0.57, df = 34, P < 0.001, 1-tailed). The illusion scores are normalized (mean corrected) in each session.
By performing parametric modulation analysis using the subjective ratings for the illusory experiences (illusion scores) as covariates, we found that activity in the left posterolateral cerebellum was significantly correlated with subjective visuokinesthetic perception of hand movement [Crus I: peak (−12, −66, −27), t = 3.9, P < 0.001, uncorrected; Fig. 3b, red region, 3c]. The peak of this cerebellar activation was located just beside (1 voxel anterior) the left cerebellar cluster obtained in the fMRI experiment 1 (Fig. 3b, orange region), indicating activity in this region is involved in visuokinesthetic combination that subserves multisensory perception of hand movement.
Results of PPI Analyses
PPI analysis (see Supplementary Materials) revealed significantly enhanced coupling of activity between the right-sided IPL (Fig. 4a) and the left cerebellum under the CONG condition compared with that under the INCONG condition (Lobule IV: peak [−27, −69, −18], t = 4.0, P < 0.001, uncorrected; Fig. 4b–d). Likewise, a trend of enhanced coupling of activity was found between the right-sided area 44/45 (Fig. 4a) and the left cerebellum (Crus I: peak [−36, −72, −27], t = 3.0, P = 0.005, uncorrected).
Results from flip analysis (a) and from PPI analyses (b–d). (a) Right-dominant activities in the IPL (yellow circle) and in areas 44/45 revealed by the main effect of illusions. A sagittal plane (x = 60) is displayed. (b) Left cerebellar activation of a representative participant, which showed enhanced coupling of activity with right IPL activity under the CONG condition (P < 0.005, uncorrected for display purpose). A horizontal plane (z = −27) is displayed. (c) and (d) Relationship of activities between the right IPL and the left cerebellum in the representative participant (c: CONG; d: INCONG). The regression slopes were 0.52 and 0.29 for the CONG and INCONG conditions, respectively. The activities (x-axis for right IPL; y-axis for left cerebellum) are mean adjusted (arbitrary units).
Discussion
The behavioral experiment showed that velocities of visual hand motions are incorporated into the kinesthetic sensation of hand movement only when the movement directions sensed by the 2 independent sensory modalities are concordant (CONG). The fMRI studies showed that the left posterolateral cerebellum was selectively activated during visuokinesthetic processing under the CONG condition and that this activation was consistently elicited by stimulation of both the left and right hands. Furthermore, the level of left cerebellar activation correlated with the subjective visuokinesthetic perception of hand movement. Finally, we found enhancement of functional coupling between the left cerebellum and the right parietal cortex specifically in multisensory processing under the CONG condition. These lines of evidences suggest that the left cerebellum, in concert with the right parietal cortex, participates in visuokinesthetic combination for perception of hand movement.
Effects of Vision on Illusory Hand Movement
In our behavioral experiment, visual information regarding hand motions affected the kinesthetic illusion of hand flexion under both the CONG and INCONG conditions (Fig. 1b). This finding is consistent with the notion of “visual dominance over kinesthesia” (Botvinick and Cohen 1998; Hagura et al. 2007). The striking difference between the conditions was that the velocity of visual flexion motions graded the angular magnitude of the illusory flexion movement only under the CONG condition (Fig. 1b). This grading of illusion by the directionally concordant visual hand motion was also replicated in the CONG condition of fMRI experiment 3 (Fig. 3a), in which the participants viewed the hand motions displayed on the monitor in front of their face in the scanner. Thus, the location where the visual hand motions was displayed did not change this effect. These results indicate that, in addition to the general effect of visual dominance over kinesthesia (Hagura et al. 2007), a specific neuronal process to incorporate information on visual velocity into the kinesthetic experience is present when the directions sensed by vision and kinesthesia are concordant (CONG). Because participants reported that the hand motions they viewed were felt as actual state of their own vibrated hand only under the CONG condition, it is plausible that the brain continuously matches and combines visual and kinesthetic information to maintain perceptual coherence, linking seen and felt movements. This context-specific combination of vision and kinesthesia may be brought about because movement directions sensed by vision and kinesthesia are always matched during ordinary motor behavior (cf., Mercier et al. 2008).
Visuokinesthetic Combination in the Left Lateral Cerebellum
The hand sections of cortical motor areas were activated under both CONG and INCONG conditions (Fig. 2a–b). These areas participate in limb-specific kinesthetic processing even in situations when vision is unavailable (Naito et al. 2005, 2007). The 2 conditions also activated the frontoparietal cortices (mostly in the right hemisphere; areas 44/45 and IPL; see Figs 2a–d and 4a), which also process general kinesthetic information (limb nonspecific); it has been suggested that these areas contribute to creation of higher order body representations (Berlucchi and Aglioti 1997; Berti et al. 2005; Naito et al. 2005, 2007; Committeri et al. 2007). Therefore, the frontoparietal activation observed in the present study may be related to higher order visuokinesthetic processing, an interpretation consistent with previous findings (Ehrsson et al. 2004).
The left posterolateral cerebellum was specifically activated under the CONG condition (Fig. 2e). Activation of this region during illusory hand movements was not found in our previous studies (see above), suggesting that this cerebellar region is exclusively involved in multisensory processing of concordant visual and kinesthetic information regarding hand movements (also see Fig. 2f). It is unlikely that this cerebellar activity represents the positional discrepancy sensed by vision and kinesthesia (cf., Imamizu et al. 2000) because we observed no significant increase of activity in the cerebellum under the INCONG condition, where the discrepancy should be more prominent (Fig. 2e–f). Just as the behavioral results suggest that vision and kinesthesia are combined only under the CONG condition, it follows that the activity in the left posterolateral cerebellum is associated with a neuronal process of visuokinesthetic combination.
The tasks used in the experiments were completely passive, where no actual movement, no intention to move, and no sense of effort are required (Kito et al. 2006). Thus, it is reasonable to interpret our observation of cerebellar activity as a “multisensory” process of visuokinesthetic combination. This type of sensory combination could reasonably take place in the cerebellum because it has been demonstrated in nonhuman primates that both visual (Ungerleider et al. 1984; Schmahmann and Pandya 1991; Stein and Glickstein 1992; Glickstein 2000) and kinesthetic (Murphy et al. 1973; Bauswein et al. 1983; van Kan et al. 1993) information reaches the cerebellum, either indirectly via the cerebro–pontine–cerebellar pathway or directly via the spinocerebellar pathway. This view also fits with the notion that the cerebellum plays a sensory role when the brain acquires multisensory information that is relevant to the tasks to be performed (Gao et al. 1996; Parsons et al. 1997; Miall and Reckess 2002).
The human posterolateral cerebellum is activated by visuomotor tasks that require multisensory (integrative) processing of visual and kinesthetic information (Jueptner and Weiller 1998; Imamizu et al. 2000, 2004). Patients with lesions in the lateral cerebellum often exhibit deficits in visuomotor tracking tasks, and these deficits worsen during execution of tasks that include visual feedback of hand motion. This indicates that cerebellar damage disturbs computation of online visuokinesthetic combination in estimating the state of hand (Beppu et al. 1984; Haggard et al. 1995; Liu et al. 1999). In nonhuman primates, neurons in the lateral cerebellum seem to process online visual feedback from the monkey's own hand movements (Liu et al. 2003). All these observations support our claim that the cerebellum is involved in continuous visuokinesthetic combination to update limb position, which can be utilized for adaptive online motor control of limb position.
The left cerebellum has not been stressed as a locus for visuokinesthetic combination because most previous studies have focused more on the aspect of visuomotor control of hand movement (Jueptner and Weiller 1998; Imamizu et al. 2000, 2004) rather than on the process of multisensory combination that subserves the spatial perception of hand movement. A previous report of activity of the left cerebellum during the course of multisensory combination regarding a static hand (Ehrsson et al. 2004) may support our present interpretation. Thus, left cerebellar activation may be important as the brain configures the limb location/displacement by combining multisensory afferent information.
The precise mechanism of the cerebellar activation observed in the present study is still uncertain. However, the cerebellum has been reported to participate in the process of prediction of the sensory consequence of action (Wolpert et al. 1995; Wolpert and Miall 1996), especially regarding the visual consequence (perception) (Lindner et al. 2006) and also in the estimation of the state of the effector for online correction of action (Miall et al. 2007). Thus, one may speculate that the cerebellar activation is related to a process that matches the 2 states of hand position/displacement estimated by continuous visual and kinesthetic inputs to further predict the forthcoming state of the hand.
Left Cerebellar Activity Is Associated with Visuokinesthetic Perception
The activity in the left posterolateral cerebellum, which was similar to the region associated with visuokinesthetic combination (Fig. 3b), was positively correlated with the visuokinesthetic perception of hand movement (Fig. 3b–d). This indicates that the left cerebellum is not only related to online multisensory combination but is also involved in neuronal processes that mediate the perception of limb movement. Most previous studies of sensory (bodily) perception have focused on the role of the cerebral cortex (Berlucchi and Aglioti 1997; Berti et al. 2005; Committeri et al. 2007), and the present finding may shed light on the role of the cerebellum in mediating sensory perception. This view is supported by neuropsychological observations (visual perception: Nawrot and Rizzo 1995, 1998; Thier et al. 1999; kinesthetic perception: Grill et al. 1994). Because previous studies have demonstrated that the cerebellum is also involved in higher order functions, which were once thought to be primarily within the purview of the cerebral cortex (Kim et al. 1994; Allen et al. 1997; Parsons et al. 1997), it is conceivable that the cerebellum and the cerebral cortex interact (cerebro–cerebellar interaction) to achieve higher order functions, including bodily perception.
Cerebro–Cerebellar Interaction between the Right Parietal Cortex and the Left Cerebellum Mediates Visuokinesthetic Combination (Perception)
The PPI analysis revealed enhanced coupling of activity between the left cerebellum and the right-sided IPL under the CONG condition (Fig. 4a–d). Although this analysis does not provide direct evidence for anatomical connections or causality, we postulate a cerebro–cerebellar functional coupling, which could be due to contralateral anatomical connections (cerebellar projection to the IPL via the thalamus or projection from the parietal cortex to the cerebellum via the pontine nuclei), as found in nonhuman primates (Sasaki et al. 1977, 1979; Middleton and Strick 1998; Clower et al. 2001; Dum and Strick 2003).
Our observation of IPL activation probably corresponds to human cytoarchitectonic area PF, the largest region of the IPL (Caspers et al. 2006). In nonhuman primates, neurons in this area respond to both visual and somesthetic inputs (Hyvarinen 1982). Damage to the human right IPL disturbs the ability to process the spatial location of a limb (Committeri et al. 2007). On the other hand, a low density of gray matter, particularly in the left posterolateral cerebellum, can cause deficits in visuomotor integration of hand movement in adolescents (Allin et al. 2005). These lines of evidences suggest a critical role for a cerebro–cerebellar interaction between the right parietal (IPL) and left cerebellum in combining visual and kinesthetic information (and for perception). The left cerebellum could be an important locus for combining exteroceptive (visual) and interoceptive (kinesthetic) information to create an online estimate of hand location/displacement during movement (see above). This function may be achieved in collaboration with activity of the right parietal cortex, and this cerebro–cerebellar interaction may mediate the process of progressively updating one's body configuration.
For the cerebral cortex, the language function predominantly engages the left cerebral hemisphere (Wada and Rasmussen 1960; Binder et al. 1997; Springer et al. 1999; Vikingstad et al. 2000). On the other hand, there is emerging evidence supporting the notion that the right cerebral cortices play crucial roles in bodily perception, that is, body image (see references above). This functional lateralization also seems to be extended to the cerebellum through its contralateral anatomical connection with the cerebral cortex. For example, a patient with agenesis of the right cerebellar hemisphere showed difficulty in learning reading and writing in his early childhood (Tavano et al. 2007). As described above (Allin et al. 2005), an anatomical deficit in the left posterolateral cerebellum may cause difficulty in visuomotor integration of hand movement. These clinical observations imply possible functional lateralization in the human cerebellum in relation to its connected cerebral hemisphere. The left cerebellar activation in the present study and its interaction with the right cerebral cortex in combining visuokinesthetic information for perception of limb movement are in good agreement with the proposed role of the right cerebral cortex in “bodily perception.”
In conclusion, our results suggest that the human left cerebellum is an important component of a network that progressively updates one's body image using multisensory information.
Funding
Brain Science Foundation, Japan to EN; the Ministry of Education, Culture, Sports, Science, and Technology (the 21st Century Center of Excellence Program [D-2 to Kyoto University, EN and MM) and Grant-in-Aid for Scientific Research [C], 19500290 to EN), Japan Society for the Promotion of Science (17-2087 to NH).
The authors are grateful to Prof. James Lackner, Dr H. Henrik Ehrsson, Mr Nicholas Cothros, and Mr Satoshi Hirose for their valuable comments on the early version of the manuscript and also to Dr Takanori Kochiyama for his comments on the analysis of data. Conflict of interest: None declared.
References
Author notes
Current address: National Institute of Information and Communication Technology, Research Department 1, Kobe Advanced ICT Research Center, Biophysical ICT Group, ATR Computational Neuroscience Laboratories, 2-2-2 Hikaridai, Seika-cho, Soraku-gun, Kyoto, 619-0288, Japan
