Abstract

Little is known about the neuronal mechanisms underlying the temporal ordering of tactile signals. We examined the brain regions involved in judgments of the temporal order of successive taps delivered to both hands. Participants received identical stimuli while engaging in 2 different tasks: Judging the temporal order and judging the numerosity of points of tactile stimulation. Comparisons of the functional magnetic resonance imaging data obtained during the 2 tasks revealed regions that were more strongly activated with the judgments of the temporal order than with the judgments of numerosity under both arms-uncrossed and -crossed conditions: The bilateral premotor cortices, the bilateral middle frontal gyri, the bilateral inferior parietal cortices and supramarginal gyri, and the bilateral posterior part of the superior and middle temporal gyri. Stronger activation was found in some of these areas that implicated for remapping tactile stimuli to spatial coordinates after the participants crossed their arms. The activation in the perisylvian areas overlapped with the human visual-motion–sensitive areas in the posterior part. Based on these results, we propose that the temporal order of tactile signals is determined by combining spatial representations of stimuli in the parietal and prefrontal cortices with representations of “motion” or “changes” in the multisensory perisylvian cortex.

Introduction

The brain can determine the order of 2 stimuli that are separated in time by 20–50 ms (Hirsh and Sherrick 1961; Pöppel 1997). Recent studies have shed light on the neuronal mechanisms underlying this temporal-order judgment (TOJ). Davis et al. (2009) recently reported that the bilateral temporal parietal junctions (TPJ) were activated during judgments of the temporal order of 2 visual stimuli. Woo et al. (2009) also used a visual TOJ task and showed that transcranial magnetic stimulation (TMS) of the right posterior parietal cortex affected the participants' performance. Bernasconi et al. (2010a, 2010b) demonstrated the importance of activity in the posterior sylvian regions in auditory TOJ by investigating auditory evoked potentials. These studies agree that the posterior sylvian regions are involved in both auditory and visual TOJs. This is consistent with the idea that performance on TOJ tasks is independent of the stimulus modalities (Hirsh and Sherrick 1961; Pöppel 1997) and that a decision center for TOJ is shared across different sensory modalities (Efron 1963; Sternberg and Knoll 1973). However, Fujisaki and Nishida (2009) recently showed that TOJ is not strictly independent of stimulus modalities: The just noticeable difference was smallest for tactile–tactile TOJ and increased in the order of audio-tactile, visuo-tactile, and audio-visual TOJs. Alais and Cass (2010) examined whether improvement in auditory TOJ after training transfers to visual TOJ, or vice versa. They found that improvement in 1 modality did not transfer to the other, disproving the idea that performance on TOJ tasks is independent of the stimulus modalities. Thus, it is important to examine the neural mechanisms of TOJ for each specific combination of stimulus modalities and to discriminate modality-specific mechanisms from those shared across different sensory modalities.

In this study, we examined the brain regions involved in judgments of the temporal order of 2 tactile stimuli, delivered 1 to each hand, in particular because there is accumulated psychophysics literature regarding tactile TOJ (e.g., Hirsh and Sherrick 1961; Yamamoto and Kitazawa 2001a, 2001b; Shore et al. 2002; Roder et al. 2004; Schicke and Roder 2006; Kitazawa et al. 2007). We and other researchers previously reported that crossing the arms caused a reversal of the temporal order of tactile stimuli when they were required to judge which hand was stimulated first (or second) (Yamamoto and Kitazawa 2001a, 2001b; Shore et al. 2002; Wada et al. 2004; Schicke and Roder 2006). In another study, Shibuya et al. (2007) showed that participants were not able to ignore visual distracters that were presented simultaneously with the tactile stimuli, even when participants were instructed to ignore the visual stimuli. These results showed that participants do not base their tactile TOJ solely on a somatotopic map, but rather on an external spatial map on which both visual, tactile, and proprioceptive signals converge (Kitazawa et al. 2007).

Furthermore, given that Shibuya et al. (2007) found that tactile TOJ was affected by visual distracters that elicited “apparent motion”, it is plausible that the area related to the processing of motion information may have cooperative roles in the tactile TOJs. From these results, we previously proposed a “motion-projection hypothesis” (Kitazawa et al. 2007). This hypothesis assumes that successive tactile stimuli evoke a type of motion signal in the motion-related areas of the brain and that these are also related to external spatial representations in the parietal or prefrontal cortex. The order of 2 tactile events is then reconstructed by integrating the motion signal with the spatial locations of the 2 tactile stimuli (Fig. 1). It is worth noting that the hypothesis assumes that motion-related areas are involved in both arms-uncrossed (Fig. 1A) and arms-crossed (Fig. 1B) conditions. The hypothesis explains how inverted motion signals, whether they are generated by visual distracters or successive taps to crossed hands, cause inverted judgments (e.g., Fig. 1B).

Figure 1.

Motion-projection hypothesis. Tactile events are hypothesized to be ordered in time by combining information regarding “what happened where” represented in external spatial coordinates and information regarding “when”. The “when” information is hypothesized to be captured in a motion vector. The brain may compare the vector with the 2 points in the spatial coordinates and then determine which was the start and which was the end so that the temporal order of 2 events is reconstructed. Illustrations show the process of correct judgment in the arm-uncrossed condition (A) and that of inverted judgment in the arm-crossed condition (B). The same networks are hypothesized to be activated in both cases.

Figure 1.

Motion-projection hypothesis. Tactile events are hypothesized to be ordered in time by combining information regarding “what happened where” represented in external spatial coordinates and information regarding “when”. The “when” information is hypothesized to be captured in a motion vector. The brain may compare the vector with the 2 points in the spatial coordinates and then determine which was the start and which was the end so that the temporal order of 2 events is reconstructed. Illustrations show the process of correct judgment in the arm-uncrossed condition (A) and that of inverted judgment in the arm-crossed condition (B). The same networks are hypothesized to be activated in both cases.

The purpose of the present study was to test predictions from the hypothesis that 1) external spatial coordinates, rather than somatotopic coordinates, are involved, and 2) motion-related areas are involved in judgments of the temporal order of 2 tactile stimuli, delivered 1 to each hand. For this purpose, participants received identical tactile stimuli while engaging in 2 different tasks: Judging temporal order or judging numerosity (the number of pins providing the tactile stimulation, NJ task). We then compared the functional magnetic resonance imaging (fMRI) data obtained during the 2 tasks. We used the NJ task, because we found in a preliminary experiment that the NJ did not depend on the arm posture. We also compared data under the 2 arm conditions, because arm crossing would increase the demand for remapping tactile signals to spatial coordinates, and elicit greater activation in such areas as the human homolog of the macaque ventral intraparietal cortices (hVIP) that have been implicated for the process of remapping (Lloyd et al. 2003; Bolognini and Maravita 2007; Azanon, Longo et al. 2010).

To compare distribution of hypothesized motion-related areas in tactile TOJ with visual-motion areas, we added a random-dot motion task. Finally, considering the reported involvement of the posterior perisylvian regions in the TOJ tasks (Davis et al. 2009; Woo et al. 2009; Bernasconi et al. 2010a, 2010b), we compared the perisylvian regions that are involved in tactile TOJ with those of the posterior perisylvian language areas in the same participants.

Materials and Methods

Participants

Twelve males (aged 20–27 years) participated in the study. All participants were neurologically normal and strongly right-handed according to the Edinburgh Inventory (Oldfield 1971). The study received approval from the Institutional Review Board, and all participants provided written informed consent according to the institutional guidelines.

Tasks

Each participant was placed in a magnetic resonance (MR) scanner with his arms uncrossed under 1 condition (arms-uncrossed condition) and with his arms crossed under the other condition (arms-crossed condition). A non-magnetic 8-pin Braille stimulator, driven by piezoelectric actuators for each pin, was developed for the study (KGS Corporation, Saitama, Japan, Fig. 2A). One button for response was located on each stimulator with 8 movable pins in 4 × 2 arrays with inter-pin intervals of 3 mm (Fig. 2A). Two tactile stimuli were delivered to the ventral surface of each participant's index finger by driving some of the pins out so that they protruded from the surface of the button by approximately 0.4 mm. The index finger of each hand was placed on stimulators that were fixed to the thighs of each participant; the distance between the stimulators was about 20 cm. During the experiments, participants were required to close their eyes and have their ears plugged.

Figure 2.

Experimental designs. (A) A set of non-magnetic 8-pin Braille stimulators (left) and an enlarged view, from above, of the stimulus pins on the button switch for the response (right). Eight pins on the button were arranged in a 4 × 2 array with an inter-pin interval of 3 mm. Tactile stimuli were delivered when some of the pins protruded slightly and came into contact with the participant's finger surface located on the button. (B) A part of the task sequence of the TOJ and NJ tasks. Pins that provide stimulation protrude for 1 s, and all pins are then withdrawn (“GO” timing). Each participant responds to a forced-choice question by pushing the “second” button in the TOJ tasks and the button with the greater number of pins in the NJ tasks.

Figure 2.

Experimental designs. (A) A set of non-magnetic 8-pin Braille stimulators (left) and an enlarged view, from above, of the stimulus pins on the button switch for the response (right). Eight pins on the button were arranged in a 4 × 2 array with an inter-pin interval of 3 mm. Tactile stimuli were delivered when some of the pins protruded slightly and came into contact with the participant's finger surface located on the button. (B) A part of the task sequence of the TOJ and NJ tasks. Pins that provide stimulation protrude for 1 s, and all pins are then withdrawn (“GO” timing). Each participant responds to a forced-choice question by pushing the “second” button in the TOJ tasks and the button with the greater number of pins in the NJ tasks.

In each trial, 2 successive stimuli, right then left or left then right, were delivered with a fixed stimulus onset asynchrony (SOA) by 1–7 pins that protruded from each stimulator. The pins used for each stimulation protruded for 1 s and were then retracted (see the “GO” timing in Fig. 2B). After the retractions, each participant responded by pressing a button as quickly as possible.

In the TOJ task, each participant was required to judge the order of the stimuli and respond in a forced-choice manner by pressing the button corresponding to the side that was stimulated last. In the NJ task, the participant was required to judge the number of pins used in each stimulus and respond by pressing the button corresponding to the side that involved the greater number of pins. The reaction time from the GO signal was recorded for each trial.

Under the arms-uncrossed condition, participants engaged in 2 sessions: Once for the TOJ task and once for the NJ task with their arms uncrossed. Each session consisted of 32 trials, and the inter-trial interval was set at 12.5 s (400 s in total). The SOA (50 ms: n = 2; 70 ms: n = 5; 80 ms: n = 1; 100 ms: n = 2; 120 ms: n = 1; 200 ms: n = 1) and the difference in the number of pins (1: n = 1; 2: n = 10; 3: n = 1) were chosen for each participant prior to the fMRI experiment to produce a correct-response rate of approximately 80% to adjust for task difficulty. The same number of pins and the same SOA were used for the TOJ and NJ tasks.

Under the arms-crossed condition, participants were required to cross their left arms over their right arms. In the MR scanner, a cloth was placed between the arms to avoid conductive effects. The participants took part in 2 sessions, once for each of the TOJ and NJ tasks with their arms crossed. All task conditions, except for the position of the arms, were identical to those under the arms-uncrossed condition. The orders of tasks and conditions were balanced among participants.

To examine the relationships between the areas related to TOJs and the areas related to the processing of motion information, that is, putative human visual-motion area V5/middle temporal visual area (MT) and medial superior temporal visual area (MST), we added a random-dot motion task, in which participants were asked to keep looking at a fixation point on a screen through a mirror while random dots moved or remained stationary. Under the “motion” condition, the dots moved at a constant velocity (a few degrees per second), and the direction of motion changed every 6 s (up, down, right, and left). Under the “static” condition, the dots remained stationary for 24 s. The conditions were alternated 3 times.

To further compare the perisylvian areas involved in tactile TOJs with the posterior language areas, we added a listening task (Kansaku et al. 2000). In the task, the participants were instructed to close their eyes and listen attentively to auditory stimuli delivered through headphones for 380 s. The stimulus amplitude, which remained constant across participants, had an average sound pressure level of 97 dB at the distal end of the audio system (Resonance Technology, Northridge, CA, United States of America). Prior to the session, the participants were told that the sound stimuli contained a story, about which several questions would be asked after the experiment. Each experiment consisted of 9 epochs: 5 control epochs alternating with 4 narrative epochs. The first control epoch was 60 s long and all other epochs were 40 s long. The narrative of a short essay (160 s) was delivered during the 4 test epochs (epochs 2, 4, 6, and 8). To exclude activation elicited by processing of voice signals in general, test epochs were replayed in reverse during each control epoch.

Imaging

Functional magnetic resonance imaging data were acquired with a 3.0T MR scanner (GE Medical Systems, Milwaukee, WI, United States of America). Functional images sensitive to the blood oxygen level-dependent (BOLD) contrast (Ogawa et al. 1993) were obtained from a T2* gradient-echo echo-planar imaging pulse sequence with a 200-mm field-of-view, 6-mm slice thickness, 2-mm interslice gap, and a 64 × 64-data matrix. For the TOJ and NJ tasks, 164 image volumes were acquired per session with a repetition time (TR) of 2500 ms, echo time (TE) of 29.8 ms, and flip angle of 70°; for the random-dot motion task, 48 image volumes were acquired with a TR of 4000 ms, TE of 29.8 ms, and flip angle of 90°; for the language task, 90 image volumes were acquired with a TR of 4000 ms, TE of 29.8 ms, and flip angle of 90°. The image volumes covered the entire brain with 18 slices. High-resolution T2-weighted anatomical images of the same slice locations as the functional images were also acquired for each participant using the fast spin-echo pulse sequence with TR of 6000 ms, TE of 180 ms, flip angle of 90°, a 200-mm field-of-view, 6-mm slice thickness, 2-mm interslice gap, and a 256 × 256 data matrix.

Data Analysis

The 2-way analysis of variance (within-subject design) was applied to the mean correct-response rate and the mean reaction time to examine the main effect of the task (TOJ/NJ), the main effect of the arm posture (uncrossed/ crossed), and their interactions.

Functional images were analyzed with statistical parametric mapping software (Friston et al. 1995) (SPM2; Wellcome Department of Cognitive Neurology, London, United Kingdom; Friston et al. 1995) and implemented with Matlab 6.5 (MathWorks, Natick, MA, United States of America). The image processing for each experimental run proceeded in the following 5 steps: 1) Motion correction, 2) slice-timing adjustment, 3) coregistration of the anatomical T2 images with the mean functional images in a run, 4) spatial normalization of all images to the Montreal Neurological Institute (MNI) reference brain, and 5) spatial smoothing with a Gaussian kernel of 12-mm full-width at half-maximum.

The statistical analysis was performed in 2 levels, using a mixed-effects design (Woods 1996; Price and Friston 1997). In the first level, the time series of each participant was analyzed separately as a fixed-effect analysis. Using the model parameters estimated by the least-mean–square method, the resulting set of voxel values for each comparison constituted a statistical parametric map (SPM) of the t statistic. In the second-level analysis, all activations were isolated using a 1-sample t-test of the individual contrast as a random-effect analysis.

Perception of “Motion” and Judgment Reversal

In our motion-projection hypothesis, “motion” was hypothesized to be induced by successive taps in both uncrossed and crossed conditions. We further hypothesized that the degree of judgment reversal in the crossed condition depended on the strength of the inverted “motion”. To test this hypothesis, we recruited 8 new participants (7 males and 1 female) aged 21–38 years (mean 31 years old), who participated in 2 experiments. Each experiment consisted of 200 trials, in each of which they received successive stimuli 1 to each hand from the same Braille stimulators that were used in fMRI experiments with their eyes closed. The number of pins for stimulation was fixed to 4 in both experiments. The SOA for each trial was chosen from 20 SOAs (−1000, −500, −250, −200, −150, −100, −50, −30, −20, −10, +10, +20, +30, … , +1000 ms) in a pseudorandom order (10 times for each SOA in 1 experiment). The positive SOA indicated that the right hand was stimulated first and the negative one indicated the left hand was stimulated first. In 1 experiment, they were asked to judge whether they felt a flow (motion) in a direction from 1 point to the other (1) or not (0). In the other experiment, they were required to judge the order of stimuli. They responded by pushing a button for response by one of their index fingers. They were required to make a response in a forced-choice manner, within 2 s after the delivery of the second stimulus. The orders of the 2 experiments were counterbalanced across the participants.

Results

Behavioral Results in the Scanner

Under the arms-uncrossed condition, we adjusted the SOA (50–200 ms; mean = 88 ms, standard error of the mean [SEM] = 12 ms) and differences in the number of pins (1–3; mean = 2, SEM = 0.1) for each participant so that a correct-response rate of approximately 80% was achieved in both the TOJ and the NJ tasks. Accordingly, the correct-response rates were 81 ± 3% (mean ± SEM) for the TOJ and 83 ± 3% for the NJ tasks (Fig. 3A). Under the arms-crossed condition, the SOA and the pin-number difference were the same as those under the arms-uncrossed condition. The correct-response rate for the TOJ task dropped to 57 ± 5% but that for the NJ tasks was nearly 80% (84 ± 3%). The 2-way analysis of variance (within-subject design) showed that the main effect of the task (TOJ/NJ, F1,11= 22.4, P < 0.001), the main effect of the arm posture (uncrossed/crossed, F1,11= 11.0, P < 0.001), and their interactions (F1,11= 20.6, P < 0.001) were significant. Post hoc tests of simple main effects showed that the mean correct-response rate under the arms-crossed condition in the TOJ task (TOJ crossed) was significantly smaller than that under the arms-uncrossed condition in the TOJ task (P < 0.001) and smaller than that under the arms-crossed condition in the NJ task (P < 0.001, Fig. 3A).

Figure 3.

Behavioral results. (A) Correct-response rates for TOJ and NJ tasks. Scores for the TOJ task under the arms-crossed condition were significantly lower than that of the scores under other task conditions. (B) Reaction times for the TOJ and NJ tasks. The reaction times were measured from “GO” to the initiation of pressing the button. Reaction times were significantly longer when the arms were crossed than when they were not crossed.

Figure 3.

Behavioral results. (A) Correct-response rates for TOJ and NJ tasks. Scores for the TOJ task under the arms-crossed condition were significantly lower than that of the scores under other task conditions. (B) Reaction times for the TOJ and NJ tasks. The reaction times were measured from “GO” to the initiation of pressing the button. Reaction times were significantly longer when the arms were crossed than when they were not crossed.

For the mean reaction time (Fig. 3B), the 2-way analysis of variance showed that the main effect of the arm posture was significant (F1,11= 7.8, P = 0.017) but not the main effect of the task (F1,11= 1.9, P = 0.20) or their interaction (F1,11= 2.0, P = 0.19). Thus the mean reaction time under the arms-crossed condition (TOJ crossed or NJ crossed, 527 ± 53 ms; mean ± SEM) was significantly longer than those under the arms-uncrossed condition (TOJ uncrossed or NJ uncrossed, 455 ± 45 ms).

The results show that arm crossing caused judgment reversals in the TOJ task, but not in the NJ task. In addition, arm crossing increased reaction time. The behavioral results in the TOJ task were basically similar to those reported previously (Yamamoto and Kitazawa 2001a, 2001b; Shore et al. 2002).

TOJ versus NJ Contrast

Under both arms-uncrossed and -crossed conditions, greater activation was elicited bilaterally in the following areas during the TOJ task compared with the NJ task (Fig. 4A). The bilateral premotor cortices extending to the posterior part of the middle frontal gyri (BA 6/9), the bilateral middle frontal gyri (BA 46), the bilateral inferior parietal cortices and supramarginal gyri (BA 40), the bilateral posterior part of the superior and the middle temporal gyri (BA 21/22), the inferior part in the right anterior insula, and the bilateral caudate. Details are presented in Table 1.

Table 1

Regions showing increased activity during the TOJ task compared with the NJ task under the arms-uncrossed and -crossed conditions (conjunction analysis)

Region L/R BA Peak coordinates
 
Z-value 
Talairach
 
MNI
 
x y z x y z 
Precentral gyrus/middle frontal gyrus 6/9 −38 −1 40 −40 40 3.23 
Middle frontal gyrus/inferior frontal gyrus 46/10 −40 42 15 −42 48 3.08 
Insula/inferior frontal gyrus – 25 27 28 30 −6 3.03 
Posterior superior temporal gyrus/middle temporal gyrus/inferior parietal lobule/supramarginal gyrus 21/22
39/40 
−49 −47 −52 −48 10 2.95 
Middle temporal gyrus/posterior superior temporal gyrus 21/22 53 −38 58 −38 2.89 
Supramarginal gyrus 40 56 −39 36 62 −36 38 2.87 
Precentral gyrus/middle frontal gyrus 6/9 37 38 42 12 36 2.71 
Caudate – −16 17 −16 10 14 2.64 
Cerebellum (tonsil) – −14 −67 −38 −14 −74 −40 2.52 
Caudate – 13 10 16 2.43 
Cerebellum (tonsil) – 14 −62 −32 16 −68 −34 2.40 
Cerebellum (lingual) – −1 −48 −18 −52 −20 2.37 
Inferior parietal lobule 40 −37 −45 41 −38 −42 46 2.35 
Middle frontal gyrus 9/46 28 32 26 32 38 20 2.16 
Insula – −32 −8 −10 −34 −8 −14 2.10 
Thalamus – 21 −20 19 24 −18 18 2.08 
Anterior cingulate 32 −14 29 21 −14 34 16 1.76 
Region L/R BA Peak coordinates
 
Z-value 
Talairach
 
MNI
 
x y z x y z 
Precentral gyrus/middle frontal gyrus 6/9 −38 −1 40 −40 40 3.23 
Middle frontal gyrus/inferior frontal gyrus 46/10 −40 42 15 −42 48 3.08 
Insula/inferior frontal gyrus – 25 27 28 30 −6 3.03 
Posterior superior temporal gyrus/middle temporal gyrus/inferior parietal lobule/supramarginal gyrus 21/22
39/40 
−49 −47 −52 −48 10 2.95 
Middle temporal gyrus/posterior superior temporal gyrus 21/22 53 −38 58 −38 2.89 
Supramarginal gyrus 40 56 −39 36 62 −36 38 2.87 
Precentral gyrus/middle frontal gyrus 6/9 37 38 42 12 36 2.71 
Caudate – −16 17 −16 10 14 2.64 
Cerebellum (tonsil) – −14 −67 −38 −14 −74 −40 2.52 
Caudate – 13 10 16 2.43 
Cerebellum (tonsil) – 14 −62 −32 16 −68 −34 2.40 
Cerebellum (lingual) – −1 −48 −18 −52 −20 2.37 
Inferior parietal lobule 40 −37 −45 41 −38 −42 46 2.35 
Middle frontal gyrus 9/46 28 32 26 32 38 20 2.16 
Insula – −32 −8 −10 −34 −8 −14 2.10 
Thalamus – 21 −20 19 24 −18 18 2.08 
Anterior cingulate 32 −14 29 21 −14 34 16 1.76 

BA, Brodmann areas; L/R, left/right hemisphere.

P < 0.01 (Z > 2.33); P < 0.05 (Z > 1.64), uncorrected.

Figure 4.

Brain areas activated by TOJ. The figure shows SPMs obtained with a conjunction analysis (conjunction null) for ([TOJ uncrossed > NJ uncrossed] × [TOJ crossed > NJ crossed]). The results are from a 12-participant group analysis at P < 0.05 and <0.01 (uncorrected). The level of P = 0.05 for the false discovery rate (FDR) is also shown in the color scale. When we compared the data obtained during the TOJ and NJ tasks, we used inclusive masking with the contrast at the TOJ task (P < 0.001, uncorrected). The numbers below each transverse image refer to MNI z-plane coordinates. The x signs on the transverse image (z = + 42) represent the location of the right human homolog of the monkey VIP ([26, −54, 42] reported in Lloyd et al. 2003) and the corresponding location in the left hemisphere.

Figure 4.

Brain areas activated by TOJ. The figure shows SPMs obtained with a conjunction analysis (conjunction null) for ([TOJ uncrossed > NJ uncrossed] × [TOJ crossed > NJ crossed]). The results are from a 12-participant group analysis at P < 0.05 and <0.01 (uncorrected). The level of P = 0.05 for the false discovery rate (FDR) is also shown in the color scale. When we compared the data obtained during the TOJ and NJ tasks, we used inclusive masking with the contrast at the TOJ task (P < 0.001, uncorrected). The numbers below each transverse image refer to MNI z-plane coordinates. The x signs on the transverse image (z = + 42) represent the location of the right human homolog of the monkey VIP ([26, −54, 42] reported in Lloyd et al. 2003) and the corresponding location in the left hemisphere.

As expected, some of these regions (e.g., the bilateral premotor cortices and the bilateral inferior parietal cortices) are closely related to spatial coordinates as discussed later in detail. To further characterize the activation in the bilateral middle temporal gyri, we tested the overlap between motion-related brain regions that were activated by the moving random dots. The activation around the middle temporal gyri (Fig. 5B, red) overlapped with more of the anterior parts of the regions showing greater activation during the presentation of moving compared with static dots (Fig. 5B, blue). The regions that were activated in response to the moving random dots corresponded to the standard locations of the human V5/MT and MST reported by previous imaging studies (Watson et al. 1993; Dukelow et al. 2001; Orban et al. 2003; Kolster et al. 2010). These areas were distinct from the posterior language areas (Fig. 5A,B, green).

Figure 5.

Brain areas activated by TOJ (red), random-dot motion (blue), and a listening task (green). (A) Lateral views. (B and C) Activations on 2 transverse planes at z = + 6 mm (B) and +42 mm (C). Activation maps are shown in red for the TOJ > NJ contrast (the same conjunction analysis as in Fig. 4), in blue for the motion > no motion contrast during the task involving moving random dots, and in green for the forward replay > reverse replay contrast during the narrative listening task. All contrasts were thresholded at P = 0.05 (uncorrected). White represents commonly activated area in TOJ > NJ overlapped with the area activated by the moving random dots. Note the overlap in the posterior regions of the bilateral middle temporal gyri (B), and in the bilateral dorsal premotor cortices and in the bilateral inferior parietal cortices (C). The x signs in (B) represent the standard locations of the human V5/MT (a: [−46, −66, 8] (Dukelow et al. 2001), b: [−46, −78, 6] (Kolster et al. 2010), c: [−55, −74, 4] (Orban et al. 2003), and d: [−46, −73, 3] (Watson et al. 1993) in the MNI coordinates). The yellow dots in (B) represent reported peak activations in response to biological motion stimuli (1: [−55, −21, 7], [61, −24, 6] (lip-reading, Calvert et al. 1997), 2: [−46, −71, 8], [47, −59, 9] (gaze, Wicker et al. 1998), 3: [−48, −55, 7], [54, −50, 3] (gaze, Puce et al. 1998), 4: [−50, −52, 4], [55, −50, 3] (mouth movement, Puce et al. 1998), 5: [−47, −57, 14], [55, −65, 5] (gaze, Hoffman and Haxby 2000), and 6: [51, −36, 5] (hand movement, Grezes et al. 1999). The x signs in (C) represent the same locations in the human VIP as in Figure 4.

Figure 5.

Brain areas activated by TOJ (red), random-dot motion (blue), and a listening task (green). (A) Lateral views. (B and C) Activations on 2 transverse planes at z = + 6 mm (B) and +42 mm (C). Activation maps are shown in red for the TOJ > NJ contrast (the same conjunction analysis as in Fig. 4), in blue for the motion > no motion contrast during the task involving moving random dots, and in green for the forward replay > reverse replay contrast during the narrative listening task. All contrasts were thresholded at P = 0.05 (uncorrected). White represents commonly activated area in TOJ > NJ overlapped with the area activated by the moving random dots. Note the overlap in the posterior regions of the bilateral middle temporal gyri (B), and in the bilateral dorsal premotor cortices and in the bilateral inferior parietal cortices (C). The x signs in (B) represent the standard locations of the human V5/MT (a: [−46, −66, 8] (Dukelow et al. 2001), b: [−46, −78, 6] (Kolster et al. 2010), c: [−55, −74, 4] (Orban et al. 2003), and d: [−46, −73, 3] (Watson et al. 1993) in the MNI coordinates). The yellow dots in (B) represent reported peak activations in response to biological motion stimuli (1: [−55, −21, 7], [61, −24, 6] (lip-reading, Calvert et al. 1997), 2: [−46, −71, 8], [47, −59, 9] (gaze, Wicker et al. 1998), 3: [−48, −55, 7], [54, −50, 3] (gaze, Puce et al. 1998), 4: [−50, −52, 4], [55, −50, 3] (mouth movement, Puce et al. 1998), 5: [−47, −57, 14], [55, −65, 5] (gaze, Hoffman and Haxby 2000), and 6: [51, −36, 5] (hand movement, Grezes et al. 1999). The x signs in (C) represent the same locations in the human VIP as in Figure 4.

Arms-Crossed versus Arms-Uncrossed Contrasts

During the TOJ task, greater activation was elicited in the following areas under the arms-crossed condition compared with the arms-uncrossed condition (Fig. 6): The left dorsal premotor cortex (BA 6), the left dorsolateral prefrontal cortex (BA 9/44), the dorsal part of the left supramarginal gyrus (BA 40), the left superior parietal cortex (BA 7), and the right middle temporal gyrus (BA 21). Details are presented in Table 2.

Table 2

Regions showing increased activation during the TOJ under arm-crossed condition compared with the TOJ task under the arms-uncrossed condition

Region L/R BA Peak coordinates
 
Z-value 
Talairach
 
MNI
 
x y z x y z 
Middle frontal gyrus −22 −3 52 −22 54 3.20 
Dorsolateral prefrontal cortex 9/44 −49 28 −52 14 26 2.85 
Supramarginal gyrus 40 −29 −44 37 −30 −42 42 2.76 
Middle temporal gyrus 21 41 −39 46 −40 2.72 
Superior parietal lobule/inferior parietal lobule 7/40 −31 −57 54 −32 −54 62 2.40 
Middle temporal gyrus 21 −44 −44 −1 −46 −46 2.39 
Insula – 27 23 30 26 −8 1.96 
Supramarginal gyrus 40 54 −39 39 60 −36 42 1.91 
Middle frontal gyrus 41 11 29 46 16 26 1.83 
Insula 13 42 −8 46 −6 1.76 
Insula − −31 −2 −32 1.68 
Postcentral gyrus 2/3 −57 −14 38 −60 −10 40 1.67 
Region L/R BA Peak coordinates
 
Z-value 
Talairach
 
MNI
 
x y z x y z 
Middle frontal gyrus −22 −3 52 −22 54 3.20 
Dorsolateral prefrontal cortex 9/44 −49 28 −52 14 26 2.85 
Supramarginal gyrus 40 −29 −44 37 −30 −42 42 2.76 
Middle temporal gyrus 21 41 −39 46 −40 2.72 
Superior parietal lobule/inferior parietal lobule 7/40 −31 −57 54 −32 −54 62 2.40 
Middle temporal gyrus 21 −44 −44 −1 −46 −46 2.39 
Insula – 27 23 30 26 −8 1.96 
Supramarginal gyrus 40 54 −39 39 60 −36 42 1.91 
Middle frontal gyrus 41 11 29 46 16 26 1.83 
Insula 13 42 −8 46 −6 1.76 
Insula − −31 −2 −32 1.68 
Postcentral gyrus 2/3 −57 −14 38 −60 −10 40 1.67 

Other details as in Table 1.

P < 0.01 (Z > 2.33); P < 0.05 (Z > 1.64), uncorrected.

Figure 6.

Brain areas activated by arm crossing during the TOJ task. Activation map shows the TOJ crossed > TOJ uncrossed contrast. The results are of a 12-participant group analysis at P < 0.05 and <0.01 (uncorrected). Other conventions are the same as in Figure 4. No voxels reached the FDR of 0.05.

Figure 6.

Brain areas activated by arm crossing during the TOJ task. Activation map shows the TOJ crossed > TOJ uncrossed contrast. The results are of a 12-participant group analysis at P < 0.05 and <0.01 (uncorrected). Other conventions are the same as in Figure 4. No voxels reached the FDR of 0.05.

During the NJ task, the arms-crossed versus arms-uncrossed contrast yielded activations in the same areas (BA 6, 9, 44, 40, and 21) but more extensively and bilaterally (Fig. 7, Table 3). However, it is worth noting that none of these areas were activated stronger in the NJ task than in the TOJ task under the arms-crossed condition (P < 0.05, uncorrected). On the contrary, many of these areas were activated even stronger during the TOJ task when compared with the NJ task, as shown by the results of conjunction analysis in Figure 4 ([TOJ crossed > NJ crossed] × [TOJ uncrossed > NJ uncrossed]).

Table 3

Regions showing increased activation during the NJ under arm-crossed condition compared with the NJ task under the arms-uncrossed condition

Region L/R BA Peak coordinates
 
Z-value 
Talairach
 
MNI
 
x y z x y z 
Medial frontal gyrus/superior frontal gyrus 6/9 −16 55 −16 10 56 4.31 
Middle temporal gyrus/superior temporal gyrus 21/22 −42 −58 −44 −60 12 4.17 
Middle temporal gyrus/superior temporal gyrus 21/22 52 −54 14 58 −54 16 4.08 
Medial frontal gyrus/superior frontal gyrus 6/9 28 56 32 12 56 3.00 
Superior parietal gyrus/supramarginal gyrus 39/40 −26 −72 26 −26 −72 32 3.89 
Middle frontal gyrus/inferior frontal gyrus 45/46 −40 32 −42 36 3.85 
Inferior frontal gyrus/insula 47 36 27 40 30 −6 3.83 
Posterior cingulate 29/30 −9 −53 12 −8 −54 14 3.40 
Cerebellum (vermis) – −63 −41 −70 −44 3.37 
Posterior cingulate 29/30 10 −48 13 12 −48 14 3.30 
Medial frontal gyrus −11 26 43 −10 34 40 3.22 
Inferior parietal lobule/supramarginal gyrus 39/40 34 −45 40 38 −42 44 3.20 
Insula 22 45 −9 −3 50 −8 −8 3.15 
Cerebellum (tonsil) – −17 −55 −30 −18 −60 −32 3.06 
Precuneus 31 −69 19 −70 24 2.81 
Middle frontal gyrus −38 37 34 −40 44 30 2.78 
Precuneus 21 −61 30 24 −60 34 2.76 
Middle frontal gyrus/inferior frontal gyrus 45/46 51 17 27 56 22 22 2.72 
Superior occipital gyrus 19 34 −72 29 38 −72 34 2.65 
Region L/R BA Peak coordinates
 
Z-value 
Talairach
 
MNI
 
x y z x y z 
Medial frontal gyrus/superior frontal gyrus 6/9 −16 55 −16 10 56 4.31 
Middle temporal gyrus/superior temporal gyrus 21/22 −42 −58 −44 −60 12 4.17 
Middle temporal gyrus/superior temporal gyrus 21/22 52 −54 14 58 −54 16 4.08 
Medial frontal gyrus/superior frontal gyrus 6/9 28 56 32 12 56 3.00 
Superior parietal gyrus/supramarginal gyrus 39/40 −26 −72 26 −26 −72 32 3.89 
Middle frontal gyrus/inferior frontal gyrus 45/46 −40 32 −42 36 3.85 
Inferior frontal gyrus/insula 47 36 27 40 30 −6 3.83 
Posterior cingulate 29/30 −9 −53 12 −8 −54 14 3.40 
Cerebellum (vermis) – −63 −41 −70 −44 3.37 
Posterior cingulate 29/30 10 −48 13 12 −48 14 3.30 
Medial frontal gyrus −11 26 43 −10 34 40 3.22 
Inferior parietal lobule/supramarginal gyrus 39/40 34 −45 40 38 −42 44 3.20 
Insula 22 45 −9 −3 50 −8 −8 3.15 
Cerebellum (tonsil) – −17 −55 −30 −18 −60 −32 3.06 
Precuneus 31 −69 19 −70 24 2.81 
Middle frontal gyrus −38 37 34 −40 44 30 2.78 
Precuneus 21 −61 30 24 −60 34 2.76 
Middle frontal gyrus/inferior frontal gyrus 45/46 51 17 27 56 22 22 2.72 
Superior occipital gyrus 19 34 −72 29 38 −72 34 2.65 

Other details as in Table 1.

P < 0.01 (Z > 2.33); P < 0.05 (Z > 1.64), uncorrected.

Figure 7.

Brain areas activated by arm crossing during the NJ task. Activation map shows the NJ crossed > NJ uncrossed contrast. The results are of a 12-participant group analysis at P < 0.05 and <0.01 (uncorrected). Other conventions are the same as in Figure 4.

Figure 7.

Brain areas activated by arm crossing during the NJ task. Activation map shows the NJ crossed > NJ uncrossed contrast. The results are of a 12-participant group analysis at P < 0.05 and <0.01 (uncorrected). Other conventions are the same as in Figure 4.

Perception of “Motion” and Judgment Reversal

Eight newly recruited participants felt a sense of “motion” in >90% of trials when they received successive stimuli with an SOA of 100 ms (uncrossed condition; open circles in Fig. 6A) and 150 ms (crossed condition, triangles). The sense of “motion” decreased at shorter and longer SOAs. A level of 50% was achieved at 30 ms and somewhere between 250–500 ms (Fig. 6A). At 500 ms, the ratio dropped below 30% and further to <20% at 1000 ms in both conditions. The results show that the participants felt a sense of “motion” irrespective of whether the arms were crossed or uncrossed and that it peaked at SOAs of 100–200 ms.

The same participants reported inverted judgments when the arms were crossed (Fig. 6B, crosses), as indicated by smaller correct-response rates than those in the arm-uncrossed condition (open squares). The degree of judgment reversal, quantified by the difference of the rate of correct responses in the 2 arm postures, almost paralleled the strength of the sense of “motion”: The difference peaked around 100–250 ms and decreased thereafter toward longer SOAs of 500 and 1000 ms.

Discussion

We compared the fMRI data obtained during the TOJ and NJ tasks in the arms-uncrossed and arms-crossed conditions and identified the following regions that were more prominent in the TOJ task: The bilateral premotor cortices extending to the posterior part of the middle frontal gyri (BA 6/9), the bilateral middle frontal gyri (BA 46), the bilateral inferior parietal cortices and supramarginal gyri (BA 40), the bilateral posterior part of the superior and the middle temporal gyri (BA 21/22), the inferior part in the right anterior insula, and the bilateral caudate.

In the following sections, we discuss 4 points: 1) The involvement of external spatial coordinates, 2) the involvement of “motion” areas, and 3) effects of arm-crossing; finally, we discuss the validity and limitations of the motion-projection hypothesis regarding tactile TOJs.

Involvement of External Spatial Coordinates

Previous studies showed that crossing the arms caused a reversal of the subjective temporal order of 2 tactile stimuli, delivered 1 to each hand (Yamamoto and Kitazawa 2001a, 2001b; Shore et al. 2002; Wada et al. 2004; Schicke and Roder 2006). From those results, we suggested that it is not until the signals from skin receptors are referred to the hand positions in space that the signals are ordered in time (Yamamoto and Kitazawa 2001a, 2001b; Kitazawa et al. 2007). We first discuss whether the activations in the bilateral premotor cortex (BA 6) and the bilateral inferior parietal cortices (BA 40) extending dorsally to the intraparietal sulcus (TOJ uncrossed > NJ uncrossed and TOJ crossed > NJ crossed, Fig. 4) imply hypothesized involvement of external spatial coordinates.

Electrophysiological experiments in macaques have described neuronal populations that respond to tactile and visual stimuli and encode the location of visual stimuli in the body-centered reference frames in the ventral intraparietal area (Duhamel et al. 1998; Avillac et al. 2005), and in the premotor cortex (Graziano et al. 1994; Graziano and Gross 1998; Pesaran et al. 2006).

According to previous human imaging studies, the inferior parietal lobule has been implicated in somatosensory spatial discrimination (Akatsuka et al. 2008) and integration of tactile and proprioceptive information (Milner et al. 2007). In a recent fMRI study with human participants, the anterior part of the intraparietal sulcus, the inferior parietal lobule (supramarginal gyrus), and the dorsal and ventral portions of the premotor cortex exhibited selective BOLD adaptation to an object moving near the right hand (Brozzoli et al. 2011).

Azañon, Longo et al. (2010) provided a more direct evidence for the contribution of the posterior parietal cortex to localizing tactile stimuli in external space. The authors showed that single-pulse TMS delivered to the right hVIP (indicated by a cross in the right hemisphere in Figs 4A and 5B) impaired judgment of the height of touch to the left arm. The center of stimulation was close (within a few millimeters) to the activation in the parietal lobule we found in the TOJ versus NJ contrast.

Taken together, we suggest that the premotor and/or the inferior parietal areas are involved in the process of localizing tactile stimuli in the body- or hand-centered reference frames. Activation of these areas, as revealed by the conjunction of TOJ > NJ contrasts, suggests that body- or hand-centered spatial coordinates, or at least the process of coordinate conversion, are involved in tactile TOJ. This basically agrees with the involvement of spatial coordinates as predicted by the motion-projection hypothesis, though it does not prove the hypothesis.

Involvement of the “Motion” Areas

In an additional experiment outside the scanner, we confirmed that participants felt a sense of “motion” in >70% of trials when 2 stimuli were delivered 1 to each hand with SOAs of 50–200 ms. The sense of “motion” occurred irrespective of whether the arms were crossed or uncrossed (Fig. 8A), even though the participants closed their eyes. Because the SOAs used in the scanner (50–200 ms) fell in this range, the same sense of “motion” would have occurred during the TOJ task in the scanner.

Figure 8.

Sensation of “motion” evoked by successive tactile stimuli delivered one to each hand. (A) The rate of trials in which participants felt a sense of “motion” is plotted against the SOA. Note the similarity of data in the arms-crossed (triangles) and arms-uncrossed (open circles) conditions. (B) Judgment reversal due to arm crossing. The rate of correct response in the tactile TOJ is plotted against the SOA. Squares show data in the arms-uncrossed condition and crosses show those in the arms-crossed condition. A dotted line shows the difference between the arms-uncrossed and the arms-crossed conditions. (A and B) Each symbol shows the mean of 160 trials (20 trials × 8 participants; right-hand–first and left-hand–first stimuli are combined for the SOAs with the same absolute value). Error bars show the SEM.

Figure 8.

Sensation of “motion” evoked by successive tactile stimuli delivered one to each hand. (A) The rate of trials in which participants felt a sense of “motion” is plotted against the SOA. Note the similarity of data in the arms-crossed (triangles) and arms-uncrossed (open circles) conditions. (B) Judgment reversal due to arm crossing. The rate of correct response in the tactile TOJ is plotted against the SOA. Squares show data in the arms-uncrossed condition and crosses show those in the arms-crossed condition. A dotted line shows the difference between the arms-uncrossed and the arms-crossed conditions. (A and B) Each symbol shows the mean of 160 trials (20 trials × 8 participants; right-hand–first and left-hand–first stimuli are combined for the SOAs with the same absolute value). Error bars show the SEM.

The activation around the middle temporal gyri overlapped with the most anterior parts of the regions activated by random dots in motion (Fig. 5B), which included the standard locations of the human V5/MT and MST (Watson et al. 1993; Dukelow et al. 2001; Orban et al. 2003; Kolster et al. 2010). The finding agrees with a previous report that the human MT can be activated by local tactile motion stimuli (Hagen et al. 2002). Considering a previous report that tactile motion generated by successive stimuli delivered 1 to each hand interacted with auditory motion (Soto-Faraco et al. 2004a, 2004b), the area of activation in the middle temporal gyri might receive auditory input as well.

We suggest 2 reasons why the activation around the middle temporal gyri lied anterior to the peak coordinates of the human V5/MT and MST. Firstly, apparent motion evoked by successive taps to the 2 hands separated by about 20° would activate neurons in MST with large receptive fields, but not those in the MT with much smaller receptive fields within the contralateral hemifield. This would partly explain why we observed activation anteriorly, because MST lies anterior to MT (Smith et al. 2006). Secondly, activation from TOJ extended anteriorly to involve regions around the superior temporal sulcus that respond to biological motion stimuli. Peaks of activation from biological motion actually fell in or near to the activation from TOJ (Fig. 5B, 1–6). It is thus likely that the activation from TOJ involved areas that represent body-related motion.

It is also worth noting that the aforementioned “space” related areas (the bilateral dorsal premotor areas and the dorsal part of the bilateral supramarginal gyri) were also activated by the motion stimuli (Fig. 5C). Thus the motion signal hypothesized in our model could be represented over these areas including the human V5/MT and MST, the bilateral dorsal premotor areas, and the dorsal part of the bilateral supramarginal gyri.

Although the participants in the present study closed their eyes during the task, the results clearly show that successive taps generated a type of “motion” signal in the brain where both tactile and visual signals converge. The results support the second prediction from the motion-projection hypothesis that motion-related areas are involved in the tactile TOJ.

It should be noted, however, that the “motion” does not necessarily mean a smooth apparent motion that cannot be distinguished from real continuous motion, because the tactile stimuli to the 2 hands were separated by 20 cm. It is reported that smooth and continuous tactile apparent motion can be elicited by placing 4 or more stimulators with a separation of 5 mm along a finger (Kirman 1974). Obviously, the condition in the present study was not the same as the optimal condition. Still, the stimuli evoked a sensation that there appears to be a single event traveling from 1 point to the other. The sense of “motion” in our present study seems be comparable to that in the visual domain reported by Soto-Faraco et al (2004a, 2004b). In the study, the authors used 2 light emitting diode separated by 30 cm at a distance of 40 cm (separated by 40° in the visual angle). Participants rated the quality of “apparent motion” on a scale of 1–5 in their study (Soto-Faraco et al. 2004a, 2004b), and the peak of the mean rating score (>4) peaked at a range of SOA from 50 to 200 ms as we found for the tactile apparent “motion” in the present study.

It may be argued that the activity in the motion area was a mere by-product and did not contribute to tactile TOJs. However, Shibuya et al. (2007) previously reported that concurrently presented visual stimuli that elicited apparent motion greatly affected tactile TOJs. These findings strongly suggest that we cannot avoid taking motion signals into account when we judge the order of tactile stimuli. Based on these previous findings, we infer that the activation in the middle temporal gyri played a key role in the tactile TOJs by providing a type of “motion” signal.

Effects of Arm-Crossing

During the TOJ task as well as the NJ task, greater activation was elicited under the arms-crossed condition compared with the arms-uncrossed condition. The contrast was much stronger during the NJ task (Fig. 7) when compared with the TOJ task (Fig. 6), but the yielded areas involved basically similar regions such as the dorsal premotor cortex (BA 6), the left dorsolateral prefrontal cortex (BA 9/44), the dorsal part of the left supramarginal gyrus (BA 40), the left superior parietal cortex (BA 7), and the right middle temporal gyrus (BA 21).

Lloyd et al. (2003) reported that the hVIP (x = + 26, y = − 54, z = + 42) was activated stronger by tactile stimuli to the right hand when the hand was crossed over the midline (versus uncrossed) with the eyes closed. The right hVIP, which was later shown essential for the remapping of crossed hand positions to the visual coordinates (Bolognini and Maravita 2007), was actually activated more strongly in the NJ-crossed condition when compared with the NJ-uncrossed condition (Fig. 7).

It is also worth noting that the counterpart in the left hVIP was activated in the TOJ-crossed versus TOJ-uncrossed contrast (Fig. 6) as well as in the NJ-crossed versus NJ-uncrossed contrast (Fig. 7). This may indicate that not only the right but also the left hVIP is contributing to the remapping of crossed hand positions to the visual coordinates. We should further note that the left hVIP region was activated more strongly under the TOJ arms-crossed condition when compared with the NJ arms-crossed condition (Fig. 4). Thus, the activation in the left hVIP was the largest in the TOJ arms-crossed condition among the 4 task conditions. Taken together, the resource for tactile remapping may be additively recruited by demands arising from the arm posture (arms crossed > arms uncrossed) as well as those from the target of judgment (TOJ > NJ), even though the remapping process is automatically triggered whenever the hand is touched (Kitazawa 2002; Azanon, Camacho et al. 2010).

Validity and Limitations of the Motion-Projection Hypothesis

The motion-projection hypothesis assumes that successive tactile stimuli evoke a type of motion signal in the motion-related areas of the brain and that these are also related to spatial representations in the brain. The order of 2 tactile events is then reconstructed by integrating the motion signal with the spatial locations of the 2 tactile stimuli so that the judgment is made regarding which hand was stimulated first and second (Fig. 1). As we have discussed so far, the present results agreed with 2 basic predictions of the hypothesis. Stronger activations in the parietal and prefrontal cortices when compared with the control NJ task agreed with the first prediction that external spatial coordinates, rather than somatotopic coordinates, are involved. Activation of the middle temporal gyri that overlapped with visual-motion areas supported the second prediction that motion-related areas are involved.

It may be further argued that processing of change or motion is sufficient for ordering 2 events in time, but this is not true. Once the change between 2 events is encoded, the information about the 2 original events should be lost in these areas. To judge the temporal order of 2 events, the brain has to keep information about the 2 events in other areas. It is as if a vector is represented in the TPJ, and the 2 points are represented in the spatial coordinates: One is the start and the other is the end. The brain may compare the vector with the 2 points in the spatial coordinates and then determine which was the start and which was the end so that the temporal order of 2 events is reconstructed.

Craig (2003) and Craig and Busey (2003) examined TOJs of 2 tactile stimuli presented to 2 fingerpads, when each of 2 stimuli simulated a local motion on the skin. They found that the judgments were affected by the local tactile motions though the participants were told to ignore local motions: When the direction of the local motions was consistent with the global motion defined by the 2 successive stimuli, the judgments were biased toward the correct judgment. On the other hand, when the local motion was directed in reverse, the judgments were biased toward the incorrect judgment. When the local motion was neutral, judgments were not affected. These results clearly show that local tactile motions, which should be represented initially in the somatotopic coordinates, affect the judgment only after the motions are remapped to spatial coordinates where the spatial locations of the 2 fingers are represented. This is again consistent with the motion-projection hypothesis in that motion signals affected the judgments in spatial coordinates.

Davis et al. (2009) recently investigated visual TOJ using fMRI and found activations in the left inferior parietal cortex, the premotor cortices, and the inferior frontal gyri, which generally agreed with the present tactile TOJ results. They specifically argued that the TPJ plays a key role in temporal processing in the brain as a part of the “when” pathway (Spierer et al. 2009). The TPJ in their study involved BA 21, 22, 41, 42, and 48 and almost matched the areas of activation discussed in the 2 previous sections. These TPJ areas receive inputs from different sensory modalities (Macaluso and Driver 2001; Matsuhashi et al. 2004; Kansaku et al. 2006) and play a role in tactile TOJ. Therefore, our results strongly support the TPJ hypothesis that the TPJ has a key role in temporal processing in the brain.

Neural networks activated during TOJ (e.g., the TPJ and premotor cortex) partly overlapped with those of the ventral attention network (Corbetta and Shulman 2002), which is thought to be specialized for the detection of salient or unexpected stimuli under conditions of stimulus-driven attention. Davis et al. (2009) suggested that temporal processing requires greater attention to external stimuli and that this explains the activation of the network in TOJ; however, the role of this network may not be explicable simply in terms of attention, given that we matched the difficulty of the TOJ and NJ tasks under the arms-uncrossed condition. As we discussed, we infer that the TPJ areas contribute to the detection and coding of any rapid “changes” in multisensory signals. Assuming this, we do not think that the TPJ is activated in TOJ due to its role in attention, but rather that the changes encoded in the TPJ areas are essential for both attention and TOJ.

Results in the present study generally supported, or at least did not contradict with, predictions from the motion-projection hypothesis. However, we should note several limitations. Firstly, the model applies to tactile TOJ in which participants are required to judge which hand was stimulated first (or second). Roberts and Humphrey (2008) reported that crossing the arms had no effect on tactile TOJ when order was judged by the frequency or duration of the tactile stimuli, in other words, by the spectro-temporal characteristics of these stimuli. A different model should be developed for tactile TOJ that does not require discrimination of one hand from the other. Secondly, the fMRI technology is a tool for exploring the brain regions that may be involved in the experimental tasks. Their functional relevance should be tested in the next step. For example, involvement of the “motion area” around the middle temporal cortex might merely reflect a correlate of the perceived motion, but be entirely independent of TOJ computation. Thirdly, the process of integrating information in the spatial coordinates and motion signals is hypothetical and surely requires further elucidation. Finally, the direction of the motion signals that plays an essential role in the judgment reversal was not discriminated by the BOLD contrast. One possible next step for investigating their functional relevance is to add TMS to the areas detected in the fMRI experiments (Hallett 2007).

Funding

This work was supported by Grants-in-Aid for Scientific Research (A) #18200024 and #21240029 to S.K., Grants-in-Aid for Scientific Research (B) #23300151 to K.K., Grants-in-Aid for Exploratory Research #23650220 to T.T. and #21650062 to K.K. from the Ministry of Education, Culture, Sports, Science and Technology (MEXT), and by a grant from the High-tech Research Center Program at Juntendo University.

Notes

Conflict of Interest: None declared.

References

Akatsuka
K
Noguchi
Y
Harada
T
Sadato
N
Kakigi
R
Neural codes for somatosensory two-point discrimination in inferior parietal lobule: an fMRI study
Neuroimage
 , 
2008
, vol. 
40
 (pg. 
852
-
858
)
Alais
D
Cass
J
Multisensory perceptual learning of temporal order: audiovisual learning transfers to vision but not audition
PLoS ONE
 , 
2010
, vol. 
5
 pg. 
e11283
 
Avillac
M
Deneve
S
Olivier
E
Pouget
A
Duhamel
JR
Reference frames for representing visual and tactile locations in parietal cortex
Nat Neurosci
 , 
2005
, vol. 
8
 (pg. 
941
-
949
)
Azanon
E
Camacho
K
Soto-Faraco
S
Tactile remapping beyond space
Eur J Neurosci
 , 
2010
, vol. 
31
 (pg. 
1858
-
1867
)
Azanon
E
Longo
MR
Soto-Faraco
S
Haggard
P
The posterior parietal cortex remaps touch into external space
Curr Biol
 , 
2010
, vol. 
20
 (pg. 
1304
-
1309
)
Bernasconi
F
Grivel
J
Murray
MM
Spierer
L
Interhemispheric coupling between the posterior sylvian regions impacts successful auditory temporal order judgment
Neuropsychologia
 , 
2010a
, vol. 
48
 (pg. 
2579
-
2585
)
Bernasconi
F
Grivel
J
Murray
MM
Spierer
L
Plastic brain mechanisms for attaining auditory temporal order judgment proficiency
Neuroimage
 , 
2010b
, vol. 
50
 (pg. 
1271
-
1279
)
Bolognini
N
Maravita
A
Proprioceptive alignment of visual and somatosensory maps in the posterior parietal cortex
Curr Biol
 , 
2007
, vol. 
17
 (pg. 
1890
-
1895
)
Brozzoli
C
Gentile
G
Petkova
VI
Ehrsson
HH
FMRI adaptation reveals a cortical mechanism for the coding of space near the hand
J Neurosci
 , 
2011
, vol. 
31
 (pg. 
9023
-
9031
)
Calvert
GA
Bullmore
ET
Brammer
MJ
Campbell
R
Williams
SC
McGuire
PK
Woodruff
PW
Iversen
SD
David
AS
Activation of auditory cortex during silent lipreading
Science
 , 
1997
, vol. 
276
 (pg. 
593
-
596
)
Corbetta
M
Shulman
GL
Control of goal-directed and stimulus-driven attention in the brain
Nat Rev Neurosci
 , 
2002
, vol. 
3
 (pg. 
201
-
215
)
Craig
JC
The effect of hand position and pattern motion on temporal order judgments
Percept & Psychophys
 , 
2003
, vol. 
65
 (pg. 
779
-
788
)
Craig
JC
Busey
TA
The effect of motion on tactile and visual temporal order judgments
Percept Psychophys
 , 
2003
, vol. 
65
 (pg. 
81
-
94
)
Davis
B
Christie
J
Rorden
C
Temporal order judgments activate temporal parietal junction
J Neurosci
 , 
2009
, vol. 
29
 (pg. 
3182
-
3188
)
Duhamel
JR
Colby
CL
Goldberg
ME
Ventral intraparietal area of the macaque: congruent visual and somatic response properties
J Neurophysiol
 , 
1998
, vol. 
79
 (pg. 
126
-
136
)
Dukelow
SP
DeSouza
JF
Culham
JC
van den Berg
AV
Menon
RS
Vilis
T
Distinguishing subregions of the human MT+ complex using visual fields and pursuit eye movements
J Neurophysiol
 , 
2001
, vol. 
86
 (pg. 
1991
-
2000
)
Efron
R
The effect of handedness on the perception of simultaneity and temporal order
Brain
 , 
1963
, vol. 
86
 (pg. 
261
-
284
)
Friston
KJ
Frith
CD
Frackowiak
RS
Turner
R
Characterizing dynamic brain responses with fMRI: a multivariate approach
Neuroimage
 , 
1995
, vol. 
2
 (pg. 
166
-
172
)
Fujisaki
W
Nishida
S
Audio-tactile superiority over visuo-tactile and audio-visual combinations in the temporal resolution of synchrony perception
Exp Brain Res
 , 
2009
, vol. 
198
 (pg. 
245
-
259
)
Graziano
MS
Gross
CG
Visual responses with and without fixation: neurons in premotor cortex encode spatial locations independently of eye position
Exp Brain Res
 , 
1998
, vol. 
118
 (pg. 
373
-
380
)
Graziano
MS
Yap
GS
Gross
CG
Coding of visual space by premotor neurons
Science
 , 
1994
, vol. 
266
 (pg. 
1054
-
1057
)
Grezes
J
Costes
N
Decety
J
The effects of learning and intention on the neural network involved in the perception of meaningless actions
Brain
 , 
1999
, vol. 
122
 
Pt 10
(pg. 
1875
-
1887
)
Hagen
MC
Franzen
O
McGlone
F
Essick
G
Dancer
C
Pardo
JV
Tactile motion activates the human middle temporal/V5 (MT/V5) complex
Eur J Neurosci
 , 
2002
, vol. 
16
 (pg. 
957
-
964
)
Hallett
M
Transcranial magnetic stimulation: a primer
Neuron
 , 
2007
, vol. 
55
 (pg. 
187
-
199
)
Hirsh
IJ
Sherrick
CE
Jr
Perceived order in different sense modalities
J Exp Psychol
 , 
1961
, vol. 
62
 (pg. 
423
-
432
)
Hoffman
EA
Haxby
JV
Distinct representations of eye gaze and identity in the distributed human neural system for face perception
Nat Neurosci
 , 
2000
, vol. 
3
 (pg. 
80
-
84
)
Kansaku
K
Johnson
A
Grillon
ML
Garraux
G
Sadato
N
Hallett
M
Neural correlates of counting of sequential sensory and motor events in the human brain
Neuroimage
 , 
2006
, vol. 
31
 (pg. 
649
-
660
)
Kansaku
K
Yamaura
A
Kitazawa
S
Sex differences in lateralization revealed in the posterior language areas
Cereb Cortex
 , 
2000
, vol. 
10
 (pg. 
866
-
872
)
Kirman
JH
Tactile apparent movement: the effects of number of stimulators
J Exp Psychol
 , 
1974
, vol. 
103
 (pg. 
1175
-
1180
)
Kitazawa
S
Where conscious sensation takes place
Conscious Cogn
 , 
2002
, vol. 
11
 (pg. 
475
-
477
)
Kitazawa
S
Moizumi
S
Okuzumi
A
Saito
F
Shibuya
S
Takahashi
T
Wada
M
Yamamoto
S
Haggard
P
Kawato
M
Rossetti
Y
Reversal of subjective temporal order due to sensory and motor integrations
Attention and performance XXII
 , 
2007
New York
Academic Press
(pg. 
73
-
97
)
Kolster
H
Peeters
R
Orban
GA
The retinotopic organization of the human middle temporal area MT/V5 and its cortical neighbors
J Neurosci
 , 
2010
, vol. 
30
 (pg. 
9801
-
9820
)
Lloyd
DM
Shore
DI
Spence
C
Calvert
GA
Multisensory representation of limb position in human premotor cortex
Nat Neurosci
 , 
2003
, vol. 
6
 (pg. 
17
-
18
)
Macaluso
E
Driver
J
Spatial attention and crossmodal interactions between vision and touch
Neuropsychologia
 , 
2001
, vol. 
39
 (pg. 
1304
-
1316
)
Matsuhashi
M
Ikeda
A
Ohara
S
Matsumoto
R
Yamamoto
J
Takayama
M
Satow
T
Begum
T
Usui
K
Nagamine
T
, et al.  . 
Multisensory convergence at human temporo-parietal junction - epicortical recording of evoked responses
Clin Neurophysiol
 , 
2004
, vol. 
115
 (pg. 
1145
-
1160
)
Milner
TE
Franklin
DW
Imamizu
H
Kawato
M
Central control of grasp: manipulation of objects with complex and simple dynamics
Neuroimage
 , 
2007
, vol. 
36
 (pg. 
388
-
395
)
Ogawa
S
Lee
TM
Barrere
B
The sensitivity of magnetic resonance image signals of a rat brain to changes in the cerebral venous blood oxygenation
Magn Reson Med
 , 
1993
, vol. 
29
 (pg. 
205
-
210
)
Oldfield
RC
The assessment and analysis of handedness: the Edinburgh inventory
Neuropsychologia
 , 
1971
, vol. 
9
 (pg. 
97
-
113
)
Orban
GA
Fize
D
Peuskens
H
Denys
K
Nelissen
K
Sunaert
S
Todd
J
Vanduffel
W
Similarities and differences in motion processing between the human and macaque brain: evidence from fMRI
Neuropsychologia
 , 
2003
, vol. 
41
 (pg. 
1757
-
1768
)
Pesaran
B
Nelson
MJ
Andersen
RA
Dorsal premotor neurons encode the relative position of the hand, eye, and goal during reach planning
Neuron
 , 
2006
, vol. 
51
 (pg. 
125
-
134
)
Pöppel
E
A hierarchical model of temporal perception
Trends Cogn Sci
 , 
1997
, vol. 
1
 (pg. 
56
-
61
)
Price
CJ
Friston
KJ
Cognitive conjunction: a new approach to brain activation experiments
Neuroimage
 , 
1997
, vol. 
5
 (pg. 
261
-
270
)
Puce
A
Allison
T
Bentin
S
Gore
JC
McCarthy
G
Temporal cortex activation in humans viewing eye and mouth movements
J Neurosci
 , 
1998
, vol. 
18
 (pg. 
2188
-
2199
)
Roberts
RD
Humphreys
GW
Task effects on tactile temporal order judgments: when space does and does not matter
J Exp Psychol Hum Percept Perform
 , 
2008
, vol. 
34
 (pg. 
592
-
604
)
Roder
B
Rosler
F
Spence
C
Early vision impairs tactile perception in the blind
Curr Biol
 , 
2004
, vol. 
14
 (pg. 
121
-
124
)
Schicke
T
Roder
B
Spatial remapping of touch: confusion of perceived stimulus order across hand and foot
Proc Natl Acad Sci USA
 , 
2006
, vol. 
103
 (pg. 
11808
-
11813
)
Shibuya
S
Takahashi
T
Kitazawa
S
Effects of visual stimuli on temporal order judgments of unimanual finger stimuli
Exp Brain Res
 , 
2007
, vol. 
179
 (pg. 
709
-
721
)
Shore
DI
Spry
E
Spence
C
Confusing the mind by crossing the hands
Brain Res Cogn Brain Res
 , 
2002
, vol. 
14
 (pg. 
153
-
163
)
Smith
AT
Wall
MB
Williams
AL
Singh
KD
Sensitivity to optic flow in human cortical areas MT and MST
Eur J Neurosci
 , 
2006
, vol. 
23
 (pg. 
561
-
569
)
Soto-Faraco
S
Spence
C
Kingstone
A
Congruency effects between auditory and tactile motion: extending the phenomenon of cross-modal dynamic capture
Cogn Affect Behav Neurosci
 , 
2004a
, vol. 
4
 (pg. 
208
-
217
)
Soto-Faraco
S
Spence
C
Kingstone
A
Cross-modal dynamic capture: congruency effects in the perception of motion across sensory modalities
J Exp Psychol Hum Percept Perform
 , 
2004b
, vol. 
30
 (pg. 
330
-
345
)
Spierer
L
Bernasconi
F
Grivel
J
The temporoparietal junction as a part of the “when” pathway
J Neurosci
 , 
2009
, vol. 
29
 (pg. 
8630
-
8632
)
Sternberg
S
Knoll
R
Kornblum
S
The perception of temporal order: fundamental issues and a general model
Attention and performance
 , 
1973
New York
Academic Press
(pg. 
629
-
685
)
Wada
M
Yamamoto
S
Kitazawa
S
Effects of handedness on tactile temporal order judgment
Neuropsychologia
 , 
2004
, vol. 
42
 (pg. 
1887
-
1895
)
Watson
JDG
Myers
R
Frackowiak
RSJ
Hajnal
JV
Woods
RP
Mazziotta
JC
Shipp
S
Zeki
S
Area-V5 of the human brain – evidence from a combined study using positron emission tomography and magnetic-resonance-imaging
Cereb Cortex
 , 
1993
, vol. 
3
 (pg. 
79
-
94
)
Wicker
B
Michel
F
Henaff
MA
Decety
J
Brain regions involved in the perception of gaze: a PET study
Neuroimage
 , 
1998
, vol. 
8
 (pg. 
221
-
227
)
Woo
SH
Kim
KH
Lee
KM
The role of the right posterior parietal cortex in temporal order judgment
Brain Cogn
 , 
2009
, vol. 
69
 (pg. 
337
-
343
)
Woods
RP
Modeling for intergroup comparisons of imaging data
Neuroimage
 , 
1996
, vol. 
4
 (pg. 
S84
-
94
)
Yamamoto
S
Kitazawa
S
Reversal of subjective temporal order due to arm crossing
Nat Neurosci
 , 
2001a
, vol. 
4
 (pg. 
759
-
765
)
Yamamoto
S
Kitazawa
S
Sensation at the tips of invisible tools
Nat Neurosci
 , 
2001b
, vol. 
4
 (pg. 
979
-
980
)