Previous neurophysiological studies performed in macaque monkeys have shown that the secondary somatosensory cortex (SII) is essentially engaged in the processing of somatosensory information and no other sensory input has been reported. In contrast, recent human brain-imaging studies have revealed the effects of visual and auditory stimuli on SII activity, which suggest multisensory integration in the human SII. To determine whether multisensory responses of the SII also exist in nonhuman primates, we recorded single-unit activity in response to visual and auditory stimuli from the SII and surrounding regions in 8 hemispheres from 6 awake monkeys. Among 1157 recorded neurons, 306 neurons responded to visual stimuli. These visual neurons usually responded to rather complex stimuli, such as stimulation of the peripersonal space (40.5%), observation of human action (29.1%), and moving-object stimulation outside the monkey's reach (23.9%). We occasionally applied auditory stimuli to visual neurons and found 10 auditory-responsive neurons that exhibited somatosensory responses. The visual neurons were distributed continuously along the lateral sulcus covering the entire SII, along with other somatosensory neurons. These results highlight the need to investigate novel functional roles—other than somesthetic sensory processing—of the SII.
It is believed that the secondary somatosensory cortex (SII), which is located in the upper bank of the lateral sulcus (UBLS), is a unimodal sensory cortex that contributes to the processing of tactile information for object recognition and of proprioceptive information for motor control. Electrophysiological studies of awake macaque monkeys investigating the neural responses to somatosensory and visual stimulation throughout the parietal operculum (PO) often demonstrated the existence of bimodal neurons in the PO region caudal to the SII (Robinson and Burton 1980b, c; Dong et al. 1994); however, these studies never found visually responsive neurons in the SII.
Despite the nonhuman-primate studies supporting the idea that the SII is a unimodal sensory area, recent human brain-imaging studies have often demonstrated visual effects on SII activity. Bremmer et al. (2001) studied the motion processing system in the human brain using fMRI and found that moving visual stimulation activated the SII as well as the cortical region in the deep intraparietal sulcus (IPS) and ventral premotor area. The activation of the SII by the observation of another person's body being touched has been repeatedly confirmed under various conditions (Keysers et al. 2004; reviewed by Keysers et al. 2010). More recent studies showed that viewing another person's actions also activates the SII (Agnew and Wise 2008; Gazzola and Keysers 2009; Nummenmaa et al. 2014).
Considering this discrepancy between the electrophysiological studies in monkey and human brain-imaging studies, asking whether visual effects on the SII are specific to humans is an interesting research question. Recently, Raos et al. (2014) performed a radioactive deoxyglucose study of macaque monkeys, which showed that regions that are activated by the observations of forelimb movements prevailed widely over the cortex, including the SII. This result encouraged us to explore visual responsive neurons in the SII of macaque monkeys.
In the present study, we used complex visual stimulation, such as the observation of human actions and presentation of moving objects within and outside the peripersonal space of the monkey. Additionally, in some cases, we tested the auditory responses because recent human imaging studies have revealed modifications in the SII activity by sound stimuli (Bremmer et al. 2001; Gazzola et al. 2006; Beauchamp and Ro 2008; Etzel et al. 2008). We found a substantial number of neurons that responded to visual stimuli in the SII as well as other cortical regions in the UBLS. A fraction of these neurons also responded to auditory stimuli. Here, we describe the response properties and distribution of these multisensory neurons in the PO, including the SII.
Materials and Methods
Subjects and Preparation
Eight hemispheres from 6 Japanese macaque monkeys (Macaca fuscate, 5 males and 1 female; body weight, 6.5–10.0 kg) were used in this study. Two of the 6 animals were used only in the present experiments. The other 4 animals were used in our previous studies (Tanaka et al. 2004; Taoka et al. 2013). All experimental procedures were approved by the RIKEN Animal Experimental Committee and were in accordance with the National Institutes of Health Guide for the Care and Use of Laboratory Animals.
After the monkeys were familiarized with primate chairs and the experimental settings (the experimenters, the task table, the primate chair, etc.), they underwent 2 surgeries under deep sodium pentobarbital anesthesia (30 mg/kg, i.v.) for the implantation of head-fixation devices followed by the placement of recording chambers. In the first operation, the head-fixation apparatus was implanted on the skull using stainless-steel screws to fix the head to the primate chair. After recovery from surgery and habituation to being seated in the chair with its head fixed, the monkey was trained to perform a simple task. The task required the monkey to place each hand for a few seconds on 2 boxes set on the table near both sides of the monkey's body; the monkey received a small reward (∼1 cm2 of apple or sweet potato) from the experimenter's hand. A few weeks after the first surgical operation, a second surgery was performed to implant a cylindrical stainless-steel recording chamber (20 mm in diameter) on the skull. To cover the SII, a skull opening (20 mm in diameter) was made stereotaxically, 8–10 mm anteriorly and 20–25 mm laterally (ear bar zero), and the chamber was fixed to the skull with dental acrylic. A more detailed surgical procedure is described elsewhere (Tanaka et al. 2004; Taoka et al. 2013).
Single-unit activity was recorded from the SII and surrounding cortical areas in the UBLS using varnished tungsten electrodes (1–2 MΩ at 1 kHz, FHC, Inc., USA), which were driven by a micromanipulator (MO-95, Narishige) attached to the recording chamber. During the recording session, the awake animal was seated in the primate chair with its head fixed. Neural activity was amplified and monitored by an oscilloscope and a loudspeaker system. Only well-isolated single-unit activities were recorded. Amplified signals were fed into the audio channel of a video recorder (DVCAM-45, Sony) for later off-line analysis. To correlate recorded neural activity with the timing of sensory stimulation or the actions of animals, the animals' behavior was video-recorded using 4 CCD cameras set around the experimental table: 2 for a top and side view of the monkey and 1 for a close-up image of the upper face to monitor approximate eye movements, which enabled us to check whether neural activity was synchronized with eye movement or not. We excluded the neural activity that was affected by eye movement from analyses of visual responses but such neurons were rarely found. The last one was set on a camera platform with a flexible arm to obtain video images of the visual stimuli that were presented by the experimenter, or of the forelimb movements of the monkey during the self-movements test (see below). Mixed video images from the 4 cameras were generated by a digital quad switcher (SW-D410, Victor) and videotaped (29.97 frames/s) together with the neural signals recorded in an audio channel. The time delay between the video images and signal recordings was calculated and adjusted for later analyses. The sounds used in auditory stimulation experiments (see below) were recorded with a microphone that was placed above the monkey's chair, hung from the ceiling, and fed into the other audio channel of the video recorder. At the end of selected electrode penetrations, 1–3 electrical lesions (current of 10 μA for 10 s) were made at different depths for later histological identification of the track trajectory. For more details, see Supplementary Methods.
Identification of the Somatosensory Receptive Field
To identify the somatosensory receptive field (RF), a variety of somatosensory stimuli were manually applied to the body of the animal, such as simple touches, rubbing, tapping, and pressing the body surface and bending the hair using the experimenter's hand, a hand-held metal probe, or a paintbrush. Passive joint manipulation was also performed when the monkey was relaxed. Regarding the intraoral structures, we only examined the rostral part, including the incisors, canines, and the front part of the tongue because of the difficulty in accessing. We were also unable to investigate a part of the hindlimb around the knee, because it was covered by the monkey's chair. To exclude visual effects on the neural activity during somatosensory stimulation, RF identification was performed both with the animal's eyes open and blinded with an eye mask.
Test of Self-Movements of the Monkey's Forelimb
An active feeding test was also performed for all visual neurons to examine whether self-movement of the monkey could evoke neural activity. After the monkey placed its hands on the 2 boxes for a few seconds, the experimenter presented the food piece at various positions on the table within the monkey's reach range (∼40 cm). After reaching, the monkey picked up the food piece and put it in its mouth at its own pace using the hand that was closer to the reward. More than 5 tests were administered for each hand.
Visual stimuli were presented to the animal by the experimenters who sat on either side of the monkey. We principally used 3 types of visual stimulation: The presentation of a human hand and objects in front of the animal within and outside of its reach range and the observation of human action. Before each stimulation, the animal placed both hands on the 2 boxes on the table. After a few seconds of hand resting (resting period), the stimulation was started. Monkeys were fed a food piece after every trial.
Presentation of a human hand within and outside the monkey's reach
In the first stage of the test, we attempted to evoke neural spikes by presenting the experimenter's hand at various places outside the monkey's reach (at a range of ∼40–80 cm from the monkey) on the left or right side and at its head level. Then, the hand was moved in various directions: horizontal, vertical, rotating, close to and away from the monkey's body. Next, the experimenter moved the hand in the same fashion but within a range of ∼40 cm from the monkey. Space around the lower body parts that were under the table was not tested. When an effective stimulation was found, we carefully checked the preference for the position and direction of the movements. The effective ranges of the stimulation, that is, the visual receptive fields (v-RFs), were roughly measured. Stimulation close to the monkey's body surface was carefully applied to avoid displacing the monkey's hair by a direct touch or airflow generated by the moving stimuli. Finally, to test the preference for objects, we used various objects instead of the experimenter's hand: a piece of food, a small ball (10 cm in diameter), a brush, and a food container.
Observation of human action
To examine neural responses to the observation of human actions, the experimenter performed a variety of forelimb actions outside of the monkey's reach range. The typical action was a series of forelimb movement: reaching for a food container, grasping a food piece with the fingers, and lifting and handing it to the monkey. When the reaching actions evoked neural activities, reaching toward objects other than the food container was also performed. To find more effective stimuli, we occasionally performed some simple actions like manipulating the food piece or food container, holding the food piece with both hands, opening and closing the lid of the food container, inserting fingers into the container, moving fingers inside the container, etc.
Auditory stimulation was also applied in cases where well-isolated single-neuron activity was maintained even after the completion of visual inspection. Natural sounds produced by the experimenter were presented, for example, the sounds of gently hitting a metal bar with a metal probe (click sound), rubbing both hands together, stroking clothes with the hand, clapping hands, tapping on the floor with the foot, jingling keys, etc. The stimuli were presented both in front of and behind the animal. During the presentation of auditory stimuli to the monkey, the loudspeaker for monitoring neural responses was turned off.
Data Collection and Statistical Analysis
The videotaped data were converted to digital formats on a personal computer. To examine the correlation between neural activity and the timing of sensory stimulation, recorded neural spikes were aligned with the timing of various events, such as the onset and cessation of sensory stimulation and movements performed by the experimenter (e.g., reaching, touching, and moving objects). Raster and peristimulus averaged histograms (bin width = 50 or 100 ms) were produced by a software (SaruMonitor, ver. 2, FiatLux, see Supplementary Methods). When neural activity was steadily recorded in >5 trials in each stimulus condition, the data were statistically analyzed. The number of spikes recorded during the test period (200 or 300 ms) was compared with that of the resting period (200 or 300 ms before the trial, while the monkey placed both hands on the boxes) using the Mann–Whitney U-test (P < 0.05).
Histological Procedures for the Reconstruction of Recording Sites
After the recording session, the animals were perfused with 0.9% saline through the heart under an overdose of pentobarbital sodium followed by 10% formalin for reconstruction of the electrode tracks. Using a micro-drive manipulator, we inserted several reference guide wires into the recording chamber parallel to the electrode penetrations before removing the brain from the skull. The explored region was blocked and removed from the brain. After dehydration and celloidin embedding, the brain block was sliced at a thickness of 40 μm at a right angle to both the lateral sulcus (LS) and cortical surface. For the identification of electrode trajectories, the sections were stained using the Nissl method and inspected for the presence of gliosis around the electrode tracks and the electrolytic lesions.
Reconstructed electrode trajectories were assigned to electrode penetrations based on the entry points on the cortical surface, electrical lesions and the depth of the penetrations. Because the electrode trajectories were traversed over several sections, successive traced images were superimposed on 1 image. After taking into consideration the shrinkage of the brain block during the dehydration and celloidin-embedding steps, recording sites were estimated with reference to the depth reading of the micromanipulator and were plotted on the superimposed image.
Figure 1 shows the locations of the SII and several adjacent regions in the UBLS area: Area 7b, the retroinsula (Ri) and the parietal rostroventral area (PR). All 4 areas have somatosensory inputs. Moreover, the SII, area 7b and Ri have topologically organized representations of the body surface (Friedman et al. 1980; Robinson and Burton 1980a; Disbrow et al. 2003). Recent electrophysiological mapping studies in the classically defined SII in anesthetized macaques (Krubitzer et al. 1995) demonstrated 2 somatotopic maps, the SII and the parietal ventral area (PV). In this article, we describe the 2 areas as a single SII region that has been classically defined as a single cortical region. The estimation of SII boundaries with other areas was carried out both cytoarchitecturally (Jones and Burton 1976; Friedman et al. 1980; Burton et al. 1995) and electrophysiologically. Nevertheless, we frequently had trouble defining the precise boundaries in the caudal-most and rostral-most regions of the SII, where architectonic features changed gradually and differences in the neural properties were not sufficiently reliable to draw a clear border.
Defining the border between the SI and the SII was not difficult compared with other cortical regions (Taoka et al. 2013). The SI has a thinner Layer IV and a clearer lamination boundary between Layers IV and V, whereas the SII has a sublamination in Layer III and a thicker Layer IV. The response properties to somatosensory stimuli were also helpful in defining the border.
The SI near the LS corresponds to Area 3b, 1, and 2. In Areas 3b and 1 bordering the SII, the neurons have RFs in the intraoral structures, the sizes of which are rather small. RF location changes systematically according to its clear somatotopic organization as the electrode penetration progresses (Toda and Taoka 2001, 2002). When the RF positions changed suddenly (for example, neurons with RFs inside the inner mouth structure were recorded followed by neurons with RFs on the surface of the face), or when RF determination became difficult, it indicated that the electrode had entered the SII. Area 2 adjacent to a more caudal part of the SII is the SI mouth region. However, SII neurons represent the hand; thus, we could easily discriminate SII neurons from SI neurons.
On the other hand, delimitation of the SII borders with Area 7b and the Ri was rather difficult. Compared with the SII, a thinner Layer IV and clearer lamination of Layers V and VI characterize Area 7b. Fewer pyramidal neurons in Layer IIIB and a clearer columnar appearance of Area 7b are also distinctive features. In the bordering region, however, these differences become less obvious. With regard to neuronal properties, 7b neurons in the bordering region with the SII often respond to somatosensory stimuli to the face/head and the forelimb (Robinson and Burton 1980b; Dong et al. 1994), whereas the SII neurons adjacent to Area 7b usually represent the proximal parts of the forelimb or the trunk (Robinson and Burton 1980a; Fitzgerald et al. 2004; Taoka et al. 2013). We often encountered large RFs covering the trunk, forelimb, and head, including the face, so that the differences in RF locations between the SII and Area 7b neurons were not always reliable for their discrimination. Furthermore, neural properties in the bordering regions did not change systematically as the electrode progressed, for example, after recording neurons with the features of 7b neurons, SII-like responses were observed over a few units, and then 7b-like neurons were recorded, suggesting that the neurons of Area 7b and the SII are intermingled around the border.
A similar difficulty in defining cortical boundaries was observed in the bordering cortical regions of the SII and Ri. The caudal part of the SII is bordered by the Ri in the deepest part near the LS fundus (Fig. 1). In the Ri, because of the sparse distribution of cells in Layer V and the rather densely packed cells in Layer VI, the demarcation of the layers was clearer than that of the SII. Layer IV in the Ri is thinner and less homogeneous than that of the SII (Burton et al. 1995). Ri neurons in the boundary with the SII usually represent the head/face or the trunk whereas SII neurons in the boundary area represent the hindlimb (Robinson and Burton 1980b). Despite these cytoarchitectural and neurophysiological differences, it was often difficult to define the border between the SII and Ri because SII-like responses and Ri-like responses were often intermingled when recording electrodes passed near the border.
It was also difficult to define the SII anterior border with the PR. Cipolloni and Pandya (1999) stated that the cortical region, cytoarchitecturally similar to the SII, extended rostrally to reach the border between the parietal and the frontal opercular regions. Furthermore, because of the scarceness of neurophysiological studies of awake macaque monkeys in the PR, we were unable to utilize neural response properties for areal discrimination. Accordingly, we could not draw the border of this region. For all the reasons mentioned earlier, we did not establish clear boundaries of the SII with the adjoining regions (Area 7b, Ri, and PR) in the UBLS. Isolated neurons recorded from the bordering regions were not excluded from the analyses in the present study.
Distribution and Somatosensory Response Properties of Neurons in the Explored Regions
We recorded 1431 single-unit activities from the SII and adjacent regions (PR, area 7b, and Ri) from 8 hemispheres of 6 awake monkeys. Among them, 1157 neurons (80.4%) were tested for somatosensory RF identification and visual stimulation. The explored regions of the 8 hemispheres varied along the rostrocaudal axis according to the differences in the location of the recording chambers; one covered almost the entire SII whereas the others missed the caudal-most or rostral-most part of the SII.
Figure 2 shows an example of a hemisphere in which the explored region covered almost the entire SII along the rostrocaudal axis. In this hemisphere, 602 single units were recorded from 45 electrode penetrations. Among them, somatosensory RFs could be identified for 285 single units. These units were grouped according to the differences in RF locations. Their estimated locations are depicted by circles with different colors (Fig. 2C).
As shown in Figure 2C, the recording electrodes in this hemisphere generally penetrated into the SII in the UBLS. In the 5 most caudal sections (f–j in Fig. 2C), the electrode entered the exposed surface of the parietal region near the LS defined as Area 7b, then penetrated into the UBLS defined as the SII (some of the penetrations reached the deepest part of the UBLS around the border between the SII and the Ri). In more rostral sections (a–e in Fig. 2C), the exposed surface near the LS was defined as the SI. We excluded the SI neurons from the analyses of the present study (only estimated locations of SI neurons are depicted in Fig. 2 using short vertical bars). Thus, the 602 recorded neurons in this hemisphere included Area 7b and Ri neurons in the bordering regions as well as in the SII.
In the caudal-most section (j in Fig. 2C), the deepest part of the UBLS around the border between the SII and the Ri, neurons with RFs in the trunk/hindlimb coexisted with neurons having RFs in the forelimb or the face/head. In the section rostral to this (i in Fig. 2C), most of the neurons recorded within the bank had RFs in the arm. In the following 5 sections (d–h in Fig. 2C), somatosensory RFs existed mostly in the hand. In the 3 rostral sections (a–c in Fig. 2C), an increase in the number of neurons with RFs in the mouth and face/head was observed and almost all of the neurons recorded had RFs in the mouth in the most rostral section (a in Fig. 2C). These results show the presence of somatotopic organization in the SII; the face/mouth were represented rostrally, whereas the trunk/leg were represented caudally, and the hand was represented in the middle of the SII, as reported in previous studies (Robinson and Burton 1980a; Burton et al. 1995; Krubitzer et al. 1995; Fitzgerald et al. 2004; Taoka et al. 2013). As shown in the somatotopic map in Figure 2C, the explored region in this hemisphere covered almost the entire SII along the rostrocaudal axis.
In the other 6 hemispheres, explored regions were shifted caudally. Therefore, the face/mouth representations in the rostral part of the SII were not explored entirely and the recorded neurons included 7b and Ri neurons in the bordering regions.
In the remaining hemisphere, shown in Figure 3, the recording chamber was located more rostrally than in the other 7 hemispheres, so that the explored region did not cover the caudal part of the SII and extended into the PO region anterior to the SII that presumably reaches the PR (Fig. 1). A total of 357 single units were recorded from 41 electrode penetrations and 183 units were identified for somatosensory RFs in this hemisphere.
Most of the insertion points around the central sulcus (CS) were distant from the LS (Fig. 3B); the explored regions in most of the sections (a–h in Fig. 3C) were restricted to the deeper part of the UBLS, facing the insular cortex. In the 2 caudal-most sections (i and j in Fig. 3C), most of the neurons had RFs in the hand, indicating that the explored area was the SII hand representation region. In the section anterior to these 2 sections (h in Fig. 3C), neurons with RFs in the mouth as well as those with combined RFs in the mouth and hand were recorded. Considering the SII somatotopic map in which the mouth/face representation occupied the rostral-most part of the SII, the explored region of these 3 sections (h–j) was in the anterior part of the SII.
In the 7 more rostral sections (a–g in Fig. 3C), somatotopic representations seemed to shift along the rostrocaudal axis; in the 4 caudal sections (d–g in Fig. 3C), neurons with RFs in the hand were found together with those with RFs in the face/head and those with combined-type RFs (>2 parts colored in light blue). In the 3 rostral sections (a–c in Fig. 3C), neurons with RFs in the mouth represented the majority. Considering the fact that the body representation pattern in those 7 sections did not fit the somatotopy of the SII, the region was probably in the PR.
Visual Response Properties of Neurons in the Explored Regions
We found 306 neurons that responded to visual stimuli (in all 8 hemispheres) in the present study (Table 1). As shown in Table 1, most of the neurons responded to rather complex stimuli, such as: 1) stimulation in the peripersonal space of the monkey, 2) observation of human actions, and 3) moving-object stimulation outside the reach range of the monkey. Except for the neurons of “static presentation” (n = 5) and “unspecified” (n = 6), all neurons responded to any of the 3 types of stimuli. The “convergence-type” neurons (which responded to the first 2 types of stimuli, n = 9) were rarely found. Here, we describe their response properties according to the 3 types of stimulation.
Neurons that responded to stimulation in the peripersonal space of the monkey
Among the 306 visual neurons, 124 neurons (40.5%) responded to stimulation in the peripersonal space of the monkey (Table 1). They did not constantly respond to stimuli presented outside of the monkey's reach range (within 40 cm) but became active when the stimulation was presented close to the monkey's body. As shown in Figure 4, the neurons responded to the experimenter's hand moving horizontally in front of the monkey's upper body. We calculated the number of spikes during the 200-ms period after the moving stimuli started in each trial and compared them with those recorded during the 200-ms resting period. We found that the number of spikes recorded during the moving stimuli in the left-to-right direction was significantly larger than that of the resting period (Mann–Whitney U-test, P < 0.05), but no significant difference was found with the right-to-left movements (Fig. 4A).
|Effective visual stimuli||n (%)a||Visual only|
|Peripersonal stimuli||124 (40.5%)||37 (29.8%)||73 (58.9%)||14 (11.3%)|
|Action observation||89 (29.1%)||31 (34.8%)||12 (13.5%)||46 (51.7%)|
|Moving stimuli||73 (23.9%)||39 (53.4%)||22 (30.1%)||12 (16.4%)|
|Convergence||9 (2.9%)||4 (44.4%)||2 (22.2%)||3 (33.3%)|
|Static presentation||5 (1.6%)||2 (40.0%)||0 (0.0%)||3 (60.0%)|
|Unspecified stimuli||6 (2.0%)||2 (33.3%)||3 (50.0%)||1 (16.7%)|
|Total||306 (100%)||115 (37.6%)||112 (36.6%)||79 (25.8%)|
|Effective visual stimuli||n (%)a||Visual only|
|Peripersonal stimuli||124 (40.5%)||37 (29.8%)||73 (58.9%)||14 (11.3%)|
|Action observation||89 (29.1%)||31 (34.8%)||12 (13.5%)||46 (51.7%)|
|Moving stimuli||73 (23.9%)||39 (53.4%)||22 (30.1%)||12 (16.4%)|
|Convergence||9 (2.9%)||4 (44.4%)||2 (22.2%)||3 (33.3%)|
|Static presentation||5 (1.6%)||2 (40.0%)||0 (0.0%)||3 (60.0%)|
|Unspecified stimuli||6 (2.0%)||2 (33.3%)||3 (50.0%)||1 (16.7%)|
|Total||306 (100%)||115 (37.6%)||112 (36.6%)||79 (25.8%)|
a% of the total number of visual neurons.
b% of the number in each category.
Figure 4A (front and side views in the middle panel) depicts the effective range of the stimulation, that is, the v-RF, which was three-dimensionally constructed extending from the lower part of the face (around the mouth and nose) to the chest and covering approximately the body width. When the stimulation was more than ∼20 cm away from the body, the responsiveness was clearly lower.
In this group of “peripersonal” neurons (n = 124), the most frequently observed visual RF were in the face (n = 84), followed by the forelimb (n = 37) and the trunk (n = 20), with complex v-RFs (n = 13) covering >2 body parts, such as the neuron shown in Figure 4. Most of the v-RFs were bilateral ones crossing the midline of the body (n = 86, 70.0%), whereas 19 contralateral and 2 ipsilateral v-RFs were also found. Regarding the remaining 17 neurons, we were unable to identify the laterality of v-RFs because of the poor isolation of single-unit activity after a long inspection of neural response properties.
Most of the 124 neurons (n = 97, 78.2%) showed directional selectivity to the presented stimuli. The most frequently observed direction was approaching the body surface (n = 76, 61.3%), followed by moving away from the body (n = 12, 9.7%), rotating (n = 4, 3.2%), upward (n = 2, 1.6%), downward (n = 2, 1.6%), and right-to-left (n = 1) (Fig. 4A). The remaining 26 neurons showed no directional selectivity, that is, the movement of the experimenter's hand alone within v-RFs was sufficient for their activation. We also tested the responsiveness of these neurons to visual stimulation with other objects. However, we did not find neurons that responded preferentially to a certain object; they always responded to other objects in a manner that was similar to their response to the experimenter's hand movement.
The neuron shown in Figure 4 also had a somatosensory RF around the mouth (Fig. 4B). When the stroking stimuli were applied in the left-to-right direction, the neuron discharged (P > 0.05). However, no significant responses were observed when the direction was reversed. The neurons had somatosensory RFs in the bilateral hands and the belly (Fig. 4C). Such bimodal neurons responding both to visual and somatosensory stimulations accounted for 73 neurons (58.9%, Table 1). All of the somatosensory RFs roughly corresponded to v-RFs but we occasionally encountered v-RFs that covered only a part of the somatosensory RFs, similar to the neuron in Figure 4.
Neurons that responded to observation of human actions
Eighty-nine (29.1%) of 306 visual neurons responded selectively to the observation of human actions performed outside the range of the monkey's reach (Table 1). The most effective human actions were the experimenter's actions of reaching and/or grasping a food piece in the container. We also observed neurons that became active during the observation of the manipulation of objects such as the food piece and the food container.
Figure 5 shows 3 examples of neurons that responded to the observation of human actions with different timings and conditions of grasping. The neuron depicted in Figure 5A became active when the experimenter picked up a food piece from the food container with the fingers. To investigate the precise timing of spike firing, the neural spikes were aligned with the timing of the end of the “lifting-up” motion (a food piece held by the fingers was lifted upward about 10 cm above the container). The neural firing began to increase in the middle of the “lifting-up” motion and reached its peak at the end of this motion. The number of spikes recorded during the period from −100 to +100 ms was compared statistically with that recorded during the 200 ms of control period, that is, the period 200 ms before the start of the action. The neural activity during the “lifting-up” action was significantly higher relative to those recorded during the control period (P < 0.05).
The neuron depicted in Figure 5B became active when the experimenter touched a piece of food in the food container. The raster and averaged histograms were aligned to the moment at which the experimenter touched the food with the fingers. Neural activity increased just after the experimenter touched a piece of food and returned to the baseline level within ∼400 ms. The spikes evoked during the 200 ms period just after the experimenter touched the food increased significantly compared with the control period (P < 0.05).
The neuron depicted in Figure 5C did not clearly respond to any phases of the human reaching and grasping action, rather, it discharged constantly in response to the experimenter's bimanual holding of a piece of food. Neural activities were aligned to the moment at which the experimenter's fingers on one hand contacted the food piece that was then pinched by the other hand. The spike discharges increased just before the meeting of both hands and lasted while the experimenter held the food bimanually (∼1 s). Neural activity during the action of holding the food (0–200 ms) was significantly greater than that of the control period (P < 0.05).
Figure 6 shows an example of a “reaching” neuron that showed constant neural activity during the observation of the reaching action. To investigate the correlation between human reaching action and neural activity, evoked spikes were aligned at 2 different time points: the onset of reaching (Fig. 6A) and the moment of touching the food (Fig. 6B). The firing rates increased gradually after the experimenter started the reaching action (Fig. 6A) and suddenly stopped when the experimenter touched the food (Fig. 6B).
Among the 89 neurons that fired during the human action observation, neurons that became active in response to human reaching actions (such as the neuron shown in Fig. 6) were observed most commonly (n = 49), followed by neurons responding to grasping the food piece (n = 17) and lifting up or taking the food piece out of the container (n = 12). We sometimes presented other actions to the monkey to identify more effective actions for evoking neural spikes and found that 22 neurons (some of which also responded to observations of reaching and/or grasping actions, so that the total number exceeded 89) responded actively to observations of opening/closing the lid of the food container, inserting fingers or moving fingers inside the food container, etc.
Most of the “action observation” neurons (51.7%) also became active during the self-movements test (Table 1). We found that a fraction of neurons had rough correlations between effective motions in the observation and in the self-movement tests; for example, the neurons shown in Figure 5A,B responded to the observation of picking up the food and became active during the monkey's self-action of picking up the food. However, this was not true for all neurons. For example, the neuron depicted in Figure 5C did not respond to the observation of picking up the food but became active during the monkey's self-motion of picking up the food. In the present study, we did not perform this kind of off-line analysis for all neurons because video-recordings of the self-movement tests were not conducted for all tested neurons.
Neurons with somatosensory RFs were not in the majority among the neurons that responded to action observation. Only 12 of 89 neurons (13.5%) had somatosensory RFs (Table 1). The somatosensory RFs observed were the face/mouth (n = 9), the forelimb (n = 5), and the trunk (n = 2) and included large somatosensory RFs that covered >1 body part (n = 3; 1 somatosensory RF covered almost the entire body surface).
Neurons that responded to object motion stimulation
We found 73 neurons (23.9%) that responded to the presentation of moving objects outside of the monkey's reach (Table 1). These neurons did not respond to the static presentation of objects at any location and exhibited no clear and constant neural activity in response to human actions. Some of the neurons also responded to stimulation presented near the monkey's body. However, those responses were not specific to the stimulation of the peripersonal space of the monkey. Some neurons had directional selectivity of the moving stimuli, such as up-and-down (n = 15), sideways (n = 8), and circular (n = 3) motions, because of which, those neurons were categorized into this group. Twenty-two neurons (30.1%) responded to somatic stimuli. The somatosensory RFs were the face/mouth (n = 22), the forelimb (n = 17), leg (n = 4), and the trunk (n = 6). Twelve neurons became active during self-movement (16.4%).
Convergence-type visual neurons
We occasionally encountered visual neurons of the convergence-type (n = 9, 2.9%), that is, neurons that responded both to the visual stimulation in the peripersonal space and the observation of human actions. Other convergence-type neurons were not observed in the present study.
The neuron in Figure 7 became active when the monkey observed the human action of reaching and grasping the food piece. The neural spikes in the rasters and the averaged histogram in Figure 7A–C were aligned at 3 different time points: the onset of reaching (Fig. 7A), touching the food piece (Fig. 7B), and the onset of lifting the food up after grasping it (Fig. 7C). The firing rate increased gradually after the experimenter started reaching (Fig. 7A), and a certain level of firing rate was maintained during the grasping motion (Fig. 7B); the spikes disappeared before the experimenter started picking up the food piece (Fig. 7C). We compared the number of spikes in the 200-ms period after the “touching the food” action with those of the rest period and found that the observation of the action evoked statistically significant responses (P < 0.05). This neuron also had v-RFs around the mouth and the right hand (Fig. 7D,E). When the experimenter's hand moved close to the surface of the monkey's mouth or hand, the spike discharges increased. However, they decreased when the experimenter's hand moved away from the body. The neural activity shown in Figure 7D,E is aligned with the timings at which the visual stimuli reached the minimum distance (∼2 cm) from the body surface. We detected 9 convergence-type neurons that responded both to the observation of action and stimulation of the peripersonal space.
The neuron shown in Figure 7 did not respond to somatic stimulation even around the mouth and the right hand but discharged in the self-movement test with either hand. Among the 9 convergence-type neurons, we found only 2 with somatosensory RFs and 3 that responded to active movement.
Other visual neurons
Among the remaining 11 neurons, we found only 5 that responded to static presentation of objects. They did not show any preference for the location at which the objects were placed or for the objects used.
Regarding the remaining 6 neurons, we did not identify any effective visual stimuli that evoked spike discharges as long as we used the visual stimulations mentioned earlier. However, they frequently showed visual responsiveness to movements of the experimenter's body, such as standing up, and to the sudden presentation of objects in front of the monkey, etc.
Auditory responses of visual neurons
After the identification of visual response properties, we examined the visual neurons for auditory response only when well-isolated neural activity was maintained. We were able to test about one-thirds of the visual neurons and found 10 neurons that showed responses to auditory stimuli. The neuron shown in Figure 8 responded to the sounds that the experimenter made by stroking his lab coat using his hand behind the monkey. Thus, the possibility was excluded that the observation of stroking actions might have elicited the neural activities. Other sounds such as click sounds, footsteps, and jingling of keys were not effective. The neuron had a v-RFs near the face (Fig. 8B). Among the 10 auditory neurons identified, the effective sounds were usually those from stroking clothes with the hand Occasionally, we encountered neurons that responded to click sounds but none that responded to the jingling of keys, footsteps, or human voices.
The neuron in Figure 8 also had a somatosensory RF in the contralateral hand, and only the rubbing of the skin surface was effective. This neuron also responded to rubbing stimuli on the contralateral side of the face. However, we could not eliminate the possibility that the sounds generated by rubbing the face evoked the neural responses. All of the auditory neurons found in the present study had somatosensory RFs (similar to the neuron depicted in Fig. 8), mostly covering a large body surface including the trunk, forearm, and face (n = 7). Thus, these types of neurons were defined as trimodal neurons that responded to auditory and somatosensory stimulation as well as visual stimulation.
Effects of Somatosensory Stimuli and Self-Movements on the Neural Activity of Visual Neurons
In the present study, we found 3 major classes of neurons that responded to either peripersonal stimuli, action observation, or moving stimuli. Table 1 shows that the ratios of neurons whose activity were affected by somatosensory stimuli and self-movements were considerably different among the 3 groups. In the “peripersonal stimuli” neurons, the percentage of neurons with somatosensory RFs, that is, bimodal neurons, was high (58.9%), but was lower (13.5%) in the “action observation” neurons. A chi-square test showed that the differences were highly significant (P < 0.001). In contrast, neurons that responded to self-movements were observed more frequently (51.7%) among the “action observation” neurons but less frequently (11.3%) among the “peripersonal stimuli” neurons (chi-square test, P < 0.001). Regarding the “moving stimuli” neurons, the percentage of visual-only neurons (i.e., no effect on neural activity when stimuli other than visual were observed) was significantly higher compared with the other neurons (53.4%, P < 0.005). The percentage was significantly lower in the “peripersonal stimuli” neurons (29.8%, P < 0.05).
Distribution of Visual/Auditory Neurons in the SII and Surrounding Regions
We investigated the distribution of visual neurons in the regions explored in the present study and found that they were distributed widely in the UBLS, covering the entire SII and its bordering regions, namely, the PR, Area 7b, and the Ri. In addition, the visual neurons were usually intermingled with unimodal somatosensory neurons.
We examined whether visual neurons were recorded together with somatosensory neurons in the same penetration. In most penetrations shown in Figures 2 and 3, both visual neurons and somatosensory neurons were recorded together. Only 2 penetrations had exclusively visual neurons (penetrations 10 and 13 in d of Fig. 3C). In all 8 hemispheres, the activities of visual neurons were recorded along a total of 119 penetrations. In most cases, visual neurons were recorded with somatosensory neurons with the exception of 14 penetrations, suggesting that visual neurons were usually distributed with somatosensory neurons at least in the regions explored in the present study.
Figures 2B and 3B depict the entry points of penetrations along which visual neurons were recorded (marked by red circles), indicating a wide distribution of visual penetration along the LS. Similar observations were also found in the other 6 hemispheres. In g–j of Figure 2, visual neurons are densely distributed near the exposed surface in Area 7b. In the deepest part of the UBLS (in j of Fig. 2), a few visual neurons were found. Those neurons were a mixed population of Area 7b, Ri, and SII neurons.
In a–f in Figure 2 and h–j in Figure 3, which are considered as the SII (already mentioned in the first subsection of the Results section), visual neurons were observed in the UBLS of all sections except for e in Figure 2. In more rostral sections, a–f in Figure 3, which correspond to the PR (see the first paragraph of Results section), many visual neurons were found in the deeper bank of the LS in 3 caudal sections (d–f in Fig. 3), whereas in the rostral-most sections, a–c in Figure 3, where most of the neurons had somatosensory RFs in the mouth, no visual neurons existed.
Trimodal neurons that responded to auditory stimuli, as well as to both visual and somatosensory stimuli, were also recorded in the SII (in b and j in Fig. 2 and i in Fig. 3). Accordingly, we concluded that the visual-/auditory-responsive neurons are distributed continuously along the LS from the bordering region of SII, 7b, and Ri to the adjoining PR covering the entire SII. The SII is a multisensory area receiving other sensory inputs in addition to somatosensory sensations.
In the present study, we recorded single-neuron activity in the PO, that is, the SII and surrounding 3 cortical regions, Area 7b, Ri, and PR (Fig. 1). The main aim of our study was to verify sensory inputs other than somesthetic information into the PO. The questions we tried to answer were as follows: 1) whether visual-/auditory-responsive neurons exist in the SII and 2) if indeed they exist, how are they distributed in the PO? Accordingly, the neurons present in the cortical regions surrounding the SII were not excluded. We described the locations of visual-/auditory-responsive neurons in Area 7b, Ri, and PR as well as in the SII. To investigate visual effects on neural responses, we applied various visual stimuli and found that a number of neurons exhibited visual properties. The visual neurons that we found rarely responded to static object presentation, but to rather complex stimuli, such as moving objects presented within and outside the monkey's peripersonal space, and observation of human actions. Furthermore, at least some of the visual neurons also responded to auditory stimuli. The visual-/auditory-responsive neurons were distributed throughout the entire SII, along the LS from the bordering region of SII, 7b, and Ri to the adjoining PR. This is the first study that demonstrated sensory inputs other than somesthetic, such as visual and auditory sensations in the SII in macaque monkeys.
In previous neurophysiological studies of awake macaque monkeys that investigated the response to visual and somatosensory stimuli in the PO, neurons that responded to visual stimuli were reported in Area 7b and Ri but not in the SII. Robinson and Burton (1980b, c) found bimodal neurons that responded to both visual and somatosensory stimuli in Area 7b buried in the UBLS, the region caudally adjacent to the SII. The authors mentioned that these bimodal neurons were found less frequently in the bank than those in the exposed cortical surface of Area 7b. A similar result was described by Dong et al. (1994) who studied the face representation region in Area 7b. Both studies claimed that the SII neurons do not respond to visual stimulation. In a caudal part of the LS bank region, the parietoinsula vestibular cortex (PIVC), Grüsser et al. (1990) observed neurons that responded to visual stimuli eliciting optokinetic responses in macaque monkeys. Recently, Chen et al. (2010, 2011) studied the PIVC and its surrounding areas, including the caudal-most part of the SII, with optical flow stimulation, but did not find responsiveness to visual stimulation. Even in the other primate groups such as New World and prosimian primates, visual responsive neurons in the PO including the SII have not been demonstrated electrophysiologically, at least among studies using awake monkeys. Therefore, it is believed that neurons that respond to visual stimulation are not very frequent in the PO and that no visual input exists in the SII.
In contrast with these previous studies, we found a number of visual neurons in the SII in the present study. What causes such differences in the results of similar electrophysiological studies? One of the possible reasons is the difference in the visual stimuli used between the experiments; the visual stimuli used in the previous studies were mostly rather simple (e.g., presentation of objects such as a small light, turning the room light on and off, etc.) (Robinson and Burton 1980b, c; Dong et al. 1994). However, the visual neurons found in the present study hardly responded to such simple stimuli, that is, among a total of 306 visual neurons, only 5 neurons responded to static object presentations (Table 1). Although Dong et al. (1994) adopted moving objects as visual stimuli in the peripersonal space of the monkey, they could not record visual responses in the SII. Their study examined only the face representation, but the explored area in the SII was limited to its caudal-most part (Fig. 1) where neurons represent the trunk/leg but not the face. This may explain why the authors missed neurons that responded to peripersonal visual stimulation in the SII. Robinson and Burton (1980b, c) also used moving objects as visual stimuli outside the monkey's peripersonal space but failed to find “moving-stimuli” visual neurons. The purpose of their studies was to investigate “somatosensory areas buried in the LS.” Therefore, the authors probably used somatosensory stimuli as the primary stimulation. On the other hand, the majority of “moving-stimuli” neurons found in our study were unimodal (“visual only” neurons in Table 1), and the frequency of bimodal neurons with somatosensory RFs was only ∼30%. Their failure to find “moving-stimuli” neurons could possibly be attributed to the low percentage of somatosensory responses among the “moving-stimuli” neurons.
In the present study, we also investigated the PR in one hemisphere and found neurons that responded to visual stimuli as well as somatosensory stimuli (Fig. 3). Only limited electrophysiological studies have been performed in the macaque PR. Ogawa et al. (1989) reported neurons with somatosensory RFs in the intraoral structures in a shallower part of the UBLS. Recently, a neuroanatomical study in monkeys showed rich neural connections of the PR with the SII (Krubitzer and Kaas 1990; Disbrow et al. 2003), suggesting that the PR is a somatosensory area in the rostral-most part of the PO. In fact, we observed that most of the neurons there had somatosensory RFs usually on the hand, mouth, or face, along with combined RFs. This is the first report that describes the properties of PR neurons using awake macaque monkeys, although the PR region explored in the present study was restricted to the deeper half of the upper bank. We found various types of visual neurons, including neurons that also responded to auditory stimulation. Accordingly, in the PO region, a multisensory area extends rostrocaudally covering the PR, the SII, Area 7b, and Ri.
A fraction of the 306 visual neurons identified in the present study were responsive to sound stimuli, thereby representing trimodal neurons, as all of them had somatosensory RFs. Robinson and Burton (1980b, c) investigated auditory responses in both the upper and lower banks of the LS. However, they did not report auditory neurons in the PO. This is the first report that demonstrates auditory input in the SII and PR in macaque monkeys. The number of confirmed trimodal neurons (n = 10) was small. However, this does not necessarily mean a scarce distribution of auditory-responsive neurons in the explored region within UBLS. First, auditory stimulation applied only to a limited number of visual neurons because it was difficult to maintain well-isolated neural activity after a long inspection of visual responsiveness. As a result, the number of neurons tested for auditory responsiveness was only about one-third of that of visual neurons. Furthermore, it is possible that we missed auditory neurons that would respond to other sounds not used in the present study.
Recently, anatomical connections between the SII and auditory related areas have been reported in both humans and monkeys. Disbrow et al. (2003) found that the rostral part of the macaque SII, which the authors termed the PV area, is connected with the temporal operculum region that corresponds to an auditory-associated area (Kaas and Hackett 2000). A more recent study using human diffusion tensor imaging revealed the neural link between the SII and an auditory-associated area (Ro et al. 2013). Considering the modulatory effects of auditory stimuli on neural activity in the SII, as reported in human imaging studies (Bremmer et al. 2001; Gazzola et al. 2006; Beauchamp and Ro 2008; Etzel et al. 2008), it is possible that auditory inputs are more common in the SII. Further investigation is necessary to illustrate the overall nature of multisensory input in the SII.
In our present study, the majority of visual neurons were categorized into 3 groups according to the differences in effective stimuli, that is, neurons responding to peripersonal stimuli, action observation, and moving stimuli. Additionally, the effect of somatosensory stimuli and self-movements of the forelimb on the visual neurons were clearly different among the 3 groups (Table 1). In particular, the “peripersonal” and “action observation” neurons show distinct response properties against somatosensory stimuli and self-movements. Most of the “peripersonal” neurons had tactile RFs whereas neurons activated during self-movements were observed less. Considering the high ratio of visual neurons with directional selectivity, where the approaching object was the most effective stimulus, the “peripersonal” neurons may contribute to the information processing of moving objects near the body and self-body being touched by the approaching objects (Graziano and Cooke 2006). In contrast, most of the “action observation” neurons responded to self-movements, but the neurons with tactile RFs were less frequently observed. These results suggest that this class of neurons may be involved in the information processing related to action execution irrespective of whether the actions were performed by the first or third person (Rizzolatti et al. 2014).
Visual neurons with similar response properties as of the 3 groups have been already reported in other cortical regions such as the ventral premotor areas (Gentilucci et al. 1988; di Pellegrino et al. 1992), the intraparietal areas (Colby et al. 1993; Iriki et al. 1996; Graziano et al. 2000), the inferior parietal areas (Leinonen et al. 1979; Gallese et al. 2002; Rozzi et al. 2008), and the superior temporal areas (Bruce et al. 1981; Perrett et al. 1990). Neurons in the frontal and parietal areas are thought to contribute to controlling visually guided forelimb movements and spatial recognition of objects including the self-body. Neuroanatomical studies have found rich connections of the SII with those cortical regions (Matelli et al. 1986; Cipolloni and Pandya 1999; Disbrow et al. 2003; Borra et al. 2008; Gerbella et al. 2011; Gharbawie et al. 2011), suggesting that the most plausible visual sensory input in the SII comes from those higher associative cortices. Considering the absence of cortical connections with unimodal visual sensory areas in the SII and the scarce responsiveness of SII neurons to simple visual stimulation, the SII is not the area that processes visual information; rather, it only receives previously processed information.
Investigation of somatosensory input in the SII in macaque monkeys has indicated that the SII has rich connections with the SI (Robinson and Burton 1980a; Burton et al. 1995). Therefore, the SII is thought to be the part of the cortex that contributes solely to processing tactile information for object recognition (Hsiao 2008). Recently, more attention has been paid to SII neural activity during forelimb movements, especially the hand, suggesting SII function as a motor control based on processing proprioception (Binkofski et al. 1999; Hinkley et al. 2007; Taoka et al. 2013). The present study, however, found that the SII is a multisensory area. This emphasizes the need for a novel definition of the functional role of the SII other than the attributed sensorimotor function arising from the idea that the SII is a unimodal sensory area.
This work was supported by JSPS KAKENHI Grant Number 17500205. Funding to pay the Open Access publication charges for this article was provided by RIKEN Brain Science Institute.
Conflict of Interest: None declared.