## Abstract

There is growing evidence that in primate cerebral cortex the areas along the ‘dorsal pathway’ are involved in the transformation of visual motion information towards a motor command. To pursue this cortical flow of information from visual motion areas to the motor cortex, single-cell activity was recorded from visual areas MT/MST (middle temporal area/medial superior temporal area) and from primary motor cortex (M1) while monkeys tracked moving targets with their right hand. Spike activity of 353 directionally tuned motor cortex cells was combined to a time-varying population vector, and similarly a time-resolved visual population vector was calculated from 252 MT/MST cells. Both population vectors code faithfully for the direction of the collinear motion of target and hand. For a given direction, the length of the population vectors varied over time during the performance of the task. The temporal evolution of both population responses reflects the different relationship between the early visual responses to the moving target and the directional motor command controlling the hand movement. The results indicate that during the visual tracking task visual and motor populations which code for similar directions of movement are co-activated with considerable temporal overlap. Despite this co-activation in both modalities, we failed to observe any significant synchronization between areas MT/MST and M1.

## Introduction

When we perform reaching movements to moving targets, our visual system has to process the information about target motion, and this information is finally used to update the motor system during the control of hand movements. The cortical mechanisms involved in such a continuous transformation of visual information towards a motor action are not well understood. The cortical areas along the ‘dorsal stream’ are candidates for such a transformation, as they combine the processing of visual information with neuronal activity related to eye and limb movements (Ungerleider and Mishkin, 1982; Maunsell and Newsome, 1987; Van Essen et al., 1992). In recent years there have been several studies which aimed to analyze the different levels of visuo-motor transformation along the areas of the dorsal stream (Battaglia-Mayer et al., 1998, 2000; Goodale, 1998; Burnod et al., 1999). From this work one can conclude that the control of visually guided reaching requires a coordinated activation of multiple areas located in the medio-temporal and parietal cortex. To our knowledge, there has been no approach in primates to compare spike activity recorded simultaneously from different areas along this pathway during a functionally demanding visuo-motor task. In our experiments, we therefore set out to combine recordings from two positions along the dorsal ‘vision-for-action’ pathway while a monkey performed visually guided tracking movements. We selected visual motion areas MT/MST (middle temporal area/medial superior temporal area) and the primary motor cortex (M1) for simultaneous recordings with two multi-electrode systems to compare cortical activity on a visual ‘input level’ with neuronal activity from a motor cortical ‘output level’. In the current study, we searched for possible neuronal interaction on the basis of population responses calculated separately for directionally tuned visual and motor cells. We also analyzed all pairs of cells recorded simultaneously from visual and motor areas to test for possible synchronization of neuronal activity between these areas.

The areas MT and MST, located at the superior temporal sulcus, are specialized for the coding of the direction and velocity of moving stimuli. These areas can be regarded as an entry point of visual motion processing to the dorsal pathway. Cells in areas MT and MST are most sensitive to moving stimuli (Dubner and Zeki, 1971; Zeki, 1974; van Essen et al., 1981; Maunsell and van Essen, 1983a,b; Tanaka et al., 1986). They project to cortical area 7a and to other areas within the intraparietal sulcus: namely, the lateral intraparietal area (LIP) the ventral intraparietal area (VIP) (Boussaoud et al., 1990) and to the parieto-occipital area (Colby et al., 1988). Posterior parietal areas themselves are linked with premotor cortex, the superior colliculus and pontine nuclei (Wise et al., 1997; Lacquaniti and Caminiti, 1998). All these areas are known to influence various aspects of the visual control of eye, limb and body movements. In summary, one can conclude that the dorsal stream has the functional properties and interconnections that are needed for the moment-to-moment control of visually guided actions.

The main motor output of the cerebral cortex is routed through M1, which has a prominent role in the specification, initiation and execution of motor acts. The discharge of neurons in M1 relates to muscular activity, output force and torque (Georgopoulos et al., 1992; Wise, 1993) and there is now strong evidence that it also relates to the kinematics of movement, i.e. parameters such as direction and velocity, and to higher-order processing of sensorimotor information (Georgopoulos et al., 1986; Carpenter et al., 1999; Port et al., 2001; Lee et al., 2001). The majority of cells in the arm area of M1 is directionally tuned and most cells show a single preferred direction. Based on the ubiquitous presence of direction selectivity in the activity of motor cortical cells, the directional signal from a large number of cells can be read out as a population vector (Georgopoulos et al., 1986; Schwartz, 1994). Such a weighted vector sum of the population activity gave access to the temporal dynamics of the cortical control during ongoing hand movements (Schwartz, 1993) and it has been used successfully to monitor covert operations during cognitive tasks (Carpenter et al., 1999).

With regard to directional coding, the functional properties of direction selective cells in the visual motion areas MT and MST show similarities to cells in M1. Cells in MT and MST typically respond also with a high rate for a particular direction of stimulus movement and progressively less for directions further away from the preferred direction. Accordingly, we extended the population vector analyses to the data obtained from the visual motion areas. By processing the data from both modalities with an identical analysis, we are able to directly compare visual and motor activity on the population level. The population vectors from visual and motor areas code accurately for the direction of movement in retinal and in hand-centered coordinates, respectively. In a second step of analysis, we were able to elucidate the temporal relationship between the activity on the population level. With this approach, we here show that during visually guided tracking movements, the visual motion areas MT and MST mainly follow the time course of stimulus movement with 80 ms latency, whereas, at the same time, the motor cortex prepares for the upcoming linear tracking movement with a lead time of up to 300 ms. The motor activity was closely related to the kinematics of the hand, and the visual population response was dominated by the movement of the visual target. Additional to these well-known relationships, we could show that the motor population was partially related to parameters of the visual stimulus guiding the movement. However, the visual response did show only a weak relation to hand velocity which appeared too late in time to have influence on the control of movement.

The current paper proves that on the population level, simultaneous activity in both areas is present during visually guided tracking. The prerequisite for the cross-correlation study — a simultaneous activation in different modalities during a single behavioral task — was fulfilled. According to the so-called ‘binding hypothesis’ [for review, see (von der Malsburg, 1999; Singer, 1999)] one might expect that the distributed activity in spatially separated cortical areas show some amount of temporal synchronization, which could be interpreted as a sign of a dynamical cooperation between these areas. It is obvious that the visual areas MT/MST and M1 have quite different functional roles in an (hypothetical) ‘vision to action’ pathway. On the other hand, the idea seemed tempting to us that the behavioral demands of the tracking task should be sufficient to generate task related cortical synchronization and we therefore analyzed the simultaneously recorded cell activity from both areas for any sign of synchronization. However, with cross-correlation analysis of single cell spike trains we have not detected any sign of interaction between pairs of cells from M1 and visual areas MT and MST.

## Materials and Methods

Single-cell recordings were conducted in awake, behaving monkeys performing visually guided manual tracking movements. Recordings were made from two hemispheres of two monkeys [two male monkeys (Macaca mulatta, 5.5 and 6.1 kg)]. All procedures were in accordance with published guidelines on the use of animals in research (European Communities Council Directive 86/609/ECC).

### Animal Preparation

All monkeys were surgically prepared for chronic neurophysiological recordings. Monkeys were pretreated with atropine and sedated with ketamine hydrochloride. Under general anesthesia [pentobarbital sodium, 10 mg/kg i.v.] and sterile surgical conditions each animal was implanted with a device for holding the head. A scleral search coil was implanted to monitor eye position according to the method published by Judge et al. (Judge et al., 1980) and was connected to a plug on top of the scull. In both animals, two recording chambers were implanted over two separate craniotomies to allow recordings with two independent multi-electrode systems. One chamber was placed over the occipital cortex in a parasagittal stereotaxic plane tilted back 60 from the vertical. A second chamber was implanted over the frontal cortex centered just anterior to the central sulcus. The placement of both chambers was guided by magnetic resonance images, which were taken from each animal before the first surgery. Recording chambers, eye coil plug, and head holder were all embedded in dental acrylic that itself was anchored to the skull by self-tapping screws. Analgesics were applied postoperatively and recordings started no sooner than 1 week after surgery.

### Experimental Set-up for Control of Behavioral Data

During the experiments, the animals were seated comfortably in a primate chair allowing them to move both hands freely. Both animals were trained to use their right hand only for moving a two-joint manipulandum placed in front of them in a horizontal plane above the level of their hip. The sensor of a digitizing table was attached to the handle of the manipulandum and was moving almost frictionless above the digitizing table. This allowed to measure the position of the hand with a spatial resolution of 0.1 mm. The hand position data were sampled with 75 Hz by a PC and were displayed in real time as a feedback cursor (red dot, radius 1.4° visual angle). All visual stimuli were presented on a translucent vertical screen placed 114 cm in front of the animal which subtended a viewing angle 57° wide and 43° high. Moving the handle of the manipulandum 10 mm towards the screen caused an upward movement of the feedback cursor by 1.17° visual angle. During the experiment, black curtains darkened the area surrounding the animal.

The feedback cursor and all additional visual stimuli were generated by a single PC which controlled the hand data as well. A high performance graphic board (ELSA Winner, 2000 Pro/X, Aachen, Germany) served to generate the real time video output, which was back-projected to the translucent screen with a video projection system (Electrohome ECP 4100, Kitchener, Ontario, Canada, 75 Hz frame rate, 800 × 600 pixels video resolution). Eye position was measured with the scleral search coil technique and the analog output of the eye monitor system (Primelec, Regensdorf, Switzerland) was fed to the same control PC. By this setup, hand and eye position were continuously controlled with a single PC and the information about the hand position was used in real time to update the video display of the feedback cursor. The same computer was used to store all neuronal data collected during the experiments. The software for the generation of the visual stimuli, the online control of the animals behavior and for the collection of all data was developed by one of the authors (W.K.).

### Visually Guided Tracking Task

During all phases of the task, the monkey had to fixate a stationary, central green spot which was always drawn on top of all other stimuli on the vertical screen. The fixation window had a radius 2.1° in visual space and was not visible to the animal. A trial was aborted immediately if the animal broke fixation. This requirement of central and immobile fixation was included to avoid any possible influence of eye position on cell activity.

### Center→Out Task

To obtain the directional tuning of cells from M1, the animals were trained to make straight movements from a central start position to one of eight equally distributed peripheral targets (70 mm in hand space). This center→out task has been used in a similar form by several other investigators before [e.g. (Georgopoulos et al., 1982; Fu et al., 1993]. In contrast to previous studies, we added the requirement for the central fixation during this task to avoid switching between conditions with constrained eye movements (as during the tracking task) and conditions with free eye movements. The eight center→out conditions were presented to the animals intermingled with the four tracking conditions in a randomized block design. We recorded a minimum of five successfully completed repetitions from four tracking conditions and eight center→out conditions.

### Visual Control Task

The preferred directions of the visual cells were measured by having the monkeys fixate a central spot and moving a whole field random dot pattern (57° × 43° visual angle) on a circular pathway (7° radius) in clockwise or counterclockwise direction. The computer-generated pattern was always covering the full screen, causing the impression as it moves behind a large aperture. The translatory motion of the random pattern along the circular path produced at a fixed position on the screen (for example, at a receptive field of a neuron) a movement in continuously changing directions (Schoppmann and Hoffmann, 1976; Hoffmann and Distler, 1989). Activity of all direction sensitive cells was modulated according to the continuous change of stimulus direction in its receptive field. During completion of a full circle, all possible directions were covered. In a single trial, the pattern moved for 3.5 s, and needed 3.33 s to complete the circular path. For each cell, a minimum of five repetitions with translations in clockwise and counterclockwise directions was measured. Additionally, the receptive fields of most visual cells were determined qualitatively with handheld stimuli while the animals were fixating.

### Extracellular Recording and Data Acquisition

Neuronal activity was recorded from M1 and from areas MT and MST at the superior temporal sulcus using two separate multi-electrode manipulators (Thomas Recording, Giessen, Germany). Activity of single cells was detected in real-time by means of a computer controlled multi-channel spike sorter (Plexon Inc., Dallas, TX). Time stamps for detected spikes were stored with 10 μs resolution by the same PC controlling the behavior of the animal.

### Calculation of Preferred Directions of Motor Cells

The directional tuning of the motor cells was calculated from the discharge rates during five repetitions of the center→out movements in eight directions. The mean spike discharge rate for each direction of movement was calculated for the time from the appearance of the peripheral target until the feedback cursor reached the target. The preferred direction was calculated using standard directional statistics [‘mean direction’ (Mardia, 1972)]. The statistical significance of the directional tuning was tested with a non-parametric, statistical bootstrapping technique, similar to a method used by Lurito (Lurito et al., 1991). For this test, the length of the mean resultant, R, was calculated (Mardia, 1972) using the discharge rates during all 40 movements to weight the corresponding movement directions. A new sample was generated by assigning randomly the observed discharge rates to movement directions, and the mean resultant was calculated. This procedure was repeated 100 times, and the lengths of 100 resultants were rank ordered. If the length of the observed mean resultant, R, was greater than the 95th percentile in the distribution of the mean resultants, the cell was considered to be directionally tuned. If the cell was directionally tuned, the direction of the mean resultant was taken as the cell's preferred direction. This test was chosen to allow a similar approach for the data from motor and visual cortex without making assumptions about a particular shape of the directional tuning (e.g. a cosine model).

### Calculation of Preferred Directions of Visual Cells

The spike activity during the visual control task was analyzed to obtain the directional tuning of the visual cells. To avoid an influence of transient responses to motion onset, the activity during the first 150 ms after the start of the pattern motion was excluded. Each spike measured during the following 3.333 ms (time for the completion of pattern motion on a full circular path), was transformed in a unit vector pointing in the direction of stimulus movement at the time of spike occurrence. For the pattern movements in clockwise and counterclockwise direction, the sum of all unit vectors was calculated separately. The preferred direction of a cell was taken as the mean vector of both directions. A possible influence of the response latency on the preferred direction of a cell was cancelled out by averaging data from clockwise and counterclockwise translation.

For testing the statistical significance of the preferred direction of the visual cells, we again used the non-parametric, statistical bootstrapping technique described above. For this test, the spike trains of an individual trial were rearranged by shuffling the sequence of spike intervals, and each spike was transformed in a unit vector pointing in the direction of stimulus motion at the corresponding time. The resultant vector constructed from the shuffled data was noted. The procedure was repeated 100 times for each trial recorded during clockwise and counterclockwise stimulus movement, and the length of the resulting vectors were rank ordered. If the length of the observed mean, R, was greater than the 95th percentile in the distribution of the shuffled mean resultants, the directional tuning for the given condition (clockwise or counter-clockwise) was considered to be significant. Only when this 95th percentile was reached for both stimulus directions, the average direction from both stimulus directions (clockwise and counterclockwise) was taken as the preferred direction of the cell.

### Calculation of Neuronal Population Vectors

The neuronal population vector is the weighted sum of vectorial contributions of individual cells (Georgopoulos et al., 1988). For the calculation of the population vector, peristimulus time histograms (13.3 ms bin-width, 75 Hz) were computed for each cell which proved to have a statistically significant preferred direction. This vector sum was calculated in an ongoing, time-varying fashion for all four conditions, and we used counts of fractional intervals as a measure of the intensity of cell discharge. For a given time bin, each cell made a vectorial contribution in the direction of its preferred direction and of magnitude equal to the change in cell activity compared to the rate observed during the last 0.5 s preceding the onset of the moving bar in the periphery (‘control rate’, that is, while the monkey was fixating the center and while holding the handle at the center of the plane). The population vector P for the jth stimulus condition (i.e. tracking direction) and the kth time bin is

$\mathbf{P}_{\mathit{jk}}\ =\ {{\sum}_{\mathit{i}}^{\mathit{N}}}\ \mathit{w}_{\mathit{ijk}}\mathbf{C}_{\mathit{i}}$
where Ci is the preferred direction of the ith cell and wi,j,k is a weighting function
$\mathit{w}_{\mathit{i,j,k}}\ {=}\ \mathit{d}\ _{\mathit{i,j,k}}\ {\mbox{--}}\ \mathit{a}_{\mathit{i}}$
where di,j,k is the discharge rate of the ith cell for the jth conditions and kth time bin, and ai is the control rate for the ith cell.

The data from M1 and from visual areas MT/MST were combined separately in two different population vectors which evolved over time during the performance of the tracking task.

### Multiple Linear Regression of Population Vector

To quantify the relation of the population vectors to the kinematics of the target and the hand, we performed a multiple linear regression of the population response with the averaged time course of hand and target kinematics. In this regression, the length of the population vector was expressed as a function of hand position, hand velocity, target position and target velocity. To analyze the temporal relationship between these parameters and the population activity, we shifted the data of the hand and the target independently relative to the population data. Accordingly, we calculated a single regression for each combination of time shifts:

$\mathit{f}_{\mathit{t}}\ {=}\ \mathit{b}_{0}\ {+}\ \mathit{b}_{1}\mathit{position}^{hand}_{\mathit{t}{+}{\tau}_{1}}\ {+}\ \mathit{b}_{2}\mathit{velocity}^{hand}_{\mathit{t}{+}{\tau}_{1}}\ \mathit{t}\ {+}\ {\tau}_{1}\ {\leq}\ \mathit{T}\ {+}\ \mathit{b}_{3}\mathit{position}^{target}_{\mathit{t}{+}{\tau}_{2}}\ {+}\ \mathit{b}_{4}\mathit{velocity}^{target}_{\mathit{t}{+}{\tau}_{2}}\ {+}\ {\varepsilon}_{\mathit{t}}\ \mathit{t}\ {+}\ {\tau}_{2}\ {\leq}\ \mathit{T}$
where b0, . . ., b4 are regression coefficients, ε is an error term, and T is the period from 500 ms prior to onset of target movement until the delivery of the reward. The inequalities mean that the position and velocity data included within the shifted time courses were always contained in the behavioral meaningful period T. This approach was inspired by a study of Ashe and Georgopoulos (Ashe and Georgopoulos, 1994), where a comparable analysis was performed by expressing the ongoing impulse activity of single cells as a function of target direction and of position, velocity and acceleration of the hand.

### Cross-correlation Analysis

To analyze the temporal structure in the spike trains of simultaneously recorded pairs of cells from areas MT/MST and M1, we calculated cross-coincidence histograms (CCHs) from the corresponding spike trains. The CCH comprises all intervals between the spikes from both cells within the time window under study, up to a maximal delay of typically ±128 ms. All CCHs were computed with a bin width of 1 ms. Only cells with a total of more than 500 spikes during the time course of the recording were included in this analysis. Each CCH was tested for any prominent modulation (oscillatory or non-oscillatory) indicating synchronized activity in both cells. The particular steps of the synchronization analysis was guided by a previous study of our group (Cardoso de Oliveira et al., 1997), in which non-oscillatory synchronization in area MT has been observed especially during an expectation period while the monkey expected a low contrast pattern in a direction discrimination task. Accordingly to the previous study, we analyzed only CCHs which contained more than 1000 entries with a maximum delay ofless than 100 ms. Briefly, the subsequent steps were as follows: the shift-predictor was low-pass filtered and subtracted from the raw CCH and the resulting difference correlogram was expressed in Z scores, i.e. in units of standard deviation. Only peaks in this difference correlograms which exceeded a Z score of 3.0 were taken as significant. To avoid false positives, two additional criterion had to be fulfilled: (i) the peak had to be significant after filtering the correlogram by a three-point averaging filter which assured that the peak was not caused by a single bin exceeding the significance level, (ii) we tested whether the correlation was consistently detectable when the trials were divided in two sub-groups. Only if in both groups a significant correlation occurred, the cell pair was scored as significantly correlated.

### Histology and Reconstruction

During the last days of recording we made electrolytic microlesions (10 μA for 12 s) in the motor cortex and in the recording area located at the superior temporal sulcus of both monkeys. Standard histological procedures were used to identify the location of the electrolytic lesions and to reconstruct the relative spatial position of the electrode tracks during the recording sessions from the first monkey. All recording positions located at the superior temporal sulcus which yielded cells with significant directional bias were compatible with a location in area MT or area MST. Based on this preliminary separation between MT and MST cells, we calculated two distinct population vectors for MT and MST cells from the first animal. We could not find any significant difference between results obtained from these sub-populations and therefore combined the data from all directionally tuned visual cells to a single population vector. The recording positions from the precentral cortex proved to be located in the area rostral of the anterior bank of the central sulcus, which is in correspondence to the functional properties of the cells showing a clear relation to movements of the proximal arm.

## Results

### Directional Tuning of Cells from Visual and Motor Areas

We recorded the activity of 605 arm-related cells from M1 from two animals. Of those, 294 and 311 cells were recorded in the first and second monkey, respectively. Most cells changed activity in relation to proximal movements of the contralateral arm as judged by examination of the animal outside the behavioral task. A fraction of 353 motor cells (58.3%) showed a significant directional tuning in the center→out task. For the individual animals, this ratio was 189 out of 294 (= 64.3%) for the first monkey and 164 out of 311 (= 52.7%) for the second monkey.

From 426 cells located in visual areas MT and MST we recorded the spike activity during the visually guided tracking task, whereof 174 cells were from the first and 252 cells were from the second monkey. Two hundred and fifty-two cells (59.3%) showed a statistically significant preferred direction when tested in the visual control task (64.4 and 55.6% of the cells in the first and second monkey, respectively). The distribution of preferred directions of the directionally tuned cells from both the visual and the motor cells ranged throughout the directional continuum (Fig. 2). None of the visual and motor sub-populations from either animal showed any statistically significant directional bias (Rayleigh test for uniformity, P > 0.1 in both cases).

### Motor Population Vector

The mean velocity profile of the hand does not match the constant velocity of the target very well. This is probably related to the relative high velocity of the tracking movement the monkey had to perform. To ensure that the moving target causes an appropriate activation in areas MT and MST, a visual velocity of the target of 7°/s was selected, which is already at the lower end of velocities preferred by these visual motion areas. A higher speed of the moving target would have increased the difficulties for the monkey to make proper tracking movements.

### Visual Population Vector

Taking the data from one hemisphere only, the gradients in the population response during the horizontal movements could be interpreted as a code for the instantaneous distance of the target from the horizontal meridian. However, the positional information of the visual target is encoded also through the retinotopic organization of area MT, and this coding is not well captured by the vector interpretation of the cell response [in contrast to a positional population code, for example used by Jancke et al. (Jancke et al., 1999)]. With the mirroring of the vector data we stayed in the conceptual frame of the vector model, which favors the correspondence between the magnitude of the visual population vector and target speed.

The same transformation of mirroring the horizontal component and subsequent averaging was performed for the motor data, and the results for both populations during the horizontal movements are shown in Figure 5. For better visualization of the data, vectors were rotated 90° counterclockwise so that vectors coding for rightward movement are now pointing upwards. This rotation avoids that most vectors fall in line with the temporal axis and allows a better estimation of the vector's length. This rotation is purely for visualization and does not affect the data or has any implications for further steps of the analysis.

### Results from Multiple Linear Regression

$f_{\mathit{t}}\ {=}\ {\mbox{--}}0.02\ {+}\ 5.06\mathit{position}^{hand}_{\mathit{t}{+}{\tau}_{1}}\ {+}\ 22.84\mathit{velocity}^{hand}_{\mathit{t}{+}{\tau}_{1}}\ {+}\ 2.26\mathit{position}^{target}_{\mathit{t}{+}{\tau}_{2}}\ {+}\ 13.16\mathit{velocity}^{target}_{t{+}{\tau}_{2}}\ {+}\ {\varepsilon}_{\mathit{t}}$
with τ1 = +293 ms and τ2 = –240 ms.

For the visual population vector, the similar analysis yielded a maximum R2 = 0.953 for a shift of hand data by –260 ms and a shift of target data by –80 ms. The maximum value for regression was obtained with hand data from 260 ms before and the response to the visual stimulus movement 80 ms before. The corresponding regression equation was:

$f_{\mathit{t}}\ {=}\ {\mbox{--}}0.02\ {+}\ 0.10\mathit{position}^{hand}_{\mathit{t}{+}{\tau}_{1}}\ {+}\ 7.58\mathit{velocity}^{hand}_{\mathit{t}{+}{\tau}_{1}}\ {\mbox{--}}\ 2.17\mathit{position}^{target}_{\mathit{t}{+}{\tau}_{2}}\ {+}\ 24.98\mathit{velocity}^{target}_{\mathit{t}{+}{\tau}_{2}}\ {+}\ {\varepsilon}_{\mathit{t}}$
with a shift of the hand data by τ1 = –260 ms and τ2 = –80 ms.

In addition to the temporal relationship between the kinematic parameters and the length of the population vector, we wanted to evaluate the relative importance of all parameters entered in the multiple linear regression. To compare the relative importance of a given variable in the regression equation, we calculated the standardized regression coefficients, which were obtained by expressing the observations as Z scores (i.e. in standard deviation units). This facilitates a comparison among variables with different units (i. e. position versus speed). The standardized coefficients were calculated for the regressions yielding the highest R2 values. Rank ordering the standardized coefficients showed that for the motor population vector, the speed of the hand is the most important parameter, followed by target speed, hand position and target position. To test whether the length of the population vector was related to one or more of the parameters tested, the t statistic and its probability level were calculated for each coefficient. For the motor vector, all four parameters tested contributed significantly at the 5% level (after Bonferroni correction for multiple comparisons) to the variation of the population vector. For the visual population vector, target speed ranked highest, followed by hand speed. Both hand and target position did not contribute significantly to the regression model of the visual population vector.

## Discussion

This study compares population activity in M1 and in visual motion areas MT and MST during visually guided tracking movements. A behavioral task was designed in such a way that the visual information about the moving target was essential to fulfill the spatial and temporal requirement of the motor response. We found that both areas were active over an extended period of time during the manual tracking, and that the direction of movement is coded faithfully and co-linear in both areas on the population level. The temporal evolution of activation in the motor and in the visual areas was dominated by the velocity profile of the corresponding hand and target movement, respectively. These results emphasize the notion that a continuous processing across multiple areas takes place during visually guided hand movements.

Based on the co-activation of both populations coding for similar movement directions, we set out to gain insight about the temporal aspects of the transformation of visual motion information towards a motor command and the possible interaction between these areas. Since movement direction was constant during the task, the parameter that varied in time was speed. We therefore collapsed the data obtained during movements in different tracking directions to a single set of data. In the subsequently obtained ‘generalized’ population activity, the temporal variability of the population response was most closely related to the velocity profiles of hand and target.

The temporal relationship between the kinematics of target and hand movement and the length of the population vector was further analyzed by multiple linear regression. The regression analysis revealed that during the visual tracking task the motor cortex codes for the kinematics of upcoming movement with a lead time of ~300 ms. The visual motion areas MT and MST follow the onset of stimulus motion by ~80 ms.

In previous studies that analyzed the motor population vector, a time lag of 120 ms between motor cortical activity and the limb movement has been described for continuous drawing movements (Schwartz, 1993). A study by Ashe and Georgopoulos (Ashe and Georgopoulos, 1994) compared the coding of movement parameters in M1 and area 5 during center→out movements. They used a comparable multiple linear regression model to analyze single-cell activity from both areas and obtained a median shift for the highest R2 of –90 and +30 ms for motor cortex and area 5, respectively. The remarkably extended lead time of the motor population during the tracking task in our study might be accounted for by the fact that the direction of the movement was indicated quite early during the task. As the target started to move 1250 ms before the animal had to start the hand movement, the information about the movement reached the motor cortex early before movement onset. The latency of the visual response to the onset of the target is in similar range as latencies obtained from single-cell responses during visual stimulation with flashed stimuli (Schmolesky et al., 1998).

Motor cortex activity as well as visual driven activity in MT and MST is most tightly coupled to the velocity of the hand and the target, respectively. A similar ranking was observed by Ashe and Georgopoulos (Ashe and Georgopoulos, 1994) in M1. In their study, they found a strong influence by the direction of movement in 55% of the cells. Hand velocity influenced most strongly the activity of 27% of the cells, followed by hand position and hand acceleration. In our population data, the influence of motion direction is implicitly included in the direction of the population vector. As we averaged the population data from different directions, the direction of movement is not included as a parameter in the regression. In our data, the regression of the visual population vector underlines the functional specialization of areas MT and MST for the processing of motion information. The results from our regression analysis emphasizes the tight coupling to the velocity of the visual stimulus.

### Possible Influence of Set-related Activity on Preferred Direction of Motor Cells

A comparable visually guided tracking task has been used in a study that focused on the activity in M1 and in the cerebellum (Johnson et al., 1999). In a fraction of cells from M1, the authors described a significant change in the preferred direction during the time course of the tracking. In contrast to their study, we tested the preferred directions of the motor cells with a separate center→out task. We used the preferred directions derived from this control task as constant values for the calculation of the population vector. The change in preferred direction observed by Johnson et al. (Johnson et al., 1999) was equally distributed without any bias towards a particular direction. We therefore do not expect that such a possible modulation of direction preference had a systematic influence on the resulting population vector in our analysis.

### Simultaneous Recordings Do Not Reveal Inter-modal Synchronization

The negative result from the synchronization analysis might not be too surprising as the visual and motor cortical areas under study are separated by several cortical processing levels or even cerebro-cerebellar loops. On the other hand, a main proposal of the binding hypothesis gives rise to the expectation that cortical activity could be dynamically synchronized, even when the corresponding areas are separated by large distances (Eckhorn et al., 1988; Engel et al., 1991) [for a review see (Singer, 1999)]. Given that the cortical activity is related to a single external object — or to a single behavioral action — the neuronal coding of such an entity might be particularly supported by internally generated synchronization. In the conceptual frame of the binding hypothesis, such synchronization should even occur across different modalities, for example, to support the linking of the dispersed neuronal representation of a single object (Roelfsema et al., 1997). The behavioral visuo-motor task used in our study was designed partly with respect to these requirements, as real-time processing of visual information was mandatory to control the sustained motor response. We failed to elicit inter-modal neuronal synchronization on the level of paired spike trains recorded simultaneously during several repetitions of the behavioral task. However, in ~20% of the neurons recorded from MT and MST, we observed pronounced oscillatory modulation of spike activity in the gamma range (>40 Hz), which was synchronized without phase lag between spatially distant visual cells. Such stimulus dependent oscillatory modulation and synchronization in areas MT/MST was observed only during a purely visual task which did not require a manual motor response (for example, our visual control task described in the Materials and Methods section). These stimulus-dependent oscillations and their synchronization in areas MT/MST will be described in a separate paper. During the manual tracking task, we did not find any such oscillatory activity in areas MT/MST nor any synchronization between neuronal activity in MT/MST and M1.

Figure 1.

Scheme of visually guided tracking task. (A) Initial center hold phase: only the fixation target (green dot in the center of the screen), the manual feedback cursor (filled red circle, 1.4° radius) and a center hold window (open white circle, 2.1° radius) were visible to the animal. The monkey had to fixate the central dot and, at the same time, keep the feedback cursor in the center hold window for a randomized center hold time (800–1500 ms). (B) Pre-tracking phase: a light gray bar (14° long and 1.4° wide) appeared in the periphery in one of four possible locations (8.75° from the center) and started immediately to move with constant velocity (7°/s) towards the center of the screen. During this period, the animal had to maintain the initial hand and eye position for an additional 1250 ms. (C) Tracking phase: at the end of the pre-tracking phase, the bar crossed the mid-position of the center hold window. This mid-position also equals the location of the fixation target. At that time, the center hold window disappears and the monkey had to start a manual tracking movement during which the maximum distance between the center of the bar (marked by a white circle) and the feedback cursor is always less than 2.1° (corresponding to a window of 18 mm radius in hand space). (D) Target hold phase. The movement of the bar stopped 1250 ms after the mid-point has been crossed, when a path symmetrical to the center of the screen is completed. In this position, the bar remained visible for an additional second, during which the monkey had to maintain the hand position in the spatial window of 18 mm radius.

Figure 1.

Scheme of visually guided tracking task. (A) Initial center hold phase: only the fixation target (green dot in the center of the screen), the manual feedback cursor (filled red circle, 1.4° radius) and a center hold window (open white circle, 2.1° radius) were visible to the animal. The monkey had to fixate the central dot and, at the same time, keep the feedback cursor in the center hold window for a randomized center hold time (800–1500 ms). (B) Pre-tracking phase: a light gray bar (14° long and 1.4° wide) appeared in the periphery in one of four possible locations (8.75° from the center) and started immediately to move with constant velocity (7°/s) towards the center of the screen. During this period, the animal had to maintain the initial hand and eye position for an additional 1250 ms. (C) Tracking phase: at the end of the pre-tracking phase, the bar crossed the mid-position of the center hold window. This mid-position also equals the location of the fixation target. At that time, the center hold window disappears and the monkey had to start a manual tracking movement during which the maximum distance between the center of the bar (marked by a white circle) and the feedback cursor is always less than 2.1° (corresponding to a window of 18 mm radius in hand space). (D) Target hold phase. The movement of the bar stopped 1250 ms after the mid-point has been crossed, when a path symmetrical to the center of the screen is completed. In this position, the bar remained visible for an additional second, during which the monkey had to maintain the hand position in the spatial window of 18 mm radius.

Figure 2.

Distribution of preferred directions of cells from both monkeys. (A and B) Preferred directions of all cells from M1 showing a statistically significant directional tuning during the center→out task. (A) 189 cells from monkey 1, (B) 164 cells from monkey 2. (C and D) Preferred directions of visual cells recorded in areas MT and MST that proved to have a significant directional tuning in the visual control task. (C) 112 cells from monkey 1, (D) 140 cells from monkey 2. Note that all sub-populations cover the directional continuum without having any strong directional bias.

Figure 2.

Distribution of preferred directions of cells from both monkeys. (A and B) Preferred directions of all cells from M1 showing a statistically significant directional tuning during the center→out task. (A) 189 cells from monkey 1, (B) 164 cells from monkey 2. (C and D) Preferred directions of visual cells recorded in areas MT and MST that proved to have a significant directional tuning in the visual control task. (C) 112 cells from monkey 1, (D) 140 cells from monkey 2. Note that all sub-populations cover the directional continuum without having any strong directional bias.

Figure 3.

Temporal evolution of population vectors for the (A) motor population and (B) visual population. For both populations, the time-varying population vector is shown during the tracking movements in four different directions (indicated by arrows on the left). For each condition, the population vector is calculated every 13.3 ms and plotted as a line originating at the corresponding value on the time axis. These vectors show the temporal variability in the directional signal encoded in the visual and motor population. For horizontal movements, the population vectors tends to point in the direction of the time axis. Therefore, the gray line is added as a description of the actual length of the population vector. The continuous thin and thick lines give the time course of the position and velocity of the target, respectively. The dotted thin and thick lines represent the averaged hand position and velocity from all trials contributing to the population vector. Before averaging, all data were aligned to target onset (left vertical dashed line). Only the position and velocity traces collinear to the movement direction of the target are plotted (i.e. the x-components during horizontal movements and the y-components during vertical movements). Note that the direction of the motor population vector shows only little variability in a single condition and codes for the similar direction in the time before and during the manual tracking. The length of the population shows a similar time course for all four directions of tracking movements. The visual data show a slightly higher variability in the direction of the population vector, and a consistent pattern in the temporal evolution of vector length is missing. The differences in the temporal evolution of the population vector length during the horizontal movements is probably a consequence of the fact that most receptive fields were located in the right hemifield.

Figure 3.

Temporal evolution of population vectors for the (A) motor population and (B) visual population. For both populations, the time-varying population vector is shown during the tracking movements in four different directions (indicated by arrows on the left). For each condition, the population vector is calculated every 13.3 ms and plotted as a line originating at the corresponding value on the time axis. These vectors show the temporal variability in the directional signal encoded in the visual and motor population. For horizontal movements, the population vectors tends to point in the direction of the time axis. Therefore, the gray line is added as a description of the actual length of the population vector. The continuous thin and thick lines give the time course of the position and velocity of the target, respectively. The dotted thin and thick lines represent the averaged hand position and velocity from all trials contributing to the population vector. Before averaging, all data were aligned to target onset (left vertical dashed line). Only the position and velocity traces collinear to the movement direction of the target are plotted (i.e. the x-components during horizontal movements and the y-components during vertical movements). Note that the direction of the motor population vector shows only little variability in a single condition and codes for the similar direction in the time before and during the manual tracking. The length of the population shows a similar time course for all four directions of tracking movements. The visual data show a slightly higher variability in the direction of the population vector, and a consistent pattern in the temporal evolution of vector length is missing. The differences in the temporal evolution of the population vector length during the horizontal movements is probably a consequence of the fact that most receptive fields were located in the right hemifield.

Figure 4.

Location of receptive fields during MT/MST recordings from the second monkey. The coverage of horizontal eccentricity by the receptive fields of all cells recorded from the second monkey is depicted as bar histograms. Note that each receptive field may contribute to a continuum of eccentricities according to its horizontal width. Data from all recorded cells of the second monkey with a clear receptive field (minimal response field) are combined (n = 270). From the first monkey, receptive field location were plotted only qualitatively and data were not included in this analysis. The thin line shows the mirrored distribution (x-axis mirrored), the thick line describes the sum from the measured and the mirrored distribution. In this sum, the coverage of receptive field for the visual space covered by the movement is more homogeneous than for the non-mirrored, original data.

Figure 4.

Location of receptive fields during MT/MST recordings from the second monkey. The coverage of horizontal eccentricity by the receptive fields of all cells recorded from the second monkey is depicted as bar histograms. Note that each receptive field may contribute to a continuum of eccentricities according to its horizontal width. Data from all recorded cells of the second monkey with a clear receptive field (minimal response field) are combined (n = 270). From the first monkey, receptive field location were plotted only qualitatively and data were not included in this analysis. The thin line shows the mirrored distribution (x-axis mirrored), the thick line describes the sum from the measured and the mirrored distribution. In this sum, the coverage of receptive field for the visual space covered by the movement is more homogeneous than for the non-mirrored, original data.

Figure 5.

Combination of population data from both horizontal movements. The x-components during the leftward movement were mirrored at the vertical axes. The y-components of the population vector were averaged without any preceding transformation. The influence of a biased activation (e.g. due to an heterogeneous distribution of receptive fields of visual cells) is cancelled out by this averaging. The position and velocity data of the hand and target were transformed and averaged in the same way as the population vector components. The averaged horizontal vectors represent a generalized vector that is no longer related to a single direction. To avoid the horizontal vector pointing in the direction of the time axis, all vectors were rotated by 90° counterclockwise.

Figure 5.

Combination of population data from both horizontal movements. The x-components during the leftward movement were mirrored at the vertical axes. The y-components of the population vector were averaged without any preceding transformation. The influence of a biased activation (e.g. due to an heterogeneous distribution of receptive fields of visual cells) is cancelled out by this averaging. The position and velocity data of the hand and target were transformed and averaged in the same way as the population vector components. The averaged horizontal vectors represent a generalized vector that is no longer related to a single direction. To avoid the horizontal vector pointing in the direction of the time axis, all vectors were rotated by 90° counterclockwise.

Figure 6.

Grand average from movements in all four directions: combination of population data from horizontal and vertical movements. For this average, the results from the mirrored averages are combined: the x-component of data measured during the horizontal movements (as shown in Fig. 5) and the y-component of data measured during the vertical movements. These combinations yield the overall temporal modulation of the population activity and the averaged profiles of hand position and hand velocity.

Grand average from movements in all four directions: combination of population data from horizontal and vertical movements. For this average, the results from the mirrored averages are combined: the x-component of data measured during the horizontal movements (as shown in Fig. 5) and the y-component of data measured during the vertical movements. These combinations yield the overall temporal modulation of the population activity and the averaged profiles of hand position and hand velocity.

Figure 7.

Results from multiple linear regression calculated for different combinations of time shifts between the population response and hand and target data. The coefficient of determination R2 (z-axis) is plotted as a function of the temporal shifts (x-axis: shift of target data, y-axis: shift of hand data). (A) Results from regression of motor population data. (B) Results from regression of visual population data.

Figure 7.

Results from multiple linear regression calculated for different combinations of time shifts between the population response and hand and target data. The coefficient of determination R2 (z-axis) is plotted as a function of the temporal shifts (x-axis: shift of target data, y-axis: shift of hand data). (A) Results from regression of motor population data. (B) Results from regression of visual population data.

Figure 8.

Peak height of CCHs expressed in Z-scores for those CCHs containing a significant peak. Significant peaks were obtained only when CCHs were calculated between cells from a single structure (MT/MST, left half, or M1, right half of figure). For CCHs calculated between both areas, not a single CCH with a significant peak was observed.

Figure 8.

Peak height of CCHs expressed in Z-scores for those CCHs containing a significant peak. Significant peaks were obtained only when CCHs were calculated between cells from a single structure (MT/MST, left half, or M1, right half of figure). For CCHs calculated between both areas, not a single CCH with a significant peak was observed.

We thank Dr C. Distler for performing surgery on the monkeys and for the histological processing. This work was supported the Deutsche Forschungsgemeinschaft, SFB 509 ‘Neurovision’ and by a Helmholtz stipend to W. Kruse from the Bundestministerium für Bildung, Wissen-schaft, Forschung und Technologie (BMBF).

## References

Ashe J, Georgopoulos AP (
1994
) Movement parameters and neural activity in motor cortex and area 5.
Cereb Cortex

6
:
590
–600.
Battaglia-Mayer A, Ferraina S, Marconi B, Bullis JB, Lacquaniti F, Burnod Y, Baraduc P, Caminiti R (
1998
) Early motor influences on visuomotor transformations for reaching. A positive image of optic ataxia.
Exp Brain Res

123
:
172
–189.
Battaglia-Mayer A, Ferraina S, Mitsuda T, Marconi B, Genovesio A, Onorati P, Lacquaniti F, Caminiti R (
2000
) Early coding of reaching in the parietooccipital cortex.
J Neurophysiol

83
:
2374
–2391.
Boussaoud D, Ungerleider LG, Desimone R (
1990
) Pathways for motion analysis: cortical connections of the medial superior temporal and fundus of the superior temporal visual areas in the macaque.
J Comp Neurol

296
:
462
–495.
Burnod Y, Baraduc P, Battaglia-Mayer A, Guigon E, Koechlin E, Ferraina S, Lacquaniti F, Caminiti R (
1999
) Parieto-frontal coding of reaching: an integrated framework.
Exp Brain Res

129
:
325
–346.
Cardoso de Oliveira S, Thiele A, Hoffmann KP (
1997
) Synchronization of neuronal activity during stimulus expectation in a direction discrimination task.
J Neurosci

17
:
9248
–9260.
Carpenter AF, Georgopoulos AP, Pellizzer G (
1999
) Motor cortical encoding of serial order in a context-recall task.
Science

283
:
1752
–1757.
Colby CL, Gattas R, Olson CR, Gross G (
1988
) Topographical organization of cortical afferents to extrastriate visual area PO in the macaque: a dual tracer study.
J Comp Neurol

269
:
392
–413.
Dubner R, Zeki SM (
1971
) Response properties and receptive fields of cells in an anatomically defined region of the superior temporal sulcus in the monkey.
Brain Res

35
:
528
–532.
Eckhorn R, Bauer R, Jordan W, Brosch M, Kruse W, Munk M, Reitboeck HJ (
1988
) Coherent oscillations: a mechanism of feature linking in the visual cortex? Multiple electrode and correlation analyses in the cat.
Biol Cybern

60
:
121
–130.
Engel AK, Konig P, Kreiter AK, Singer W (
1991
) Interhemispheric synchronization of oscillatory neuronal responses in cat visual cortex.
Science

252
:
1177
–1179.
Fu Q-G, Suarez JI, Ebner TJ (
1993
) Neuronal specification of direction and distance during reaching movements in the superior precentral premotor area and primary motor cortex of monkeys.
J Neurophysiol

70
:
2097
–2116.
Gergopoulos AP, Kalaska JF, Caminiti R, Massey JT (
1982
) On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex.
J Neurosci

2
:
1527
–1537.
Georgopoulos AP, Schwartz A, Kettner RE (
1986
) Neuronal population coding of movement direction.
Science

233
:
1416
–1419.
Georgopoulos AP, Kettner RE, Schwartz AB (
1988
) Primate motor cortex and free arm movements to visual targets in three-dimensional space. II. Coding of the direction of movement by a neuronal population.
J Neurosci

8
:
2928
–2937.
Georgopoulos AP, Ashe J, Smyrnis N, Taira M (
1992
) Motor cortex and the coding of force.
Science

256
:
1692
–1695.
Goodale MA (
1998
) Visuomotor control: where does vision end and action begin?
Curr Biol

8
:
R489
–R491.
Hoffmann KP, Distler C (
1989
) Quantitative analysis of visual receptive fields of neurons in nucleus of the optic tract and dorsal terminal nucleus of the accessory tract in the macaque monkey.
J Neurophysiol

62
:
416
–428.
Jancke D, Erlhagen W, Dinse HR, Akhavan AC, Giese M, Steinhage A, Schoner G (
1999
) Parametric population representation of retinal location: neuronal interaction dynamics in cat primary visual cortex.
J Neurosci

19
:
9016
–9028.
Johnson MTV, Coltz JD, Hagen MC, Ebner TJ (
1999
) Visuomotor processing as reflected in the directional discharge of premotor and primary motor cortex neurons.
J Neurophysiol

81
:
875
–894.
Judge SJ, Richmond BJ, Chu FC (
1980
) Implantation of magnetic search coils for measurement of eye position: an improved method.
Vision Res

20
:
535
–538.
Lacquaniti F, Caminiti R (
1998
) Visuo-motor transformations for arm reaching.
Eur J Neurosci

10
:
195
–203.
Lee D, Port NL, Kruse W, Georgopoulos AP (
2001
) Neuronal clusters in the primate motor cortex during interception of moving targets.
J Cogn Neurosci

13
:
319
–331.
Lurito JT, Georgakopoulos T, Georgopoulos AP (
1991
) Cognitive spatial-motor process 7. The making of movements at an angle from a stimulus direction: studies of motor cortical activity at the single cell and population levels.
Exp Brain Res

87
:
562
–580.
Mardia KV (1972) Statistics of directional data. New York: Academic Press.
Maunsell JHR, van Essen DC (
1983
) The connections of the middle temporal visual area (MT) and their relationship to a cortical hierarchy in the macaque monkey.
J Neurosci

3
:
2563
–2586.
Maunsell JHR, van Essen DC (
1983
) Functional properties of neurons in middle temporal visual area of the macaque monkey. II Binocular interactions and sensitivity to binocular disparity.
J Neurophysiol

41
:
1148
–1167.
Maunsell JHR, Newsome WT (
1987
) Visual processing in monkey extrastriate cortex.
Annu Rev Neurosci

10
:
363
–401.
Port NL, Kruse W, Lee D, Georgopoulos AP (
2001
) Motor cortical activity during interception of moving targets.
J Cogn Neurosci

13
:
306
–318.
Roelfsema PR, Engel AK, Konig P, Singer W (
1997
) Visuomotor integration is associated with zero time-lag synchronization among cortical areas.
Nature

385
:
157
–161.
Schoppmann A, Hoffmann KP (
1976
) Continuous mapping of directional selectivity in the cat's visual cortex.
Neurosci Lett

2
:
177
–181.
Schmolesky MT, Youngchang W, Hanes DP, Thompson KG, Leutgeb S, Schall JD, Leventhal AG (
1998
) Signal timing across the macaque visual system.
J Neurophysiol

79
:
3272
–3278.
Schwartz AB (
1993
) Motor cortical activity during drawing movements: population response during sinusoid tracing.
J Neurophysiol

70
:
28
–36.
Schwartz AB (
1994
) Direct cortical representations of drawing.
Science

265
:
540
–542.
Singer W (
1999
) Neuronal synchrony: a versatile code for the definition of relations?
Neuron

24
:
49
–65.
Tanaka K, Hikosaka K, Saito H-A, Yukie M, Fukada Y, Iwai E (
1986
) Analysis of local and wide-field movements in the superior temporal visual areas of the macaque monkey.
J Neurosci

6
:
134
–144.
Ungerleider LG, Mishkin M (1982) Two cortical visual systems. In: The analysis of visual behavior (Ingle DJ, Goodale MA, Mansfield RJ, eds), pp. 549–586. Cambridge, MA: MIT Press.
Van Essen DC, Maunsell JHR, Bixby JL (
1981
) The middle temporal visual area in the macaque: myeloarchitecture, connections, functional properties and topographic organization.
J Comp Neurol

199
:
293
–326.
Van Essen DC, Anderson CH, Fellemann DJ (
1992
) Information-processing in the primate visual system: an integrated systems perspective.
Science

255
:
419
–423.
von der Malsburg C (
1999
) The what and why of binding: the modeler's perspective.
Neuron

24
:
95
–104.
Wise SP (
1993
) Monkey motor cortex: movements, muscles, motoneurons and metrics.
Trends Neurosci

16
:
46
–49.
Wise SP, Boussaoud D, Johnson PB, Caminiti R (
1997
) Premotor and parietal cortex: corticocortical connectivity and combinatorial computations.
Annu Rev Neurosci

20
:
25
–42.
Zeki SM (
1974
) Functional organization of a visual area in the posterior bank of the superior temporal sulcus of the rhesus monkey.
J Physiol

236
:
549
–573.