Given an instruction regarding which effector to move and what location to move to, simply adding the effector and spatial signals together will not lead to movement selection. For this, a nonlinearity is required. Thresholds, for example, can be used to select a particular response and reject others. Here we consider another useful nonlinearity, a supralinear multiplicative interaction. To help select a motor plan, spatial and effector signals could multiply and thereby amplify each other. Such an amplification could constitute one step within a distributed network involved in response selection, effectively boosting one response while suppressing others. We therefore asked whether effector and spatial signals sum supralinearly for planning eye versus arm movements from the parietal reach region (PRR), the lateral intraparietal area (LIP), the frontal eye field (FEF), and a portion of area 5 (A5) lying just anterior to PRR. Unlike LIP neurons, PRR, FEF, and, to a lesser extent, A5 neurons show a supralinear interaction. Our results suggest that selecting visually guided eye versus arm movements is likely to be mediated by PRR and FEF but not LIP.
A central question in neuroscience is where and how particular computations are performed. Frequently a specific node (e.g., a neuron, circuit, nucleus, area, or network) is identified as a candidate for a particular operation. Evidence for or against the involvement of that node can be obtained by tracing the origin and destination of tracts related to the node, by recording which signals are present, and by intervening in processing at that node either negatively, for example, with lesions or reversible inactivations, or positively, for example, with microstimulation or optogenetic stimulation. The strongest case can be made when multiple lines of inquiry converge to the same result. Here we focus on how to make the best use of recording studies to gain insight into neurophysiological mechanisms.
Many studies conclude that a particular computation may occur within a particular node if that node encodes all of the necessary inputs as well as the output of the proposed computation. In these cases, signals may be segregated across different elements in the node (e.g., different neurons within a single area, or different voxels within a particular region of interest), or they might be combined in various ways within the same nodes. A yet stronger argument can be made for having identified the site in which a particular computation occurs when the output signal(s) can be detected only after the appearance of the input signals. We now propose, and provide a case study for, yet another criterion for judging what computation is being performed at a given node.
Perhaps the simplest way to combine signals is to add them together. In a system in which different neurons receive different combinations of inputs, linear summation preserves information but cannot perform an “all-or-none” computation, such as deciding which of several possible options to pursue. This decision requires some form of nonlinearity. For example, to choose whether to execute a reach or saccade to the left or right, signals related to a “reach” instruction might be added to a signal representing a target on the left, and this sum might be compared with “reach” plus “target right”, “saccade” plus “target left”, and “saccade” plus “target right.” A nonlinear readout mechanism might then select the largest of the 4 signals or ask if any of the 4 sums exceeds a threshold (Hanes et al. 1998; Gold and Shadlen 2000; Roitman and Shadlen 2002; Huk and Shadlen 2005; Hanks et al. 2006; Ferrera et al. 2009; Ray et al. 2009; Pouget et al. 2011; Ding and Gold 2012). Alternatively, a motor plan could be selected using an “and” operation, in which a particular output is activated if and only if a particular combination of inputs occurs, for example, reach to the left if and only if representations for a target on the left and a reach instruction are active. A multiplicative or supralinear operation can be helpful as a preliminary stage, even if it does not by itself produce a Boolean (all or none) output. A final all or none output could be achieved by subsequent additional nonlinearities such as competitive inhibition (Cisek and Kalaska 2005) or thresholding (Gold and Shadlen 2007).
Here we focus on spatial and effector inputs for reaches and saccades. Effector and spatial information are independent from one another. We can decide to move an arm or the eyes even prior to knowing a precise spatial goal for the movement (Stanford et al. 2010). Many cortical regions involved in planning movements encode information related to the effector to be used (e.g., one or the other arm or the eyes) as well as information about the spatial location of the target (Snyder et al. 1997; Hoshi and Tanji 2000; Kermadi et al. 2000; Calton et al. 2002; Astafiev et al. 2003; Cisek et al. 2003; Connolly et al. 2003; Dickinson et al. 2003; Cisek and Kalaska 2005; Medendorp et al. 2005; Beurze et al. 2007; Cui and Andersen 2007; Levy et al. 2007; Chang et al. 2008; Pesaran et al. 2010). Neurons in the parietal reach region (PRR) and in the lateral intraparietal area (LIP) are spatially tuned and show differential responses when instructed to prepare a saccade or reach (Snyder et al. 1997; Colby and Goldberg 1999; Cui and Andersen 2007; Pesaran et al. 2010), even in the absence of spatial information about the target (Calton et al. 2002; Dickinson et al. 2003). Neurons in these areas, as well as in the dorsal portion of area 5 (A5) (Cui and Andersen 2011; Bremner and Andersen 2014), show differential responses to the sum of spatial and effector information. Effector- and spatial-specific deficits are seen after reversible lesions of these areas (Liu et al. 2010; Battaglia-Mayer et al. 2013; Hwang et al. 2012; Yttri et al. 2013, 2014). Neurons in the frontal eye fields (FEF) show all 3 properties (responses to pure spatial and to pure effector information; differential responses to combined spatial and effector information; effector- and spatial-specific deficits from lesions) and are known to play a causal role in directing saccades but not reaches (Bruce and Goldberg 1985; Schall 1991; Hanes et al. 1998; Lawrence and Snyder 2006, 2009; Wardak et al. 2006; Crapse and Sommer 2009; Ray et al. 2009).
In the present study, animals performed a task in which information about where to move was separated in time from information about what body part to move (eyes or arm) (Hoshi and Tanji 2000, 2002; Calton et al. 2002; Dickinson et al. 2003; Lawrence and Snyder 2006; Bernier et al. 2012). We investigated the response to these 2 separate pieces of information compared with the response to the combined information, and in particular, whether these responses combined linearly or nonlinearly. We recorded from 4 dorsal stream areas in the parietal and frontal cortices: PRR, area 5 (A5; a small portion of area 5 just anterior to PRR, likely corresponding to the anterior portion of MIP), LIP, and FEF. Our results demonstrate that nonlinearities occur in PRR and A5 when planning reaches and in FEF, but not in LIP, when planning saccades. Nonlinearities in FEF and A5 depend on the order in which effector and spatial information is delivered, whereas the nonlinearities in PRR are independent of stimulus order. Our results suggest that PRR, FEF, and perhaps A5 take part in movement selection.
Materials and Methods
Two male rhesus macaques (Macaca mulatta) were trained to make either eye or arm movements to visible targets while we collected single-unit spikes from PRR, A5, LIP, and FEF. All procedures conformed to the Guide for the Care and Use of Laboratory Animals and were approved by the Washington University Institutional Animal Care and Use Committee. Animals were seated in a custom-designed monkey chair (Crist Instruments, Hagerstown, MD, USA) with a fully open front that allowed for unconstrained arm movements to visual stimuli. Stimuli were back-projected by a CRT projector (Electrohome, Kitchener, Ontario, Canada) onto a touch panel (see below) located 25 cm in front of the animal. Unlike an LCD projector, a CRT projector casts no extraneous light, so that other than the visual stimuli, experiments took place in complete darkness in a sound-attenuated room.
Eye position was monitored by a scleral search coil (CNC Engineering, Seattle, WA, USA), and arm position was monitored by a 43.2 cm touch panel (Keytec, Richardson, TX, USA) with custom electronics to minimize electrical interference and produce a temporal resolution of 8 ms. Different trial types were performed, but all involved movements from the center of the touch panel out to a peripheral location. All trials began with the eyes and the arm fixating and touching the central fixation target, respectively (Fig. 1A,B). On reach trials, the animals made dissociated center-out arm movements while maintaining the fixation at the center. On saccade trials, the animals made dissociated center-out eye movements while continuing to touch the center target.
The directional tuning of each cell was first mapped using a receptive field (RF) mapping task. These trials began with the eye and arm at a blue central target (0.9° × 0.9°). The animals had to fixate within 2.5° (radius) and touch within 6° of the center of this target. The central target then was extinguished and one of the 8 peripheral targets (0.9° × 0.9°) appeared at 20° eccentricity. The animal responded with a center-out coordinated eye and arm movement. The target associated with the greatest evoked activity during the 100–200 ms interval following target onset was chosen online as the “preferred direction”, and the opposite direction constituted the “null direction.”
Once the RF was mapped, the animals performed interleaved LE trials (Fig. 1A) and EL trials (Fig. 1B). Both trial types began when the animals looked within 2.5°of a blue central target and touched within 6° of it. Next, on LE trials, a blue peripheral spatial target appeared, and then, after a variable delay of 500–800 ms, a red or green foveal cue appeared. On EL trials, the order of appearance was reversed: the foveal effector cue appeared first, followed after a delay by the peripheral target. Both stimuli remained present until the end of the trial. Immediately after the appearance of the second stimulus, one animal was required to look at the spatial target if the foveal effector cue was red and to reach for the spatial target if the cue was green. The second animal learned the reverse mapping. All stimuli were 1.5° × 1.5°. The blue peripheral spatial target was 20° from the fovea, in either the preferred or null direction of the cell being recorded from. Correct trials were rewarded with a drop of juice.
On both LE and EL trials, the animals had 500 ms (saccade trials) or 900 ms (reach trials) to move the cued effector into a 6° (saccade trials) or 8° (reach trials) radius window about the peripheral spatial target, while leaving the uncued effector at the center target. Upon an error (moving prematurely, moving the wrong effector, acquiring the target too late or inaccurately), the trial was aborted and a new randomly selected trial began. Error trials were excluded from the analyses. Each type of trial (LE and EL), direction of movement (preferred and null), and effector type (saccade and reach) was performed 10 times in random order, for a combined total of 80 trials per neuron.
Figure 1D shows the reciprobit distribution of eye and arm movement RTs on LE and EL trials (Carpenter and Williams 1995). The reciprobit analysis is based on the empirical finding that saccadic RTs show a positive skew, and that the distribution is much closer to a Gaussian if the reciprocal of 1/RT is plotted instead. This follows from a linear rise to threshold model, in which the slopes of the rise have a normal distribution. Parameters of interest are the rate at which information is obtained (which sets the slope of the rise to threshold), how far away the threshold lies from the starting point (which together with slope affects rise time), and the standard deviation of the slopes. The cumulative distribution of reciprocal RTs approaches a straight line when plotted on a probit scale. Changes in the mean rate of rise to threshold cause the straight line to shift from right to left or left to right. Changes in how far away the threshold lies from the starting point cause the straight line to rotate clockwise or counterclockwise about the point (x = 1/infinity, y = 100%).
Recording Procedures and Anatomical Localization
All recordings from all 4 cortical areas were made from 2 adult male rhesus macaques (monkeys G and I). Recording chambers were placed contralateral to the hand used in the studies. Extracellular recordings were made using tungsten electrodes (FHC or Alpha Omega). Recording chambers were centered at 5–8 mm posterior and 12 mm lateral (Horsley-Clarke coordinates) for the posterior parietal cortex recording and 25 mm anterior and 20 mm lateral for the frontal cortex recording. These chambers were placed flush to the skull. While we searched for cells, animals performed nondelayed, center-out combined eye and arm movements to 20° peripheral targets in each of 8 directions around the clock, spaced 45° apart. Sometimes, the animals also performed either memory saccades plus reaches or interleaved LE and EL tasks. Cells that modulated firing rate at any point during the search were tested in the RF mapping task. In most cases, the preferred direction of a cell was clear from this sequence, and to save time we collected data only in the preferred and null directions in the main LE and EL tasks. Sometimes adjacent target locations would both give strong responses. In such cases, we collected both directions (and their opposite directions) and later picked the direction with stronger responses based on the 200–1000 ms interval after target onset. In just under 10% of cases, we tested 8 equally spaced directions and then fit a cosine function to select the best direction, again based on the 200–1000 ms interval.
We used structural magnetic resonance imaging (MRI) to localize and confirm our recording sites (Calton et al. 2002; Chang et al. 2008). Briefly, each animal was anesthetized and a surface coil was positioned around the chamber. A 3 T Siemens MRI scanner running an MPRAGE sequence was used to obtain a high resolution (0.5 × 0.5 × 0.5 mm) structural scan. To localize our recording sites, we placed a custom-designed cylinder containing MR contrast agent (gadoversetamide) into the recording chamber. Vertical bars inside the cylinder at known locations displaced the contrast agent, allowing us to reconstruct the position and orientation of the recording chamber and grid relative to the cortex. In several scanning sessions, we injected 1–2 μL of 0.1 mol/L manganese in monkey G at known coordinates. By visualizing this injection in the scan, we determined the accuracy of our initial localization to be within 1–2 mm of our intended target.
PRR was localized to a region that lies primarily on the medial bank of the IPS and contains many neurons with clear responses to visual target onsets and greater sustained activity during a memory-guided reach task than during a memory-guided saccade task (Snyder et al. 1997; Calton et al. 2002). PRR overlaps portions of the anatomically defined medial intraparietal area (MIP) (Colby and Duhamel 1991; Colby and Goldberg 1999), parietal-occipital area (Lewis and Van Essen 2000b), and area V6a (Galletti et al. 1999; Fattori et al. 2005). Figure 3 of the study by Calton and colleagues (2002) shows the anatomical reconstruction of the PRR neurons recorded and used in this study. The reconstructed recording locations of PRR neurons straddle the border between V6a and the MIP (Lewis and Van Essen 2000a, 2000b). This region partially overlaps with V6a/PO (Galletti et al. 1999). The study by Chang and colleagues (2009) in their Figure 1 shows the 3-dimensional reconstructions of typical PRR cells within MIP and V6a/PO boundaries.
LIP was identified as a region on the lateral wall of the IPS containing many cells with crisp responses to visual onsets and greater sustained activity during a memory-guided saccade trial than during a memory-guided reach trial (Dickinson et al. 2003).
Area 5 (A5) recordings were obtained from a small portion of area 5 located on the medial aspect of the IPS and within 4 mm of the anterior boundary of PRR (Calton et al. 2002). Many of these cells fall in the anterior portion of area MIP. We first determined the boundaries of PRR (Calton et al. 2002) (see above) and then classified cells that fell anterior or superficial to PRR as A5 cells (Cui and Andersen 2011; Bremner and Andersen 2014) (Fig. 2B). The boundaries of PRR were based on quantitative rather than qualitative criteria: the frequency of cells with clear visual and memory activity and the robustness of those effects. To ensure consistency, boundaries were constrained to be planar. A5 cells within 1 mm of the PRR border were excluded.
FEF, in the anterior bank of the arcuate sulcus, was defined as any region within 200 μm of a site at which electrical microstimulation of <50 μA evoked a consistent saccadic eye movement (Lawrence and Snyder 2006, 2009). To make this determination, animals fixated a target. After 400 ms, the target disappeared, and 100 ms later, a 70-ms interval of either stimulation (biphasic, 250 μs/phase, 350 Hz, 70-ms duration) or no stimulation was delivered with equal probability. The animal was rewarded on every stimulation trial and also on every no-stimulation trial in which the eyes remained within 4.5° of the extinguished fixation point. Only neurons collected from stimulation sites resulting in perturbations > 2° were used in the current analysis. Significant perturbations (2-sample t-test, P < 0.05) ranged from 2.2° to 28° (mean ± SEM of 8.2 ± 0.3°).
Our principle aim was to compare the neuronal response to combined spatial and effector information to the sum of the responses when only spatial information or only effector information was known. The response when only spatial information is known was obtained from the last 300 ms prior to effector cue onset in LE trials. The “pre-effector spatial specificity” was then determined as the difference in activity between preferred and null targets for the preferred effector. Similarly, the response when only effector information is known was obtained from the last 300 ms prior to spatial target onset in EL trials (Calton et al. 2002; Dickinson et al. 2003; Lawrence and Snyder 2006). The “pre-location effector specificity” then was determined as the difference in activity between the preferred and the null effector during this epoch.
The response to the combined information (i.e., either “post-effector spatial specificity” or “post-location effector specificity”) was obtained from intervals determined by the eye or arm movement RTs. This was important since animals were free to initiate a movement once both pieces of information were available and we wished to capture activity related only to planning or motor selection-related processes, and to minimize movement-related activity. We first determined the time at which only 10% of movements (reaches and saccades) were initiated (202 ms and 144 ms for saccades on LE and EL trials, respectively, and 310 and 232 ms for reaches, respectively). We then measured activity in the 50-ms interval immediately preceding this time (Fig. 1C, orange rectangles). LIP and FEF respond to saccades much more strongly than to reaches, so in those areas we analyzed activity in the interval from 152 to 202 ms following effector cue onset of LE trials (i.e., post-effector spatial specificity) and 94–144 ms following target onset on EL trials (i.e., post-location effector specificity). PRR and A5 respond to reaches much more strongly than to saccades, so in those areas we analyzed the activity in the interval from 260 to 310 ms following effector cue onset of LE trials (i.e., post-effector spatial specificity) and 182–232 ms following target onset on EL (i.e., post-location effector specificity).
To test for nonlinearity, we examined the summation of effector and spatial signals in 2 independent ways. First, we determined how effector instructions gate spatial specificity on LE trials, by comparing post-effector spatial specificity against pre-effector spatial specificity. We compared these 2 activity levels to test whether spatial specificity is enhanced (indicating a supralinear interaction), suppressed (sublinear), or remains unaltered (linear) once effector information is provided. Second, we examined the summation of effector and spatial signals in the reverse order—that is, how the spatial instruction gates effector specificity. For this, we compared post-location effector specificity against pre-location effector specificity on EL trials. We compared these 2 activity levels to test whether effector specificity is enhanced (supralinear interaction), suppressed (sublinear), or remains unaltered (linear) once spatial information is provided.
Reaction Time Profiles for Integrating Reach and Saccade Information
We presented animals with spatial targets that instructed where to move but not how (location), and effector cues that instructed how to move (reach or saccade) but not where (effector), separated by a delay period. This allowed us to temporally dissociate the 2 pieces of information and to measure responses to each. On Location-Effector (LE) trials (Fig. 1A), a target appeared in the periphery, followed by an effector cue at the fovea. Effector-Location (EL) trials were similar, but the order of the effector cue and spatial target was reversed (Fig. 1B). In both trial types, animals could move once they had both effector and spatial information, that is, once both the effector cue and target location had been delivered.
Reaction times (RTs) were faster on EL compared with that on LE trials (Fig. 1C) (Table 1). Median saccade RTs were 56 and 42 ms faster in EL compared with that in LE trials (P < 0.0001 in each animal, Wilcoxon rank sum test). Variability was also significantly reduced in EL compared with that in LE saccade trials (both P < 0.0001, F test). For reaches, RTs were 72 ms faster in EL compared with that in LE trials in each animal (both P < 0.0001, Wilcoxon rank sum test). Variability was also significantly reduced in EL compared with that in LE reach trials (both P < 0.0001, F test). Saccades were faster and less variable than reaches, and both saccades and reaches were faster and less variable on EL compared with that on LE trials. This is consistent with the fact that RTs to exogenous information (e.g., the spatial target in EL trials) are generally faster than RTs to endogenous information (the central effector cue in LE trials) since less processing is required in the former case (Jonides 1981).
|EL saccade (ms)||EL reach (ms)||LE saccade (ms)||LE reach (ms)|
|Monkey G||170 ± 30||263 ± 40||226 ± 31||335 ± 46|
|Monkey I||190 ± 47||332 ± 70||232 ± 50||404 ± 77|
|EL saccade (ms)||EL reach (ms)||LE saccade (ms)||LE reach (ms)|
|Monkey G||170 ± 30||263 ± 40||226 ± 31||335 ± 46|
|Monkey I||190 ± 47||332 ± 70||232 ± 50||404 ± 77|
The distribution of reciprocal RTs on a probit scale (see Materials and Methods) is consistent with a decision-like process of selecting a motor plan (Fig. 1D). As predicted by the LATER model (Carpenter and Williams 1995), which posits that movements begin when a decision variable crosses a threshold value, the data form straight lines for all but the tails of the distributions. (The sole exception involves reach RTs in EL trials from monkey I. This violation could arise because the data are combined across movements in different directions.) This supports a mechanism with a rate-limiting step that resembles a rise to bound (threshold) model but provides no indication about where in the nervous system such a thresholding operation might occur.
Adding Effector Information to Spatial Information
Neuronal activity was recorded from PRR (138 cells) (Calton et al. 2002), A5 (70 cells) (Calton et al. 2002), LIP (65 cells) (Dickinson et al. 2003), and FEF (101 cells) (Lawrence and Snyder 2006) in the same 2 monkeys (G and I). A5 cells recorded in the current study were just anterior to PRR and showed somewhat similar characteristics as PRR neurons (see below and the inset in Fig. 2B).
Spatial and effector information may add together linearly or they may combine nonlinearly, with one signal modulating or gating the response to the other. We do not expect baseline activity to sum. To assay responses independent of baseline, we subtract firing rate obtained on complementary trials, that is, preferred minus null direction trials or saccade minus reach trials. If addition is linear, then presenting the second piece of information will not change the firing difference produced by the first piece of information. For example, we asked whether the spatial specificity on LE trials, that is, the difference in activity on preferred compared to null trials, is altered when the effector cue appears. An alteration indicates a nonlinearity; no alteration indicates linear addition.
Figure 2 shows peristimulus time histograms of population activity on LE trials (spatial target presented before effector cue) in each cortical region (see references Calton et al. 2002; Dickinson et al. 2003; Lawrence and Snyder 2006 for single cell examples from each area). The orange traces show the degree of spatial specificity, that is, the difference in responses to preferred minus null target directions. If effector and spatial information sum linearly, the orange difference traces will not change in response to the delivery of the effector cue. A supralinear interaction would be manifest as an upward deviation of the orange trace, while a sublinear interaction would be manifest as a downward deviation.
The PRR population shows supralinear additivity on LE trials. Providing a spatial target results in a large change in firing rate, most of which is sustained during a delay period (Fig. 2A, left panel, orange trace). Later in the trial, when a reach instruction is given, the activity associated with spatial information increases (Fig. 2A, middle panel, orange trace). The difference in activity—post-effector minus pre-effector spatial specificity during 260–310 ms following effector cue onset—is significant (9.92 ± 2.54 sp/s, P < 0.0005, paired t-test; see Materials and Methods for selection of the time interval). When considering firing rates of individual cells, 92 out of 138 cells (67%) had higher firing post-effector than pre-effector, that is, they fall above the unity line (Fig. 3A) (significantly greater than expected by chance, P < 0.0005, binomial test). When only the 67 cells with significant spatial tuning in the last 300 ms before effector cue onset are considered (filled points), 42 (63%) fall above the unity line (P = 0.05, binomial test). Finally, of the 63 cells that showed a significant change in tuning after the effector cue appears (bootstrap test, P < 0.05), 44 (69%) fall above the line (P < 0.005, binomial test), confirming that the information about which effector to move combines supralinearly with spatial information in PRR.
A5 cells also are supralinear on LE trials, though to a lesser extent then the PRR cells (larger spatial selectivity after effector information, 260–310 ms following effector cue onset: 4.53 ± 1.96 sp/s, P = 0.02, paired t-test). The population firing rates increase with the delivery of the spatial information (Fig. 2B) and further increase when a reach instruction is delivered. Of 70 individual cells, 46 cells (66%) showed higher firing post- compared with pre-effector instruction, that is, the majority of the cells fall above the unity line (Fig. 3B) (P = 0.01, binomial test). This includes 18 of the 31 cells (58%) with significant spatial tuning (filled points) (P = 0.47, binomial test). Finally, of the 26 cells that showed a significant change in spatial specificity with effector instruction (bootstrap test, P < 0.05), the direction of that change was an increase in 18 (69%) (P = 0.08, binomial test). Therefore, the information about which effector to move combines supralinearly with spatial information in A5, although the magnitude of the effect, at both the population and individual cell levels, is substantially lower than in PRR.
In contrast to PRR and A5 cells, LIP cells linearly combine effector and spatial information on LE trials (152–202 ms following effector cue onset; see Materials and Methods) (mean difference, −2.0 ± 1.0 sp/s, P = 0.05, paired t-test; Fig. 2C). Later in time, effector specificity appears to increase (e.g., 200 ms after effector onset). The increase anticipates movement onset, and so it is unlikely to reflect proprioceptive feedback. The late increase in firing could drive the movement or, alternatively, reflect an efference copy of the movement, perhaps serving as a forward predictor or state estimator. At the individual cell level, out of all 65 LIP cells, only 22 (34%) show an increase in firing after the effector instruction is presented (Fig. 3C) (P = 0.01, binomial test). This includes 12 of the 23 cells (52%) with significant spatial tuning (filled points) (P = 1, binomial test). Finally, of the 11 cells that showed a significant change in specificity with effector instruction (bootstrap test, P < 0.05), the direction of that change was an increase in 4 (36%) (P = 0.55, binomial test). These results indicate that the information about which effector to move combines linearly or weakly sublinearly with spatial information in LIP.
Finally, the FEF population, like that of PRR, shows strong supralinear additivity on LE trials (152–202 ms following effector cue onset) (mean difference, 18.19 ± 3.99 sp/s, P < 0.0001; paired t-test; Fig. 2D). Of 101 cells, 71 cells (70%) have higher firing post- compared with pre-effector instruction, that is, about two-thirds of cells fall above the unity line (Fig. 3D) (P < 0.0001, binomial test). This includes 31 of the 39 cells (79%) with significant effector specificity in at least one of the 2 intervals (filled points) (P < 0.0005, binomial test). Finally, of the 68 cells that showed a significant change in specificity with effector instruction (bootstrap test, P < 0.05), the direction of that change was an increase in 53 (78%) (P < 0.0001, binomial test). These results indicate that the information about which effector to move combines supralinearly with spatial information in FEF.
Adding Spatial Information to Effector Information
Next, we examined whether and how effector specificity (differences in activity evoked by a preferred vs. a nonpreferred effector instruction) is altered by the presence or absence of spatial information. Figure 4 is analogous to Figure 2, but shows data from EL (effector cue presented before spatial target) rather than LE trials. The orange traces show the degree of effector specificity, that is, the difference in responses to preferred minus null effector instructions. The critical test is again whether the orange trace in the middle panels remains flat after the second piece of information (target location) is delivered.
In the absence of spatial information, 3 of the 4 cortical regions show a significant bias to one effector compared with the other (Fig. 4). PRR neurons increase their activity when cued to prepare a reach compared with a saccade, and the bias is maintained to the end of the delay period (last 300 ms prior to target onset, 3.99 ± 0.73 sp/s, P < 0.0001, Wilcoxon sign rank test) (Calton et al. 2002). LIP and FEF neurons show the reverse effect, increasing their activity when cued to prepare a saccade compared with a reach. Specificity appears earlier in LIP than in FEF, but it is significant in both areas at the end of the delay period (both LIP: 2.08 ± 0.60 sp/s, P < 0.001, Wilcoxon sign rank test; FEF: 1.56 ± 0.51 sp/s, P < 0.001) (Lawrence and Snyder 2006). A5 neurons show a weaker and nonsignificant increase in activity following an instruction to prepare for a reach compared with an instruction to prepare for a saccade (1.06 ± 0.52 sp/s; P = 0.09, Wilcoxon sign rank test).
We are interested in whether the increase in activity after target presentation is greater for one effector compared with the other, indicating a nonlinear combination of effector and spatial signals. For the reach-preferring regions (PRR and A5), the orange traces show the reach response minus the saccade response. For the saccade-preferring areas (LIP and FEF), the orange traces show the saccade response minus the reach response. If effector and spatial information sum linearly, the orange difference trace will be flat following target onset (to the right of the solid vertical line). A supralinear interaction would cause the orange trace to deviate upward, while a sublinear interaction would cause the orange trace to deviate downward.
In PRR, effector and spatial signals combine supralinearly on EL trials. Effector specificity is significantly greater after compared with before the spatial information was supplied (182–232 ms following target onset; see Materials and Methods) (mean difference, 10.54 ± 2.40 sp/s, P < 0.0001; paired t-test; Figure 4A, middle panel; compare the relative change in the green vs. red trace, or the upward deviation in the orange trace). Of all 138 PRR cells, 64% show a higher relative difference in firing after compared with before delivery of the spatial information, that is, half of the cells fall above the unity line in Figure 5A, indicating larger mean values for postspatial effector specificity than prespatial effector specificity. This includes 56 of the 83 cells (68%) with significant effector specificity in at least one of the 2 intervals (filled points). Finally, of the 71 cells that showed a significant change in effector specificity as a result of target appearance (P < 0.05, bootstrap test), the direction of that change is an increase in 52 (73%). All 3 ratios are significantly greater than what would be expected by chance (all P < 0.005, exact binomial test), confirming once more that information about the spatial location of the target combines supralinearly with effector information in PRR.
In contrast, A5 cells linearly combined effector and spatial information—the mean effector specificity neither increased nor decreased as a result of target appearance (182–232 ms following target onset) (mean difference, 3.64 ± 2.07 sp/s, P = 0.08, paired t-test; Fig. 4B). Of 70 A5 cells, 59% fall above the unity line (Fig. 5B). This includes 22 of the 34 cells (65%) with significant effector specificity in at least one of the 2 intervals (filled points). Finally, of the 30 cells that showed a significant change in specificity with target appearance (bootstrap test, P < 0.05), the direction of that change is an increase in 18 (60%). None of these 3 ratios is significantly different from chance (all P > 0.12, exact binomial test), indicating that when spatial information follows effector information, the combination is linear in A5.
The population of LIP cells also combines effector and spatial information linearly. The mean effector specificity decreased by a nonsignificant amount as a result of target appearance (94–144 ms following target onset; see Materials and Methods) (mean difference, –1.65 ± 1.42 sp/s, P = 0.25, paired t-test; Fig. 4C). Of 65 LIP cells, 43% fall above the unity line (Fig. 5C). This includes 8 of the 22 cells (36%) with significant effector specificity in at least one of the 2 intervals (filled points). Finally, of the 10 cells that showed a significant change in specificity with target appearance, the direction of that change is an increase in 2 (20%). None of these ratios are significantly different from chance (all P > 0.10, exact binomial test), indicating once again that spatial and effector information add linearly in LIP.
Finally, FEF shows linear additivity in EL trials. The mean specificity remains unchanged as a result of target appearance (94–144 ms following target onset) (mean difference, 1.41 ± 1.26 sp/s, P = 0.27, paired t-test; Fig. 4D). Effector specificity does rise in a later interval following target onset (3rd column in Fig. 4D), but this effect occurs so late that it is more likely to be an efference copy than a causal signal. Of 101 FEF cells, 54% fall above the unity line (Fig. 5D). This includes 12 of the 23 cells (52%) with significant effector specificity in at least one of the 2 intervals. Finally, of the 17 cells that showed a significant change in specificity with target appearance, the direction of that change was an increase in 7 (41%). None of these ratios were significantly different from chance (all P > 0.55, exact binomial test). Thus, the way in which spatial and effector information combine in FEF depends on the order of the instructions. When effector information is provided first, the information adds linearly (this paragraph and Figs 4D and 5D). However, when spatial information is provided first, the information combines supralinearly (Figs 2D and 3D).
Figure 6 summarizes our findings. When spatial signals are provided first (LE trials, panel A), effector and spatial signals in PRR and FEF combine in a strongly supralinear manner. A5 shows moderate supralinearity, and LIP shows linear responses. When effector signals are provided first (EL trials, panel B), effector and spatial signals combine supralinearly in PRR and linearly in FEF, LIP, and A5.
We asked whether information about location (where to move) and effector (what type of movement to make) combine linearly or nonlinearly within particular cortical regions. To determine this, we compare responses to location information alone, effector information alone, and the combination of location and effector information, all within the same neuron. Previous studies from our own and other labs have reported data from 2 of these 3 components. For example, Cui and Andersen (2007, 2011) have looked at responses to location information alone and the combination of effector and location information. To conclude whether the information combines linearly or nonlinearly, it is essential to obtain data from all 3 combinations. Testing for linear versus nonlinear summation of inputs can help identify the locus at which particular motor plans are selected. Given multiple potential targets for movement, we select at most one target and move to it, rather than moving towards the average location of all targets of interest. With strictly linear processing, no information is lost and therefore an all-or-none choice cannot be made. Instead, a nonlinearity is required. Finding where nonlinearities occur helps to isolate the neurons and circuits that perform particular selection processes.
Note that we are not arguing that early processing is linear and late processing is nonlinear. Nonlinearities occur even at early stages of processing. For example, both the retina and V1 contain nonlinear processing stages (Marr 2010). As another example, Cartesian reference frame transformations, which are believed to occur in the parietal cortex, are linear at the algorithmic level but may require nonlinear operations when the spatial information is encoded using radial basis functions rather than a Cartesian rate code (Pouget and Snyder 2000). Conversely, linear computations may be required to implement a selected plan. Thus, linear and nonlinear operations may occur at any point in the processing chain. However, particular nonlinearities can serve as markers for selection processes.
Based on this principle, we suggest that PRR and FEF are involved in selecting a motor plan for a reach or a saccade to a particular spatial location, while LIP is not involved. This final point that LIP is not involved in selecting between plans to reach or saccade is further supported by the fact that LIP responses at the time of movement hardly distinguish saccades from reaches (Fig. 4C, rightmost column). Thus, we have 2 seemingly contradictory sets of findings. On the one hand, LIP encodes saccade intention (Snyder et al. 1997; Andersen and Cui 2009). Effector specificity prior to receipt of location information is greater in LIP than in FEF (Fig. 6B and Dickinson et al. 2003). On the other hand, LIP combines spatial and effector information linearly and hardly distinguishes between saccades and reaches when the movements occur. This indicates that LIP does not determine whether a saccade or reach will be performed.
These results can be reconciled in at least 2 ways. LIP might act as a salience map, encoding or even directing the spatial allocation of attention in an effector-independent manner (Goldberg et al. 2006). In this case, the finding of effector specificity during the planning stage means that a plan to move the eyes to a particular location must recruit or require a greater allocation of attention than a plan to reach to that location. An alternative reconciliation of the 2 sets of findings is that LIP activity is saccade specific and necessary but not sufficient for saccade generation. In this case, the linear combination of information and the equal responses at the time of movement are explained by LIP responding in an obligatory manner to exogenous visual transients, acting to provide the substrate for a potential saccade regardless of prior instructions. We favor this second interpretation. Note that with either interpretation, the presence of effector specificity or intention-related activity does not provide evidence of a computation to select one particular effector over another; for that, a nonlinear interaction is required. Furthermore, our results imply that a structure or structures downstream of LIP (closer to the motor output) must act as a final gate on the saccade command. Both the current results as well as elegant experiments using the stop signal delay paradigm argue strongly that the final gate is in FEF (Hanes et al. 1998; Pouget et al. 2011).
The finding that LIP is not involved in the decision to reach or saccade does not argue against a role of LIP in other types of decisions, including perceptual decision-making (Huk and Shadlen 2005; Gold and Shadlen 2007). Accumulating sensory evidence over time to arrive at a correct perceptual judgment is fundamentally a sensory computation, which is different from the selection of one effector over another. Perceptual decisions regarding the stochastic distribution of events (Shadlen and Newsome 2001) or the worth of those events (Louie et al. 2011) do not require integrating spatial and effector information. Instead, these perceptual decisions focus solely on sensory information and require nonlinear operations within a different domain. These operations may be accomplished using a drift-diffusion operation combined with a (nonlinear) threshold (Gold and Shadlen 2007; Ratcliff and McKoon 2007), or a mechanism using (multiplicative) competitive inhibition (Cisek and Kalaska 2005).
FEF is an interesting case. Nonlinear interactions occur when a spatial signal appears and is then followed by an effector instruction (LE trials); when the effector instruction comes first, the transformation is linear (EL trials). Another way to view this is that, in FEF neurons, an effector cue evokes a substantially larger response when preceded by a spatial target inside compared with outside of the receptive field, but a spatial target evokes similar responses regardless of whether it is preceded by a saccade or reach instruction. This is consistent with previous studies showing that FEF has a role not only in initiating saccades (Hanes et al. 1998), but also in directing attention (Thompson et al. 1997; Moore and Fallah 2004). The onset of a spatial target may capture attention independent of effector system. If so, then the FEF response to a target may be independent of, and therefore add linearly to, a previous effector cue. In contrast, when an effector cue appears after a spatial cue, there is no additional allocation of attention. Instead, the processing would reflect only movement selection, leading to a nonlinear response.
The neurons recorded in A5 show a dependence on order that is similar to that seen in FEF: they are supralinear on LE trials but linear on EL trials. These A5 cells were recorded just anterior to PRR. PRR is defined functionally as a region near the posterior end of the medial bank of the intraparietal sulcus (IPS) containing a high proportion of cells with strong visual responses and sustained memory activity for reaching. The borders are not crisply defined, but it appears to span parts of anatomical areas V6a and MIP. Our A5 recordings are likely in anterior MIP and may represent a transition between PRR, which plays a role in effector selection, and more anterior regions, which do not. In support of this idea, note that, although the overall pattern of responses in the 2 populations was similar (compare Figs 2A,B and 4A,B), spatial tuning was substantially stronger in PRR compared with A5 before (10.78 ± 1.21 vs. 6.67 ± 1.14 sp/s, P = 0.01, 2-sample t-test) as well as after (20.68 ± 2.91 vs. 11.20 ± 2.11 sp/s, P < 0.01) effector information was provided (Fig. 6A). Similarly, effector specificity was substantially stronger in PRR compared with A5 both before (3.99 ± 0.73 vs. 0.97 ± 0.5 sp/s, P < 0.001) and after (14.53 ± 2.40 sp/s vs. 4.62 ± 2.10 sp/s, P < 0.005) spatial information was provided (Fig. 6B). If our anterior MIP (A5) recordings represent a transition away from PRR and toward less effector-specific areas, then the dependence on order seen in A5 may actually reflect a power issue. In PRR, supralinearity is less robust on EL compared with LE trials, and if effect sizes are reduced more or less equally in anterior MIP, then this alone may explain the apparent dependence on order in anterior MIP.
Neurons in the dorsal premotor cortex (PMd), like those in PRR, combine spatial and reach effector information in a manner that is consistent with a role in deciding to move either the left or the right arm. Some PMd neurons show strong nonlinear summation patterns for moving either the right or the left arm to a target located in either the left or right hemifield (i.e., activity tuned to a specific interaction between effectors and targets) (Hoshi and Tanji 2000). The activity of many PMd neurons is also dependent on the order of effector and spatial instruction, similar to the current finding in FEF (Hoshi and Tanji 2000). PRR projects directly to ipsilateral PMd (Tanné et al. 1995; Johnson et al. 1996; Wise et al. 1997). Thus, the selection of a reach plan appears to be computed across a distributed network that includes at least PRR and PMd.
Finally, we consider several limitations of our study. Through training, it is possible that the loci of neural computations may shift from one node to another in the network. Because our animals perform their tasks extensively prior to data collection, the neural circuits across areas are likely to be at steady state at the time of recording. Future studies could address whether summation patterns change as an animal learns a new task or during the subsequent extensive practice period. Another potential caveat is that on EL reach trials, animals may covertly plan eye movements. For this reason, it is difficult to rule out that the summation patterns during EL reach trials observed in the present study are truly independent of any possible saccade planning. However, we previously showed that delay activity during EL trials in PRR predicts reach but not saccade RTs (Snyder et al. 2006), mitigating the concern that covert saccade planning might contribute to summation patterns in PRR during EL reach trials.
In summary, we show that PRR, FEF, and to a lesser extent A5 are all specifically involved in integrating spatial sensory information with information regarding what type of movement should be executed. This is an important stage in the selection of a motor plan. In contrast, the linear operations found in LIP suggest that this area is not itself involved in critical aspects of motor selection, although it may provide critical inputs to areas that are involved in these processes. We suggest that, within the distributed network of regions in the cerebral cortex involved in effector-specific motor planning, only a subset of regions are directly involved in selecting the motor plan.
The work presented here was supported by the National Eye Institute (R01-EY012135 to L.H.S.) and the National Institute of Mental Health (R00-MH099093 to S.W.C.C.).
We thank Bijan Pesaran and Benjamin Hayden for helpful comments on the manuscript. Conflict of Interest: None declared.