The brain computes spatial relationships as necessary to achieve behavioral goals. Loss of this spatial cognitive ability after damage to posterior parietal cortex may contribute to constructional apraxia, a syndrome in which a patient's ability to reproduce spatial relationships between the parts of an object is disrupted. To explore neural correlates of object-relative spatial representation, we recorded neural activity in parietal area 7a of monkeys performing an object construction task. We found that neurons were activated as a function of the spatial relationship between a task-critical coordinate and a reference object. Individual neurons exhibited an object-relative spatial preference, such that different neural populations were activated when the spatial coordinate was located to the left or right of the reference object. In each case, the representation was robust to translation of the reference object, and neurons maintained their object-relative preference when the position of the object varied relative to the angle of gaze and viewer-centered frames of reference. This provides evidence that the activity of a subpopulation of parietal neurons active in the construction task represented relative position as referenced to an object and not absolute position with respect to the viewer.
Humans assemble and utilize complex objects such as tools or buildings, a fact often considered to provide a physical demonstration of human intelligence. Most objects that humans build and use are assemblies of discrete components. Constructed objects are therefore the products of a set of spatial cognitive operations on the part of an architect, engineer, or builder that has computed and specified how the parts of an object ought to be fit together. Some of the most basic spatial computations involved can be isolated in a simple behavioral context defined by the requirement to assemble a copy of a model object. To do this successfully, the spatial relationships between the components of the model must be analyzed and reproduced. The process is captured by the actions of a child assembling building blocks to conform to a configuration provided by printed instructions. Spatial information embedded in the instructions is periodically sampled and analyzed to direct each action during assembly. This cycle of spatial analysis followed by physical assembly is nicely reflected by the pattern of eye and arm movements as humans assemble copies of model objects. Subjects direct gaze first to the model object and then to the corresponding location within the copy object immediately before placing new parts at the proper relative position in the copy (Ballard et al. 1992; Hayhoe et al. 1998). This suggests that the brain strategically samples spatial information embedded in the model object before directing each movement during construction of the copy and further that neural correlates of the spatial analysis involved and the spatial motor control required may be dissociable in time during this process.
Spatial cognitive processes that enable object construction appear to rely particularly on the functional integrity of posterior parietal cortex. Damage to this cortical area often produces constructional apraxia, a syndrome in which patients are unable to accurately reproduce the spatial structure of model objects (Kleist 1934; Black and Strub 1976; Villa et al. 1986; Ruessmann et al. 1988). Given the above, a simple prediction is that neurons in posterior parietal cortex represent spatial relationships among object components. Loss of these neurons after parietal damage could then provide part of the explanation why parietal patients construct objects that are spatially disorganized. To examine the possibility that parietal neurons support relational representations of space that localize object components with respect to each other, we developed an object construction task that monkeys could perform and recorded neural activity in posterior parietal area 7a.
We previously reported that, during this task, the firing rate of area 7a neurons varied systematically with a task-critical spatial datum, namely, where component parts were missing from incomplete copies relative to model objects stored in working memory (Chafee et al. 2005). This neural signal suggested that parietal neurons were involved in a comparative operation by which differences between copies and models were assessed in order to direct subsequent construction. Support for this was provided by the observation that the neural signal predicted where monkeys were going to place the next component in the copy object on correct and error trials. Importantly, this neural signal did not correlate with sensorimotor aspects of the task because neural activity did not correspond to the position of a visual stimulus or the direction of the required motor response. We interpreted this neural activity as a physiological correlate of a spatial cognitive operation involved in directing object construction.
In the present experiment, we test the prediction that during object construction, single parietal neurons support an object-referenced representation of space. The object construction task required determining whether object components were located on the left or right side of the object to which they belonged. We sought to determine whether parietal neurons coded this spatial relationship consistently when the position of the reference object changed relative to the angle of gaze and the position of the viewer. The objective was to determine if neural activity was selective for an intraobject spatial relationship and if this selectivity could be dissociated from retinocentric or viewer-centered representations of space that have been previously described in parietal cortex. We present supporting evidence below that parietal neurons encode not only spatial locations or directions but also spatial relationships, when these are important to a given behavioral objective.
Materials and Methods
Neural activity was recorded bilaterally from area 7a in the posterior parietal cortex of 2 male rhesus macaques (4 and 6 kg). Two recording chambers (7 mm internal diameter) were implanted over area 7a bilaterally in each animal in an aseptic surgery under isoflurane (1–2%) gas anesthesia. Four titanium posts were fixed to the skull with screws, and a halo was attached to provide a mechanical anchor to stabilize head position. Postsurgical analgesia was administered for several days (Buprenex, 0.05 mg/kg bid, intramuscularly). Further information regarding recording technique and reconstruction of electrode penetrations in area 7a can be found in our prior report (Chafee et al. 2005). Care and treatment of the animals conformed to the Principles of Laboratory Animal Care of the National Institutes of Health (NIH) (NIH publication no. 86–23, revised in 1995). The Internal Animal Care and Use Committees of the University of Minnesota and the Minneapolis Veterans Affairs Medical Center approved all experimental protocols.
The object stimuli used in the construction task consisted of varying arrangements of identical square elements and were adapted from stimuli developed by Driver and Halligan (1991). Squares were blue and subtended 1.4° of visual angle; adjacent squares were separated by a gap of 0.25°. Object squares were laid out in a 5 × 5 square grid, 8° on a side. All objects included a frame consisting of squares in the base row and central column of the grid forming an inverted “T” configuration. The frame provided a base and central axis to all objects. Distinct model configurations consisted of this frame plus 1 or 2 additional squares. Monkeys were trained to perform the task using a set of 36 distinct model configurations differing in the position of the squares present in addition to those making up the object frame. (Additional squares were located either on the left or on the right side of the object, at one of 4 different vertical levels within the 5 × 5 grid. For a depiction of the full set of object configurations employed in training, see Fig. 1 of Chafee et al. 2005.) Neural recording was conducted with a subset of these configurations. This subset consisted of objects in which additional object squares were present in either the topmost or the middle row of the object grid, on either the left or the right side of the object midline. The construction task was administered in 2 experimental series (Fig. 1), differing in whether the position of the model object (Series A) or the copy object (Series B) varied relative to the fixation target across trials (as described further below). In Series A (Fig. 1A–G), model objects consisted of the object frame plus 1 or 2 additional squares present on either the left or the right side of the object, in either the top or the middle row in the grid. In Series B (Fig. 1H–M), model objects consisted of the object frame plus one additional square, present on either the left or the right side of the object, in either the top or the middle row of the grid. We restricted the total number of different objects used during recording in order to restrict the number of trials required to complete each trial set. This was advantageous as the time that cortical neurons can be stably isolated is limited. The activity of each set of isolated neurons was recorded as monkeys performed either 128 trials (Series A) or 160 trials (Series B) of the object construction task.
Both monkeys performed Series A; the second monkey performed Series B of the construction task, as well as a probe task, in blocks. Each trial began with the appearance of a red dot serving as a gaze fixation target presented at the center of the visual display. Monkeys were required to maintain gaze within 1.3° to 1.7° of this central target throughout the trial up to the delivery of reward. Moving gaze fixation beyond this window terminated the trial. After a preliminary period of fixation (500 ms), a model object was presented for 750 ms (Fig. 1A,H; Model). In Series A, the model object was presented offset 4.2° horizontally either to the left or to the right of the fixation target (just more than half the object width) so that the object fell entirely within the left or right visual hemifields (Fig. 1A). The direction of the shift, left or right, was randomized across trials. In Series B, the model object was always presented centered on the fixation target at the center of the display (Fig. 1H). After the model disappeared, a 750-ms delay period followed in which only the gaze fixation target was visible (Fig. 1B,I; Delay). After the delay period, a copy object was presented (Fig. 1C,J; Copy). In Series A, the copy object was always centered on the fixation target (Fig. 1C). In Series B, the copy object was offset 4.2° horizontally to the left or right of the fixation target, at random across trials (Fig. 1J). In both series, the copy object was identical to the model object preceding it on each trial except that one square had been removed (squares within the central column or base row comprising the object frame were never removed). In considering the model object, we refer to the square removed to produce the copy as the “critical square.” In considering the copy object, we refer to the location where a square was absent relative to the preceding model as the “missing critical square.” After the copy object had been visible for 750 ms, a pair of choice squares appeared flanking the copy object (Fig. 1D,K; Choice array). Choice squares were arranged either in a horizontal array, with one square on either side of the copy object (Fig. 1D), or in a vertical array, with both squares on the same side of the copy object at different vertical positions (Fig. 1K). The configuration of the choice array, whether horizontal or vertical, varied randomly across trials (in both Series A and B). The monkey replaced the missing critical square by selecting one of the 2 choice squares for addition to the copy object. The selection was controlled by varying the timing of the motor response and not its direction. After the choice squares had been visible for 300–600 ms, first one choice square (Fig. 1E,L; first choice) and then the other choice square (Fig. 1F; second choice) increased in brightness for a period of 700–1000 ms in random sequence (represented by the open choice square in Fig. 1E,F,L). Only one of the 2 choice squares was brightly illuminated at any one time. The monkeys depressed a single response key with their left foot to indicate their selection. The choice square that was bright at the time that the response key was pressed translated smoothly inward in a horizontal direction to occupy the grid position in the same row and immediately adjacent to the copy object. (In the case that the monkey depressed the response key during the first choice period, the second choice was not shown.) Addition of the selected choice square to the copy object produced a new configuration. If the correct choice square was selected, it replaced the missing critical square and reproduced the configuration of the preceding model (Fig. 1G; Completion). If the incorrect choice square was selected, the configuration of the constructed object differed from the preceding model (Fig. 1M; Completion; note the constructed object configuration in Fig. 1M is different from the model object in Fig. 1H). After the selected choice was added to the copy object, the newly constructed configuration remained visible for a period of 300 ms (Fig. 1G,M). If the choice was correct, the monkey was then rewarded with a drop (0.2 mL) of juice. The sequential choice response mechanism was used to report the monkeys' decision in order to dissociate locations in objects from targets for movement. The spatial direction of the movement vector the monkey had to execute to depress the response key was invariant across trials. Neural signals associated with spatial variables in the task therefore were not likely to reflect spatial aspects of this required motor response.
In a probe task (Fig. 11), we flashed a visual probe stimulus on a minority (25%) of Series B construction trials. The monkey could not anticipate which trials were probe trials, and the reaction time to detect the probe stimulus measured behavioral correlates of spatial representation during object construction. The probe stimulus consisted of a red square the size of the other squares in the copy object. It was flashed for 50 ms during the copy period, 150–750 ms after the appearance of the copy object. The visual probe was located either at the same position in the copy object as the missing critical square or at the mirror location on the opposite side of the copy object. On probe trials, the monkey was required to depress the response key between 50 and 600 ms after probe onset for reward. On the remaining 75% of trials randomly interleaved with the attention probe trials, the monkey performed the construction task as described above.
Neural activity was recorded using a 16-channel multielectrode recording matrix (Thomas Recording, GMbH, Giessen, Germany). The electrode signals were amplified (at a gain of 20 000), filtered (bandpass between 0.5 and 5 kHz), and action potentials were discriminated in each channel using a time-amplitude window discriminator (DDIS-1, Bak Electronics, Mount Airy, MD) or waveform discriminator (Multi Spike Detector, Alpha Omega Engineering, Nazareth, Israel). Two people performed and monitored the unit isolation during recording. Typically, the action potentials of between 20 and 30 neurons were isolated simultaneously. If isolation of a neuron's activity was not maintained throughout the set of administered trials, the data from that neuron were discarded. Spike timing was sampled with 40 μs resolution (DAP 5200a Data Acquisition Processor, Microstar Laboratories, Bellevue, WA). Computer files containing the timing of action potentials relative to stimulus and behavioral events were saved for all neurons isolated. Neurons in which the mean discharge rate averaged across the model, delay, and copy periods of the task exceeded 0.5 Hz were included in subsequent analyses. In both animals, ∼15% of isolated neurons failed to meet this spontaneous activity criterion; the remaining 85% of all cells isolated comprise the present database. Electrode penetrations were confined to area 7a in the inferior parietal gyrus; recording locations were determined by magnetic resonance imaging visualization of electrodes and confirmed by postmortem localization (Chafee et al. 2005).
The relation between neural firing rate and the spatial position of object squares was assessed by a 2-way analysis of covariance (ANCOVA). The ANCOVA was implemented as a least-squares regression estimating the parameters of a general linear model (GLM) in which the categorical factors were represented by dummy variables. The 2 factors in the ANCOVA defined the relative position of the critical square in 2 alternative spatial frameworks. The first factor was object-referenced relative position. This factor had 2 levels that specified whether the critical square was located on the left or right side of the reference object relative to its midline. The second factor was viewer-referenced position. This factor had 2 levels specifying whether the critical square was located to the left or right of the gaze fixation target. Object-referenced and viewer-referenced positions of the critical square, coded as categorical spatial variables (left or right in each framework), were statistically independent in the design. In the analysis of the Series A data, the dependent variable was firing rate during the model period, and the 2 spatial factors represented the object and viewer-referenced position of the critical square in the model object. In the analysis of the Series B data, the dependent variable was firing rate during the copy period, and the 2 spatial factors represented the object and viewer-referenced position of the critical square missing from the copy object.
In Series A, we focused our analysis on neural activity during the model period because we varied the position of the model object in Series A. In Series B, we focused our analysis on neural activity during the copy period because we varied the position of the copy object in Series B. By examining activity during the period in which we varied the position of the corresponding reference object, we could determine whether neural activity varying as a function of the position of squares in (or missing from) the reference object coded position relative to the object or the viewer.
Two continuous covariates were included in the model, the baseline firing rate (during the 500 ms interval of central gaze fixation preceding model onset) and experimental time (elapsed time since the start of neural recording). These covariates corrected for trial-to-trial fluctuations in baseline activity and linear trends in firing rate across the recording session. We estimated the regression coefficients for the parameters of the linear model using the REGRESS function in the Matlab Statistical Toolbox (The Mathworks Inc., Natick, MA), and the significance of object and viewer-referenced factors was assessed at P < 0.05.
Critical squares located on the left side of the reference object occupied one pair of retinocentric positions (Fig. 2A; open squares labeled L). Critical squares located on the right side of the reference object occupied another, slightly offset pair of retinocentric positions (Fig. 2A; filled squares labeled R). Figure 2A illustrates the retinocentric receptive fields (dotted outlines varying in size from approximately 4° to 16°) of 3 hypothetical neurons overlapping 1 (neuron B), 2 (neuron A), or 3 (neuron C) critical square locations. The firing rates of neurons A–C (Fig. 2A) would be expected to vary as a function of square position in approximately the manner illustrated by the bar graphs of corresponding color below (the 4 bars indicate relative levels of activity evoked by critical squares at the 4 retinocentric positions above). In each case, mean firing rate is generally greater when critical squares are located in the left (neurons A, B) or right (neuron C) visual hemifields. Such a pattern of spatial preference would be expected to yield significance for the viewer-referenced factor in the ANCOVA.
A different type of spatial selectivity is illustrated in Figure 2B. In this case, elevated activity is associated with critical squares located within one of 2 spatially separated retinocentric regions corresponding to the preferred side of the reference object when the object appears at 2 different locations in the display. Hypothetical neurons preferring object-left (Fig. 2B; neuron A) and object-right (Fig. 2B; neuron B) are shown. Neurons with this form of spatial selectivity during the construction task would be expected to be significantly affected by the object-referenced factor in the ANCOVA.
Neurons significantly affected by both viewer- and object-referenced position or their interaction could exhibit several types of spatial selectivity. One such type is illustrated by neuron B in Figure 2A. In neurons such as this, selective for a single retinocentric location, firing rate could be significantly higher when trials were segregated according to both object-referenced and viewer-referenced position. (Neuron B exhibits elevated activity both when critical squares are located on the right side of the reference object and in the left visual hemifield). Thus, it was particularly important to confirm that single parietal neurons existed in which activity related significantly to object-referenced position only, and not to viewer-referenced position, or to the interaction between these factors.
To examine population activity, we plotted normalized population spatial tuning functions based on the position of the critical square expressed in viewer-referenced coordinates (Figs 5 and 6). We plotted separate tuning functions for neurons with left and right spatial preference defined either with respect to the reference object or with respect to the viewer. To construct the population spatial tuning functions, we first computed the mean firing rate of each neuron during either the model or copy period when the critical square was located in each of either 8 (Series A) or 4 (Series B) different horizontal locations on either side of the fixation target in the display. (Tuning functions were defined with respect to the horizontal position of critical squares.) We then normalized each neuron's spatial tuning function to its peak (by dividing each rate by the maximum observed) and then averaged these single-neuron, normalized spatial tuning functions across all of the neurons in the population. We further constructed population spike density functions (SDFs) to visualize the time course of activity in neurons exhibiting the strongest object-referenced and viewer-referenced signals (neurons were included in each population if their activity related significantly to the corresponding spatial factor in the ANCOVA at P < 0.001). In each population, we constructed separate SDFs using 4 groups of trials. In the object-referenced neuronal population (e.g., Fig. 8A,B), trials were divided according to the combination of 2 binary factors describing the position of the critical square in object-referenced (preferred or nonpreferred) and viewer-referenced (ipsilateral or contralateral) spatial reference frames. In the viewer-referenced neural population (e.g., Fig. 8C,D), trials were similarly divided according to the combination of 2 binary factors describing the position of the critical square in viewer-referenced (preferred or nonpreferred) and object-referenced (left or right) spatial reference frames. Each trial, represented by a vector of spike times, was converted to a continuous SDF by convolution with a Gaussian kernel (σ = 20 ms) using the KSDENSITY function of the Matlab Statistical Toolbox (The Mathworks Inc.). Then for each neuron, the single-trial SDFs in each of the 4 groups of trials above were averaged to produce 4 mean SDFs for that neuron. These single-neuron SDFs were then averaged across all neurons in the population to produce the population activity time courses illustrated.
In the visual probe experiment (Fig. 11), reaction time data were modeled as a linear function of 2 independent variables using multiple linear regression (univariate GLM in the SPSS statistical package; SPSS, Inc., Chicago, IL). The independent variables were the position of the visual probe stimulus relative to the missing critical square, treated as a categorical variable (either the same or mirror opposite location in the copy object), and the retinal eccentricity of the visual probe stimulus, expressed in degrees visual angle.
In Series A, Monkey 1 performed 83% of object construction trials correctly. In the choice sequence, it responded correctly during the first choice period on 79% of trials and during the second choice period on 87% of trials. The average reaction time of Monkey 1 in the first and second choice periods was 386 and 335 ms, respectively (relative to the change in the illumination of each choice that signaled its availability to the monkey). Monkey 2 performed 90% of Series A object construction trials correctly. In the choice sequence, it responded correctly during the first choice period on 88% of trials and during the second choice period on 91% of trials. The average reaction time of Monkey 2 in the first and second choice periods was 324 and 303 ms, respectively. In Series B, Monkey 2 performed 93% of trials correctly. In the choice sequence, it responded to the first and second choices correctly on 92% and 93% of trials. The mean reaction time to the presentation of the first and second choice was 324 and 308 ms.
We recorded the activity of 1517 neurons in 2 monkeys performing the object construction task. (This neural database is partially overlapping with the database described in Chafee et al. 2005. Some neurons tested on various control conditions in that report are excluded from the present study, whereas the Series B data described here were not included in the prior report.) Neural recordings were confined to area 7a in the inferior parietal gyrus. (The regions of area 7a sampled during neural recording in the 2 monkeys are illustrated in Chafee et al. 2005.) In Series A, we recorded the activity of 748 and 265 neurons in Monkeys 1 and 2. In Series B, we recorded the activity of 504 neurons in Monkey 2. The neuronal sample was divided between the left and right cerebral hemispheres of both monkeys. (Series A: 374 neurons in both the left and right hemispheres of Monkey 1; 178 and 87 neurons in the left and right hemispheres of Monkey 2. Series B: 238 and 266 neurons in left and right hemispheres of Monkey 2).
Two Predictions Regarding the Neural Representation of Object-Referenced Spatial Position
Neural activity coding the position of the critical square relative to the reference object during the construction task should satisfy at least 2 conditions. First, neural activity ought to exhibit translation invariance. The activity of neurons preferring a given spatial relationship should be equivalently elevated by any configuration of critical square and reference object that satisfies the preferred relationship, regardless of where the square and the object appear in space relative to the viewer. Second, neural activity coding the spatial relationship between the critical square and the reference object should vary when the critical square remains fixed in space relative to the viewer, but the reference object moves so as to alter its spatial relationship to the critical square. To test the first prediction, we kept the spatial relationship between the critical square and the reference object fixed, but varied the position of the pair relative to the gaze fixation target and therefore the viewer. To test the second prediction, we kept the position of the critical square fixed relative to the viewer, while changing the position of the reference object. Confirming these 2 predictions tests the hypothesis that neural activity during object construction codes the relative position of the critical square with respect to the reference object and not the absolute position of the critical square with respect to the viewer.
Effects of Varying Viewer-Referenced Position while Holding Object-Referenced Position Constant
Figure 3 illustrates the activity of 2 neurons in area 7a demonstrating an object-referenced spatial preference. One neuron exhibited a spatial preference for the right side of the reference object (Fig. 3A–D). The other neuron exhibited a spatial preference for the left side of the reference object (Fig. 3E–H). The data are from Monkey 2 in Series B. The signal coding object-referenced position was clearest in single neurons during the copy period (see below for quantification), although it was present in both task periods.
In the first neuron, activity during the copy period (Fig. 3A–D) was elevated when the missing critical square was located on the right side of the copy object (Fig. 3B,D). Less activity was evident when the missing critical square was located on the left side of the copy object (Fig. 3A,C; the position of the missing critical square is specified by a comparison between model and copy objects illustrated above each panel). The activity of this neuron exhibited translation invariance in so far as firing rate was comparably elevated when the missing critical square was located on the preferred side of the copy object, irrespective of whether the square and object were located to the left (Fig. 3B) or right (Fig. 3D) of the gaze fixation target. Neural activity was not a simple function of the pattern of visual input in that firing rate varied to reflect the changing position of the missing critical square when visual input during the copy period did not vary (as in Fig. 3A,B). In the ANCOVA we employed (Materials and Methods), the discharge rate of this neuron during the copy period related exclusively to the object-referenced position of the missing critical square (Fobject = 56.40, P < 0.001). Activity did not relate to the viewer-referenced position of the missing critical square (Fviewer = 2.67, P = 0.104) or to the interaction between the 2 spatial factors (Finter = 2.13, P = 0.146). In the second neuron (Fig. 3E–H), activity was elevated to a comparable level when the missing square was on the left side of the copy object, regardless of whether the copy object was presented to the left (Fig. 3E) or right (Fig. 3G) of the gaze fixation target. Similarly, the discharge rate of the second neuron during the copy period related exclusively to the object-referenced position of the missing critical square (Fobject = 410.58, P < 0.001). Activity in this case again did not relate either to the viewer-referenced position of the missing critical square (Fviewer = 0.15, P = 0.703) or to the interaction between the 2 factors (Finter = 0.09, P = 0.767). In both neurons, activity varied as a function of a virtual spatial coordinate (the missing critical square) derived from a sequence of stimuli by application of a cognitive rule and appeared to represent that coordinate in an object-referenced framework.
We applied the above 2-way ANCOVA to firing rates during the model period (Series A) and the copy period (Series B) of the trial to examine the influence of critical square position on neural activity. The 2 factors in the ANCOVA were the object-referenced position of the critical square (left or right with respect to the object midline) and the viewer-referenced position of the critical square (left or right with respect to the fixation target). The Venn diagram in Figure 4 illustrates the percentage of neurons (of those exhibiting any spatial effect in the analysis) in which activity related significantly (P < 0.05) either to the object-referenced factor, the viewer-referenced factor, or their interaction (numbers of neurons in parentheses). In one group of neurons, activity varied as a function of the object-referenced factor exclusively. This group accounted for 28% and 49% of neurons with spatially selective activity during the model and copy periods, respectively. The activity of other neurons varied as a function of the viewer-referenced position of the critical square. Neurons coding square position in object-referenced and viewer-referenced coordinates coexisted in posterior parietal cortex.
To examine the spatial tuning of neurons, we constructed normalized population spatial tuning functions (Fig. 5). We defined the position of the critical square as its horizontal distance from the fixation target in degrees of visual angle. Neurons were separated into groups on the basis of whether their activity related to the object-referenced or viewer-referenced factor in the ANCOVA (P < 0.05) and by the spatial preference of the neuron (left or right) in each reference frame. Thus, for example, the object-referenced population included all neurons for which activity related significantly (P < 0.05) to the object-referenced factor (165 neurons in the model period and 223 neurons in the copy period, see Fig. 4). The spatial tuning of these neurons was bimodal—2 peaks in activity were aligned to the 2 retinocentric positions in our design that corresponded to a single, preferred side of the reference object when the object was located to the left or right of the fixation target (Fig. 5A–C; trials with both ambiguous and determinate models as defined below were included; error bars indicate ±1 standard error of the mean). Object-right preferring neurons (Fig. 5A–C; blue lines) were maximally activated when the critical square fell within 2 discrete ranges of retinocentric space (corresponding to the blue-shaded regions) located at different distances from the fixation target. Neurons with object-left preference exhibited a similarly asymmetrical and bimodal spatial tuning function mirror-reflected about the vertical meridian (Fig. 5A–C; red lines). Object-referenced spatial tuning was evident during both the model period (Fig. 5A,B) and the copy period (Fig. 5C) and in both Monkey 1 (Fig. 5A) and Monkey 2 (Fig. 5B,C). We confirmed that the bimodal pattern of spatial tuning at the population level (Fig. 5) was also characteristic of the single neurons in the population, the large majority of which also exhibited bimodal spatial tuning with 2 peaks in activity aligned either with the left (Fig. 6A) or with the right (Fig. 6B) sides of the reference object. Neural activity during the copy period is shown (Fig. 6). A similar pattern of bimodality in spatial tuning was evident during the model period, though the data were noisier, in part because the signal was weaker and in part because more critical square locations were tested in Series 1.
In viewer-referenced neurons, by contrast, neural population tuning functions were monotonic. Activity was greater when the critical square was located either to the left (Fig. 5D–F; red spatial tuning functions) or to the right (Fig. 5D–F; blue spatial tuning functions) of the gaze fixation target. This was the case in Monkey 1 (Fig. 5D) and Monkey 2 (Fig. 5E,F) and during both the model (Fig. 5D,E) and copy periods (Fig. 5F).
Effects of Varying Object-Referenced Position while Holding Viewer-Referenced Position Constant
The second prediction above is that neural activity coding the position of critical squares relative to objects should vary when critical squares remain fixed in viewer-referenced space, but the reference object changes position to alter the spatial relationship between square and object. To test this prediction, we compared the spatial preference of neurons during the model and copy periods of the same trial in Series A. Model and copy objects were offset by one half of the object width so that fixed regions of retinocentric space corresponded first to one and then the other side of the reference object within the same trial. For example, when the model object appeared offset to the left of fixation (Fig. 7A), the region of retinocentric space immediately to the left of the fixation target (shaded red) corresponded to the right side of the reference object. When the copy object appeared centered on the fixation target later within the same trial (Fig. 7B), the same retinocentric region (shaded red) now corresponded to the left side of the reference object.
Neurons activated during the model period were often activated during the copy period as well although typically to a lesser extent (e.g., compare the increase in population activity in model and copy periods in Fig. 8A). This allowed us to compare the spatial preference of neurons across the 2 task periods within the same trial. Figure 7 illustrates population spatial tuning functions computed using neural activity recorded during the model period (blue lines) and copy period (red lines), for neurons with object-left (Fig. 7C) and object-right (Fig. 7D) preference (neurons were included in this analysis if their activity in the model period related significantly to object-referenced position at P < 0.05). Peaks and troughs in the spatial tuning functions indicate retinocentric positions that were preferred and nonpreferred by the neural population during each task period. Population activity evoked by critical squares appearing in fixed retinocentric regions depended on the relative position of the reference object. For example, in neurons with object-left preference (Fig. 7C), critical squares immediately to the right of the fixation target (in region 2) were preferred during the model period but nonpreferred during the copy period. This can be seen by the fact that this retinocentric region contains a relative maximum in the blue tuning function constructed from model period activity and a relative minimum in the red tuning function constructed from copy period activity (Fig. 7C). The opposite pattern is seen for critical squares to the left of fixation (in region 1). Thus, the activity of this neural population represented each fixed retinocentric region as corresponding to opposite sides of the reference object (preferred and nonpreferred) depending on the relative position of the reference object. Neurons with object-right preference exhibited a similarly flexible representation of fixed retinocentric regions as corresponding to opposite sides of the reference object depending on relative location, although the spatial tuning functions were reflected about the fixation target (Fig. 7D). These data are evidence that firing rate did not bear a fixed relationship to retinocentric position, but varied as a function of the relative position of the critical square with respect to the reference object.
Comparing Object-Referenced Signals in the Model and Copy Periods
To compare the strength of neural activity related to the object-referenced position of the critical square in the model and copy periods, we first identified the population of neurons in which activity related to object-referenced position in the ANCOVA at P < 0.05 (165 neurons in the model period, 223 neurons in the copy period). We then compared the distributions of P values associated with the object-referenced factor in these populations of neurons. The average P value associated with the object-referenced factor was 0.017 in the population of neurons active in the model period and 0.009 in the population of neurons active in the copy period, a significant difference (Wilcoxon rank-sum test, P < 10−8) indicative of a more consistent object-referenced signal in the copy period. We also computed an index quantifying the difference in neuronal firing rate attributable to object-referenced position. To compute this index, we determined the mean firing rate of each neuron during the model or copy periods when the critical square was located on the left or right side of the reference object (collapsing across trials in which the reference object appeared to the left or right of the fixation target). We then computed the absolute value of the difference between these rates divided by their sum; |Rleft – Rright|/(Rleft + Rright); where R is the mean firing rate during the model or copy periods and “left” and “right” refer to the object-referenced position of the critical square on each trial. The index is bounded between 0 and 1. The average object-referenced activity index was 0.19 in the population of neurons active in the model period and 0.23 in the population of neurons active in the copy period, a significant difference (Wilcoxon rank-sum test, P = 0.0056) again indicative of a stronger signal in the copy period.
Considering whether Object-Referenced Tuning Arises by a Random Process
We utilized the ANCOVA to identify neurons in which activity varied with object-referenced position and then constructed population tuning (Fig. 5) and activity (Fig. 8) functions using this statistically defined group of cells. One can question whether object-referenced spatial tuning emerged in the analysis as a result of selecting neurons from a population of neurons with random spatial tuning for retinocentric position. In such a population, by chance some neurons would exhibit greater activity when critical squares were on the left side of objects, whereas others would exhibit greater activity when critical squares were on the right side of objects. Dividing the population into right- and left-preferring groups on this basis and averaging their activity would be expected to produce object-referenced spatial tuning, even in the case that the neurons were not, as a population, selective for object-referenced position.
To assess the likelihood of this possibility, we tested 2 further predictions of the alternative hypothesis that spatial tuning arises by chance association between mean firing rate and retinocentric position. If spatial tuning is random and neurons are selective for random subsets of retinocentric positions, then alternative patterns of spatial preference for various retinocentric positions should be equally prevalent. We performed an additional ANCOVA to determine the relative frequencies of 2 specific alternative types of spatial selectivity defined in retinocentric coordinates. In the first type, neural activity was selective for the specific pair of retinocentric positions corresponding either to the left or to the right sides of the reference object. Neurons of this type exhibited bimodal, object-aligned spatial tuning and were significantly influenced by the object-referenced factor in the ANCOVA. In the second type of spatial tuning we assessed, neural activity was elevated when the critical square was located in the pair of retinocentric positions that were either closest to or farthest from the fixation target. Neurons with this type of spatial selectivity exhibited either upright or inverted U-shaped spatial tuning functions. We detected U-shaped spatial tuning by including a second factor in the ANCOVA that coded the retinocentric eccentricity of the critical square as a binary factor (near to or far from the fixation target). We found that bimodal tuning was more common than U-shaped tuning (although both forms were detected). Considering neurons significantly affected by one and not the other of the 2 spatial factors, or the interaction, in the ANCOVA above (at P < 0.05), the ratio of bimodal to U-shaped spatial tuning was 134 to 60 neurons in the model period and 182 to 28 neurons in the copy period. The probabilities of obtaining samples skewed in favor of object-referenced tuning by drawing from a population in which the 2 forms were equally prevalent by chance were P < 10−7 and P < 10−27 during the model and copy periods, respectively (binomial distribution assuming the 2 forms of spatial tuning were equally probable). The results were similar when neurons significantly influenced by both factors or by the interaction were included in the counts of neurons with each type of tuning, in which case the probabilities of obtaining the corresponding samples by chance were P < 10−5 and P < 10−14 during the model and copy periods.
To further confirm that object-referenced spatial tuning was not an artifact of selecting the subset of randomly tuned neurons that happened to conform to an object-referenced hypothesis, we conducted a bootstrap analysis. We computed the above object-referenced activity index using the model period activity of every neuron in the sample (we did not prescreen the data by limiting this analysis to the subset of significantly object-referenced neurons as determined by the ANCOVA). During the model period, object-referenced indices in the population ranged from 0 to 0.75, with a mean value of 0.092. (The mean value of the index was reduced relative to the values above because all neurons were included in the present case, those with and without object-referenced activity, as well as neurons that were not active in the task.) We then shuffled the data by randomly reassigning the 8 mean firing rates observed in each neuron to the 8 possible retinocentric positions of the critical square and recomputed the object-referenced index for each neuron as well as the mean value of the index in the population. If object-referenced tuning occurred by chance, shuffling firing rate with respect to retinocentric position on a cell-by-cell basis should have no net effect on the average strength of object-referenced tuning seen in the population. We shuffled mean firing rate and retinocentric position in each neuron 10 000 times and recomputed the mean population object-referenced index after each iteration. The number of times that the shuffling produced a mean population object-referenced index equal to or greater than the value we observed provided a measure of the likelihood of obtaining the observed value by chance. Only 2 of the 10 000 iterations produced a greater mean object-referenced index in the population than the observed value. This indicates that the probability of obtaining by chance a population of randomly tuned neurons with a degree of object-referenced tuning equal to or greater than what was observed was 2 in 10 000 or P ≈ 0.0002. We conclude object-referenced tuning occurs more frequently and with greater strength than predicted by chance.
Population Activity Time Course Is Consistent with Object-Referenced Spatial Representation
We constructed population SDFs (Materials and Methods) to examine the time course of the average activity of statistically defined populations of object-referenced and viewer-referenced neurons. In the object-referenced population, population activity functions nearly overlap during the model period (Fig. 8A) and copy period (Fig. 8B) when critical squares were located on or missing from the preferred side of the reference object (red and green SDF), regardless of whether the critical square and reference object were located in the visual hemifield that was contralateral (red SDF) or ipsilateral (green SDF) to the recorded neuron. Activity of the viewer-referenced populations (Fig. 8C,D) depended in contrast on whether the critical square (and reference object) was located in the preferred (red and orange SDFs) or the nonpreferred (blue and green SDFs) visual hemifield.
It was possible that neurons included in the object-referenced population had a viewer-referenced preference, but that this preference was not systematic on a neuron-to-neuron basis with respect to whether critical squares and objects were located in the ipsilateral or contralateral visual hemifield. To examine this question, we replotted activity time courses for the object-referenced neural population after determining the viewer-referenced preference (significant or not) of each neuron. In this case, population activity exhibited a joint influence of object-referenced and viewer-referenced position such that activity was stronger when critical squares (and objects) fell on the preferred side of the object (Supplementary Fig. 1A; red and green lines vs. blue and orange lines) and also within the preferred visual hemifield (Supplementary Fig. 1A; red vs. green line). When the population was limited to object-referenced neurons (removing neurons for which the probability associated with the viewer-referenced factor in the ANCOVA was P ≤ 0.1), the difference in population activity as a function of viewer-referenced position was substantially reduced (Supplementary Fig. 1B; red and green lines more nearly overlap). This observation confirms the existence of a relatively pure object-referenced signal in some neurons. However, object-referenced and viewer-referenced factors influenced the activity of some neurons jointly to varying degrees, and the separation between the neural populations coding these factors was not absolute.
In the population significantly influenced by object-referenced position during the copy period (Fig. 8B), the population signal coding the position of the critical square in the copy object emerged prominently during the preceding delay period, before the appearance of the copy object. In a prior report, we demonstrated that the emergence of this population signal ahead of the appearance of the copy object reflects the degree to which the position of the missing critical square can be predicted on the basis of the model object alone (Chafee et al. 2005). Thus, the early emergence of the population signal coding the object-referenced position of the missing critical square in the present experiment (Fig. 8B) might reflect foreknowledge of this relative position afforded by the model object. To test the influence of prediction on neural activity, we contrasted population activity on trials in which the model object did (Fig. 9A; Determinate model trials) and did not (Fig. 9B; Ambiguous model trials) specify the location of the critical square with certainty (model and copy configurations on determinate and ambiguous model trials are shown in Fig. 9C,D, respectively). Determinate models had a single square that could be removed to produce the copy object (Fig. 9C; in models with 2 squares present in the same row and on the same side of the object, only the outermost square could be removed). Ambiguous models had 2 squares at different positions that could be removed to produce the copy, so that monkeys could not know the location of the missing square in advance until the copy object was presented (Fig. 9D). (We restricted this analysis to the 30 neurons in which activity related significantly, P < 0.001, to the object-centered factor during the model period in Fig. 8A.) When the model object determined whether the critical square was going to be on the left or right side of the model and copy objects (regardless of the eventual position of those objects in the display), neural activity coding the object-referenced relative position of the critical square emerged shortly after model onset (Fig. 9A; note the divergence of red and green preferred activity functions considered together from orange and blue nonpreferred activity functions during the model period at the arrow). When the model by itself did not determine whether the critical square was going to be on the left or right side of the copy object (and additional information provided by the configuration of the copy object was required), the population signal specifying the object-referenced position of the critical square did not emerge until after the appearance of the copy object (Fig. 9B; note the delayed divergence of red and green preferred activity functions from orange and blue nonpreferred activity functions at the arrow, after the onset of the copy object). Population activity coded the object-referenced position of the critical square specifically at the time in the trial when enough information had been provided to fix its relative position with respect to the reference object.
Object-Referenced Spatial Preference Persists when Object Shape Does Not Vary
On ambiguous trials, distinct sets of copy object shapes localized the missing critical square to the left and right side of the copy object (compare copy objects on object-left and object-right trials; Fig. 9D). We sought to confirm that this difference in copy object shape was not responsible for the emergence of the population signal reflecting the position of the missing critical square during the copy period on ambiguous model trials (Fig. 9B). Toward that end, we identified 2 sets of trials (Supplementary Fig. 2A; Set 1 and Set 2) selected to equate the shapes of copy objects on trials in which the missing critical square was located on the left and right side of the copy object. (Note that the same copy object shapes were presented when the missing critical square was on the left and the right side of the copy object in each set. Object-left and object-right trials were further equated for the number of repetitions of each copy object shape.) In these trial sets, half of the model objects were determinate, half ambiguous with respect to the side of the critical square. At the time of the onset of the copy object, population activities on preferred trials (red and green activity functions) and nonpreferred trials (orange and blue activity functions) overlapped (Supplementary Fig. 2B). Approximately 200 ms after the appearance of the copy object, population activity again diverged as a function of whether the missing critical square was located on the preferred or nonpreferred side of the copy object (Supplementary Fig. 2B; arrow). This divergence of population activity during the copy period was not due to a difference in the shape of the copy objects because the same copy objects were presented on preferred and nonpreferred trials. Object shape may influence neural activity in parietal cortex during the object construction task. However, it does not appear that object shape alone can account for object-referenced spatial selectivity.
Hemispheric Bias in Object-Referenced Spatial Representation
We found a contralateral bias in the neural representation of object-referenced space. Most object-referenced neurons (significantly related to the object-referenced factor in the ANCOVA at P < 0.05) were most active when the critical square was located on the contralateral side of reference objects (Fig. 10). Of object-referenced neurons active during the model period, 108 neurons exhibited a preference for the contralateral side of the reference object and 57 neurons exhibited a preference for the ipsilateral side of the reference object (P < 0.001, Fisher's exact test on counts in a 2 × 2 tabulation of neurons by cerebral hemisphere and object-referenced preference). This bias for the contralateral side of the reference object was evident when the reference object appeared alternatively in the ipsilateral and contralateral visual hemifields relative to each recorded neuron. Similarly, of object-referenced neurons active during the copy period, 137 neurons exhibited a preference for the contralateral side of the reference object and 86 neurons exhibited a preference for the ipsilateral side of the reference object (P = 0.002, Fisher's exact test). Considering viewer-referenced neurons active during the model period, a contralateral bias was also found with 126 neurons preferring critical squares (and reference objects) presented in contralateral viewer-referenced space and 71 neurons preferring critical squares presented in ipsilateral viewer-referenced space (Fisher's exact test, 2-sided P < 0.001). No such bias was seen for neurons coding the viewer-referenced position of the missing critical square during the copy period (46 neurons with contralateral preference, 53 neurons with ipsilateral preference; Fisher's exact test, 2-sided P = 0.295).
Behavioral Correlates of Object-Referenced Spatial Representation
We measured reaction time on visual probe trials to detect a probe stimulus flashed briefly next to the copy object across trials in which the copy object appeared to the left or right of the fixation target (Fig. 11). On visual probe trials, the monkey viewed a model object followed by the copy object and presumably computed the position of the missing critical square as on normal construction trials (probe trials were unpredictable, occurring on a random 25% of construction trials, and no indication was given that a probe trial was in progress). On a probe trial, 150–750 ms after the copy object appeared, a red square was flashed for 50 ms on one or the other side of the copy object. When the probe stimulus appeared, the monkey was rewarded for immediately pressing the response key, after which the probe trial ended.
On half of probe trials, the probe stimulus appeared in the mirror opposite location as the missing critical square on the opposite side of the copy object (Fig 11A; probe stimuli are represented by red squares, missing critical squares by open squares with an inscribed “X”). On the other half of probe trials, the probe stimulus appeared in the same location as the missing critical square (Fig. 11B). If the neural representation of the missing critical square affected behavioral performance, one would predict that the monkey's detection of the probe would be faster on trials when the probe stimulus appeared in the same location as the missing critical square versus the mirror opposite location in the copy object. We found evidence that this was the case—reaction time was significantly faster when the probe stimulus appeared in the same location as the missing critical square (Figs. 11D, blue symbols), compared with when the probe stimulus appeared in the mirror opposite location in the copy object (Fig. 11D, red symbols; F(1,584) = 7.95, P = 0.005). The eccentricity of the probe also influenced reaction time, with detection of probes farther away from fixation delayed relative to the detection of probes nearer to fixation (Fig. 11D, F(3,584) = 4.28, P = 0.005). We examined the reaction time of the monkey to press the response key after the first choice square brightened on the normal construction trials that were intermingled with visual probe trials, when the first choice in the choice sequence was correct. We found that the reaction time to report the brightening of the correct choice square (Fig. 11D, green symbols) was faster than the reaction time to report the flash of the probe stimulus (Fig. 11D, red and blue symbols). Most trials were construction trials, and the monkey was highly trained in this trial type. Probe trials required a shift in response strategy and were less frequent, possibly accounting for the overall delay in reaction time to detect probe onset relative to brightening of the correct choice square.
There are at least 2 possible interpretations for the additional delay in reaction time when the probe stimulus was in the mirror opposite location relative to the missing square (Fig. 11D, red symbols), in comparison to trials in which the probe and missing critical square shared the same location (Fig. 11D, blue symbols). One interpretation is that covert spatial attention was deployed to the location of the missing critical square, accounting for the quicker reaction time to detect the probe when it was presented at this location. An alternative interpretation invokes a Stroop effect. When visual probes appeared on the same side of the copy object as the correct choice, the monkey could ignore the color of the square; pressing the response key was correct whether the square was red (a visual probe) or blue (a correct choice). When the probe stimulus appeared on the side opposite the missing critical square, color became critical for deciding whether to press the response key (in the case the square was red) or not (in the case the square was blue). Thus, color and position could be considered to conflict. However, the data confirm a behavioral correlate of an object-referenced representation of space in either case (whether spatial attention or Stroop effect). That is because prolonged reaction times were observed when the visual probe stimulus fell within a region of space defined by its spatial relationship to the reference object.
Humans have the ability to selectively compute spatial relationships between objects. Because the number of spatial relationships embedded in even moderately complex scenes is large, it seems likely that we compute only those relationships that are needed to achieve a given behavioral objective. In that light, the computation of spatial relationships between objects provides an instance of spatial cognition in which the brain actively probes the visual input for task-critical spatial information. Spatial analytic processes of this type are likely to be essential for a broad array of human behaviors, one of many possible examples being our ability to construct and utilize tools. Most tools are assemblies of discrete components that have to be assembled into a specific configuration (e.g., satisfy a set of spatial relationships) to be useful. Using most tools similarly requires controlling the spatial relationship between the tool and other objects so that the tool can effectively interact with them. The behavioral effects of posterior parietal cortex damage in humans suggest that this area is essential to the representation of relative spatial information needed to direct these behavioral operations, as the apraxias produced by parietal damage are often characterized by deficits in both object construction (Villa et al. 1986; Ruessmann et al. 1988) and tool use (Leiguarda and Marsden 2000). A simple prediction then is that posterior parietal neurons represent relative spatial information, or that in other words, the intensity of neural discharge in this cortical area varies systematically as a function of the spatial relationship between a point in space and a reference object. To test this prediction, we recorded neural activity in posterior parietal area 7a of monkeys performing an object construction task. This enabled us to examine whether and how spatial relationships between the parts of the object were reflected in the activity of single posterior parietal neurons.
One prior line of posterior parietal research in monkeys has suggested that posterior parietal cortex serves as a sensorimotor interface, providing visual control of movement (Bushnell et al. 1981; Andersen 1995; Bracewell et al. 1996; Snyder et al. 1997b, 2000; Synder, Batista, et al. 1998). Another has indicated that parietal cortex plays a role in visual attention (Mountcastle et al. 1981; Steinmetz et al. 1994; Robinson et al. 1995; Steinmetz and Constantinidis 1995; Gottlieb et al. 1998; Powell and Goldberg 2000; Constantinidis and Steinmetz 2001a, 2001b; Goldberg et al. 2002; Bisley and Goldberg 2003). As has been often noted, sensorimotor control and attention may be 2 sides of one coin as target selection is a necessary prerequisite for target-directed movement (and thus a useful capability for a sensorimotor interface), particularly in environments with many potential targets. In experiments characterizing neural correlates of spatial vision, visual attention, and motor control, parietal neurons have been found to represent the spatial location of visual stimuli in several distinct coordinate systems, including those whose origin is defined in relation to the eye, head, body, or world (Andersen et al. 1985, 1990; Snyder, Grieve, et al. 1998; Andersen and Buneo 2002; Bisley and Goldberg 2003). These various spatial representations were discovered in the activity of posterior parietal neurons that possessed retinocentric visual receptive fields. More specifically, the activity of parietal neurons remained selective for stimuli appearing at a fixed location on the retina, and the intensity of discharge evoked was in some cases modulated by postural factors, such as differences in the position of the eyes and head when the visual stimulus was presented.
Additional work on the functions of parietal cortex in monkeys has recently demonstrated that the cognitive functions of this cortical area extend beyond attention and sensorimotor control. For example, this work has shown that posterior parietal cortex plays a role in decision making by characterizing the systematic relationship between the activity of parietal neurons and the value of planned actions as well as the strength of the sensory evidence instructing them (Shadlen and Newsome 2001; Roitman and Shadlen 2002; Sugrue et al. 2004). Other studies have shown that parietal neurons encode abstract rules (Stoet and Snyder 2004), elapsed time (Leon and Shadlen 2003; Janssen and Shadlen 2005), and numerical quantity (Nieder and Miller 2004). Spatial and temporal aspects of spatial problem solving in the context of a visual maze task have similarly been related to neural activity in parietal area 7a (Crowe et al. 2004, 2005). Although posterior parietal cortex appears to serve as a sensorimotor interface, it does not appear that its functions are restricted to sensory attention and sensorimotor control. The present experiments extend research into the involvement of posterior parietal neurons in spatial cognition by addressing the question whether parietal neurons code relative spatial information. To that end, we tested whether the activity of posterior parietal neurons varied systematically with the spatial relationship existing between a point in space and a reference object. Available evidence suggests that the integrity of posterior parietal cortex in humans is important for this form of spatial representation, as parietal damage disrupts object construction and object-based forms of attention (Kleist 1934; Black and Strub 1976; Villa et al. 1986; Ruessmann et al. 1988; Driver and Halligan 1991; Driver et al. 1994; Tipper and Behrmann 1996).
The present experiments provide evidence that parietal neurons encode the spatial relationship between a point in space and a reference object (Figs 3, 4, 5A–C, 6, 7C,D, 8A,B). This conclusion is by necessity restricted to the limited set of reference objects, positions, and the single task context employed in the present experiments, and for that reason, it is not yet possible to determine with certainty whether the object-referenced spatial representation evident during object construction would generalize beyond this context. However, under the conditions imposed by the object construction task, a population of posterior parietal neurons was seen to exhibit a signal which varied systematically as a function of whether a task-critical coordinate was located to the relative left or right of a reference object. In a largely distinct population of parietal neurons, activity varied as a function of whether a task-critical coordinate was located to the left or right of the fixation target, which itself was aligned to the midline of viewer-centered frames of reference (Figs 4, 5D–F, 8C,D). We focus our discussion on the evidence supporting the novel conclusion that individual parietal neurons can represent spatial information that is object referenced.
Neural activity participating in an object-referenced spatial representation should meet at least 2 requirements. First, neural activity should remain constant when the retinocentric position coded by that activity varies, so long as the preferred spatial relationship between the retinocentric position and the reference object is maintained. Alternatively, neural activity should vary when the retinocentric position coded by that activity remains fixed, if a shift in the position the reference object alters the spatial relationship between it and the retinocentric position in question.
Evidence that the Spatial Representation Is Object Referenced
The following observations regarding the population of object-referenced neurons to our view support the hypothesis that the activity of these neurons represents an object-referenced spatial relationship during object construction:
Neural activity consistently varied as a function of whether a spatial coordinate was located to the left or right of a reference object presented at different retinocentric locations (Figs 3, 5–9; Supplementary Figs 1 and 2).
The activity of a considerable proportion of parietal neurons related significantly to object-referenced position, and not to viewer-referenced position, or the interaction between these factors in the ANCOVA. (Fig. 4. Of 504 neurons studied in Series B e.g., the activity of 143 neurons, or 28% of the sample, related exclusively to object-referenced position.)
Spatial tuning functions plotted in retinocentric coordinates exhibited multiple peaks, with each peak in activity aligned to a preferred side of the reference object presented at different positions in the display (Figs 5A–C, 6). This satisfies the first criterion enumerated above—that neural activity remains constant when retinocentric position varies so long as the preferred spatial relationship between retinocentric position and reference object is maintained.
Neural activity varied to represent critical squares located in a single, fixed retinocentric region according to the relative position of that region with respect to the reference object (Fig. 7C,D). This satisfies the second criterion enumerated above—that neural activity be shown to vary when retinocentric position remains constant, in the case that the spatial relationship between the retinocentric position and the reference object changes.
Object-referenced spatial tuning did not appear to reflect random selectivity for the various retinocentric positions tested. Neurons with bimodal object-aligned spatial tuning were significantly more frequent than neurons with alternative forms of spatial tuning that should be equally prevalent if spatial tuning were random.
Further, an index of object-referenced spatial selectivity computed from all neurons in the sample (without preselection of neurons using the ANCOVA) was significantly greater than predicted by a chance association between firing rate and retinocentric position.
Neural activity coded the preferred spatial relationship specifically at the variable time within the trial that this spatial relationship became defined by the combination of objects in the task (Fig. 9; Supplementary Fig. 2B).
We found behavioral evidence consistent with an object-referenced representation of space during the construction task. Reaction time to detect a probe stimulus was faster if the probe appeared on a particular side of the reference object (the same side as the missing critical square) across trials in which the reference object appeared at different locations (Fig. 11).
Most neurons in the left parietal cortex preferred the right side of the reference object and vice versa (Fig. 10). This hemispheric bias is similar to that observed in humans who, after suffering a parietal lesion, neglect the contralateral side of objects (Driver and Halligan 1991; Driver et al. 1994; Tipper and Behrmann 1996).
We interpret the above observations as providing support for the hypothesis that signals in individual parietal neurons can vary with the spatial relationship between a point in space and a reference object. Some alternative interpretations are considered below.
Considering whether Neural Activity Codes Object Shape
It is known that activity in parietal cortex is modulated by object shape (Sereno and Maunsell 1998; Sereno et al. 2002; Sereno and Amador 2006), raising the possibility that object-referenced neural activity reflected the shape of the reference object and not a spatial relationship between a point and the object. Several aspects of the data suggest that object shape could not account for object-referenced activity. Most importantly, population activity varied as a function of the spatial relationship between the critical square and the reference object when the shape of the reference object did not vary. For example, neural activity during the copy period varied as a function of whether the missing critical square was located to the left or right side of the copy object across trials in which the shape of the copy object remained the same on every trial (Figs 3, 4, 5C, 6, 8B). In addition, in Series A, the neural signal reflecting whether the critical square was on the left or right side of the reference object emerged during the model period on determinate model trials (Fig. 9A) but not until after the appearance of the copy object on ambiguous model trials (Fig. 9B). In the latter case, differential activity emerged immediately after the onset of the copy object even in the case that physically identical copy objects were presented on object-left and object-right trials (Supplementary Fig. 2B). Further, the spatial preference of each neuron was robust to variation in the shape (and also position) of the reference object that varied across model and copy periods within each trial of the task (Fig. 7C,D).
The above observations are not inconsistent with prior reports indicating that object shape influences neural activity in posterior parietal cortex (Sereno and Maunsell 1998; Sereno et al. 2002; Sereno and Amador 2006). However, the observations above argue that an influence of object shape did not entirely account for object-referenced spatial selectivity, because neural activity varied when object shape did not, and remained constant when shape varied, so long as the preferred spatial relationship between point and reference object was maintained.
Considering whether Neural Activity Codes a Rule
Object-referenced spatial selectivity may alternatively be construed to reflect a task-specific spatial rule. That is possible in the present case. The activity of the area 7a neurons we studied may have varied with relative spatial position but only in the context of the object construction task and only in reference to the specific subset of critical square and reference object positions we tested. However, the spatial representation exhibited several properties that do not immediately suggest a rule interpretation. The neural representation did not prescribe the direction or timing of the motor response (the former was constant and the later unpredictable when the neural signal was observed). Rules are typically expected to express “if-then” contingencies, the latter component of which appears to not be represented by neural activity in the present case (more specifically neural activity did not appear to specify what monkeys were supposed to do contingent on the relative position of the critical square). It will be important to test relative spatial representation in parietal cortex under different task contingencies and rules, to more directly establish whether relative spatial representation will generalize across task specifics, as has been done in the supplementary eye fields (SEF) (Tremblay et al. 2002). However, we believe that a rule interpretation does not negate the primary conclusion that the activity of parietal neurons can reflect a spatial relationship. Any rule applicable in the construction task that is consistent with the neural activity we observed would be contingent upon a spatial relationship. That is, to fit the data and the task, the rule would be of the general form expressed in the following statement: “if the critical square is located on the relative left side of the object, then …”, and there would be one such rule for each object-relative position, reflected in the activation of different subsets of parietal neurons.
Considering whether Neural Activity Codes a Motor Plan
Object-referenced neural activity could have reflected a motor plan to move to a point defined by its spatial relationship with respect to a reference object. Prior studies have shown that the activity of parietal neurons can reflect motor plans that monkeys never execute (Snyder et al. 1997a). Moreover, motor planning activity specifying movement direction in object-centered coordinates has been well characterized in the SEF, by Olson et al. (Olson and Gettner 1995, 1999; Olson and Tremblay 2000; Tremblay et al. 2002; Olson 2003). That series of experiments has shown that neural activity varies as a function of the object-referenced direction of a forthcoming saccade (Olson and Gettner 1995), that the activity does not depend on the visual attributes of the instructional cue (Olson and Gettner 1999), or whether the target object is a continuous object or a group of objects (Olson and Tremblay 2000).
It seems likely that object-relative spatial information coded by parietal neurons can be utilized to direct movement, given the sensorimotor functions presently considered to be performed by this cortical area. However, it may be the case that the spatial information represented in parietal cortex is at least one step removed from the final neural command to move. During the object construction task, it appeared that object-referenced relative positions were computed that did not serve as targets for movement. (Monkeys were not rewarded for, or allowed to make, movements toward the objects.) At a minimum, the present data add to that which shows that neural activity in parietal cortex does not bear an obligatory relationship to motor output in so far as neural activity can code spatial information that is not necessarily translated into movement. In this case, neural activity could reflect a motor command that is canceled by a downstream structure, or it could reflect spatial information that is not in itself a motor command, though it could still be used to elicit spatially congruent motor commands in downstream structures. In a prior study, we found that most neurons coding the location of the missing square during the copy period of the construction task, when object-referenced signals were prominent in the present experiment, were not active when saccades were planned and executed to the same retinocentric position in a delayed saccade task (Chafee et al. 2005). Area 7a appears to play a relatively minor role in saccade control as more neurons are activated after saccade initiation, and microstimulation thresholds necessary to evoke saccades are relatively high in comparison to area lateral intraparietal area (LIP; Barash et al. 1991; Thier and Andersen 1998). These observations suggest that neural activity in area 7a during the construction task does not reflect an unexecuted saccade command—however further work will be required to elucidate the relationship between object-referenced spatial representation in area 7a and motor control in tasks that, unlike the object construction task, require movement to points on objects.
A prior study (Sabes et al. 2002) investigated the neural representation of object-centered spatial information in area LIP when monkeys made saccades to specific parts of a reference object. The finding in that study was that neural activity varied as a function of the orientation of the reference object, and also of the oculocentric direction of the saccade (Sabes et al. 2002). However, activity did not vary systematically as a function of the object-centered direction of the saccade. The difference between the results of that study and the present one with respect to object-referenced spatial representation in parietal cortex may be due to any of several factors, including a difference in the parietal areas sampled (area LIP vs. area 7a), a difference in behavioral contexts (delayed saccades vs. object construction), and also a difference in the spatial manipulation of the reference object, translation in the present case versus rotation in the prior study (Sabes et al. 2002).
Comparison of Neural Activity to Models of Object-Based Neglect
Driver and Pouget (2000) have argued that object-centered deficits in attention can theoretically result from the loss of neurons with viewer-referenced (retinocentric) receptive fields. In this account, damage to parietal cortex in one hemisphere imposes a monotonically decreasing gradient of visual representation and salience established by the loss of neurons representing the contralateral visual hemifield. By virtue of this gradient, the contralateral half of objects is less salient than the ipsilateral half, regardless of where the objects appear on the retina. Deneve and Pouget (2003) similarly showed that object-centered spatial deficits can be reproduced by “lesioning” neural network models in which neurons possess retinocentric visual receptive fields. In this model, the magnitude of neural activity evoked by placing a part of an object within the retinocentric receptive field of a neuron was modulated by the orientation of the object and also by the relative retinocentric position of the object part relative to the rest of the object. Thus, object-centered spatial deficits do not necessitate explicit object-centered coding at a cellular level (Driver and Pouget 2000; Pouget and Sejnowski 2001; Deneve and Pouget 2003). Our data can be interpreted as exemplifying an effect of relative retinocentric position on neural activity, such that neurons code “left of” or “right of” with respect to a specific retinocentric locus that has become anchored to and moves with a reference object. (e.g., Such a neural representation of relative position might not remain aligned to the principal axis of the reference object were the object to rotate). However, an assumption of prior models has been that parietal neurons code space in a strictly retinocentric framework, albeit in a manner that is modulated by relative retinocentric position. In the present experiment, some parietal neurons appeared to carry a signal coding relative position in a manner that was independent of retinocentric position (as witnessed by the absence of a significant effect of viewer-referenced position on the activity of a considerable proportion of neurons).
Our results favor the hypothesis that neural activity during object construction represents the spatial relationship between a task-critical spatial coordinate and a reference object. It will be important to examine whether this spatial representation generalizes across a broader variety of spatial positions, reference objects, and task contexts than tested here. However, the present data provide, to our knowledge, the first evidence that object-referenced information is coded by single parietal neurons. This spatial representation could, in principle, affect both visual processing and motor control simultaneously, to influence object-referenced forms of both attention and movement planning. As is the case with all studies of a single cortical area, it is not clear from the present data whether area 7a is the generator of the object-referenced signals we observed. Parietal cortex is embedded within a number of distributed cortical systems, and it seems nearly certain that the signal we observed is a property of these distributed systems (Mountcastle 1978) and not the individual cortical areas that make them up. For example, not only do neurons in prefrontal cortex exhibit object-centered spatial selectivity (Niki 1974; Olson and Gettner 1995, 1999; Olson and Tremblay 2000; Tremblay et al. 2002; Olson 2003) but neurons in inferotemporal cortex do as well. Inferotemporal neurons code both the shape of visual contours and their relative positions within objects (Pasupathy and Connor 2001, 2002). Further study is needed to elucidate the role of these various areas and the networks that link them together in the neural representation of object space.
Humans with damage to parietal cortex in one cerebral hemisphere often have disrupted spatial representation of the contralateral half of objects, regardless of where the objects appear with respect to the viewer (Driver and Halligan 1991; Driver et al. 1994; Tipper and Behrmann 1996). We found a qualitatively similar hemispheric bias in the neural representation of object-relative space in monkey posterior parietal cortex. Neurons in each cerebral hemisphere preferentially represented the contralateral half of object-relative space when the object itself appeared at different locations in the display (Fig. 10). Further, we found behavioral evidence that performance was enhanced within a region of space corresponding to a particular side of the reference object when the position of the object varied (Fig. 11). This convergence of neurophysiological and behavioral data is consistent with the involvement of parietal cortex in representing space referenced to objects, potentially unifying 2 disparate effects of parietal damage in humans. Constructional apraxia (Kleist 1934; Black and Strub 1976; Villa et al. 1986; Ruessmann et al. 1988) and object-centered neglect (Driver and Halligan 1991; Driver et al. 1994; Tipper and Behrmann 1996) may reflect motor and perceptual consequences of an insult to the neural mechanism representing spatial relationships, such as the spatial relationships between the parts of an object that define object structure.
Supplementary figures can be found at: http://www.cercor.oxfordjournals.org/.
The authors thank Apostolos Georgopoulos for his important intellectual contribution to this work, and for his unfailing support. We thank D. Boeff and D. Evans for expert technical assistance. We thank A. Fortes and A. Basford for comments on the manuscript. This work was supported by United States Public Health Service NIH grant NS17413, the Department of Veterans Affairs, and the American Legion Brain Sciences Chair. Conflict of Interest: None declared.