Abstract

We recorded the activity of parietal area 7a neurons in monkeys performing an object construction task. In each trial, a model object consisting of a variable arrangement of squares was presented, followed after a delay by a copy of the model object that was missing a single square. Monkeys replaced the missing square to reconstruct the model configuration. Activity of many 7a neurons varied systematically with the position of the missing square and predicted where monkeys were going to add parts to the object they were building. The location of the missing square was a computed spatial datum important to object construction which did not correlate with the retinal location of a visual stimulus or the direction of the required motor response. The population of cells coding this coordinate was generally inactive when the same spatial locations were made relevant by visual targets to which monkeys either planned saccades or directed attention in other behavioral contexts. The data suggest that some parietal neurons participate in neural representations of space that reflect spatial cognitive as opposed to sensorimotor processing, coding the results of spatial computations performed on visual stimuli to meet cognitive objectives.

Introduction

The spatial functions of posterior parietal neurons in monkeys have often been investigated by recording neural activity while monkeys viewed, attended to or moved toward peripheral visual stimuli (Mountcastle et al., 1975; Robinson et al., 1978; Hyvarinen, 1981). A principle of parietal neurophysiology deriving from this work has been that many posterior parietal neurons have visual receptive or movement fields — regions of space where visual sensory input or motor output is associated with neural activation. Thus the spatial coordinates coded by parietal neurons have most often represented the location of an attended stimulus or the direction of a planned movement. This has been confirmed in recent experiments examining the relation between neural activity in posterior parietal cortex and various forms of spatial cognition, including spatial attention (Mountcastle et al., 1981; Steinmetz et al., 1994; Gottlieb et al., 1998; Bisley and Goldberg, 2003), spatial working memory (Gnadt and Andersen, 1988; Chafee and Goldman-Rakic, 1998), motor intention (Bracewell et al., 1996; Mazzoni et al., 1996; Snyder et al., 1997, 1998a) and decision processing (Platt and Glimcher, 1999; Shadlen and Newsome, 2001; Sugrue et al., 2004). Typically in these studies, cognitive processing was seen to modulate either the intensity of neural discharge (as in the case of attention or decision processing) or the duration of neural discharge (as in the case of working memory or motor intention) that represented the location of a stimulus or the direction of a movement. Thus cognitive processing has been correlated with the modulation of either the timing or the intensity of sensorimotor representations in parietal cortex. It is an interesting question whether all forms of spatial representation encode the locations of stimuli or movements, or whether the brain can also sustain abstract spatial representations that do not. In order to solve a spatial problem, for example, it may be necessary for cortical neurons to compute and represent abstract spatial parameters that do not correlate with the locations of their visual receptive or movement fields and do not represent the locations of stimuli or the directions of movement. The question seems an important one in assessing the potential variety of neural processes engaged in spatial cognition, and whether and to what degree the neural representation of space can be dissociated from sensorimotor function.

In the present experiment, we investigated neural activity in parietal cortex during object construction. Our choice of this behavior for study was motivated by evidence that parietal damage can interfere with a person's ability to assemble or construct objects, while sparing basic motor function. This defect in the spatial organization of movement is known as constructional apraxia (Kleist, 1934), and follows damage to various cortical areas (Benson and Barton, 1970; Arena and Gainotti, 1978; Carlesimo et al., 1993), but is most common and severe following lesions of the right posterior parietal cortex (Black and Strub, 1976; Villa et al., 1986; Ruessmann et al., 1988). Consequently object construction is likely to depend at least in part on spatial processing normally supported by the intact posterior parietal cortex.

Objects are typically constructed in order to match a desired configuration or ‘blueprint’, and consequently the act of construction involves largely directing the addition of new parts. As the constructive process progresses, a builder has to compare the object that they have constructed so far to a central representation of the object they ultimately want to construct. The difference between these spatial representations determines where new parts should go. This comparative process is typically formalized in clinical tests of constructional ability in which a model object is presented to patients, who are then instructed to produce a copy of it. The task facing the patient in these cases is to draw or assemble the parts of their copy together in such a way that the resulting structure matches the model object. The construction task we developed for monkeys incorporated this feature. Monkeys were trained to construct a copy of a model object, and to do this, they were required to compare the copy to the model and form a spatial judgement regarding the difference between them. If a comparative spatial operation such as this was performed during object construction, it seemed reasonable that this process would be reflected in (correlated with) neural activity in posterior parietal cortex, given the impact of parietal damage on constructional ability.

We found that during object construction, parietal neurons coded a spatial coordinate that correlated with the location of parts that were missing from copy objects in comparison to model objects monkeys were attempting to reproduce. These data related to one of the overall objectives of the work, which was to determine whether spatial information represented in the activity of parietal neurons could be identified as the product of spatial cognitive as opposed to sensorimotor processing. At the center of this distinction was the question of whether or not neural activity in parietal cortex could code spatial information that did not correspond to the location of a visual stimulus or the direction of a pending movement. The present results suggest that area 7a neurons, when confronted with complex and cognitively demanding spatial problems such as object construction, can represent spatial information that does not correlate with sensorimotor variables, but instead reflects the product of a computation performed on visual stimuli to meet cognitive objectives.

Materials and Methods

Two male rhesus macaque monkeys (weighing 6 and 4 kg) were prepared for single neuron recording using standard aseptic surgical technique under Isoflurane (1–2%) gas anesthesia. In each animal, two recording chambers (7 mm i.d.) were implanted bilaterally, one each above a craniotomy overlying area 7a in the left and right cerebral hemisphere. Four titanium posts were attached to the skull with titanium screws, and a halo was attached to provide an anchor point to stabilize head position during neural recording experiments. Analgesia was provided for a period of several days following surgery (Buprenex, 0.05 mg/kg BID, i.m.). Care and treatment of the animals conformed to the Principles of Laboratory Animal Care of the NIH (NIH publication no. 86-23, revised in 1995). The Internal Animal Care and Use Committees of the University of Minnesota and the Minneapolis Veterans Affairs Medical Center approved all experimental protocols.

Monkeys viewed visual stimuli appearing on a 19″ color computer monitor (Gateway EV900, 60 Hz refresh) centered at eye level, located at a distance of 42 cm. The left foot of each monkey rested on a single response pedal. Extension of the ankle produced pressure on the pedal closing a switch, and the time of switch closure relative to the stimulus sequence in the behavioral task determined whether or not the trial was correct (see below). Correct trials were rewarded with a drop (0.1 ml) of juice. Monkeys worked for either three or four trial sets each day. Each set contained 200–300 trials, requiring 45 min to 1 h to complete.

Data Collection

Eye position was measured using an infrared video eye tracking system sampling eye position at 60 Hz (ISCAN Inc., Burlington, MA). Single neuron activity was recorded with a 16-microelectrode matrix (Eckhorn system, Thomas Recording, GMbH, Giessen, Germany). The electrode signals were amplified (at a gain of 20 000), filtered (bandpass between 0.5 and 5 kHz), and action potentials were discriminated in each channel using a time-amplitude window discriminator (DDIS-1, Bak Electronics, Mount Airy, MD) or waveform discriminator (Multi Spike Detector, Alpha Omega Engineering, Nazareth, Israel). Spike timing was sampled with 40 μs resolution (DAP 5200a data acquisition processor, Microstar Laboratories, Bellevue, WA). Computer files containing the timing of action potentials relative to stimulus and behavioral events were saved for all neurons encountered. Data were obtained in groups of simultaneously recorded neurons (typically 25–30), and neurons whose average discharge rate exceeded 0.5 Hz during the model, delay and copy periods of the construction task were included in subsequent analyses. In both animals, ∼15% of isolated neurons failed to meet this spontaneous activity criterion; the remaining 85% of all cells isolated comprise the present database.

We conducted two series of experiments. In experimental series 1, we recorded the activity of 1614 area 7a neurons in monkeys M1 and M2 as they performed the visual object construction task and also the delayed saccade task. In experimental series 2, we recorded the activity of 367 area 7a neurons in monkey M2 performing the visual object construction task, the mixed-choice construction task and the peripheral attention task (see below). The locations of recording sites were estimated from the magnetic resonance (MR) image of a metal microelectrode placed at the center of each chamber entering the inferior parietal gyrus (Fig. 2c). Microelectrode penetrations were restricted to a circular region 4.8 mm in diameter centered on this point and were confined to area 7a in the inferior parietal lobule. A small minority of penetrations encroached on the bounding intraparietal and superior temporal sulci, although neurons within the banks of these sulci were not sampled. MR localization of recording location was confirmed in the first monkey at sacrifice (Fig. 2d). MR and histological localization were in close agreement (compare Fig. 2c,d, ‘Monkey 1’).

The Visual Object Construction Task

In the construction task, monkeys reconstructed the configuration of a remembered model object by adding elements to an incomplete copy object. Each model and incomplete copy object was made up of a group of square elements in various configurations. (Each square element was blue in color and subtended 1.4° of visual angle. Adjacent squares were separated by a 0.25° gap.) These stimuli were adapted from those used by Driver and colleagues to demonstrate object-based neglect after parietal lobe damage (Driver et al., 1994). The squares making up each object were placed on a regular five by five grid (8.3° in both width and height). Each model and incomplete copy included an object frame (Fig. 1a), consisting of a base row and central column of square elements forming an inverted ‘T’. The frame provided a base and central axis to the object. Model objects varied in the number and position of square elements added to this frame. Models contained either one (Fig. 1b) or two (Fig. 1c) additional squares on either the left or right sides of the object at various vertical positions, producing a set of 36 different configurations. Monkeys were trained on the task using the complete set of model objects, but a subset was used to record neural activity in order to reduce the number of trials that had to be performed for each set of neurons. We employed two basic variants of the construction task. In one of these, the model and copy objects appeared at the same location in the visual display, both centered on the central fixation target (Fig. 2a, ‘Model’ and ‘Incomplete copy’ panels of ‘Trial 1’). In this case, monkeys performed the task using the subset of eight model configurations shown within the solid square outlines in Figure 1b,c. In the second variant, the model and copy objects appeared at different locations in the visual display. The model object was offset a fixed distance to one side of the fixation target or the other at random, such that the model appeared entirely within either the left or right visual hemifields (Fig. 2a, ‘Model’ panel of ‘Trial 2’). The incomplete copy appeared as before centered on the fixation target (Fig. 2a, ‘Incomplete copy’ panel of ‘Trial 2’). In this case, monkeys performed the task using the subset of 14 model configurations shown within both the solid and dashed square outlines in Figure 1b,c.

Figure 1.

Configuration of visual stimuli presented as model objects in the construction task. (A) The object ‘frame’. All model objects included a central column and base row of square elements, forming an inverted ‘T’ form. Model objects consisted of this frame plus either one or two additional squares added to the frame at various positions. (B) One-square model objects consisted of the frame plus one square added at the positions shown. (C) Two-square model objects consisted of the frame plus two squares added at the positions shown. Monkeys were trained to perform the construction task using the entire set of 36 model objects (B, C). Neural activity was recorded using a subset of model objects to reduce the total number of trials required to study the activity of a given set of isolated neurons. On trials in which model and copy objects were presented at the same location, the subset of eight model configurations within the solid square outlines was used. On trials in which model and copy objects were presented at different locations, the subset of 14 model configurations within the solid and the dashed square outlines was used.

Figure 1.

Configuration of visual stimuli presented as model objects in the construction task. (A) The object ‘frame’. All model objects included a central column and base row of square elements, forming an inverted ‘T’ form. Model objects consisted of this frame plus either one or two additional squares added to the frame at various positions. (B) One-square model objects consisted of the frame plus one square added at the positions shown. (C) Two-square model objects consisted of the frame plus two squares added at the positions shown. Monkeys were trained to perform the construction task using the entire set of 36 model objects (B, C). Neural activity was recorded using a subset of model objects to reduce the total number of trials required to study the activity of a given set of isolated neurons. On trials in which model and copy objects were presented at the same location, the subset of eight model configurations within the solid square outlines was used. On trials in which model and copy objects were presented at different locations, the subset of 14 model configurations within the solid and the dashed square outlines was used.

Figure 2.

(A) Sequential events of the object construction task in two trials (represented by the horizontal row of panels marked ‘Trial 1’ and ‘Trial 2’). Each trial a model and incomplete copy object appeared in sequence. The copy was identical to the prior model, except that it was missing a single square. The monkeys' task was to replace the missing square by selecting between two choice squares appearing later in the trial. Fixation of a visual target at the center of the display was required throughout. Model and copy stimuli were both visible for 750 ms, and were separated by a 750 ms delay. Following the copy period, two choice squares appeared flanking the incomplete copy object, arranged in either a horizontal array (‘Trial 1’) or a vertical array (‘Trial 2’). First one and then the other choice square increased in luminance in random sequence (‘1st choice’, ‘2nd choice’). The monkey selected one of the two choice squares by controlling the time at which it depressed a single response key in relation to the sequential illumination of the choice squares. The choice square that was bright at the time that the response key was depressed was added to the incomplete copy object to create a new configuration (‘Complete copy’). If the monkey chose the correct choice square, the model configuration was recreated (‘Trial 1’), if not an erroneous configuration was produced (‘Trial 2’). (B) The location of the missing square. The missing square could occupy either of two horizontal locations (left or right) or two vertical locations (high or low). In this example, the missing square (square with inscribed ‘×’) was at the left and low location. (C) MRI based localization of recording sites in posterior parietal area 7a in two monkeys. Microelectrode penetrations were confined to a circular region 4.8 mm (black dots) in diameter located in the inferior parietal gyrus of each cerebral hemisphere. (D) Microelectrode penetrations (dots) determined at sacrifice and superimposed on a photograph of the cortical surface in Monkey 1. Recording locations are in close agreement with MRI reconstruction. IPS: intraparietal sulcus. STS: superior temporal sulcus.

Figure 2.

(A) Sequential events of the object construction task in two trials (represented by the horizontal row of panels marked ‘Trial 1’ and ‘Trial 2’). Each trial a model and incomplete copy object appeared in sequence. The copy was identical to the prior model, except that it was missing a single square. The monkeys' task was to replace the missing square by selecting between two choice squares appearing later in the trial. Fixation of a visual target at the center of the display was required throughout. Model and copy stimuli were both visible for 750 ms, and were separated by a 750 ms delay. Following the copy period, two choice squares appeared flanking the incomplete copy object, arranged in either a horizontal array (‘Trial 1’) or a vertical array (‘Trial 2’). First one and then the other choice square increased in luminance in random sequence (‘1st choice’, ‘2nd choice’). The monkey selected one of the two choice squares by controlling the time at which it depressed a single response key in relation to the sequential illumination of the choice squares. The choice square that was bright at the time that the response key was depressed was added to the incomplete copy object to create a new configuration (‘Complete copy’). If the monkey chose the correct choice square, the model configuration was recreated (‘Trial 1’), if not an erroneous configuration was produced (‘Trial 2’). (B) The location of the missing square. The missing square could occupy either of two horizontal locations (left or right) or two vertical locations (high or low). In this example, the missing square (square with inscribed ‘×’) was at the left and low location. (C) MRI based localization of recording sites in posterior parietal area 7a in two monkeys. Microelectrode penetrations were confined to a circular region 4.8 mm (black dots) in diameter located in the inferior parietal gyrus of each cerebral hemisphere. (D) Microelectrode penetrations (dots) determined at sacrifice and superimposed on a photograph of the cortical surface in Monkey 1. Recording locations are in close agreement with MRI reconstruction. IPS: intraparietal sulcus. STS: superior temporal sulcus.

Monkeys initiated the construction trial sequence by directing their gaze to a small fixation target (a red spot of light) presented at the center of the display. The monkey was required to maintain its gaze position within 1.3–1.7° of the fixation target until the trial was complete (Fig. 2a; the approximate size of the fixation window relative to the model object is indicated by the dotted circle in the ‘Model’ panel of ‘Trial 1’). If at any time before the end of the trial gaze position deviated by more than this distance from the fixation target the trial was terminated. After 500 ms of fixation, a model object was presented for 750 ms (Fig. 2a, ‘Model’) and then disappeared. The disappearance of the model marked the beginning of a 750 ms delay period in which only the fixation target was visible (Fig. 2a, ‘Delay’). At the end of the delay period an incomplete copy object appeared (Fig. 2a, ‘Incomplete copy’). The copy object remained visible for 750 ms, before an array of choice squares appeared (Fig. 2a, ‘Choice array’). Neural activity during the 750 ms period beginning with the appearance of the incomplete copy object and ending with the appearance of the choice array is the focus of this report. Each incomplete copy object was identical to the model object preceding it on a given trial, except that one square element had been removed, referred to as the ‘missing square’ (Fig. 2b). The missing square was always one of the additional squares placed adjacent to the object frame in the model and not one of the squares making up the frame itself. In the case of model objects having two squares on the same side of the object and at the same vertical position (second row of objects in Fig. 1c), the missing square was always the outermost of the two squares added to the frame. In order to successfully complete the trial, monkeys were required to add a new square to the incomplete copy object in order to replace the missing square and so reconstruct the model configuration. This was done by selecting one of the two choice squares in the choice array. One choice square was the correct choice, and was located at the same vertical position and on the same side of the incomplete copy object as the missing square. (In Fig. 2a, ‘Trial 1’, the correct choice is the leftmost of the two choice squares. In Fig. 2a, ‘Trial 2’, the correct choice is the lower of the two choice squares.) The incorrect choice square was a distracter. The array of choice squares randomly appeared in one of two orientations, forming either a horizontal array (Fig. 2a, ‘Choice array’ panel of ‘Trial 1’) or a vertical array (Fig. 2a, ‘Choice array’ panel of ‘Trial 2’). In horizontal choice arrays, correct and incorrect choice squares were located on opposite sides of the copy object. In vertical choice arrays, correct and incorrect choice squares were located on the same side of the copy object. After a variable interval (300–600 ms), one of the choice squares selected at random increased in luminance for a period of 700–1000 ms (Fig. 2a, ‘1st choice’). If the monkey pressed the response key while the first choice square was bright (as in Fig. 2a, ‘1st choice’ panel of ‘Trial 2’), the first choice was selected for addition to the incomplete copy object. If the monkey did not press the response key in this interval, the first choice square dimmed to its original luminance at the same time that the second choice square brightened. The second choice square then remained bright for a period of 700–1000 ms (Fig. 2a, ‘2nd choice’ panel of ‘Trial 1’). If the monkey pressed the response key during this interval, the second choice square in the sequence was selected for addition to the incomplete copy object. In either case, at the time that the response key was pressed, the dim choice square disappeared and the bright hoice square translated smoothly inward toward the model in a horizontal direction, coming to rest adjacent to the outermost square in the copy object to produce a new configuration (Fig. 2a, ‘Complete copy’). The complete copy remained visible for a period of 300 ms before disappearing at the end of the trial. If the monkey selected the correct choice square, its addition produced a complete copy object whose configuration matched the preceding model, and the trial was rewarded after the complete copy period (Fig. 2a; compare ‘Model’ and ‘Complete copy’ panels of ‘Trial 1’). If the monkey selected the incorrect choice square, an erroneous configuration resulted. The complete copy object did not match the preceding model, and the trial was not rewarded (Fig. 2a; compare ‘Model’ and ‘Complete copy’ panels of ‘Trial 2’).

Delayed Saccade Control Task

We recorded the activity of area 7a neurons while monkeys performed a delayed saccade task. After a period of central fixation (500 ms), a saccade target was presented at one of eight peripheral locations (Fig. 5a). Fixation and saccade targets were both red spots. Saccade target locations corresponded to the centers of missing squares and choice squares in the construction task (object squares were not visible in the saccade task but are shown as dotted outlines to illustrate their locations relative to saccade targets). After a fixed delay of 750 ms, the central fixation target disappeared, and the monkey made a saccade toward the peripheral target. The trial was rewarded if that saccade ended within 1.3–1.7° of the peripheral target location and gaze was maintained at that location for an additional 500 ms.

Dual Attention–Construction Task

We used a dual task paradigm in order to probe the allocation of visual attention during object construction. In this dual task, all model objects consisted of the frame plus a single additional square (Fig. 1b). Most trials (85 or 75%) were standard construction trials as described above. A minority of trials (15 or 25%) were attention probe trials. The monkey was not cued as to which trial type was in progress — construction and probe trials progressed through model, delay and copy periods presenting identical visual stimuli. On probe trials, 150–750 ms after the incomplete copy object appeared, a visual probe stimulus was flashed for 50 ms and the choice square array did not appear. The monkey's task was to depress the response key as soon as it detected the probe (the key press had to occur between 50 and 600 ms after probe onset). The probe stimulus was a single red square whose contrast was greater than threshold (the monkey successfully detected 93% of probe stimuli). The horizontal position of the probe stimulus was varied relative to the position of the missing square (Fig. 9), and was expressed in degrees offset from that position. The probe was presented either at the position of the missing square (position 0°), at the mirror opposite location within the copy object (position −3.3°), at the location of the correct choice square (position 3.3°) or at the location of the incorrect choice square (position −6.6°).

Mixed-choice Construction Task

The horizontal position of the missing square and the correct choice square were dissociated in a mixed choice task. Choice squares were arranged in a vertical array and presented in one of two locations relative to the missing square. On standard trials, the choice array appeared on the same side of the copy object as the missing square, and so the correct choice and the missing square shared the same horizontal position relative to the central axis of the copy object (Fig. 10a, left). On crossed trials, the choice array appeared on the opposite side of the copy object as the missing square, and so the correct choice and missing square had different horizontal positions relative to the central axis of the copy object (Fig. 10a, right). In either case, the correct choice was located at the same vertical position as the missing square. The monkey was cued as to whether a standard or crossed trial was in progress by the color of the fixation target (red indicated a standard trial, green indicated a crossed trial). On crossed trials, the correct choice square translated across the central axis of the copy object and came to rest at the missing square location on the opposite side of the object.

Peripheral Attention Task

We recorded the activity of 7a neurons while monkeys directed attention toward a peripheral visual stimulus in order to detect its dimming (Bushnell et al., 1981; Colby et al., 1996). Trials began when monkeys fixated a central fixation target (a blue spot of light). After 500 ms a second blue spot target identical to the fixation target appeared in a peripheral location (Fig. 11a, left). Monkeys were required to maintain fixation of the central fixation target while directing attention to the peripheral target in order to detect a subtle decrease in its luminance that occurred at an unpredictable time. The peripheral target was presented in one of four locations selected at random. Peripheral target locations corresponded to the centers of the four possible locations of the missing square relative to the fixation target in the construction task (the missing square locations are indicated by the dotted square outlines; Fig. 11a, left). After a random interval of between 400 and 1400 ms, the peripheral target dimmed slightly (Fig. 11a, right). The monkey was rewarded for pressing the response key between 50 and 500 ms following the dimming of the peripheral target.

Data Analysis

All data analyses were implemented in Matlab (The MathWorks, Inc., Natick, MA). We employed an analysis of covariance (ANCOVA) to assess the influence of task variables on neural activity. The ANCOVA was implemented as a general linear model (Kleinbaum et al., 1988; Howell, 1997) estimated by the REGRESS function in the Matlab Statistical Toolbox. All ANCOVA analyses included two covariates. The first covariate was the baseline firing rate during a 500 ms period while the monkey fixated the central target before the appearance of the model object at the beginning of each trial. The second covariate was the time elapsed since the beginning of data collection for a given set of isolated neurons and the start of each trial in the set. The influence of the independent variables on neural activity was therefore assessed independently of either fluctuation in basal activity across trials or linear trends in activity across the recording session. An α level of 0.05 was used unless otherwise noted.

In experimental series 1, we analyzed neural activity on the construction and delayed saccade tasks. In the construction task, the dependent variable in the analysis was the firing rate of a given neuron on each trial during the incomplete copy period. The copy period began with the appearance of the incomplete copy object, and ended with the appearance of the choice array 750 ms later. The two fixed factors in the analysis were the horizontal position (left/right) and the vertical position (high/low) of the missing square in the incomplete copy object coded as categorical factors (Fig. 2b). These factors together defined the location of the missing square relative to the central axis (horizontal position) and base (vertical position) of the incomplete copy object. In the delayed saccade task, the dependent variable was the firing rate on each trial during the delay period. This period began with the appearance of the saccade target in peripheral vision, and ended with the offset of the fixation target, which was the go signal to make a saccade to the target. The two fixed factors were the horizontal position (left/right) and vertical position (high/low) of the saccade target, again coded as categorical factors. Trials with near targets (at the locations of removed squares in the incomplete copy object) and far targets (at the locations of choice squares) were pooled (Fig. 5a).

In experimental series 2, we employed an ANCOVA to analyze the activity of neurons in a mixed-choice construction task. The dependent variable in this analysis was neural activity during the copy period. The independent variables were the horizontal position of the missing square (left/right), and the horizontal position of the choice square (left/right), with respect to the central axis of the copy object. A relation between neural activity and the position of the choice square was anticipatory in nature because the neural activity analyzed was measured during the copy period, and the choice array had not yet appeared at this point in the trial (Fig. 2a). Horizontal positions of missing and choice squares were included as two independent factors in this design because these positions were uncorrelated in the mixed-choice task. In experimental series 2, we also analyzed the activity of a group of neurons that were recorded on both the standard construction and the peripheral attention tasks. In this group of neurons, we simplified the ANCOVA by collapsing the coding of position into a single factor with four levels (to code the location both of the missing square and also the peripheral target). This simplified the comparison between tasks to the pattern of significance obtained across two (instead of four) factors, as was the case in analyzing the mixed-choice task data.

We constructed spike density functions (SDF) from the pooled activity of neural populations. First we identified (partially overlapping) populations of neurons whose activity varied significantly either with the horizontal or the vertical position of the missing square as assessed by the above ANCOVA. For the purposes of constructing population spike density functions in Figures 4 and 5, we used a stringent statistical criterion to select cells with the strongest signals in the population (including only those neurons whose firing rates were affected by the relevant spatial variable at P < 0.001). Trials were segregated into preferred and nonpreferred groups as a function of the location of the missing square along the horizontal or vertical axis that was associated with the greatest activity in each neuron. Each trial, represented by a sequence of spike times, was then converted into a SDF evaluated at 1 ms intervals using the KSDENSITY kernel smoothing function in the Matlab Statistical Toolbox. A Gaussian kernel of σ = 15 was used to construct population spike density functions (in the case of the spike density functions constructed from the activity of the single neuron in Figure 3, a kernel of σ = 20 ms was used). Individual trial SDF were then averaged across the trials in preferred and nonpreferred groups separately for each neuron and normalized (by subtracting the minimum at each point and dividing by the range). These average normalized SDF for each neuron were again averaged across all the neurons in each population to yield the population SDF in Figures 4, 5, 10 and 11. In Figures 4 and 5, the standard error of the mean of the population SDF was computed across neurons and is represented by the region of lighter shading surrounding each SDF. When neural activity was contrasted across tasks, the preferred horizontal or vertical position of each neuron was defined by the activity of that neuron on the standard construction task, and this preferred position was then used to segregate trials in either the delayed saccade task (Fig. 5d), the mixed-choice construction task (Fig. 10d,f) or the peripheral attention task (Fig. 11e) into preferred and nonpreferred groups. The difference between population activity on preferred and nonpreferred trials in the various control tasks therefore illustrates the degree to which neurons in the population maintained a consistent spatial preference across the tasks.

Figure 3.

Example of a neuron in area 7a in which activity during the copy period reflected the location of the missing square. (A–L) Trials are segregated according to the combination of model and incomplete copy objects that were presented, as illustrated above each raster. The time of the sequential events in the construction trial are indicated by the four vertical lines through each raster (excluding the y axis) which indicate (left to right) the onset of the model object, the offset of the model object (beginning of the delay period), onset of the copy object, and onset of the choice array, as labeled in (L). The incomplete copy period is indicated by the vertical bar of gray shading. At the bottom of each raster a spike density function (σ = 20 ms) illustrates mean neural activity over time. Divisions indicate increments of 25 Hz on the vertical axis and 250 ms on the horizontal axis. (B, F, J) Neural activity evoked by the copy object is a function of the location of the missing square relative to the preceding model. Activity increases on trials in which the missing square is located at the lower left position within the object (compare model and copy objects). (A–D) Neural activity varied with the location of the missing square when the visual form of the copy object did not vary. On these trials, models consisted of the object frame plus a single square, and consequently the copy object consisted of the object frame only in every case.

Figure 3.

Example of a neuron in area 7a in which activity during the copy period reflected the location of the missing square. (A–L) Trials are segregated according to the combination of model and incomplete copy objects that were presented, as illustrated above each raster. The time of the sequential events in the construction trial are indicated by the four vertical lines through each raster (excluding the y axis) which indicate (left to right) the onset of the model object, the offset of the model object (beginning of the delay period), onset of the copy object, and onset of the choice array, as labeled in (L). The incomplete copy period is indicated by the vertical bar of gray shading. At the bottom of each raster a spike density function (σ = 20 ms) illustrates mean neural activity over time. Divisions indicate increments of 25 Hz on the vertical axis and 250 ms on the horizontal axis. (B, F, J) Neural activity evoked by the copy object is a function of the location of the missing square relative to the preceding model. Activity increases on trials in which the missing square is located at the lower left position within the object (compare model and copy objects). (A–D) Neural activity varied with the location of the missing square when the visual form of the copy object did not vary. On these trials, models consisted of the object frame plus a single square, and consequently the copy object consisted of the object frame only in every case.

Figure 4.

Spike density functions (SDF) illustrate the average normalized activity of neural populations in area 7a in which activity varied significantly (P ≤ 0.001) with either the horizontal (A, C) or vertical (B, D) position of the missing square. In each panel, population activity is separately illustrated on preferred (blue SDF) and nonpreferred (red SDF) trials (preferred trials were defined according the position of the missing square associated with the greatest activity in each neuron). Trials are further segregated according to whether the model object included one (A, B), or two (C, D) squares in addition to the object frame. Models with one additional square were determinate in the sense that they unambiguously determined where the missing square in the copy object was going to be (only the one square present in the model in addition to the object frame could be removed). Models with two additional squares were indeterminate in the sense that either of the two additional squares present in the model could be removed to produce the copy on a given trial, and the location of the missing square was therefore ambiguous until the copy object appeared. (A, B) On determinate model trials, population activity differentiates the horizontal (A) and vertical (B) position of the missing square early in the trial; shortly after model onset (note black arrows at points were blue and red SDF diverge during the model period). (C, D) On indeterminate model trials, population activity differentiates the horizontal (C) and vertical (D) position of the missing only after the copy object appears. (Note black arrows at points where blue and red SDF diverge during the copy period.)

Figure 4.

Spike density functions (SDF) illustrate the average normalized activity of neural populations in area 7a in which activity varied significantly (P ≤ 0.001) with either the horizontal (A, C) or vertical (B, D) position of the missing square. In each panel, population activity is separately illustrated on preferred (blue SDF) and nonpreferred (red SDF) trials (preferred trials were defined according the position of the missing square associated with the greatest activity in each neuron). Trials are further segregated according to whether the model object included one (A, B), or two (C, D) squares in addition to the object frame. Models with one additional square were determinate in the sense that they unambiguously determined where the missing square in the copy object was going to be (only the one square present in the model in addition to the object frame could be removed). Models with two additional squares were indeterminate in the sense that either of the two additional squares present in the model could be removed to produce the copy on a given trial, and the location of the missing square was therefore ambiguous until the copy object appeared. (A, B) On determinate model trials, population activity differentiates the horizontal (A) and vertical (B) position of the missing square early in the trial; shortly after model onset (note black arrows at points were blue and red SDF diverge during the model period). (C, D) On indeterminate model trials, population activity differentiates the horizontal (C) and vertical (D) position of the missing only after the copy object appears. (Note black arrows at points where blue and red SDF diverge during the copy period.)

Figure 5.

Neurons coding the location of the missing square generally did not code the location of targets in a delayed saccade task. (A) Saccade targets were placed at the eight locations shown (red dots) centered on four possible locations of the missing square (near targets), and the four possible locations of the choice squares (far targets) in the construction task. The dotted square outlines illustrate the relative positions of object squares in the construction task to saccade targets in the saccade task (object squares are included for purposes of illustration and were not visible during the saccade task). (B) Positional preferences of single neurons active on both the construction and saccade tasks. Each point in the scatter plot indicates the directional index (see Materials and Methods) computed for a single neuron during construction (abscissa) and saccade (ordinate) tasks. Neurons were included in this plot if their activity related significantly either to horizontal position (blue circles) or vertical position (red triangles) in both tasks, coding both the missing square and the saccade target along the same spatial axis. This population included 139 neurons, and constituted a minority (16%) of the larger population of neurons in which activity related to the position of the missing square in the construction task. The directional indices were only weakly correlated, and many neurons preferred different positions in the two task contexts (represented by the points in the upper left and lower right quadrants). (C) Normalized activity of the population of neurons coding the horizontal position of the missing square during the construction task (ANCOVA, P ≤ 0.001; n = 152 neurons). (D) The activity of the same 152 neurons on the delayed saccade task; trials are segregated according to the horizontal position of the saccade target, and preferred and nonpreferred positions are defined by each neuron's horizontal position preference on the construction task. Activity in this neural population is related to the horizontal position of the missing square but not the saccade target. (E, F) Activity of populations of 7a neurons in which activity varied significantly with the horizontal position of the saccade target (P ≤ 0.001), during either the delay period (E; n = 61 neurons) or the saccade period (F; n = 48 neurons) of the delayed saccade task. The activity of 7a neurons exhibited spatial selectivity during the saccade task, but this population of neurons was largely distinct from the group of neurons whose activity coded the position of the missing square during object construction.

Figure 5.

Neurons coding the location of the missing square generally did not code the location of targets in a delayed saccade task. (A) Saccade targets were placed at the eight locations shown (red dots) centered on four possible locations of the missing square (near targets), and the four possible locations of the choice squares (far targets) in the construction task. The dotted square outlines illustrate the relative positions of object squares in the construction task to saccade targets in the saccade task (object squares are included for purposes of illustration and were not visible during the saccade task). (B) Positional preferences of single neurons active on both the construction and saccade tasks. Each point in the scatter plot indicates the directional index (see Materials and Methods) computed for a single neuron during construction (abscissa) and saccade (ordinate) tasks. Neurons were included in this plot if their activity related significantly either to horizontal position (blue circles) or vertical position (red triangles) in both tasks, coding both the missing square and the saccade target along the same spatial axis. This population included 139 neurons, and constituted a minority (16%) of the larger population of neurons in which activity related to the position of the missing square in the construction task. The directional indices were only weakly correlated, and many neurons preferred different positions in the two task contexts (represented by the points in the upper left and lower right quadrants). (C) Normalized activity of the population of neurons coding the horizontal position of the missing square during the construction task (ANCOVA, P ≤ 0.001; n = 152 neurons). (D) The activity of the same 152 neurons on the delayed saccade task; trials are segregated according to the horizontal position of the saccade target, and preferred and nonpreferred positions are defined by each neuron's horizontal position preference on the construction task. Activity in this neural population is related to the horizontal position of the missing square but not the saccade target. (E, F) Activity of populations of 7a neurons in which activity varied significantly with the horizontal position of the saccade target (P ≤ 0.001), during either the delay period (E; n = 61 neurons) or the saccade period (F; n = 48 neurons) of the delayed saccade task. The activity of 7a neurons exhibited spatial selectivity during the saccade task, but this population of neurons was largely distinct from the group of neurons whose activity coded the position of the missing square during object construction.

We recovered the location of the missing square by decoding neural population activity using linear discriminant analysis (Johnson and Wichern, 1998; Averbeck et al., 2003). Linear discriminant analysis (LDA) is a multivariate statistical technique that classifies observations to categories on the basis of a group of discriminating variables. In the present analysis, observations were trials of the construction task, and each trial was represented by a population activity vector consisting of the mean firing rate of each neuron in a population during the copy period of the task. The population activity vector therefore represented the distribution of firing rates across the neurons in the population on a given trial while the copy object was visible. There were four categories in the analysis, corresponding to the four possible locations of the missing square in the copy object. Classification of trials to categories was carried out using the CLASSIFY function of the Matlab statistical toolbox. A discriminant function for each category was derived by computing the parameters of a multivariate normal probability density distribution whose mean was equal to the mean population activity vector averaged across the trials in a given category and whose covariance was estimated by pooling trials across categories (see equation 3 of Averbeck et al., 2003). Equal prior probabilities were assumed (as the square could be removed from any one of the four potential locations with equal probability). The analysis proceeded by using the discriminant functions to compute a set of scores for each trial associated with its membership in each of the categories. The trial was then classified to the category with the highest score. This was carried out using fivefold cross-validation — the mean and covariance of the discriminant functions for the categories were defined on the basis of four-fifths of the trials and the remaining fifth was treated as being of unknown category and was classified using LDA. The procedure was replicated for successive fifths of the data until all trials were classified. Generally, LDA performance scales with the number of neurons included in the analysis (Averbeck et al., 2003). The present analysis was based on a population of 7a neurons whose firing rate during the copy period varied significantly with the location of the missing square. This population included neurons that belonged to different ensembles recorded at different times (that is not all of the neurons in the population were recorded simultaneously). Neurons recorded at different times were aggregated into this larger population by considering a single trial in the analysis to be a particular repetition of a given pair of model and copy objects. The firing rates of all neurons in the population on a given trial defined in this way were then grouped into a single population activity vector.

In a separate analysis, we recovered the location of the missing square when monkeys chose the incorrect choice square and built the incorrect object. We identified error trials and divided them into four groups. These four groups included trials in which the missing square was located on the left side of the copy but the monkey added a square on the right, and vice versa; and trials in which the missing square was located at the upper position within the copy but the monkey added a square at the lower position, and vice versa. We constructed neural populations for each error type. The neurons included for each error type were those in which (i) activity on correct trials varied significantly with the position of the missing square along the spatial axis (horizontal or vertical) associated with that type of error (ANCOVA; P ≤ 0.05); and (ii) a neuron was recorded while the monkey committed at least 10 errors of that type. Once the populations had been identified, categories for LDA classification were constructed on the basis of the activity in each population on correct trials. In the case of each population and error type, the classification included two categories. In populations recorded during horizontal errors, the two categories corresponded to the left and right side of the copy object. In populations recorded during vertical errors, the two categories corresponded to the high and low positions in the copy object. We then classified error trials to these categories using LDA as above. On error trials, one position in each classification corresponded to the location of the missing square whereas the other corresponded to the location of the square that the monkey erroneously added to the object (on error trials these locations were different).

Results

We recorded the activity of 1981 neurons in posterior parietal area 7a (Fig. 2c,d) of two male rhesus macaque monkeys (M1 and M2) as they performed the visual object construction task (1092 of these neurons were recorded within the first monkey, 889 in the second). Of these cells, 1614 were recorded from both monkeys in experimental series 1 (standard construction task plus delayed saccade task). An additional 367 neurons were recorded from M2 in experimental series 2 (standard construction task plus mixed-choice and peripheral attention tasks, see Materials and Methods). Monkey M1 performed 83% of construction trials correctly in which the model object appeared randomly in left or right visual hemifields, and 94% of trials correctly in which model and probes both appeared centered in the display. Overall performance in series 1 was 87% correct. Monkey M2 performed 90% of construction trials correctly in which the model object appeared randomly in left or right visual hemifields, and 91% of trials correctly in which model and probes both appeared centered in the display. In series 2, Monkey M2 performed 97% of mixed-choice construction trials correctly, with comparable performance on standard (99% correct) and crossed (97% correct) trials. Performance was more accurate in monkeys M1 and M2 on trials with vertical choice arrays (88% and 91% respectively) than with horizontal choice arrays (86% and 89% correct respectively), although the difference was slight (only 3% difference in both animals).

Neural Activity in 7a Correlates with Spatial Judgement during Object Construction

In the construction task, monkeys viewed two objects in sequence — first the model object and then, after a delay, the incomplete copy object. Both objects consisted of varying arrangements of identical squares presented on a video monitor. The incomplete copy was identical to the preceding model each trial, except that a single square had been removed (Fig. 2).

We found that after the incomplete copy appeared, 7a neurons became active as a function of the location of the square that had been removed from the incomplete copy object relative to the preceding model. An example of this pattern of activity in a single 7a neuron is shown in Figure 3. During the incomplete copy period (indicated by the vertical gray bar of shading through each raster), neural activity increased on trials in which model and incomplete copy objects together connoted a square removed from the lower left position in the incomplete copy object (Fig. 3b,f,j). This increase in activity was not a function of the pattern of visual input at the time it occurred. On trials in which the model consisted of the object frame plus one additional square, the incomplete copy object consisted of the frame only. In spite of the fact that the incomplete copy object was physically identical across these trials, its appearance evoked markedly different levels of neural activity depending on where the square had been removed from the copy object (Fig. 3ad). On trials in which the model consisted of the object frame plus two additional squares, the neuron exhibited the same preference, and incomplete copy objects in which squares had been removed from the lower left position excited the neuron (Fig. 3f,j); physically identical copy objects from which squares had been removed at other locations did not (compare copy objects in Fig. 3f and l, and those in Fig. 3j and h). Activity during the incomplete copy period varied across trials with the same model object (compare model objects in Fig. 3e and f, and those in Fig. 3j and k), and so was not likely to represent either a working memory of the model configuration or the reactivation of a visual receptive field stimulated by the model. Activity during the incomplete copy period did not reflect an association between a particular pair of model and incomplete copy objects because it generalized across all object pairs in the set that together defined a missing square at the preferred position (Fig. 3b,f,j).

We assessed the influence of the location of the missing square in the copy object on neural activity during the copy period using a two-way ANCOVA. The two orthogonal fixed factors in the analysis (horizontal and vertical position) jointly determined the location of the missing square (Fig. 2b). The activity of approximately one in four area 7a neurons during the incomplete copy period that met a minimal activity criterion (see Materials and Methods) varied significantly as a function either of the horizontal position (27%) or the vertical position (24%) of the square removed from the incomplete copy object relative to the preceding model (Table 1, ‘Combined’). The proportions of neurons exhibiting these effects were comparable in the two monkeys considered individually (Table 1, ‘Monkey 1’ and ‘Monkey 2’). The group of neurons whose activity was significantly related to either the horizontal or vertical position of the missing square (or both spatial factors) included 43% of all spontaneously active area 7a neurons isolated for study (700 of 1614 cells).

Table 1

Results of a two-way ANCOVA in which the horizontal position and the vertical position of the missing square were fixed factors

Main effects in ANCOVA (position of missing square)
 
Monkey 1
 
Monkey 2
 
Combined
 
Horizontal position 27% (298/1092) 28% (146/522) 27% (444/1614) 
Vertical position 27% (296/1092) 18% (97/522) 24% (393/1614) 
Interaction 10% (111/1092) 8% (43/522) 9% (154/1614) 
Horizontal or vertical
 
45% (489/1092)
 
40% (211/522)
 
43% (700/1614)
 
Main effects in ANCOVA (position of missing square)
 
Monkey 1
 
Monkey 2
 
Combined
 
Horizontal position 27% (298/1092) 28% (146/522) 27% (444/1614) 
Vertical position 27% (296/1092) 18% (97/522) 24% (393/1614) 
Interaction 10% (111/1092) 8% (43/522) 9% (154/1614) 
Horizontal or vertical
 
45% (489/1092)
 
40% (211/522)
 
43% (700/1614)
 

The dependent variable in the analysis was neural activity during the copy period. Trials in which the model and copy object appeared at different locations, and trials in which the objects appeared at the same location, were pooled for this analysis. The row labeled ‘Horizontal position’ tallies the percentages of neurons (and proportion of the total sample in parenthesis) for which the horizontal position of the missing square significantly affected firing rate, regardless of the effect of vertical position. Likewise, the row labeled ‘Vertical position’ tallies the percentages and proportion of neurons for which the vertical position of the missing square significantly affected firing rate, regardless of the effect of horizontal position. The row labeled ‘Interaction’ tallies the percentages and numbers of neurons for which the interaction between these two spatial factors significantly influenced neural activity. Finally, the row labeled ‘Horizontal or vertical’ tallies the percentages and proportion of neurons in which activity related either to horizontal position alone, to vertical position alone, or to both spatial factors.

Trials in which model and incomplete copy objects appeared at the same or different locations in the display are pooled in Table 1. Trials in Table 2 are restricted to those in which model and incomplete copy objects appeared at different locations (the model object was offset into the left or right visual hemifield while the copy object was centered on the fixation target). The effects of the position of the square removed from the incomplete copy were equally prevalent in this subset of the data. This is evidence that neural activity in 7a coding where the square had been removed from the incomplete copy object did not require that the corresponding square in the model object appear at the same retinotopic location.

Table 2

Results of a two-way ANCOVA in which the horizontal position and the vertical position of the missing square were fixed factors

Main effects in ANCOVA (position of missing square)
 
Combined (Monkeys 1 and 2)
 
Horizontal position 27% (275/1016) 
Vertical position 25% (251/1016) 
Interaction 8% (84/1016) 
Horizontal or vertical
 
44% (445/1016)
 
Main effects in ANCOVA (position of missing square)
 
Combined (Monkeys 1 and 2)
 
Horizontal position 27% (275/1016) 
Vertical position 25% (251/1016) 
Interaction 8% (84/1016) 
Horizontal or vertical
 
44% (445/1016)
 

The dependent variable in the analysis was neural activity during the copy period. Data included in this analysis were restricted to trials in which the model and copy appeared at different locations (other conventions as in Table 1). Neurons whose activity related to the horizontal or vertical position of the missing square were approximately as common in this sub-sample as when all available trials were combined (Table 1). The relationship between neural activity and the location of the missing square did not require that the model and copy objects appear the same retinotopic location.

The set of model objects included some in which two squares were stacked together on the same side and at the same height in the model object (second row of model objects in Fig. 1c). In these objects, the missing square (always the outermost of the two squares added to the frame) was located at a farther eccentricity from the central axis of the object than in other model configurations. We tested the effect of the eccentricity of the missing square on the activity of those neurons coding the horizontal position of the missing square. We found that the activity of approximately one in five of these neurons (62 of 275 neurons coding horizontal position in Table 2) was significantly influenced not only by the side but also by the eccentricity of the missing square relative to the central axis of the object (the activity of these neurons was significantly affected by both the side and eccentricity of the missing square at P ≤ 0.05 in a two-way ANCOVA employing side and eccentricity as factors).

We selected neurons in which activity during the copy period varied significantly (P ≤ 0.001) with either the horizontal position (Fig. 4a,c; 152 neurons) or the vertical position (Fig. 4b,d; 108 neurons) of the missing square. We then constructed average normalized population spike density functions (SDF) illustrating the time course of activity in these two populations (see Materials and Methods). In each panel (Fig. 4ad), separate SDF illustrate population activity on trials in which the missing square was in the preferred position (blue SDF), or the nonpreferred position (red SDF) of each neuron included in the population. (The preferred position of each neuron was defined as the position of the missing square associated with the greatest activity during the copy period.) On the subset of trials in which the model object consisted of the frame plus one additional square (Fig. 1b), the copy object consisted of the object frame only (Fig. 1a), and so the visual form of the copy object did not vary. Population activity on these trials clearly differentiated the horizontal (Fig. 4a) or vertical (Fig. 4b) position of the missing square during the copy period (note the separation between blue and red SDF), in spite of the fact the same copy object was presented on every trial. This confirmed at the population level what was seen at the single cell level (Fig. 3) — that neural activity during the copy period was not a function of the form of the copy object. On these trials, we also noted that the population signal coding the position of the missing square emerged early in the trial, shortly after the onset of the model object (Fig. 4a,b; note black arrows pointing at the point of divergence between the blue and red SDF during the model period). We wondered whether the early emergence of this population signal might be due to the fact that on this subset of trials monkeys could anticipate the location of the missing square with certainty on the basis of the model object alone. (Models with one additional square determined the position of the missing square unambiguously because only the one additional square could be removed to produce the copy object, and consequently the missing square could be located at only one position). To examine this possibility, we constructed neural population activity functions (Fig. 4c,d) using trials in which model objects included two squares in addition to the object frame (Fig. 1c). In this case, monkeys could not anticipate the location of the missing square on the basis of the model object alone, because either of the two additional squares present in the model could be removed to produce the copy. On these trials, we found that the emergence of the population signal coding the horizontal position (Fig. 4c) and the vertical position (Fig. 4d) of the missing square was delayed until after the copy object appeared (Fig. 4c,d; note black arrows pointing at the divergence between blue and red SDF during the copy period). Neural population activity therefore varied with the missing square both in space and in time — coding the horizontal and vertical position of the missing square at the point in time that the position of the missing square was fixed by model and copy objects.

Potential Relation of Neural Activity to Oculomotor Factors

It was not likely that neural activity during the incomplete copy period reflected motor planning for the upcoming response. The monkey pressed a single response key with its foot every trial and movement direction did not vary. To examine whether neural activity might reflect oculomotor intention, we tested the activity of 7a neurons on both the construction task and a delayed saccade task. Saccade targets were placed at locations that included those of the missing squares, and also the squares in the choice array (Fig. 5a). Under the hypothesis that neural activity during object construction reflected the oculocentric direction of a planned saccade, the same population of neurons should be active in the two tasks and exhibit the same spatial preference for missing squares and saccade targets. Instead we found that the object construction and delayed saccade tasks recruited largely distinct neural populations. Of the 444 neurons in the sample whose activity varied with the horizontal position of the missing square during object construction (Table 1, ‘Combined’), activity in only 83 neurons (19%) varied significantly with the horizontal position of the saccade target in the delayed saccade task. Likewise, of the 393 neurons in the sample whose activity varied with the vertical position of the missing square during object construction (Table 1, ‘Combined’), activity in only 56 neurons (14%) varied with the vertical position of the saccade target in the delayed saccade task. Therefore ∼84% of the population whose activity coded the location of the missing square along one or the other spatial axis during object construction failed to code the location of the saccade target along that axis in the context of the delayed saccade task. In the 139 neurons in which activity varied significantly with both the position of the missing square and the saccade target, we calculated two positional preference indices (see Materials and Methods) for each neuron recorded in the two tasks. The index quantified the degree to which each neuron preferred left versus right or high versus low positions. The position preferences of single neurons were weakly correlated between the saccade and construction tasks (r = 0.246, P = 0.003); but there was considerable scatter and the indices were not tightly aligned. Many neurons preferred saccade targets and missing squares at opposite horizontal or vertical positions (Fig. 5b; points in upper left and lower right quadrants).

We selected the subpopulation of neurons whose activity was most strongly related to the horizontal position of the missing square during object construction (Fig. 5c; the neurons included were those in which activity significantly related to the main effect of horizontal position at P ≤ 0.001). We then examined the activity of these neurons during the delayed saccade task, segregating trials into preferred and nonpreferred groups according to whether the saccade target was located at the same horizontal position each neuron preferred in the construction task or not. We found that whereas neural activity in this population clearly varied with the horizontal position of the missing square during the construction task (Fig. 5c), population activity did not vary with the horizontal position of the target in the delayed saccade task (Fig. 5d). The directional signal evident in the activity of this neural population when monkeys were building objects was largely absent when monkeys were planning saccades.

Whereas few neurons active during object construction were active during the saccade task, the saccade task in itself was effective in driving a largely distinct population of neurons in area 7a. For example, the activity of 7a neurons varied significantly (P ≤ 0.05) as a function of the horizontal position of the saccade target both during the delay period (266 neurons) and the saccade period (271 neurons) of the delayed saccade task. Population activity clearly differentiated the position of the saccade target during both task epochs (Fig. 5e,f; neurons included were those in which neural activity related significantly to the horizontal position of the target at P ≤ 0.001; 61 and 48 neurons during delay and saccade periods respectively). Neural activity during the delay period of the saccade task may have been related to the visual stimulation provided by the continuing presence of the saccade target, or to the monkey's directing attention toward the target. Neural activation during the saccade period was largely postsaccadic. Population activity peaked approximately 50 ms after the eyes entered the response window surrounding the saccade target.

During the construction task, neural activity may have alternatively related to a variation in eye position within the fixation window. To test this possibility, we assessed the distribution of sampled eye position within the fixation window as a function of the location of the missing square. We found that the average eye position within the fixation window varied by only 0.1° across trials with different missing square locations (Fig. 6). A negligible variation in firing rate would be predicted by a variation in eye position this small (Andersen et al., 1990b).

Figure 6.

Gaze position of the first monkey during the copy period of the construction task. Color contour density plots of sampled eye position are segregated into four panels as a function of the location of the missing square (square with inscribed ‘×’). Red indicates highest density, blue lowest density. Eye position within the fixation window does not vary appreciably with the location of the missing square.

Figure 6.

Gaze position of the first monkey during the copy period of the construction task. Color contour density plots of sampled eye position are segregated into four panels as a function of the location of the missing square (square with inscribed ‘×’). Red indicates highest density, blue lowest density. Eye position within the fixation window does not vary appreciably with the location of the missing square.

Neural Activity Predicts Performance in the Construction Task

We decoded neural population activity using linear discriminant analysis (LDA; see Materials and Methods) with fivefold cross-validation (Johnson and Wichern, 1998) to provide a measure of how well population activity represented the location of the missing square. The analysis classified trials to one of four categories corresponding to the four possible missing square locations in the experiment. The classification was based on the pattern of activity observed in a population of 7a neurons. The frequency with which the classification was correct indicated the strength of the prediction about the location of the missing square we could make on the basis of the firing rates of 7a neurons. The results of the trial classification procedure are shown in Figure 7. Trials were first segregated into four groups based on the known position of the missing square. Trials in each group were then classified to four categories, corresponding to the potential locations of the missing square. The heights of the four vertical columns in each copy object (Fig. 7ad) are proportional to the frequency that trials of each type were classified to each of the four corresponding categories in the analysis. In each case, the tallest column representing the category to which the majority of trials were classified corresponds to the true location of the missing square on each trial. Overall, 85% of trials were correctly classified on the basis of the pattern of population activity.

Figure 7.

Decoding neural population activity to recover the location of the missing square. Linear discriminant analysis (LDA) was used lo classify trials of neural population activity to one of four categories representing the four possible locations of the missing square in the copy object. The frequency with which trials were classified to the correct category provided a measure of the strength with which population activity coded the location of the missing square. The classification was based on the activity of 128 neurons from Monkey 1. Activity in these neurons varied significantly (P < 0.05) with the location of the missing square during a variant of the construction task in which model and probe objects were both presented at the center of the display. (A–D) The number of trials of population activity classified to each of the four categories is proportional to the relative heights of the columns at the four potential missing square locations in each object. (A) Classification of trials when the missing square was in the upper left location. LDA correctly classifies 87% of these trials to the correct category on the basis of the pattern of neural population activity. (B) Classification of trials when the missing square was in the upper right location. LDA classfies 87% of trials correctly. (C) Classification of trials when the missing square was in the lower left position — 88% of trials are correctly classified. (D) Classification of trials when the missing square was in the lower right position — 78% of trials are correctly classified. The location of the square missing from the copy object can be decoded from the pattern of activity in a population of area 7a neurons on a single trial with 85% accuracy on average.

Figure 7.

Decoding neural population activity to recover the location of the missing square. Linear discriminant analysis (LDA) was used lo classify trials of neural population activity to one of four categories representing the four possible locations of the missing square in the copy object. The frequency with which trials were classified to the correct category provided a measure of the strength with which population activity coded the location of the missing square. The classification was based on the activity of 128 neurons from Monkey 1. Activity in these neurons varied significantly (P < 0.05) with the location of the missing square during a variant of the construction task in which model and probe objects were both presented at the center of the display. (A–D) The number of trials of population activity classified to each of the four categories is proportional to the relative heights of the columns at the four potential missing square locations in each object. (A) Classification of trials when the missing square was in the upper left location. LDA correctly classifies 87% of these trials to the correct category on the basis of the pattern of neural population activity. (B) Classification of trials when the missing square was in the upper right location. LDA classfies 87% of trials correctly. (C) Classification of trials when the missing square was in the lower left position — 88% of trials are correctly classified. (D) Classification of trials when the missing square was in the lower right position — 78% of trials are correctly classified. The location of the square missing from the copy object can be decoded from the pattern of activity in a population of area 7a neurons on a single trial with 85% accuracy on average.

On error trials, the square missing from the copy object and the square added to the copy object were located at different positions. We identified four types of error trials on this basis: trials in which the missing square was on the left side of the object and the added square was on the right side (Fig. 8a), or vice versa (Fig. 8b), and trials in which the missing square was at the high position and the added square was at the low position (Fig. 8c), or vice versa (Fig. 8d). We classified each of these error trials to one of two categories corresponding to the position of the missing square or the added square on each error trial (see Materials and Methods). LDA classified most error trials to the category corresponding to the position of the added square and not the missing square (Fig. 8ad). This indicated that the population activity pattern predicted where the monkey was going to add a square when it committed an error in construction, establishing an association between neural activity and the animals' behavioral performance. The overall accuracy of classification performance on error trials (Fig. 8) was lower than that on correct trials (Fig. 7). This was likely to reflect the availability of fewer neurons contributing to the population on which the error trial classification was based.

Figure 8.

Neural population activity predicted errors in construction before they occurred. On error trials, the square that was missing from the copy object and the square the monkey added to the copy object were located at different positions. (AD) Four error trial types were defined by a particular combination of missing and added square locations. The majority of error trials of each type classified to the category corresponding to where monkeys added a square to the copy object, and not where the missing square was on that trial. (A) Classification of trials in which the missing square was on the left side of the copy, but the monkey added a square to the right side. Most trials were classified to the right category, anticipating the monkey's spatial error. (B) Classification of trials in which the missing square was on the right side of the copy, but the monkey added a square to the left side. Most trials were classified to the left category. (C) Classification of trials in which the missing square was at the high position in the copy but the monkey added a square at the low position. Most trials were classified to the low category. (D) Classification of trials in which the missing square was at the low position in the copy but the monkey added a square to the high position. Most trials were classified to the high position. In the case of each error trial type, the majority of trials were classified to the category corresponding to the erroneous addition, and consequently to the spatial choice made by the animal made during object construction.

Figure 8.

Neural population activity predicted errors in construction before they occurred. On error trials, the square that was missing from the copy object and the square the monkey added to the copy object were located at different positions. (AD) Four error trial types were defined by a particular combination of missing and added square locations. The majority of error trials of each type classified to the category corresponding to where monkeys added a square to the copy object, and not where the missing square was on that trial. (A) Classification of trials in which the missing square was on the left side of the copy, but the monkey added a square to the right side. Most trials were classified to the right category, anticipating the monkey's spatial error. (B) Classification of trials in which the missing square was on the right side of the copy, but the monkey added a square to the left side. Most trials were classified to the left category. (C) Classification of trials in which the missing square was at the high position in the copy but the monkey added a square at the low position. Most trials were classified to the low category. (D) Classification of trials in which the missing square was at the low position in the copy but the monkey added a square to the high position. Most trials were classified to the high position. In the case of each error trial type, the majority of trials were classified to the category corresponding to the erroneous addition, and consequently to the spatial choice made by the animal made during object construction.

Deployment of Covert Attention during Object Construction

To measure the deployment of covert spatial attention during the construction task, we employed a dual-task paradigm. The monkey performed a set of trials including two trial types. Most trials in the set were standard construction trials. A minority of trials in the set were probe trials (probe trials comprised either 15 or 25% of trials in each set). Probe trials, like standard construction trials, began with the sequential presentation of a model and copy object (the monkey was not cued as to which type, standard construction or probe trial, was in progress). Then, on probe trials, after a random interval following the appearance of the copy object, a probe stimulus (red square) was flashed for 50 ms at one of four locations next to the copy object. Probe positions varied relative to the location of the missing square as defined by the model and copy objects on that trial (probe position was expressed as degrees horizontal offset from the location of the missing square). Probe stimuli (Fig. 9) appeared either at the location of the missing square (position 0°), at the mirror opposite location in the copy object (position −3.3°), at the location of the correct choice square (position +3.3°) or at the position of the incorrect choice square (position −6.6°) on a given trial. Because probe trials were infrequent, the expectation was that spatial attention would be distributed as it was normally during construction task performance. The RT to detect the probe would then provide a means to measure this distribution of attention.

Figure 9.

To assess the deployment of covert attention during construction performance, we probed the locus of attention by flashing a visual stimulus for 50 ms which the monkey had to simply detect for reward in the midst of a minority of construction trials. The probe was presented at a random interval after the appearance of the copy object, and its location was varied relative to the location of the missing square. (The locations of probe stimuli are represented by light gray squares flanking the object frame in this figure, and are given in degrees offset from the missing square on a given trial.) Probes at position 0° were located at the position of the missing square; probes at 3.3° were located at the position of the correct choice square on the same side of the object as the missing square. Probes at negative offsets were located on the opposite side of the object. Reaction time was at a minimum when the probe stimulus was presented at the location of the missing square (position 0°); it was significantly faster (see text) when it appeared here than when it appeared at an equal eccentricity on the mirror opposite side in the copy object (position −3.3°; asterisks).

Figure 9.

To assess the deployment of covert attention during construction performance, we probed the locus of attention by flashing a visual stimulus for 50 ms which the monkey had to simply detect for reward in the midst of a minority of construction trials. The probe was presented at a random interval after the appearance of the copy object, and its location was varied relative to the location of the missing square. (The locations of probe stimuli are represented by light gray squares flanking the object frame in this figure, and are given in degrees offset from the missing square on a given trial.) Probes at position 0° were located at the position of the missing square; probes at 3.3° were located at the position of the correct choice square on the same side of the object as the missing square. Probes at negative offsets were located on the opposite side of the object. Reaction time was at a minimum when the probe stimulus was presented at the location of the missing square (position 0°); it was significantly faster (see text) when it appeared here than when it appeared at an equal eccentricity on the mirror opposite side in the copy object (position −3.3°; asterisks).

Mean reaction time to detect the probe varied significantly across the four probe locations [Fig. 9; one-way ANOVA; F(3,1163) = 3.03, P = 0.008]. RT was at a minimum when the probe appeared at the location of the missing square (position 0°), and was significantly less in this case than when the probe appeared at the mirror opposite location on the other side of the copy object [planned comparison between mean RT at positions 0° and −3.3°; asterisks in Fig. 9; t(1163) = −2.071, P = 0.039]. Probes at the missing square and mirror opposite locations were the same distance from the fixation target and shared the same retinal eccentricity. Consequently an influence of stimulus eccentricity on reaction time (Carrasco et al., 1995) could not account for this effect.

In Figure 9 trial sets in which probe trials were comparatively frequent (25% of trials) or infrequent (15% of trials) were pooled. The difference in RT to detect probes at positions 0° and −3.3° positions achieved significance only when all data were included. The size of the RT effect, particularly between adjacent probe positions, was small in relation to the variance in RT, perhaps because probes were closely spaced (∼3° apart), and all probes were within 6° of the location of the missing square.

We compared sets of trials with infrequent and frequent probe trials separately. The minimum in the RT function was located at the missing square in either case. We found that increasing the frequency of probe trials from 15% to 25% reduced the impact of the missing square location on RT [mean RT no longer varied significantly across probe location; F(3,506) = 1.50, P = 0.215]. Increasing probe trial frequency also reduced percent correct performance on the construction task slightly from 95% to 90% correct [t(1250) = −3.56, P < 0.001].

Representing Multiple Locations Simultaneously: Distinct Neural Populations Code Missing and Choice Squares during Construction

On standard construction trials, neural activity related to the location of the missing square may in fact have coded the location of the correct choice square the monkey was about to select, as these locations were never varied with respect to one another in the standard task. To resolve this ambiguity, we recorded the activity of 367 neurons during a mixed-choice construction task dissociating the location of the missing and choice squares (Fig. 10a; see Materials and Methods). On standard trials, the correct choice square and the missing square were both located on the same side of the copy object. On crossed trials, missing and choice squares were located on opposite sides of the copy object. The monkey was cued as to whether a standard or crossed trial was in progress by the color of the fixation target. We analyzed neural activity during the copy period of the mixed-choice task using a two-way ANCOVA. The two factors in this analysis were the horizontal position of the missing square, and the horizontal position of the correct choice square. Neural activity during the copy period preceded the appearance of the choice square array, so any relation between neural activity and the location of the choice square was anticipatory in nature. In this neural sample, the activity of 57 neurons was significantly related to the location of the missing square, whereas the activity of 30 neurons was significantly related to the location of the choice square (Fig. 10b; P ≤ 0.05). Thus neurons coding the missing square outnumbered those coding the choice square by ∼2 to 1. The activity of only three neurons was statistically related to both spatial factors in the analysis.

Figure 10.

Neural population activity on a mixed-choice construction task which decoupled the locations of missing and choice squares. (A) The spatial arrangement of missing and choice squares in the mixed-choice task. Two trial types were randomly interleaved, and which trial type was in progress was cued to the monkey by the color of the fixation target. A red fixation target indicated a standard construction trial in which the choice square would appear on the same side of the copy object as the missing square (A, left). A green fixation target indicated a crossed construction trial in which the choice square would appear on the opposite side of the copy object as the missing square (A, right). Vertical choice arrays were used. (B) The firing rates of 367 neurons during the copy period of the mixed-choice task was analyzed in a two-way ANCOVA in which the horizontal position of the missing square and the horizontal position of the choice square were fixed factors. The Venn diagram shows the results of this analysis. In 57 neurons, neural activity varied significantly (P ≤ 0.05) with the horizontal position of the missing square. In 30 neurons, activity varied significantly with the horizontal position of the choice square. The activity of only three neurons varied with both spatial factors. (C, D) Normalized average population activity of the 57 neurons in which firing rate varied significantly with the horizontal position of the missing square. Trials are segregated on the basis of the position of the missing square into preferred and nonpreferred groups according to the position preference of neurons on standard trials. Neurons maintain the same position preference on crossed trials (D) that they exhibited on standard trials (C), demonstrating that neural activity remained a function of the position of the missing square on both trial types, and a change in the anticipated position of the correct choice square had little effect. (E, F) Normalized average population activity of the 30 neurons in which firing rate related to the horizontal position of the choice square. This group of neurons switch their position preference defined with respect to the missing square on crossed trials (F; note that red and blue SDF are inverted) relative to standard trials (E). The switch in the position preference of these neurons between standard and crossed trials is consistent with coding the anticipated location of the choice square.

Figure 10.

Neural population activity on a mixed-choice construction task which decoupled the locations of missing and choice squares. (A) The spatial arrangement of missing and choice squares in the mixed-choice task. Two trial types were randomly interleaved, and which trial type was in progress was cued to the monkey by the color of the fixation target. A red fixation target indicated a standard construction trial in which the choice square would appear on the same side of the copy object as the missing square (A, left). A green fixation target indicated a crossed construction trial in which the choice square would appear on the opposite side of the copy object as the missing square (A, right). Vertical choice arrays were used. (B) The firing rates of 367 neurons during the copy period of the mixed-choice task was analyzed in a two-way ANCOVA in which the horizontal position of the missing square and the horizontal position of the choice square were fixed factors. The Venn diagram shows the results of this analysis. In 57 neurons, neural activity varied significantly (P ≤ 0.05) with the horizontal position of the missing square. In 30 neurons, activity varied significantly with the horizontal position of the choice square. The activity of only three neurons varied with both spatial factors. (C, D) Normalized average population activity of the 57 neurons in which firing rate varied significantly with the horizontal position of the missing square. Trials are segregated on the basis of the position of the missing square into preferred and nonpreferred groups according to the position preference of neurons on standard trials. Neurons maintain the same position preference on crossed trials (D) that they exhibited on standard trials (C), demonstrating that neural activity remained a function of the position of the missing square on both trial types, and a change in the anticipated position of the correct choice square had little effect. (E, F) Normalized average population activity of the 30 neurons in which firing rate related to the horizontal position of the choice square. This group of neurons switch their position preference defined with respect to the missing square on crossed trials (F; note that red and blue SDF are inverted) relative to standard trials (E). The switch in the position preference of these neurons between standard and crossed trials is consistent with coding the anticipated location of the choice square.

We segregated standard and crossed trials according to the position of the missing square, and defined the preferred position of each neuron on the basis of its activity on standard trials. The 57 neurons in which activity related significantly to the position of the missing square exhibited a consistent spatial preference across standard and crossed trials (Fig. 10c,d). Activity in these neurons continued to code the location of the missing square in spite of the fact that the choice array had switched from the same side of the copy object (Fig. 10c) to the opposite side of the copy object (Fig. 10d). Conversely, the 30 neurons in which activity related significantly to the position of the choice square switched their spatial preference from standard (Fig. 10e) to crossed trials (Fig. 10f; note that red and blue SDF are inverted). This was consistent with neural activity in this group of cells coding the anticipated position of the choice array as it switched from one side of the copy object to the other. Thus two physiologically distinct groups of 7a neurons were concurrently active during the copy period, one which coded the position of the missing square and one which coded the anticipated position of the choice square before the choice array appeared. Both locations were relevant to construction task performance.

Largely Different Populations of area 7a Neurons Code the Location of the Missing Square during Construction, and the Location of Attended Visual Stimuli

One important question in the present experiments is whether neurons in 7a are generically active whenever a particular spatial locus becomes behaviorally relevant and draws attention. We compared the activity of 7a neurons in monkey M2 on the construction task and on a peripheral attention task (Fig. 11a; see Materials and Methods). This task was similar to a classical attention task that has been previously used to study the influence of attention on neural activity in posterior parietal cortex (Bushnell et al., 1981; Colby et al., 1996). In the peripheral attention task, monkeys fixated a central spot target while a peripheral spot target was presented at one of four locations (Fig. 11a). Peripheral targets were located at positions relative to the fixation target that corresponded to the centers of the missing squares in the construction task (Fig. 11a, left). The monkey was required to press the response key when the peripheral target dimmed slightly (Fig. 11a, right). The time between target onset and target dimming was randomly selected between six intervals (400–1400 ms in 200 ms increments).

Figure 11.

Comparison of the activity of a sample of 246 neurons recorded during both the object construction and peripheral attention tasks. (A) Events in the peripheral attention task. Monkeys maintained their gaze on a fixation target while a spot target was presented at one of four peripheral locations corresponding to the positions of the missing square in the construction task (the relative locations of missing squares are indicated by the dotted outlines). After a random interval, the peripheral target dimmed slightly, and the monkey was rewarded for detecting this luminance change and pressing the response key. (B) Frequency distribution of reaction times relative to the dimming of the peripheral target at time 0. The distribution shows a clear peak centered at 325 ms. (C) Distribution of reaction times relative to the onset of the peripheral target at time 0. The peripheral target dimmed after one of six randomly selected intervals following target onset (indicated by the vertical arrows at 400, 600, 800, 1000, 1200 or 1400 ms). There is a peak in the reaction time distribution corresponding to each interval, delayed by ∼350 ms. The relatively tight coupling between the time the target changed luminance and the time the monkey pressed the response key provides evidence that the monkey attended to the target. (D) Venn diagram illustrating the numbers of neurons in which activity varied significantly as a function of the missing square or the peripheral target (or both). The activity of each neuron was analyzed during the copy period of the construction task and also the delay period of the attention task using two separate ANCOVAs. (The delay period of the attention task was the interval after the onset and before the dimming of the peripheral target). In each analysis, position (either of the missing square or the peripheral target) was coded as a single factor with four levels. The activity in 60 neurons in this sample related significantly to the location of the missing square during object construction, whereas the activity of 20 neurons related significantly to the location of the peripheral target during the attention task (P < 0.05). In six neurons, activity varied significantly with both spatial factors. (E, F) Averaged normalized spike density functions (SDF) illustrate the activity of the two largely distinct neural populations during the first 750 ms following the onset of the peripheral target in the attention task. (E) Normalized activity of the subpopulation of 60 neurons in which activity related significantly to the location of the missing square during object construction. Activity of these neurons on the attention task is shown. Trials of the attention task are segregated into preferred and nonpreferred groups according to the position of the attention target relative to the positional preference of each neuron on the construction task. Activity is nearly equivalent on preferred (blue SDF) and nonpreferred (red SDF) trials, indicating that neurons in this population did not differentiate the position of the peripheral target during the attention task. (F) Normalized activity of the subpopulation of 20 neurons in which activity related significantly to the location of the peripheral target in the attention task (activity on the attention task is shown). This smaller and largely distinct subpopulation of 7a neurons coded the location of the peripheral target (note difference between blue and red SDF).

Figure 11.

Comparison of the activity of a sample of 246 neurons recorded during both the object construction and peripheral attention tasks. (A) Events in the peripheral attention task. Monkeys maintained their gaze on a fixation target while a spot target was presented at one of four peripheral locations corresponding to the positions of the missing square in the construction task (the relative locations of missing squares are indicated by the dotted outlines). After a random interval, the peripheral target dimmed slightly, and the monkey was rewarded for detecting this luminance change and pressing the response key. (B) Frequency distribution of reaction times relative to the dimming of the peripheral target at time 0. The distribution shows a clear peak centered at 325 ms. (C) Distribution of reaction times relative to the onset of the peripheral target at time 0. The peripheral target dimmed after one of six randomly selected intervals following target onset (indicated by the vertical arrows at 400, 600, 800, 1000, 1200 or 1400 ms). There is a peak in the reaction time distribution corresponding to each interval, delayed by ∼350 ms. The relatively tight coupling between the time the target changed luminance and the time the monkey pressed the response key provides evidence that the monkey attended to the target. (D) Venn diagram illustrating the numbers of neurons in which activity varied significantly as a function of the missing square or the peripheral target (or both). The activity of each neuron was analyzed during the copy period of the construction task and also the delay period of the attention task using two separate ANCOVAs. (The delay period of the attention task was the interval after the onset and before the dimming of the peripheral target). In each analysis, position (either of the missing square or the peripheral target) was coded as a single factor with four levels. The activity in 60 neurons in this sample related significantly to the location of the missing square during object construction, whereas the activity of 20 neurons related significantly to the location of the peripheral target during the attention task (P < 0.05). In six neurons, activity varied significantly with both spatial factors. (E, F) Averaged normalized spike density functions (SDF) illustrate the activity of the two largely distinct neural populations during the first 750 ms following the onset of the peripheral target in the attention task. (E) Normalized activity of the subpopulation of 60 neurons in which activity related significantly to the location of the missing square during object construction. Activity of these neurons on the attention task is shown. Trials of the attention task are segregated into preferred and nonpreferred groups according to the position of the attention target relative to the positional preference of each neuron on the construction task. Activity is nearly equivalent on preferred (blue SDF) and nonpreferred (red SDF) trials, indicating that neurons in this population did not differentiate the position of the peripheral target during the attention task. (F) Normalized activity of the subpopulation of 20 neurons in which activity related significantly to the location of the peripheral target in the attention task (activity on the attention task is shown). This smaller and largely distinct subpopulation of 7a neurons coded the location of the peripheral target (note difference between blue and red SDF).

We examined the distribution of reaction times to depress the response key relative to the dimming of the peripheral target. If the monkey attended to the target, its motor response ought to be tightly coupled to the time that the target changed luminance. The frequency distribution of reaction time calculated relative to the time that the target dimmed was unimodal, with a relatively sharp peak at 325 ms (Fig. 11b). We then examined the frequency distribution of reaction time calculated relative to the onset of the peripheral target. This distribution of reaction time exhibited six peaks each corresponding to one of the 6 random intervals between target onset and target dimming (Fig. 11c; the times of target dimming are indicated by vertical arrows, peaks in the RT distribution are delayed by ∼350 ms). The coupling of the motor response to the dimming of the target is evidence that the monkey attended to the target in order to detect a change in its luminance.

Of the 246 neurons recorded during the performance of both the construction and peripheral attention tasks, the activity of 60 neurons was significantly related to the location of the missing square in the construction task, and the activity of 20 neurons was related to the location of the peripheral target in the attention task (Fig. 11d). These were largely distinct populations, and the activity of few neurons (six cells) related to the locations of both the missing square and the peripheral target. Thus three times as many neurons in the sample were engaged to code a spatial location relevant to object construction than to code the location of an attended visual stimulus.

We computed the normalized activity of these two populations of neurons during the performance of the peripheral attention task. The first population consisted of the 60 neurons in the sample whose activity coded the location of the missing square during object construction (Fig. 11d). In this population of cells, we segregated peripheral attention trials into preferred and nonpreferred groups according to whether or not the attention target was located at the position of the missing square preferred by each neuron on the construction task. This population did not exhibit a consistent positional preference for missing squares and attended visual stimuli — as indicated by the fact that population activity functions on preferred (blue SDF) and nonpreferred (red SDF) trials of the attention task largely overlap (Fig. 11e). This pattern is consistent with the result of the ANCOVA, indicating that the activity of few neurons (six) in this group coded the location of the attended stimulus. The second population of neurons we examined included the 20 neurons in which activity varied significantly with the position of the attended target in the attention task (Fig. 11d). In this population of cells, we segregated peripheral attention trials into preferred and nonpreferred groups on the basis of the location of the peripheral target associated with greatest activity on the attention task. The relation between activity in these cells and the location of the attended visual stimulus is indicated by the difference between population activity functions on preferred and nonpreferred trials (Fig. 11f).

Discussion

Damage to human parietal cortex can impair spatial cognitive function. For example, constructional apraxia (Kleist, 1934; Black and Strub, 1976; Villa et al., 1986; Ruessmann et al., 1988) is a syndrome in which patients lose the ability to accurately process and reproduce object structure — they lose, for example, the cognitive ability to accurately represent the spatial arrangement of the parts of an object. This is often tested clinically by presenting a model object to a patient and asking them to produce a copy of it, either by drawing the model or arranging a duplicate set of its parts. Patients produce copies that are internally disorganized — the parts are placed in disarray relative to one another. The implication is that loss of normal parietal function has impaired the brain's ability to analyze and reproduce object structure, although little is known regarding what spatial analytic operations may be supported by parietal cortex or how these operations may be mediated by parietal neurons. Our objective in the present experiment was to examine these questions by developing an object construction task that monkeys could perform, to enable concurrent recording of neural activity in posterior parietal cortex. By probing spatial representations at the neural level during object construction, we sought to gain information regarding the nature of spatial analytic operations that were taking place during object construction and how parietal neurons may be involved.

The object construction task we employed incorporated the basic model–copy design used in clinical tests of human constructional ability. Monkeys viewed a model object, consisting of a contiguous grouping of identical square elements. After a delay, a copy object appeared. The copy was an incomplete version of the model, identical except that it lacked a single square element, which we refer to as the missing square. Monkeys were required to replace this missing square by adding a new square at the same position to the incomplete copy object. If this was accurately done, the addition of the square produced a new object (the complete copy object) whose structure matched the structure of the preceding model. This defined successful construction.

Construction is a process of accruing parts. To add a new part to an object, at the simplest level the brain has to figure out where to place that part. From this perspective, object construction hinges on a spatial judgement, that is, where each new part should be placed. When trying to copy an object, this judgement takes a particular form. The brain must determine, or judge, exactly how an object under construction is incomplete, or in other words, where parts are missing within the object being built relative to the model object that one is trying to copy. At each stage in the process, as each part is added, this amounts to finding where the emerging copy object and the model configuration differ. Often, one configuration is visible (the copy that is being built) and one is stored in working memory (the model object). Even in the case where a model object is continuously available to view (as in most clinical tests of constructional ability), the model is typically out of view or in peripheral vision when a patient is examining their own workspace and deciding where to place the next part of the copy. Consequently, working memory may be integral to copy tasks in which subjects view models briefly to acquire spatial information, and then saccade to the workspace in order produce the copy. The difference between the visible copy and the remembered model defines where parts are missing in one relative to the other and where new parts should be added. Comparing the structure of the object that one is attempting to build to a central representation of a desired configuration, or blueprint, therefore appears to be a fundamental component of object construction. In the case where objects are physically assembled, the spatial comparison defined above may provide the target that the motor system is subsequently engaged to reach, namely the position in the object where the next part should be placed. The two processes, a targeting step dependent on a spatial analytic process assessing object structure and a motor implementation step involving physical movement, may be dissociable. The construction task we designed minimized motor planning and execution aspects of object construction, and was based on this premise. We hoped in this way to isolate patterns of neural activity related to a spatial analysis of object structure from neural activity coding the metrics of the movements that would normally implement the constructive operation.

By recording neural activity in parietal cortex during object construction, we found that parietal neurons represented the void within the copy object under construction, the place where the next part should go. Specifically, during the time that the copy object was visible, neural activity coded the location of the square that was missing from the copy relative to the immediately preceding model. This neural signal could provide the analysis-based targeting function necessary to direct construction. We found in fact that this neural signal predicted where monkeys were going to add squares to the copy object on both correct and error trials (Figs 7 and 8). The task was designed in such a way that the location of the missing square did not correlate with the direction of the movement required in order to add the square and complete the constructive operation.

From the perspective of prior parietal research, one interesting aspect of the neural signal we found during object construction was that neurons represented a position in empty space — a position that was devoid of a visual stimulus and was made significant by a cognitive operation. Prior work in area 7a in primates has characterized the involvement of this cortical area in spatial vision (Robinson et al., 1978; Motter and Mountcastle, 1981; Andersen and Mountcastle, 1983; Andersen et al., 1985, 1987, 1990b; Motter et al., 1987; Steinmetz et al., 1987; Snyder et al., 1998b) and spatial attention (Mountcastle et al., 1981, 1987; Steinmetz et al., 1994; Robinson et al., 1995; Steinmetz and Constantinidis, 1995; Constantinidis and Steinmetz, 2001a,b). In both roles, the activity of 7a neurons signaled the presence of a visual stimulus within their retinotopic visual receptive fields. In both roles, the intensity of neural activity evoked by the stimulus was found to be modulated by extraretinal factors, such as the position of the eyes or head (Andersen and Mountcastle, 1983; Andersen et al., 1985, 1990b; Snyder et al., 1998b) or the location or state of covert attention (Mountcastle et al., 1981, 1987; Steinmetz et al., 1994; Robinson et al., 1995; Steinmetz and Constantinidis, 1995; Constantinidis and Steinmetz, 2001a,b). In this prior work, the presence of a visual stimulus within the visual receptive field typically initiated neural activity in area 7a, and the location of the stimulus relative to a neurons retinotopic receptive field was a key spatial variable coded in neural activity. During object construction, the activity of 7a neurons was not initiated by the appearance of a visual stimulus within their receptive fields. There was no stimulus at the position that neurons were engaged to represent. Therefore neurons did not code the position of a visual stimulus. Instead, 7a neurons coded locations that were computed. The sensory independence demonstrated by this finding may have important implications for the involvement of parietal cortex in higher order spatial cognitive processing that is also likely to depend on the brain's ability to represent and manipulate spatial information apart from the particulars of concurrent sensory input.

Neurons Coded the Location of the Missing Square in the Construction Task

We found that the activity of a population of 7a neurons during the period that the copy object was visible varied systematically as a function of the location of the missing square and not as a function of the form of the copy object. This was evidence that neural activity coded a computed spatial datum and not the location of a visual stimulus. For example, neural activity during the time the copy object was visible clearly related to the location of the missing square across trials in which the physically identical copy object was presented (Figs 3 and 4a,b). The visual stimulation provided by the copy object was constant. The fact that neural activity continued to vary with the missing square in this case indicated that neural activity was not a function strictly of where light fell on the retina relative to the visual receptive fields of 7a cells. Further evidence that this was the case was provided by the fact that neural activation generalized across diverse pairs of model and copy objects that together connoted a common missing square location (Fig. 3).

One alternative interpretation of these data is that neural activity was due to the modulation of visual receptive field sensitivity by eye position, as shown in prior work (Andersen et al., 1985). However, gaze fixation was required during the construction task and mean eye position within the fixation window varied by a small fraction of a degree as a function of the location of the missing square (Fig. 6). Another alternative interpretation of these data might be that neural activity was related to the reactivation of a visual receptive field stimulated earlier in the trial, or to spatial working memory of the retinal location of a previously visible stimulus. We found, however, that the neural population continued to code the location of the missing square on trials in which the retinal position of the model was shifted relative to the copy object (compare Table 2 to Table 1). On these trials, no prior stimulus appeared at the retinal location of the missing square in the copy object, thus reactivation of a visual receptive field or spatial working memory were unlikely to account for the activity we observed.

Neurons whose activity varied with the location of the missing square were prevalent in area 7a. The activity of 43% of all area 7a neurons encountered that met a minimal spontaneous activity requirement (see Materials and Methods) varied significantly with either the horizontal or vertical position of the missing square (Table 1). One possible account for the prevalence of this signal in parietal cortex is that spatial representations in area 7a conform to the processing demands of tasks monkeys have been trained to perform. This possibility is generally consistent with the finding that parietal neurons represent different types of information as a function of task context. For example, when the color of a target is made significant in a saccade task, neurons in LIP acquire color sensitivity which they do not otherwise exhibit (Toth and Assad, 2002).

Potential Relation of Neural Activity to the Oculocentric Direction of a Planned Saccade

Other interpretations of the present data more closely related to the known sensory and motor properties of parietal neurons should be considered. One of these is that neural activity during the object construction task reflected motor planning or intention (Bracewell et al., 1996; Mazzoni et al., 1996; Snyder et al., 1997, 1998a). Specifically, neural activity may have reflected a motor plan that was never physically executed to move in a direction that was coupled to the location of the missing square. It is important to consider the potential relation to motor planning, particularly with regard to the question framed above as to whether cortical systems always represent spatial information that is defined in relation to the receptive fields or movement fields of the participating neurons. By design, movement direction did not vary in the construction task — monkeys controlled the constructive operation by varying the timing and not the direction of motor output. For that reason, neural activity coding the location of the missing square did not reflect a neural representation of the direction of the movement which the monkey planned and executed for reward while performing the task.

The monkey may have planned a movement it never made, however, and neural activity in area 7a may have related to this motor plan. We examined this question in the context of saccade planning, as various subdivisions of the inferior parietal cortex have been implicated in the cortical control of eye movements (Gnadt and Andersen, 1988; Andersen, 1989; Barash et al., 1991; Bracewell et al., 1996). We compared the activity of the same individual neurons on the construction task and on a delayed saccade task. In the saccade task, monkeys planned saccades toward spot targets that were placed at the same spatial positions where missing squares were located during the construction task (we also placed saccade targets at the positions of the choice squares). We focused our analyses on the subpopulation of neurons in which activity varied significantly with the location of the missing square during construction, and found that in the large majority of these neurons, activity did not vary significantly with the location of the target in the delayed saccade task. Populations of neurons in area 7a were indeed active during the delay and saccade epochs of the saccade task (Fig. 5e,f); however, these populations were largely distinct from the population of 7a neurons that coded spatial information relevant to the performance of the object construction task. We found that in a minority of neurons, activity varied significantly with the location of the missing square and also the location of the saccade target. However, the spatial preferences of these neurons in the two tasks were not tightly aligned (Fig. 5bd). In a different behavioral context, we have similarly observed that parietal neurons coding task-relevant spatial information in spatial cognitive and saccade tasks do not necessarily code the same spatial coordinate in each of them (Crowe et al., 2004). Together, these data suggest that individual parietal neurons can participate in diverse spatial representations and that the spatial information they code in each may be to a degree independent. The failure of 7a neurons coding the location of the missing square to activate during the saccade task is evidence their activity did not represent an oculocentric saccade plan.

This interpretation is generally consistent with prior evidence that area 7a does not play a primary role in saccade planning and control. The activity of some area 7a neurons varies with the direction of saccades, but in most of these cases the onset of neural activity follows the initiation of the saccade (Barash et al., 1991), and so does not participate in planning the eye movement. Additional evidence that 7a does not play a direct role in saccade control has been provided by the results of microstimulation experiments; saccades are not evoked by the injection of electrical currents into area 7a that readily evoke saccades in neighboring area LIP (Thier and Andersen, 1998). Finally, the anatomical connectivity of area 7a does not suggest an immediate role in oculomotor control, at least in contrast to other cortical eye fields. For example, projections between area 7a and the superior colliculus or the frontal eye fields are weak or absent (Lynch et al., 1985; Andersen et al., 1990a).

The above findings suggest that neural activity in area 7a during the construction task did not represent the oculocentric direction of a saccade that monkeys planned but never executed (we consider the possibility that neurons were engaged to plan a saccade in object-centered coordinates below). Nonetheless, it seems likely that neural activity in 7a during object construction played a role in response planning, at least in a more abstract sense. For example, the construction task required monkeys to make a forced choice between alternative choice squares in order to determine the final configuration of the constructed object. This choice was based on the spatial relation between the alternative choice squares and the missing square. Consequently a neural representation of the location of the missing square may have been an important component of the mechanism by which that selection among alternative choice squares was made. Which choice square was selected in turn determined when the motor response occurred in relation to the choice illumination sequence, and ultimately the form of the object that was built. In this sense, neural activity may have served to guide behavior without coding the endpoint of a movement. If so, the early emergence of this signal (Fig. 4a,b), long in advance of the motor response, may be thought to reflect a form of response planning, but not necessarily in the simplest sense of coding a movement vector.

Potential Relation of Neural Activity to Object-centered Representations of Space

The primary finding in this study was that parietal neurons represented a spatial locus defined by a cognitive process during object construction. Neurons generally were not active to code the same spatial locus when defined by direct visual input, namely by placing a stimulus at the preferred location to serve either as the endpoint of a saccade (Fig. 5) or as the focus of attention (Fig. 11). This demonstrates that neural activity during construction was not related to the retinotopic or oculocentric coding of a spatial location when attention was directed to a visual target or a saccade was planned to a visual target. However, the differential activity of neurons between these contexts, object construction versus the control tasks, is an observation with at least two alternative interpretations because the construction and control tasks differed along at least two dimensions simultaneously. First, spatial cognitive processes involved in building objects were required in the construction task and were not required in the control tasks. Second, visual objects were displayed in the construction task and were not displayed in the control tasks.

The physical presence of a visual object may have influenced neural activity independently of cognitive operations involved in building objects. For example, Olson and colleagues have shown that neurons in the supplementary eye fields (SEF) code location in object-centered coordinates (Olson and Gettner, 1995, 1999; Olson and Tremblay, 2000; Tremblay et al., 2002). Importantly, SEF neurons exhibit object-centered spatial selectivity as long as movements are directed to the part of an object, even when object-centered spatial judgements are not required (Olson and Gettner, 1995; Tremblay et al., 2002). By analogy, 7a neurons in the present study may have failed to consistently activate on the control tasks because the control tasks involved behaviors directed toward spot targets (Figs 5 and 11). Visual objects were not present in the control tasks and consequently monkeys could not form object-centered spatial representations. The population of 7a neurons we studied may therefore have been engaged if the control tasks had required planning saccades or directing attention to parts of objects, in which case their activity would be more generally related to object-centered spatial coding, engaged for multiple behaviors in which this form of spatial representation was required.

It remains an open possibility that the neurons we studied would be active in tasks other than object construction, that support the formation of object-centered spatial representations. Determining the degree to which 7a neurons are exclusively specialized for construction versus other object-based tasks is an important question that will require further study. However, the present data establish the involvement of area 7a neurons in a specific spatial judgement relevant to object construction in a relatively simple and straightforward sense. The location of the missing square itself constituted a constructional judgement. The neural representation of this particular spatial datum establishes a link between a spatial analytic process assessing the structure of the objects presented in the construction task and neural activity in parietal cortex. This link is further substantiated by the finding that the location coded by neural activity predicts behavior. Neurons code the position where monkeys will place the new square within the object before the fact on correct and error trials (Figs 7 and 8). In this sense, the spatial representation of 7a neurons predicts the form of the object that is built. Thus neurons appear to provide the analysis-driven targeting function that might normally inform the motor system as to where to place the next part in an object during assembly. This targeting mechanism may be engaged in other contexts, such as when directing a saccade or attention to a part of an object. In representing specifically the location of the missing square in the context of object construction, however, the spatial targeting mechanism evident in the activity of 7a neurons displays a form of spatial intelligence suited to solving the spatial problem of building objects. The fact that neurons represented the location of the missing square also demonstrates that 7a neurons can code space independently of the retinal locations of visual targets, a fact that was further supported by the finding that the same neurons were not activated when a visual target was placed at spatially congruent locations.

Available evidence suggests that the influence of constructional judgements and object-centered spatial coding are not mutually exclusive, and that the two factors jointly determine spatial representation in parietal cortex. For example, preliminary evidence indicates that 7a neurons code locations in an object-centered framework during construction (Chafee et al., 2003). Consequently, the presence of a visual object influences the spatial representation of neurons in area 7a, as it does in the SEF. These data favor the view that constructional judgements and object-centered spatial coding are simultaneously influential factors whose intersection shapes neural activity in parietal cortex during object construction.

Relation of Neural Activity to Spatial Attention

The area of greatest overlap between the present data and the results of previous research in parietal cortex appears to relate to the established correlation between parietal function and spatial attention. We found evidence that spatial attention was covertly directed toward the location of the missing square during object construction (Fig. 9), the same location represented by the majority of 7a neurons that were active during the copy period (Fig. 10c,d). This deployment of attention was top-down in the construction task, attention moved to a point in empty space defined by a cognitive operation.

A correlation between the state or location of attention and neural activity in parietal cortex has been extensively documented (Mountcastle et al., 1981, 1987; Steinmetz et al., 1994; Robinson et al., 1995; Gottlieb et al., 1998; Kusunoki et al., 2000; Constantinidis and Steinmetz, 2001a,b; Bisley and Goldberg, 2003). For example, in area LIP, neurons preferentially code the locations of salient stimuli (Gottlieb et al., 1998; Kusunoki et al., 2000), and covert attention is found at the location represented by neural activity elicited by saccade targets or visual distractors during a delayed saccade task (Bisley and Goldberg, 2003). Likewise, prior studies in area 7a have shown that, as in area LIP, neurons preferentially represent the locations of salient visual stimuli (Constantinidis and Steinmetz, 2001b). These studies have also shown that attention has a suppressive effect on the magnitude of neural activity evoked by the appearance of a visual stimulus within the retinotopic receptive fields of 7a neurons (Steinmetz et al., 1994; Robinson et al., 1995; Constantinidis and Steinmetz, 2001a). This finding has been interpreted as indicating that 7a neurons are active in relation to the movement of attention (Steinmetz and Constantinidis, 1995), a hypothesis that is generally consistent with the positive correlation between response magnitude and stimulus salience (as attention often moves toward salient stimuli).

The suppressive effect of attention on visual sensitivity, which has been well documented in area 7a (Steinmetz et al., 1994; Robinson et al., 1995; Constantinidis and Steinmetz, 2001a), appears to be a correlate of attention that is distinct form the one we have found in the same cortical area during object construction. For example, during construction, attention was generally correlated with an increase and not a decrease in the firing rate of neurons coding the attended locus (Figs 3, 4 and 10). An additional distinction was that attention during construction did not manifest as a variable gain on the visual sensitivity of parietal neurons, because parietal neurons became active without a visual stimulus appearing in their receptive fields. It may therefore be that the neural activity we observed during object construction was more closely related to the previously reported correlation between neural activity in area 7a and behavioral salience (Constantinidis and Steinmetz, 2001b), except that in the present case, neural activity correlated with the behavioral salience of an abstract spatial coordinate and not the salience of a visual stimulus.

Parietal Activity and Abstract Spatial Processing

The present data indicate that abstract spatial information logically related to the solution of a spatial problem is present in the activity of neurons in area 7a. In this regard, our results are generally consistent with prior evidence that parietal neurons participate in neural representations of abstract information. For example, LIP neurons continue to code the inferred direction of motion of a moving stimulus while the stimulus itself is occluded from view (Eskandar and Assad, 1999). This effect cannot be mediated by the stimulation of visual receptive fields during the period that the coded stimulus is invisible. Other studies have shown that the activity of neurons in the intraparietal sulcus systematically relates to the number of items in a visual display regardless of their location (Nieder and Miller, 2004), and that neural activity in LIP represents elapsed time in a manner that parallels time perception (Leon and Shadlen, 2003). It is difficult to account for these findings solely in terms of the activation of the visual receptive or movement fields of parietal neurons. The present data suggest that, like the neural representation of numerosity, visual motion, or time, the neural representation of space in parietal cortex can likewise be abstracted from sensory or motor processing. In this regard the present results share some features with other neural data we have recently reported in area 7a during the performance of a visual maze task (Crowe et al., 2004). In that study, neural activity was systematically tuned to the direction of a path through a maze that a monkey was required to solve in the absence of eye movements. In both the maze and construction tasks we found evidence that the activity of area 7a neurons coded an abstract spatial parameter that was relevant to the spatial problem at hand.

In the present experiment, we have observed this form of task-tailored, abstract spatial representation in area 7a in a behavioral context involving the top-down control of spatial attention. Top-down attention is strategically deployed by humans to satisfy cognitive task constraints in natural visuomotor tasks (Shinoda et al., 2001; Triesch et al., 2003), including a block copy task (Hayhoe et al., 1998) that is directly analogous to the object construction task we have presently employed. The present data indicate that attention is strategically deployed during object construction in the nonhuman primate as well, and that this deployment is correlated with the activity of neurons in parietal area 7a. These findings may provide some insight into why damage to parietal cortex in humans can produce constructional apraxia (Kleist, 1934; Black and Strub, 1976; Villa et al., 1986; Ruessmann et al., 1988). Without the benefit of normal parietal function patients may lose the ability to deploy top-down attention strategically and this may interfere with constructional performance. We found in the present experiments that attention and constructional performance were both correlates of neural activity in parietal cortex.

Correlation versus Causality in Object Construction, the Scope and Limit of Interpretation Supported by the Neural Data

Though nearly ubiquitous in studies of this type, the distinction between correlation and causality places important limits on how the current results should be interpreted. Specifically, the present data do not establish that 7a mediates the spatial cognitive processing required to build objects. Instead, they document a correlation between neural activity and spatial cognitive function during object construction. Establishing a causal link, as opposed to a correlation, between the function of neurons and a behavioral or cognitive act requires the convergence of data from neural recording, lesion and microstimulation experiments at a minimum (the last two of which we have not conducted). Given a simplifying assumption (namely that neural firing rate is acceptable shorthand for information processing in the brain), the difference between correlation and causality is one essentially of anatomical scope. If the pattern and evolution of neural activity throughout the brain as a whole is considered causal with respect to the spatial cognitive processing that enables object construction, the question then reduces to what part of this functionally integrated but anatomically distributed processing takes place within area 7a.

It is nearly certain that the neural and cognitive operations required to construct objects are mediated by a distributed cortical system (Mountcastle, 1978, 1995) of which parietal cortex is only a part. This distributed system includes projections linking posterior parietal and dorsolateral prefrontal cortex directly (Cavada and Goldman-Rakic, 1989; Chafee and Goldman-Rakic, 1998, 2000), but also involves a parallel network of projections linking parietal and prefrontal cortex to numerous other cortical and sub-cortical targets in common (Selemon and Goldman-Rakic, 1988). The division of computational load among the nodes of this integrated system during complex spatial behaviors such as object construction is not yet known. Neurons in area 7a may serve as a passive spatial register storing the results of spatial computations performed by other nodes in this more broadly distributed cortical system. Alternatively, neurons in area 7a may participate in spatial computation directly, which in this case amounts to controlling the evolution of neural activity in the system in such a manner that the coordinate of the missing square is reliably computed and encoded. Differentiating between these possibilities is generally beyond the scope of single neuron recording experiments.

Granting the above limit, the present results illustrate that the representational capability of parietal neurons extends to abstract spatial variables computed by cognitive operations. This capability was not immediately apparent from previous work in which parietal neurons coded a spatial coordinate that most often coincided with the position of a visual stimulus. The present results also show that during object construction, parietal neurons represent a spatial datum that captures the difference between a model object and its copy, and predicts the position where parts are placed as the copy is built. Importantly, the coded spatial datum does not correlate with the direction of the motor response planned and executed during the task. This suggests that parietal cortex is part of a network providing a spatial targeting function during object construction that is based on a spatial analysis of object structure, and that selects positions within objects. Although the targeting mechanism may normally instruct the motor system, it appears to be at least partially independent in the present context, as points in space are selected during construction that do not coincide with the direction of the forthcoming movement.

We thank D. Lee, H. Merchant, T. Naselaris, G. Pellizzer and L. Romanski for providing constructive criticism and comments regarding this work. We thank D. Boeff for electrical engineering and computer systems assistance, and D. Evans for surgical and animal care assistance. Supported by USPHS grant NS17413, the United States Department of Veterans Affairs, and the American Legion Brain Sciences Chair.

References

Andersen RA (
1989
) Visual and eye movement functions of the posterior parietal cortex.
Annu Rev Neurosci
 
12
:
377
–403.
Andersen RA, Mountcastle VB (
1983
) The influence of the angle of gaze upon the excitability of the light-sensitive neurons of the posterior parietal cortex.
J Neurosci
 
3
:
532
–548.
Andersen RA, Essick GK, Siegel RM (
1985
) Encoding of spatial location by posterior parietal neurons.
Science
 
230
:
456
–458.
Andersen RA, Essick GK, Siegel RM (
1987
) Neurons of area 7 activated by both visual stimuli and oculomotor behavior.
Exp Br Res
 
67
:
316
–322.
Andersen RA, Asanuma C, Essick G, Siegel RM (
1990
a) Corticocortical connections of anatomically and physiologically defined subdivisions within the inferior parietal lobule.
J Comp Neurol
 
296
:
65
–113.
Andersen RA, Bracewell RM, Barash S, Gnadt JW, Fogassi L (
1990
b) Eye position effects on visual, memory, and saccade-related activity in areas LIP and 7a of macaque.
J Neurosci
 
10
:
1176
–1196.
Arena R, Gainotti G (
1978
) Constructional apraxia and visuoperceptive disabilities in relation to laterality of cerebral lesions.
Cortex
 
14
:
463
–473.
Averbeck BB, Crowe DA, Chafee MV, Georgopoulos AP (
2003
) Neural activity in prefrontal cortex during copying geometrical shapes. II. Decoding shape segments from neural ensembles.
Exp Brain Res
 
150
:
142
–153.
Barash S, Bracewell RM, Fogassi L, Gnadt JW, Andersen RA (
1991
) Saccade-related activity in the lateral intraparietal area. I. Temporal properties; comparison with area 7a.
J Neurophysiol
 
66
:
1095
–1108.
Benson DF, Barton MI (
1970
) Disturbances in constructional ability.
Cortex
 
6
:
19
–46.
Bisley JW, Goldberg ME (
2003
) Neuronal activity in the lateral intraparietal area and spatial attention.
Science
 
299
:
81
–86.
Black FW, Strub RL (
1976
) Constructional apraxia in patients with discrete missile wounds of the brain.
Cortex
 
12
:
212
–220.
Bracewell RM, Mazzoni P, Barash S, Andersen RA (
1996
) Motor intention activity in the macaque's lateral intraparietal area. II. Changes of motor plan.
J Neurophysiol
 
76
:
1457
–1464.
Bushnell MC, Goldberg ME, Robinson DL (
1981
) Behavioral enhancement of visual responses in monkey cerebral cortex. I. Modulation in posterior parietal cortex related to selective visual attention.
J Neurophysiol
 
46
:
755
–772.
Carlesimo GA, Fadda L, Caltagirone C (
1993
) Basic mechanisms of constructional apraxia in unilateral brain-damaged patients: role of visuo-perceptual and executive disorders.
J Clin Exp Neuropsychol
 
15
:
342
–358.
Carrasco M, Evert DL, Chang I, Katz SM (
1995
) The eccentricity effect: target eccentricity affects performance on conjunction searches.
Percept Psychophys
 
57
:
1241
–1261.
Cavada C, Goldman-Rakic PS (
1989
) Posterior parietal cortex in rhesus monkey. II. Evidence for segregated corticocortical networks linking sensory and limbic areas with the frontal lobe.
J Comp Neurol
 
287
:
422
–445.
Chafee MV, Goldman-Rakic PS (
1998
) Matching patterns of activity in primate prefrontal area 8a and parietal area 7ip neurons during a spatial working memory task.
J Neurophysiol
 
79
:
2919
–2940.
Chafee MV, Goldman-Rakic PS (
2000
) Inactivation of parietal and prefrontal cortex reveals interdependence of neural activity during memory-guided saccades.
J Neurophysiol
 
83
:
1550
–1566.
Chafee MV, Crowe DA, Averbeck BB, Georgopoulos AP (
2003
) Parietal neurons encode object-centered space in a construction task. Soc Neurosci Abstr 658.4.
Colby CL, Duhamel JR, Goldberg ME (
1996
) Visual, presaccadic, and cognitive activation of single neurons in monkey lateral intraparietal area.
J Neurophysiol
 
76
:
2841
–2852.
Constantinidis C, Steinmetz MA (
2001
a) Neuronal responses in area 7a to multiple stimulus displays. II. Responses are suppressed at the cued location.
Cereb Cortex
 
11
:
592
–597.
Constantinidis C, Steinmetz MA (
2001
b) Neuronal responses in area 7a to multiple-stimulus displays. I. Neurons encode the location of the salient stimulus.
Cereb Cortex
 
11
:
581
–591.
Crowe DA, Chafee MV, Averbeck BB, Georgopoulos AP (
2004
) Neural activity in primate parietal area 7a related to spatial analysis of visual mazes.
Cereb Cortex
 
14
:
23
–34.
Driver J, Baylis GC, Goodrich SJ, Rafal RD (
1994
) Axis-based neglect of visual shapes.
Neuropsychologia
 
32
:
1353
–1365.
Eskandar EN, Assad JA (
1999
) Dissociation of visual, motor and predictive signals in parietal cortex during visual guidance.
Nat Neurosci
 
2
:
88
–93.
Gnadt JW, Andersen RA (
1988
) Memory related motor planning activity in posterior parietal cortex of macaque.
Exp Brain Res
 
70
:
216
–220.
Gottlieb JP, Kusunoki M, Goldberg ME (
1998
) The representation of visual salience in monkey parietal cortex.
Nature
 
391
:
481
–484.
Hayhoe MM, Bensinger DG, Ballard DH (
1998
) Task constraints in visual working memory.
Vision Res
 
38
:
125
–137.
Howell DC (
1997
) Statistical methods for psychology. Belmont, CA: Wadsworth Publishing Company, Inc.
Hyvarinen J (
1981
) Regional distribution of functions in parietal association area 7 of the monkey.
Brain Res
 
206
:
287
–303.
Johnson RA, Wichern DW (
1998
) Applied Multivariate Statistical Analysis. Saddle River, NJ: Prentice Hall.
Kleinbaum DG, Kupper LL, Muller KE (
1988
) Applied regression analysis and other multivariable methods. Belmont, CA: Wadsworth Publishing Company, Inc.
Kleist K (
1934
) Konstruktive (optische) Apraxie. Leipzig: Barth.
Kusunoki M, Gottlieb J, Goldberg ME (
2000
) The lateral intraparietal area as a salience map: the representation of abrupt onset, stimulus motion, and task relevance.
Vision Res
 
40
:
1459
–1468.
Leon MI, Shadlen MN (
2003
) Representation of time by neurons in the posterior parietal cortex of the macaque.
Neuron
 
38
:
317
–327.
Lynch JC, Graybiel AM, Lobeck LJ (
1985
) The differential projection of two cytoarchitectonic subregions of the inferior parietal lobule of macaque upon the deep layers of the superior colliculus.
J Comp Neurol
 
235
:
241
–254.
Mazzoni P, Bracewell RM, Barash S, Andersen RA (
1996
) Motor intention activity in the macaque's lateral intraparietal area. I. Dissociation of motor plan from sensory memory.
J Neurophysiol
 
76
:
1439
–1456.
Motter BC, Mountcastle VB (
1981
) The functional properties of the light-sensitive neurons of the posterior parietal cortex studied in waking monkeys: foveal sparing and opponent vector organization.
J Neurosci
 
1
:
3
–26.
Motter BC, Steinmetz MA, Duffy CJ, Mountcastle VB (
1987
) Functional properties of parietal visual neurons: mechanisms of directionality along a single axis.
J Neurosci
 
7
:
154
–176.
Mountcastle VB (
1978
) An organizing principle for cerebral function: the unit module and the distributed system. In: The Mindful Brain (Edelman GE, Mountcastle VB, eds.), pp. 7–50. Cambridge, MA: The MIT Press.
Mountcastle VB (
1995
) The evolution of ideas concerning the function of the neocortex.
Cereb Cortex
 
5
:
289
–295.
Mountcastle VB, Lynch JC, Georgopoulos A, Sakata H, Acuna C (
1975
) Posterior parietal association cortex of the monkey: command functions for operations within extrapersonal space.
J Neurophysiol
 
38
:
871
–908.
Mountcastle VB, Andersen RA, Motter BC (
1981
) The influence of attentive fixation upon the excitability of the light-sensitive neurons of the posterior parietal cortex.
J Neurosci
 
1
:
1218
–1225.
Mountcastle VB, Motter BC, Steinmetz MA, Sestokas AK (
1987
) Common and differential effects of attentive fixation on the excitability of parietal and prestriate (V4) cortical visual neurons in the macaque monkey.
J Neurosci
 
7
:
2239
–2255.
Nieder A, Miller EK (
2004
) A parieto-frontal network for visual numerical information in the monkey.
Proc Natl Acad Sci USA
 
101
:
7457
–7462.
Olson CR, Gettner SN (
1995
) Object-centered direction selectivity in the macaque supplementary eye field.
Science
 
269
:
985
–988.
Olson CR, Gettner SN (
1999
) Macaque SEF neurons encode object-centered directions of eye movements regardless of the visual attributes of instructional cues.
J Neurophysiol
 
81
:
2340
–2346.
Olson CR, Tremblay L (
2000
) Macaque supplementary eye field neurons encode object-centered locations relative to both continuous and discontinuous objects.
J Neurophysiol
 
83
:
2392
–2411.
Platt ML, Glimcher PW (
1999
) Neural correlates of decision variables in parietal cortex.
Nature
 
400
:
233
–238.
Robinson DL, Goldberg ME, Stanton GB (
1978
) Parietal association cortex in the primate: sensory mechanisms and behavioral modulations.
J Neurophysiol
 
41
:
910
–932.
Robinson DL, Bowman EM, Kertzman C (
1995
) Covert orienting of attention in macaques. II. Contributions of parietal cortex.
J Neurophysiol
 
74
:
698
–712.
Ruessmann K, Sondag HD, Beneicke U (
1988
) On the cerebral localization of constructional apraxia.
Int J Neurosci
 
42
:
59
–62.
Selemon LD, Goldman-Rakic PS (
1988
) Common cortical and subcortical targets of the dorsolateral prefrontal and posterior parietal cortices in the rhesus monkey: evidence for a distributed neural network subserving spatially guided behavior.
J Neurosci
 
8
:
4049
–4068.
Shadlen MN, Newsome WT (
2001
) Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey.
J Neurophysiol
 
86
:
1916
–1936.
Shinoda H, Hayhoe MM, Shrivastava A (
2001
) What controls attention in natural environments?
Vision Res
 
41
:
3535
–3545.
Snyder LH, Batista AP, Andersen RA (
1997
) Coding of intention in the posterior parietal cortex.
Nature
 
386
:
167
–170.
Snyder LH, Batista AP, Andersen RA (
1998
a) Change in motor plan, without a change in the spatial locus of attention, modulates activity in posterior parietal cortex.
J Neurophysiol
 
79
:
2814
–2819.
Snyder LH, Grieve KL, Brotchie P, Andersen RA (
1998
b) Separate body- and world-referenced representations of visual space in parietal cortex.
Nature
 
394
:
887
–891.
Steinmetz MA, Constantinidis C (
1995
) Neurophysiological evidence for a role of posterior parietal cortex in redirecting visual attention.
Cereb Cortex
 
5
:
448
–456.
Steinmetz MA, Motter BC, Duffy CJ, Mountcastle VB (
1987
) Functional properties of parietal visual neurons: radial organization of directionalities within the visual field.
J Neurosci
 
7
:
177
–191.
Steinmetz MA, Connor CE, Constantinidis C, McLaughlin JR (
1994
) Covert attention suppresses neuronal responses in area 7a of the posterior parietal cortex.
J Neurophysiol
 
72
:
1020
–1023.
Sugrue LP, Corrado GS, Newsome WT (
2004
) Matching behavior and the representation of value in the parietal cortex.
Science
 
304
:
1782
–1787.
Thier P, Andersen RA (
1998
) Electrical microstimulation distinguishes distinct saccade-related areas in the posterior parietal cortex.
J Neurophysiol
 
80
:
1713
–1735.
Toth LJ, Assad JA (
2002
) Dynamic coding of behaviourally relevant stimuli in parietal cortex.
Nature
 
415
:
165
–168.
Tremblay L, Gettner SN, Olson CR (
2002
) Neurons with object-centered spatial selectivity in macaque SEF: do they represent locations or rules?
J Neurophysiol
 
87
:
333
–350.
Triesch J, Ballard DH, Hayhoe MM, Sullivan BT (
2003
) What you see is what you need.
J Vis
 
3
:
86
–94.
Villa G, Gainotti G, De Bonis C (
1986
) Constructive disabilities in focal brain-damaged patients. Influence of hemispheric side, locus of lesion and coexistent mental deterioration.
Neuropsychologia
 
24
:
497
–510.

Author notes

1Brain Sciences Center, Veterans Affairs Medical Center, Minneapolis, MN 55417, USA, 2Department of Neuroscience, University of Minnesota, Minneapolis, MN 55455, USA, 3Department of Neurology, University of Minnesota, Minneapolis, MN 55455, USA and 4Center for Cognitive Sciences, University of Minnesota, Minneapolis, MN 55455, USA