Although some change in the neural representation of an object must occur as it becomes familiar, the nature of this change is not fully understood. In humans, it has been shown that the N170—an evoked visual potential—is enhanced for classes of objects for which people have visual expertise. In this study, we explored whether monkeys show a similar modulation in event-related potential (ERP) amplitude as a result of long-term familiarity by recording ERPs with chronically implanted electrodes over extended training periods spanning many sessions. In each of 3 experiments, we found larger amplitude visual evoked responses to highly familiar images for the time period of 120–250 ms after stimulus onset. This difference was found when the monkeys were trained in an individual-level discrimination task, in a task that required only color discrimination, and even following a viewing-only task. We thus observed this familiarity effect across several tasks and different object categories and further found that the difference between “familiar” and “novel” became smaller as the animals gained experience with the previously unfamiliar objects across multiple test sessions. These data suggest that changes in visual responses associated with familiarity are evident early in the evoked visual response, are robust, and may be automatic, driven at least in part by repeated object exposure.
Some form of previous experience is, by definition, critical for recognition, but it is not clear how or if this experience is evident in neurophysiological signals recorded from the brain. The ability to recognize familiar objects or individuals seen before provides obvious evolutionary advantage. At the same time, objects that are frequently encountered, such as conspecifics or particular food items, have a high likelihood of being important because these items are most likely to guide action on a regular basis. In this paper, we consider how object familiarity and the task conditions leading to familiarity affect the brain's evoked neural response to known objects.
Many recent studies of visual event-related potentials (ERPs) have identified a response component 160–200 ms after stimulus onset that is sensitive to stimulus category (Jeffreys and others 1992; Carmel and Bentin 2002; Rossion, Curran, and Gauthier 2002; Itier and Taylor 2004). This component, generally called the N170, is enhanced for faces compared with other objects, and individuals with deficits in face recognition show a decreased neural response for faces at the level of this component (Bentin and others 1999; Eimer and McCarthey 1999). This finding has been characterized by some as evidence of face specificity of the N170 (Bentin and others 1996). Others, however, have proposed that the N170 shows enhanced response to categories of objects for which the subjects possess some level of visual expertise. In support of the latter proposal, Tanaka and Curran (2001) reported an enhanced N170 in dog and car experts for objects within their domain of expertise. They found no such enhancement when car experts were tested with dogs or vice versa. As additional evidence, Rossion, Gauthier, and others (2002) tested human subjects using both faces and a novel category of objects with which subjects had extensive stimulus training (Greebles). They used both upright and inverted versions of the stimuli. Rossion and others found that inversion of faces and greebles affected the N170 similarly for greeble experts, but only face inversion had a demonstrable effect on the N170 in greeble novices. These data indicate that inversion disrupts the N170 for classes of objects for which people are experts but not for classes of objects for which people are not experts.
Additional studies have shown interference between faces and stimuli within categories of expertise, as measured by the N170 (Gauthier and others 2003; Rossion and others 2004; Gauthier and Curby 2005). Gauthier and others (2003) tested car experts and found modulation of the N170 when cars were interleaved with faces in a 2-back working memory task. They did not find modulation in car novices. In addition, Rossion and others (2004) showed that training with greebles resulted in similar interference between the greebles and faces for the N170. Taken together, these data suggest that the N170 is modified by object classes other than faces and that a partially overlapping or shared substrate may underlie the N170 for faces and objects of other highly trained categories.
Relatively few studies have explored similar neural signals in monkeys. Pineda and others used ERPs to examine the role of familiarity in face recognition in monkeys (Pineda and Nava 1993; Pineda and others 1994). Pineda and others (1994) presented squirrel monkeys with images of both monkey and human faces. The stimuli were divided into familiar faces (monkeys and humans with whom the subjects had frequent contact) and unfamiliar faces (monkeys and humans with whom the subjects had no prior contact). They found that familiar monkey faces resulted in increased ERP amplitude 60–200 ms after stimulus onset and decreased amplitude 250–600 ms after stimulus onset compared with unfamiliar monkey faces. There were no significant differences for human faces. Although these results suggest that stimulus familiarity can affect neural responses, they do not directly probe the conditions and progression of the effects of stimulus familiarity on ERPs. In that study, familiarity was based solely on social interactions and not on experimental training, so the relative degree of familiarity that subjects perceived through photographs of their conspecifics and the humans was unknown. In addition, because only faces of human and nonhuman primates were tested, it is not clear whether effects of familiarity at the level of ERPs might only be observed for face stimuli in monkeys.
In contrast to the relative paucity of monkey visual ERP studies, numerous investigators have recorded from single neurons within various areas of extrastriate cortex to test the effects of experience on neural signals. Hölscher and others (2003), for example, investigated the effects of long-term visual experience on single-cell responses in the perirhinal cortex of monkeys. They trained 3 rhesus macaque monkeys to perform a delayed match-to-sample task with trials consisting of either all familiar or all novel object images. Hölscher and others found that perirhinal neurons responded more actively to familiar compared with novel objects and found some evidence for a gradual buildup of responsiveness to a set of stimuli over long periods, for example, 400 repetitions over 1–2 weeks. Rainer and others (2004) tested for learning-dependent changes in V4 neurons. They found that V4 cell responses carried more information about the identity of familiar compared with novel images when the images were degraded in noise. This suggests that as we gain experience with objects, the neural representation becomes more robust to suboptimal viewing conditions. Booth and Rolls (1998) examined the effects of real-world experience on responses of inferotemporal (IT) cells. They allowed the monkeys to play with a set of real objects to engender familiarity and then recorded from single cells using static images of the familiar as well as novel objects. They found a small set of neurons that responded to multiple views of the same familiar object. In contrast, they found no such neurons for novel objects. This result demonstrates the role that experience plays in the development of representations of real-world objects. Several other studies suggest that IT cells become tuned to the features or combinations of features that are important for recognizing trained images (Baker and others 2002; Sigala and Logothetis 2002; Sigala 2004). We note that not all studies have found increased activity for well-known objects. Freedman and others (2005), for example, found that overall responses of IT neurons were actually somewhat reduced for familiar stimuli, compared with novel stimuli, although the responses to familiar stimuli were significantly more selective.
Together, these studies provide evidence that extensive experience with a set of objects changes the underlying activation of individual neurons in numerous areas of extrastriate cortex. It is unclear, however, what the overall pattern of response across many neurons and brain areas might look like and whether the results would be similar to findings for the N170 in humans.
In this study, we distinguish between the effects of long-term object familiarity and those due to short-term repetition. In the typical repetition suppression study, stimulus adaptation is measured over relatively short intervals, such as within a single trial or a single training session (Miller and others 1991; Riches and others 1991; Fahy and others 1993; Li and others 1993; Xiang and Brown 1998). In contrast, in the present study, object familiarity is measured over the course of several days or even weeks. As the relationship between repetition suppression and long-term familiarity is still unclear, we have focused here only on the latter.
We extend previous findings of effects of long-term visual experience in several important ways. First, we recorded from sites along ventral visual areas thought to be homologous to the electrode sites used for analysis in human studies of the N170 (Tanaka and Curran 2001; Rossion, Gauthier, and others 2002), so that we could begin to bridge the gap between human and monkey studies. In addition, we controlled the amount of experience the monkeys received with the trained and untrained stimuli within the context of the experiment, rather than relying on knowledge about a preexisting category such as faces. To do this, we measured ERPs to both overlearned (400–600 repetitions) and novel exemplars. We also used chronically implanted electrodes, which allowed us to study changes in neural response over the course of training. Finally, we used 3 different behavioral tasks to examine how different training paradigms affect changes in visual evoked responses.
In this experiment, we show that ERP amplitude reflects the level of experience for specific exemplars within a category of learned objects. We trained 2 monkeys to discriminate between individuals within a category of objects (birds). Once the monkeys had completed a large number of trials with an initial set of exemplars, we added new objects from the same category and compared evoked responses between the highly familiar and newly introduced stimuli. We then tracked these responses over multiple sessions.
The subjects were 2 adult male rhesus macaque monkeys (Macaca mulatta), weighing 9–13 kg. Prior to the experiment, the monkeys were familiarized with sitting in a primate chair and button pressing for juice reward. Both had participated in unrelated behavioral studies.
The monkeys were surgically implanted with a single piece titanium head restraint post and an array of 12 electrodes. The electrodes were attached to the skull using 1.5-mm titanium screws (Bioplate Inc., Los Angeles, CA, http://www.bioplate.com) placed as shown in Figure 1. The placement of the electrodes was chosen to cover sites in monkey cerebral cortex thought to be homologous to the human regions known to show the most pronounced N170 enhancement for faces in normal human subjects (Rossion, Gauthier, and others 2002). Insulated wire leads were attached to the screws by titanium washers and connected to a multipin connector (Omnetics Connector Corporation, Minneapolis, MN, http://www.omnetics.com) that was anchored to the skull within a titanium chamber. All animal surgeries were performed under aseptic conditions using isoflurane anesthesia and were approved by the Institutional Animal Care and Use Committee at Brown University and carried out in accordance with the guidelines published in the National Institutes of Health (NIH) Guide for the Care and Use of Laboratory Animals (NIH publication no. 86-23, revised 1987).
The stimuli consisted of 16 color photos of birds (see Fig. 2A) obtained from the Hemera Photo Objects Premium Image Collections I and II (Hemera Technologies Corporation, Seattle, WA). In addition, the animals were shown 16 color photos of objects with which they previously had significant training (a minimum of 400 trials with these objects preceding the beginning of this experiment, the specific objects differed for each monkey). The stimuli subtended 6 degrees of visual angle along their largest dimension.
The animals were tested in experimental setups consisting of a separate animal testing room and adjoining experimenters' workstations. Each setup contained a graphics stimulator running an OpenGL-based display program, a control console, a local area network of 4 computers running a real-time operating system (QNX) for experimental control, infrared video monitors, audio amplifiers for sound generation, microphones, and eye-tracking hardware. Experimental control and data collection of behavioral measures were conducted using custom-written software, providing deterministic control and acquisition of button responses and eye position. All behavioral data, such as button responses and eye position signals, were available for online control and stored to disk for offline analysis. Each animal testing room was electrically shielded and sound isolated. Eye movements were recorded using an ISCAN RK-726PCI video eye-tracking system (ISCAN, Inc., Burlington, MA).
Individual electroencephalography (EEG) signals were amplified by a factor of 10 000 using a Grass model 15RXi amplifier (Astro-Med Inc., West Warwick, RI) with an analog band-pass filter of 0.3–300 Hz. The reference electrode was placed on the head restraint post, which was attached directly to the anterior half of the skull. Evoked potentials were recorded at a sampling rate of 2500 Hz for 100 ms prior to trial onset and for 500 ms following the animal's response.
The stimuli were presented on a computer monitor positioned 120 cm from the monkey. The monkeys initiated trials by fixating for 450 ms on a blue fixation spot that subtended 0.3 degrees of visual angle presented in the center of the monitor. The fixation spot was removed from view once the fixation requirement was met; the stimulus was shown 200 ms later. The stimulus remained on until the monkey made a button response or until 5000 ms had elapsed. The monkeys were given juice reinforcement for correct responses (i.e., choosing the button associated with a particular object). The monkeys were allowed to examine the stimuli freely during each experimental trial; however, we limited all our data analyses to those trials during which the monkeys' gaze remained within 3 degrees of the center of the screen (half the stimulus dimension) for the first 300 ms following stimulus onset. The intertrial interval was 1000 ms.
During the training phase, the monkeys were shown a total of 16 objects, 8 birds and 8 previously learned objects. Responses were divided evenly between the 2 response buttons, so that 4 birds were assigned to the right button and 4 were assigned to the left. The monkeys were shown 4 repetitions of each stimulus during a block, resulting in 64 trials per block. The stimuli were randomized within each block. Thus, immediate repeats of stimuli were possible. This phase lasted for several days, until the monkeys had seen each bird between 400–600 times.
Following the training phase, the monkeys began the novel exemplar phase. During this phase of the experiment, an additional 8 objects from each category were added to those shown during the training phase. Each block contained 4 repetitions of each of the 16 birds (8 familiar and 8 novel) and the 16 previously trained objects, yielding a total of 128 trials per block. The stimuli were randomized within each block. Monkey S completed 23 daily sessions in the novel exemplar phase, and monkey T completed 19 daily sessions.
Differences in accuracy and reaction times between the familiar and novel exemplars were assessed by obtaining average proportion correct scores and mean reaction times for each of the individual images used (8 familiar and 8 novel). The set of familiar and novel measures were then compared by a t-test for each session.
We isolated a period of 120–250 ms after stimulus onset for analysis of the evoked potential. This time period was selected for a number of reasons. First, this time period corresponds approximately to the time scale of the N170 in humans (Carmel and Bentin 2002; Rossion, Curran, and Gauthier 2002; Itier and Taylor 2004). We started our analysis window at 120 ms because this is a time when cells in monkey IT cortex show evidence of clear stimulus selectivity. In this way, we could focus on differences related to image identity and not those that might reflect low-level perceptual differences between novel and familiar objects. We cut off the analysis at 250 ms so as to bind the analysis epoch by a time that consistently preceded the monkeys' manual responses, which occurred as early as 300 ms.
We used independent component analysis (ICA) in an attempt to extract and isolate elements of the evoked potentials reflecting differences in object familiarity. Each independent component's activation time course corresponded to the activity of some hypothetical source, and its projection at an electrode corresponded to the contribution of that source to the signal observed at that electrode. The signal from each electrode can result from a combination of line noise, artifacts from eye movements and muscles of the scalp and jaw, and traces of some number of neural processes. Each such source is detected by each electrode to a different degree, depending on location. To the extent that these sources are stably localized across the spatial field of electrodes, and vary in activation independently of each other, they can be viewed as independent components. Given that their activities combine sufficiently linearly and with negligible delays at each electrode, ICA is a useful tool for the analysis of evoked potentials (for a more extensive discussion on the applicability of ICA to analysis of evoked potentials, see Makeig and others 1996). For this analysis, we low-pass filtered and downsampled the recorded signal to 500 Hz and used the ICA toolbox (http://www.cnl.salk.edu/∼jung/ica.html) to generate one mixing matrix for each monkey's data. Each matrix was generated with 1000 trials randomly selected from all sessions of all experiments in which that monkey participated. For each animal, we used 9 of the 12 channels, yielding 9 independent components; 3 channels with possible connection problems were excluded from the analysis, as signals from these often saturated the amplifiers and the A/D conversions.
We used permutation tests to discover the most appropriate component and to statistically analyze the ERP difference between the novel and familiar objects. For each component and each electrode, we calculated the area between the average projection in novel trials and the average projection in familiar trials, within the analysis period of 120–250 ms after stimulus onset. We also calculated this area measure for 100 random reassignments of the trials between the 2 categories. The component with the largest difference between the actual area measure and the average area measure of the permutations was selected for additional analysis, along with the channel to which it most strongly projected. We ran a second test using 9000 permutations and considered the area measure significant at P < 0.05 if, in the resulting distribution, it fell in the 95th percentile divided by a factor of 9 (i.e., corrected for the number of components). Days with few trials tended to have some noise in the average curves; this noise increased the raw area measure for those days. To alleviate this artificial inflation, we divided each day's area measure by the average area measure of the 9000 permutations to obtain a normalized area measure.
Results and Discussion
By the end of the training phase, both monkeys were nearly perfect at classifying their learned set of birds. In the subsequent novel exemplar phase, monkey S saw each of the bird stimuli 460 times over the course of 23 daily sessions; monkey T saw 385 stimulus repetitions over the course of 19 daily sessions. By the second session of the novel stimulus phase (see Fig. 3A,B), monkey S showed no significant difference for accuracy (t14 = 1.68, P > 0.05) or response times (t14 = −1.94, P > 0.05) for the old birds compared with the new birds. As shown in Figure 3C, monkey T showed no significant accuracy differences between the original and novel birds by the second session (t14 = 1.86, P > 0.05). The difference between monkey T's response times for novel and original birds was, however, significantly different for several sessions during novel exemplar training, although these differences decreased with training (Fig. 3D). The reaction time differences between the 2 animals may have resulted from fact that monkey S had had more prior training in transfer tasks than had monkey T. In any case, similar patterns of ERP response changes with experience would indicate that even if the specific learning strategies adopted by the monkeys differ, the underlying encoding process may be similar.
Single channel ERP data showed that in the first session of the novel exemplar phase, the evoked response to the original trained objects was significantly larger than that for the novel objects (Fig. 4, session 1). By session 18, this difference was greatly reduced. We used ICA to simultaneously analyze data from all the electrodes, not just those with the strongest visual evoked potential (Comon 1994; Bell and Sejnowski 1995), to focus our analysis on one component for each monkey—that component of the EEG signal related most directly to the familiarity of the visual stimuli. The selected component was projected onto the channel at which it was strongest, reconstructing part of the signal observed at that channel (projections onto other channels yield scalar multiples of this signal). We then examined the area, from 120 to 250 ms after stimulus onset, between the average curves for the familiar and novel trials (normalized as described above).
Based on this quantitative analysis, we found significantly larger response amplitudes between 120 ms and 250 ms after stimulus onset for familiar stimuli compared with new exemplars in the first session of training for both monkeys (Fig. 5). This same familiarity effect was significant on most days of training with the new exemplars, even when there were no significant behavioral differences. This difference in the neural signal had diminished considerably by the final session of testing in this experiment (see session 23 of Fig. 5A and session 19 of Fig. 5B), and the general trend was a steady reduction in the difference between familiar and novel stimuli over the course of the experiment.
In this first experiment, we thus found a significantly larger ERP signal for more-familiar stimuli (as measured by number of repetitions). Despite the random assignment of birds to the familiar and novel groups, these images were quite complex and differed perceptually in numerous ways. Therefore, one possible explanation for the neural signal difference is that over the sets of familiar and novel stimuli there were random low-level differences (e.g., slight differences in local contrast or area), which could have resulted in different levels of neural activation. However, if the response difference we observed was based only on low-level stimulus differences between the original and novel birds, we would expect this difference to remain constant throughout the novel exemplar phase. As is clear from Figure 5, this was not the case. We found a systematic decrease in the differences between the familiar and novel sets over time. Thus, it is unlikely that the results reported were due to image related differences between the original and novel exemplars.
Another possible explanation for the results of Experiment 1 is that the difference in the neural signal reflects differences in response planning or execution. When the novel stimuli were introduced, both monkeys were performing well over 90% correct for the familiar stimuli. For each of them, the behavioral differences between the novel and original birds were significantly different in the first session (see Fig. 3), as it took time to associate the correct response with each new stimulus. Thus, the difference observed in the evoked potentials that was initially large but decreased over time may reflect this learning process, rather than image familiarity per se. To address this issue, we designed a task in which response selection would be easily transferred to a novel set of images, to determine what role the learning of new associations played in the distinction between well-known and novel stimuli.
In Experiment 1, we found a significant difference in ERPs for familiar and unfamiliar exemplars of a single object category. In that experiment, the monkeys learned to individuate specific birds in order to solve the task, and learning the novel exemplars required several repetitions over multiple blocks. Based on those results, it is unclear whether the observed higher ERP amplitude for familiar stimuli was the result of the monkeys' active discrimination among items in an explicit recognition task or, another, more automatic process. In the study of Hölscher and others (2003), in which overlearned objects elicited more activity than novel ones, the monkeys were also required to closely attend to the object identities in order to solve the task. We wondered if a less demanding discrimination task would eliminate the effects of specific exemplar familiarity. If the familiarity effect was due to a more automatic encoding process, we would expect that the level of task difficulty would not be as critical as stimulus exposure. Using a simple classification rule that generalized to novel stimuli, we could test whether the differences between novel and familiar exemplars were due to the confidence of the response or to stimulus familiarity. Finally, using a new category of objects, we could assess the overall robustness of this experience-dependent effect. For these reasons, in Experiment 2, we trained a monkey in a simple discrimination task with a new category of objects (teapots) whose class boundary (left or right button press) was based on color. We then tested for the familiarity effect by introducing new teapots that could immediately be categorized by applying the same classification rule.
The subject was monkey S from Experiment 1.
The stimuli consisted of photos of 24 teapots obtained from Hemera Photo Objects Premium Image Collections I and II (Hemera Technologies Corporation) and from the World Wide Web. We used Adobe Photoshop 7.0 (Adobe Systems, Inc., San Jose, CA) to change the color of the teapots so that 12 were blue and 12 were purple (examples shown Fig. 2B).
The apparatus was identical to Experiment 1.
Individual trials proceeded as in Experiment 1. During the training phase, the monkey was shown 16 teapots. The responses were divided between the 2 response buttons based on color, so that the 8 blue teapots were assigned to the right button and the 8 purple teapots were assigned to the left button. The monkey was shown 4 repetitions of each stimulus during a block, resulting in 64 trials per block. The monkey received approximately 620 repetitions of each stimulus during the initial training phase. Immediately following training, the monkey was shown 8 new teapots in the novel exemplar phase. Blocks consisted of 4 repetitions of each of the 24 teapots (16 familiar and 8 novel) to yield a total of 96 trials in each block. During both training and the novel exemplar phase, the stimuli were completely randomized within each block. In the novel exemplar phase, the monkey received approximately 390 trials with each exemplar over the course of 15 sessions.
Results and Discussion
During the initial training, monkey S achieved over 95% accuracy. Following this training, the novel teapots were introduced into the original set. In several sessions we did observe a significant accuracy difference between the novel and familiar teapots (Fig. 6A; sessions 1, 5, 7, 8, & 12, t14 = 2.25, 3.76, 2.52, 2.68, 2.19 P < 0.05), but the effect was extremely small (∼1% to 7%). Response time results showed a significant difference over 4 of the first 5 sessions (t14 = −4.74, −2.53, −3.63, −2.40; P < 0.05), which was no longer significant by session 6 (see Fig. 6B; t14 = 0.27, P > 0.05). As expected, accuracy for the novel exemplars was much closer to performance for the familiar exemplars for the first block in this experiment (90% and 100% correct, respectively), compared with the first block in Experiment 1 (43% and 100% correct, respectively). A similar pattern was found for response times: differences were much greater in session 1 of Experiment 1 (figure 3B) than in session 1 of Experiment 2 (figure 6B). Despite this improved performance compared to Experiment 1, it must be noted that there were significant differences between novel and familiar stimuli in several sessions, suggesting the monkey was attending to the identities.
As in Experiment 1, we used ICA to isolate a component that accounted for a large part of the difference between familiar and novel ERPs and considered projections of this component for our quantitative analysis. Again we found significantly larger response amplitude for familiar stimuli compared with novel stimuli on the first day of training. By session 15, this difference remained but was greatly reduced. Figure 7 shows the normalized area difference for each daily session of the experiment. This figure shows that the general trend was a gradual decrease in the difference between familiar and novel over the course of the experiment, although this difference was significant throughout the experiment.
In Experiment 2, we used a simple color discrimination task to determine whether a less demanding task might still produce a significant difference in the evoked potentials. Still, we found a significant difference between novel and familiar exemplars, indicating that even when the monkey is not required to attend to anything more than an object's color, the responses to familiar items still differ from those to novel ones. This is indicated by both the behavioral data and the ERP data. Similar to Experiment 1, we show that the magnitude of the ERP response was “larger” for familiar objects than for novel objects. In addition, the monkey showed smaller (though sometimes significant) performance differences between novel and familiar stimuli compared to Experiment 1 (see Fig. 6). However, we found ERP differences very similar to those reported in Experiment 1 (see Fig. 7). This result suggests that the difference in the neural signal between novel and familiar objects does not reflect a decisional or response bias. Finally, we report similar results in Experiments 1 and 2 using different sets of objects and slightly different tasks, indicating that these findings are robust across stimulus and task changes.
In Experiment 3, we further explored whether the enhanced neural response for familiar exemplars is related to the overt response by training a monkey in a passive-viewing task. The monkey was only required to fixate a small spot, which was then removed. A visual stimulus was presented, followed by the spot reappearing in a new location, which the monkey then refixated for a juice reward. Thus, in contrast to Experiments 1 and 2, the monkey was not required to make any response to the visual stimulus and could, in principle, ignore the stimulus entirely while performing the task. A significant difference between familiar and novel stimuli in this passive-viewing task would indicate that this familiarity effect does not require an overt recognition response. This result would also suggest that the process through which familiarity affects the neural representation of objects occurs automatically for objects that are seen repeatedly and not just for those associated with specific responses.
The subject was monkey S from Experiment 1.
The stimuli consisted of 24 individual pocket watches obtained from Hemera Photo Objects Premium Image Collections I and II (Hemera Technologies Corporation). The colors of the stimuli were adjusted using Adobe Photoshop 7.0 (Adobe Systems, Inc.) so that all the objects were one of 2 colors, 12 gold and 12 red (see Fig. 2).
The apparatus was identical to Experiment 1.
In Experiment 3, the monkey was trained to perform a fixation task, which did not require him to make a response to the stimulus. The monkey initiated trials by fixating on a yellow fixation spot that subtended 0.3 degrees of visual angle. The fixation spot was then removed, and the monitor was blanked for 200 ms; the stimulus was then presented for 600 ms. A second fixation spot was presented 250 ms later and was presented in a randomly selected location 6 degrees above, below, to the right, or to the left of the center. The monkey was required to fixate the second spot to receive juice reinforcement. As in the first 2 experiments, we discarded trials in which the monkey did not maintain the focus of his gaze within 3 degrees of stimulus center for the first 300 ms of stimulus presentation.
During the training phase, the monkey was shown 16 pocket watches (8 gold and 8 red) in the fixation task but was not required (or allowed) to make a response to them. Training blocks consisted of 4 repetitions of each of 16 objects for a total of 64 trials. In the training phase, the monkey had approximately 400 repetitions of each stimulus prior to the introduction of the novel exemplars. During the novel exemplar testing, 8 new exemplars of the pocket watches were added to the original set. These new stimuli were also divided by color. The 24 stimuli (16 familiar and 8 novel) were shown 4 times each for a total of 96 trials in each block. During both training and the novel exemplar phase, the stimuli were randomized within each block. The monkey saw the novel and familiar exemplars approximately 340 times over the course of 20 daily sessions.
Results and Discussion
We analyzed the ERP data using ICA and permutation tests, as in the previous experiments. On the first day of the novel exemplar phase, we once again found a large difference between the familiar and unfamiliar exemplars in the 120- to 250-ms analysis epochs. By the final day of the experiment (session 20, approximately 340 trials), this difference was greatly reduced. Figure 8 shows the normalized area measure for each daily session of the experiment; the difference between familiar and novel items tended to decrease over the 20 sessions of the experiment.
In Experiment 3, we found larger amplitude responses for familiar compared with novel stimuli using a third category of objects. For these images, the monkey was never required to respond explicitly to the stimuli. That we still found a difference in the ERP signal between familiar and novel stimuli shows that the familiarity effect we report here does not depend on explicit classification training and suggests that changes in neural responses for familiar stimuli is at least in part a result of repeated exposure. In addition, because there was no explicit response, the difference between familiar and unfamiliar objects cannot be attributed to the overt nature of the manual tasks used in Experiments 1 and 2. This type of familiarization by repeated exposure is similar to our experience with many objects encountered on a daily basis. Regardless of whether or not we interact with these objects on any particular occasion, their frequency of occurrence would be expected to change their neural representation, tuning the visual system to match the statistical regularities present in the environment.
In the present study, we found significantly larger ERP signals for familiar exemplars compared with novel exemplars when tested over the course of hundreds of stimulus repetitions. The finding of specific enhancement for highly familiar stimuli was robust across individual-level and color discriminations and even resulted following extensive passive viewing. We also demonstrated that over the course of several days and weeks of training with the “novel” stimuli, this difference diminishes. This phenomenon was shown for 2 monkeys in Experiment 1 and in 3 separate experiments for a single monkey (monkey S). For monkey S, the 3 experiments occurred sequentially over the course of several months. Thus, the diminished difference observed following extensive experience cannot be attributed to changes in the interface between the electrodes and the signal source with passage of time.
Although we chose to use perceptual categories of common objects to test category-specific familiarity, these categories were unlikely to be familiar to the monkeys prior to training. Indeed, because these objects were sometimes shown intermingled with other objects (Experiment 1), we cannot assume that the monkeys automatically categorized the test items as human observers would (e.g., collections of birds or teapots), especially given the small number of exemplars trained for each category. However, there is strong evidence that nonhuman animals do indeed form perceptual categories similar to humans (Wasserman and others 1988) and that training with just a few category exemplars (e.g., 12) leads to significant generalization to novel exemplars (Bhatt and others 1988). Thus, we expect that the monkeys perceived the visual similarity among the exemplars belonging to the same class, even though the categories were likely not familiar.
Despite training for several weeks, the difference between the evoked potentials for familiar and novel objects did not entirely vanish (see Figs 5, 7, and 8). We have found that several “months” later, even with additional experience with the set of novel exemplars, the difference between the first (familiar) and second (novel) sets of stimulus exemplars to still exist. There are several possible explanations for the persistence of this effect. One explanation is that the specific training method might be responsible. Specifically, during novel exemplar testing, the monkeys are given an equal number of new and old exemplars within each training block. It is important to remember that prior to the novel exemplar phase, the monkeys had already viewed the original exemplars 400–600 times. Thus, if the total number of repetitions of each exemplar is measured, the monkeys “always” experienced a larger number of repetitions of the original exemplars; the persistence in the familiarity difference may be directly related to the number of repetitions. If the repetitions were made equal (by showing the novel exemplars more times than the original exemplars during the novel exemplar phase), the difference between familiar and novel might disappear. Another possibility is that the first exemplars learned in a category may be given special status. This phenomenon would be similar to age of acquisition effects reported for written word recognition (Morrison and Ellis 1995). These experiments show that those words learned first are recognized more quickly than those learned later, even when word frequency is controlled. Additional experiments will be necessary to determine which explanation better accounts for the familiarity effects we report here.
Multiple Component Analysis
Throughout this study, we used one independent component for our analyses. Projections of the selected component were strong, relatively free from noise, and accounted for a large portion of the difference between familiar and novel trials. Still, there were other components whose activation time courses depended on stimulus familiarity. By projecting all these components together, we can reconstruct a larger portion of the actual ERP signal observed while still excluding signals (and noise) that carried no information about stimulus familiarity. Figure 9 shows the daily normalized difference measures for monkey S derived from all 6 components that showed a significant difference between conditions on the first day of any of the experiments. The decreasing difference is once again obvious, and the inclusion of most of the recorded signal makes comparisons between experiments more meaningful. The single-component analysis yields a very clean signal and excludes as much as possible the activity not related to familiarity; the multicomponent analysis confirms that no critical information was discarded or obscured by excluding the rest of the components. This figure also allows a comparison of all 3 experiments on a single graph. It is evident that in each experiment, there is a systematic decrease in the response difference between the original and novel exemplars, as the novel stimuli become more familiar. It is also important to note that these changes take place extremely slowly, here occurring over a 4- to 6-week period.
Although we limited our analyses to trials during which the animals' gaze was focused on the visual stimulus, small saccadic eye movements were not entirely eliminated by this constraint. One possible explanation for our data is that the observed differences in ERPs might somehow be due to different viewing strategies. To test this, we analyzed the dwell time before the first saccade (i.e., the initial saccade latency) for both familiar and novel stimuli in the 3 experiments (later fixations were not considered, as the response often came immediately after the first saccade). The monkeys generally looked longer at the familiar exemplars than the novel exemplars before executing a small second saccade. In Experiment 1, the mean latency difference across all days for monkey S was 20 ms and for monkey T 73 ms. For Experiments 2 and 3, monkey S's latency differences were extremely small (6 ms). These differences were significant (Wilcoxon signed rank test, P < 0.05) with the exception of Experiment 2 (P > 0.1). Of note, the latency differences for each experiment varied from session to session but did not systematically increase or decrease with time. In contrast, we saw large and consistent changes in the magnitude of the difference between familiar and novel in the visual evoked potential.
To assess any direct relation between the saccade latencies and the ERP differences, we correlated the dwell times and the normalized area measure difference between familiar and novel objects across experimental sessions. A significant correlation could indicate that our familiarity difference was somehow related to differences in saccade patterns for the familiar and novel stimuli. However, only 2 correlations were significant: monkey T in Experiment 1 (r = 0.68, P < 0.005) and monkey S in Experiment 2 (r = −0.72, P < 0.005), and these were in opposite directions. This analysis suggests that the increased ERP amplitudes observed for familiar images are not a direct result of the dwell time prior to the first saccade, but it is nonetheless intriguing that the eye movement data seem to provide an implicit behavioral signature of image familiarity.
The observed familiarity effects also do not appear related to differences in gaze eccentricity. Although our analyses excluded trials in which the monkeys' gaze left a circle of 3-degree radius within the first 300 ms, none of the results we have described changed qualitatively when all trials were included in the analyses. To further verify that eye movements were not responsible for our results, we divided the trials from monkey S in Experiment 1 into 2 groups: those in which gaze remained within one degree of fixation (less than 2% of trials) and those in which the maximum excursion was between 1 and 3 degrees, during the first 300 ms. ICA projections for both of these groups of trials, divided between original and novel exemplars and averaged across all days of the experiment, are shown in Figure 10. Although the traces for the highly constrained trials are somewhat noisy (stemming from the reduction in trials), the large familiarity difference is still clearly evident, further suggesting that differences in viewing strategy are not responsible for the differences in the ERP signals.
Our finding of an enhanced response for familiar stimuli may seem at odds with a number previous reports showing “decreases” in neural signal magnitude as a result of stimulus repetition. When recording from single neurons, a “smaller” neural signal has often been observed for repeated stimuli compared with the initial presentation of a stimulus (Miller and others 1991; Riches and others 1991; Fahy and others 1993; Li and others 1993; Xiang and Brown 1998). These studies repeat specific stimuli within the course of a recording session, sometimes separated by a few seconds or less. Similarly, functional magnetic resonance imaging (fMRI) studies in humans have also shown stimulus-specific adaptation for the blood oxygen level–dependent (BOLD) signal (Buckner and Koutstaal 1998; Henson and others 2000, 2002; James and others 2000; Grill-Spector and Malach 2001; Henson 2003). These studies report a reduction in activity in higher level visual areas when an item is repeated, usually within a block of repetitions lasting several seconds. Thus, if the same face is repeated several times within a block, the BOLD signal is significantly lower than in blocks containing several different faces. It is widely believed that the underlying mechanism for repetition suppression is likely to be the same type of neuronal adaptation found for stimulus repetition in single-cell recording (Grill-Spector and Malach 2001).
There is evidence, however, that these repetition suppression effects are distinct from the type of long-term learning effects that result from weeks of training, such as those reported here. For example, Henson and others (2000) used fMRI to compare repetition suppression for famous faces and familiar symbols to suppression for unfamiliar faces and symbols. Similar to previous studies, they found significant repetition suppression for familiar faces and symbols in a region of right fusiform gyrus. Interestingly, they also found a larger neural response to familiar faces and symbols in bilateral areas of fusiform cortex (close to areas that are known to respond to faces and objects of visual expertise). These results indicate that familiarity and repetition suppression may be driven by separate mechanisms. This is supported by the data of Hölscher and others (2003) described above, who found repetition suppression when stimuli were repeated within a session as well as an overall larger neural signal for highly familiar stimuli (400 repetitions shown over the course of 7–13 days). In agreement with these studies, Pineda and others (1994) found a larger signal soon after stimulus onset for familiar faces (103 ms after stimulus onset) and a smaller signal later (250–600 ms after stimulus onset).
To test whether we could find short-term repetition suppression effects in our data, we performed a reanalysis of both the behavioral and ERP responses. Specifically, we examined conditions in which the same stimulus was shown on 2 consecutive trials during Experiments 1 and 2. Due to the large number of stimuli used in this experiment, the imposed fixation limits, and the random ordering of trials within blocks, this represented a small but significant number of the total trials (approximately 1%). For these data, paired t-tests revealed no significant difference in reaction time between the first and second presentations of target images within a run in any experiment. We then looked to see if we could find any evidence for ERP adaptation. Using the same type of ICA projection and permutation tests described above, we found no significant difference in the ERP signal between the first and second presentation for any of the experiments. Our relatively short recording period (ending whenever the monkey responded, approximately 400 ms after stimulus onset) may explain why we did not find a neural reduction for repeated stimuli. Repetition suppression effects are often found at later time intervals, for example 300–600 ms after stimulus onset (Pineda and others 1994; Rugg 1995). Our recording epoch was shorter because we were interested in perceptual effects similar to the N170, but in future studies, a longer analysis period may allow one to show both an increase early (120–250 ms) and a decrease later (300–600 ms) related to long-term and short-term familiarity, respectively.
The data reported here complement and extend previous monkey single-unit studies (e.g., Miller and others 1991; Riches and others 1991; Li and others 1993; Xiang and Brown 1998; Baker and others 2002; Sigala and Logothetis 2002; Hölscher and others 2003; Sigala 2004). These studies have shown changes in the neural response for increasing stimulus familiarity. At present, it is unclear how activity at the single-neuron level recorded from various areas in extrastriate cortex contributes to the overall activity measured by ERPs and fMRI. At the same time, despite potential problems due to differences in anatomical structures between monkeys and humans, our data are quite consistent with ERP findings in humans. Numerous human ERP studies have explored long-term familiarity within a class of objects by measuring ERPs for faces and have reported a differential ERP response to familiar faces compared with novel faces (Seeck and others 1993; Schweinberger and others 1995, 2002; Caharel and others 2002). In many cases, this difference is a larger neural response for familiar than for novel stimuli recorded soon after stimulus onset (150–300 ms). Thus, this study shows that using ERPs in monkeys may serve as a viable bridge between human and monkey research.
Several neural mechanisms could underlie the larger ERP amplitude associated with extensive training with visual images. The difference could be the result of increased intensity of single-cell responses for objects with which the monkeys have a large amount of experience. This explanation accords with the data reported by Hölscher and others (2003). They reported higher spike rates for single cells in the perirhinal cortex for familiar objects than for novel objects. As the evoked potentials are the result of large populations of active neural elements (Mitzdorf 1985), a second possibility is simply that more cells in visual areas such as IT cortex respond when a familiar object is shown compared with a novel object. Thus, as an object becomes more familiar, more neurons may be recruited to encode the visual properties of that object (Kobatake and others 1998). We have no direct evidence, though, of any increase in the absolute number of cells responding to the familiar stimuli. Finally, another possible explanation is that neuronal inputs to the source regions of the signals we analyzed respond more synchronously to familiar objects than to novel ones, setting up more coherent changes in current densities that are then observable in the evoked potential. This synchronized activity could result in more effective neural transmission (Roy and Alloday 2001; Kara and Reid 2003) and ultimately more efficient processing of well-known objects. It is of course quite possible that a combination of these mechanisms is the cause of the observed effects. Additional research, employing other methodologies such as intracranial local field potential measures and single-unit recordings, will be necessary to understand more fully the underlying mechanism for the type of long-term familiarity effects we report here. The present study, however, provides some additional insight into the behavioral conditions that are sufficient to induce changes in neural processing of highly familiar stimuli.
We would like to thank Julie Lamin and Ryan E. B. Mruczek for assisting with data collection and the members of the Perceptual Expertise Network collaborative network for their valuable comments and suggestions. This research was funded by a grant from the James S. McDonnell Foundation and NIH-EY014681. Conflict of Interest: None declared.