Abstract

Sounds that have been produced with one's own motor system tend to be remembered better than sounds that have only been perceived, suggesting a role of motor information in memory for auditory stimuli. To address potential contributions of the motor network to the recognition of previously produced sounds, we used event-related potential, electric current density, and behavioral measures to investigate memory for produced and perceived melodies. Musicians performed or listened to novel melodies, and then heard the melodies either in their original version or with single pitch alterations. Production learning enhanced subsequent recognition accuracy and increased amplitudes of N200, P300, and N400 responses to pitch alterations. Premotor and supplementary motor regions showed greater current density during the initial detection of alterations in previously produced melodies than in previously perceived melodies, associated with the N200. Primary motor cortex was more strongly engaged by alterations in previously produced melodies within the P300 and N400 timeframes. Motor memory traces may therefore interface with auditory pitch percepts in premotor regions as early as 200 ms following perceived pitch onsets. Outcomes suggest that auditory–motor interactions contribute to memory benefits conferred by production experience, and support a role of motor prediction mechanisms in the production effect.

Introduction

Perception involves the comparison of incoming sensory information with information stored in memory. If a newly encountered stimulus corresponds to a memory trace that is associated with a previously encountered stimulus, recognition can occur. Several behavioral manipulations, including repetition (Hintzman 1976), elaboration (Craik and Lockhart 1972), and organization (Mandler 1967), have previously been used to show benefits in memory retention and subsequent recognition. Recent research in the auditory domain suggests an additional behavioral memory aid: sensorimotor production. Sounds that have been produced with one's own motor system tend to be recognized better than sounds that have simply been perceived (MacLeod et al. 2010). Spoken words tend to be remembered better than words that are mouthed without sound (Gathercole and Conway 1988), or listened to without movement (MacDonald and MacLeod 1998). Similarly, musical melodies performed on a piano with normal auditory feedback are recognized at higher rates than melodies that have only been perceived, or have been produced without sound (Brown and Palmer 2012). This production effect on memory recognition has been attributed to the presence of an additional dimension along which produced stimuli can be discriminated from nonproduced stimuli during recognition (Conway and Gathercole 1987; Dodson and Schacter 2001; Ozubko and MacLeod 2010).

The neural mechanisms of production-based improvement of memory recognition are currently unknown, and the more general links between sensorimotor processing of auditory events and memory processes remain largely unexplored. Sensorimotor experience on tasks such as learning to speak a new language or play a musical instrument is known to yield significant structural (Schlaug 2001; Draganski and May 2008; Hyde et al. 2009) and functional (Hund-Georgiadis and von Cramon 1999; Jäncke et al. 2000; Ungerleider et al. 2002; Golestani and Zatorre 2004) changes in the brain. Musicians, for example, develop strong associations between precise movements and their specific auditory outcomes over years of practice (Palmer 1997; Zatorre et al. 2007), and reciprocal cortical auditory–motor interactions have been observed during both music performance (Baumann et al. 2005; Bangert et al. 2006) and music perception (Haslinger et al. 2005; Koelsch et al. 2006; Brown and Martinez 2007). Superior temporal areas and motor planning regions are often co-activated in response to either pure auditory or silent motor tasks (Bangert et al. 2001; Bangert and Altenmüller 2003; D'Ausilio et al. 2006). The premotor and the inferior frontal cortices have been implicated in a perception–execution matching system in which the motor system is activated by perceived sounds (Bangert et al. 2006; Fiebach and Schubotz 2006; Lahav et al. 2007). Activation of motor associations during auditory perception is a potential mechanism for effects of production experience on auditory memory.

An important aspect of complex auditory stimuli, such as language and music, is that these stimuli are sequential: Elements (e.g., words and tones) follow one another in specific orders, permitting the prediction of upcoming sounds on the basis of current and previous sounds during perception (Jones and Boltz 1989; Federmeier 2007; Tillmann 2012). Perceiving the first few tones of a familiar melody, for example, activates memory traces corresponding to that melody, which then in turn leads listeners to predict future tones. The perception of auditory sequences that have been previously produced may therefore involve sensorimotor prediction mechanisms, in which the motor system generates a model of the motor plan associated with an upcoming auditory event (the perceptual outcome of the motor plan) (Schubotz 2007). According to current motor control models, an efferent copy of a simulated action plan originates in supplementary motor and premotor areas and is communicated to the parietal cortex during perception (Schubotz and von Cramon 2003; Haggard and Whitford 2004; Rauschecker and Scott 2009). In contrast to old/new or remember/know recognition paradigms in which subjects make judgments regarding a single-stimulus event (or several events perceived simultaneously, e.g., in the visual modality), predictive processing and listener expectations may have a considerable influence on how successfully the cognitive system is able to recognize matches and mismatches of newly encountered material with previously encountered material, particularly for auditory sequences. Models of motor simulation and prediction remain largely untested, and it is possible that the involved brain regions play a role in processing memory deviants in previously produced auditory sequences.

Neural recognition processes unfold rapidly, following sound onsets. Pitch percepts corresponding to incoming sensory acoustic information are thought to be fully formed and compared with stored memory representations within 200 ms (Näätänen and Winkler 1999). The auditory N200 event-related potential (ERP), arising from a network of generators including the anterior cingulate, primary auditory cortex, and frontal regions, has been taken to signal a mismatch between sensory input and stored memory representations, with amplitudes scaling according to the strength of the memory violation (Folstein and Van Petten 2008). Subsequent P300 and N400 potentials coincide with the cognitive evaluation of the newly perceived stimulus (Polich 2007; Kutas and Federmeier 2011). Whereas the P300 and accompanying frontal activation have been linked to shifts of auditory attention (Escera et al. 2002) and mental imagery processes (Navarro Cebrian and Janata 2010), and may index the cognitive significance of a stimulus (Nieuwenhuis et al. 2005), the N400 and accompanying medial temporal lobe activation are thought to index stimulus familiarity (Voss and Federmeier 2011). Motor involvement in auditory recognition processes or the use of motor simulation or motor imagery during auditory perception would recruit the motor network (Jeannerod 2001; Bangert et al. 2006; Lotze 2013), which could then respond to events deviating from motor memory traces, thereby enhancing ERP responses to memory deviants. Motor activity should be more pronounced in the hemisphere contralateral to movements, following a functional cortical organization corresponding to that of overt movements (Johnson 1998; Carrillo-de-la-Peña et al. 2008). If the motor system does not contribute to production-based memory enhancement during subsequent perception, we would expect equivalent involvement of the motor system in the perception of both previously perceived and previously produced auditory stimuli.

The goal of the current study was to investigate the neural correlates of effects of auditory–motor learning on memory recognition processes. More specifically, we aimed to elucidate the time course of cortical motor contributions to the processing of deviations from stored memory representations (i.e., expectancy violations), as well as the relationship between production-based memory enhancement and scalp electrophysiological responses to perceived auditory events. Behavioral, ERP, and electric current density source localization measures were used to examine skilled musicians' processing of memory violations in auditorily presented melodies that had previously been learned by either production (movements paired with normal auditory feedback) or perception-only. Therefore, neural responses to the same physical stimulus items were measured, while perceivers' prior sensorimotor experience of the stimuli was manipulated.

On the basis of the production effect on memory for auditory stimuli (Brown and Palmer 2012), we expected skilled pianist participants to more accurately identify memory-violating pitches in previously produced melodies compared with previously perceived melodies. Memory violations were expected to elicit N200, P300, and N400 ERP responses, with larger amplitudes following production experience in comparison to perception-only experience. Given the evidence of neural motor contributions to the perception of previously performed music in skilled musicians and the purported role of motor preparation areas in sensorimotor prediction (Schubotz 2007), we expected the motor network, and the premotor cortex in particular, to show greater involvement in the processing of pitches deviating from production-based memories compared with perception-based memories. As the melodies learned by pianists involved movements of the right hand only, we also predicted that regions activated within the motor network would show greater leftward lateralization in response to violations in melodies learned by production compared with melodies learned by perception.

Materials and Methods

Participants

Twenty-six adult pianists from the Lyon community participated in the study. Six were excluded from analyses due to excessive electroencephalography (EEG) artifacts. The remaining 20 pianists (15 women, age M = 21.7 years, SD = 3.1 years) had between 6 and 20 years of piano experience (M = 11.7 years, SD = 2.8 years). Eighteen participants were right-handed and 2 were left-handed. All participants reported playing piano regularly and none possessed absolute pitch. No participants reported any hearing problems. Participants provided written informed consent, and the study was conducted in accordance with the principles laid down in the Declaration of Helsinki.

Materials

Twelve 12-note melodies in 4/4 time signature, conforming to conventions of Western tonal music, were used in the study (see Fig. 1 for an example of a melody). The melodies were selected from a larger corpus (Brown and Palmer 2012) and were assigned to one of 2 sets so that each set was equated in terms of previously acquired recognition accuracy scores (Brown and Palmer 2012). Audio recordings of naturally performed melodies were obtained from 2 skilled pianists with Cubase 6 software (Steinberg Media Technologies GmbH) from an M-Audio Keystation 88es MIDI piano keyboard. A 500-ms interonset interval metronome, which sounded for 8 quarter notes prior to the start of each recording, set the performance tempo. These recordings were presented to participants during the perception learning condition with a Cubase HALion One piano timbre. The same timbre was used for the auditory feedback heard during the production learning condition.

Figure 1.

Top: One of the notated stimulus melodies. Fingers used to strike piano keys were notated below the musical staff for melodies in both learning conditions. Finger numbers were indicated only for tones for which there were multiple possible fingerings. Bottom: The same stimulus melody containing an altered pitch (circled) that was heard during the memory recognition test.

Figure 1.

Top: One of the notated stimulus melodies. Fingers used to strike piano keys were notated below the musical staff for melodies in both learning conditions. Finger numbers were indicated only for tones for which there were multiple possible fingerings. Bottom: The same stimulus melody containing an altered pitch (circled) that was heard during the memory recognition test.

During the memory recognition test, melodies from the perception and production learning conditions were presented as computer-generated MIDI recordings with 500-ms per quarter note interonset intervals (with no expressive timing variations), and with the same timbre as in the learning conditions. MIDI velocity was constant for all pitches. A single pitch alteration was placed in 50% of the melodies. Altered pitches were from the diatonic key established by each melody, maintained the melodic contour of the original melody, and remained within a major third of the original target pitch. Pitch alterations occurred in one of 8 different serial locations and never occurred on the first 3 pitches or the last pitch of a melody. Alterations were never repetitions of the preceding or following tone. The degree to which target pitches were primed on a sensory level within the preceding melodic context was controlled by ensuring that each altered pitch appeared equivalently or more often in the preceding context than the corresponding original pitch. Altered pitches were placed only on quarter notes, and were therefore always 500 ms in duration. Eighteen of the altered pitches were preceded by quarter notes and 12 were preceded by eighth notes. The altered pitches were aligned equally often with weakly accented metrical beats and with strongly accented beats, as determined by a four-tier metrical hierarchy (Lerdahl and Jackendoff 1983). Finally, altered pitches were designed to be produced by the same right-hand finger that was used to produce the original target pitch during learning, and original and altered pitches were distributed across fingers within the right hand. The sets of melodies assigned to the 2 learning conditions were matched on each of these features. An example of a melody and a pitch alteration is shown in Figure 1.

Participants were seated in a soundproof, electrically shielded chamber while EEG was recorded, and melodies were presented over EEG-compatible air-delivery headphones (ER-2 Tubephones, Etymotic Research, Inc.). During the memory recognition test, EEG was recorded with 95 Ag/AgCl active electrodes (ActiCAP, Brain Products GmbH) configured according to the international 10–20 system. Participants' eyes remained open during EEG recording. The signal was recorded with a BrainAmp amplifier at a resolution of 16 bits, a sampling rate of 500 Hz, and with an analog low pass of 1000 Hz and high pass of 0.016 Hz. The ground electrode was placed at position AFz and the reference electrode on the tip of the nose. Electrodes below and above the right eye monitored vertical eye movements and 2 electrodes (F9 and F10) monitored horizontal eye movements. Electrode impedances were kept below 30 kΩ.

Design

The study used a repeated-measures 2 (perception/production learning conditions) × 2 (altered/original target pitches) within-participant design. In the learning phase, half of the participants received one set of melodies in the production learning condition and the other set of melodies in the perception learning condition, whereas the other half of participants received the reverse melody assignment. The order of the perception and production learning conditions was counterbalanced across participants. In the memory recognition test, melodies were presented over 5 blocks. Within each block, each of the 12 learned melodies was presented once in its original form and once in its altered form, with order of melodies randomized within each block. Each altered pitch occurred only once at a given serial position within the melodic context; thus, each altered melody was unique within the context of the experiment and was therefore heard only once by participants over the course of the experiment. This resulted in 30 (6 melodies × 5 blocks) recognition trials per experimental condition (perception/production learning × original/altered target pitch) and a total of 120 recognition trials.

Procedure

Participants first completed a musical background questionnaire, followed by a piano performance sight-reading test. Participants who were able to perform a short single-hand melody from notation to a note-perfect criterion within 2 attempts were admitted to the experiment. All pianists who were invited to participate met this criterion. Following completion of the sight-reading test, participants were outfitted with EEG caps and electrodes.

Learning Phase

Participants learned 12 novel melodies: 6 melodies were learned in the perception learning condition and the 6 other melodies were learned in the production learning condition. In the perception learning condition, pianists heard 10 renditions of each melody over headphones. In the production learning condition, pianists performed 10 successive renditions of each melody. The musical notation for each melody remained in view during both learning conditions. Fingers used to strike piano keys were notated below the musical staff for melodies in both learning conditions; finger numbers were indicated only for tones for which there were multiple possible fingerings. For the production learning condition, each trial began with an initial metronome sounded at 500 ms per quarter note (the same IOI at which perceived melodies were presented) for 8 quarter notes prior to the start of each performance, and stopped when participants began to perform. Normal auditory feedback triggered by piano key presses was delivered via headphones during performances. Thus, the conditions for the production learning condition were identical to those under which the prerecorded melodies (used in the perception learning condition) were performed. Participants were instructed prior to the learning conditions that their memory for the melodies would be tested following learning. The learning phase lasted ∼35 min. EEG activity was not recorded during the learning phase.

Memory Recognition Test

Following the learning phase, participants were presented over headphones with the computer-generated recordings of the 12 originally learned melodies repeated 5 times and 60 unique altered melodies divided across 5 blocks. EEG was simultaneously recorded and participants were asked to identify whether or not each melody contained an incorrect pitch. At the beginning of each trial, a fixation cross appeared in the center of the computer screen, and after 2000 ± 500 ms, a melody was presented auditorily (melody notation was not shown during the memory recognition test). The fixation cross remained on the screen for the entire duration of the melody, and participants were instructed to fixate on the cross for the entire duration. Participants were instructed to avoid blinking and moving during the presentation of the melodies. After listening to each melody, participants indicated whether the melody contained an alteration (Yes/No) and how confident they were in their judgment using a Likert scale (adapted from Opacic et al. 2009), ranging from 1 (not sure at all) to 5 (very sure); 0 corresponded to “guessing.” No time limit was imposed for recognition or confidence responses. Participants were told that they could blink and relax before pressing a key to proceed to the next trial. The time interval between the end of the learning phase and the start of the recognition trials was ∼5 min.

Post-test

Participants then listened to each original melody (with no altered pitches) and indicated whether they had first learned the melody by listening to it or performing it (Listened/Performed), and their level of confidence in their judgment on the same confidence rating scale as was used in the memory test. Each melody was presented once, in different random orders for each participant. EEG activity was not recorded during the post-test.

Data Recording and Analysis

Behavioral data

Learning phase

Errors in pitch accuracy during the production learning condition were identified by computer comparison of pianists' performances with the information in the notated musical score (Large 1993). Pitch omissions were counted as errors as well. Corrections (errors in which pianists stopped after an error and corrected) were excluded from error rate computations and analyzed separately.

Memory recognition test

Mean accuracy scores in the recognition test were analyzed using a 2 (learning condition) × 2 (target pitch) repeated-measures analysis of variance (ANOVA). Response accuracy was coded categorically as either correct or incorrect. Recognition accuracy was also analyzed in terms of hits (correct identification of an altered melody) minus false alarms (incorrect identification of an original melody) scores with a one-way ANOVA (comparing the 2 learning conditions). Confidence ratings were evaluated with a 2 (learning condition) × 2 (target pitch) × 2 (response accuracy) repeated-measures ANOVA.

Post-test

Post-test data were analyzed in a one-way ANOVA on the proportion of melodies that were correctly identified in the production learning and perception learning conditions. A 2 (learning condition) × 2 (response accuracy) ANOVA on post-test confidence ratings was also conducted.

EEG data

EEG signals were analyzed using BrainVision Analyzer 2.0.2 (Brain Products GmbH). Electrodes were re-referenced offline to the average of all scalp electrodes. The EEG signals were bandpass-filtered between 1 and 30 Hz. Data were segmented into 600 ms epochs beginning 100 ms prior to the onset of the target pitch (altered pitch or contextually identical original pitch in the presented melodies) and terminating at the onset of the subsequent pitch. Artifact rejection was performed automatically using a ±50 μV rejection threshold at electrodes Fz, Cz, Pz, and Oz, as well as the horizontal and vertical electro-oculogram, and manually by removing any trials seemingly contaminated with eye movements or muscle activity on any of the electrodes. Artifacts were considered excessive when more than half of the trials from a given condition of the experiment exceeded the ±50 μV rejection threshold at electrodes Fz, Cz, Pz, Oz, or the horizontal or vertical electro-oculogram. Trials for which participants' responses were incorrect were excluded from averages, leaving a mean of 20.9 trials (SD = 5.5) in the perception-altered condition, 23.2 (SD = 5.2) trials in the production-altered condition, 21.1 trials (SD = 5.2) in the perception-original condition, and 21.8 trials (SD = 4.7) in the production-original condition. Trial numbers were roughly equivalent across conditions; slight differences reflected the higher number of trials included in the ERP analysis for the production altered and original conditions compared with the perception altered and original conditions.

Event-related potentials

Average ERPs for each participant and each of the 4 experimental conditions were time-locked to the onset of the target pitch using EEG activity occurring up to 100 ms prior to the target pitch as a baseline. Mean ERP amplitudes were statistically evaluated at 9 topographical regions of interest (ROIs; see bottom middle subplot of Fig. 2): left anterior (F1, F3, F5, F7, AF3, AF7, AFF1h), right anterior (F2, F4, F6, F8, AF4, AF8, AFF2h), midline anterior (Fz), left central (C1, C3, C5, T7), right central (C2, C4, C6, T8), midline central (Cz), left posterior (P1, P3, P5, P7, PO3, PO7, PPO1h), right posterior (P2, P4, P6, P8, PO4, PO8, PPO2h), and midline posterior (Pz). Electrode ROIs were adapted from Miranda and Ullman (2007). Forty-millisecond time windows for statistical analysis of ERP components were centered on grand-averaged peak amplitude latencies as follows: 100–140 ms (label N100), 180–220 ms (labeled N200), 240–280 ms (labeled N400), 270–310 ms (labeled P300), and 420–460 ms (labeled LPC). Peak amplitude latencies were identified on the basis of previous research and visual inspection of the grand averages, and calculated by averaging peak amplitude latencies across midline electrodes Fz, Cz, and Pz.

Figure 2.

Grand-averaged ERPs elicited by the 4 experimental conditions for trials in which participants correctly identified the presented melody as altered or original. Activity with each of 9 topographical regions of interest (ROIs) is shown (bottom center; see Materials and Methods for more details): Left anterior (electrodes F1, F3, F5, F7, AF3, AF7, AFF1h), right anterior (F2, F4, F6, F8, AF4, AF8, AFF2h), midline anterior (Fz), left central (C1, C3, C5, T7), right central (C2, C4, C6, T8), midline central (Cz), left posterior (P1, P3, P5, P7, PO3, PO7, PPO1h), right posterior (P2, P4, P6, P8, PO4, PO8, PPO2h), and midline posterior (Pz). Activity within left and right lateral ROIs is averaged across all electrodes contained within the ROI. Negative is plotted upward.

Figure 2.

Grand-averaged ERPs elicited by the 4 experimental conditions for trials in which participants correctly identified the presented melody as altered or original. Activity with each of 9 topographical regions of interest (ROIs) is shown (bottom center; see Materials and Methods for more details): Left anterior (electrodes F1, F3, F5, F7, AF3, AF7, AFF1h), right anterior (F2, F4, F6, F8, AF4, AF8, AFF2h), midline anterior (Fz), left central (C1, C3, C5, T7), right central (C2, C4, C6, T8), midline central (Cz), left posterior (P1, P3, P5, P7, PO3, PO7, PPO1h), right posterior (P2, P4, P6, P8, PO4, PO8, PPO2h), and midline posterior (Pz). Activity within left and right lateral ROIs is averaged across all electrodes contained within the ROI. Negative is plotted upward.

Mean ERP component amplitudes were assessed in 3 levels of analysis following a procedure used by Miranda and Ullman (2007). The first-level analysis determined whether effects of independent variables differed significantly across scalp regions, the second-level analysis allowed us to identify the scalp region (single ROI or group of ROIs) where the component was most prominent, and the third-level analysis tested influences of independent variables on ERPs within the ROI(s) where the component was determined to be statistically maximal.

In the first-level analysis, ERP amplitudes at midline ROIs were tested in 2 × 2 × 3 repeated-measures ANOVAs with factors learning condition (perception, production), target pitch (altered, original), and position (anterior, central, posterior). Mean amplitudes at lateral ROIs were tested in 2 × 2 × 2 × 3 repeated-measures ANOVAs with factors learning condition (perception, production), target pitch (altered, original), hemisphere (right, left), and position (anterior, central, posterior). Only interactions involving one or both independent variables (learning condition, target pitch) and one or both scalp-distribution factors (hemisphere, position) that were determined to be significant in the first-level analysis are reported. In the second-level analysis, repeated-measures ANOVAs performed on only those factors involved in each significant first-level interaction. ROIs (midline or lateral) where components showed larger absolute voltages were analyzed. In the third-level analysis, repeated-measures ANOVAs that included learning condition and target pitch as factors were conducted on ROIs where the component was determined by the second-level analysis to be most prominent.

Scalp topographic maps showing ERP component distributions were generated by plotting amplitude values on the scalp. Activity was averaged across the time window used for the analysis of each component. Scalp topographic maps representing difference waves for each component were also generated by subtracting mean amplitudes corresponding to original pitches from mean amplitudes corresponding to altered pitches within the analysis time window for each component.

Source localization

Standardized low-resolution brain electromagnetic tomography (sLORETA) was used to compute cortical activity (source current density in µA/mm3) corresponding to ERP components that showed effects of learning condition. The sLORETA method is a standardized discrete, 3D distributed, linear, minimum norm solution to the inverse problem (Pascual-Marqui 2002), which has been validated in several simultaneous EEG/fMRI studies (Olbrich et al. 2009; Mobascher et al. 2009) and allows accurate localization of deep cortical structures, including the anterior cingulate cortex (Pizzagalli et al. 2001) and mesial temporal lobes (Zumsteg et al. 2006).

In the current implementation of sLORETA, computations were made in a realistic head model (Fuchs et al. 2002), using the MNI152 template (Mazziotta et al. 2001), with the 3D solution space restricted to cortical gray matter, as determined by the probabilistic Talairach atlas (Lancaster et al. 2000). Standard electrode positions on the MNI152 scalp were taken from Jurcak et al. (2007) and Oostenveld and Praamstra (2001). The intracerebral volume was partitioned in 6239 voxels at a 5-mm spatial resolution. Thus, sLORETA images represent the standardized electric activity at each voxel in neuroanatomical Montreal Neurological Institute (MNI) space as the exact magnitude of the estimated current density. Source current densities for each participant corresponding to the perception of memory violations in previously produced (production-altered condition) melodies and source current densities corresponding to the perception of memory violations in previously perceived (perception-altered condition) melodies were compared within the time windows of the N200, P300, and N400 using a voxel-wise randomization test of log F-ratios. sLORETA performed 5000 permutations of the randomized statistical nonparametric mapping (SnMP), and critical log F-ratios and significance values were corrected for multiple comparisons. Log F-ratio values for each voxel were thresholded based on a corrected significance threshold of P < 0.01 for the source localization of each of the ERP components.

Results

Behavioral Results

Learning Phase

Less than 1% of tones per performance in the production learning condition were produced erroneously (M pitch error rate per trial = 0.0097, SE = 0.0019). A mean of 95.5% of all performances contained no errors (SE = 1.3%), indicating that participants performed the presented melodies with high accuracy. Corrections (pitch errors in which pianists stopped after an error and corrected) occurred in a mean of 3.7% of all performances (SE = 1.5%). The serial locations of errors produced during the learning phase rarely matched the serial locations of altered target pitches in the memory recognition test: Across all participants, melodies, and performances, errors were produced in only 0.3% of all possible opportunities for errors to match the locations of alterations, and corrections were produced in only 0.8% of all possible opportunities.

Memory Recognition Test

Analyses of mean recognition accuracy scores (Fig. 3) indicated a significant effect of learning condition, F1,19 = 5.05, P < 0.05. Participants were more accurate at judging whether or not a melody contained an altered pitch for melodies from the production learning condition (M percentage correct responses = 83.8%, SE = 1.9%) compared with melodies from the perception learning condition (M percentage correct responses = 80.6%, SE = 2.4%). There was no significant main effect of target pitch and no significant interactions (all Ps > 0.46).

Figure 3.

Top: Mean percentage of correct responses in the memory recognition test by learning condition (perception/production) and target pitch (altered/original). Middle: Mean confidence ratings in the memory recognition test by learning condition (perception/production), target pitch (altered/original), and response accuracy (correct/incorrect). Bottom: Post-test mean confidence ratings in the memory recognition test by learning condition (perception/production) and response accuracy (correct/incorrect). Error bars represent one standard error. *P < 0.05.

Figure 3.

Top: Mean percentage of correct responses in the memory recognition test by learning condition (perception/production) and target pitch (altered/original). Middle: Mean confidence ratings in the memory recognition test by learning condition (perception/production), target pitch (altered/original), and response accuracy (correct/incorrect). Bottom: Post-test mean confidence ratings in the memory recognition test by learning condition (perception/production) and response accuracy (correct/incorrect). Error bars represent one standard error. *P < 0.05.

Analyses of hits – false alarms scores indicated a significant main effect of learning condition, F1,19 = 5.05, P < 0.05. Higher scores were achieved for melodies that had been produced (M = 0.676, SE = 0.040) compared with melodies that had only been perceived (M = 0.611, SE = 0.040). Years of piano instruction correlated positively with hits – false alarms for the production learning condition, r(18) = 0.58, P < 0.05, and the perception learning condition, r(18) = 0.58, P < 0.05.

Analyses of participants' confidence ratings on the memory test (Fig. 3) revealed main effects of learning condition, F1,19 = 18.16, P < 0.001, of target, F1,19 = 10.06, P < 0.01, and of response accuracy, F1,19 = 117.48, P < 0.001. Participants indicated higher confidence when identifying produced melodies (M = 3.61, SE = 0.094) compared with perceived melodies (M = 3.27, SE = 0.108). They also indicated higher confidence when the identified melody contained an altered pitch (M = 3.59, SE = 0.095) compared with an original pitch (M = 3.29, SE = 0.109). Finally, participants indicated higher confidence for correct responses (M = 3.97, SE = 0.070) than for incorrect responses (M = 2.91, SE = 0.096). Learning condition showed a marginally significant interaction with accuracy, F1,19 = 3.18, P = 0.09. There were no other significant interactions between learning condition, target pitch, or response accuracy variables (all Ps > 0.66).

Post-test

Analyses of post-test accuracy revealed a significant main effect of learning condition, F1,19 = 7.33, P < 0.05. Participants were more accurate at remembering how they had first learned a melody when the melody was learned by production (M = 0.675, SE = 0.055) compared with when it was learned by perception (M = 0.541, SE = 0.038).

Analysis of post-test confidence ratings (Fig. 3) indicated a significant main effect of response accuracy, F1,19 = 13.94, P < 0.01, which interacted significantly with learning condition, F1,19 = 11.79, P < 0.005: Participants rated their confidence level for only the production condition melodies as higher for correctly identified melodies compared with incorrectly identified melodies (Tukey HSD = 0.67, α = 0.05). This was not the case for the perception condition confidence ratings. There was no main effect of learning condition on confidence ratings in the post-test (P = 0.76).

ERP Results

Figure 2 shows grand-averaged ERP waveforms time-locked to target pitches averaged across correct response trials. Visual inspection revealed an auditory P1–N1–P2 complex elicited by both the altered and original target pitches in both learning conditions. Subsequent ERP components elicited by altered targets included an early negative component maximal around 180–220 ms (labeled N200), a later negative component around 240–280 ms (labeled N400), a positive component maximal around 270–310 ms (labeled P300), and a later positive maximal around 420–460 ms (labeled LPC). Scalp topographies corresponding to time ranges for altered pitches and altered/original voltage differences are shown in Figure 4.

Figure 4.

Top: Voltage (in µV) scalp topographies for altered pitches by learning condition (perception/production). Bottom: Voltage (in µV) scalp topographies showing the distribution of the difference between original and altered pitch conditions. For both altered pitch topographies and difference topographies, activity averaged over 40 ms surrounding each component's grand-averaged peak is shown (N200, 180–220 ms; P300, 270–310 ms; N400, 240–280 ms; LPC, 420–460 ms).

Figure 4.

Top: Voltage (in µV) scalp topographies for altered pitches by learning condition (perception/production). Bottom: Voltage (in µV) scalp topographies showing the distribution of the difference between original and altered pitch conditions. For both altered pitch topographies and difference topographies, activity averaged over 40 ms surrounding each component's grand-averaged peak is shown (N200, 180–220 ms; P300, 270–310 ms; N400, 240–280 ms; LPC, 420–460 ms).

Negative Components

Analysis of amplitudes within the N100 time window at midline ROIs yielded a significant main effect of position, F2,38 = 13.37, P < 0.001: The N100 was more prominent at anterior ROIs than at central and posterior ROIs (Tukey HSD = 0.44, α = 0.05). There were no other significant main effects or interactions (all Ps > 0.15).

Analysis of amplitudes within the N200 time window at midline ROIs yielded a significant three-way interaction between learning condition, target pitch, and position variables, F2,38 = 8.50, P < 0.001. Second-level analysis (see Materials and Methods) of the three-way interaction showed a significant interaction between target pitch and position, F2,38 = 6.10, P = 0.005. Altered pitches elicited larger negative potentials than original pitches at both anterior and central ROIs (Tukey HSD = 0.71, α = 0.05). Anterior and central ROIs were selected as the target region for third-level analysis with the factors learning condition, target pitch, and anterior–central ROI position (see Fig. 5). There was a main effect of target pitch on N200 amplitudes, F1,19 = 21.49, P < 0.001, and a significant interaction between learning condition and target, F1,19 = 6.93, P < 0.05. The amplitude of the N200 was larger (more negative) for the altered pitches than for the original pitches, and this difference was greater for the production learning condition than for the perception learning condition (Tukey HSD = 0.63, α = 0.05). There was no significant main effect of position (anterior/central), F1,19 = 0.77, P = 0.39, and there were no other significant main effects or interactions (all Ps > 0.14).

Figure 5.

Mean amplitude values of correct response grand-averaged event-related potentials (ERPs) that were elicited by target pitches. Amplitudes from third-level ERP analysis are shown. These amplitudes were pooled across electrodes within a priori ROIs for which the component was statistically determined to be most prominent (the anterior and central midline ROIs for the N200, anterior midline ROI for the P300, posterior lateral ROIs for the N400, and posterior midline ROI for the LPC). *P < 0.05.

Figure 5.

Mean amplitude values of correct response grand-averaged event-related potentials (ERPs) that were elicited by target pitches. Amplitudes from third-level ERP analysis are shown. These amplitudes were pooled across electrodes within a priori ROIs for which the component was statistically determined to be most prominent (the anterior and central midline ROIs for the N200, anterior midline ROI for the P300, posterior lateral ROIs for the N400, and posterior midline ROI for the LPC). *P < 0.05.

Analysis of mean amplitudes in the N400 time range at lateral ROIs showed a significant interaction between learning condition and position, F2,38 = 5.72, P < 0.01, as well as a significant interaction between target pitch and position, F2,38 = 14.97, P < 0.001. Second-level analysis of the interaction between learning condition and position revealed a main effect of position, F2,38 = 23.04, P < 0.001. The negativity was restricted to posterior ROIs (Tukey HSD = 0.46, α = 0.05). Second-level analysis of the interaction between target pitch and position also revealed a main effect of position, F2,38 = 23.04, P < 0.001. Posterior ROIs showed a larger negativity than anterior and central ROIs (Tukey HSD = 0.46, α = 0.05). Third-level analysis on posterior ROIs revealed a significant main effect of learning condition, F1,19 = 6.38, P < 0.05. Produced target pitches elicited a significantly larger (more negative) N400 than perceived target pitches (Fig. 5). There was also a significant main effect of target pitch, F1,19 = 16.51, P < 0.001. Altered pitches elicited a larger (more negative) N400 than original pitches. There was a marginal main effect of hemisphere within the posterior ROIs, F1,19 = 3.24, P = 0.09. The right-lateralized posterior ROI showed larger negative amplitudes than the left-lateralized posterior ROI. Hemisphere also showed a marginally significant interaction with learning condition, F1,19 = 3.34, P = 0.08. Produced targets tended to elicit more negative responses at left-lateralized ROIs than perceived targets. There were no other significant main effects or interactions (all Ps > 0.18). The individual averages of one of the 2 left-handed participants showed the same pattern of N400 lateralization as the grand averages, whereas the other left-handed participant's individual averages showed no lateralization.

Positive Components

Analysis of amplitudes at midline ROIs within the time range of the P300 yielded a significant interaction between target pitch and position, F2,38 = 14.88, P < 0.001, as well as a marginally significant interaction between learning condition and position, F2,38 = 2.97, P = 0.06. Second-level analysis of the interaction between target pitch and position indicated a significant main effect of position. The positivity was most prominent at the anterior ROI (Tukey HSD = 0.46, α = 0.05). Analysis of the interaction between learning condition and position also indicated a significant main effect of position, F2,38 = 31.80, P < 0.001. The positivity was most prominent at the anterior ROI (Tukey HSD = 0.46, α = 0.05). Third-level analysis on anterior ROIs revealed a significant main effect of target pitch, F1,19 = 19.84, P < 0.001. Altered pitches elicited a larger P3 than original pitches (Fig. 5). The main effect of learning on P300 amplitudes at the anterior midline ROI did not reach significance, F1,19 = 2.58, P = 0.13. However, the main effect of learning was significant when evaluating mean amplitudes across lateral ROIs, F1,19 = 5.29, P < 0.05. Produced pitches elicited larger amplitudes in the time range of the P300 than perceived pitches (Fig. 5). There were no other significant main effects or interactions (all Ps > 0.88).

Within the time range of the LPC, there was a significant interaction between target pitch and position, F2,38 = 5.15, P < 0.05. Second-level analysis showed a significant main effect of position, F2,38 = 7.94, P < 0.001. The LPC was most prominent at posterior ROIs. Third-level analysis showed a significant main effect of target pitch, F1,19 = 17.75, P < 0.001. The LPC was larger for the altered pitches than for original pitches (Fig. 5). There were no other significant main effects or interactions (all Ps > 0.58).

Correlations of ERPs and Behavioral Measures

Correlation Between ERP Amplitudes and Recognition Accuracy

Hits – false alarms scores corresponding to the production learning condition showed marginally significant correlations with mean amplitudes in the production learning condition within the time ranges of the N200, r(18) = −0.37, P = 0.10; the P300, r(18) = 0.37, P = 0.10; and the N400, r(18) = −0.43, P = 0.06. This was also observed for the amplitudes within the time range of the N200 for the perception learning condition, r(18) = −0.41, P = 0.07. No other component amplitudes correlated with accuracy scores in the memory task for perception and production conditions.

Correlation Between ERP Amplitudes and Confidence Ratings

The amplitude of the P300 elicited by altered pitches for previously produced melodies positively correlated with the related confidence ratings in the memory recognition task, r(18) = 0.44, P = 0.05, while this was not the case for the perception condition, r(18) = 0.05, P = 0.84. Participants demonstrating larger P300 amplitudes in response to altered pitches in previously produced melodies rated higher levels of confidence in their memory recognition judgments for produced melodies containing altered pitches (Fig. 6). No other component amplitudes correlated significantly with confidence ratings in the memory task.

Figure 6.

Correlation of mean P300 amplitudes elicited by altered pitches in previously produced melodies with recognition confidence ratings for the production-altered condition.

Figure 6.

Correlation of mean P300 amplitudes elicited by altered pitches in previously produced melodies with recognition confidence ratings for the production-altered condition.

Source Localization Results

Figure 7 shows differences in source current density activity elicited by altered target pitches in previously produced melodies compared with previously perceived melodies. Differences are shown in terms of log F-ratios corresponding to the time ranges of ERP responses that showed effects of the learning condition. Brain regions showing increased activity in the production-altered condition compared with the perception-altered condition within the time ranges of the N200, P300, and N400 are shown in Table 1, and brain regions showing increased activity for the perception-altered condition compared with the production-altered condition within the time ranges of the N200, P300, and N400 are shown in Table 2.

Table 1

sLORETA results: Brain regions showing significantly increased activity during pitch alterations in previously produced melodies compared with previously perceived melodies

Brain region N200
 
P300
 
N400
 
(x, y, zLog F-ratio (x, y, zLog F-ratio (x, y, zLog F-ratio 
midFG (25, 20, 60) 1.39 (20, 10, 65) 0.90 (25, −10, 45) 1.22 
(−25, 15, 60) 1.46 (−30, −10, 50) 1.11   
medFG (0, 40, 45) 1.61 (0, 5, 50) 1.07 (10, −25, 50) 1.34 
(−5, 40, 45) 1.50 (−5, 5, 50) 1.11 (−10, −25, 50) 1.31 
ACC (0, 45, 10) 0.94 (0, 50, 0) 1.15 (20, 45, 10) 1.09 
(−10, 35, −5) 1.05 (−5, 45, 0) 1.12   
IFG (−20, 30, −5) 1.03     
Precuneus     (5, −35, 45) 1.25 
(−25, −85, 35) 0.99   (−5, −35, 45) 1.15 
preCG   (25, −15, 50) 0.94 (20, −25, 55) 1.32 
  (−25, −15, 50) 1.14 (−20, −25, 55) 1.07 
postCG     (20, −30, 55) 1.25 
  (−50, −20, 55) 0.92 (−20, −30, 50) 1.08 
SFG   (15, 10, 55) 0.97 (5, −5, 70) 1.15 
  (−5, 60, 0) 1.03   
OG   (−5, 50, −20) 0.88   
CG   (5, 0, 45) 1.16 (20, −25, 45) 1.35 
  (−10, 5, 45) 1.26 (0, −25, 40) 1.25 
ITG   (−50, −75, −5) 0.90   
paraCL     (5, −25, 45) 1.29 
    (0, −30, 45) 1.24 
MTG     (−65, −40, −20) 1.25 
Insula     (−40, −45, 20) 1.08 
Brain region N200
 
P300
 
N400
 
(x, y, zLog F-ratio (x, y, zLog F-ratio (x, y, zLog F-ratio 
midFG (25, 20, 60) 1.39 (20, 10, 65) 0.90 (25, −10, 45) 1.22 
(−25, 15, 60) 1.46 (−30, −10, 50) 1.11   
medFG (0, 40, 45) 1.61 (0, 5, 50) 1.07 (10, −25, 50) 1.34 
(−5, 40, 45) 1.50 (−5, 5, 50) 1.11 (−10, −25, 50) 1.31 
ACC (0, 45, 10) 0.94 (0, 50, 0) 1.15 (20, 45, 10) 1.09 
(−10, 35, −5) 1.05 (−5, 45, 0) 1.12   
IFG (−20, 30, −5) 1.03     
Precuneus     (5, −35, 45) 1.25 
(−25, −85, 35) 0.99   (−5, −35, 45) 1.15 
preCG   (25, −15, 50) 0.94 (20, −25, 55) 1.32 
  (−25, −15, 50) 1.14 (−20, −25, 55) 1.07 
postCG     (20, −30, 55) 1.25 
  (−50, −20, 55) 0.92 (−20, −30, 50) 1.08 
SFG   (15, 10, 55) 0.97 (5, −5, 70) 1.15 
  (−5, 60, 0) 1.03   
OG   (−5, 50, −20) 0.88   
CG   (5, 0, 45) 1.16 (20, −25, 45) 1.35 
  (−10, 5, 45) 1.26 (0, −25, 40) 1.25 
ITG   (−50, −75, −5) 0.90   
paraCL     (5, −25, 45) 1.29 
    (0, −30, 45) 1.24 
MTG     (−65, −40, −20) 1.25 
Insula     (−40, −45, 20) 1.08 

Note: MNI coordinates of peak increases in standardized current density activity elicited by altered pitches for the production condition compared with the perception condition within the time ranges of the N200, P300, and N400, and peak log F-ratio values significant at P < 0.01, corrected.

midFG, middle frontal gyrus; medFG, medial frontal gyrus; ACC, anterior cingulate cortex; IFG, inferior frontal gyrus; preCG, precentral gyrus; postCG, postcentral gyrus; SFG, superior frontal gyrus; OG, orbital gyrus; CG, cingulate gyrus; ITG, inferior temporal gyrus; paraCL, paracentral lobule; MTG, middle temporal gyrus.

Table 2

sLORETA results: Brain regions showing significantly increased activity during pitch alterations in previously perceived melodies compared with previously produced melodies

Brain region N200
 
P300
 
N400
 
(x, y, zLog F-ratio (x, y, zLog F-ratio (x, y, zLog F-ratio 
SMG (50, −50, 20) −1.59 (55, −50, 20) −1.11   
STG (60, −60, 20) −1.42 (50, −45, 20) −1.09   
IPL (55, −45, 25) −1.28 (55, −45, 25) −1.10   
MTG (60, −60, 10) −1.11     
Insula (40, −45, 20) −0.94     
Brain region N200
 
P300
 
N400
 
(x, y, zLog F-ratio (x, y, zLog F-ratio (x, y, zLog F-ratio 
SMG (50, −50, 20) −1.59 (55, −50, 20) −1.11   
STG (60, −60, 20) −1.42 (50, −45, 20) −1.09   
IPL (55, −45, 25) −1.28 (55, −45, 25) −1.10   
MTG (60, −60, 10) −1.11     
Insula (40, −45, 20) −0.94     

Note: MNI coordinates of peak decreases in standardized current density activity elicited by altered pitches for the production condition compared with the perception condition within the time ranges of the N200, P300, and N400, and peak log F-ratio values significant at P < 0.01, corrected.

SMG, supramarginal gyrus; STG, superior temporal gyrus; IPL, inferior parietal lobule; MTG, middle temporal gyrus.

Figure 7.

Standardized low-resolution brain electromagnetic tomography (sLORETA) images depicting brain voxels that differed in standardized current density responses to altered target pitches in previously produced versus previously perceived melodies within the time windows of the N200 (top), P300 (middle), and N400 (bottom). Voxels that showed the largest increases in standardized current density for the production condition compared with the perception condition are indexed in yellow, and voxels showing largest increases for the perception condition compared with the production condition are indexed in blue. Brighter colors indicate larger differences in terms of statistical log F-ratios. MNI coordinates: N200, x = −10, y = −50, z = 45; P300, x = −10, y = −50, z = 45; N400, x = 20, y = −30, z = 45.

Figure 7.

Standardized low-resolution brain electromagnetic tomography (sLORETA) images depicting brain voxels that differed in standardized current density responses to altered target pitches in previously produced versus previously perceived melodies within the time windows of the N200 (top), P300 (middle), and N400 (bottom). Voxels that showed the largest increases in standardized current density for the production condition compared with the perception condition are indexed in yellow, and voxels showing largest increases for the perception condition compared with the production condition are indexed in blue. Brighter colors indicate larger differences in terms of statistical log F-ratios. MNI coordinates: N200, x = −10, y = −50, z = 45; P300, x = −10, y = −50, z = 45; N400, x = 20, y = −30, z = 45.

Motor preparation areas showed consistently stronger activation for the production-altered condition than the perception-altered condition across the N200, P300, and N400 time ranges. The middle (BA 6; peak MNI coordinates: x = −25, y = 15, z = 60, log F-ratio = 1.46, P < 0.01) and medial (BA 8; peak MNI coordinates x = 0, y = 40, z = 45; log F-ratio = 1.61, P < 0.01) frontal gyri showed largest activation increases for the production condition compared with the perception condition within the timeframe of the N200. The same brain areas showed significantly greater contributions to the generation of the P300 and N400 potentials for the production condition compared with the perception condition.

Increases in activation for the production condition compared with the perception condition were more prominent in the left hemisphere than the right during the time ranges of both the N200 and P300. In particular, the left inferior frontal gyrus (BA 47) showed selective engagement in the early processing of altered pitches in previously produced melodies (peak MNI coordinates within the time range of the N200: x = −20, y = −30, z = −5, log F-ratio = 1.03, P < 0.01). Leftward primary motor cortex (precentral gyrus; BA 4) was also activated more strongly for the production condition within the time ranges of the P300 (peak MNI coordinates: x = −25, y = −15, z = 50, log F-ratio = 1.14, P < 0.01) and the N400 (peak MNI coordinates: x = 20, y = −25, z = 55, log F-ratio = 1.32, P < 0.01).

The anterior cingulate has previously been identified as one of several generators contributing to auditory N200 and P300 components. Within the time range of the P300, the anterior cingulate cortex (BA 24; peak MNI coordinates: x = −10, y = 5, z = 45, log F-ratio = 1.26, P < 0.01; BA 32; peak MNI coordinates: x = 0, y = 50, z = 0, log F-ratio = 1.15, P < 0.01) showed largest increases in activation for the production-altered condition compared with the perception-altered condition. The anterior cingulate also contributed significantly to the N200 (peak MNI coordinates x = −10, y = 35, z = −5; log F-ratio = 1.05, P < 0.01), and the N400 (peak MNI coordinates: x = −15, y = 35, z = 20, log F-ratio = 1.01, P < 0.01). Whereas increases in activation during the timeframe of the N200 for the production condition tended to be more focal, the P300, and especially the N400, were characterized by increases in a widely distributed network of generators, with structures in frontal, temporal, and parietal areas demonstrating enhanced responses to memory violations in previously produced melodies.

The right supramarginal and middle temporal gyri, as well as the inferior parietal lobule, were significantly more active during the perception-altered condition than the production-altered condition within the N200 and P300 time ranges. These effects were strongest within the time range of the N200 in the supramarginal gyrus (BA 40; peak MNI coordinates: x = 50, y = −50, z = 20, log F-ratio = −1.59, P < 0.01).

Discussion

The current study examined the neural correlates of memory recognition of musical melodies learned by production (movements accompanied by auditory feedback) or by perception. The results support our principal hypothesis: amplified neural electrophysiological potentials arising from cortical motor structures in response to memory violations were associated with more accurate recognition of the violations in previously produced melodies in comparison to previously perceived melodies. Motor planning areas showed greater involvement in the early detection of pitch alterations, associated with the N200 potential, and premotor and primary motor regions were implicated in later cognitive responses to the deviant pitches, associated with P300 and N400 responses. These findings suggest that auditory–motor interactions within cortex contribute to the benefits afforded by production experience on recognition memory. The findings also support the notion that listeners' memory-based expectations influence the recognition of previously produced and perceived auditory sequences, consistent with theories of predictive motor simulation (Schubotz 2007). We first consider the behavioral findings, followed by a discussion of ERP and source localization findings.

Behavioral Findings

Production Learning Increases Recognition of Memory Violations

Pianists' learning of melodies by production increased subsequent auditory recognition of memory-violating pitch alterations in the melodies, compared with melodies that had been learned by perception-only. Previously, perceivers have been shown to recognize melodies learned by production among nonlearned distracters at higher rates than melodies learned by perception (Brown and Palmer 2012). The current finding extends this result, showing that production learning enhances subsequent recognition of single pitch alterations, consistent with a “production advantage” in memory for auditory stimuli. In the domain of language, production typically increases recognition rates by about 10% when stimuli are presented visually at test (MacLeod et al. 2010). The musical stimuli in the current study showed smaller effects (a 4% increase for production), possibly due to the difficulty of the recognition task in comparison to speech studies, although the current effects are on par with previous studies on auditory–motor learning of music (Finney and Palmer 2003; Brown and Palmer 2012). Differences between musical and linguistic stimuli may also contribute to variability in the magnitude of memory enhancement observed following production. For example, single words are capable of conveying meaning via semantics, whereas meaning in music, which is a matter of debate, is typically conveyed through a series of temporally proximal auditory events.

Close coupling of movements and auditory feedback, on which the production effect on memory may hinge, is a common feature across instances of sound production. Associations between speech-like productions and resulting sounds emerge within the first year of infancy (Kuhl and Meltzoff 1996), and manipulations of auditory feedback during adults' vocal productions yields compensatory motor adjustments (Kawahara 1994), indicating strong links between auditory feedback and movements in language speakers. Memory for music following production may also depend on the close coupling of actions with auditory feedback, as production learning under conditions in which auditory feedback is based on recordings of other performers or on computer-generated (constant pitch velocity) sounds does not yield improved recognition during subsequent perception compared with self-generated auditory feedback (Brown and Palmer 2012). Auditory–motor learning of melodies may constitute a dual load task by comparison to auditory-only learning, as musicians must learn and attend to not only the sequence of pitches comprising a melody, but also the unique sequence of movements required to produce the melody. A dual load attentional learning task would have predicted inferior recognition memory for previously produced melodies compared with previously perceived melodies. The current findings instead favor a framework in which an integrated auditory–motor memory trace (e.g., Hommel et al. 2001) is more deeply encoded and/or more easily cued for retrieval than a purely auditory memory trace. Indeed, auditory information may be processed more deeply during performance than during perception, in order to ensure the accuracy of the auditory feedback associated with one's movements (a levels-of-processing account of memory encoding; Craik and Lockhart 1972).

Increased Episodic Memory for Production Learning

Participants in the current study were also more accurate and confident in recalling the type of learning by which they had learned a given melody in the production condition compared with the perception condition. This finding suggests that sensorimotor memories encoded during the production learning condition were at least partially explicit, and mirrors the finding from the language literature that memory for whether a word has been studied by reading it aloud or reading it silently is greater for words that are learned by production, that is, reading aloud (Ozubko et al. 2012). Thus, pianists were not only able to remember “what” they learned, but also “how” they learned it, suggesting that specific episodic information was encoded during the learning phase, particularly during production learning. Production effects in the domain of language have suggested a privileged role of recollection or episodic memory over familiarity processes during recognition, as memory for produced words is enhanced on explicit but not implicit memory tests (MacDonald and MacLeod 1998). However, studies using remember/know judgments and the receiver operating characteristic procedure reveal influences of both recollection and familiarity (Ozubku et al. 2012). Taken together, prior and current findings support the domain generality of enhanced auditory recognition of self-produced auditory events, and suggest that increases in recognition accuracy can be attributed to multiple memory mechanisms.

EEG Findings

N200 Response to Memory Violations in Produced and Perceived Melodies

Memory-violating pitches in melodies learned by production and perception elicited the N200 ERP component, and production learning modulated the amplitude of this component: A larger N200 response was observed for memory violations in previously produced melodies compared with previously perceived melodies. Enlarged N200 amplitudes following production suggest a greater mismatch between auditorily presented deviants and production-based memory traces, compared with perception-based memory traces. The N200 response was associated with increased neural activation in frontal motor preparation regions including premotor and supplementary motor regions, as well as the anterior cingulate, left inferior frontal gyrus, and left precuneus. Motor preparation regions have been associated with the generation (Deiber et al. 1998), learning (Pau et al. 2013), and imagery (Lotze et al. 1999) of movement sequences, and, in tandem with the inferior frontal gyrus, auditory–motor integration (Baumann et al. 2005; Bangert et al. 2006; Lahav et al. 2007). The precuneus is activated while imagining sequences of finger taps and other types of movements (Hanakawa et al. 2003; Malouin et al. 2003), and may contribute to episodic memory retrieval, as it is engaged in processing old compared with new sentences (Tulving et al. 1994) and episodic compared with semantic memory for melodies (Platel et al. 2003). Precuneus activity is additionally associated with self-cognition, particularly in reference to autobiographical memories (Addis et al. 2007), and action simulation of oneself versus others (Ruby and Decety 2001), which could indicate greater self-related activation for a situation where one is more actively involved (performing) compared with just perceiving. The anterior cingulate has been linked to motor and perceptual error detection, and is involved in the generation of early negative potentials associated with motor errors (Gehring et al. 2012). One recent study showed anterior cingulate activation during the perception of dissonant pitch changes in recently performed musical scales, suggesting a potential role of this region in perceptual error detection in performed music (Maidhof et al. 2009).

The fronto-parietal network associated with the N200 may have aided the comparison of newly formed auditory percepts with sensorimotor stimulus representations stored in memory. Left-lateralized increases in activation for the production condition may reflect participants' right-hand learning of melodies. Whereas the response to memory violations in previously produced melodies was characterized by increases in activation in motor areas, memory violations in previously perceived melodies elicited increases in supramarginal, superior temporal, and middle temporal regions within the time range of the N200. Effects were strongest within the supramarginal and superior temporal gyri, regions implicated in pitch perception and short-term memory (Zatorre et al. 2002; Gaab et al. 2003; Schönwiesner et al. 2007). Together, these findings point toward a greater role of motor planning regions in perceptual comparisons involving sensorimotor memory traces, and a greater role of auditory regions in comparisons involving auditory-only memory traces. Networks underlying responses to memory violations in the current study resemble the antero-dorsal stream described by Rauschecker and Scott (2009), which comprises premotor, parietal, and sensory regions. Our findings support a bidirectional pathway model of sensorimotor integration, in which perceptual and motor representations interact via an antero-dorsal pathway (Rauschecker 2011).

Role of Predictive Processing in the “Production Effect” on Memory

Electrophysiological measures within the time range of the N200 did not differentiate originally learned target pitches on the basis of learning condition. This pattern of responses contrasts with the differential processing of altered pitches (i.e., memory violations) within the time frame of the N200, which showed larger amplitudes for previously produced melodies than previously perceived melodies. If learning condition alone were driving the ERP effects, we would expect a main effect of learning on N200 amplitudes. However, an interaction effect was observed, which suggests that participants predicted, or expected, upcoming pitches while listening to melodies during the memory recognition test, with the prediction processes differing across learning conditions. Pitches that matched participants' memory traces (and were therefore predicted) showed no influences of learning condition, whereas pitches that violated participants' memory traces (and were therefore unpredicted) elicited an N200 potential. This finding fits with theories of predictive processing during perception, which propose that correctly predicted events are of little informational value to perceivers, and are therefore processed less than unpredicted events (Friston 2012).

Neural models of sensorimotor prediction propose that supplementary motor and premotor areas generate an efferent copy of a predicted movement, which is passed to parietal and temporal association cortices during auditory perception (Schubotz and von Cramon 2003; Haggard and Whitford 2004; Rauschecker and Scott 2009). One implication of such forward models is that the recognition of memory-violating pitches may involve not just one, but 2 mismatch comparisons (a pitch mismatch and a movement mismatch). The current findings inform models of motor prediction by suggesting that top-down motor predictions may interface with bottom-up sensory information in premotor or supplementary motor areas, which were most strongly activated within the time range of the N200 in the production condition compared with the perception condition.

A major question regarding pitch prediction mechanisms during music perception concerns whether predictions arise from knowledge of specific tone sequences (veridical memory), from sensory information stored in short-term memory (sensory memory), or from more general knowledge of a given musical system (schematic memory). Crucially, pitch alterations in the current study were diatonic (in-key) tones, which had occurred previously in the melody, and therefore deviated only from veridical memory representations, and not from rule-based pitch schemas or sensory memory traces. These diatonic tones should therefore not give rise to violations based on predictive processes related to cognitive tonal expectancies or sensory dissonance. Violations of schematic pitch expectations in musical pitch sequences elicit early anterior negative ERPs (e.g., the early right anterior negativity [ERAN] and right anterior temporal negativity [RATN]), even when the violating pitches do not create sensory dissonance, suggesting that cognitive, rule-based expectancies are active during music perception (Koelsch et al. 2007; Marmel et al. 2011). Diatonicity of pitch alterations in the current study would minimize contributions of ERAN and/or RATN potentials. As each alteration was unique within the context of the melody in which it appeared, effects also cannot be due to the learning of target pitch locations or other information. Scalp distributions of the early negative components elicited by schematically violating pitches are typically right-lateralized (Patel et al. 1998; Koelsch et al. 2000, 2005). The negativity elicited in the current experiment showed no rightward lateralization, further suggesting the absence of ERAN and/or RATN potentials.

The mismatch negativity (MMN), which was initially (Nätäänen and Picton 1986), and is still sometimes (Patel and Azzam 2005), referred to as the N2a subcomponent of the N200, has often been used to index mechanisms underpinning sensory memory. Auditory oddball investigations have shown that the MMN is elicited by infrequent pitches within an auditory context (Sams et al. 1985). However, the N200 in the current study likely comprised N2b and N2c subcomponents (Pritchard et al. 1991), with minimal contributions of the MMN, for several reasons. First, the memory-violating pitch alterations in the current study appeared in the preceding melodic context more often on average than the corresponding contextually identical original pitches. Therefore, the altered pitches were “primed” by the melodic context more strongly than the original pitches. Consequently, the early negativity observed in response to altered pitches cannot be due to the deviance of altered pitches from an invariant melodic context or from sensory memory traces. Second, the negativity elicited by diatonically altered pitches in the current study was maximal at midline electrodes, whereas the MMN elicited by irregular pitches is typically right-lateralized (Koelsch et al. 2001; Koelsch 2009). In agreement with the current findings, Miranda and Ullman (2007) report a negativity maximal at midline electrode sites elicited by diatonic alterations in familiar melodies; the negativity elicited by nondiatonic alterations was not maximal at the midline. Third, whereas the auditory MMN is thought to be generated by predominantly auditory sensory areas, including auditory cortex (Koelsch 2009; Näätänen et al. 2001), increases in N200 amplitudes in the current study corresponded to decreases in activity within auditory sensory regions. Thus, the current findings indicate that knowledge of specific sensorimotor pitch sequences can play a role in generating predictions during perception, in the absence of schematic and sensory violations, and with no overt movements accompanying perception.

Production Learning Enhances P300 and N400 Responses to Memory Violations

Amplitudes of P300 and N400 potentials following the N200 were also enhanced by production learning compared with perception learning, and increased P300 amplitudes tended to associate with greater recognition accuracy and confidence in the production condition. Larger P300 and N400 responses to memory violations than original pitches fit with a predictive view of neural information processing, in which it is important that memory violations be attended to and encoded, in order for learning to take place (Friston and Stephan 2007).

The superior frontal gyrus, cingulate gyrus, motor planning, and primary motor areas showed strongest responses to memory violations in previously produced melodies than previously perceived melodies within the time range of the P300. Increased activation of the precentral gyrus, which contains the primary motor cortex, within the time range of the P300 for the production condition compared with the perception condition, is suggestive of motor imagery processes, as mental imagery of finger and hand movements is associated with increased primary motor cortex activation (Lotze et al. 1999; Carrillo-de-la-Peña et al. 2008). Also supporting a motor imagery account of the P300 is the finding that amplitudes within the P300 time range increased not only in response to memory violations, but also to original pitches. Motor representations of music learned by production are potentially reactivated by listening-only (cf. Bangert et al. 2001; Bangert and Altenmüller 2003; D'Ausilio et al. 2006; Lahav et al. 2007), and by auditory imagery of familiar tunes (Halpern and Zatorre 1999; Leaver et al. 2009). However, co-occurring anteriorly distributed N200 and P300 potentials have together been termed the “distraction potential,” because they appear to coincide with longer response times to visual stimuli in cross-modal auditory distracter tasks (Escera and Corral 2007), as well as autonomic nervous system responses such as heart rate deceleration and decreases in skin conductance (Lyytinen et al. 1992). The frontal regions activated within the time range of the P300 are consistent with networks activated during attentional orienting (Peelen et al. 2004), supporting an interpretation of the P300 as reflecting stimulus-driven shifts of auditory attention following unexpected sounds (Schröger and Wolff 1998; Escera et al. 2002; Rinne et al. 2006). Thus, the current findings are consistent with both motor imagery and attention capture accounts of the P300.

The finding that memory violations, as well as original pitches, elicited a larger N400 for previously produced melodies than for previously perceived melodies suggests that the produced melodic contexts were more familiar to participants. (It may appear from the grand averages that the N400 component represents the negative end of the P300 dipole. However, the observed N400 closely resembles the N400 previously reported to be elicited by veridical pitch expectancy violations in familiar music during listening, in terms of both latency and scalp distribution (Besson and Faïta 1995; Miranda and Ullman 2007). The observed N400 was also maximal within the typical time range for this component (250–500 ms following the unexpected stimulus onset) and, unlike the P300, was associated with medial temporal lobe activity, consistent with previous work suggesting that this region is critical in generating the N400 (Kutas and Federmeier 2011).) Larger N400 amplitudes are elicited by memory violations that follow highly familiar auditory contexts, compared with less familiar contexts (Miranda and Ullman 2007; Mestres-Missé et al. 2008), suggesting the N400 serves as an index of stimulus familiarity during recognition (Daltrozzo et al. 2010; Voss and Federmeier 2011). Stronger activation of the middle temporal gyrus in response to memory violations for the production condition compared with the perception condition is consistent with studies showing a critical role of the left temporal lobe in the generation of the N400, and a relation to memory recognition or retrieval (Van Petten and Luka 2006). However, the N400 is known to correspond to a widely distributed network of generators within the temporal and frontal lobes (Kutas and Federmeier 2011), as was observed here.

Response-Related Late Posterior Positivity

Memory-violating pitches in the current experiment also elicited a late posterior positive potential showing no influences of learning condition. As this positive potential was maximal at latencies more than 100 ms following the preceding anterior positivity, we interpret this potential as an LPC (as opposed to the P3b subcomponent of the P300; Squires et al. 1975). Posterior positive potentials tend to be elicited when deviant stimuli are task-relevant or response-dependent (Snyder and Hillyard 1976; Pritchard 1981), and may reflect the updating of working memory representations (context updating theory; Donchin 1981; Donchin and Coles 1988; Polich 2007). For example, a late posterior positivity was observed when participants were asked to detect altered pitches in perceived musical scales, but not when told to continue performing following an altered feedback pitch during the performance of pitch scales on a musical instrument (Maidhof et al. 2009). The LPC elicited by altered pitches in the previously produced and the previously perceived melodies may indicate participants' detection of altered (memory-violating) pitches in the memory task. The finding that LPC amplitudes did not depend on learning modality is consistent with the view that late posterior positive potentials depend on the task relevance of a stimulus (passive listening versus task-oriented listening), and not on the degree of deviation of the perceived stimulus from memory traces.

Relationship Between Musical Training and Effects of Production Experience on Memory

Our musician participants showed a positive association between recognition accuracy and musical training: More years of musical instruction correlated with improved detection of memory violations. Higher recognition accuracy, in turn, tended to be associated with larger N200 responses to memory violations. It has been suggested that auditory–motor associations may develop over time with practice (Jäncke 2012), such that expert musicians demonstrate more priming of motor responses than novices (Lappe et al. 2008). As the current study suggests motor activation as one mechanism driving the production effect, one might expect effects of production experience on recognition memory to differ across musical skill levels, depending on the state of the learned auditory–motor associations. One musical feature that engages the motor system, even when the music is novel or unfamiliar, is the musical beat, or tactus (Grahn and Brett 2007). In the case of rhythm and meter perception, musical training appears to have only moderate influences on motor activity (Chen et al. 2008), but embodiment of the musical beat or groove could perhaps also aid memory when movements are aligned with self-generated auditory signals. Future investigations could explore individual differences in neural auditory–motor interactions across production tasks varying in complexity, and test how these differences relate to memory recognition mechanisms.

Conclusions

Our findings shed new light on links between production and perception by identifying for the first time neural correlates of the “production advantage” in memory recognition. Production experience increased recognition of learned musical melodies above and beyond recognition rates achieved following listening-only experience. Production influenced electrophysiological processing of subsequently perceived memory violations at early (N200) and later (P300, N400) stages of pitch processing. Motor planning as well as primary motor regions showed greater engagement during pitch processing following production learning, suggesting a role of motor memory or simulation processes in the production effect. The N400 potential, a marker of stimulus familiarity in recognition memory, was also enhanced by production learning, suggesting increased familiarity of previously produced melodies. These findings support a role of motor prediction mechanisms in the perception of previously produced sequences, and suggest that motor memories interface with pitch percepts as early as 200 ms following sound onsets in premotor regions of cortex. Thus, responses to perceived sounds may rely not only on veridical auditory information stored in memory, but also on the sensory modality through which the sound was encoded. Increased behavioral and neural responses to previously produced stimuli advance the ideas that production experience (e.g., typing, writing, speaking, and other modes of bimodal sensory expression) can be used as a strategy for memory enhancement, and that the motor system plays an active role during the encoding as well as the perception of previously learned items.

Funding

ERASMUS MUNDUS Auditory Cognitive Neuroscience exchange grant and National Science Foundation Graduate Research Fellowship to B.M., Canada Research Chairs grant and Natural Sciences and Engineering Research Council grant (NSERC 298173) to C.P., Centre National de la Recherche Scientifique (CNRS UMR5292) to B.T.

Notes

We thank Alexandra Corneyllie, Tatiana Selchenkova, Frances Spidle, Sasha Ilnyckyj, and Alexander Demos for their assistance. This work was conducted in the framework of the LabEx CeLyA (“Centre Lyonnais d'Acoustique,” ANR-10-LABX-60) and in the Sequence Production Lab, McGill University, Canada. Conflict of Interest: None declared.

References

Addis
DR
Wong
AT
Schacter
DL
.
2007
.
Remembering the past and imagining the future: common and distinct neural substrates during event construction and elaboration
.
Neuropsychologia
 .
45
:
1363
1377
.
Bangert
M
Altenmüller
EO
.
2003
.
Mapping perception to action in piano practice: a longitudinal DC-EEG study
.
BMC Neurosci
 .
4
:
26
.
Bangert
M
Haeusler
U
Altenmüller
E
.
2001
.
On practice: how the brain connects piano keys and piano sounds
.
Ann N Y Acad Sci
 .
930
:
425
428
.
Bangert
M
Peschel
T
Schlaug
G
Rotte
M
Drescher
D
Hinrichs
H
Heinze
H
Altenmüller
E
.
2006
.
Shared networks for auditory and motor processing in professional pianists: evidence from fMRI conjunction
.
Neuroimage
 .
30
:
917
926
.
Baumann
S
Koeneke
S
Meyer
M
Lutz
K
Jäncke
L
.
2005
.
A network for sensory-motor integration
.
Ann N Y Acad Sci
 .
1060
:
186
188
.
Besson
M
Faïta
F
.
1995
.
An event-related potential (ERP) study of musical expectancy: comparison of musicians with nonmusicians
.
J Exp Psychol Hum Percept Perform
 .
21
:
1278
1296
.
Brown
RM
Palmer
C
.
2012
.
Auditory–motor learning influences auditory memory for music
.
Mem Cogn
 .
40
:
567
578
.
Brown
S
Martinez
MJ
.
2007
.
Activation of premotor vocal areas during musical discrimination
.
Brain Cogn
 .
63
:
59
69
.
Carrillo-de-la-Peña
MT
Galdo-Alvarez
S
Lastra-Barreira
C
.
2008
.
Equivalent is not equal: primary motor cortex (MI) activation during motor imagery and execution of sequential movements
.
Brain Res
 .
1226
:
134
143
.
Chen
JL
Penhune
VB
Zatorre
RJ
.
2008
.
Listening to musical rhythms recruits motor regions of the brain
.
Cereb Cortex
 .
18
:
2844
2854
.
Conway
MA
Gathercole
SE
.
1987
.
Modality and long-term memory
.
J Mem Lang
 .
26
:
341
361
.
Craik
FIM
Lockhart
RS
.
1972
.
Levels of processing: a framework for memory research
.
J Verb Learn Verb Behav
 .
11
:
671
684
.
Daltrozzo
J
Tillmann
B
Platel
H
Schön
D
.
2010
.
Temporal aspects of the feeling of familiarity for music and the emergence of conceptual processing
.
J Cogn Neurosci
 .
22
:
1754
1769
.
D'Ausilio
A
Altenmüller
E
Olivetti Belardinelli
M
Lotze
M
.
2006
.
Cross-modal plasticity of the motor cortex while listening to a rehearsed musical piece
.
Eur J Neurosci
 .
24
:
955
958
.
Deiber
MP
Ibanez
V
Honda
M
Sadato
N
Raman
R
Hallett
M
.
1998
.
Cerebral processes related to visuomotor imagery and generation of simple finger movements studied with positron emission tomography
.
Neuroimage
 .
7
:
73
85
.
Dodson
CS
Schacter
DL
.
2001
.
If I had said it I would have remembered it: reducing false memories with a distinctiveness heuristic
.
Psychon Bull Rev
 .
8
:
155
161
.
Donchin
E
.
1981
.
Surprise!… surprise?
Psychophysiology
 .
18
:
493
513
.
Donchin
E
Coles
MG
.
1988
.
Is the P300 component a manifestation of context updating?
Behav Brain Sci
 .
11
:
357
427
.
Draganski
B
May
A
.
2008
.
Training-induced structural changes in the adult human brain
.
Behav Brain Res
 .
192
:
137
142
.
Escera
C
Corral
MJ
.
2007
.
Role of mismatch negativity and novelty-P3 in involuntary auditory attention
.
Int J Psychophysiol
 .
21
:
251
264
.
Escera
C
Corral
MJ
Yago
E
.
2002
.
An electrophysiological and behavioral investigation of involuntary attention towards auditory frequency, duration and intensity changes
.
Cogn Brain Res
 .
14
:
325
332
.
Federmeier
KD
.
2007
.
Thinking ahead: the role and roots of prediction in language comprehension
.
Psychophysiology
 .
44
:
491
505
.
Fiebach
CJ
Schubotz
RI
.
2006
.
Dynamic anticipatory processing of hierarchical sequential events: a common role for Broca's area and ventral premotor cortex across domains?
Cortex
 .
42
:
499
502
.
Finney
S
Palmer
C
.
2003
.
Auditory feedback and memory for music performance: sound evidence for an encoding effect
.
Mem Cogn
 .
31
:
51
64
.
Folstein
JR
Van Petten
C
.
2008
.
Influence of cognitive control and mismatch on the N2 component of the ERP: a review
.
Psychophysiology
 .
45
:
152
170
.
Friston
K
.
2012
.
Prediction, perception and agency
.
Int J Psychophysiol
 .
83
:
248
252
.
Friston
KJ
Stephan
KE
.
2007
.
Free-energy and the brain
.
Synthese
 .
159
:
417
458
.
Fuchs
M
Kastner
J
Wagner
M
Hawes
S
Ebersole
JS
.
2002
.
A standardized boundary element method volume conductor model
.
Clin Neurophysiol
 .
113
:
702
712
.
Gaab
N
Gaser
C
Zaehle
T
Jäncke
L
Schlaug
G
.
2003
.
Functional anatomy of pitch memory: an fMRI study with sparse temporal sampling
.
Neuroimage
 .
19
:
1417
1426
.
Gathercole
SE
Conway
MA
.
1988
.
Exploring long-term modality effects: vocalization leads to best retention
.
Mem Cogn
 .
16
:
110
119
.
Gehring
WJ
Liu
Y
Orr
JM
Carp
J
.
2012
.
The error-related negativity (ERN/Ne)
. In:
Luck
SJ
Kappenman
E
, editors.
Oxford handbook of event-related potential components
 .
New York
:
Oxford University Press
. p.
231
291
.
Golestani
N
Zatorre
RJ
.
2004
.
Learning new sounds of speech: reallocation of neural substrates
.
Neuroimage
 .
21
:
494
506
.
Grahn
JA
Brett
M
.
2007
.
Rhythm and beat perception in motor areas of the brain
.
J Cogn Neurosci
 .
19
:
893
906
.
Haggard
P
Whitford
B
.
2004
.
Supplementary motor area provides an efferent signal for sensory suppression
.
Cogn Brain Res
 .
19
:
52
58
.
Halpern
AR
Zatorre
RJ
.
1999
.
When that tune runs through your head: a PET investigation of auditory imagery for familiar melodies
.
Cereb Cortex
 .
9
:
697
704
.
Hanakawa
T
Immisch
I
Toma
K
Dimyan
MA
Van Gelderen
P
Hallett
M
.
2003
.
Functional properties of brain areas associated with motor execution and imagery
.
J Neurophysiol
 .
89
:
989
1002
.
Haslinger
B
Erhard
P
Altenmüller
E
Schroeder
U
Boecker
H
Ceballos-Baumann
AO
.
2005
.
Transmodal sensorimotor networks during action observation in professional pianists
.
J Cogn Neurosci
 .
17
:
282
293
.
Hintzman
DL
.
1976
.
Repetition and memory
. In:
Bower
GH
, editor.
The psychology of learning and motivation: advances in research and theory
 .
New York
:
Academic Press
. p.
47
91
.
Hommel
B
Müsseler
J
Aschersleben
G
Prinz
W
.
2001
.
The Theory of Event Coding (TEC): a framework for perception and action planning
.
Behav Brain Sci
 .
24
:
849
937
.
Hund-Georgiadis
M
Von Cramon
DY
.
1999
.
Motor-learning-related changes in piano players and non-musicians revealed by functional magnetic-resonance signals
.
Exp Brain Res
 .
125
:
417
425
.
Hyde
KL
Lerch
J
Norton
A
Forgeard
M
Winner
E
Evans
AC
Schlaug
G
.
2009
.
Musical training shapes structural brain development
.
J Neurosci
 .
29
:
3019
3025
.
Jäncke
L
.
2012
.
The dynamic audio–motor system in pianists
.
Ann N Y Acad Sci
 .
1252
:
246
252
.
Jäncke
L
Shah
NJ
Peters
M
.
2000
.
Cortical activations in primary and secondary motor areas for complex bimanual movements in professional pianists
.
Brain Res Cogn Brain Res
 .
10
:
177
183
.
Jeannerod
M
.
2001
.
Neural simulation of action: a unifying mechanism for motor cognition
.
Neuroimage
 .
14
:
S103
S109
.
Johnson
SH
.
1998
.
Cerebral organization of motor imagery: contralateral control of grip selection in mentally represented prehension
.
Psychol Sci
 .
9
:
219
222
.
Jones
MR
Boltz
M
.
1989
.
Dynamic attending and responses to time
.
Psychol Rev
 .
96
:
459
491
.
Jurcak
V
Tsuzuki
D
Dan
I
.
2007
.
10/20, 10/10, and 10/5 systems revisited: their validity as relative head-surface-based positioning systems
.
Neuroimage
 .
34
:
1600
1611
.
Kawahara
H
.
1994
.
Interactions between speech production and perception under auditory feedback perturbations on fundamental frequencies
.
J Acoust Soc Jpn
 .
15
:
201
202
.
Koelsch
S
.
2009
.
Music-syntactic processing and auditory memory: similarities and differences between ERAN and MMN
.
Psychophysiology
 .
46
:
179
190
.
Koelsch
S
Fritz
T
Müller
K
Friederici
AD
.
2006
.
Investigating emotion with music: an fMRI study
.
Hum Brain Mapp
 .
27
:
239
250
.
Koelsch
S
Gunter
TC
Friederici
AD
Schröger
E
.
2000
.
Brain indices of music processing: “non-musicians” are musical
.
J Cogn Neurosci
 .
12
:
520
541
.
Koelsch
S
Gunter
T
Schröger
E
Tervaniemi
M
Sammler
D
Friederici
AD
.
2001
.
Differentiating ERAN and MMN: an ERP study
.
Neuroreport
 .
12
:
1385
1389
.
Koelsch
S
Gunter
TC
Wittfoth
M
Sammler
D
.
2005
.
Interaction between syntax processing in language and in music: an ERP study
.
J Cogn Neurosci
 .
17
:
1565
1577
.
Koelsch
S
Jentschke
S
Sammler
D
Mietchen
D
.
2007
.
Untangling syntactic and sensory processing: an ERP study of syntactic and sensory processing
.
Psychophysiology
 .
44
:
476
490
.
Kuhl
PK
Meltzoff
AN
.
1996
.
Infant vocalizations in response to speech: vocal imitation and developmental change
.
J Acoust Soc Am
 .
100
:
2425
.
Kutas
M
Federmeier
KD
.
2011
.
Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP)
.
Annu Rev Psychol
 .
62
:
621
647
.
Lahav
A
Saltzman
E
Schlaug
G
.
2007
.
Action representation of sound: audiomotor recognition network while listening to newly acquired actions
.
J Neurosci
 .
27
:
308
314
.
Lancaster
JL
Woldorff
MG
Parsons
LM
Liotti
M
Freitas
CS
Rainey
L
Kochunov
PV
Nickerson
D
Mikiten
SA
Fox
PT
.
2000
.
Automated Talairach Atlas labels for functional brain mapping
.
Hum Brain Mapp
 .
10
:
120
131
.
Lappe
C
Herholz
SC
Trainor
LJ
Pantev
C
.
2008
.
Cortical plasticity induced by short-term unimodal and multimodal musical training
.
J Neurosci
 .
28
:
9632
9639
.
Large
EW
.
1993
.
Dynamic programming for the analysis of serial behaviors
.
Behav Res Meth Instrum Comput
 .
25
:
238
241
.
Leaver
AM
Van Lare
J
Zielinski
B
Halpern
AR
Rauschecker
JP
.
2009
.
Brain activation during anticipation of sound sequences
.
J Neurosci
 .
29
:
2477
2485
.
Lerdahl
F
Jackendoff
RS
.
1983
.
A generative theory of tonal music
.
Cambridge, MA
:
MIT Press
.
Lotze
M
.
2013
.
Kinesthetic imagery of musical performance
.
Front Hum Neurosci
 .
7
:
280
. .
Lotze
M
Montoya
P
Erb
M
Hülsmann
E
Flor
H
Klose
U
Birbaumer
N
Grodd
W
.
1999
.
Activation of cortical and cerebellar motor areas during executed and imagined hand movements: an fMRI study
.
J Cogn Neurosci
 .
11
:
491
501
.
Lyytinen
H
Blomberg
AP
Näätänen
R
.
1992
.
Event-related potentials and autonomic responses to a change in unattended auditory stimuli
.
Psychophysiology
 .
29
:
523
534
.
MacDonald
PA
MacLeod
CM
.
1998
.
The influence of attention at encoding on direct and indirect remembering
.
Acta Psychol
 .
98
:
291
310
.
MacLeod
CM
Gopie
N
Hourihan
KL
Neary
KR
Ozubko
JD
.
2010
.
The production effect: delineation of a phenomenon
.
J Exp Psychol Learn Mem Cogn
 .
36
:
671
685
.
Maidhof
C
Vavatzanidis
N
Prinz
W
Rieger
M
Koelsch
S
.
2009
.
Processing expectancy violations during music performance and perception: an ERP study
.
J Cogn Neurosci
 .
22
:
2401
2413
.
Malouin
F
Richards
CL
Jackson
PL
Dumas
F
Doyon
J
.
2003
.
Brain activations during motor imagery of locomotor-related tasks: a PET study
.
Hum Brain Mapp
 .
19
:
47
62
.
Mandler
G.
(
1967
).
Organization and memory
. In
Spence
KW
Spence
JT
, editors.
The psychology of learning and motivation: advances in research and theory
 .
New York
:
Academic Press
. p.
328
372
.
Marmel
F
Perrin
F
Tillmann
B
.
2011
.
Tonal expectations influence early pitch processing
.
J Cogn Neurosci
 .
23
:
3095
3104
.
Mazziotta
J
Toga
A
Evans
A
Fox
P
Lancaster
J
Zilles
K
Woods
R
Paus
T
Simpson
G
Pike
B
et al
2001
.
A probabilistic atlas and reference system for the human brain: International Consortium for Brain Mapping (ICBM)
.
Philos Trans R Soc Lond B Biol Sci
 .
356
:
1293
1322
.
Mestres-Missé
A
Càmara
E
Rodriguez-Fornells
A
Rotte
M
Münte
TF
.
2008
.
Functional neuroanatomy of meaning acquisition from context
.
J Cogn Neurosci
 .
20
:
2153
2166
.
Miranda
RA
Ullman
MT
.
2007
.
Double dissociation between rules and memory in music: an event-related potential study
.
Neuroimage
 .
38
:
331
345
.
Mobascher
A
Brinkmeyer
J
Warbrick
T
Musso
F
Wittsack
HJ
Stoermer
R
Saleh
A
Schnitzler
A
Winterer
G
.
2009
.
Fluctuations in electrodermal activity reveal variations in single trial brain responses to painful laser stimuli: a fMRI/EEG study
.
Neuroimage
 .
44
:
1081
1092
.
Näätänen
R
Picton
TW
.
1986
.
N2 and automatic versus controlled processes
.
Electroencephalogr Clin Neurophysiol Suppl
 .
38
:
169
186
.
Näätänen
R
Tervaniemi
M
Sussman
E
Paavilainen
P
Winkler
I
.
2001
.
‘Primitive intelligence’ in the auditory cortex
.
Trends Neurosci
 .
24
:
283
288
.
Näätänen
R
Winkler
I
.
1999
.
The concept of auditory stimulus representation in cognitive neuroscience
.
Psychol Bull
 .
125
:
826
.
Navarro Cebrian
A
Janata
P
.
2010
.
Electrophysiological correlates of accurate mental image formation in auditory perception and imagery tasks
.
Brain Res
 .
1342
:
39
54
.
Nieuwenhuis
S
Aston-Jones
G
Cohen
JD
.
2005
.
Decision making, the P3, and the locus coeruleus-norepinephrine system
.
Psychol Bull
 .
131
:
510
532
.
Olbrich
S
Mulert
C
Karch
S
Trenner
M
Leicht
G
Pogarell
O
Hegerl
U
.
2009
.
EEG-vigilance and BOLD effect during simultaneous EEG/fMRI measurements
.
Neuroimage
 .
45
:
319
332
.
Oostenveld
R
Praamstra
P
.
2001
.
The five percent electrode system for high-resolution EEG and ERP measurements
.
Clin Neurophysiol
 .
112
:
713
719
.
Opacic
T
Stevens
K
Tillmann
B
.
2009
.
Knowledge unspoken: implicit learning of structured human dance movement
.
J Exp Psychol Learn Mem Cogn
 .
35
:
1570
1577
.
Ozubko
JD
MacLeod
CM
.
2010
.
The production effect in memory: evidence that distinctiveness underlies the benefit
.
J Exp Psychol Learn Mem Cogn
 .
36
:
1543
1547
.
Ozubko
JD
Gopie
N
MacLeod
CM
.
2012
.
Production benefits both recollection and familiarity
.
Mem Cogn
 .
40
:
326
338
.
Palmer
C
.
1997
.
Music performance
.
Annu Rev Psychol
 .
48
:
115
138
.
Pascual-Marqui
RD
.
2002
.
Standardized low-resolution brain electromagnetic tomography (sLORETA): technical details
.
Methods Find Exp Clin Pharmacol
 .
24D
:
5
12
.
Patel
AD
Gibson
E
Ratner
J
Besson
M
Holcomb
PJ
.
1998
.
Processing syntactic relations in language and music: an event-related potential study
.
J Cogn Neurosci
 .
10
:
717
733
.
Patel
SH
Azzam
PN
.
2005
.
Characterization of N200 and P300: selected studies of the event-related potential
.
Int J Med Sci
 .
2
:
147
154
.
Pau
S
Jahn
G
Sakreida
K
Domin
M
Lotze
M
.
2013
.
Encoding and recall of finger sequences in experienced pianists compared with musically naïve controls: a combined behavioral and functional imaging study
.
Neuroimage
 .
64
:
379
387
.
Peelen
MV
Heslenfeld
DJ
Theeuwes
J
.
2004
.
Endogenous and exogenous attention shifts are mediated by the same large-scale neural network
.
Neuroimage
 .
22
:
822
830
.
Pizzagalli
DA
Pascual-Marqui
RD
Nitschke
JB
Oakes
TR
Larson
CL
Abercrombie
HC
Schaefer
SM
Koger
JV
Benca
RM
Davidson
RJ
.
2001
.
Anterior cingulate activity as a predictor of degree of treatment response in major depression: evidence from brain electrical tomography analysis
.
Am J Psychiatry
 .
158
:
405
415
.
Platel
H
Baron
JC
Desgranges
B
Bernard
F
Eustache
F
.
2003
.
Semantic and episodic memory of music are subserved by distinct neural networks
.
Neuroimage
 .
20
:
244
256
.
Polich
J
.
2007
.
Updating P300: an integrative theory of P3a and P3b
.
Clin Neurophysiol
 .
118
:
2128
2148
.
Pritchard
WS
.
1981
.
Psychophysiology of P300
.
Psychol Bull
 .
89
:
506
540
.
Pritchard
WS
Shappell
SA
Brandt
ME
.
1991
.
Psychophysiology of N200/N400: a review and classification scheme
. In:
Ackles
PK
Jennings
JR
Coles
MGH
, editors.
Advances in psychophysiology (Vol. 4)
 .
Greenwich, CT
:
JAI Press
. p.
43
106
.
Rauschecker
JP
.
2011
.
An expanded role for the dorsal auditory pathway in sensorimotor control and integration
.
Hear Res
 .
271
:
16
25
.
Rauschecker
JP
Scott
SK
.
2009
.
Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing
.
Nat Neurosci
 .
12
:
718
724
.
Rinne
T
Särkkä
A
Degerman
A
Schröger
E
Alho
K
.
2006
.
Two separate mechanisms underlie auditory change detection and involuntary control of attention
.
Brain Res
 .
1077
:
135
143
.
Ruby
P
Decety
J
.
2001
.
Effect of subjective perspective taking during simulation of action: a PET investigation of agency
.
Nat. Neurosci
 .
4
:
546
550
.
Sams
M
Paavilainen
P
Alho
K
Näätänen
R
.
1985
.
Auditory frequency discrimination and event-related potentials
.
Electroencephalogr Clin Neurophysiol
 .
62
:
437
448
.
Schlaug
G
.
2001
.
The brain of musicians
.
Ann N Y Acad Sci
 .
930
:
281
299
.
Schönwiesner
M
Novitski
N
Pakarinen
S
Carlson
S
Tervaniemi
M
Näätänen
R
.
2007
.
Heschl's gyrus, posterior superior temporal gyrus, and mid-ventrolateral prefrontal cortex have different roles in the detection of acoustic changes
.
J Neurophysiol
 .
97
:
2075
2082
.
Schröger
E
Wolff
C
.
1998
.
Attentional orienting and reorienting is indicated by human event-related brain potentials
.
Neuroreport
 .
9
:
3355
3358
.
Schubotz
RI
.
2007
.
Prediction of external events with our motor system: towards a new framework
.
Trends Cogn Sci
 .
11
:
211
218
.
Schubotz
RI
von Cramon
DY
.
2003
.
Functional–anatomical concepts of human premotor cortex: evidence from fMRI and PET studies
.
Neuroimage
 .
20
:
S120
S131
.
Snyder
E
Hillyard
SA
.
1976
.
Long-latency evoked potentials to irrelevant, deviant stimuli
.
Behav Biol
 .
16
:
319
331
.
Squires
NK
Squires
KC
Hillyard
SA
.
1975
.
Two varieties of long-latency positive waves evoked by unpredictable auditory stimuli in man
.
Electroencephalogr Clin Neurophysiol
 .
38
:
387
401
.
Tillmann
B
.
2012
.
Music and language perception: expectations, structural integration, and cognitive sequencing
.
Top Cogn Sci
 .
4
:
568
584
.
Tulving
E
Kapur
S
Craik
FI
Moscovitch
M
Houle
S
.
1994
.
Hemispheric encoding/retrieval asymmetry in episodic memory: positron emission tomography findings
.
Proc Natl Acad Sci USA
 .
91
:
2016
2020
.
Ungerleider
LG
Doyon
J
Karni
A
.
2002
.
Imaging brain plasticity during motor skill learning
.
Neurobiol Learn Mem
 .
78
:
553
564
.
Van Petten
C
Luka
BJ
.
2006
.
Neural localization of semantic context effects in electromagnetic and hemodynamic studies
.
Brain Lang
 .
97
:
279
293
.
Voss
JL
Federmeier
KD
.
2011
.
FN400 potentials are functionally identical to N400 potentials and reflect semantic processing during recognition testing
.
Psychophysiology
 .
48
:
532
546
.
Zatorre
RJ
Belin
P
Penhune
VB
.
2002
.
Structure and function of auditory cortex: music and speech
.
Trends Cogn Sci
 .
6
:
37
46
.
Zatorre
RJ
Chen
JL
Penhune
VB
.
2007
.
When the brain plays music: auditory-motor interactions in music perception and production
.
Nat Rev Neurosci
 .
8
:
547
558
.
Zumsteg
D
Lozano
AM
Wennberg
RA
.
2006
.
Depth electrode recorded cerebral responses with deep brain stimulation of the anterior thalamus for epilepsy
.
Clin Neurophysiol
 .
117
:
1602
1609
.