Abstract

Dense-array electrocortical and functional hemodynamic measures of human brain activity were collected to assess the relationship between 2 established neural measures of emotional reactivity. Recorded in parallel sessions, the slow-wave late positive potential (LPP) and visual cortical blood oxygen level–dependent (BOLD) signals were both modulated by the rated intensity of picture arousal. The amplitude of the LPP correlated significantly with BOLD intensity in lateral occipital, inferotemporal, and parietal visual areas across picture contents. Estimated strength of modeled regional sources did not correlate significantly with regional BOLD intensity. These data suggest that the enhanced positive slow wave seen over posterior sites during emotional picture processing represents activity in a circuit of visual cortical structures, reflecting a perceptual sensitivity to the motivational relevance of visual scenes.

Introduction

Currently, the most common methods of assessing emotional processing in the human brain are event-related potentials (ERP), derived from the electroencephalograph (EEG), and blood oxygen level–dependent (BOLD) contrast, assessed with functional magnetic resonance imaging (fMRI). Nevertheless, there are relatively few data explicating the relationship between these measures. Their relationship is of particular interest as the 2 measures are complementary, with ERPs monitoring neural change in real time, and fMRI providing superior spatial localization. Given that the processing of emotional stimuli is associated with increased BOLD signal in multiple structures, it is of great interest to determine which, if any, of the neural structures correspond with ERPs measured at the scalp and thus reflect common neural activation.

When participants view photographs that vary in emotional impact, a centroparietal ERP can be recorded at the scalp that is characterized by a slow positive-going voltage, beginning roughly 400 ms after picture onset and continuing until picture offset. The amplitude of this waveform, or late positive potential (LPP), is systematically related to participants' ratings of their experience of emotional arousal (Cacioppo and others 1994; Cuthbert and others 2000; Keil and others 2002; Schupp and others 2004). Sensitivity to picture arousal is also evident in increased BOLD signal across extrastriate occipital, parietal, and inferior temporal visual cortex (Breiter and others 1996; Lang and others 1998; Sprengelmeyer and others 1998; Bradley and others 2003; Sabatinelli and others 2004). These effects have been interpreted as reflecting enhanced processing of motivationally relevant stimuli and may reflect subcortical reentrant feedback (Spiegler and Mishkin 1981; Amaral and Price 1984; Lang and others 1997; Davis and Lang 2003; Sabatinelli and others 2005).

Studies of simple sensory and motor tasks have reported generally good correspondence between electrocortical and hemodynamic measures (Stippich and others 1998; Arthurs and others 2000; Arthurs and Boniface 2003; Singh and others 2003). However, less consistent correspondence has been reported in other complex sensory and cognitive tasks (Horowitz and others 2002; Vitacco and others 2002; Foucher and others 2003; Arthurs and others 2004). For example, a small set of intracranial electrodes implanted in inferior temporal visual cortex were virtually silent during complex visual processing—a task in which BOLD signal in that cortical region is abundant (Huettel and others 2004). Thus, although past work suggests that both measures will be similarly responsive to emotionally arousing pictures, the actual relationship between real-time neural voltage and hemodynamic BOLD signal remains unclear.

The present research examines the brain's response when presented with emotionally arousing (pleasant and unpleasant) and neutral picture stimuli, assessing scalp-recorded EEG (256 channel) and fMRI in the same experimental paradigm. The primary aim is to assess the potential correspondence between the scalp-recorded LPP and fMRI BOLD signal from the visual cortex, both acquired during emotional picture processing. If individuals tend to show consistent patterns of modulation in electrocortical and hemodynamic measures of pleasant, neutral, and unpleasant picture processing, we would expect the group average correlation to be significant. If the 2 measures are independently modulated, no correlation should be evident. In additional analyses (taking advantage of the dense sensor array), correlations were separately assessed between BOLD signal and 2 other measures derived from the EEG—source modeling of the LPP and regional time-frequency modulation (EEG spectral power).

Methods

Participants

Eighteen undergraduate and graduate psychology students (10 women, mean age 20 years, standard deviation 2.6 years) participated in both fMRI and EEG experimental sessions and were compensated $40 USD (a subset of fMRI data from the current sample [10 of 16 subjects] was also part of a larger [18 subjects] published study [Sabatinelli and others 2005]). Two participants' data were excluded due to excessive head motion in the magnetic resonance (MR) scanner or ocular artifact in the EEG. All volunteers were screened for claustrophobia and reported remaining awake and alert throughout both procedures.

Stimuli and Procedure

The picture series consisted of a pseudorandomly ordered selection of 60 color pictures from the International Affective Picture System (IAPS; Lang and others 1999) including 20 pleasant, 20 neutral, and 20 unpleasant scenes (IAPS stimuli used in this experiment include pleasant: 4611, 4641, 4658, 4659, 4666, 4676, 4677, 4680, 4681, 4690, 1440, 1460, 1463, 1530, 1540, 1590, 1610, 1710, 1750, 1920; neutral: 2191, 2214, 2215, 2372, 2383, 2393, 2394, 2480, 2595, 7550, 5740, 7036, 7041, 7050, 7100, 7130, 7161, 7224, 7234, 7500; unpleasant: 1010, 1019, 1050, 1052, 1090, 1110, 1111, 1113, 1114, 1120, 3000, 3051, 3060, 3068, 3069, 3071, 3100, 3101, 3266, 3400). IAPS normative ratings (Lang and others 1999) for arousal (1 = least arousing, 9 = most arousing) averaged 5.32, 3.26, and 6.41 and for valence (1 = most unpleasant, 9 = most pleasant) averaged 7.30, 4.30, and 3.46. Picture stimuli were also balanced by category to be equivalent in brightness, contrast, mean spatial frequency, and 90% quality JPEG file size. Pictures were rear projected (Kodak Ektapro 9000), subtending a 20∫ visual angle, on a translucent screen situated at the subject's feet, visible via an adjustable mirror. Stimulus presentation was time locked to data acquisition by a PC compatible computer interfaced to the MR scanner/EEG amplifier, running Virtual Psychological Monitor (VPM) stimulus control software (Cook and others 1987). The picture series was divided into 2 9-min blocks of 30 pictures each, separated by a rest period of approximately 1 min. Participants were asked to remain still and fixate on a laser point at the center of the slide screen throughout 60 picture (6 s) and dark-screen (12 s) intervals.

fMRI Acquisition and Analysis

The scanning sequence began with the acquisition of an 8 min, 90 slice sagittal volume, using a standard T1-weighted sequence on a 3-T General Electric (GE) Signa scanner and a specialized quadrature head coil (Fitzsimmons and others 1997). This volume was used to specify the location of a 10-slice coronal functional prescription, acquired using a T2*-weighted gradient echo, echo planar sequence (64 × 64, 20-cm field of view 90° flip angle, echo time 40ms, time repetition 1500 ms). The prescription originated 1 cm anterior to the occipital pole and extended anteriorly (5-mm slices, 1-mm gap). Trials with head motion greater than 1 mm (<2%) were removed from the time series. The raw data were then slice-time corrected, linearly detrended, high-pass filtered at 3 Hz, spatially filtered with a 2 voxel (6.25 mm) full-width at half maximum kernel, and coregistered with each participant's structural volume using BrainVoyager 4.6 (Brain Innovation, Maastrict, The Netherlands, www.brainvoyager.com). These volumes and coregistered functional data were then transformed into standardized coordinate space (Talairach and Tournoux 1988).

BOLD Regions of Interest

For each participant, an analysis of variance (ANOVA) identified voxels following the time course of picture presentation, after convolution with a hemodynamic response function. These individual functional maps were thresholded at an uncorrected alpha level of P < 0.000001, and 100-ml regions of interest (ROIs) were sampled bilaterally in each participant's significant voxels within the lateral occipital cortex, inferotemporal cortex, medial parietal precuneus, and primary visual cortex. All correlation analyses were performed with ROI data averaged across left and right hemispheres. These ROIs have been shown to be active during complex picture processing in past studies (Bradley and others 2003; Sabatinelli and others 2004). The choice of the comparatively small, but individually significant 100-ml ROI volume enabled all subjects to contribute equal and meaningful regional values. ROI signal change scores were calculated from 6 to 13.5 s after picture onset (the peak of signal change), using the scan immediately preceding each picture onset as baseline. Thus the signal change score reflects an increase or decrease relative to the immediately preceding intensity value.

EEG Acquisition and Analysis

Dense-array scalp EEG was continuously recorded from 257 sensors (Electrical Geodesics, Inc., Eugene, OR; www.egi.com) throughout the picture series. The data were digitized at 250 Hz, band-pass filtered between 0.1 and 100 Hz. The recording reference electrode was Cz, and all channels were rereferenced to the average off-line. Epochs extending from 200 ms prior to 1 s after picture onset were selected and analyzed with Brain Electrical Source Analysis (BESA) 5.0 (Megis Software, www.besa.de). Using the standard procedures implemented in BESA software, ocular and cardiac artifacts were corrected, and trials with excessive motion artifact were excluded, with no participant's data retaining less than 75% of trials for each picture category.

ERP Source Modeling

To model the group average scalp topography, 9 discrete regional sources were placed in a 4-shell ellipsoidal head model (Scherg and Von Cramon 1986). Regional sources as used here do not provide precise neuroanatomical localization but rather represent the multi-electrode ERP data in an anatomically meaningful low-dimensional space, which overcomes some of the problems associated with using voltage maps (Scherg and others 2002). As a key advantage, the interpretation of source current amplitude enhancement as an increase in brain electric activity is possible, whereas in voltage maps, the polarity of deflections often poses problems in interpretation (e.g., dependency of the reference electrode). Regional sources in BESA are sensitive to current changes irrespective of their orientation in space and therefore capture electrocortical activity originating from a wider range of areas. Regional sources therefore represent an ideal tool for representing distributed cortical activity, as can be expected in the present study. When using this technique, it is important to ensure that activity is not misrepresented in another cortical area, far from the true underlying source. We used source sensitivity maps to ensure that our model (see below) as a whole provides spatial specificity, such that the modeled sources are primarily sensitive to local activity. This can be achieved by using a sufficient number of model sources, evenly distributed across the cortex.

The model used here includes a medial primary visual source and 4 bilateral sources in inferior temporal, parietal, central, and frontal areas, resulting in 9 regional sources. The distance from midline was guided by the center of mass Talairach locations of significant fMRI BOLD clusters seen in this sample in inferior temporal and parietal cortex, however, due to the restraints of source modeling (approximate cortical equidistance, nonoverlapping spatial sensitivity), the locations of BOLD ROIs and electrocortical sources cannot be exactly the same. Given the probabilistic nature of source modeling using scalp-recorded EEG, the minor offsets are not critical.

Sensitivities for this source model were spatially specific (i.e., limited overlap, see Fig. 4). This 9-source model was applied to each participant's picture category average, resulting in a time course of source strength at each regional source for each picture condition. The average of this time course from 400 to 900 ms after picture onset was output for statistical testing and correlation with BOLD signal.

EEG Spectral Power

Time-frequency maps were calculated for 15 brain regions with BESA's source coherence module, from 4 to 30 Hz in 2 Hz increments, deviating from a 100-ms baseline prior to picture onset and extending 900 ms into the picture interval. The alpha (8–12 Hz) and beta (18–26 Hz) spectral changes in each region were output for each participant's average EEG change for each picture category, from 400 to 900 ms after picture onset. Given the signal-to-noise ratio inherent in the present data, we did not extend these analyses to other frequency ranges, which generally showed less pronounced signal changes to affective pictures.

EEG Acquisition Environment: The MR Simulator

A 3-T GE Signa scanner front fascia was acquired and provides the basis for the realistic appearance of an MR simulator (Fig. 1). The fascia is mounted on a wooden framework, with a fiberglass bore, including a removable hatch above the subject's head to allow adjustment of recording equipment and face-to-face communication. A rolling fiberglass bed rides within a curved channel in both the table and bore, driven by belt and electric motor from controls at the front of the simulator, as on the GE scanner (Sabatinelli and others 2000).

Figure 1.

Comparison of the 3-T GE Signa scanner used in the collection of the hemodynamic data and the MR simulator used for the collection of the dense-array EEG data. Virtually all physical and perceptual characteristics are matched for the 2 environments.

Figure 1.

Comparison of the 3-T GE Signa scanner used in the collection of the hemodynamic data and the MR simulator used for the collection of the dense-array EEG data. Virtually all physical and perceptual characteristics are matched for the 2 environments.

The noise of structural and functional scanning sequences was digitally recorded and reproduced using VPM software on a PC computer (Cook and others 1987). The digital sound file was preamplifed, adjusted in frequency range to compensate for the masking effects of the fiberglass bore, and amplified with a Crown CE2000 stage amplifier (www.crownaudio.com) to the same intensity as would be perceived by an earplugged subject in the MR scanner. Twin Bose 502c speaker arrays (www.bose.com) were situated outside of the simulator bore, directed at the head of the subject. A single Bose 802c midrange speaker cabinet was butted against a structural crossmember of the simulator, thus providing both sound and vibration to the subject. Extensive electromagnetic field (EMF) shielding was performed on the audio equipment and bore tube to limit the introduction of electromagnetic noise into the EEG.

Results

Visual Cortical BOLD Signal

A random-effects ANOVA of picture-driven signal change in the group-averaged functional volume revealed large clusters of significant activity in pericalcarine cortex (Talairach coordinates 3, −93, −3), lateral occipital (±38, −87, 12), medial parietal precuneus (±21, −72, 42), and inferior temporal cortex (±39, −64, −12). Figure 5b shows a coronal cross section through significant bilateral inferior temporal and parietal clusters. ROI analyses sampling these areas within each subject revealed that signal intensity was sensitive to picture valence, F3,13 = 15.97, P < 0.001, showing stronger increases during pleasant and unpleasant, relative to neutral picture presentations (quadratic trend F1,15 = 27.22, P < 0.001). Signal change values for each ROI and picture condition are shown in Table 1, with quadratic trend tests for each ROI. No linear trend tests were significant, thus the enhancement of ROI intensity was equivalent for pleasant and unpleasant stimuli. No laterality main effects or interactions were significant.

Table 1

Emotional picture processing effects on hemodynamic and electrocortical measures of brain activity

Measure Pleasant Neutral Unpleasant F2,14 values P values Linear Quadratic 
Pericalcarine BOLD (%Δ) 1.85 (0.17) 1.60 (0.18) 2.04 (0.20) 14.73 <0.001 NS <0.001 
Lateral occipital BOLD (%Δ) 1.32 (0.10) 1.00 (0.07) 1.33 (0.10) 8.07 <0.01 NS <0.01 
Parietal BOLD (%Δ) 0.87 (0.08) 0.61 (0.06) 0.99 (0.08) 7.61 <0.01 NS <0.01 
IT BOLD (%Δ) 1.27 (0.09) 1.00 (0.11) 1.39 (0.10) 13.93 <0.001 NS <0.001 
LPP (μvolt) 1.21 (0.68) −0.81 (0.70) 0.96 (0.71) 13.08 <0.01 NS <0.001 
Parietal source (nAmp) 25.4 (3.1) 21.1 (2.7) 25.9 (3.0) 10.47 <0.01 NS <0.001 
IT source (nAmp) 26.4 (2.1) 24.2 (2.0) 28.0 (2.7) 1.61 NS NS NS 
Pericalcarine source (nAmp) 20.5 (1.8) 18.5 (1.6) 21.0 (2.0) 3.58 NS NS <0.05 
Central source (nAmp) 23.8 (1.9) 22.7 (2.6) 25.0 (2.2) 0.63 NS NS NS 
Frontal source (nAmp) 38.0 (2.7) 33.0 (2.7) 39.2 (3.0) 1.92 NS NS NS 
Measure Pleasant Neutral Unpleasant F2,14 values P values Linear Quadratic 
Pericalcarine BOLD (%Δ) 1.85 (0.17) 1.60 (0.18) 2.04 (0.20) 14.73 <0.001 NS <0.001 
Lateral occipital BOLD (%Δ) 1.32 (0.10) 1.00 (0.07) 1.33 (0.10) 8.07 <0.01 NS <0.01 
Parietal BOLD (%Δ) 0.87 (0.08) 0.61 (0.06) 0.99 (0.08) 7.61 <0.01 NS <0.01 
IT BOLD (%Δ) 1.27 (0.09) 1.00 (0.11) 1.39 (0.10) 13.93 <0.001 NS <0.001 
LPP (μvolt) 1.21 (0.68) −0.81 (0.70) 0.96 (0.71) 13.08 <0.01 NS <0.001 
Parietal source (nAmp) 25.4 (3.1) 21.1 (2.7) 25.9 (3.0) 10.47 <0.01 NS <0.001 
IT source (nAmp) 26.4 (2.1) 24.2 (2.0) 28.0 (2.7) 1.61 NS NS NS 
Pericalcarine source (nAmp) 20.5 (1.8) 18.5 (1.6) 21.0 (2.0) 3.58 NS NS <0.05 
Central source (nAmp) 23.8 (1.9) 22.7 (2.6) 25.0 (2.2) 0.63 NS NS NS 
Frontal source (nAmp) 38.0 (2.7) 33.0 (2.7) 39.2 (3.0) 1.92 NS NS NS 

Note: Relative signal change values of electrocortical and hemodynamic brain measures for pleasant, neutral, and unpleasant pictures. The first 4 measures are expressed in percent BOLD signal change (deviated from the prepicture baseline of each trial). The scalp-recorded LPP is expressed in microvolts (deviated from the prepicture baseline of each trial), and the modeled source strength derivatives of the LPP are expressed in nanoamperes, also deviated from prepicture baseline. IT, inferotemporal; NS, not significant.

Picture Onset ERP: The LPP

The scalp topography of voltage change after picture onset included a posterior positive slow wave, seen clearly in the difference map of arousing (pleasant and unpleasant) and neutral picture presentation epochs (Fig. 2a). The topography, amplitude, and right-hemisphere bias (Cuthbert and others 2000; Keil and others 2002) of the LPP were equivalent across pleasant and unpleasant picture conditions (see also Fig. 5a), thus for clarity, the difference map of arousing and neutral pictures conditions is shown. A 27-channel cluster of sensors was averaged over the centroparietal areas (Fig. 2b, in red) and showed a reliable effect of picture valence from 400 to 900 ms after onset (F2,14 = 13.08, P = 0.001, Table 1), with greater relative scalp positivity during pleasant and unpleasant, compared to neutral, picture periods. The picture onset ERP is shown in Figure 3a, with the epoch ranging from 400 to 900 ms representing the LPP.

Figure 2.

(a) Difference map of scalp voltage topography between affective (pleasant and unpleasant, see Fig. 5a for independent maps) and neutral picture perception, from 400 to 900 ms after picture onset, seen from the back of the head. Red values indicate relative positivity. (b) Location of 27 sensors selected for estimate of LPP amplitude shaded in red.

Figure 2.

(a) Difference map of scalp voltage topography between affective (pleasant and unpleasant, see Fig. 5a for independent maps) and neutral picture perception, from 400 to 900 ms after picture onset, seen from the back of the head. Red values indicate relative positivity. (b) Location of 27 sensors selected for estimate of LPP amplitude shaded in red.

Figure 3.

Time course development of the LPP (a) and visual cortical BOLD (b) for pleasant, neutral, and unpleasant picture contents. The LPP waveform represents 27 centroparietal sensors (Fig. 2b), whereas the BOLD waveform represents average activity in inferotemporal, lateral occipital, and parietal ROIs. Picture onset is indicated by the arrow at time zero.

Figure 3.

Time course development of the LPP (a) and visual cortical BOLD (b) for pleasant, neutral, and unpleasant picture contents. The LPP waveform represents 27 centroparietal sensors (Fig. 2b), whereas the BOLD waveform represents average activity in inferotemporal, lateral occipital, and parietal ROIs. Picture onset is indicated by the arrow at time zero.

Modeled Source Strength

The Talairach coordinates and source sensitivity maps showing the regional independence of sources are shown in the left panel of Figure 4. The source intensity across the first second of the presentation interval for all trials is shown for all sources in the right panel of the figure and accounted for 96.7% of the variance of the scalp voltage. Sensitivity maps of the 9 regional dipoles revealed that the sources reflected independent regions of cortex, with the greatest contribution toward the scalp potential originating from parietal, inferior temporal, and frontal areas. This pattern is consistent with previous studies using inverse modeling to estimate the origin of slow potentials in a similar epoch (Keil and others 2002; Linden 2005). As listed in Table 1, only parietal source strength varied significantly with picture valence (F2,14 = 10.47, P < 0.01), though occipital source strength approached significance (F2,14 = 3.58, P = 0.06) and yielded a reliable quadratic trend (F1,15 = 6.53, P < 0.05). No sources differed according to hemisphere.

Figure 4.

Source sensitivity maps and source strength waveforms (in nanoamperes) for the 9 discrete regional sources used to model the LPP slow wave, over the first second of picture presentation. The sensitivity maps show the independence of cortical representation for each source (depicted in one hemisphere), with bright yellow indicating the greatest sensitivity and red the least sensitivity. The source waveforms show the activation time course of each modeled source through the first second of picture presentation, for x, y, and z dimensions. Talairach coordinates for each source are shown above the waveform.

Figure 4.

Source sensitivity maps and source strength waveforms (in nanoamperes) for the 9 discrete regional sources used to model the LPP slow wave, over the first second of picture presentation. The sensitivity maps show the independence of cortical representation for each source (depicted in one hemisphere), with bright yellow indicating the greatest sensitivity and red the least sensitivity. The source waveforms show the activation time course of each modeled source through the first second of picture presentation, for x, y, and z dimensions. Talairach coordinates for each source are shown above the waveform.

Spectral Power

Marked decreases in spectral power were elicited by picture onset, most prominently over parietal regions in the alpha (8–12 Hz) and beta (18–26 Hz) bands. Averaged power decrease from 400 to 900 ms post picture onset was not sensitive to picture valence (see Table 1).

Hemodynamic Electrocortical Correspondence

As can be seen in Table 2, the centroparietal scalp LPP, averaged from 400 to 900 ms after picture onset, correlated significantly with BOLD signal in lateral occipital, medial parietal, and inferotemporal ROIs. No correlations of independent modeled sources and BOLD signal change reached significance. In Figure 3, a time course comparison of the electrocortical LPP (a) and visual cortical ROI signal change (b, excluding pericalcarine activity) demonstrate the relative onset of signal differentiation by picture content. Figure 5 depicts the group-averaged correspondence of picture-driven scalp topography (a) and BOLD signal change (b) in a cross section of posterior visual cortex for pleasant, neutral, and unpleasant pictures, showing the augmentation of signal in both measures that is recruited during the presentation of emotionally arousing relative to neutral pictures.

Figure 5.

Comparison of ERP scalp topography (a) and fMRI BOLD contrast overlay (b) for pleasant, neutral, and unpleasant picture presentations. The ERP map represents average microvolt change from 400 to 900 ms after picture onset, with red indicating positivity and blue indicating negativity. The BOLD contrast overlay, coronal slice y = −68, represents a random-effects analysis of picture elicited activity, peaking roughly 8 s after picture onset, with red indicating more reliable increases in oxygenated blood flow and yellow indicating a threshold of P < 0.000000001.

Figure 5.

Comparison of ERP scalp topography (a) and fMRI BOLD contrast overlay (b) for pleasant, neutral, and unpleasant picture presentations. The ERP map represents average microvolt change from 400 to 900 ms after picture onset, with red indicating positivity and blue indicating negativity. The BOLD contrast overlay, coronal slice y = −68, represents a random-effects analysis of picture elicited activity, peaking roughly 8 s after picture onset, with red indicating more reliable increases in oxygenated blood flow and yellow indicating a threshold of P < 0.000000001.

Table 2

Relationships between hemodynamic, scalp potential, and modeled source strength during emotional picture perception

Regional BOLD Scalp LPP Parietal source Temporal source Occipital source Central source Frontal source 
Pericalcarine 0.45 0.29 0.25 0.12 0.30 0.20 
Lateral occipital 0.65** 0.24 0.20 −0.02 0.21 0.13 
Parietal 0.61* 0.37 0.21 0.04 0.20 0.18 
IT 0.55* 0.36 0.33 0.24 0.21 0.29 
Regional BOLD Scalp LPP Parietal source Temporal source Occipital source Central source Frontal source 
Pericalcarine 0.45 0.29 0.25 0.12 0.30 0.20 
Lateral occipital 0.65** 0.24 0.20 −0.02 0.21 0.13 
Parietal 0.61* 0.37 0.21 0.04 0.20 0.18 
IT 0.55* 0.36 0.33 0.24 0.21 0.29 

Note: Pearson's correlation coefficients of ROI BOLD signal with the scalp-recorded LPP (centroparietal mean from 400 to 900 ms) and derivative measures of the ERP, representing modeled regional source strength. IT, inferotemporal; numbers in parentheses indicate averaged combinations of source estimates. Asterisks indicate significance level, where *P < 0.05, **P < 0.01.

Discussion

Consistent with prior work, the voltage strength of a positive slow wave recorded from parietal regions of the scalp (the LPP) reflected the rated emotional intensity of picture stimuli (Cacioppo and others 1994; Cuthbert and others 2000; Keil and others 2002; Schupp and others 2004). The strength of this scalp-positive waveform correlated significantly with BOLD signal in lateral occipital, inferior temporal, and medial parietal cortex. Thus, despite wide differences in signal origin and latency, the 2 measures of cortical reactivity showed comparable modulation by emotional pictures.

The slow-wave LPP, evident on the scalp as a broad, long-lasting region of relative voltage positivity over posterior sites, may reflect a composite of activity from extrastriate occipital, inferior temporal, and medial parietal cortex. That is, within-subject correlations indicate that the amplitude of the scalp LPP was positively associated with BOLD signal in all 3 secondary visual areas. Moreover, when the scalp LPP is decomposed into regional sources, correlations of the BOLD signal with activity in these modeled individual sources are greatly reduced and nonsignificant. This may be due, alternatively, to an imprecise model, less than optimal electrocortical signal, or perhaps, the fact that the scalp LPP reflects concurrent activity across the visual system in this first second of picture perception. The latter possibility is consistent with data derived from an alternate method of slow-wave ERP source localization, the minimum-norm estimate, employed in a prior dense-array study of emotional picture processing (Keil and others 2002), which implicated occipitotemporal and parietal cortex as underlying the LPP. Multiple sources have also been implicated in a model of the late P300 component of the onset ERP (which precedes but overlaps with the temporal range of the LPP), which also included inferior temporal and superior parietal cortex (Linden 2005).

Regarding the temporal relationship between electrocortical and hemodynamic measures of brain activity, Logothetis and others (2001) have demonstrated that, though offset in time, there is a strong association between local field potentials recorded intracranially and BOLD signal in primate visual cortex. In contrast, single and multi-unit activity did not correspond as clearly with BOLD signal and in some conditions did not correspond at all. Ogawa and others (2000) have demonstrated a parallel effect during rat forepaw stimulation, in which staggered stimuli produce equivalent refractory periods in both electrocortical and BOLD signals in sensory cortex. The emotional modulation of the LPP (400 to 900 ms post picture onset) and BOLD (6–13.5 s post picture onset) could thus reflect the same mass neural activity in visual cortex. The delayed and extended nature of the indirect hemodynamic BOLD contrast may act as a temporal filter through which stimulus-coincident neural activation that first appears in the scalp potential is later seen in integrated and amplified form—integrated over time through the gradual nature of the vascular response and amplified by the relative oversupply of oxygenated blood that is the source of the BOLD contrast.

The correlation of BOLD and LPP modulation seen in this data set is reliable despite the fact that these measures were recorded in separate sessions. Although it is feasible to collect simultaneous or interleaved EEG inside an MR scanner, it presents a considerable technical challenge and inevitably compromises signal quality in both data sets. In processing of simultaneously recorded data, it is difficult to correct for motion artifact within the static field, radio frequency (RF) energy emitted from the transmit coil, and gradient field artifacts induced in the recording sensor without altering the EEG present in overlapping frequencies (Muri and others 1998; Allen and others 2000; Goldman and others 2000; Kruggel and others 2000; Sijbersa and others 2000; Bonmassar and others 2002). Interleaved image acquisition and EEG data collection reduce artifacts originating from RF energy and gradient switching (Bonmassar and others 2001; Liebenthal and others 2003; Thees and others 2003), but the comparison of EEG topography and BOLD signal is necessarily offset in time, and the onset of gradient noise is confounded with MR image acquisition. With either method, collection of artifact-free MR images is hampered by the presence of electrocortical sensors, which can shield or distort transmitted and received RF energy, and presents some risk to the participant (Baudewig and others 2000; Krakow and others 2000; Angelone and others 2004). The use of a magnetic resonance imaging simulator precludes these difficulties.

The comparison of the electrocortical and hemodynamic data recorded here relies, of course, on the assumption that the effects of the emotional pictures are consistent from session to session—that the evoked brain state is comparable from the MR scanner to the MR simulator. Prior studies using a highly similar paradigm have determined that a widely accepted measure of emotional arousal, the probe-startle reflex (Bradley and others 1993) and the LPP (Codispoti and others 2006), show consistent modulation by emotion across multiple repetitions of the same picture, suggesting that the impact of emotional stimuli across presentations is comparable.

In conclusion, a common pattern of emotion modulation was obtained for both the late posterior positive component (LPP) of the ERP, and in the hemodynamic BOLD signal in extrastriate visual cortex. This convergence of EEG and fMRI measures of affect is remarkable and implies a common involvement with motivational circuits. As shown in previous picture-viewing research, the magnitude of BOLD signal in inferotemporal visual cortex covaries closely with amygdala activity (Sabatinelli and others 2005). This covariation strongly supports the presence of reentrant projections from amygdala to the visual system that Amaral and others (1992) have described in nonhuman primates. This reentrant connection presumably prompts recruitment of a wider neuron pool in the posterior brain, associated with more intensive processing of motivationally relevant images (Holland and others 2000; Maunsell 2004). The increased activation in widespread higher-order visual areas is readily able to reach the scalp, thus accounting for the LPP waveform's high sensitivity to emotional input.

This work was supported by grants from the National Institute of Mental Health P50-MH072850 and the National Institute of Dental and Craniofacial Research DE13956. Conflict of Interest: None decalred.

References

Allen
PJ
Josephs
O
Turner
R
A method for removing imaging artifact from continuous EEG recorded during functional MRI
Neuroimage
 , 
2000
, vol. 
12
 (pg. 
230
-
239
)
Amaral
DG
Price
JL
Amygdalo-cortical projections in the monkey (Macaca fascicularis)
J Comp Neurol
 , 
1984
, vol. 
230
 (pg. 
465
-
496
)
Amaral
DG
Price
JL
Pitkanen
A
Carmichael
ST
Aggleton
JP
Anatomical organization of the primate amygdaloid complex
The amygdala: neurobiological aspects of emotion, memory, and mental dysfunction
 , 
1992
New York
Wiley-Liss
(pg. 
1
-
66
)
Angelone
LM
Potthast
A
Segonne
F
Iwaki
S
Belliveau
JW
Bonmassar
G
Metallic electrodes and leads in simultaneous EEG-MRI: specific absorption rate (SAR) simulation studies
Bioelectromagnetics
 , 
2004
, vol. 
25
 (pg. 
285
-
295
)
Arthurs
OJ
Boniface
SJ
What aspect of the fMRI BOLD signal best reflects the underlying electrophysiology in human somatosensory cortex?
Clin Neurophysiol
 , 
2003
, vol. 
114
 (pg. 
1203
-
1209
)
Arthurs
OJ
Johansen-Berg
H
Matthews
PM
Boniface
SJ
Attention differentially modulates the coupling of fMRI BOLD and evoked potential signal amplitudes in the human somatosensory cortex
Exp Brain Res
 , 
2004
, vol. 
157
 (pg. 
269
-
274
)
Arthurs
OJ
Williams
EJ
Carpenter
TA
Pickard
JD
Boniface
SJ
Linear coupling between functional magnetic resonance imaging and evoked potential amplitude in human somatosensory cortex
Neuroscience
 , 
2000
, vol. 
101
 (pg. 
803
-
806
)
Baudewig
J
Paulus
W
Frahm
J
Artifacts caused by transcranial magnetic stimulation coils and EEG electrodes in T(2)*-weighted echo-planar imaging
Magn Reson Imaging
 , 
2000
, vol. 
18
 (pg. 
479
-
484
)
Bonmassar
G
Purdon
PL
Jaaskelainen
IP
Chiappa
K
Solo
V
Brown
EN
Belliveau
JW
Motion and ballistocardiogram artifact removal for interleaved recording of EEG and EPs during MRI
Neuroimage
 , 
2002
, vol. 
16
 (pg. 
1127
-
1141
)
Bonmassar
G
Schwartz
DP
Liu
AK
Kwong
KK
Dale
AM
Belliveau
JW
Spatiotemporal brain imaging of visual-evoked activity using interleaved EEG and fMRI recordings
Neuroimage
 , 
2001
, vol. 
13
 (pg. 
1035
-
1043
)
Bradley
MM
Lang
PJ
Cuthbert
BN
Emotion, novelty, and the startle reflex: habituation in humans
Behav Neurosci
 , 
1993
, vol. 
107
 (pg. 
970
-
980
)
Bradley
MM
Sabatinelli
D
Lang
PJ
Fitzsimmons
JR
King
W
Desai
P
Activation of the visual cortex in motivated attention
Behav Neurosci
 , 
2003
, vol. 
117
 (pg. 
369
-
380
)
Breiter
HC
Etcoff
NL
Whalen
PJ
Kennedy
WA
Rauch
SL
Buckner
RL
Strauss
MM
Hyman
SE
Rosen
BR
Response and habituation of the human amygdala during visual processing of facial expression
Neuron
 , 
1996
, vol. 
17
 (pg. 
1
-
13
)
Cacioppo
JT
Crites
SL
Gardner
WL
Bernston
GG
Bioelectric echoes from evaluative categorizations: I. A late positive brain potential that varies as a function of trait negativity and extremity
J Pers Soc Psychol
 , 
1994
, vol. 
67
 (pg. 
115
-
125
)
Codispoti
M
Ferrari
V
Bradley
MM
Repetitive picture processing: autonomic and cortical correlates
Brain Res
 , 
2006
, vol. 
1068
 (pg. 
213
-
220
)
Cook
EW
III
Atkinson
L
Lang
KG
Stimulus control and data acquisition for IBM PCs and compatibles
Psychophysiology
 , 
1987
, vol. 
24
 (pg. 
726
-
727
)
Cuthbert
BN
Schupp
HT
Bradley
MM
Birbaumer
N
Lang
PJ
Brain potentials in affective picture processing: covariation with autonomic arousal and affective report
Biol Psychol
 , 
2000
, vol. 
52
 (pg. 
95
-
111
)
Davis
M
Lang
PJ
Gallagher
M
Nelson
RJ
Emotion
Handbook of psychology. Volume 3, Biological psychology
 , 
2003
New York
Wiley
(pg. 
405
-
439
)
Fitzsimmons
JR
Scott
JD
Peterson
DM
Wolverton
BL
Webster
CS
Lang
PJ
Integrated RF coil with stabilization for fMRI in human cortex
Magn Reson Med
 , 
1997
, vol. 
38
 (pg. 
15
-
18
)
Foucher
JR
Otzenberger
H
Gounot
D
The BOLD response and the gamma oscillations respond differently than evoked potentials: an interleaved EEG-fMRI study
BMC Neurosci
 , 
2003
, vol. 
4
 (pg. 
1
-
11
)
Goldman
RI
Stern
JM
Engel
J
Jr
Cohen
MS
Acquiring simultaneous EEG and functional MRI
Clin Neurophysiol
 , 
2000
, vol. 
111
 (pg. 
1974
-
1980
)
Holland
PC
Han
JS
Gallagher
M
Lesions of the amygdala central nucleus alter performance on a selective attention task
J Neurosci
 , 
2000
, vol. 
20
 (pg. 
6701
-
6706
)
Horowitz
SG
Skudlarski
P
Gore
JC
Correlations and dissociations between BOLD signal and P300 amplitude in an auditory oddball task: a parametric approach to combining fMRI and ERP
Magn Reson Imaging
 , 
2002
, vol. 
20
 (pg. 
319
-
325
)
Huettel
SA
McKeown
MJ
Song
AW
Hart
S
Spencer
DD
Allison
T
McCarthy
G
Linking hemodynamic and electrophysiological measures of brain activity: evidence from functional MRI and intracranial field potentials
Cereb Cortex
 , 
2004
, vol. 
14
 (pg. 
165
-
173
)
Keil
A
Bradley
MM
Hauk
O
Rockstroh
B
Elbert
T
Lang
PJ
Large-scale neural correlates of affective picture processing
Psychophysiology
 , 
2002
, vol. 
39
 (pg. 
641
-
649
)
Krakow
K
Allen
PJ
Symms
MR
Lemieux
L
Josephs
O
Fish
DR
EEG recording during fMRI experiments: image quality
Hum Brain Mapp
 , 
2000
, vol. 
10
 (pg. 
10
-
15
)
Kruggel
F
Wiggins
CJ
Herrmann
CS
von Cramon
DY
Recording of the event-related potentials during functional MRI at 3.0 Tesla field strength
Magn Reson Med
 , 
2000
, vol. 
44
 (pg. 
277
-
282
)
Lang
PJ
Bradley
MM
Cuthbert
BN
Lang
PJ
Simons
RF
Balaban
MT
Motivated attention: affect, activation and action
Attention and orienting: sensory and motivational processes
 , 
1997
Hillsdale, NJ
Erlbaum
(pg. 
97
-
136
)
Lang
PJ
Bradley
MM
Cuthbert
BN
International affective picture system (IAPS): technical manual and affective ratings
 , 
1999
FL: The Center for Research in Psychophysiology
Gainesville
Lang
PJ
Bradley
MM
Fitzsimmons
JR
Cuthbert
BN
Scott
JD
Moulder
B
Nangia
V
Emotional arousal and activation of the visual cortex: an fMRI analysis
Psychophysiology
 , 
1998
, vol. 
35
 (pg. 
199
-
210
)
Liebenthal
E
Ellingson
ML
Spanaki
MV
Prieto
TE
Ropella
KM
Binder
JR
Simultaneous ERP and fMRI of the auditory cortex in a passive oddball paradigm
Neuroimage
 , 
2003
, vol. 
19
 (pg. 
1395
-
1404
)
Linden
DEJ
The P300: where in the brain is it produced and what does it tell us?
Neuroscientist
 , 
2005
, vol. 
6
 (pg. 
563
-
576
)
Logothetis
NK
Pauls
J
Augath
M
Trinath
T
Oeltermann
A
Neurophysiological investigation of the basis of the fMRI signal
Nature
 , 
2001
, vol. 
412
 (pg. 
150
-
157
)
Maunsell
JHR
Neuronal representations of cognitive state: reward or attention?
Trends Cogn Sci
 , 
2004
, vol. 
8
 (pg. 
261
-
264
)
Muri
RM
Felblinger
J
Rosler
KM
Jung
B
Hess
CW
Boesch
C
Recording of electrical brain activity in a magnetic resonance environment: distorting effects of the static magnetic field
Magn Reson Med
 , 
1998
, vol. 
39
 (pg. 
18
-
22
)
Ogawa
S
Lee
T
Stepnowski
R
Chen
W
Zhu
X
Ugurbil
K
An approach to probe some neural systems interaction by functional MRI at neural scale down to milliseconds
Proc Natl Acad Sci USA
 , 
2000
, vol. 
97
 (pg. 
11026
-
11031
)
Sabatinelli
D
Bradley
MM
Fitzsimmons
JR
Lang
PJ
Parallel amygdala and inferotemporal activation reflect emotional intensity and fear relevance
Neuroimage
 , 
2005
, vol. 
24
 (pg. 
1265
-
1270
)
Sabatinelli
D
Fitzsimmons
JR
King
WM
Lang
PJ
Constructing a 0 Tesla MR scanner
Neuroimage
 , 
2000
, vol. 
11
 pg. 
S567
 
Sabatinelli
D
Flaisch
T
Bradley
MM
Fitzsimmons
JR
Lang
PJ
Affective picture perception: gender differences in visual cortex?
Neuroreport
 , 
2004
, vol. 
15
 (pg. 
1109
-
1112
)
Scherg
M
Ille
N
Bornfleth
H
Berg
P
Advanced tools for digital EEG review: virtual source montages, whole-head mapping, correlation and phase analysis
J Clin Neurophysiol
 , 
2002
, vol. 
19
 (pg. 
91
-
112
)
Scherg
M
Von Cramon
D
Evoked dipole source potentials of the human auditory cortex
Electroencephalogr Clin Neurophysiol
 , 
1986
, vol. 
65
 (pg. 
344
-
360
)
Schupp
HT
Cuthbert
BN
Bradley
MM
Hillman
CH
Hamm
A
Lang
PJ
Brain processes in emotional perception: motivated attention
Cogn Emotion
 , 
2004
, vol. 
18
 (pg. 
593
-
611
)
Sijbersa
J
Van Audekerke
J
Verhoye
M
Van der Linden
A
Van Dyck
D
Reduction of ECG and gradient related artifacts in simultaneously recorded human EEG/MRI data
Magn Reson Imaging
 , 
2000
, vol. 
18
 (pg. 
881
-
886
)
Singh
M
Kim
S
Kim
TS
Correlation between BOLD-fMRI and EEG signal changes in response to visual stimulus frequency in humans
Magn Reson Med
 , 
2003
, vol. 
49
 (pg. 
108
-
114
)
Spiegler
BJ
Mishkin
M
Evidence for the sequential participation of inferior temporal cortex and amygdala in the acquisition of stimulus-reward associations
Behav Brain Res
 , 
1981
, vol. 
3
 (pg. 
303
-
317
)
Sprengelmeyer
R
Rausch
M
Eysel
UT
Przuntek
H
Neural structures associated with recognition of facial expressions of basic emotions
Proc R Soc Lond B Biol Sci
 , 
1998
, vol. 
265
 (pg. 
1927
-
1931
)
Stippich
C
Freitag
P
Kassubek
J
Soros
P
Kamada
K
Kober
H
Scheffler
K
Hopfengartner
R
Bilecen
D
Radu
EW
Vieth
JB
Motor, somatosensory and auditory cortex localization by fMRI and MEG
Neuroreport
 , 
1998
, vol. 
9
 (pg. 
1953
-
1957
)
Talairach
J
Tournoux
P
Co-planar stereotaxic atlas of the human brain
 , 
1988
New York
Thieme Medical Publishers
Thees
S
Blankenburg
F
Taskin
B
Curio
G
Villringer
A
Dipole source localization and fMRI of simultaneously recorded data applied to somatosensory categorization
Neuroimage
 , 
2003
, vol. 
18
 (pg. 
707
-
719
)
Vitacco
D
Brandeis
D
Pascual-Marqui
R
Martin
E
Correspondence of event-related potential tomography and functional magnetic resonance imaging during language processing
Hum Brain Mapp
 , 
2002
, vol. 
17
 (pg. 
4
-
12
)