Affectively arousing visual stimuli have been suggested to automatically attract attentional resources in order to optimize sensory processing. The present study crosses the factors of spatial selective attention and affective content, and examines the relationship between instructed (spatial) and automatic attention to affective stimuli. In addition to response times and error rate, electroencephalographic data from 129 electrodes were recorded during a covert spatial attention task. This task required silent counting of random-dot targets embedded in a 10 Hz flicker of colored pictures presented to both hemifields. Steady-state visual evoked potentials (ssVEPs) were obtained to determine amplitude and phase of electrocortical responses to pictures. An increase of ssVEP amplitude was observed as an additive function of spatial attention and emotional content. Statistical parametric mapping of this effect indicated occipito-temporal and parietal cortex activation contralateral to the attended visual hemifield in ssVEP amplitude modulation. This difference was most pronounced during selection of the left visual hemifield, at right temporal electrodes. In line with this finding, phase information revealed accelerated processing of aversive arousing, compared to affectively neutral pictures. The data suggest that affective stimulus properties modulate the spatiotemporal process along the ventral stream, encompassing amplitude amplification and timing changes of posterior and temporal cortex.
The rapid selection and evaluation of visual information facilitates adaptive behavior. In the laboratory, these processes have been studied using tasks that requiring selection of a specific stimulus location, color, size, etc., frequently employing neurophysiological parameters to describe the timing within brain systems that underlie the selection phenomena (Hillyard and Anllo-Vento, 1998). For instance, a priori knowledge about the spatial arrangement of task-relevant visual events can be used to direct attention to particular parts of the visual field. This results in enhanced behavioral performance for spatially cued target stimuli, as well as enhanced amplitude of the visual electrocortical response (Müller and Hillyard, 2000). Consequently, electrophysiological findings in human and non-human subjects have led theorists to assume that cortical facilitation of specific networks may underlie attention to object features such as color, shape, temporal order or location in space (Luck et al., 1997).
In studies of spatial selective attention, participants are typically asked to detect targets in certain regions of the visual field while maintaining their gaze on a central fixation point. This covert attention task has been associated with activation of a widespread network, encompassing extrastriate visual cortex, frontoparietal cortical areas, as well as deeper structures such as the superior colliculi and the cingulate gyrus (Corbetta, 1998). Electrocortical effects in response to spatially attended stimuli are present in visual cortex as early as 100 ms, and several authors have emphasized the enhanced activation in right-hemisphere structures mediating spatial orienting and attention (Mesulam, 1998).
Attention and Emotion
Similar to the findings for directed attention research, sensory processing is also facilitated during free viewing when the visual stimuli have emotional significance (Lang et al., 1998; Schupp et al., 2003). Event-related potentials (ERPs) in response to affective pictures have reliably shown a late (latency >300 ms) ERP amplitude enhancement for emotionally arousing stimuli (Cuthbert et al., 2000), which appears to be generated in bilateral visual areas of the occipital and temporal cortex, as well as in right-hemispheric parietal areas (Keil et al., 2002).
Early ERP differences in time windows between 100 and 300 ms after stimulus onset have also been reported, suggesting that even extrastriate visual processes may be modulated by affective arousal (Pizzagalli et al., 1999; Junghöfer et al., 2001). Using a visual hemifield design with affective pictures from the International Affective Picture System (IAPS), Keil et al. (2001) reported enhancement of the N1 amplitude (∼160 ms) for affectively arousing stimuli, compared to neutral stimuli. This difference was most pronounced with left visual field presentation and at right hemispheric sensors. Recently, Smith et al. (2003) suggested that the P1 component of the ERP in an oddball task is sensitive to the emotional characteristics of a stimulus. This enhanced attentional processing of emotion-arousing stimuli has also been reported in research assessing reaction time (Öhman et al., 2001), autonomic and somatic reflexes (Lang et al., 1997) and brain imaging (Bradley et al., 2003). The general phenomenon has been related to an evolutionary preference for stimuli of appetite and threat and has been described as motivated or ‘natural selective attention’.
Steady-state Visual Evoked Potentials
To study interactions between spatial selective attention and motivated attention, the present study employs the steady-state visual evoked potential (ssVEP) (Regan, 1989). This method has been used previously to investigate affective (Keil et al., 2003a) and attentional processes in the visual modality (Müller et al., 1998). The ssVEP is a brain response elicited by a repetitive visual stimulus that is periodically modulated in intensity at a fixed rate of 6–8 Hz or greater. This procedure typically results in an oscillatory brain response with the same fundamental frequency as the driving stimulus.
Work using source analysis techniques to enhance spatial specificity of scalp signals suggests that the ssVEP is generated in parieto-occipital cortices, with the source locations depending on the stimulation frequency (Müller et al., 1997). Frequency domain analyses of the oscillatory signal reflecting stimulus-related brain activity result in two dependent variables: (i) amplitude, which reflects the magnitude of oscillatory activity at the stimulation frequency, and (ii) phase, which provides information on latency and/or source orientation (Burkitt et al., 2000). Phase delay or acceleration can thus be used as a measure of temporal dynamics of the signal when topographical distribution of phase values is stable across time and conditions (Keil et al., 2003a).
Steady-state potentials in the visual modality have been shown to be sensitive to visual spatial selective attention (Morgan et al., 1996). In a number of visual hemifield studies, authors reported that the amplitude of the ssVEP recorded at sites contralateral to the attended location was enhanced compared to non-attended conditions (Müller and Hillyard, 2000). In addition, amplitude and phase alterations of ssVEPs were associated with affective stimulus characteristics. In a recent study, it was found that the ssVEP elicited by affective pictures showed a modulation as a function of the dimension of emotional arousal (Keil et al., 2003a). This effect of higher amplitude and accelerated phase for arousing pictures, compared to affectively neutral ones, was strongest at central posterior as well as right parieto-temporal sites.
The same pattern of results was observed for the magnetic counterpart of the ssVEP, the steady-state visual evoked magnetic field, which varied as a function of subjectively rated emotional arousal. This finding points to the involvement of parieto-frontal attention networks in emotion perception and motivated attention (Moratti et al., 2004). Interestingly, Kemp et al. (2002) showed a decrease of the ssVEP elicited by a diffuse peripheral flicker when affective pictures were presented as a competing event. Summarizing this literature, findings suggest that the networks mediating spatial attention and motivated attention may overlap, with visual cortex being activated early on in the stream of processing during both attentional manipulations. Alternatively, it has been suggested that emotional processing depends on attentional resources (e.g. Eimer et al., 2003). For instance, Pessoa et al. (2002) reported that with emotional face stimuli, affective modulation of the blood oxygen level dependent response occurs only when attentional resources are being directed to the faces.
The research problem
Based on the literature reported so far, at least two alternative predictions can be made regarding the manner in which spatial attention and emotional stimulus properties may interact. First, if attentional resources are necessary to give rise to amplitude enhancement for affectively arousing stimuli, then affective modulation of the ssVEP signal should not take place when an affective stimulus is being presented in the non-attended hemifield. Thus, amplification of the ssVEP in response to affectively arousing stimuli should be observed in the attended hemifields only. Alternatively, if there is an automatic allocation of resources for affective stimuli, we would expect that visual cortical processing should benefit from both motivated and spatial attention, thus predicting amplitude enhancement as an additive function of both types of attention. Using a cued target detection task in the visual hemifields and affective pictures varying in emotional arousal (i.e. affectively neutral versus unpleasant arousing pictures), we crossed the factors of spatial attention and affective stimulus properties, resulting in four major conditions: (i) neutral pictures in both the attended and non-attended hemifield (NN); (ii) neutral pictures in the attended hemifield while unpleasant pictures were present in the ignored hemifield (NU); (iii) unpleasant pictures in the attended hemifield while neutral pictures were present in the non-attended hemfield (UN); (iv) unpleasant pictures in both hemifields (UU).
We expected amplitude enhancement and phase acceleration of the ssVEP with increasing presence of aversive information in the to-be-attended hemifield. In addition, we hypothesized that threat pictures present in the non-attended visual field would also have an amplificatory effect on the ssVEP amplitude (NN < NU < UN < UU). Alternatively, if unpleasant pictures require attention to lead to enhanced ssVEP amplitude, one would expect NN = NU < UN = UU. Given the prominent role of the right hemisphere both in spatial selective attention (Hillyard et al., 1997) and the discrimination of affectively arousing stimuli (e.g. Junghöfer et al., 2001; Keil et al., 2001), we hypothesized that amplitude and phase differences are most pronounced with left hemispheric presentation and at right hemisphere recording sites.
Materials and Methods
Twelve healthy university students with normal or corrected vision consented to participate in this study. Data from one participant were excluded from analyses because of excessive noise (see below), resulting in 11 participants (7 female), with an average age of 23 years and 4 months (SD = 2.1 years). They received class credit or 10 euros (∼US$8).
Two categories of pictures from the IAPS (Lang et al., 1999) were formed to be either aversive and high in emotional arousal or affectively neutral and low in arousal. The IAPS provides normative subjective ratings for >1200 colored pictures, in terms of their self-rated emotional arousal, valence and dominance. Each picture category comprised 24 stimuli. Stimuli depicting persons were used throughout in order to minimize variability in terms of non-affective stimulus parameters. Aversive pictures showed interpersonal attack scenes, whereas neutral pictures showed activities of daily life. Attack pictures were used because they evoke pronounced changes in reflex physiology (Bradley et al., 2001) as well as in brain metabolism (Bradley et al., 2003). Normative arousal ratings on the Self-Assessment Manikin (SAM) (Lang, 1980) 9-point scale were 5.9 for attack pictures and 2.1 for neutral pictures. Normative pleasure ratings were 2.7 and 5.6 respectively. All stimuli were displayed on a 19 in. computer monitor with a vertical refresh rate of 60 Hz. Pictures were presented in both hemifields in a flickering mode, with an on-period of 33.33 ms and an off-period of 66.67 ms, resulting in a driving frequency of 10 Hz. They subtended a horizontal visual angle of 5.2°, the eccentricity of the center of the pictures to either side being 3.9°. The distance between the screen and the participants' eyes was 1.7 m. We employed a chin rest to keep these parameters constant within and between participants. A central fixation point was present in the center of the screen at all times.
Design and Procedure
The experimental design involved a target detection task (see Fig. 1). Trials started with an arrow cue being shown for 1000 ms, pointing either to the right or left hemifield. Locked to the offset of the cue, two different IAPS pictures appeared in the hemifields, synchronously flickering at 10 Hz. These trains had a duration of 6000 ms. Participants were asked to count occasional random-dot patterns replacing picture stimuli in the attended hemifield, and to ignore targets occurring in the other while they maintained fixation. Immediately after termination of the stimulus train, participants entered the number of random dot targets counted in the relevant hemifield, using a response pad. Inter-trial intervals varied randomly between 5000 and 10 000 ms. Participants knew that the number of targets would range between 0 and 3. Accordingly, the first four fingers were placed on the respective keys of a response pad, labeled with numbers from 0 to 3.
Prior to the electroencephalographic (EEG) recordings, the experimental task was practiced until the participants were familiar with it, using stimuli that were not part of the experimental set. Subsequently, the electrode net was applied and participants entered an electrically shielded chamber, where EEG recordings were conducted. Two blocks of 60 trials were run, leading to a total of five repetitions of each picture in different configurations. Thus, both the unattended and attended field could contain aversive or neutral pictures, resulting in a 2 (attended hemifield) × 2 (attended category) × 2 (unattended category) design. After the EEG recordings, subjects viewed the 48 different pictures again in a pseudo-randomized order and were asked to rate the respective picture on the two dimensions of affective valence and arousal, using SAM. In this last block, no hemifield presentation was done and subjects viewed each picture without being constrained in any way.
Analysis of Behavioral Data
Distributions of manual responses indicating number of targets present in the attended hemifield were visually screened for outliers, which were removed from the distribution. Mean reaction times were computed for correct responses only. Error rates were expressed as percent correct responses.
EEG Recordings and EEG Data Analysis
Electroencephalographs were recorded continuously from 129 electrodes using an Electrical Geodesics system. Data were referenced to Cz, digitized at a rate of 250 Hz, and online band-pass filtered between 0.1 and 100 Hz. Epochs of 200 ms pre-stimulus and 6000 ms post-stimulus onset were obtained off-line. Artifact rejection was also performed off-line, following the procedure proposed by Junghöfer et al. (2000). Using this approach, trials with artifacts were identified based on the distribution of mean, maximum and SD across time points and channels. Sensors contaminated with artifacts were replaced by statistically weighted, spherical spline interpolated values. The maximum number of approximated channels in a given trial was set to 20. As a result, 11% of trials were rejected; the numbers of remaining trials did not differ between experimental conditions. For interpolation and all subsequent analyses, data were arithmetically re-referenced to the average reference. In addition to the artifact control procedure described above, eye movements and blinks were controlled by visually inspecting the vertical and horizontal electro-oculogram (EOG) for each trial. Artifact-free epochs were averaged separately for six combinations of attended hemifield, attended category, and unattended category. A 200 ms pre-stimulus data segment was subtracted as baseline. To eliminate effects of initial ERPs to stimulus train onset, a time period of 1000–6000 ms after train onset was used for further processing.
As a test for presence of a stimulus-locked 10 Hz ssVEP signal different from noise in each participant, we used the circular T-square algorithm proposed by Victor and Mast (1991). To this end, the 1000–6000 ms post-stimulus part of the averaged potentials for each condition was divided into five temporally equidistant, non-overlapping sections of 1000 ms length, each of which served as an independent sample for estimating the 10 Hz Fourier components. These segments were de-meaned and de-trended using linear regression. Subsequently, a 500 ms window containing five cycles was shifted in steps of 100 ms (i.e. 25 data points) and the potential within the shifting windows in the time domain was further averaged, to increase sensitivity to stimulus-locked brain responses as well as improve signal-to-noise ratio of the estimate. The resulting ssVEP data epochs of 500 ms were transformed into the frequency domain using fast Fourier transform on 128 data points, whereby three data points were padded with zeros. Fourier components of 10 Hz were calculated for eight experimental conditions and five data segments respectively. Consequently, 8 (condition) × 5 (segments) independent estimates for the 10 Hz Fourier coefficients were submitted to the circular T-square algorithm. For assessment of signal presence, we tested for all subjects and EEG channels if the signal was different from zero. We required that significant (P < 0.05)
All analyses were carried out using sensor space information (voltage data) rather than source space information (e.g. minimum norm estimate as has been employed by Moratti et al., 2004) to ensure that both amplitude and phase of the ssVEP could be reliably measured, based on the same time series. Measuring the phase of distributed source density data is difficult because of multiple orientations of sources used in the source model. In the case of EEG recordings, there are three orientations to consider (two tangential ones and one radial one, being orthogonal), which may be sensitive to different aspects of ssVEP phase. Using voltage data, however, raises the question of misinterpretation regarding the origin of the signal generating a given voltage topography. For the present study we compared amplitude maps derived from voltage data and the minimum norm estimate, a linear estimation technique originally proposed by Hämäläinen and Ilmoniemi (1984) to confirm correspondence between posterior voltage maximum and visual cortical sources, as has been demonstrated in other work (Müller et al., 1997). To this end, 10 Hz Fourier components reflecting 10 Hz activity of the ssVEP were submitted to the minimum norm implementation suggested by Hauk et al. (2002). Distributed sources for the 10 Hz ssVEP were determined across affective conditions, for ‘attend left’ and ‘attend right’ instructions separately. This step resulted in the source density maps shown in Figure 2, demonstrating that source activity underlying the scalp voltages was most pronounced in posterior cortical areas, contralateral to the attended hemifield.
For evaluation of experimental effects on ssVEP amplitude and phase, 10 Hz Fourier components were obtained for a 500 ms data segment resulting from the moving average across the entire post-stimulus onset part of the epoch following the initial ERP, i.e. 1000–6000 ms. This was done to further increase the signal-to-noise ratio for the stimulus-locked response. Differences of ssVEP phase and amplitude between experimental conditions were then evaluated statistically by means of three different approaches.
First, effects of affective category and spatial attention on posterior ssVEP amplitude were examined using analysis of variance (ANOVA) on posterior regional means. This approach has the advantage of relying on regional averages, thus enhancing the reliability of the estimate used as a dependent variable (Keil et al., 2003b). Hence, amplitude was calculated as the magnitude of the 10 Hz Fourier components for each affective category and electrode. Amplitude values were then grouped into four posterior regional means. These means encompassed both medial and lateral electrodes on both hemispheres, thus being sensitive for changes of ssVEP signal as a function of hemifield (see Fig. 3). F-values were computed for these means using ANOVA with the within-subjects factors ‘hemisphere’ (left, right), ‘laterality’ (medial, lateral), ‘Attended Hemifield’ (left, right), ‘Attended Category’ (neutral, unpleasant) and ‘Non-Attended Category’ (neutral, unpleasant). For self-reported affective arousal and valence, paired t-tests comparing mean ratings for picture categories (neutral, unpleasant) were used.
To enhance spatial resolution of the statistical analysis, electrode sites showing the hypothesized additive relationship between spatial attention and aversive arousal (i.e. attack content) were identified by conducting trend tests (contrast analyses) at each sensor, modeling amplitude enhancement as a linear function of increased spatial attention and aversive arousal for each hemifield condition. We assumed that within a hemifield condition, sites showing additive amplification should have smallest amplitudes when two neutral pictures were shown, whereas maximum amplitude was expected in the presence of two unpleasant pictures. In cases where pictures differed in terms of their category, greater amplitudes were expected when the unpleasant picture was in the to-be-attended hemifield. Thus, ssVEP amplitudes were subject to contrast analyses assuming that neutral–neutral < neutral–unpleasant < unpleasant–neutral < unpleasant–unpleasant. Because of the problems associated with multiple significance testing, P-values were determined according to the permutation method proposed by Blair and Karniski (1993; see also Karniski et al., 1994). As recommended by these authors, values of a given participant were always permuted within the subject and were never freely assigned to random cells, as these values usually cannot be considered independent. Permutation distributions were generated for hemifield conditions separately and the more conservative criterion was selected for all analyses. Distributions were based on 8000 draws for entire topographies (i.e. one draw taking into account 129 electrodes) and the greatest F-value of each draw entered the test distribution. For the present data set, this procedure resulted in critical F-values of 6.03 (P < 0.05) and 9.12 (P < 0.01).
A third approach aimed at evaluating condition-related modulations of ssVEP phase. To this end, we computed Rayleigh-tests modeling the differences of phase between conditions with affectively arousing information versus conditions without arousing information in the to-be-attended hemifield. This was done separately for each sensor and the visual hemifields, as described above for contrast analyses. Again, significance thresholds were determined using permutation of complex phase values between conditions, as described above. The resulting critical values of the Rayleigh statistic, R, were 0.66 (P < 0.05) and 0.83 (P < 0.01).
As expected, interpersonal attack (unpleasant) pictures were rated as less pleasant [mean pleasure rating 2.68, SD = 0.91] and more arousing than neutral pictures [mean pleasure rating 4.94, SD = 0.53; t(10) = 8.9, P < 0.001]. Likewise, self-rated emotional arousal differed between the categories, unpleasant pictures [mean arousal rating 5.06, SD = 1.78] showing higher scores than neutral pictures [mean arousal rating 2.26, SD = 0.80; t(10) = 6.5, P < 0.001]. In terms of the target detection task, all participants were able to perform with satisfying accuracy. The average error rate was 9.3%. Effects of experimental condition on behavioral data were evaluated by means of ANOVAs having within-subject factors of ‘Attended Hemifield’ (left, right), ‘Attended Category’ (neutral, unpleasant) and ‘Non-Attended Category’ (neutral, unpleasant). These analyses revealed that error rate was enhanced (mean error rate 10.3%) when unpleasant pictures were present in the non-attended hemifield, compared to presence of neutral pictures [mean error rate 8.0%; F(1,10) = 4.6, P < 0.05].
Reaction times (RTs) were higher after targets were embedded in unpleasant pictures, presented in the attended visual field [mean RT = 762.5, SD = 215.7], compared to neutral stimuli [mean RT = 715.4, SD = 199.9; F(1,10) = 6.5, P < 0.05]. This effect was most pronounced when unpleasant stimuli were attended in the left hemifield [attended category × hemifield: F(1,10) = 4.6, P < 0.05, see Fig. 4].
Regional Means Analysis
Topographies of ssVEP amplitude confirmed selection of the electrode clusters, showing a posterior maximum and a small degree of lateralization (see Fig. 5). ANOVAs on the four posterior regional means showed that spatial attention resulted in enhanced activation of the hemisphere contralateral to the task-relevant hemifield [hemifield × hemisphere: F(1,10) = 7.5, P < 0.05]. As expected, effects of spatial attention on ssVEP were greatest at lateral electrode sites, for the left-hemifield task [F(1,10) = 22.9, P < 0.01]. Medial sites generally showed higher ssVEP amplitude compared to lateral sites [F(1,10) = 11.3, P < 0.01], suggesting projection of the ssVEP to medial sensors, which is consistent with the source model presented in Figure 2.
We observed a main effect of attended picture category [F(1,10) = 5.4, P < 0.05], reflecting greater ssVEP amplitude for unpleasant, compared to neutral pictures. As indicated by an interaction of attended category with laterality [F(1,10) = 13.4, P < 0.01], this enhancement for unpleasant stimuli was strongest at medial, compared to lateral electrodes. An interaction of laterality, attended category and non-attended category [F(1,10) = 7.0, P < 0.05] indicated linear increase of posterior ssVEP amplitude as an additive function of spatial attention and aversive content for the medial electrodes (see Fig. 6). Unattended category interacted with hemifield and laterality [F(1,10) = 6.1, P < 0.01], showing enhancement of ssVEP amplitude for non-attended arousing pictures only with attention being directed to the left hemifield.
Topographical Contrast Analyses
As described above, planned comparison tests (contrast analyses) were calculated for each EEG-sensor, modeling linear effects of spatial selective attention and emotional content on ssVEP amplitude, to enhance spatial specificity. These were conducted for each hemifield condition separately. The procedure resulted in F-values at each electrode site, reflecting the degree to which the two experimental factors of spatial selective attention and emotional content were linear-additive at a given site. Figure 7 shows the topographical distribution of these values, complementing results obtained by regional means ANOVA. High F-values were generally observed at posterior and parieto-temporal electrodes located contralateral to the hemifield that was to be attended. In parallel to regional means analyses, the ‘attend left’ condition was related to stronger sensitivity of ssVEP amplitude to affective picture content. In particular, medial posterior and right temporal sites showed F-values >10, which suggests a strong effect, as this value is well above the permutation-determined significance criterion of 9.12 for the P < 0.01 threshold. No such effects were seen on the left hemisphere.
Less clear effects were seen when targets in the right visual field were to be attended. Again, medial posterior as well as parieto-temporal sites contralateral to the attended visual field were amplitude-modulated as a function of affective content. As can be seen from Figure 7, these differences were smaller, however, and the 5% significance criterion was met at medial posterior and parieto-temporal sites only.
To evaluate the phase differences, we used the univariate Rayleigh test, comparing conditions with affectively arousing stimuli in the attended visual field with the conditions with affectively neutral pictures in the attended field at each electrode site. This test was applied to each electrode and the resulting topographical maps are shown in Figure 8. Significant phase differences were observed at right-hemispheric sites during attention to the left visual field only. R-values exceeded the significant threshold at both inferior temporal and superior fronto-temporal sites. Maximum values were 0.86 (P < 0.01) at inferior temporal and 0.81 (P < 0.05) at fronto-temporal electrodes, suggesting reliable phase differences at these sites. Examination of phase at these sites revealed that a small but consistent acceleration of phase for unpleasant compared with neutral content could be observed. This suggests temporally contingent activation of anterior (i.e. fronto-temporal) cortical sources during each cycle of the ssVEP.
What is the relationship between spatial selection and automatic, ‘natural selective attention’ towards affectively arousing stimuli? The present study employed ssVEPs to measure the amplitude and phase of electrocortical activity during orienting towards pictures varying in terms of affective valence and arousal, shown under different instructions with respect to spatial attention. Both behavioral data and ssVEPs indicated a strong effect of emotional content on processing of the rapidly flickering pictures in the to-be attended hemifield. We found ssVEP phase acceleration and amplitude modulation to be additive functions of spatial attention and emotional content.
As a main effect of spatial attention, contralateral cortical facilitation was related to viewing attended material in both visual fields. In addition, aversive content further increased ssVEP amplitude, which was most pronounced when unpleasant pictures were presented to the left visual field. The ssVEP protocol used in the present study led to a high number of epochs — reflecting initial visual cortical processing — which were elicited in rapid succession. Since the eliciting stimulus remains the same over a period of seconds, top-down (re-entrant) modulation of the visual potential reflected in the ssVEP is possible, however, according to the affective content of the flickering stimulus. Previous work indeed suggests that local connectivity, as measured by spectral coherence, is correlated with increases during steady-state stimulation (Clementz et al., 2004). Thus, the ssVEP amplitude appears to reflect changes in neural mass activity, which are related to both initial sensory processing and re-entrant modulation. Our findings therefore seem to be in line with network perspectives of emotion, which emphasize that emotional processes encompass aspects of stimulus representation, memory and action, among others (Bower et al., 1994; Lang et al., 1997). Such theories predict that, at attended sites of the visual field, the amplitude of stimulus processing is increased as a function of affective relevance, possibly reflecting more coherent and more widespread networks (Keil, 2004). A study by Armony and Dolan (2002) supports this notion. In a hemifield paradigm, they used angry faces as conditioned stimuli (CS), and differentially predicted unpleasant bursts of white noise. As a main result, attention was automatically shifted towards the location of CS+ (i.e. the angry face predicting noise), which was accompanied by enhanced hemodynamic activity in a widespread neural network including fronto-parietal cortex.
Interference between Spatial and Natural Selective Attention
We did not find any evidence for interference between spatial selection and emotional content in the ssVEP data. The systems mediating sensory gain as a function of selection or affective relevance may rely on overlapping or partly identical cortical areas, but did not compete for resources in the present paradigm. In terms of spatially non-attended stimuli, several authors have proposed that affectively arousing stimuli automatically draw attention, resulting in enhanced processing accuracy without explicit instruction (Öhman et al., 2001). Recent work using emotional faces has demonstrated rapid (90 ms) spatial allocation of resources to angry faces compared to happy faces shown in the hemifields, resulting in early enhancement of the ERP (Pourtois et al., 2004). Our results indicate that non-attended affective information specifically modulated the ssVEP when present in the left visual field. This suggests that arousing information does not depend on spatial attentional resources when present in the left visual field. Interestingly, selective attention was reported to be a prerequisite for processing the emotional content of facial expressions (e.g. Pessoa et al., 2002). Eimer et al. (2003) observed differential early ERP amplitudes for emotional versus neutral faces only in conditions where spatial attention was not explicitly directed away from facial stimuli. The question whether processing of affectively arousing visual scenes differs from processing of affective facial expressions therefore remains an interesting topic of future research. Work employing measures of affective reflex physiology have indicated strong physiological reactivity to threat and mutilation pictures as opposed to angry or sad faces (Bradley et al., 2001). This implies that differences in arousal might contribute also to neurophysiological differences between these stimulus categories.
Topographical analysis of the linear relationship between the two types of attention examined here suggested a prominent role of areas projecting to posterior and fronto-temporal electrodes. In particular, the present topographical distribution of amplitude and phase modulations is consistent with sources located in occipital and occipito-temporal visual cortex, contralateral to the attended visual hemifield. Using magnetoencephalography and a distributed source modeling technique, Müller et al. (1997) localized the visual ssVEP in posterior occipital and ventral occipital cortex, which is consistent with the distribution observed in the present study. As concluded from earlier work based on latency measures of electrocortical recordings (Keil et al., 2003a; Schupp et al., 2003), and from visual cortical modulations seen in functional imaging data (Bradley et al., 2003), it seems likely that affective arousal acts to facilitate sensory processing, as has been suggested for spatial selective attention (Hillyard and Anllo-Vento, 1998).
Temporal and Spatial Dynamics of the ssVEP: Methodological Issues
Several authors have discussed methodological issues in terms of interpretation of ssVEP phase (Silberstein et al., 2001). In particular, interpretation as a direct measure of latency depends on the assumption that orientation and location of the structures generating the electric/magnetic field is stable across time and conditions, among other factors (Nunez et al., 1997). This is, however, not necessarily the case, and is difficult to determine, even when using techniques to improve spatial resolution of the ssVEP topography (Moratti et al., 2004). Differences in ssVEP phase during differential spatial selection of affectively arousing versus neutral stimuli suggested accelerated timing for attended arousing stimuli in the present experiment. Alternatively, additional temporal cortical sources or changed source configuration may have contributed to the phase differences observed.
Future work may address this issue by combining source space analyses with valid algorithms to estimate phase at a given source location. However, source space analyses using distributed source models as could be applied to our data create difficulties when estimating the phase of the signal. For instance, the minimum norm estimate (Hauk et al., 2002), which can easily be used as a pre-processing step in combination with time–frequency analyses (Moratti et al., 2004), produces several phase values for each model dipole orientation (i.e. three orientations in the case of EEG data) (Hauk et al., 2002), which makes interpretation of the respective phase angles difficult. In the present study, we intended to examine phase and amplitude derived from identical time series and thus did not combine source estimation techniques with frequency domain analyses. This approach has the disadvantage of being much less precise in terms of relating topographical patterns to brain structures. To minimize the latter effect, we used noise-weighted, spline-interpolated voltage data as suggested by Junghöfer et al. (1997).
Work on spatial attention has pointed to a right-hemispheric preponderance in spatial selection tasks (Corbetta, 1998). Similar findings have been reported for measures reflecting early electrocortical changes as a function of affective stimulus properties (Pizzagalli et al., 1999; Junghöfer et al., 2001). In the present study, the left hemifield attention condition was associated with stronger additive effects as measured by regional means analysis as well as F-contrast mapping, suggesting higher sensitivity of right-hemispheric structures to experimental manipulation. Statistically significant additive effects were also observed during attend-right conditions, however, and it is possible, indeed very likely, that interhemispheric information flow will interact with effects of hemifield presentation in early time segments of each trial. Thus, long presentation durations as used in the present design are not suitable for a valid analysis of hemisphere-specific processing.
Relationship between Behavioral and Electrophysiological Data
When considering behavioral data, the present task can be expected to produce interference of affective relevance in the attended stream with accuracy of report. For instance, Hartikainen et al. (2000) observed delayed reaction times in a spatial attention task, when targets were preceded by affectively arousing, compared to neutral, pictures from the IAPS. These differences were most pronounced when targets were present in the left hemisphere, following presentation of unpleasant pictures. This effect has been interpreted as reflecting enhanced attention to arousing pictures, thus interfering with the concurrent task. Behavioral work using conditioned stimuli as hemifield cues in spatial attention tasks has demonstrated that affectively arousing cues facilitate switching of attention to a given hemifield (Stormark and Hugdahl, 1996). Thus, while attention switching to a given hemifield may be enhanced by the presence of affectively arousing stimuli, concurrent processing of affective pictures and an additional task may cause interference. In the present study, participants' responses were slower when indicating the number of targets embedded in an aversive, compared to a neutral, picture stream, thus paralleling previous results with tasks that compete with affective picture processing (Isenberg et al., 1999). Furthermore, error rate was sensitive to the information shown in the non-attended visual field. When presented to the left visual field, non-attended aversive pictures significantly disrupted reporting accuracy, paralleling the findings by Hartikainen et al. (2000). Thus, at least for the left visual hemifield, behavioral data clearly point to interference due to unpleasant pictures, which are not attended. This seems to be in line with enhancement of ssVEP amplitude in that same condition as observed here, suggesting that resources are being allocated to the unpleasant stimulus even when this is not in the attended visual hemifield. In combination, behavioral and ssVEP data therefore indicate that right-hemispheric visual systems are particularly sensitive to affectively arousing content, and that aversive stimuli automatically attract attentional resources, interfering with subsequent task performance in a competing region of the field of view.
The present study suggests that ssVEP data can be used to investigate timing and amplitude of cortical responses to affective stimuli presented in different task conditions. By manipulating spatial selection and emotional content of stimuli, ssVEPs indicated cortical facilitation in the visual areas contralateral to the attended hemifield, across affective categories. This finding replicates observations obtained in spatial selective attention research (Morgan et al., 1996; Müller and Hillyard, 2000), which have been interpreted as reflecting gain control changes in specific networks underlying a given task. The time course and topography of re-entrant processes mediating sensory gain as a function of affective/motivational relevance can be examined in future studies that exploit the present approach. Furthermore, by using ssVEPs together with behavioral tasks, interactions between motivated/affective perceptual facilitation and affective action can be studied, which is a prerequisite for the understanding of emotional processes following initial stimulus analysis.
In the present study, directed and natural attention activated neural processes that overlapped in both temporal characteristics and spatial location. The behavioral data suggest that, in competition for these common resources, performance on the spatial attention task suffered. That is, both speed and accuracy decreased in the context of unpleasant arousing pictures. Furthermore, the emotional pictures prompted an enhanced electrocortical response even when attention was directed away from their spatial location. These findings are consistent with the view that emotional stimuli activate motive systems involving threat or appetite (Lang et al., 1997), modulating the early stages of stimulus processing. This motivational priority can compromise (or perhaps in some circumstances enhance) stimulus selection and response execution in subsequent transactions. Using frequency-domain analysis techniques, and with measures to insure inter-site phase locking, future efforts will focus on more clearly delineating the neural dynamics of these interacting attentional processes.
The authors would like to thank Ursula Lommen for help in data acquisition. This research was supported by the Deutsche Forschungsgemeinschaft.
1University of Konstanz and 2University of Florida