Abstract

Strong evidence suggests that memory for emotional information is much better than for neutral one. Thus, one may expect that forgetting of emotional information is difficult and requires considerable effort. The aim of this item-method directed forgetting functional magnetic resonance imaging study was to investigate this hypothesis both at behavioral and neural levels. Directed forgetting effects were observed for both neutral and emotionally negative International Affective Picture System images. Moreover, recognition rate of negative to-be-forgotten images was higher than in case of neutral ones. In the study phase, intention to forget and successful forgetting of emotionally negative images were associated with widespread activations extending from the anterior to posterior regions mainly in the right hemisphere, whereas in the case of neutral images, they were associated with just one cluster of activation in the right lingual gyrus. Therefore, forgetting of emotional information seems to be a demanding process that strongly activates a distributed neural network in the right hemisphere. In the test phase, in turn, successfully forgotten images—either neutral or emotionally negative—were associated with virtually no activation, even at the lowered P value threshold. These results suggest that intentional inhibition during encoding may be an efficient strategy to cope with emotionally negative memories.

Introduction

In everyday life situations, forgetting is often viewed as a memory failure (Neath and Surprenant 2003). In many cases, however, forgetting is not just a failure to encode, maintain, and/or retrieve information but it may have a positive adaptive function as it prevents irrelevant or outdated information from intruding on memory (Bjork 1989). In experimental studies on this phenomenon, the selection of information to forget and to remember is aided by explicit cues that enable flexible memory control.

Recently, the directed forgetting paradigm and the think/no-think paradigm are the major methods used to investigate the issue of memory control. There are 2 variants of the directed forgetting paradigm (for a review, see Basden BH and Basden DR 1996, 1998): item-method (e.g., MacLeod 1989) and list-method (e.g., Geiselman et al. 1983; Conway et al. 2000). Both paradigms are concerned with the effect of memory instruction on subsequent memory performance. They differ, however, in respect of the memory instruction timing relative to the study items (Bjork 1972). In the item-method paradigm, study items are individually cued to-be-remembered (TBR) or to-be-forgotten (TBF) on a trial-by-trial basis: “remember” (R) or “forget” (F) instruction follows the presentation of each study item. In the list-method paradigm, the study items are split into 2 lists. Following the first list (List-1), half of the participants are told to forget this list of items and the other half to keep remembering it. The 2 groups then study the second list (List-2) (e.g., Epstein 1972; Geiselman et al. 1983).

Afterward, memory is unexpectedly tested for all items, irrespective of the previous memory instructions (MacLeod 1975, 1989). Regardless of which paradigm is used, TBF items generally show impaired recall compared with TBR items. This effect is known as the directed forgetting effect (for reviews of directed forgetting methodologies and their findings, see Johnson 1994; Basden BH and Basden DR 1996; Bjork et al. 1998; MacLeod 1998). Whereas both item- and list-methods produce directed forgetting in recall, the 2 methods dissociate when items are presented in the recognition test: directed forgetting is eliminated for the list-method but maintained for the item-method (Basden et al. 1993; MacLeod 1998). This difference may suggest that the effect of R and F instructions within these 2 paradigms may be mediated by distinct underlying mechanisms (Bjork 1989; Basden et al. 1993). This observation has resulted in research that compares item-method tasks with list-method tasks in an effort to discern the mechanisms through which R and F instructions operate to influence later memory performance (Basden et al. 1993; MacLeod 1999; Conway and Fthenaki 2003).

When the item-method is used, the selective encoding explanations are generally favored (e.g., Bjork 1972). This methodology is thought to lead to segregation in memory of the individually cued TBF and TBR items and selective rehearsal of only the TBR items (Bjork 1972). This selective encoding hypothesis suggests that each item is maintained in active memory until the cue is presented, and then, if the cue is to remember the item, it is processed further (i.e., rehearsed). In contrast, when the cue is to forget, then that item is dropped from active memory and it is not further rehearsed. The selective encoding explanation that has been applied to the item-method has not fared so well when applied to the list-method. In the list-method, participants are initially instructed to remember the entire list of stimuli. The ability to forget some stimuli after they have been deeply encoded has been very often explained by inhibition at the time of retrieval (Bjork 1989; Basden et al. 2003). The retrieval inhibition account assumes that F-cued participants engage in active inhibitory processes that reduce access to List-1 items and, due to the resulting decrease in these items' interference potential, facilitate memory for List-2 items (Geiselman et al. 1983; for alternative explanations see Sahakyan and Kelley 2002; Sahakyan and Delaney 2003; Sheard and MacLeod 2005; Bäuml et al. 2008). On re-exposure of stimuli during recognition tests, the retrieval inhibition is released, and for that reason, the TBF stimuli are available not only for recognition but also for subsequent recall, thereby eliminating the directed forgetting effect.

A recently developed think/no-think paradigm relates well to the list-method directed forgetting paradigm (Anderson and Green 2001; Anderson 2003; Anderson et al. 2004; Levy and Anderson 2008; Anderson and Levy 2009). In this paradigm, participants initially learn unrelated word pairs and then complete a retrieval practice phase. On each trial of the retrieval practice phase, participants are presented with one word from some of the pairs that they initially learned. On think trials, they are asked to retrieve the paired word, but on no-think trials, they are asked to inhibit retrieval of the paired word. Results of later memory tests indicate that the no-think phase inhibits memory for the learned word pairs such that the no-think pairs are less remembered than even unpracticed pairs. This paradigm is analogous to list-method directed forgetting because it examines how inhibitory processes can affect information that was initially encoded as TBR.

Growing interest in brain correlates of forgetting has been reflected in a rapidly increasing number of studies in this field. In the list-method directed forgetting paradigm, Bäuml et al. (2008) explored oscillatory correlates of memory updating. Measuring electroencephalographies during List-2 encoding, they identified 2 effects of the F cue on oscillatory function: an increase in upper alpha power and a reduction in upper alpha phase coupling (11–13 Hz). Whereas the increase in power was related to List-2 enhancement, the reduced phase coupling was related to List-1 forgetting. Their results pointed to neural origins of forgetting and showed that alpha oscillations play a critical role in intentional updating of episodic memory, being related to top–down processes and active inhibitory function. In addition, an increase in the amount of postcue encoding led to an increase in List-1 forgetting but did not affect List-2 enhancement (Pastötter and Bäuml 2010).

In the item-method directed forgetting study, Paz-Caballero et al. (2004) investigated brain activity related to the processing of F and R instructions as revealed by the event-related potentials (ERPs). The subjects were subdivided into low- and high-forget effect groups. The F cue elicited early enhanced positive activity in the frontal and prefrontal areas only in the high-forget effect group, probably reflecting the activation of inhibitory processes in this subgroup. On the other hand, no ERP effects were found in the low-forget effect subjects: the group that still remembered a large number of items. Nowicka, Jednoróg, Marchewka, and Brechmann (2009), in turn, examined whether the ability to correctly recollect a high number of TBF stimuli is reflected in the structure of brain regions involved in both memory and the control of retrieval processes, in the item-method directed forgetting paradigm. In subjects with high recognition rates for TBF stimuli, voxel-based morphometry revealed increased gray matter volume in the left ventrolateral prefrontal cortex and the right hippocampus. In addition, gray matter volume in these regions correlated positively with the TBF recognition rate, indicating that the right hippocampus and left ventrolateral prefrontal cortex are of particular relevance in releasing TBF items from inhibition caused by the F instruction. Hsieh et al. (2009) investigated ERPs time locked to study items and to R/F cues in relation to the subsequent recognition performance. The study items in the directed forgetting task did not yield reliable subsequent memory effects. However, the R/F cues gave rise to ERPs that were predictive of the subsequent recognition performance. Specifically, F cues yielded ERPs that were more positive going over anterior brain regions, confirming results reported by Paz-Caballero et al. (2004). Interestingly, in the test/retrieval phase of item-method directed forgetting studies, the absence of the old/new effect for correctly retrieved TBF items was reported (Ullsperger et al. 2000; Nowicka, Jednoróg, Wypych, and Marchewka 2009), whereas forgotten TBF words yielded ERPs that were more negative going than ERPs for correctly rejected new items: the reversed old/new effect (Nowicka, Jednoróg, Wypych, and Marchewka 2009; Van Hooff et al. 2009). These findings may indicate the presence of some inhibitory processes activated by the F instruction at the time of encoding.

Two functional magnetic resonance imaging (fMRI) studies investigated activations during the study phase of the item-method directed forgetting paradigm (Reber et al. 2002; Wylie et al. 2008). Whereas Reber et al. (2002) were interested in differentiating encoding effort and encoding success during the study phase (they reported activations associated only with R instruction and subsequently recalled stimuli, irrespective of memory instruction), Wylie et al. (2008) focused on the question whether forgetting may be viewed as an active process. In their study, correct/incorrect responses made during the recognition of TBR and TBF words in the test phase were used post hoc to separate single trials with R/F instruction during the study phase that resulted in actual remembering/forgetting. F instruction contrasted with R instruction revealed activations in superior medial, middle frontal, middle temporal, and parahippocampal gyri. When contrasted with unintentional forgetting, intentional forgetting was associated with increased activity in the middle frontal gyrus, inferior parietal lobule, and parahippocampal gyrus.

In the think/no-think study, Anderson and Green (2001) showed that forgetting 1) increases with the number of times the specific memory is avoided, 2) resists incentives for accurate recall, and 3) is caused by processes that suppress the memory itself. Anderson et al. (2004), in turn, used fMRI to contrast brain activity during no-think and think trials and found that suppressing retrieval (i.e., controlling unwanted memories) was associated with increased activations of the lateral prefrontal cortex, which could reduce activity in the hippocampus, thereby impairing retention of unwanted memories. Individual differences in the efficacy of executive control mechanisms may underlie variation in how well people control intrusive memories (Levy and Anderson 2008; Anderson and Levy 2009). Using the think/no-think paradigm, Bergström et al. (2007) recently examined the ERP correlates of retrieval and of avoiding retrieval. The main finding of the study of Bergström et al. (2007) was that a parietal positivity was attenuated for learned no-think trials in comparison to learned think trials. That attenuation of the parietal activity was confirmed by Mecklinger et al. (2009). Findings of the think/no-think ERP studies correspond to the absence of the old/new effect, or the presence of the reversed old/new effect, found in the item-method ERP studies of directed forgetting.

An interesting issue is whether such flexible memory control leading to successful forgetting is effective in case of emotional information, the negative one in particular. It is well documented that the emotional nature of events or test items strongly influences human memory. Memories for emotional stimuli have a persistence and vividness that other memories seem to lack (Christianson 1992). Memory performance has often been found to be greater for emotionally arousing than neutral stimuli (Rubin and Friendly 1986; Bradley et al. 1992; Palomba et al. 1997; Ochsner 2000). Specifically, the recall of emotionally negative items is enhanced relative to the recall of neutral items (e.g., Danion et al. 1995; Phelps et al. 1997). Several cognitive factors have been hypothesized to account for this effect, including enhanced attention for emotional stimuli, greater elaboration during encoding of emotional stimuli, their greater distinctiveness, and the increased rehearsal of these stimuli (Reisberg and Heuer 1992). Thus, if the memory for emotional stimuli is so strong, it is likely that forgetting them will be relatively difficult.

Behavioral studies on intentional forgetting in the context of emotion have revealed inconsistent findings. Many of these studies were focused on investigating clinical disorders such as obsessive–compulsive disorder (Wilhelm et al. 1996), depression (Joormann et al. 2005; Cottencin et al. 2008), borderline personality disorder (Korfine and Hooley 2000), autism spectrum disorder (Gaig and Bowler 2008), and posttraumatic stress disorder (McNally et al. 1998; Zoellner et al. 2003). Some of them report normal directed forgetting effects for emotional stimuli (Wilhelm et al. 1996; McNally et al. 1998; McNally et al. 1999; Dumont 2000; Elzinga et al. 2000; Korfine and Hooley 2000; Moulds and Bryant 2002; Tolin et al. 2002; DePrince and Freyd 2004; Devilly et al. 2007) or even stronger effects for emotional than neutral items (Moulds and Bryant 2005), whereas others report no directed forgetting effects for emotional material (Myers and Derakshan 2004; Payne and Corrigan 2007). These discrepant results may result from differences in groups of tested subjects (only clinical, e.g., depressed groups, without a control group vs. clinical and control groups), applied paradigms of directed forgetting (list-method vs. item-method), stimuli used (words vs. complex colored images), and, finally, emotional content of stimuli (positive and negative compared with neutral vs. positive compared with negative).

Interestingly, Joslyn and Oakes (2005) demonstrated directed forgetting for stimuli as complex as autobiographical events. Using a 2-week diary paradigm, they compared recall between a group of participants who were directed to forget Week 1 memories (F group) and a group who did not receive the forget instruction (R group). Generally, the F group remembered fewer items from Week 1 than did the R group. The effect was observed both for negative and positive valence events, as well as for high– and low–emotional intensity events. These findings were replicated by Barnier et al. (2007) who observed directed forgetting effects for autobiographical memories, irrespective of their emotional content (the F instruction impaired negative, positive, and neutral memories equally).

The only fMRI study that investigated the issue of emotional information forgetting was done in the think/no-think paradigm (Depue et al. 2007). In that study, faces were used as cues and pictures of aversive scenes as targets. Aversive scenes were images taken from International Affective Picture System (IAPS) (Lang et al. 2001). Suppression of retrieval of aversive scenes in no-think trials was associated with increased activation in a number of frontal regions in the right hemisphere and reduced activation in the amygdala—a structure implicated in emotion processing. Importantly, during no-think trials, both the hippocampus and amygdala were not only less engaged than during think trials but also even less active than when people simply viewed an empty screen. These results suggest that overriding retrieval involves active disengaging of these brain regions. However, in this study, all targets were aversive images; thus, it was not possible to compare whether suppressing of emotionally negative memories and suppressing of neutral memories are associated with activations in the overlapping brain regions.

The aim of this directed forgetting fMRI study was to investigate the issue of forgetting of emotional and neutral information at both the behavioral and neural levels. Even if there is no difference between emotional and neutral items in directed forgetting effects or there is no directed forgetting effect for emotional material, some changes related to forgetting of such information might be observed at the neural level. Following Depue et al. (2007) and Payne and Corrigan (2007), we decided to use IAPS images as a set of emotional and neutral stimuli.

As we were interested in how encoding and retrieval processes contribute to intentional forgetting, fMRI data were collected during both the study phase and the recognition (test) phase. During the study phase, our attention was focused on changes in brain activity associated with the F instruction that resulted in successful forgetting, as revealed post hoc by the behavioral results from the recognition phase. In addition, we tested whether intentional forgetting differs from incidental forgetting (i.e., R trials that were unsuccessful and led to forgetting), the former resulting from the subjects' conscious action and the latter being due to memory failure. Specifically, we aimed to determine whether successful forgetting of emotional stimuli engages similar brain regions as the forgetting of neutral stimuli. We hypothesized that if forgetting of emotional images is more difficult than forgetting of neutral information, it may be associated with stronger/more widespread activations. This prediction is based on numerous studies showing that the more difficult the task the stronger/larger the activations (e.g., Gould et al. 2003; Erickson et al. 2007). Based on the study of Wylie et al. (2008), activations associated with forgetting may be expected within frontal and temporal regions. However, in the study of Wylie et al. (2008), words served as TBF and TBR stimuli, whereas in our study, we used IAPS images. For this reason, we expected to find some additional activations in more posterior regions, involved in visual processing (e.g., Lang et al. 1998; Britton et al. 2006).

During the recognition phase of the study, we investigated brain activity related to intentionally forgotten (TBF and actually forgotten) and incidentally forgotten (TBR but forgotten) emotional and neutral stimuli. We were interested to see whether processing of intentionally and incidentally forgotten stimuli differs when compared with correctly rejected new stimuli (the behavioral response was the same for all these stimuli: subjects indicated that they were new to them) and whether the emotional content matters. Assuming that forgetting of emotional stimuli is difficult and requires considerable effort, we expected that forgotten emotional items might still leave some memory traces that would result in significant activations in structures involved in memory and/or processing of emotional stimuli.

Materials and Methods

Subjects

Twenty-three healthy right-handed subjects (mean age 27.1 years) participated in the study. All subjects gave their written informed consent to the study, which was approved by the Ethical Committee of the University of Magdeburg. They were undergraduates, PhD students, or young employees of the University of Magdeburg. The data from 7 subjects were excluded from fMRI analyses because of a high number (>20%) of missed trials, that is, trials with no response (4 subjects) or the small number (<5) of trials of a given type (3 subjects). The mean age of the remaining 16 subjects (8 females) was 26.6.

Procedures

The set of stimuli consisted of images taken from the IAPS (Lang et al. 2001). Stimuli (image size 1024 × 768 pixels) were displayed using Presentation software (Neurobehavioral Systems) and back projected to a mirror system mounted on the magnetic resonance imaging (MRI) scanner head coil. Two classes of stimuli were selected according to the original scores (Lang et al. 2001): emotionally negative with low valence (mean = 2.78, standard deviation [SD] = 0.91) and high arousal (mean = 5.73, SD = 1.12) and emotionally neutral with relatively high valence (mean = 6.03, SD = 1.54) and low arousal (mean = 3.78, SD = 1.41). Stimuli were fully counterbalanced with respect to their content, valence, and arousal between the study and test phases and between TBF and TBR conditions. Emotionally negative images depicted such basic emotions as fear and disgust.

The study consisted of 2 parts. In the first part (study phase), 120 pictures (60 neutral and 60 emotionally negative) were presented to each subject; half were followed by the R instruction and the other half by the F instruction. These instructions were given visually. The order of experimental trials was pseudorandom with the constraint of no more than 3 consecutive trials with the same type of instruction or the same type of stimulus. The sequence of events in a single experimental trial was following: an image was displayed for 500 ms, followed by the fixation cross (1500 ms) and then the memory instruction (i.e., the word REMEMBER or FORGET) was presented for 1500 ms. The postcue length lasted 6, 6.5, or 7 s. It varied pseudorandomly from trial to trial, with equal probability of each of them.

In the second part (test phase), all stimuli were re-presented and mixed pseudorandomly with new pictures (60 neutral and 60 emotionally negative). Subjects had to categorize each picture, irrespective of the F/R instruction, as old (displayed in the study phase) or new using a standard response pad. Trials were mixed pseudorandomly and fully counterbalanced with respect to all experimental conditions (old/new and emotionally negative/neutral stimuli). All images were presented for 500 ms, with the interstimulus interval of 6, 6.5, or 7 s.

Imaging

Magnetic resonance imaging was carried out using a 3-Tesla Trio MRI scanner (Siemens Medical Solutions) equipped with an 8-channel phased array coil. Detailed anatomical data of the whole brain were acquired using a multiplanar rapidly acquired gradient echo sequence with 1.0-mm isotropic resolution. Functional images were acquired using an echo planar imaging pulse sequence (field of view 224 mm, matrix 64 × 64, slice thickness 3.5 mm, time echo 30 ms, time repetition 2000 ms, flip angle 80°). Thirty-four contiguous, oblique–axial images oriented parallel to the anterior–posterior commissural plane were acquired with a total of 397 volumes during the study phase and 673 brain volumes during test phase.

fMRI Data Analysis

Behavioral data were used to sort the fMRI data based on the memory instruction and behavioral outcome. Imaging data were analyzed using Statistical Parametric Mapping software (SPM8, Wellcome Department of Cognitive Neurology). First, functional images were motion corrected. Then, the structural images from single subjects were coregistered to the mean functional images. The unified normalization routine was conducted (Crinion et al. 2007) with voxel size 2 × 2 × 2 mm. Finally, data were smoothed with the 8-mm full-width, half-maximum Gaussian kernel.

Timings for all experimental conditions were entered into the design matrix. The hemodynamic response was modeled using canonical hemodynamic response function implemented in SPM software (Friston 2003). Changes in blood oxygen level–dependent (BOLD) signal were assessed both for memory instruction and the postcue period. The correction for multiple comparisons was achieved by including a cluster-level threshold of 10 contiguous voxels family-wise error (FWE) corrected, at P < 0.05. Small volume correction, as implemented in SPM8, was used where we had prior hypotheses. In such cases, we interrogated a spherical volume corresponding to number of active voxels within a peak of cluster activation (radius = 10 mm). For this small volume, a cluster of 10 contiguous voxels was sufficient to correct for multiple comparisons.

The Montreal Neurological Institute coordinates were translated to Talairach space using icbm2tal transform (Lancaster et al. 2007). Then, TalairachClient 2.4.2 was used to identify the activated structures (Lancaster et al. 2000; www.talairach.org).

In this study, stimuli presented to the subject were characterized by 2 factors: the absence/presence of emotional content and the memory instruction that followed each stimulus during the encoding (study phase). All analyses were performed in a way that enabled us to compare the influence of 2 different memory instructions combined with a specific behavioral outcome (correct or incorrect) while keeping the emotional content constant.

Results

Behavioral Results

Analyses of behavioral results were done for the whole group of tested subjects (23), and then these analyses were repeated for the group of 16 subjects who were included into fMRI analyses. In this way, we wanted to check whether exclusion of 7 out of 23 subjects (this was done to satisfy the requirements of sufficient number of trials for fMRI analyses) influenced the pattern of behavioral results. Below are presented results for both groups.

Group of 23 Subjects

Following Wylie et al. (2008), the effectiveness of the directed forgetting paradigm was checked by analyzing trials with subjects' responses, indicating that a given stimulus was “old.” Thus, recognition rates for TBR and TBF stimuli and false alarms (new stimuli judged as old ones) were analyzed using analysis of variance (ANOVA) with type of stimulus (TBR_R, TBF_R, and false alarms) and type of emotion (negative and neutral) as factors. It revealed a significant main effect of both factors (the type of stimulus: F2,21 = 97.47, P < 0.001 and emotion: F1,22 = 15.93, P < 0.001) and their interaction (F2,21 = 5.62, P < 0.05). Planned contrasts of the directed forgetting effect revealed that recognition rate for TBR items was significantly higher than that for TBF items (64.5%, SD = 3.6 vs. 51%, SD = 3.2, P < 0.001). This result clearly indicated that the R instruction and the F instruction exerted expected influence on TBR and TBF images, respectively. However, the correct recognition rate for TBF was significantly higher than the false alarms' rate (P < 0.001). The latter finding undermines the supposition that the near-chance TBF recognition rate reflected just random responding: although participants recognized fewer TBF than TBR images, they did, in fact, recognize a significant number of TBF items. Interestingly, directed forgetting effect was significant for both neutral and emotionally negative images (P < 0.001 and P < 0.005, respectively). On the other hand, emotional content of images facilitated correct recognition of TBF (P < 0.05) and induced more false alarms (P < 0.001). Results are presented in Figure 1A.

Figure 1.

Percentage of correctly recognized TBR and TBF images (TBR_R and TBF_R, respectively) and percentage of false alarms. (A) all participants (23), (B) the group of 16 subjects included into fMRI analyses. Bars represent SD; E, emotionally negative images; N, neutral images.

Figure 1.

Percentage of correctly recognized TBR and TBF images (TBR_R and TBF_R, respectively) and percentage of false alarms. (A) all participants (23), (B) the group of 16 subjects included into fMRI analyses. Bars represent SD; E, emotionally negative images; N, neutral images.

Group of 16 Subjects

Generally, findings obtained for the whole group of participants were replicated in the final group of subjects for which fMRI analyses were performed (see Fig. 1B). Specifically, ANOVA with type of stimulus (TBR_R, TBF_R, and false alarms) and type of emotion (negative and neutral) as factors revealed a significant main effect of both factors (the type of stimulus: F2,14 = 123.14, P < 0.001 and emotion: F1,15 = 11.92, P < 0.005) and their interaction (F2,14 = 14.06, P < 0.001). A significant directed forgetting effect was observed: recognition rate for TBR items was significantly higher than that for TBF items (72.8%, SD = 2.5 vs. 56.9%, SD = 3.5, P < 0.001), as revealed by planned contrasts. The directed forgetting effect was significant for both neutral and emotionally negative images (P < 0.001 and P < 0.005, respectively). Again the correct recognition rate for TBF was significantly higher than the false alarms' rate (P < 0.001), thus indicating that subjects were not responding at random level. Again the recognition rate of emotionally negative TBF images was higher than the recognition rate for neutral TBF images (P < 0.05). The results are presented in Figure 1B.

fMRI Results

Study Phase

Since the major goal of this study was to trace brain correlates of forgetting of emotional material—especially intentional forgetting—the initial analyses directly contrasted the influence of the F and R instruction in separate trials with presentations of emotionally negative and neutral IAPS images. Table 1 summarizes the fMRI results from the study phase. The results of the F > R contrast revealed clear differences between the forgetting of emotional and neutral items (Fig. 2A). In the case of emotionally negative images, forgetting such stimuli resulted in the strong activation of a distributed neural network comprising regions of the frontal (Brodmann area [BA] 6), temporal (BA 21), parietal (BA 7), occipital (BA 18 and BA 19), and limbic lobes, located mainly in the right hemisphere (see Table 1). Specifically, the right cuneus, the right precuneus, and the right parahippocampal gyrus were among structures activated by the subjects' intention to forget emotional material. In the case of neutral images, however, an effort to forget led to just one cluster of activation in the right lingual gyrus in the occipital lobe. The total number of voxels activated by F instruction were 3245 and 47 for trials with emotional and neutral images, respectively.

Table 1

The study phase—regions of significant activations

Contrast BA X Y Z T statistic Voxels 
Emotional images 
    F > R 
        R, parietal lobe, precuneus −68 40 6.01a 3190 
        L, occipital lobe, lingual gyrus 18 −22 −76 −12 5.89a  
        R, occipital lobe, cuneus 19 −84 28 5.65a  
        R, temporal lobe, middle temporal gyrus 21 48 −32 5.01b 37 
        R, limbic lobe, parahippocampal gyrus 20 62 −6 −28 4.96b 18 
        R, frontal lobe, middle frontal gyrus 22 −16 50 4.09b 10 
    F_F > R_F 
        R, occipital lobe, fusiform gyrus 19 20 −84 −12 4.97a 944 
        L, occipital lobe, lingual gyrus 18 −16 −80 −10 4.12a 255 
        L, occipital lobe, fusiform gyrus 19 −28 −76 −12 4.12a  
        R, frontal lobe, middle frontal gyrus 44 10 52 4.02b 196 
        R, frontal lobe, superior frontal gyrus 10 12 66 20 3.83b 21 
        R, parietal lobe, inferior parietal lobule 40 50 −58 46 3.58b 57 
Neutral images 
    F > R 
        R, occipital lobe, lingual gyrus 17 12 −86 −2 4.94a 47 
    F_F > R_F 
        R, occipital lobe, lingual gyrus 17 16 −86 3.60a 11 
Contrast BA X Y Z T statistic Voxels 
Emotional images 
    F > R 
        R, parietal lobe, precuneus −68 40 6.01a 3190 
        L, occipital lobe, lingual gyrus 18 −22 −76 −12 5.89a  
        R, occipital lobe, cuneus 19 −84 28 5.65a  
        R, temporal lobe, middle temporal gyrus 21 48 −32 5.01b 37 
        R, limbic lobe, parahippocampal gyrus 20 62 −6 −28 4.96b 18 
        R, frontal lobe, middle frontal gyrus 22 −16 50 4.09b 10 
    F_F > R_F 
        R, occipital lobe, fusiform gyrus 19 20 −84 −12 4.97a 944 
        L, occipital lobe, lingual gyrus 18 −16 −80 −10 4.12a 255 
        L, occipital lobe, fusiform gyrus 19 −28 −76 −12 4.12a  
        R, frontal lobe, middle frontal gyrus 44 10 52 4.02b 196 
        R, frontal lobe, superior frontal gyrus 10 12 66 20 3.83b 21 
        R, parietal lobe, inferior parietal lobule 40 50 −58 46 3.58b 57 
Neutral images 
    F > R 
        R, occipital lobe, lingual gyrus 17 12 −86 −2 4.94a 47 
    F_F > R_F 
        R, occipital lobe, lingual gyrus 17 16 −86 3.60a 11 

P < 0.05, FWE corrected;

a

exploratory analyses;

b

small volume correction; F, “forget” instruction; R, “remember” instruction; F_F, successful F instruction (F instruction leading to intentional forgetting); R_R, successful R instruction; R_F, unsuccessful R instruction (R instruction leading to incidental forgetting). Activations are given in Montreal Neurological Institute coordinates.

Figure 2.

The study phase. (A) Effect of memory instruction: intention to forget contrasted with intention to remember (F instruction > R instruction for all trials); (B) comparison of intentional and incidental forgetting (F_F > R_F); left panel, emotionally negative images; right panel, neutral images. Significant group activations are superimposed on a normalized single subject's T1 image.

Figure 2.

The study phase. (A) Effect of memory instruction: intention to forget contrasted with intention to remember (F instruction > R instruction for all trials); (B) comparison of intentional and incidental forgetting (F_F > R_F); left panel, emotionally negative images; right panel, neutral images. Significant group activations are superimposed on a normalized single subject's T1 image.

In addition, successful intentional forgetting (F_F) and incidental forgetting (R_F) were directly compared, again separately for the 2 types of stimuli. In the case of emotionally negative stimuli, this comparison revealed activations of frontal (BA 6, middle frontal gyrus; BA 10, superior frontal gyrus), parietal (BA 40, inferior parietal lobule), and occipital (BA 19, fusiform gyrus; BA 18, lingual gyrus; BA 19, cuneus) regions of the brain, whereas in the case of neutral images, only the lingual gyrus (BA 17) was active (Table 1, Fig. 2B).

Test Phase

fMRI analyses during the test phase of the experiment were focused on TBF_F and TBR_F trials. Direct contrasts between intentionally forgotten (TBF_F) and new images (correct rejection) revealed virtually no activation accompanying either neutral or emotionally negative images (Fig. 3). Similarly, no activation was observed for incidentally forgotten neutral and emotionally negative (TBR_F) images contrasted with new images (Fig. 3). Thus, intentionally and incidentally forgotten stimuli did not differ from correctly classified new images at the neural level: the behavioral responses were the same for TBF_F, TBR_F, and new items. In each case, a lack of activation was observed not only for P values that were FWE corrected, but also even if P values were uncorrected for multiple repetitions, uncorrected P values were lowered, and the extent threshold k was set at 0 voxels.

Figure 3.

The test (recognition) phase - the lack of activation for intentionally and incidentally forgotten items. The same image applies to 4 contrasts: TBF_F vs. CR and TBR_F vs. CR for neutral images and TBF_F vs. CR and TBR_F vs. CR for emotionally negative images. TBF_F are forgotten TBF images, TBR_F are forgotten TBR images and CR are correct rejections (i.e., new/unstudied images correctly classified as new).

Figure 3.

The test (recognition) phase - the lack of activation for intentionally and incidentally forgotten items. The same image applies to 4 contrasts: TBF_F vs. CR and TBR_F vs. CR for neutral images and TBF_F vs. CR and TBR_F vs. CR for emotionally negative images. TBF_F are forgotten TBF images, TBR_F are forgotten TBR images and CR are correct rejections (i.e., new/unstudied images correctly classified as new).

Discussion

The main goal of this study was to investigate the differences between intentional forgetting of emotionally negative and neutral information. The behavioral findings indicated that significant directed forgetting effects were present both for emotionally negative and neutral IAPS images: more TBR images were recalled than TBF images. However, the recognition rate of TBF was higher in the case of emotionally negative stimuli in comparison to neutral stimuli, that is, less TBF negative images were forgotten than neutral ones. Our behavioral findings are in line with findings of other studies that report directed forgetting effects for emotional information (Wilhelm et al. 1996; McNally et al. 1998, 1999; Dumont 2000; Elzinga et al. 2000; Korfine and Hooley 2000; Moulds and Bryant 2002, 2005; Tolin et al. 2002; DePrince and Freyd 2004; Joslyn and Oakes 2005; Barnier et al. 2007; Devilly et al. 2007).

At the neural level, stark differences in the forgetting of emotional and neutral information were observed during the study phase of the experiment. The intention to forget emotionally negative pictures, whether it was successful or not, resulted in strong activations involving the middle frontal gyrus, middle temporal gyrus, parahippocampal gyrus, precuneus and cuneus in the right hemisphere, and lingual gyrus in both hemispheres. However, the intention to forget neutral images resulted solely in activation of the right lingual gyrus. Interestingly, the comparison of intentional forgetting (successful F instruction) and incidental forgetting (unsuccessful R instruction) again revealed strong activation of a distributed network involving the middle frontal gyrus, superior frontal gyrus, inferior parietal lobule, fusiform gyrus, and lingual gyrus in the case of negative pictures but only a single cluster of activations in the lingual gyrus in the case of neutral pictures. These findings indicate that at the neural level, intentional forgetting can be readily distinguished from forgetting as a result of memory failure. However, in the test (recognition) phase, neutral or emotionally negative images that had been forgotten, either intentionally or incidentally, did not differ from newly encountered images, as revealed both by the subjects' response and the lack of activations. Thus, forgotten items, whether they were emotional or not, did not leave memory traces visible as changes in BOLD signal.

Our behavioral and fMRI findings provide converging evidence indicating that forgetting of emotional information is more difficult (less negative TBF was forgotten) and more effortful (widely distributed network was associated with either intention to forget or actual successful forgetting) than forgetting of neutral information. Forgetting, in general, may be viewed as an effortful process that is cognitively more demanding than is remembering (Fawcett and Taylor 2008). Our findings extend this notion indicating that in case of emotional material, forgetting is even more effortful, and they are in line with studies showing that the more difficult the task the larger the activations (e.g., Gould et al. 2003; Landau et al. 2004; Kelly and Garavan 2005; Erickson et al. 2007). Finally, results of our study favor the idea that processes activated by F instruction and occurring during the study phase are indispensable and sufficing for successful forgetting in the item-method directed forgetting paradigm.

The latter notion is directly related to the question whether the differential memory performance for TBR and TBF items in item-method directed forgetting paradigm is solely due to differential encoding of TBR and TBF items or whether multiple mechanisms (e.g., operating at retrieval) underlie the effects of intentional forgetting (Johnson 1994; Zacks and Hasher 1994; Anderson and Neely 1996). The rationale for different encoding hypothesis comes from the notion that R and F instructions may serve as cues used by subjects to decide how to process the item during encoding. If the item is cued TBR, subjects engage in more elaborate encoding, but if it is cued TBF, subjects do no devote further encoding efforts. Inefficient encoding may subsequently result in forgetting. Thus, TBR items may be viewed as deeply encoded stimuli in contrast to shallowly encoded TBF items. Following this line of reasoning, Ullsperger et al. (2000) directly addressed this issue by comparing the ERPs for TBF and TBR words with the ERPs for shallowly and deeply encoded words. Both deeply and shallowly encoded stimuli elicited similar effects (i.e., the old/new effect) that differed only quantitatively, whereas TBR and TBF stimuli elicited significantly different ERP effects—the typical old/new effect for TBR items and absence of such effect in case of TBF items. Ullsperger et al. (2000) concluded that their results may suggest that differential encoding alone cannot account for the effects of directed forgetting, and they proposed that items followed by the F instruction become inhibited and less accessible and, therefore, more difficult to retrieve.

In our study, inhibition triggered by an intention to forget emotionally negative IAPS images resulted in strong activation of a distributed neural network, extending from anterior to posterior brain regions. The involvement of frontal cortex in this network seems to be apparent. The frontal cortex hosts a rich variety of cognitive and affective functions and therefore constitutes an area in which attention, memory, and different emotional processes interact (Carretié et al. 2009). More specifically, prefrontal regions mediate inhibitory memory control as indicated by neuropsychological research showing that inhibitory deficits in memory are associated with dysfunction of prefrontal circuits (Shimamura 1994). Patients with frontal lobe injuries (especially in the right hemisphere) suffer impairments in intentionally initiating inhibitory memory processes (Conway and Fthenaki 2003). Regions within BA 6, associated with intention to forget in our study, receive multimodal inputs (Picard and Strick 2001) and are activated by visual selective attention (Kastner and Ungerleider 2000) and by cognitive tasks that demand updating in memory (Picard and Strick 2001). Moreover, Li et al. (2006) reported that more efficient response inhibition in a stop-signal task is associated with activations including BA 6 region. On the other hand, some studies (e.g., Mecklinger et al. 2009) reported a close resemblance between inhibitory control of unwanted memories and inhibitory control of prepotent motor responses (they were significantly correlated). Interestingly, in our study, frontal activations were present only for negative IAPS images and absent for neutral IAPS images. The lack of frontal activations observed for neutral stimuli may be a consequence of similar involvement of frontal regions in F and R trials, indicating, for instance, that monitoring functions may be active to the same extent in both types of trials.

One may ask, however, why structures other than prefrontal cortex were associated with forgetting. Inhibition, in general, is postulated to be a mechanism by which the prefrontal cortex exerts its effects on subcortical and posterior cortical regions to implement executive control (Aron et al. 2004). Thus, one of plausible explanations of posterior region activations refers to the top–down control (Bar et al. 2006). In the directed forgetting paradigm, the F instruction may initiate inhibitory processes in the frontal regions that may exert some influence on posterior regions of the brain, linked to visual processing, in a top–down manner. Another reason why posterior activations were associated with presentation of F instruction is the plausible process of visualization/imagination of TBF items. In the F trials of our experiment, the F instruction might initiate not only the process of active stopping/interrupting of encoding but also process of visualization/imagination of TBF images—when receiving F instruction, subjects may recollect/visualize an item presented in that F trial in order to ensure which stimulus is supposed TBF. Both processes, that is, process of visualization/imagination and process of active stopping of image encoding, may be associated with activations in posterior regions that were associated with intentional forgetting in our study: the lingual and parahippocampal gyri that were shown to be involved in novel picture encoding (Rombouts et al. 1999) and the precuneus, a region involved in internal imagery (Knauff et al. 2003; for review, see Cavanna and Trimble 2006).

Differences between forgetting of neutral and negative images may be related to differences in attentional mechanisms involved in processing of these 2 types of stimuli. Directed forgetting has also been described as resulting from attentional inhibition of information during encoding (Zacks et al. 1996). Zacks et al. (1996) argued that attentional mechanisms are engaged to expunge TBF words from working memory and to prevent their reactivation. In other words, when the process triggered by F instruction wins the race against the intention to remember, it may do so because it activates attentional control mechanisms. Generally, attentional inhibition is defined as the suppression of irrelevant information so that it will not enter to the working memory (Hasher and Zacks 1988). This hypothesis emphasizes the active inhibition of TBF items. Such an attentional inhibition may be less efficient and more difficult for negative stimuli due to attentional bias in processing of emotional information. A growing body of evidence documents that attention is mainly directed toward emotional—especially negative—stimulation (Hansen CH and Hansen RD 1988; Pratto and John 1991; Lang et al. 1997; Mogg and Bradley 1998; Fox et al. 2000; Carretié et al. 2001). This effect is called the negativity bias (for review, see Carretié et al. 2009). Emotional stimuli capture attention with the ease and are effectively processed even when attention is limited (Fox et al. 2001, Öhman et al. 2001). Negative stimuli appear to preferentially make use of the magnocellular pathway to rapidly reach subcortical and cortical processing areas (Vuilleumier et al. 2003; Pourtois et al. 2005; Vuilleumier and Driver 2007). Insula, which is among these areas (Gallese et al. 2004), receives inputs from the thalamus (Critchley 2005) and also the visual cortex (Gallese et al. 2004). It also sends back projections to the visual cortex (Rodman and Consuelos 1994), which is probably the reason why several studies have found a greater response to negative than to nonnegative stimuli in posterior brain regions (e.g., Carretié et al. 2001, 2008; Mourão-Miranda et al. 2003; Pourtois et al. 2005). Activations of the visual cortex appear to involve mainly regions such as the posterior middle temporal gyrus and parietal visual areas (Lang et al. 1998; Carretié et al. 2001, 2008). In addition, increasing emphasis has been recently placed on interactions of emotion with elaborative, attentional, and sensory processes supported by regions of the prefrontal cortex, parietal cortex, and fusiform/parahippocampal gyri (e.g., Dolcos et al. 2004; Kensinger et al. 2007; Talmi et al. 2008; for a review, see Dolcos and Denkova 2008). Interestingly, all these regions were associated with intention to forget negative stimuli in our study.

Mechanisms involved in directed forgetting effects for stimuli more complex than words, like colored pictures, are a matter of debate (Hauswald and Kissler 2008; Hourihan et al. 2009; Quinlan et al. 2010). Specifically, a question arises whether selective rehearsal is possible for nonverbal information. Recently, Quinlan et al. (2010) compared item-method directed forgetting of pictures and words that were the verbal label (or name) of each picture. In their experiment, subjects studied either pictures or words and were tested with either pictures or words, resulting in 4 conditions. When pictures were presented at study, a directed forgetting effect was evident at test (regardless of whether words or pictures were presented at test). The magnitude of the directed forgetting effect was reduced for studied pictures, relative to studied words, but the effect was present in all conditions. However, as Quinlan et al. (2010) pointed out in their discussion, the pictures that they used were highly nameable (indeed, the pictures were highly nameable by design, to permit testing of studied pictures using words, and vice versa). Thus, in their study, rehearsal referred rather to labels than to pictures per se. Hauswald and Kissler (2008) found a small-magnitude directed forgetting effect for more complex nonverbal stimuli (colored scenic photographs) that—in their opinion—cannot be reduced to a one-word verbalization.

Hourihan et al. (2009), in turn, investigated in their item-method directed forgetting study whether selective rehearsal was possible for difficult-to-name abstract symbols. The symbols were easily distinguished from each other but did not appear to have an obvious name or label. Hourihan et al. (2009) found a significant directed forgetting effect for unnamed symbols, indicating that nonverbal rehearsal can be used selectively to enhance memory for TBR pictorial stimuli. As far as our stimuli are concerned, it is rather hard to create a one-word “label” that would fully describe complex images, especially emotionally negative. Moreover, even if such a label could be created, it would fit to more than one image—in IAPS, there are many images that depict similar objects or scenes. Specifically, in our study—as mentioned in the Materials and methods—special attention was devoted to counterbalance the content of images between TBF, TBR, and “new” set of images (for instance, if there was an image of a snake selected as TBF, 2 other images of a snake would be selected to become members of TBR and new sets of images). Interestingly, this “adjusting” the content of images may be one of the reasons why strong activations of right fusiform gyrus were associated with successful intentional forgetting in our study. Kensinger et al. (2007) showed that the right fusiform gyrus is involved in processing of visual details, examining the encoding processes that led a person to remember the exact visual details of negative and neutral images. In their study, pairs of images were selected so that the 2 items (“same” and “similar”) of a pair shared the same verbal label (e.g., were both umbrellas) but differed in other perceptual features (e.g., color, shape, size, and orientation). In the recognition test, same, similar, and new images were presented. Memory for the visual details of negative items was accentuated because of enhanced visual processing of those stimuli during encoding: the right fusiform gyrus showed enhanced activity, both in extent and in magnitude, during the encoding of negative items (Kensinger et al. 2007). This is in line with previous findings suggesting that the right fusiform gyrus is important for the processing of the exemplar-specific visual details of an object (Marsolek 1999; Koutstaal et al. 2001; Simons et al. 2003; Kensinger et al. 2006). We hypothesized that processing of visual details of presented images was also critical not only for subsequent remembering but also for forgetting: to efficiently disregard an object, one should precisely know which object is supposed TBF.

Recently, Depue et al. (2006) examined cognitive control of memory for verbal and nonverbal stimuli that were either neutral or emotionally negative, utilizing a think/no-think paradigm for face–word or face–IAPS picture pairs. Results for both words and IAPS pictures showed that the inhibitory influences were larger for negative than neutral items. However, in that study, as in other think/no-think studies, inhibitory control refers to retrieval suppression of deeply encoded stimuli, whereas in an item-method directed forgetting paradigm, mechanisms leading to forgetting operate mainly at encoding, interrupting/stopping this process. This may be a reason why in our study less negative IAPS images were forgotten in comparison to neutral one, that is, effect just opposite to that reported by Depue et al. (2006).

Previous fMRI study by Wylie et al. (2008) that investigated brain correlates of forgetting as revealed by F versus R comparison showed unique patterns of activations associated with intentional forgetting. That finding clearly indicated that intentional forgetting may be viewed as an active process, reflecting an effort required for preventing TBF items from being encoded into the long-term memory. Whereas some activations were common in the study of Wylie et al. (2008) and our study (middle frontal gyrus, middle temporal gyrus, parahippocampal gyrus, and inferior parietal lobule), other activations differed probably due to some differences in the experimental design. The main difference between the 2 studies is the type of stimuli used, that is, words versus images. Moreover, in our study, the memory instruction was given explicitly by presenting either the word REMEMBER or FORGET, whereas in the study of Wylie et al. (2008), the memory instruction was provided as a string of 5 uppercase Xs that were colored either blue or yellow to indicate whether subjects were supposed to remember or forget the word from a given trial.

Interestingly, studies that used experimental paradigms that required inhibition at retrieval reported activations overlapping—in some cases—with activations associated with encoding inhibition in our study. Using the think/no-think paradigm, Anderson et al. (2004) reported that an attempt to suppress unwanted memories resulted in increased activation in many frontal (BA 45, BA 46, BA 6, and BA 9) and parietal (BA 7) regions. A part of this active inhibitory network (BA 6 and BA 7) was also active in our F > R condition. Another think/no-think study with pairs of faces (that served as cues) and emotionally negative IAPS pictures (as targets) showed that memory suppression of negative information is controlled by prefrontal right-sided regions: for the no-think versus think contrast, increased activity was observed in BA 8, BA 9/46, BA 47, and BA 10 (Depue et al. 2007). The latter region was activated by presentation of F instruction that resulted in successful intentional forgetting in our study. Altogether, some similarities of findings obtained in studies that utilize different experimental procedures, based on different mechanisms, but leading to the memory impairment (i.e., forgetting) seem to elicit inhibitory processes reflected in activation of shared neural network.

In conclusion, the findings of this item-method directed forgetting fMRI study reveal that forgetting of emotional information is supported by a widely distributed neural network, indicating more effort than forgetting of neutral information. These differences were observed in the study phase but not the test phase, which suggests that the directed forgetting effect is mainly based on inhibition at the encoding level rather than at retrieval (but see: Ullsperger et al. 2000; Nowicka, Jednoróg, Wypych, and Marchewka 2009). More generally, our results suggest that flexible control of memory may be effective even in case of unpleasant memories, but still it requires more effort than in case of neutral ones. One should realize, however, that forgetting effects are not a robust phenomenon and may depend on specific task situations and experimental manipulations (Hauswald and Kissler 2008; Quinlan et al. 2010; for discussion, see Bulevich et al. 2006).

Funding

Polish Ministry of Science and Higher Education (0717/H03/2007/32); the Center for Advanced Imaging in Germany (BMBF 01GO0505).

Conflicts of Interest: None declared.

References

Anderson
MC
Rethinking interference theory: executive control and the mechanisms of forgetting
J Mem Lang
 , 
2003
, vol. 
49
 (pg. 
415
-
445
)
Anderson
MC
Green
C
Suppressing unwanted memories by executive control
Nature
 , 
2001
, vol. 
410
 (pg. 
366
-
369
)
Anderson
MC
Levy
BJ
Suppressing unwanted memories
Curr Dir Psychol Sci
 , 
2009
, vol. 
18
 (pg. 
184
-
194
)
Anderson
MC
Neely
JH
Bjork
EL
Bjork
RA
Interference and inhibition in memory retrieval
Memory handbook of perception and cognition
 , 
1996
San Diego (CA)
Academia Press
(pg. 
237
-
313
)
Anderson
MC
Ochsner
KN
Kuhl
B
Robertson
E
Gabrieli
SW
Glover
GH
Gabrieli
JDE
Neural systems underlying the suppression of unwanted memories
Science
 , 
2004
, vol. 
303
 (pg. 
232
-
235
)
Aron
RA
Robbins
TW
Poldrack
RA
Inhibition and the right inferior frontal cortex
Trends Cogn Sci
 , 
2004
, vol. 
8
 (pg. 
170
-
177
)
Bar
M
Kassam
KS
Ghuman
AS
Boshyan
J
Schmid
AM
Dale
AM
Hämäläinen
MS
Marinkovic
K
Schacter
DL
Rosen
BR
, et al.  . 
Top-down facilitation of visual recognition
Proc Natl Acad Sci USA
 , 
2006
, vol. 
103
 (pg. 
449
-
454
)
Barnier
AJ
Conway
MA
Mayoh
L
Speyer
J
Avizmil
O
Harris
CB
Directed forgetting of recently recalled autobiographical memories
J Exp Psychol Gen
 , 
2007
, vol. 
136
 (pg. 
301
-
322
)
Basden
B
Basden
D
Gargano
G
Directed forgetting in implicit and explicit memory tests: a comparison of methods
J Exp Psychol Learn Mem Cogn
 , 
1993
, vol. 
19
 (pg. 
603
-
616
)
Basden
BH
Basden
DR
Directed forgetting: further comparisons of the item and list methods
Memory
 , 
1996
, vol. 
4
 (pg. 
633
-
653
)
Basden
BH
Basden
DR
Golding
JM
MacLeod
CM
Directed forgetting: a contrast of methods and interpretations
Intentional forgetting: interdisciplinary approaches
 , 
1998
Mahwah (NJ)
Erlbaum
(pg. 
139
-
172
)
Basden
BH
Basden
DR
Wright
M
Part-list re-exposure and release of retrieval inhibition. in directed forgetting
Conscious Cogn
 , 
2003
, vol. 
12
 (pg. 
354
-
375
)
Bäuml
KH
Hanslmayr
S
Pastötter
B
Klimesch
W
Oscillatory correlates of intentional updating in episodic memory
Neuroimage
 , 
2008
, vol. 
41
 (pg. 
596
-
604
)
Bergström
ZM
Velmans
M
de Fockert
J
Richardson-Klavehn
A
ERP evidence for successful voluntary avoidance of conscious recollection
Brain Res
 , 
2007
, vol. 
1151
 (pg. 
119
-
133
)
Bjork
RA
Melton
AW
Martin
E
Theoretical implications of directed forgetting
Coding processes in human memory
 , 
1972
Washington (DC)
Winston
(pg. 
217
-
235
)
Bjork
RA
Roediger
HL
3rd
Craik
FIM
Retrieval inhibition as an adaptive mechanism in human memory
Varieties of memory and consciousness: essays in honour of Endel Tulving
 , 
1989
Hillsdale (NJ)
Erlbaum
(pg. 
309
-
330
)
Bjork
EL
Bjork
RA
Anderson
MC
Golding
JM
MacLeod
CM
Varieties of goal-directed forgetting
Intentional forgetting: interdisciplinary approaches
 , 
1998
Hillsdale (NJ)
Erlbaum
(pg. 
103
-
117
)
Bradley
MM
Greenwald
MK
Petry
MC
Lang
PJ
Remembering pictures: pleasure and arousal in memory
J Exp Psychol Learn Mem Cogn
 , 
1992
, vol. 
18
 (pg. 
379
-
390
)
Britton
JC
Taylor
SF
Sudheimer
KD
Liberzon
I
Facial expressions and complex IAPS pictures: common and differential networks
Neuroimage
 , 
2006
, vol. 
31
 (pg. 
906
-
919
)
Bulevich
JB
Roediger
HL3rd
Balota
DA
Butler
AC
Failures to find suppression of episodic memories in the think/no-think paradigm
Mem Cogn
 , 
2006
, vol. 
34
 (pg. 
1569
-
1577
)
Carretié
L
Albert
J
López-Martín
S
Tapia
M
Negative brain: an integrative review on the neural processes activated by unpleasant stimuli
Int J Psychophysiol
 , 
2009
, vol. 
71
 (pg. 
57
-
63
)
Carretié
L
Hinojosa
J
Albert
J
López-Martín
S
De La Gándara
BS
Igoa
JM
Sotillo
M
Modulation of ongoing cognitive processes by emotionally intense words
Psychophysiology
 , 
2008
, vol. 
45
 (pg. 
188
-
196
)
Carretié
L
Martin-Loeches
M
Hinojosa
JA
Mercado
F
Emotion and attention interaction studied through event-related potentials
J Cogn Neurosci
 , 
2001
, vol. 
13
 (pg. 
1109
-
1128
)
Cavanna
AE
Trimble
MR
The precuneus: a review of its functional anatomy and behavioural correlates
Brain
 , 
2006
, vol. 
129
 (pg. 
564
-
583
)
Christianson
SA
The handbook of emotion and memory: research and theory
 , 
1992
Hillsdale (NJ)
Lawrence Erlbaum Associates
Conway
MA
Fthenaki
A
Disruption of inhibitory control of memory following lesions to the frontal and temporal lobes
Cortex
 , 
2003
, vol. 
39
 (pg. 
667
-
686
)
Conway
MA
Harries
K
Noyes
J
Racsma'ny
M
Frankish
CR
The disruption and dissolution of directed forgetting: inhibitory control of memory
J Mem Lang
 , 
2000
, vol. 
43
 (pg. 
409
-
430
)
Cottencin
O
Gruat
G
Thomas
P
Devos
P
Goudemand
M
Consoli
SM
Directed forgetting in depression
J Int Neuropsychol Soc
 , 
2008
, vol. 
14
 (pg. 
895
-
899
)
Crinion
J
Ashburner
J
Leff
A
Brett
M
Price
C
Friston
K
Spatial normalization of lesioned brains: performance evaluation and impact on fMRI analyses
Neuroimage
 , 
2007
, vol. 
37
 (pg. 
866
-
875
)
Critchley
HD
Neural mechanisms of autonomic, affective and cognitive integration
J Comp Neurol
 , 
2005
, vol. 
493
 (pg. 
154
-
166
)
Danion
J
Kaufmann-Muller
F
Grange
D
Zimmermann
M
Greth
P
Affective valence of words, explicit and implicit memory in clinical depression
J Affect Disord
 , 
1995
, vol. 
34
 (pg. 
227
-
234
)
DePrince
AP
Freyd
JJ
Forgetting trauma stimuli
Psychol Sci
 , 
2004
, vol. 
15
 (pg. 
488
-
492
)
Depue
BE
Banich
MT
Curran
T
Suppression of emotional and nonemotional content in memory: effects of repetition on cognitive control
Psychol Sci
 , 
2006
, vol. 
17
 (pg. 
441
-
447
)
Depue
BE
Curran
T
Banich
MT
Prefrontal regions orchestrate suppression of emotional memories via a two-phase process
Science
 , 
2007
, vol. 
37
 (pg. 
215
-
219
)
Devilly
GJ
Ciorciari
J
Piesse
A
Sherwell
S
Zammit
S
Cook
F
Turton
C
Dissociative tendencies and memory performance on directed-forgetting tasks
Psychol Sci
 , 
2007
, vol. 
18
 (pg. 
212
-
217
)
Dolcos
F
Denkova
E
Neural correlates of encoding emotional memories: a review of functional neuroimaging evidence
Cell Sci Rev
 , 
2008
, vol. 
5
 (pg. 
78
-
122
)
Dolcos
F
LaBar
KS
Cabeza
R
Dissociable effects of arousal and valence on prefrontal activity indexing emotional evaluation and subsequent memory: an event-related fMRI study
Neuroimage
 , 
2004
, vol. 
23
 (pg. 
64
-
74
)
Dumont
M
Directed forgetting and memory bias for emotion-congruent information in clinical depression
Curr Psychol Cogn
 , 
2000
, vol. 
19
 (pg. 
171
-
188
)
Elzinga
BM
de Beurs
E
Sergeant
JA
van Dyck
R
Phaf
RH
Dissociative style and directed forgetting
Cognit Ther Res
 , 
2000
, vol. 
24
 (pg. 
279
-
295
)
Epstein
W
Bower
GH
Mechanisms of directed forgetting
The psychology of learning and motivation
 , 
1972
, vol. 
Vol. 6
 
New York
Academic Press
(pg. 
147
-
191
)
Erickson
KI
Colcombe
SJ
Wadhwa
R
Bherer
L
Peterson
MS
Scalf
PE
Kim
JS
Alvarado
M
Kramer
AF
Training-induced functional activation changes in dual-task processing: an fMRI study
Cereb Cortex
 , 
2007
, vol. 
17
 (pg. 
192
-
204
)
Fawcett
JM
Taylor
TL
Forgetting is effortful: evidence from reaction time probes in an item-method directed forgetting task
Mem Cogn
 , 
2008
, vol. 
36
 (pg. 
1168
-
1181
)
Fox
E
Lester
V
Russo
R
Bowles
RJ
Pichler
A
Dutton
K
Facial expressions of emotions: are angry faces detected more efficiently?
Cogn Emot
 , 
2000
, vol. 
14
 pg. 
61092
 
Fox
E
Russo
R
Bowles
R
Dotton
K
Do threatening stimuli draw or hold attention in visual attention in subclinical anxiety
J Exp Psychol Gen
 , 
2001
, vol. 
130
 (pg. 
466
-
478
)
Friston
K
Frackowiak
RSJ
Friston
KJ
Frith
C
Dolan
R
Friston
KJ
Price
CJ
Zeki
S
Ashburner
J
Penny
WD
Introduction: experimental design and statistical parametric mapping
Human brain function
 , 
2003
2nd ed
Orlando (FL)
Academic Press
Gaig
SB
Bowler
DM
Free recall and forgetting of emotionally arousing words in autism spectrum disorder
Neuropsychologia
 , 
2008
, vol. 
46
 (pg. 
2336
-
2343
)
Gallese
V
Keysers
C
Rizzolatti
G
A unifying view of the basis of social cognition
Trends Cogn Sci
 , 
2004
, vol. 
8
 (pg. 
396
-
403
)
Geiselman
RE
Bjork
RA
Fishman
DL
Disrupted retrieval in directed forgetting: a link with posthypnotic amnesia
J Exp Psychol Gen
 , 
1983
, vol. 
112
 (pg. 
58
-
72
)
Gould
RL
Brown
RG
Owen
AM
Ffytche
DH
Howard
RJ
fMRI BOLD response to increasing task difficulty during successful paired associates learning
Neuroimage
 , 
2003
, vol. 
20
 (pg. 
1006
-
1019
)
Hansen
CH
Hansen
RD
Finding the face in the crowd: an anger superiority effect
J Pers Soc Psychol
 , 
1988
, vol. 
54
 (pg. 
917
-
924
)
Hasher
L
Zacks
RT
Bower
GH
Working memory, comprehension, and aging: a review and a new view
The psychology of learning and motivation
 , 
1988
, vol. 
Vol. 22
 
New York
Academic Press
(pg. 
193
-
225
)
Hauswald
A
Kissler
J
Directed forgetting of complex pictures in an item method paradigm
Memory
 , 
2008
, vol. 
16
 (pg. 
797
-
809
)
Hourihan
KL
Ozubko
JD
MacLeod
CM
Directed forgetting of visual symbols: evidence for nonverbal selective rehearsal
Mem Cognit
 , 
2009
, vol. 
37
 (pg. 
1059
-
1068
)
Hsieh
LT
Hung
DL
Tzeng
OJL
Lee
JR
Cheng
SK
An event-related potential investigation of the processing of remember/forget cues and item encoding in item-method directed forgetting
Brain Res
 , 
2009
, vol. 
1250
 (pg. 
190
-
201
)
Johnson
HM
Processes of successful intentional forgetting
Psychol Bull
 , 
1994
, vol. 
116
 (pg. 
274
-
292
)
Joormann
J
Hertel
PT
Brozovich
F
Gotlib
IH
Remembering the good, forgetting the bad: intentional forgetting of emotional material in depression
J Abnorm Psychol
 , 
2005
, vol. 
114
 (pg. 
640
-
648
)
Joslyn
SL
Oakes
MA
Directed forgetting of autobiographical events
Mem Cognit
 , 
2005
, vol. 
33
 (pg. 
577
-
587
)
Kastner
S
Ungerleider
LG
Mechanisms of visual attention in the human cortex
Annu Rev Neurosci
 , 
2000
, vol. 
23
 (pg. 
315
-
341
)
Kelly
AMC
Garavan
H
Human functional neuroimaging of brain changes associated with practice
Cereb Cortex
 , 
2005
, vol. 
15
 (pg. 
1089
-
1102
)
Kensinger
EA
Garoff-Eaton
RJ
Schacter
DL
Memory for specific visual details can be enhanced by negative arousing content
J Mem Lang
 , 
2006
, vol. 
54
 (pg. 
99
-
112
)
Kensinger
EA
Garoff-Eaton
RJ
Schacter
DL
How negative emotion enhances the visual specificity of a memory
J Cogn Neurosci
 , 
2007
, vol. 
19
 (pg. 
1872
-
1887
)
Knauff
M
Fangmeier
T
Ruff
CC
Johnson-Laird
PN
Reasoning, models, and images: behavioral measures and cortical activity
J Cogn Neurosci
 , 
2003
, vol. 
15
 (pg. 
559
-
573
)
Korfine
L
Hooley
JM
Directed forgetting of emotional stimuli in borderline personality disorder
J Abnorm Psychol
 , 
2000
, vol. 
109
 (pg. 
214
-
221
)
Koutstaal
W
Wagner
AD
Rotte
M
Maril
A
Buckner
RL
Schacter
DL
Perceptual specificity in visual object priming: fMRI evidence for a laterality difference in fusiform cortex
Neuropsychologia
 , 
2001
, vol. 
39
 (pg. 
184
-
199
)
Lancaster
JL
Tordesillas-Gutiérrez
D
Martinez
M
Salinas
F
Evans
A
Zilles
K
Mazziotta
JC
Fox
PT
Bias between MNI and Talairach coordinates analyzed using the ICBM-152 brain template
Hum Brain Mapp
 , 
2007
, vol. 
28
 (pg. 
1194
-
1205
)
Lancaster
JL
Woldorff
MG
Parsons
LM
Liotti
M
Freitas
CS
Rainey
L
Kochunov
PV
Nickerson
D
Mikiten
SA
Fox
PT
Automated Talairach atlas labels for functional brain mapping
Hum Brain Mapp
 , 
2000
, vol. 
10
 (pg. 
120
-
131
)
Landau
SM
Schumacher
EH
Garavan
H
Druzgal
TJ
D'Esposito
M
A functional MRI study of the influence of practice on component processes of working memory
Neuroimage
 , 
2004
, vol. 
22
 (pg. 
211
-
221
)
Lang
PJ
Bradley
MM
Cuthbert
BN
Lang
P
Simons
RF
Balaban
MT
Motivated attention: affect, activation, and action
Attention and orienting: sensory and motivational processes
 , 
1997
Mahwah (NJ)
Lawrence Erlbaum Associates
(pg. 
97
-
135
)
Lang
PJ
Bradley
MM
Cuthbert
BN
International affective picture system (IAPS): affective ratings of pictures and instruction manual. Technical Report A-8
 , 
2001
Gainesville (FL)
University of Florida
Lang
PJ
Bradley
MM
Fitzsimmons
JR
Cuthbert
BN
Scott
JD
Moulder
B
Nangia
V
Emotional arousal and activation of the visual cortex: an fMRI analysis
Psychophysiology
 , 
1998
, vol. 
35
 (pg. 
199
-
210
)
Levy
BJ
Anderson
MC
Individual differences in the suppression of unwanted memories: the executive deficit hypothesis
Acta Psychol
 , 
2008
, vol. 
127
 (pg. 
623
-
635
)
Li
CR
Huang
C
Constable
RT
Sinha
R
Imaging response inhibition in a stop-signal task: neural correlates independent of signal monitoring and post-response processing
J Neurosci
 , 
2006
, vol. 
26
 (pg. 
186
-
192
)
MacLeod
CM
Long-term recognition and recall following directed forgetting
J Exp Psychol Hum
 , 
1975
, vol. 
104
 (pg. 
271
-
279
)
MacLeod
CM
Directed forgetting affects both direct and indirect tests of memory
J Exp Psychol Learn Mem Cogn
 , 
1989
, vol. 
15
 (pg. 
13
-
21
)
MacLeod
CM
Golding
M
MacLeod
CM
Directed forgetting
Intentional forgetting: interdisciplinary approaches
 , 
1998
Mahwah (NJ)
Erlbaum
(pg. 
1
-
57
)
MacLeod
CM
The item and list methods of directed forgetting: test differences and the role of demand characteristics
Psychon Bull Rev
 , 
1999
, vol. 
6
 (pg. 
123
-
129
)
Marsolek
CJ
Dissociable neural subsystems underlie abstract and specific object recognition
Psychol Sci
 , 
1999
, vol. 
10
 (pg. 
111
-
118
)
McNally
RJ
Metzger
LJ
Lasko
NB
Clancy
SA
Pitman
RK
Directed forgetting of trauma cues in adult survivors of childhood sexual abuse with and without posttraumatic stress disorder
J Abnorm Psychol
 , 
1998
, vol. 
107
 (pg. 
596
-
601
)
McNally
RJ
Otto
MW
Yap
L
Pollack
MH
Hornig
CD
Is panic disorder linked to cognitive avoidance of threatening information?
J Anxiety Disord
 , 
1999
, vol. 
13
 (pg. 
335
-
348
)
Mecklinger
A
Parra
M
Waldhauser
GT
ERP correlates of intentional forgetting
Brain Res
 , 
2009
, vol. 
1255
 (pg. 
132
-
147
)
Mogg
K
Bradley
BP
A cognitive-motivational analysis of anxiety
Behav Res Ther
 , 
1998
, vol. 
36
 (pg. 
809
-
848
)
Moulds
ML
Bryant
RA
Directed forgetting in acute stress disorder
J Abnorm Psychol
 , 
2002
, vol. 
111
 (pg. 
175
-
179
)
Moulds
ML
Bryant
RA
An investigation of retrieval inhibition in acute stress disorder
J Trauma Stress
 , 
2005
, vol. 
18
 (pg. 
233
-
236
)
Mourão-Miranda
J
Volchan
E
Moll
J
de Oliveira-Souza
R
Oliveira
L
Bramati
I
Gattass
R
Pessoa
L
Contributions of stimulus valence and arousal to visual activation during emotional perception
Neuroimage
 , 
2003
, vol. 
20
 (pg. 
1955
-
1963
)
Myers
LB
Derakshan
N
To forget or not to forget: what do repressors forget and when do they forget?
Cogn Emot
 , 
2004
, vol. 
18
 (pg. 
495
-
511
)
Neath
I
Surprenant
AM
Human memory
 , 
2003
2nd ed
Belmont (CA)
Thompson Wadsworth Publishing
Nowicka
A
Jednoróg
K
Marchewka
A
Brechmann
A
Successfully overcoming the inhibitory impact of the “forget” instruction: a voxel-based morphometric study of directed forgetting
Psychophysiology
 , 
2009
, vol. 
46
 (pg. 
1008
-
1012
)
Nowicka
A
Jednoróg
K
Wypych
M
Marchewka
A
Reversed old/new effect for intentionally forgotten words: an ERP study of directed forgetting
Int J Psychophysiol
 , 
2009
, vol. 
71
 (pg. 
97
-
102
)
Ochsner
KN
Are affective events richly recollected or simply familiar? The experience and process of recognizing feelings past
J Exp Psychol Gen
 , 
2000
, vol. 
129
 (pg. 
242
-
326
)
Öhman
A
Flykt
A
Esteves
F
Emotion drives attention: detecting the snake in the grass
J Exp Psychol Gen
 , 
2001
, vol. 
130
 (pg. 
466
-
478
)
Palomba
D
Angrilli
A
Mini
A
Visual evoked potentials, heart rate responses and memory to emotional pictorial stimuli
Int J Psychophysiol
 , 
1997
, vol. 
27
 (pg. 
55
-
67
)
Pastötter
B
Bäuml
K-H
Amount of postcue encoding predicts amount of directed forgetting
J Exp Psychol Learn Mem
 , 
2010
, vol. 
36
 (pg. 
54
-
65
)
Payne
BK
Corrigan
E
Emotional constraints on intentional forgetting
J Exp Soc Psychol
 , 
2007
, vol. 
43
 (pg. 
780
-
786
)
Paz-Caballero
MD
Menor
J
Jiménez
JM
Predictive validity of event-related potentials (ERP) in relation to directed forgetting effects
Clin Neurophysiol
 , 
2004
, vol. 
115
 (pg. 
369
-
377
)
Phelps
EA
LaBar
KS
Spencer
DD
Memory for emotional words following unilateral temporal lobectomy
Brain Cogn
 , 
1997
, vol. 
35
 (pg. 
85
-
109
)
Picard
N
Strick
PL
Imaging the premotor areas
Curr Opin Neurobiol
 , 
2001
, vol. 
11
 (pg. 
663
-
672
)
Pourtois
G
Dan
E
Grandjean
D
Sander
D
Vuilleumier
P
Enhanced extrastriate visual response to band-pass spatial frequency filtered fearful faces: time course and topographic evoked-potentials
Hum Brain Mapp
 , 
2005
, vol. 
26
 (pg. 
65
-
79
)
Pratto
F
John
OP
Automatic vigilance: the attention-grabbing power of negative social information
J Pers Soc Psychol
 , 
1991
, vol. 
61
 (pg. 
380
-
391
)
Quinlan
CK
Taylor
TL
Fawcett
JM
Directed forgetting: comparing pictures and words
Can J Exp Psychol
 , 
2010
, vol. 
64
 (pg. 
41
-
46
)
Reber
PJ
Siwiec
RM
Gitelman
DR
Parrish
TB
Mesulam
MM
Paller
KA
Neural correlates of successful encoding identified using functional magnetic resonance imaging
J Neurosci
 , 
2002
, vol. 
22
 (pg. 
9541
-
9548
)
Reisberg
D
Heuer
F
Winograd
E
Neiser
U
Remembering the details of emotional events
Affect and accuracy in recall: studies of ‘flash-bulb’ memory
 , 
1992
Cambridge (UK)
Cambridge University Press
(pg. 
162
-
190
)
Rodman
HR
Consuelos
MJ
Cortical projections to anterior inferior temporal cortex in infant macaque monkeys
Vis Neurosci
 , 
1994
, vol. 
11
 (pg. 
119
-
133
)
Rombouts
SARB
Scheltens
P
Machielsen
WCM
Barkhof
F
Hoogenraad
FGC
Veltman
DJ
Valk
J
Witter
MP
Parametric fMRI analysis of visual encoding in the human medial temporal lobe
Hippocampus
 , 
1999
, vol. 
9
 (pg. 
637
-
643
)
Rubin
RC
Friendly
M
Predicting which words get recalled: measures of free recall, availability, goodness, emotionality, and pronounciability for 925 nouns
Mem Cogn
 , 
1986
, vol. 
14
 (pg. 
79
-
94
)
Sahakyan
L
Delaney
PF
Can encoding differences explain the benefits of directed forgetting in the list method paradigm?
J Mem Lang
 , 
2003
, vol. 
48
 (pg. 
195
-
206
)
Sahakyan
L
Kelley
CM
A contextual change account of the directed forgetting effect
J Exp Psychol Learn Mem Cogn
 , 
2002
, vol. 
28
 (pg. 
1064
-
1072
)
Sheard
ED
MacLeod
CM
Ohta
N
MacLeod
CM
Uttl
B
List method directed forgetting: return of the selective rehearsal account
Dynamic cognitive processes
 , 
2005
Tokyo (Japan)
Springer
(pg. 
219
-
248
)
Shimamura
AP
Gazzaniga
MS
Memory and frontal lobe function
The cognitive neurosciences
 , 
1994
Cambridge (MA)
MIT Press
(pg. 
803
-
815
)
Simons
JS
Koutstaal
W
Prince
S
Wagner
AD
Schacter
DL
Neural mechanisms of visual object priming: evidence for perceptual and semantic distinctions in fusiform cortex
Neuroimage
 , 
2003
, vol. 
19
 (pg. 
613
-
626
)
Talmi
D
Anderson
AK
Riggs
L
Caplan
JB
Moscovitch
M
Immediate memory consequences of the effect of emotion on attention to pictures
Learn Mem
 , 
2008
, vol. 
15
 (pg. 
172
-
182
)
Tolin
DF
Hamlin
C
Foa
EB
Directed forgetting in obsessive-compulsive disorder: replication and extension
Behav Res Ther
 , 
2002
, vol. 
40
 (pg. 
793
-
803
)
Ullsperger
M
Mecklinger
A
Müller
U
An electrophysiological test of directed forgetting: the role of retrieval inhibition
J Cogn Neurosci
 , 
2000
, vol. 
12
 (pg. 
924
-
940
)
Van Hooff
JC
Whitaker
TA
Ford
RM
Directed forgetting in direct and indirect tests of memory: Seeking evidence of retrieval inhibition using electrophysiological measures
Brain Cogn
 , 
2009
, vol. 
71
 (pg. 
153
-
164
)
Vuilleumier
P
Armony
JL
Driver
J
Dolan
J
Distinct spatial frequency sensitivities for processing faces and emotional expressions
Nat Neurosci
 , 
2003
, vol. 
6
 (pg. 
624
-
631
)
Vuilleumier
P
Driver
J
Modulation of visual processing by attention and emotion: windows on causal interactions between human brain regions
Philos Trans R Soc Lond B Biol Sci
 , 
2007
, vol. 
362
 (pg. 
837
-
855
)
Wilhelm
S
McNally
RJ
Baer
L
Florin
I
Directed forgetting in obsessive-compulsive disorder
Behav Res Ther
 , 
1996
, vol. 
34
 (pg. 
633
-
641
)
Wylie
GR
Foxe
JJ
Taylor
TL
Forgetting as an active process: an fMRI investigation of item-method-directed forgetting
Cereb Cortex
 , 
2008
, vol. 
18
 (pg. 
670
-
682
)
Zacks
RT
Hasher
L
Dagenbach
D
Carr
TH
Directed ignoring: inhibitory regulation of working memory
Inhibitory mechanisms in attention, memory, and language
 , 
1994
New York
Academic Press
(pg. 
241
-
264
)
Zacks
RT
Radvansky
G
Hasher
L
Studies of directed forgetting in older adults
J Exp Psychol Learn Mem Cogn
 , 
1996
, vol. 
22
 (pg. 
143
-
156
)
Zoellner
LA
Sacks
MB
Foa
EB
Directed forgetting following mood induction in chronic posttraumatic stress disorder patients
J Abnorm Psychol
 , 
2003
, vol. 
112
 (pg. 
508
-
514
)