Abstract

A relatively common form of memory distortion arises when individuals must discriminate items they have seen from those they have imagined (reality monitoring). The present fMRI investigation (at 1.5 T) focused on the processes that relate to memory assignment regardless of accuracy (e.g. that correspond with the belief that an item was presented as a picture, regardless of whether that belief is correct). Prior to the scan, participants (n = 16) viewed concrete nouns and formed mental images of the object named. Half of the names were followed by the object's photo. During the scan, participants saw the object names and indicated whether the corresponding photo had been studied. Activity in visual-processing regions (including the precuneus and fusiform gyrus) corresponded with the attribution of an item to a pictorial presentation. In contrast, activity in regions thought to be important for self-referential processing (including the ventromedial prefrontal cortex and posterior cingulate gyrus) was associated with attribution to a nonpresented source. These neural findings converge with behavioral evidence indicating that individuals use the amount of different types of information retrieved (e.g. perceptual detail, information about cognitive operations) to determine whether an item was imagined or perceived.

Introduction

We often must discriminate items we have imagined from those we have seen (reality monitoring; Johnson and Raye, 1981). The types of information retrieved likely influence the attribution made by an individual (reviewed by Johnson, 1997). For example, visually presented information typically will be retrieved with more perceptual detail than imagined information, while imagined items will be associated with more information about the cognitive processes used to generate them (Johnson and Raye, 1981; Johnson and Hirst, 1993). Thus, failures in reality monitoring are more likely to occur when individuals imagine items vividly, such that items are later remembered with sensory and perceptual detail, or when individuals generate images automatically, reducing the likelihood that cognitive operations are remembered (e.g. Johnson et al., 1977, 1981, 1988; Rabinowitz, 1989; Dobson and Markham, 1993).

Recent neuroimaging evidence has confirmed the suggestion that items studied as photos have greater visual information available at retrieval than items mentally imagined. In an event-related potential study, Gonsalves and Paller (2000) showed participants object names and asked them to form a mental image of the object. For half of those objects, the corresponding photo also was shown. Gonsalves and Paller (2000) then compared activity for items correctly attributed versus misattributed to pictorial presentation. They demonstrated that activity at posterior electrode sites was greater during correct than incorrect memory attribution to pictorial presentation. They hypothesized that the posterior activity reflected visual processing, consistent with the proposal that more visual information had been retrieved for the items studied as photos than for the items that had been imagined.

Converging evidence has come from an event-related fMRI study (Okado and Stark, 2003), using a design similar to Gonsalves and Paller (2000) except that a ‘lie test’ was added between study and test to boost memory attribution errors. The authors found that occipital regions, as well as the posterior parahippocampal gyrus, showed greater activity for the correct attribution of pictures than for the incorrect attribution of mentally imagined items. These results suggested that visuo-spatial information was more likely to be present for items studied as pictures than for items only mentally imagined.

These contrasts have held constant participants' retrieval responses but varied the attributions' accuracies. A stronger test of the hypothesis that the information retrieved will influence memory attribution is to examine the processes in common between correct and incorrect memory attributions (i.e. processes that occur whenever an individual believes a photo was studied). Although this issue was not central for Okado and Stark (2003), they did present an intriguing finding: left lateral parietal and left frontal regions seemed to show patterns of activity that tracked whether or not participants believed that a photo had been shown. They did not, however, report any visual-processing regions that showed this pattern. They also did not examine regions leading to attribution of a memory to a nonpresented source. Thus, little information is available regarding the neural processes underlying attribution of a memory to a presented versus a nonpresented source within a reality monitoring design (for discussion of findings bearing on this issue in studies of true versus false recognition of words and objects, see Schacter and Slotnick, 2004; Slotnick and Schacter, 2004).

The Present Study

The present study examined this issue, using an experimental design similar to Gonsalves and Paller (2000) in which participants had to indicate whether an object had been seen or only mentally imagined. This design allowed us to examine the activity that corresponded with assignment of an item to pictorial presentation versus activity that corresponded with assignment of an item to a nonpresented source, regardless of the accuracy of that assignment. Our hypothesis was that items attributed to pictorial presentation would be associated with more activity in visual regions (e.g. precuneus, fusiform gyrus) than items attributed to a nonpresented source. This finding would be consistent with the proposal (Johnson et al., 1993) that individuals use the amount of visual information retrieved to make decisions about whether an item was visually presented. This finding also would fit well with neuroimaging data acquired during encoding (Gonsalves et al., 2004; Kensinger and Schacter, 2005a), demonstrating that increased activity in visual processing regions (e.g. precuneus and fusiform gyrus) during mental imagery corresponds with an increased tendency to later misattribute the item to visual presentation. We had less specified hypotheses regarding the neural processes that would relate to memory attribution to a nonpresented source; given the hypothesis of Johnson and colleagues, we expected that regions associated with the cognitive operations required for forming a mental image (e.g. visuo-spatial working memory) might be related to attribution to a nonpresented source.

Materials and Methods

Participants

Participants comprised 17 right-handed, native English speaking Harvard undergraduate or graduate students (aged 18–30 years) screened to exclude those with contra-indicators for MRI scanning or with a history of depression. The data from one participant were excluded due to scanner malfunction. No participant was taking centrally active medications. Informed consent was obtained from all participants before both the behavioral and imaging portions of the study in a manner approved by the Harvard University and Massachusetts General Hospital Institutional Review Boards.

Materials and Procedure

Materials comprised 450 concrete words and 450 photo objects depicting a single object on a white background (e.g. a baseball, a tarantula; taken from Hemera Technologies Inc, 2002, Canada). Words and photo objects were selected as pairs, such that each word named a photo object (e.g. ‘umbrella’ and a picture of an umbrella). Half of the words and objects were rated as being negatively emotional objects, and the other half were rated as being neutral objects. The effects of emotional content are reported elsewhere (Kensinger and Schacter, 2005b); here, analyses collapse across emotional content. We did not find reliable effects of emotion on the neural processes corresponding to memory attribution. All of the regions discussed in this paper showed comparable effects of memory attribution for the emotional and the neutral items.

One or two days before the scanning session, each participant viewed a study list with 150 emotional words and 150 neutral words presented for 2 s each. Half of the words were followed by the corresponding photo object, presented for 2 s, and the remaining words were followed by a blank square shown for 2 s (Fig. 1). Participants were instructed to make a button press to indicate whether each word named an object that was bigger or smaller than a shoebox. They were told that the task was interested in mental imagery performance and were encouraged to use mental imagery to perform the size-judgement task. Participants were instructed to simply view the square or the photo object that occurred after the word, and they were told that no response was required to these items.

Figure 1.

Retrieval-related activity was greater for studied items assigned to pictorial presentation (i.e. items given a ‘picture’ response versus a ‘no picture’ response, A) in the posterior cingulate gyrus (BA 23/31, in blue, B), precuneus (BA 7, in green, C) and fusiform gyrus (BA 18, in pink, D). CA, correct attribution; MA, misattribution.

Figure 1.

Retrieval-related activity was greater for studied items assigned to pictorial presentation (i.e. items given a ‘picture’ response versus a ‘no picture’ response, A) in the posterior cingulate gyrus (BA 23/31, in blue, B), precuneus (BA 7, in green, C) and fusiform gyrus (BA 18, in pink, D). CA, correct attribution; MA, misattribution.

After a delay of 1–2 days, participants returned for the fMRI scan. During the functional scans, participants performed a surprise recognition task (Debriefing indicated that no participants realized that their memory would be tested for the items studied in the laboratory). The recognition task was divided across three functional scans. In each scan, participants viewed 150 words. Ffity of the words corresponded to items that had been studied in the word-only condition; 50 to items studied in the word–picture condition; and 50 to items that had not been studied. Words were pseudorandomly intermixed with fixation crosses (+) to provide jitter (Dale, 1999). For each word participants indicated whether or not the corresponding photo object had been presented at study (Table 1). Thus, a ‘no’ response was required both for novel words that had not been studied and for words that had been presented without their corresponding photo object at study.

Table 1

Possible item and response types

Response
 
New Items
 
Word-Only Item
 
Word-Picture Item
 
‘No picture’ Correct rejection Word-only correct attribution Word-picture misattribution 
‘Picture’
 
Baseline false alarm
 
Word-only misattribution
 
Word-picture correct attribution
 
Response
 
New Items
 
Word-Only Item
 
Word-Picture Item
 
‘No picture’ Correct rejection Word-only correct attribution Word-picture misattribution 
‘Picture’
 
Baseline false alarm
 
Word-only misattribution
 
Word-picture correct attribution
 

Image acquisition and data analysis

Images were acquired on a 1.5 T Siemens Sonata MRI scanner. Stimuli were back-projected onto a screen in the scanner bore, and participants viewed the words through an angled mirror attached to the head coil. Detailed anatomic data were acquired using a multiplanar rapidly acquired gradient echo (MP-RAGE) sequence. Functional images were acquired using a T2*-weighted echo planar imaging (EPI) sequence (TR = 3000 ms, TE = 40 ms, FOV = 200 mm; flip angle = 90°). Twenty-one slices (5 mm thickness, 1 mm skip between slices) were acquired in an interleaved fashion.

All preprocessing and data analysis were conducted within SPM99 (Wellcome Department of Cognitive Neurology). Standard preprocessing was performed on the functional data, including slice-timing correction, rigid body motion correction, normalization to the Montreal Neurological Institute template (resampling at 3 mm cubic voxels), and spatial smoothing (using an 8 mm full-width half-maximum isotropic Gaussian kernel).

For each participant, and on a voxel-by-voxel basis, an event-related analysis was first conducted in which all instances of a particular event type were modeled through convolution with a canonical hemodynamic response function. The following events were modeled: new items (collapsing across response), word-only correct attributions, word-only misattributions, word–picture correct attributions, and word–picture misattributions (see Table 1). All participants had at least 10 instances of every event type. Effects for each event type were estimated using a subject-specific, fixed-effects model, resulting in a beta-weight (i.e. amplitude) associated with each event type. The difference in the beta-weights for two event types of interest was then computed, and these data were then entered into a second-order, random-effects group analysis. One-sample t-tests (i.e. contrast analyses) were used to examine the consistency of activity at each voxel, using between-participant variability to estimate variance. Conjunction analyses, using the masking function in SPM99 to select voxels to include or exclude, examined the regions shared between two contrasts. Thus, one contrast of interest was computed, and the activated voxels for this contrast were used to form a mask. A second contrast of interest was then computed, and the mask from the first contrast was applied, such that the resulting conjunction analysis revealed regions activated in both of the contrast analyses. The individual contrasts included in the contrast analysis were analyzed at a threshold of P < 0.01, such that the conjoint probability of the conjunction analysis, estimated using Fisher's method (Fisher, 1950; Lazar et al., 2002) was P < 0.001.

All activations are presented in neurological coordinates (i.e. activity on the right hemisphere is presented on the right side of the brain images). Voxel coordinates are reported in Talairach coordinates (Talairach and Tournoux, 1998) and reflect the most significant voxel within the cluster of activation. Event-related time-courses were extracted from active clusters by creating regions-of-interest (ROI) as 8 mm spheres using the ROI toolbox implemented in SPM99. A Huynh–Feldt correction for non-sphericity was implemented for all ROI analyses.

Results

Behavioral Results

An ANOVA conducted on the ‘picture’ response rates with item history (word–picture, word-only, new) as a within-subject factor revealed a main effect of item type [F(2,14) = 33.43, P < 0.001, partial η2 = 0.83], with more ‘picture’ responses given to word–picture items than to word-only items [t(15) = 7.00, P < 0.001] and more ‘picture’ responses given to word-only items than to new items [t(15) = 7.15, P < 0.001]. Thus, correct attribution rates (mean ± SE = 0.62 ± 0.04) were significantly greater than misattribution rates (0.33 ± 0.04), and misattribution rates for the word-only items remained significantly above the rates of baseline false recognition responses to the new items (0.16 ± 0.04).

Neuroimaging Results

Random-effects analyses contrasted activation as a function of memory performance (comparing correct memory attributions and memory misattributions) separately for each item type (from a word-only or a word–picture trial; Table 2). We could not compare correct and incorrect responses to new items because of the low number of baseline false recognition responses. Because our central aim was to examine the neural processes that were related to reality-monitoring assignment, we were most interested in conjunction analyses that revealed activity shared by items correctly and incorrectly given a particular memory attribution. These conjunction analyses therefore revealed regions that were associated with a particular belief (e.g. that a picture was presented) both when that belief was correct and when it was incorrect.

Table 2

Contrasts for each item type (word-only or word–picture) as a function of memory attribution (‘picture’ or ‘no picture’)

Region
 
Hemisphere
 
BA
 
Talairach coordinates
 
Word–picture correct attributions (‘picture’) > word–picture misattributions (‘no picture’)    
Superior frontal gyrus −32, −6, 61 
Middle frontal gyrus 10/46 −32, 46, −5 
Inferior frontal gyrus 10/11 −41, 52, −12 
  47 −27, 40, −7 
Postcentral gyrus 1/2 −48, −29, 51 
  −35, −43, 60 
Inferior parietal lobule 40 −48, −47, 44 
  7/40 −35, −38, 52 
Anterior hippocampus  −24, −10, −15 
Putamen  21, 9, 11 
Cerebellum Bilateral   
Word–picture misattributions (‘no picture’) > word–picture correct attributions (‘picture’)    
Medial frontal/orbital frontal gyrus 10/11 −6, 45, −20 
 11/32 6, 43, −17 
  10/11 3, 31, −14 
Middle frontal gyrus 24, 28, 40 
Precentral gyrus 38, −18, 48 
   48, −9, 47 
Posterior cingulate gyrus 23/31 12, −60, 22 
Middle temporal gyrus 39 42, −74, 26 
  37/39 50, −62, 25 
Word-only correct attributions (‘no picture’) > word-only misattributions (‘picture’)    
Medial frontal/orbital frontal gyrus bilateral 11 0, 34, −12 
Precentral gyrus 45, −12, 53 
Inferior temporal gyrus 20 −56, −21, −21 
Middle/superior temporal gyrus 21/22 60, −14, −2 
Anterior hippocampus  −29, −10, −22 
Word-only misattributions (‘picture’) > word-only correct attributions (‘no picture’)    
Superior frontal gyrus −18, −5, 66 
Inferior frontal gyrus 10/47 −42, 44, −2 
Precentral gyrus −32, −20, 59 
Anterior cingulate gyrus 32 −3, 8, 44 
 32 3, 20, 40 
  32 6, 28, 29 
Inferior parietal lobe 40 −35, −30, 40 
  7/40 −30, −38, 43 
 7/40 35, −33, 38 
  7/40 42, −38, 54 
Precuneus −17, −52, 50 
Insula  35, 14, 2 
Cerebellum
 
Bilateral
 

 

 
Region
 
Hemisphere
 
BA
 
Talairach coordinates
 
Word–picture correct attributions (‘picture’) > word–picture misattributions (‘no picture’)    
Superior frontal gyrus −32, −6, 61 
Middle frontal gyrus 10/46 −32, 46, −5 
Inferior frontal gyrus 10/11 −41, 52, −12 
  47 −27, 40, −7 
Postcentral gyrus 1/2 −48, −29, 51 
  −35, −43, 60 
Inferior parietal lobule 40 −48, −47, 44 
  7/40 −35, −38, 52 
Anterior hippocampus  −24, −10, −15 
Putamen  21, 9, 11 
Cerebellum Bilateral   
Word–picture misattributions (‘no picture’) > word–picture correct attributions (‘picture’)    
Medial frontal/orbital frontal gyrus 10/11 −6, 45, −20 
 11/32 6, 43, −17 
  10/11 3, 31, −14 
Middle frontal gyrus 24, 28, 40 
Precentral gyrus 38, −18, 48 
   48, −9, 47 
Posterior cingulate gyrus 23/31 12, −60, 22 
Middle temporal gyrus 39 42, −74, 26 
  37/39 50, −62, 25 
Word-only correct attributions (‘no picture’) > word-only misattributions (‘picture’)    
Medial frontal/orbital frontal gyrus bilateral 11 0, 34, −12 
Precentral gyrus 45, −12, 53 
Inferior temporal gyrus 20 −56, −21, −21 
Middle/superior temporal gyrus 21/22 60, −14, −2 
Anterior hippocampus  −29, −10, −22 
Word-only misattributions (‘picture’) > word-only correct attributions (‘no picture’)    
Superior frontal gyrus −18, −5, 66 
Inferior frontal gyrus 10/47 −42, 44, −2 
Precentral gyrus −32, −20, 59 
Anterior cingulate gyrus 32 −3, 8, 44 
 32 3, 20, 40 
  32 6, 28, 29 
Inferior parietal lobe 40 −35, −30, 40 
  7/40 −30, −38, 43 
 7/40 35, −33, 38 
  7/40 42, −38, 54 
Precuneus −17, −52, 50 
Insula  35, 14, 2 
Cerebellum
 
Bilateral
 

 

 

All regions consist of at least five voxels, P < 0.001 uncorrected.

Assignment to Pictorial Presentation

We first performed a conjunction analysis to examine the regions in which activity was greater for ‘picture’ than ‘no picture’ responses, regardless of accuracy (i.e. word–picture correct attributions > word–picture misattributions, and word-only misattributions > word-only correct attributions). This analysis revealed activity in the fusiform gyrus, precuneus, lateral inferior parietal lobe, and prefrontal cortex (Fig. 1 and Table 3). The fusiform gyrus and precuneus have been implicated in visual processing (e.g. Fletcher et al., 1995; Ishai et al., 2000; Kosslyn and Thompson, 2000; Suchan et al., 2002) and in recollective retrieval (Shannon and Buckner, 2004), so activity in these regions is consistent with the hypothesis that individuals will assign a memory to external presentation when they retrieve the information with particularly vivid sensory detail. Activity in the lateral inferior parietal lobe and in the prefrontal cortex often is associated with retrieval of contextual information (Wheeler and Buckner, 2003, 2004), and interestingly, these two regions were those also identified by Okado and Stark (2003) as being related to memory assignment to external presentation.

Table 3

Regions in which activity was revealed by conjunction analysis to correspond with assignment of a memory to external presentation (‘picture’ > ‘no picture’ assignment for word-only and word–picture items)

Region
 
Hemisphere
 
BA
 
Talairach coordinates
 
Middle frontal gyrus 9/46 −45, 30, 31 
  10/46 −39, 46, 0 
 9/46 44, 33, 26 
Inferior frontal gyrus 47 30, 19, −11 
Precentral gyrus 4/5 −48, −30, 51 
Inferior parietal lobe 40 −42, −53, 52 
 40 47, −48, 38 
Cingulate gyrus bilateral 23/31 0, −28, 29 
Fusiform gyrus 18 −32, −76, −11 
Precuneus bilateral ±12, −68, 48 
Thalamus  −15, −20, 7 
Caudate  15, −5, 20 
Cerebellum
 
Bilateral
 

 

 
Region
 
Hemisphere
 
BA
 
Talairach coordinates
 
Middle frontal gyrus 9/46 −45, 30, 31 
  10/46 −39, 46, 0 
 9/46 44, 33, 26 
Inferior frontal gyrus 47 30, 19, −11 
Precentral gyrus 4/5 −48, −30, 51 
Inferior parietal lobe 40 −42, −53, 52 
 40 47, −48, 38 
Cingulate gyrus bilateral 23/31 0, −28, 29 
Fusiform gyrus 18 −32, −76, −11 
Precuneus bilateral ±12, −68, 48 
Thalamus  −15, −20, 7 
Caudate  15, −5, 20 
Cerebellum
 
Bilateral
 

 

 

We also examined whether there were regions that showed an interaction between memory assignment and memory accuracy (i.e. that were more active during accurate assignment to ‘picture’ presentation than during inaccurate assignment, or vice versa). We therefore computed the following: (word–picture correct attribution > word–picture misattribution) > (word-only misattribution > word-only correct attribution). Two regions showed this interaction, with more activity during the retrieval of word–picture items accurately assigned to pictorial presentation than during all other conditions: the precuneus [−20, −47, 52; Brodmann's area (BA) 7] and the left lateral parietal lobe (−38, −38, 54; BA 7/40). The activity in the precuneus is consistent with prior evidence suggesting that recollection of sensory details is greatest during accurate attribution of information to a presented source (Gonsalves and Paller, 2000; Okado and Stark, 2003). The effect in the left lateral parietal lobe may be related to the amount of contextual information retrieved for each item. It has been proposed that activity in the left lateral parietal lobe relates to the amount of contextual detail individuals feel they have retrieved (Okado and Stark, 2003; Wheeler and Buckner, 2003, 2004), and it is plausible that individuals believed they had retrieved more contextual information about the items that had been associated with visual presentation than about those that had only been mentally imagined.

Interestingly, the opposite pattern of activity [(word-only misattribution > word-only correct attribution) > (word–picture correct attribution > word–picture misattribution)] was shown in the left middle frontal gyrus (−45, 44, 0; BA 10/46): This region showed an interaction whereby it was most responsive on trials when individuals inaccurately assigned an item to pictorial presentation. The left prefrontal cortex has often been associated with demands for source retrieval (e.g. Nolde et al., 1998; Dobbins et al., 2002; Dobbins and Wagner, 2005) and with post-retrieval monitoring (reviewed by Rugg, 2005). Thus, the activity in the left middle frontal gyrus may reflect additional post-retrieval monitoring engaged on trials for which individuals knew that an item was familiar but required additional processes to determine whether or not a picture of the item was studied.

Assignment to a Nonpresented Source

To explore the regions in which activity corresponded with attribution to a nonpresented source, regardless of whether that attribution was correct or incorrect, we performed a conjunction analysis to examine the regions shared by the following contrasts: word-only correct attributions > word-only misattributions, and word–picture misattributions > word–picture correct attributions. This analysis revealed that activity in the ventromedial prefrontal cortex, posterior cingulate gyrus, and middle and superior temporal gyri all showed this pattern of response (Table 4). Interestingly, the pattern of activity in all of these regions showed deflections from baseline (Fig. 2), with greater ‘deactivation’ during retrieval of items assigned to pictorial presentation than during retrieval of items assigned to a nonpresented source. Although the meaning of below-baseline activity is still being discussed (Gusnard et al., 2001; see Archer et al., 2003; Grecius et al., 2003; McKiernan et al., 2003; Daselaar et al., 2004), the regions identified here are consistent with those that have been implicated in self-referential processes (Kelley et al., 2002; D'Argembeau et al., 2005) that are thought to be carried out during the fixation-cross baseline conditions (reviewed by Gusnard and Raichle, 2001; Northoff and Bermpohl, 2004). Thus, we suggest that the near-baseline levels of activity in these regions during the assignment of items to nonpresented sources corresponds with relatively high levels of the same types of self-referential processes that individuals typically carry out during the baseline condition. In contrast, during assignment to pictorial presentation, individuals appear to have disengaged from self-referential cognition to allow other processing (e.g. the visual and contextual-memory processing discussed above) to occur.

Figure 2.

Retrieval-related activity was greater for studied items assigned to a nonpresented source (i.e. items given a ‘no picture’ response versus a ‘picture’ response, A) in the superior temporal gyrus (BA 22/39, in pink, B), posterior cingulate gyrus (BA 23/31, in green, C) and ventromedial prefrontal cortex (BA 10/11, in blue, D). CA, correct attribution; MA, misattribution.

Figure 2.

Retrieval-related activity was greater for studied items assigned to a nonpresented source (i.e. items given a ‘no picture’ response versus a ‘picture’ response, A) in the superior temporal gyrus (BA 22/39, in pink, B), posterior cingulate gyrus (BA 23/31, in green, C) and ventromedial prefrontal cortex (BA 10/11, in blue, D). CA, correct attribution; MA, misattribution.

Table 4

Regions in which activity was revealed by conjunction analysis to correspond with assignment of a ‘no picture’ response (for both word-only and word–picture items)

Region
 
Hemisphere
 
BA
 
Talairach coordinates
 
Medial frontal/orbital frontal gyrus 10 −11, 50, 6 
  11 −3, 46, −20 
 11 3, 31, −14 
  11 6, 42, −18 
Middle frontal gyrus 26, 31, 40 
Precentral gyrus 39, −18, 48 
  47, −5, 22 
Posterior cingulate gyrus 23 −9, −52, 13 
  23 −6, −54, 27 
 23/31 12, −57, 22 
Superior temporal gyrus 22 −39, 7, −15 
 22 53, −14, 3 
  22/39 60, −54, 19 
Middle temporal gyrus 39 −53, −63, 22 
 37/39 50, −60, 17 
  21 48, 7, −30 

 

 
21
 
56, −18, −12
 
Region
 
Hemisphere
 
BA
 
Talairach coordinates
 
Medial frontal/orbital frontal gyrus 10 −11, 50, 6 
  11 −3, 46, −20 
 11 3, 31, −14 
  11 6, 42, −18 
Middle frontal gyrus 26, 31, 40 
Precentral gyrus 39, −18, 48 
  47, −5, 22 
Posterior cingulate gyrus 23 −9, −52, 13 
  23 −6, −54, 27 
 23/31 12, −57, 22 
Superior temporal gyrus 22 −39, 7, −15 
 22 53, −14, 3 
  22/39 60, −54, 19 
Middle temporal gyrus 39 −53, −63, 22 
 37/39 50, −60, 17 
  21 48, 7, −30 

 

 
21
 
56, −18, −12
 

We then examined whether there were regions that showed an interaction between memory accuracy and memory assignment (i.e. showing greater activity during retrieval of word-only items accurately assigned to a nonpresented source versus during retrieval of word–picture items inaccurately assigned to a nonpresented source, or vice versa). To examine the former we computed (word-only correct attribution > word-only misattribution) > (word–picture misattribution > word–picture correct attribution). The superior (±53, −9, 7, BA 22) and middle temporal gyrus (44, −40, 10, BA 21/22; −53, 2, −18, BA 21) showed this pattern of response, with greater activity when word-only items were correctly assigned a ‘no picture’ response than when word–picture items were incorrectly given such a response. In contrast, when we examined regions that responded more to word–picture items inaccurately assigned to a nonpresented source, i.e. (word–picture misattribution > word–picture correct attribution) > (word-only correct attribution > word-only misattribution), we found that the posterior cingulate gyrus (9, −63, 21; BA 23/31) showed this pattern of response. It was more active when word–picture items were inaccurately assigned than in all other conditions. Future research will be required to clarify the independent contributions of these regions; for now, we simply note that these different patterns of activity suggest that the regions of the temporal gyrus support processes distinct from those of the posterior cingulate gyrus.

Discussion

Prior studies have revealed the neural processes that correspond with accurate versus inaccurate retrieval on a reality-monitoring task (Gonsalves and Paller, 2000; Okado and Stark, 2003; Kensinger and Schacter, 2005b). These studies have found increased activity in medial temporal-lobe regions (the anterior hippocampus: Kensinger and Schacter, 2005b; the posterior parahippocampal gyrus: Okado and Stark, 2003), and in the occipital cortex (Gonsalves and Paller, 2000; Okado and Stark, 2003) during accurate retrieval. The present study instead focused on the activity associated with memory attribution to pictorial presentation or to a nonpresented source. Therefore, we were able to examine the retrieval-related processes that correspond with an individual's belief about an item's history regardless of the accuracy of that belief.

Attribution to a Presented Source

With regard to the processes that are related to a particular memory attribution, we were most interested in whether there were neural processes that corresponded with memory attribution to a pictorial source, regardless of the accuracy of that attribution. Our main hypothesis was that items attributed to pictorial presentation would be associated with additional retrieval-related activity in visual processing regions than items attributed to a nonpresented source. This hypothesis was supported: activity in the fusiform gyrus and precuneus corresponded with attribution of a memory to pictorial presentation. Both of these regions have been associated with visual processing (Fletcher et al., 1995; Ishai et al., 2000; Kosslyn and Thompson, 2000; Suchan et al., 2002), although the precuneus also may play a broader role in the recollection of prior experiences (Shannon and Buckner, 2004). These neural data dovetail nicely with behavioral data indicating that individuals will be most likely to attribute an item to pictorial presentation when vivid visual information is remembered and there is little memory for cognitive operations (Johnson and Raye, 1981; Johnson et al., 1981). Together, these data provide strong evidence that individuals use the amount of different types of information retrieved to guide their attributions on reality-monitoring tasks (Johnson et al., 1993; Johnson, 1997). When rich perceptual information is remembered about an item, individuals will attribute their memory to a presented source.

It is important to note that this conclusion is not incompatible with prior neuroimaging studies that found greater activity during retrieval of items accurately attributed to pictorial presentation than during retrieval of imagined items inaccurately attributed (Gonsalves and Paller, 2000; Okado and Stark, 2003). In fact, we also found evidence that a region of the precuneus, associated with visual processing and visuo-spatial attention (e.g. Fletcher et al., 1995; Kosslyn and Thompson, 2000; Culham and Kanwisher, 2001) and with the recollective qualities of a memory (see Shannon and Buckner, 2004), was differentially responsive to items correctly attributed to pictorial presentation. This result is consistent with behavioral data indicating that misattributed memories can differ phenomenologically from accurately attributed memories: participants often report that memories for items correctly believed to have been perceived include more sensory detail and associated thoughts and feelings than do memories for items inaccurately attributed to a presented source (Mather et al., 1997; Norman and Schacter, 1997). Taken together, these data suggest that more sensory information and recollective details may be available for items attributed to pictorial presentation than for those attributed to a nonpresented source. However, even for the visually attributed items, there is an effect of item history: items accurately attributed to a presented source are associated with additional activity in the precuneus than items misattributed. This suggests that there is some sensory or recollective processing that distinguishes correct attributions from incorrect attributions, but that individuals do not capitalize on this information to guide their attribution decisions (see also Slotnick and Schacter, 2004). Individuals may assign items to pictorial presentation when the sensory information retrieved surpasses some threshold; variability above that threshold may not influence the attribution decision.

The precuneus was not the only region in which activity was modulated by the accuracy of the assignment. Activity in the lateral parietal lobe also showed differential activity for items accurately versus inaccurately attributed to pictorial presentation. It is likely that this pattern of activity results from the correspondence of left lateral parietal activity to the amount of contextual information believed to have been retrieved. Prior studies have demonstrated that this region shows greater activity for items assigned an ‘old’ response than for those given a ‘new’ response (Wheeler and Buckner, 2003). There also is more left lateral parietal activity when individuals believe that they vividly ‘remember’ an item was presented than when they simply feel that the item is familiar (Wheeler and Buckner, 2004). In the present study, individuals may have retrieved (or believed they had retrieved) more contextual information about the items correctly attributed to visual presentation than about those mentally imagined items incorrectly attributed to a presented source (for more discussion, see Okado and Stark, 2003). This hypothesis is generally consistent with the differential activity in the precuneus, suggesting that there was more sensory information available for items correctly attributed than for those misattributed to pictorial presentation.

In contrast to the precuneus and lateral parietal activity, activity in the left middle frontal gyrus showed the opposite pattern: greater activity for items inaccurately attributed to pictorial presentation than for those accurately attributed. This region is thought to play a critical role in retrieval orientation on tasks requiring source monitoring (e.g. Nolde et al., 1998; Dobbins et al., 2002; Dobbins and Wagner, 2005) and in post-retrieval monitoring (reviewed by Rugg, 2005). It is plausible that these source- and post-retrieval monitoring processes were disproportionately engaged for the misattributed items. These object names may have triggered a sense of familiarity, but participants may have required additional processes to distinguish whether the item had been perceived or only imagined.

Attribution to a Nonpresented Source

A very different network of regions corresponded with attribution of items to a nonpresented source. The network closely paralleled what has been termed a ‘default network’ (e.g. Gusnard and Raichle, 2001), thought to underlie self-referential processing. These regions have been related to a variety of self-evaluative processes, including judgements of self-descriptive adjectives (e.g. Kelley et al., 2002), assessments of agency (e.g. Farrer et al., 2003) and assignments of ownership (e.g. Vogeley and Fink, 2003). These regions tend to show ‘deactivations’ (or deflections from baseline), thought to reflect the fact that self-referential processes are often carried out even during the fixation-cross baseline periods in fMRI experiments.

We interpret the activity in these regions in the present study as reflecting the fact that, during attribution to a nonpresented source, engagement of self-referential processes remained high, and close to the levels during baseline. This finding is broadly consistent with the proposal that individuals will use information regarding cognitive operations performed during mental imagery to distinguish imagined from perceived items. In contrast, for items attributed to pictorial presentation, individuals disengaged from these self-referential processes, likely to divert resources to other types of processing (e.g. visual processing).

Conclusions

In summary, the results of the present study demonstrate that not only are there neural processes that correspond with accurate reality-monitoring performance, there also are processes that relate to the memory attribution made by an individual. Items accurately or inaccurately attributed to visual presentation were associated with activity in visual processing regions (precuneus and fusiform gyrus), as well as activity in regions associated with contextual retrieval of information (lateral parietal lobe). In contrast, items attributed to a nonpresented source were associated with activity thought to reflect self-referential processing (in the ventromedial PFC, posterior cingulate gyrus, middle and superior temporal gyrus). These results are consistent with the proposal (reviewed by Johnson, 1997) that information attributed to a presented source will be associated with additional perceptual information, while information attributed to a nonpresented source will have additional information regarding the processes used to generate the item.

More broadly, these data are important in demonstrating that not only are there neural signatures that correspond with accurate attribution to different sources (e.g. Nyberg et al., 2000; Wheeler and Buckner, 2003, 2004; Slotnick and Schacter, 2004), including to imagined versus perceived sources (Lundstrom et al., 2003; Okado and Stark, 2003), there are also processes that lead to a particular memory attribution regardless of its accuracy. These results fit nicely with two prior studies (Kahn et al., 2004; Slotnick and Schacter, 2004) revealing common neural processes corresponding with ‘old’ responses on a recognition task, regardless of whether that response reflected correct recognition or false recognition. In other words, there were processes shared by items that individuals had studied and those that the individuals only believed that they had studied. The results of the present experiment take those results one step further by demonstrating that this type of overlap in processes can occur not only when activity is compared for old and new items assigned an ‘old’ response, but also when activity is examined for old items attributed to different study conditions. This convergence of results using old/new recognition and reality-monitoring tasks is consistent with the proposal (e.g. Johnson et al., 1993) that both types of retrieval judgements rely on source attributions. Thus, the neural activity subserving retrieval processes can be related both to the belief that an item was studied, and also to the belief than an item was studied in a particular way (e.g. as a picture).

This research was supported by grants MH60941 (to D.L.S.) and MH070199 (to E.A.K.) from the National Institutes of Health. The Martinos Center is supported by the National Center for Research Resources (grant P41RR14075) and by the MIND Institute. We thank Ronnie Bryan and Mariko Jameson for their assistance with participant recruitment and testing.

References

Archer JS, Abbott DF, Waites AB, Jackson GD (
2003
) fMRI ‘deactivation’ of the posterior cingulate during generalized spike and wave.
Neuroimage
 
20
:
1915
–1922.
Culham JC, Kanwisher NG (
2001
) Neuroimaging of cognitive functions in human parietal cortex.
Curr Opin Neurobiol
 
11
:
157
–163.
Dale AM (
1999
) Optimal experimental design for event-related fMRI.
Hum Brain Mapp
 
8
:
109
–114.
D'Argembeau A, Collette F, Van der Linden M, Laureys S, Del Fiore G, Degueldre C, Luxen A, Salmon E (
2005
) Self-referential reflective activity and its relationship with rest: a PET study.
Neuroimage
 
25
:
616
–624.
Daselaar SM, Prince SE, Cabeza R (
2004
) When less means more: deactivations during encoding that predict subsequent memory.
Neuroimage
 
23
:
921
–927.
Dobbins IG, Wagner AD (February 23,
2005
) Domain-general and domain-sensitive prefrontal mechanisms for recollecting events and detecting novelty. Cereb Cortex 10.1093/cercor/bhi054.
Dobbins IG, Foley H, Schacter DL, Wagner AD (
2002
) Executive control during episodic retrieval: multiple prefrontal processes subserve source memory.
Neuron
 
35
:
989
–996.
Dobson M, Markham R (
1993
) Imagery ability and source monitoring: implications for eyewitness memory.
Br J Psychol
 
84
:
111
–118.
Farrer C, Franck N, Georgieff N, Frith CD, Decety J, Jeannerod M (
2003
) Modulating the experience of agency: a positron emission tomography study.
Neuroimage
 
18
:
324
–333.
Fisher RA (
1950
) Statistical methods for research workers. London: Oliver & Boyd.
Fletcher PC, Frith CD, Baker SC, Shallice T, Frackowiak RS, Dolan RJ (
1995
) Themind's eye–precuneus activation in memory-related imagery.
Neuroimage
 
2
:
195
–200.
Gonsalves B, Paller KA (
2000
) Neural events that underlie remembering something that never happened.
Nat Neurosci
 
3
:
1316
–1321.
Gonsalves B, Reber PJ, Gitelman DR, Parrish TB, Mesulam M-M, Paller KA (
2004
) Neural evidence that vivid imagining can lead to false remembering.
Psychol Sci
 
15
:
655
–660.
Greicius MD, Krasnow B, Boyett-Anderson JM, Eliez S, Schatzberg AF, Reiss AL, Menon V (
2003
) Regional analysis of hippocampal activation during memory encoding and retrieval: fMRI study.
Hippocampus
 
13
:
164
–174.
Gusnard DA, Raichle ME (
2001
) Searching for a baseline: functional imaging and the resting human brain.
Nat Rev Neurosci
 
2
:
683
–694.
Gusnard DA, Akbudak E, Shulman GL, Raichle ME (
2001
) Medial prefrontal cortex and self-referential mental activity: relation to a default mode of brain function.
Proc Natl Acad Sci USA
 
98
:
4259
–64.
Ishai A, Ungerleider LG, Haxby JV (
2000
) Distributed neural systems for the generation of visual images.
Neuron
 
28
:
979
–990.
Johnson MK (
1997
) Source monitoring and memory distortion.
Philos Trans Biol Sci
 
352
:
1733
–1745.
Johnson MK, Hirst, W (
1993
) MEM: memory subsystems as processes. In: Theories of memory (Collins AF, Gathercole SE, Conway MA, Morris PE, eds), pp. 241–286. Hove: Erlbaum.
Johnson MK, Raye CL (
1981
) Reality monitoring.
Psychol Rev
 
88
:
67
–85.
Johnson MK, Taylor TH, Raye CL (
1977
) Fact and fantasy: the effects of internally generated events on the apparent frequency of externally generated events.
Mem Cogn
 
5
:
116
–122.
Johnson MK, Raye CL, Foley HJ, Foley MA (
1981
) Cognitive operations and decision bias in reality monitoring.
Am J Psychol
 
94
:
37
–64.
Johnson MK, Foley MA, Leach K (
1988
) The consequences for memory of imagining in another person's voice.
Mem Cogn
 
16
:
337
–342.
Johnson MK, Hashtroudi S, Lindsay DS (
1993
) Source monitoring.
Psychol Bull
 
114
:
3
–28.
Kahn I, Davachi L, Wagner AD (
2004
) Functional–neuroanatomic correlates of recollection: implications for models of recognition memory.
J Neurosci
 
24
:
4172
–4180.
Kelley WM, Macrae CN, Wyland CL, Caglar S, Inati S, Heatherton TF (
2002
) Finding the self? An event-related fMRI study.
J Cogn Neurosci
 
14
:
785
–794.
Kensinger EA, Schacter DL (
2005
a) Emotional content and reality-monitoring ability: FMRI evidence for the influence of encoding processes.
Neuropsychologia
 
43
:
1429
–1443.
Kensinger EA, Schacter DL (
2005
b) Retrieving accurate and distorted memories: neuroimaging evidence for effects of emotion.
Neuroimage
 
27
:
167
–177.
Kensinger EA, Clarke RJ, Corkin, S (
2003
) What neural correlates underlie successful encoding and retrieval? A functional magnetic resonance imaging study using a divided attention paradigm.
J Neurosci
 
23
:
2407
–2415.
Kosslyn SM, Thompson WL (
2000
) Shared mechanisms in visual imagery and visualperception: insights from cognitive neuroscience. In: The new cognitive neurosciences (Gazzaniga MS, ed.), pp. 975–985. Cambridge, MA: MIT Press.
Lazar NA, Luna B, Sweeney JA, Eddy WF (
2002
) Combining brains: a survey of methods for statistical pooling of information.
Neuroimage
 
16
:
538
–550.
Lundstrom BN, Petersson KM, Andersson J, Johansson M, Fransson P, Ingvar M (
2003
) Isolating the retrieval of imagined pictures during episodic memory: activation of the left precuneus and left prefrontal cortex.
Neuroimage
 
20
:
1934
–1943.
Mather M, Henkel LA, Johnson MK (
1997
) Evaluating characteristics of false memories: remember/know judgements and memory characteristics questionnaire compared.
Mem Cogn
 
26
:
826
–837.
McKiernan KA, Kaufman JN, Kucera-Thompson J, Binder JR (
2003
) A parametric manipulation of factors affecting task-induced deactivation in functional neuroimaging.
J Cogn Neurosci
 
15
:
394
–408.
Nolde SF, Johnson MK, D'Esposito M (
1998
) Left prefrontal activation during episodic remembering: an event-related fMRI study.
Neuroreport
 
26
:
3509
–3514.
Norman KA, Schacter DL (
1997
) False recognition in younger and older adults: exploring the characteristics of illusory memories.
Mem Cogn
 
26
:
838
–848.
Northoff G, Bermpohl F (
2004
) Cortical midline structures and the self.
Trends Cogn Sci
 
8
:
102
–107.
Nyberg L, Habib R, McIntosh AR, Tulving E (
2000
) Reactivation of encoding-related brain activity during memory retrieval.
Proc Natl Acad Sci USA
 
97
:
11120
–11124.
Okado Y, Stark C (
2003
) Neural processing associated with true and false memory retrieval.
Cogn Affect Behav Neurosci
 
3
:
323
–334.
Rabinowitz JC (
1989
) Judgements of origin and generation effects: comparisons between young and elderly adults.
Psychol Aging
 
4
:
259
–268.
Rugg MD (
2005
) Retrieval processing in human memory: electrophysiological and fMRI evidence. In: The cognitive neurosciences, 3rd edn (Gazzaniga MS, ed.), pp. 727–739. Cambridge, MA: MIT Press.
Schacter DL, Slotnick SD (
2004
) The cognitive neuroscience of memory distortion.
Neuron
 
44
:
149
–160.
Shannon BJ, Buckner RL (
2004
) Functional-anatomic correlates of memory retrieval that suggest nontraditional processing roles for multiple distinct regions within posterior parietal cortex.
J Neurosci
 
24
:
10084
–10092.
Slotnick SD, Schacter DL (
2004
) A sensory signature that distinguishes true from false memories.
Nat Neurosci
 
7
:
664
–672.
Suchan B, Yaguez L, Wunderlich G, Canavan AGM, Herzog H, Tellmann L, Homberg V, Seitz RJ (
2002
) Neural correlates of visuospatial imagery.
Behav Brain Res
 
131
:
163
–168.
Talairach J, Tournoux P (
1998
) Co-planar stereotaxic axis of the human brain. New York: Thieme.
Vogeley K, Fink GR (
2003
) Neural correlates of the first-person perspective.
Trends Cogn Sci
 
4
:
14
–21.
Wheeler ME, Buckner RL (
2003
) Functional dissociation among components of remembering: control, perceived oldness, and content.
J Neurosci
 
23
:
3869
–3880.
Wheeler ME, Buckner RL (
2004
) Functional–anatomic correlates of remembering and knowing.
Neuroimage
 
21
:
1337
–1349.