Abstract

Using functional magnetic resonance imaging, the present study investigated whether the enhanced memory performance associated with congruent relative to incongruent retrieval cues is modulated by how items are encoded. Subjects studied a list of visually presented words and pictures and attempted to recognize these items in a later memory test. Half of the studied items were tested with a congruent cue (word–word and picture–picture), whereas the remainders were tested with an incongruent cue (word–picture and picture–word). For both words and pictures, regions where study activity was greater for congruently than incongruently cued items overlapped regions where activity differentiated the 2 classes of study material. Thus, word congruency effects overlapped regions where activity elicited by study words exceeded the activity elicited by pictures. Similarly, picture congruency effects overlapped regions demonstrating enhanced activity for pictures relative to words. In addition, several regions, including dorsolateral prefrontal cortex and intraparietal sulcus, demonstrated material-nonspecific congruency effects. The findings suggest that items benefit from a congruent retrieval cue when their study processing resembles the processing later engaged by the retrieval cue. Consistent with the principle of transfer appropriate processing, the benefit of a congruent retrieval cue derives from the interaction between study and retrieval processing.

Introduction

The likelihood that an event will be remembered depends both on how it was initially encoded and how memory is cued during a retrieval attempt. The importance of the relationship between encoding and retrieval has been recognized in 2 related principles—encoding specificity (Tulving and Thomson 1973) and transfer appropriate processing (TAP; Morris et al. 1977)—that have influenced research in this area for more than 30 years. Central to both principles is the importance of “study–test overlap” in understanding the interaction between encoding and retrieval. Other things being equal, the effectiveness of a retrieval cue depends on the amount of overlap between the processing engaged by the cue and the processing engaged during encoding: the greater the overlap, the greater the likelihood of successful retrieval (Roediger et al. 1989; Roediger and Guynn 1996; but see Nairne 2002).

One example of the influence of study–test overlap comes from studies manipulating the “congruency” of study items and retrieval cues. For example, recognition memory for both pictures and words is markedly higher when the format of the test items is the same as the study format (picture–picture and word–word) than when the formats are changed (picture–word and word–picture) (Hornberger et al. 2004; Schloerscheidt and Rugg 2004). Such findings are of course easily accommodated within the encoding specificity and TAP frameworks. Importantly though, these frameworks do not only lead to the rather obvious prediction that congruent retrieval cues will be more effective than incongruent cues. They also predict that the likelihood that retrieval will be facilitated by a congruent cue will depend on the amount of overlap between study processing and the processing engaged by the cue during a subsequent retrieval attempt. That is, the likelihood that an item will benefit from a congruent retrieval cue should depend on the extent that processing engaged at study resembles the processing later engaged by the cue.

Motivated by this prediction, the present study employed functional magnetic resonance imaging (fMRI) to assess whether encoding-related neural activity differs for study items that are later successfully retrieved in response to congruent versus incongruent retrieval cues. The study employs a variant of the widely employed subsequent memory procedure, in which neural activity elicited by study items is contrasted according to performance on a later memory test (e.g., Wagner et al. 1998; Henson et al. 1999; Kirchhoff et al. 2000; see Paller and Wagner 2002 for review). Using this procedure, it has been demonstrated in a number of studies that the regions exhibiting “subsequent memory effects” differ according to study task and hence with the processes engaged during encoding (Otten and Rugg 2001a; Otten et al. 2002; Mitchell et al. 2004; Harvey et al. 2007). It has also been reported that subsequent memory effects differ in their localization depending on the format of the retrieval cues employed in the later memory test (Otten forthcoming). Thus, neural activity associated with successful memory encoding is sensitive to manipulations of both study and retrieval processing.

In the present experiment, we employed a crossed design in which words and pictures were presented at both study and test. We expected that, for both item classes, subsequent recognition performance would be higher when memory was cued by congruent (same material) than incongruent (different material) items at test. Because recognition memory can be supported by both episodic retrieval (recollection) and an acontextual “familiarity” signal, we assessed recognition performance with the remember/know procedure (Tulving 1985). This permitted the contributions of recollection and familiarity to recognition to be separately estimated (Yonelinas and Jacoby 1995), allowing an assessment of whether, as hypothesized, cue congruency modulates likelihood of recollection. We identified encoding-related activity associated with these cue congruency effects by contrasting the activity elicited at study by items that were later successfully recognized in response to congruent versus incongruent cues. For the reasons noted above, we predicted that these neural cue congruency effects would, in part, overlap with the activity elicited by the cues themselves. Because memory for the study items was tested outside the scanner, this prediction could not be tested directly. However, by contrasting the activity elicited by the 2 classes of study item, we were able to identify regions that were selectively activated during word and picture encoding regardless of how memory was later cued. Thus, we could ask whether the regions exhibiting material-selective activity overlapped with regions demonstrating material-specific cue congruency effects. In effect, we asked whether studied words that were later retrieved with congruent cues were processed in a more “word-like” manner than words retrieved with incongruent cues. Analogously, we also asked whether the study activity elicited by congruently cued pictures was more “picture-like” than the activity elicited by pictures later retrieved with incongruent cues. If cue congruency and material-selective effects do indeed overlap, this would lend support to the proposal that congruent retrieval cues are most effective when processing at study resembles the processing engendered by the cue in the subsequent test.

In addition to material-specific cue congruency effects, the present study also allowed 2 other kinds of encoding effect to be investigated. By searching for cue congruency effects common to the 2 classes of study item, material-independent effects can be identified. And, by searching for regions where encoding-related activity is greater for later recognized items retrieved with incongruent rather than congruent cues, the possibility that these items are also associated with a distinct encoding signature can be evaluated.

Materials and Methods

Subjects

Twenty-five subjects participated in the experiment (aged 18–30 years; 11 male). All were right-handed native English speakers. They were recruited from the University of California, Irvine (UCI) community and compensated for their participation. Two subjects were excluded from analysis, one for inadequate behavioral performance and the other for excessive head movement. Informed consent was obtained before participation in accord with the requirements of the UCI Institutional Review Board.

Materials

Stimuli were drawn from a pool of 394 pairs of color pictures of objects and their corresponding names. A study list comprised 240 critical items, 120 of which were words and 120 pictures, along with an additional 2 buffer items (one picture and one word). One half of the items in the word and picture sets denoted or depicted items typically found indoors, with the remainder comprising outdoor items. Test lists consisted of 360 items, made up of a random sequence of 240 studied items (120 words and 120 pictures) and 120 new items (60 pictures and 60 words), along with 2 buffers. Half of the studied items were presented at test in the same format as at study (i.e., word–word; picture–picture; congruent cue condition); for the remaining items, study and test formats were switched (i.e., word–picture; picture–word; incongruent cue condition). The 30 remaining items from the stimulus pool were used for practice trials. Study and test lists were constructed separately for each subject.

Items were displayed with VisuaStim (Resonance Technology, Northridge, CA) XGA MRI compatible head-mounted goggles with a resolution of 800 × 600 pixels. The field of display subtended visual angles of 30° × 23° at a virtual viewing distance of 1.2 m. The critical items were presented in a gray frame at the center of the visual field preceded by a fixation cross (+). The frame subtended visual angles of 8.4° × 8.4°. The maximum visual angle subtended by a picture was 6° × 6.5°. Words were presented in white upper case Helvetica 30 point font at a maximum visual angle of 8° × 1.5°. During the test phase, items were presented with the same display parameters as at study.

Procedure

The experiment consisted of a scanned incidental study phase followed by a nonscanned recognition memory test. Subjects were given instructions and practice on the study task prior to the experiment proper outside the scanner. For each study trial, a fixation cross was presented for 200 ms followed immediately by a study item, which was displayed for 1000 ms. The item was replaced by a white fixation asterisk for 1800 ms, which was followed by the onset of the next trial. This trial sequence resulted in a stimulus onset asynchrony (SOA) of 3000 ms. The study list also contained 120 randomly interspersed null trials, during which the fixation cross was continuously displayed for 1200 ms and no response was required. The presentation order of items in the study list was pseudorandom, with the constraint that there were no more than 3 consecutive trials of the same item type, including null trials. For the study task, subjects were required to indicate whether the study item was more likely to be found indoors or outdoors by pressing a button with the index or middle finger of their right hand. The assignment of finger to the indoor/outdoor response was counterbalanced across subjects. The study task proper was presented in a single block.

The test phase was administered out of the scanner approximately 15 min after the end of the study task. No mention was made of a memory test until this point in the experiment. Each test trial consisted of the presentation of a fixation cross for 100 ms followed by the presentation of a test item for 1000 ms on a computer monitor. The test item was followed by an asterisk that was displayed as the prompt for a response. Subjects were told that they would see a series of intermixed pictures and words, both studied and new. They were informed that some of the studied items would be presented in the same format as at study and some in the alternate format and that an item was to be considered old if it had been presented at study regardless of whether study and test format matched. Subjects were instructed to make a “remember/know/new” response to each test item (Tulving 1985) according to the following criteria: 1) recognition of the item was accompanied by recollection of one or more specific details about the study presentation (remember); 2) the item was judged to have presented in the study phase in the absence of recollection (know); and 3) the item was judged to be unstudied (new). To minimize contamination of old responses by guesses, subjects were instructed to use the new response if they were unsure about an item's study status. A short practice list was administered prior to the test proper, which was self-paced.

fMRI Scanning Parameters

A 1.5-T Philips Eclipse MR scanner (Philips Medical Systems, Bothell, WA) was used to acquire both T1-weighted anatomical volume images (256 × 256 matrix, 1 mm3 voxels) and T2*-weighted gradient echo-planar images (EPIs) (64 × 92 matrix, 2.6 × 3.9 mm pixels, time echo = 40 ms) per volume. EPIs comprised twenty-seven 3-mm-thick axial slices with a 1.5-mm interslice gap acquired in a descending sequential order. The study phase comprised 374 volumes at a repetition time (TR) of 2500 ms. The first 5 volumes were discarded to allow tissue equilibration effects. The ratio of SOA to TR in the study phase resulted in a sampling of the impulse response at a rate of approximately 2 Hz over trials. The T1–weighted anatomical volume covering the whole brain was acquired for each subject following the study phase.

Data Analysis

Statistical parametric mapping (SPM2; Wellcome Department of Cognitive Neurology, London, UK: http//www.fil.ion.ucl.ac.uk), implemented in MATLAB 6 (Mathworks, Natick, MA), was used for preprocessing and analysis of the fMRI data. Volumes were spatially realigned to the first volume, and each slice was temporally realigned to the middle slice of its volume. The resulting data were normalized to a standard EPI template based on the Montreal Neurological Institute (MNI) reference brain (Cocosco et al. 1997) and resampled into 3 mm3 voxels using nonlinear basis functions (Ashburner and Friston 1999). Normalized images were smoothed with an isotropic 8-mm full-width half-maximum Gaussian kernel. The time series in each voxel was high-pass filtered at 1/128 Hz to remove low-frequency noise and scaled to a grand mean of 100 across both voxels and volumes. Each subject's T1-weighted anatomical scan was normalized to a standard T1 template of the MNI brain and resampled into 2 mm3 voxels. For each subject, neural activity was modeled by delta functions (impulse events) at stimulus onset. Then, the ensuing blood oxygenation level–dependent response was modeled by convolving these neural functions with a canonical hemodynamic response function and its temporal and dispersion derivatives.

The main experimental interest was in differences between activity elicited at study by items that were later correctly recognized in response to congruent versus incongruent retrieval cues. As described below, items associated with later remember and know judgments were collapsed to form one event of interest for each study–test condition. Thus, 4 events of interest were defined: congruent word (words correctly recognized in response to a word test item); incongruent word (words recognized in response to a picture test item); congruent picture (pictures recognized in response to pictures); and incongruent picture (pictures recognized in response to words). Items associated with incorrect or omitted test judgments, and buffer items, were classified as events of no-interest. Also included in the analysis model were 6 covariates modeling movement-related residual variance (the 3 rigid body translations and 3 rotations) determined from the realignment stage. Parameter estimates for events of interest were estimated using a general linear model. Nonsphericity of the error covariance was accommodated by an auto-regressive, AR (1), model, in which the temporal autocorrelation was estimated by pooling over suprathreshold voxels (Friston et al. 1998). The parameters for each covariate and the hyperparameters governing the error covariance were estimated using restricted maximum likelihood.

In the second stage of the analysis, linear contrasts of subject-specific parameter estimates were computed, treating subjects as a random effect. Regions of overlap between the outcomes of 2 contrasts were identified by inclusively masking the relevant SPMs, whereas exclusive masking was employed to identify voxels where effects were not shared between 2 contrasts. The peak voxels of clusters exhibiting reliable effects are reported in MNI coordinates. The statistical threshold for contrasts that were not subjected to inclusive masking was set at P < 0.001 with a 4 voxel cluster extent threshold. Exclusive masks were thresholded at P < 0.05 for one-sided t contrasts and at P < 0.1 for 2-sided F contrasts (note that the more liberal the contrast, the more conservative is the exclusive masking procedure). Conjoint thresholds for inclusively masked, independent contrasts were calculated using Fisher's procedure (Fisher 1950; Lazar et al. 2002). The primary contrast was thresholded at P < 0.01, and the masking contrast at P < 0.005, to give a conjoint threshold of approximately P < 0.0005 (in fact, the conjoint significance of all inclusively masked effects exceeded P < 10−5; see Table 3). Caret5 Software (Van Essen et al. 2001; http://brainmap.wustl.edu/caret) was used to map statistically significant clusters onto the PALS atlas (Van Essen 2005).

Results

Behavioral Results

Study Data

Accuracy rates on the study task were 0.89 for words and 0.88 for pictures; unsurprisingly, these did not significantly differ. Indoor/outdoor judgments were made with a mean reaction time (RT) of 1144 ms (standard deviation [SD] = 314) for words and 1108 ms (SD = 307) for pictures. RTs associated with later correctly recognized items were segregated according to test cue conditions and are presented in Table 1. To parallel the approach taken in the fMRI analyses, RTs were contrasted according to whether items were later recognized in response to a congruent or incongruent cue. RTs were significantly longer for words than pictures (by 49 ms; F1,22 = 8.33, P < 0.01), but neither the main effect of cue congruency nor its interaction with study modality was significant.

Table 1

Study RTs (ms) for later recognized items segregated by congruency of the retrieval cue (SD in parenthesis)

 Congruent cue Incongruent cue 
Word 1133 (69) 1146 (64) 
Picture 1113 (65) 1088 (64) 
 Congruent cue Incongruent cue 
Word 1133 (69) 1146 (64) 
Picture 1113 (65) 1088 (64) 

Test Data

Table 2 shows the proportions of hits and false alarms accorded remember and know responses, as well as estimates of recollection and familiarity, as a function of study and test format. For both classes of item, false alarms were more likely to be endorsed as “known” than “remembered” (F1,22 = 26.34, P < 0.001), an effect that was slightly but significantly greater for words than pictures (F1,22 = 8.46, P < 0.01). Recognition performance was calculated for remember and know judgments according to the assumption that recollection and familiarity are independent bases for recognition (Yonelinas and Jacoby 1995). Probability of recollection was estimated as [pRHITpRFA], whereas probability of familiarity was estimated as [(pKHIT/(1 − pRHIT)) − (pKFA/(1 − pRFA))]. Analysis of variance (ANOVA) of recollection estimates indicated that recollection was higher for studied pictures than for studied words (F1,22 = 13.33, P < 0.001) and for congruent relative to incongruent cues (F1,22 = 50.31, P < 0.001). The interaction between material and cue congruency was not significant. ANOVA of familiarity estimates revealed no significant effects. Recognition performance collapsed across remember and know judgments (quantified as pHit − pFA) revealed a pattern of results qualitatively identical to that obtained for remember judgments alone.

Table 2

Mean proportions of items recognized with remember and know responses and mean estimates of recollection and familiarity (SD in parenthesis)

Study material Test cue Remember Know Newa Recollection Familiarity 
Word Word 0.51 (0.21) 0.29 (0.18) 0.20 (0.13) 0.45 (0.21) 0.34 (0.24) 
Picture 0.33 (0.21) 0.31 (0.16) 0.36 (0.20) 0.24 (0.19) 0.27 (0.19) 
Picture Word 0.40 (0.24) 0.31 (0.16) 0.29 (0.18) 0.34 (0.23) 0.31 (0.19) 
Picture 0.67 (0.22) 0.20 (0.15) 0.13 (0.10) 0.58 (0.20) 0.37 (0.19) 
New Word 0.06 (0.05) 0.22 (0.14) 0.73 (0.15)   
Picture 0.08 (0.06) 0.18 (0.11) 0.73 (0.13)   
Study material Test cue Remember Know Newa Recollection Familiarity 
Word Word 0.51 (0.21) 0.29 (0.18) 0.20 (0.13) 0.45 (0.21) 0.34 (0.24) 
Picture 0.33 (0.21) 0.31 (0.16) 0.36 (0.20) 0.24 (0.19) 0.27 (0.19) 
Picture Word 0.40 (0.24) 0.31 (0.16) 0.29 (0.18) 0.34 (0.23) 0.31 (0.19) 
Picture 0.67 (0.22) 0.20 (0.15) 0.13 (0.10) 0.58 (0.20) 0.37 (0.19) 
New Word 0.06 (0.05) 0.22 (0.14) 0.73 (0.15)   
Picture 0.08 (0.06) 0.18 (0.11) 0.73 (0.13)   
a

New response rates signify false alarms for studied items and correct rejections for unstudied items.

fMRI Results

There were insufficient trials to allow encoding-related activity to be estimated separately for items later accorded remember or know judgments. Thus, the analyses reported below were based on study activity collapsed over the remember/know distinction. A subsidiary analyses conducted on 17 subjects who contributed a least 10 remember judgments for all 4 classes of study item (studied words and pictures tested with congruent and incongruent test cues) revealed a qualitatively similar pattern of findings, albeit at lower statistical thresholds than those reported below.

Two principal analyses were conducted to identify the encoding activity associated with study–test congruency effects. The first of these analyses was directed at the identification of congruency effects that were selective for either studied words or studied pictures. The second analysis identified effects that were material independent.

Material-Selective Congruency Effects

For the reasons outlined in the Introduction, we focused on the identification of material-specific cue congruency effects that overlapped with regions exhibiting material-selective activity. For both words and pictures, this was achieved by computing material-specific congruency effects (i.e., congruent word > incongruent word and congruent picture > incongruent picture) and then inclusively masking these effects with the outcome of the relevant material-selective contrast. The material-specific congruency effects were thresholded at P < 0.01 (cluster extent threshold > 4), and the masking contrast was thresholded at P < 0.005. To eliminate voxels where a congruency effect was also evident for the alternate study material, and therefore was not material specific, the outcome of the inclusive masking procedure was exclusively masked by the alternate material-specific congruency effect, thresholded at P < 0.05. The regions identified by these analyses are listed in Table 3 and illustrated in Figure 1. Word-specific congruency effects were obtained in the left superior temporal gyrus and the right inferior parietal cortex. In the case of pictures, effects were evident in right superior parietal and right occipitotemporal cortices.

Figure 1.

Overlap between cue congruency effects (P < 0.01) and the main effect of material (P < 0.005). (a, b) Overlap between word congruency effects (blue) and regions where activity was greater for words than for pictures (yellow). (c, d) Overlap between picture congruency effects (red) and regions where there was greater activity for pictures than for words (cyan). Graphs show mean parameter estimates (arbitrary units) for the congruency effects at the peak voxel in each region. Bars indicate standard errors.

Figure 1.

Overlap between cue congruency effects (P < 0.01) and the main effect of material (P < 0.005). (a, b) Overlap between word congruency effects (blue) and regions where activity was greater for words than for pictures (yellow). (c, d) Overlap between picture congruency effects (red) and regions where there was greater activity for pictures than for words (cyan). Graphs show mean parameter estimates (arbitrary units) for the congruency effects at the peak voxel in each region. Bars indicate standard errors.

Table 3

Regions showing material-selective cue congruency effects

Coordinates Z No. of voxels Region BA 
x y z 
Word 
−66 −39 3.36 [3.21] (22) L superior temporal gyrus 22 
60 −24 21 3.21 [3.20] (9) R inferior parietal cortex 40 
Picture 
27 −63 42 3.40 [2.93] (41) R superior parietal cortex 
39 −81 21 3.16 [5.17] (10) R middle occipital/temporal gyrus 19/39 
Coordinates Z No. of voxels Region BA 
x y z 
Word 
−66 −39 3.36 [3.21] (22) L superior temporal gyrus 22 
60 −24 21 3.21 [3.20] (9) R inferior parietal cortex 40 
Picture 
27 −63 42 3.40 [2.93] (41) R superior parietal cortex 
39 −81 21 3.16 [5.17] (10) R middle occipital/temporal gyrus 19/39 

Note: Coordinates and Z values refer to the peak voxels of clusters demonstrating cue congruency effects. Z values within square brackets refer to the significance of the relevant masking (material selective) contrast at the same voxel. L, left; R, right; BA, Brodmann area (approximate).

We also sought to identify regions where material-selective encoding-related activity was greater for items later recognized in response to incongruent cues. Employing the same rationale that motivated the search for cue congruency effects described above, we inclusively masked the relevant study material contrasts (thresholded at P < 0.005) with the “reversed” material-selective contrast (P < 0.01). No suprathreshold clusters were identified for either class of study material.

Material-Independent Cue Congruency Effects

Material-independent effects were identified by computing the main effect of cue congruency (P < 0.001, cluster extent > 4) and inclusively masking this contrast with the outcome of the 2 simple effects (each thresholded at P < 0.05). The surviving voxels were then exclusively masked with the bidirectional F contrast for the interaction between material and congruency (thresholded at P < 0.1). Thus, this procedure identified regions where activity was greater for items later recognized with congruent rather than incongruent cues and where, in addition, the congruency effects for each class of study material were independently significant and did not differ significantly in their magnitudes. The outcome of this procedure is detailed in Table 4 and illustrated in Figure 2. Among the regions identified were the left middle frontal gyrus, left anterior intraparietal sulcus, and right occipitotemporal and superior parietal cortex. The reverse contrast, seeking regions where items later recognized with incongruent cues elicited greater activity than those recognized in response to congruent cues, failed to identify any suprathreshold voxels.

Figure 2.

Regions demonstrating material-independent cue congruency effects (P < 0.001). Graphs show mean parameter estimates of the effects at the peak voxel in each region. Bars indicate standard errors.

Figure 2.

Regions demonstrating material-independent cue congruency effects (P < 0.001). Graphs show mean parameter estimates of the effects at the peak voxel in each region. Bars indicate standard errors.

Table 4

Regions showing material-independent cue congruency effects

Coordinates Z No. of voxels Region BA 
x y z    
−30 33 27 3.51 (6) L middle frontal gyrus 
45 −72 24 3.90 (16) R middle temporal gyrus 39 
−27 −42 45 3.78 (30) L superior parietal cortex 
24 −66 48 3.71 (11) R superior parietal cortex 
Coordinates Z No. of voxels Region BA 
x y z    
−30 33 27 3.51 (6) L middle frontal gyrus 
45 −72 24 3.90 (16) R middle temporal gyrus 39 
−27 −42 45 3.78 (30) L superior parietal cortex 
24 −66 48 3.71 (11) R superior parietal cortex 

Note: Coordinates and Z values refer to the peak voxels of each cluster. L, left; R, right; BA, Brodmann area (approximate).

Discussion

Behavioral Data

Study RTs did not differ according to whether an item was later recognized in response to a congruent or an incongruent retrieval cue. Thus, the fMRI cue congruency effects discussed below are not secondary to gross differences in the efficiency with which congruently and incongruently cued items were processed at study. Performance on the subsequent recognition test was higher for pictures than for words and for congruent compared with incongruent cues. The first of these findings is an example of the well-known picture superiority effect (Paivio 1971, 1986), whereas the second finding—the focus of the present study—reflects the expected benefit of cue congruency. Because we assessed recognition memory with the remember/know procedure, we were able to investigate material and congruency effects separately for recollection- and familiarity-driven judgments. Whereas study material and cue congruency had substantial, additive effects on estimates of recollection, their influence on familiarity estimates was in both cases nonsignificant. As is discussed in more detail below, these findings bear significantly on the fMRI findings, in that they argue against an interpretation of fMRI cue congruency effects in terms of encoding-related activity that supports familiarity rather than recollection.

fMRI Findings

Robust differences in encoding-related activity were evident for subsequently recognized study items according to whether the items were retrieved in response to congruent or incongruent cues. These fMRI cue congruency effects were comprised of both material-specific and material-independent components. As predicted, material-specific effects overlapped with regions that responded selectively to the corresponding class of study item (i.e., words or pictures).

Before discussing the implications of these findings further, 2 general points are worthy of mention. The first point concerns the contrasts employed to identify fMRI cue congruency effects. These contrasts are different from those that are employed to identify subsequent memory effects when study items are contrasted according to subsequent performance on a common memory test (e.g., hits vs. misses or successful vs. unsuccessful source memory). In the present study, the crucial contrasts were between study items that were successfully remembered under 2 different retrieval conditions. The contrasts were motivated by the hypothesis that in order for an item to benefit from a congruent retrieval cue, its study processing must overlap with the processing accorded the cue, whereas the same study processing will have much less impact if the item is cued with an incongruent cue. This approach is appropriate for identifying encoding-related activity that is more important when memory is later tested with one type of retrieval cue rather than another. It cannot, however, identify encoding activity that supports subsequent memory independently of how it is cued.

The second point has already been alluded to. The effects of cue congruency were selective for recognition judgments associated with recollection. Thus, the benefit of a congruent retrieval cue was confined to items whose study processing supported later recollection and did not extend to processing facilitating familiarity-driven recognition. Therefore, although limitations on trial numbers precluded separate analysis of the neural activity elicited by study items later accorded remember or know judgments, it can be concluded that fMRI cue congruency effects are linked to encoding processes that support recollection rather than familiarity.

Material-Selective Cue Congruency Effects

Both words and pictures demonstrated cue congruency effects that overlapped with the outcome of contrasts between the 2 classes of study material. In the case of words, overlap was observed between word-specific congruency effects and word-selective activity in left superior temporal gyrus and right inferior parietal cortex. For pictures, congruency effects and picture-selective activity overlapped in right occipitotemporal and superior parietal cortices. We assume that the material-selective effects reflect neural activity supporting processes engaged in service of word and picture processing and that these processes would have also been engaged when words and pictures were employed as retrieval cues. Given this assumption, the present findings are consistent with the hypothesis that retrieval cues are most effective when their processing overlaps or recapitulates processing engaged during study. Thus, the findings offer neural support for a central tenet of the TAP and encoding specificity frameworks (see Introduction).

The loci of the overlap between cue congruency and material-selective effects provide some insight into the nature of the study processing that supported later retrieval in response to congruent cues. Notably, the region of the left superior temporal gyrus demonstrating overlap between word-specific congruency effects and word-selective material effects has been implicated in phonological processing of visual words in a number of prior studies (e.g., Burton et al. 2005; for review, see Joseph et al. 2001). This suggests that study–test overlap at the phonological level played an important role in mediating cue congruency effects for words, such that items were more likely to benefit from congruent cues if their phonological attributes received emphasis during study. The occipitotemporal region identified in the analysis of picture congruency effects, which is somewhat dorsal to the much-studied lateral occipital complex (Grill-Spector et al. 2001), has been reported to play a role in object recognition in numerous studies (e.g., Ishai et al. 2000; Sevostianov et al. 2002), especially when processing of structural features is emphasized (Kellenbach et al. 2003). Along with the right superior parietal region also identified by this analysis, the occipitotemporal region demonstrating picture-specific effects has also been implicated in processing of spatial features of visual objects (e.g., Haxby et al. 1994; Faillenot et al. 1999). It seems likely therefore that picture congruency effects were enhanced when low-level structural features of the study items received emphasis during encoding.

Material-Independent Effects

In addition to the material-selective cue congruency effects discussed above, effects common to both classes of study item were also evident. Two of the regions exhibiting these effects—left anterior intraparietal sulcus and left middle frontal gyrus—have been implicated in top–down attentional control (e.g., Hopfinger et al. 2000; Luks and Simpson 2004; Hahn et al. 2006) and, in the case of the intraparietal sulcus, in attentionally mediated perceptual binding (Humphreys 1998; Shafritz et al. 2002; Cusack 2005). In light of these prior results, the present findings suggest that engagement of attentional control processes increases the likelihood that a study item will later benefit from a congruent retrieval cue. We hypothesize these attentional processes support the binding of semantic and nonsemantic (surface) attributes of study items into a coherent perceptual, and hence mnemonic, representation (see also Uncapher et al. 2006). Such representations would be especially accessible to congruent cues because of the opportunity afforded for overlap between cue processing and the encoded memory representation at both semantic and nonsemantic representational levels. For study items that were later tested with incongruent cues, by contrast, whether or not the encoded memory representations included surface information was irrelevant; with an incongruent cue, no potential existed for overlap between memory representations and retrieval processing other than at the semantic level. Thus, for incongruently cued items, variability in the extent to which their surface and semantic attributes were bound together during study processing would have had little impact on later memory.

In addition to regions likely supporting attentional control, material-independent cue congruency effects were also identified in right posterior middle temporal gyrus and right superior parietal cortex. Both of these regions abut right posterior regions demonstrating picture-selective congruency effects, raising the concern that the apparent dissociations between material-selective and material-independent effects in these areas are more quantitative than qualitative. This concern is, however, ameliorated by the outcome of ANOVAs contrasting word and picture congruency effects according to their loci. The ANOVAs revealed that, in both occipitotemporal and superior parietal cortex, the magnitude of the difference between word and picture congruency effects was significantly greater at the peak voxel exhibiting the picture-selective effect than at the voxel exhibiting the material-independent effect (voxel × material interactions F1,22 = 3.73, P < 0.05 [one tailed], and F1,22 = 9.18, P < 0.001 [one tailed] for occipitotemporal and parietal regions, respectively). Thus, in both cortical regions, there is direct statistical evidence to support a dissociation between voxels demonstrating picture-selective and material-independent congruency effects. As with the selective effects already discussed, the material-independent effects in these regions may reflect the benefit to later memory of study processing that emphasizes surface as well as semantic attributes. In the case of the areas demonstrating material-independent effects, the processes they support are presumably of benefit to words and pictures alike.

Reversed Congruency Effects

We were unable to find evidence for either material-selective or material-independent “reversed” cue congruency effects, that is, regions demonstrating greater activity for study items later recognized when cued with incongruent versus congruent retrieval cues. Whereas null findings such as these must be interpreted with caution, they suggest that the successful encoding of incongruently cued items did not engage processing additional to that supporting items retrieved in response to congruent cues. One possibility is that, regardless of how it was later cued, memory benefited to the extent that semantic attributes of a study item received emphasis during encoding. By contrast, as noted already, encoding the surface attributes of a study item will only be beneficial when the subsequent retrieval cue shares those attributes, that is, when it is congruent. This line of reasoning leads to the prediction that it should be possible to devise study tasks in which reversed congruency effects are evident. For example, with words as study items, a task that required subjects to generate an image of the object denoted by each word might be expected to modulate overlap between study processing and retrieval processing even when the retrieval cues were pictorial, paving the way for the emergence of reversed congruency effects.

It was noted by a referee that the analytic approach adopted in the present study focused on identification of regions where cue congruency effects on behavior were associated with greater encoding-related activity for congruently than incongruently cued items. The approach was motivated by the widely held assumption that the magnitude of the neural activity in a functionally specialized cortical region covaries with the level of engagement of the computations supported by the region. Thus, our prediction that items will benefit from congruent retrieval cues to the extent that the processing receiving emphasis at study overlaps the processing engaged by the retrieval cue reduces to a prediction about overlapping patterns of activation. This is not to say that there may not also be regions where a relative decrease in study activity is predictive of a cue congruency effect on behavior (cf., Otten and Rugg 2001b; Otten forthcoming). The present experimental design is not well suited to the identification of such regions, however.

Concluding Comments

The present study provides a clear example of the intimacy of the relationship between processes operating at encoding and retrieval. Specifically, the findings suggest that to benefit from a congruent retrieval cue, a study item must engage processing that facilitates the binding of its surface and semantic attributes into a common memory representation. Thus, the relative ease with which retrieval is accomplished in response to congruent relative to incongruent retrieval cues reflects a mnemonic benefit that accrues only when study and retrieval processing are aligned.

Funding

National Institute of Mental Health (MH074528).

The authors thank the members of the University of California-Irvine Research Imaging Center for their assistance with fMRI data acquisition. Conflict of Interest : None declared.

References

Ashburner
J
Friston
K
Nonlinear spatial normalization using basis functions
Hum Brain Mapp
 , 
1999
, vol. 
7
 (pg. 
254
-
266
)
Burton
MW
LoCasto
PC
Krebs-Noble
D
Gullapalli
RP
A systematic investigation of the functional neuroanatomy of auditory and visual phonological processing
Neuroimage
 , 
2005
, vol. 
26
 (pg. 
647
-
661
)
Cocosco
CA
Kollokian
V
Kwan
RS
Evans
AC
Brainweb: online interface to a 3D MRI simulated brain database
Neuroimage
 , 
1997
, vol. 
5
 pg. 
S425
 
Cusack
R
The intraparietal sulcus and perceptual organization
J Cogn Neurosci
 , 
2005
, vol. 
17
 (pg. 
641
-
651
)
Faillenot
I
Decety
J
Jeannerod
M
Human brain activity related to the perception of spatial features of objects
Neuroimage
 , 
1999
, vol. 
10
 (pg. 
114
-
124
)
Fisher
RA
Statistical methods for research workers
1950
11th ed
London
Oliver and Boyd
Friston
KJ
Fletcher
P
Josephs
O
Holmes
A
Rugg
MD
Turner
R
Event-related fMRI: characterizing differential responses
Neuroimage
 , 
1998
, vol. 
7
 (pg. 
30
-
40
)
Grill-Spector
K
Kourtzi
Z
Kanwisher
N
The lateral occipital complex and its role in object recognition
Vision Res
 , 
2001
, vol. 
41
 (pg. 
1409
-
1422
)
Hahn
B
Ross
TJ
Stein
EA
Neuroanatomical dissociation between bottom-up and top-down processes of visuospatial selective attention
Neuroimage
 , 
2006
, vol. 
32
 (pg. 
842
-
853
)
Harvey
P
Fossati
P
Lepage
M
Modulation of memory formation by stimulus content: specific role of the medial prefrontal cortex in the successful encoding of social pictures
J Cogn Neurosci
 , 
2007
, vol. 
19
 (pg. 
351
-
362
)
Haxby
JV
Horwitz
B
Ungerleider
LG
Maisog
JM
Pietrini
P
Grady
CL
The functional organization of human extrastriate cortex: a PET-rCBF study of selective attention to faces and locations
J Neurosci
 , 
1994
, vol. 
14
 (pg. 
6336
-
6353
)
Henson
RNA
Rugg
MD
Shallice
T
Josephs
O
Dolan
RJ
Recollection and familiarity in recognition memory: an event-related fMRI study
J Neurosci
 , 
1999
, vol. 
19
 (pg. 
3962
-
3972
)
Hopfinger
JB
Buonocore
MH
Mangun
GR
The neural mechanisms of top-down attentional control
Nat Neurosci
 , 
2000
, vol. 
3
 (pg. 
284
-
291
)
Hornberger
M
Morcom
AM
Rugg
MD
Neural correlates of retrieval orientation: effects of study-test similarity
J Cogn Neurosci
 , 
2004
, vol. 
16
 (pg. 
1196
-
1210
)
Humphreys
GW
Neural representation of objects in space: a dual coding account
Philos Trans R Soc Lond B Biol Sci
 , 
1998
, vol. 
353
 (pg. 
1341
-
1351
)
Ishai
A
Ungerleider
LG
Matin
A
Haxby
JV
The representation of objects in the human occipital and temporal cortex
J Cogn Neurosci
 , 
2000
, vol. 
12
 
Suppl 2
(pg. 
35
-
51
)
Joseph
JE
Noble
K
Eden
GF
The neurobiological basis of reading
J Learn Disabil
 , 
2001
, vol. 
34
 (pg. 
566
-
579
)
Kellenbach
ML
Hovius
M
Patterson
K
A PET study of visual and semantic knowledge about objects
Cortex
 , 
2003
, vol. 
41
 (pg. 
121
-
132
)
Kirchhoff
BA
Wagner
AD
Maril
A
Stern
CE
Prefrontal-temporal circuitry for episodic encoding and subsequent memory
J Neurosci
 , 
2000
, vol. 
20
 (pg. 
6173
-
6180
)
Lazar
NA
Luna
B
Sweeney
JA
Eddy
W
Combining brains: a survey of methods for statistical pooling of information
Neuroimage
 , 
2002
, vol. 
16
 (pg. 
538
-
550
)
Luks
TL
Simpson
GV
Preparatory deployment of attention to motion activates higher-order motion-processing brain regions
Neuroimage
 , 
2004
, vol. 
22
 (pg. 
1515
-
1522
)
Mitchell
JP
Macrae
CN
Banaji
MR
Encoding-specific effects of social cognition on the neural correlates of subsequent memory
J Neurosci
 , 
2004
, vol. 
24
 (pg. 
4912
-
4917
)
Morris
CD
Bransford
JD
Franks
JJ
Levels of processing versus transfer appropriate processing
J Verb Learn Verb Behav
 , 
1977
, vol. 
16
 (pg. 
519
-
533
)
Nairne
JS
The myth of the encoding-retrieval match
Memory
 , 
2002
, vol. 
10
 (pg. 
389
-
395
)
Otten
LJ
Forthcoming. Fragments of a larger whole: retrieval cues constrain observed neural correlates of memory encoding
Cereb Cortex
  
Advance Access published Nov 6, 2006; doi:10.1093/molbev/bhl111
Otten
LJ
Henson
RNA
Rugg
MD
State-related and item-related neural correlates of successful memory encoding
Nat Neurosci
 , 
2002
, vol. 
5
 (pg. 
1339
-
1344
)
Otten
LJ
Rugg
MD
Task-dependency of the neural correlates of episodic encoding as measured by fMRI
Cereb Cortex
 , 
2001
, vol. 
11
 (pg. 
1150
-
1160
)
Otten
LJ
Rugg
MD
When more means less: neural activity related to unsuccessful memory encoding
Curr Biol
 , 
2001
, vol. 
11
 (pg. 
1528
-
1530
)
Paivio
A
Imagery and verbal processes
1971
Rinehart & Winston
New York: Holt
Paivio
A
Mental representation: a dual coding approach
1986
New York
Oxford University Press
Paller
KA
Wagner
AD
Observing the transformation of experience into memory
Trends Cogn Sci
 , 
2002
, vol. 
6
 (pg. 
93
-
102
)
Roediger
HL
Guynn
MJ
Bjork
EL
Bjork
RA
Retrieval process
Memory
 , 
1996
San Diego (CA)
Academic Press
(pg. 
197
-
236
)
Roediger
HL
Weldon
MS
Challis
B
Roediger
FIM
Craik
H
Explaining dissociations between implicit and explicit measures of retention: a processing account
Varieties of memory and consciousness: essays in honor of Endel Tulving
 , 
1989
Hillsdale (NJ)
Erlbaum
(pg. 
3
-
41
)
Schloerscheidt
AM
Rugg
MD
The impact of change in stimulus format on the electrophysiological indices of recognition
Neuropsychologia
 , 
2004
, vol. 
42
 (pg. 
451
-
466
)
Sevostianov
A
Horwitz
B
Nechaev
V
Williams
R
Fromm
S
Braun
AR
fMRI study comparing names versus pictures of objects
Hum Brain Mapp
 , 
2002
, vol. 
16
 (pg. 
168
-
175
)
Shafritz
KM
Gore
JC
Marois
R
The role of the parietal cortex in visual feature binding
Proc Natl Acad Sci USA
 , 
2002
, vol. 
99
 (pg. 
10917
-
10922
)
Tulving
E
Memory and consciousness
Can Psychol
 , 
1985
, vol. 
26
 (pg. 
1
-
12
)
Tulving
E
Thomson
DM
Encoding specificity and retrieval processes in episodic memory
Psychol Rev
 , 
1973
, vol. 
80
 (pg. 
359
-
380
)
Uncapher
MR
Otten
LJ
Rugg
MD
Episodic encoding is more than the sum of its parts: an fMRI investigation of multifeatural contextual encoding
Neuron
 , 
2006
, vol. 
52
 (pg. 
547
-
556
)
Van Essen
DC
A population-average, landmark- and surface-based (PALS) atlas of human cerebral cortex
Neuroimage
 , 
2005
, vol. 
28
 (pg. 
635
-
662
)
Van Essen
DC
Dickson
J
Harwell
J
Hanlon
D
Anderson
CH
Drury
HA
An integrated software system for surface-based analyses of cerebral cortex
J Am Med Inf Assoc
 , 
2001
, vol. 
8
 (pg. 
443
-
459
)
Wagner
AD
Schacter
DL
Rotte
M
Koutstaal
W
Maril
A
Dale
AM
Rosen
B
Buckner
RL
Building memories: remembering and forgetting of verbal experiences as predicted by brain activity
Science
 , 
1998
, vol. 
281
 (pg. 
1188
-
1191
)
Yonelinas
AP
Jacoby
LL
The relation between remembering and knowing as bases for recognition: effects of size congruency
J Mem Lang
 , 
1995
, vol. 
34
 (pg. 
622
-
643
)