## Abstract

Neuroimaging and lesion studies have implicated specific prefrontal cortex locations in subjective memory awareness. Based on this evidence, a rostrocaudal organization has been proposed whereby increasingly anterior prefrontal regions are increasingly involved in memory awareness. We used theta-burst transcranial magnetic stimulation (TBS) to temporarily modulate dorsolateral versus frontopolar prefrontal cortex to test for distinct causal roles in memory awareness. In three sessions, participants received TBS bilaterally to frontopolar cortex, dorsolateral prefrontal cortex, or a control location prior to performing an associative-recognition task involving judgments of memory awareness. Objective memory performance (i.e., accuracy) did not differ based on stimulation location. In contrast, frontopolar stimulation significantly influenced several measures of memory awareness. During study, judgments of learning were more accurate such that lower ratings were given to items that were subsequently forgotten selectively following frontopolar TBS. Confidence ratings during test were also higher for correct trials following frontopolar TBS. Finally, trial-by-trial correspondence between overt performance and subjective awareness during study demonstrated a linear increase across control, dorsolateral, and frontopolar TBS locations, supporting a rostrocaudal hierarchy of prefrontal contributions to memory awareness. These findings indicate that frontopolar cortex contributes causally to memory awareness, which was improved selectively by anatomically targeted TBS.

## Introduction

Monitoring of memory performance is often essential for the effective use of memory (Dunlosky and Bjork 2008; Fleming and Dolan 2012). For instance, the degree to which students accurately evaluate memory during learning affects the time allotted to study specific information, choices regarding when to terminate study, and subsequent memory performance (Metcalfe 2013; Fleming and Frith 2014). Lack of awareness of suboptimal learning and retrieval is maladaptive (Nelson and Narens 1980; Chua et al. 2014) and could hinder timely intervention in cases of memory impairment (e.g., Cosentino 2014).

Despite a crucial role for awareness in learning and memory, little is known regarding neural mechanisms. Lesion-deficit studies suggest that frontopolar prefrontal cortex damage is particularly disruptive for organized behaviors involving memory monitoring and manipulation of memory representations (Christoff and Gabrielli 2000; Shimamura 2002; Dreher et al. 2008; Burgess et al. 2010). Indeed, frontopolar cortex is the apex of rostrocaudal hierarchically organized prefrontal cortex (e.g., Fuster 1997; Koechlin and Summerfield 2007; Badre 2008). The relatively high convergence of posterior cortex onto fewer frontopolar regions suggests that frontopolar cortex may be ideally situated for monitoring memory and perhaps other types of information (Miller and Cohen 2001; Ramnani and Owen 2004; Simons et al. 2008; Passingham and Wise 2012).

The hypothesized role of frontopolar cortex in memory monitoring is consistent with evidence linking this region to integration of short-term representations over time, especially during demanding tasks (e.g., Koechlin et al. 1999; Christoff et al. 2001; Braver and Bongiolatti 2002; Badre and Wagner 2004). In working-memory and reasoning tasks that require integration, frontopolar cortex may be crucial for monitoring response conflicts by differentiating relevant from competing (i.e., interfering) representations. For instance, Badre and Wagner (2004) identified increased frontopolar cortex activity during a working-memory task requiring information integration over time, as well as frontopolar involvement in monitoring response conflict via functional coupling with anterior cingulate cortex. In some instances, such as in working memory, frontopolar-mediated integration may be adaptive by reducing overdependence on external stimuli in service of a superordinate contextual goal. However, in other types of memory tasks, such as paired-associate recognition, integration of associative information over multiple trials may be detrimental to memory by introducing interference that builds over time.

Previous work suggests that frontopolar activity and gray-matter volume are associated with awareness more so than with performance accuracy in memory tasks (Chua et al. 2014) and in visual perceptual tasks (Fleming et al. 2010). This role of frontopolar cortex in awareness has been supported by several lesion-deficit experiments that find selective awareness deficits in patients with rostral prefrontal lesions (Burgess et al. 2010). However, the permanency of brain lesions, lack of control over damage location, and possible compensatory responses after brain damage complicate inferences regarding frontopolar cortex and awareness.

Theta-burst transcranial magnetic stimulation (TBS) has been used to modulate prefrontal activity in healthy individuals. Continuously applied TBS can temporarily inhibit cortical function for a period outlasting stimulation (e.g., Dayan et al. 2013), although increased functional connectivity of stimulated regions has also been observed (Gratton et al. 2013). Prefrontal contributions to awareness have been primarily tested with TBS in perceptual rather than in memory tasks (e.g., Fleming et al. 2010; Rounis et al. 2010; Fleming et al. 2012). In one of the few studies of TBS in memory, Lee et al. (2013) found that dorsolateral TBS improved accuracy of memory responses made without awareness (cf. Voss and Paller 2009). However, Blumenfeld et al. (2014) recently found that ventrolateral TBS during word encoding impaired subsequent item recognition memory, whereas dorsolateral TBS marginally increased memory performance relative to a control location. Previous studies have not tested distinct roles of frontopolar versus other prefrontal regions in memory awareness. We therefore used TBS to test hypothesized roles of frontopolar and dorsolateral prefrontal cortex during a paired-associate memory task incorporating four measures of memory awareness.

## Materials and Methods

### Participants

Twenty-one participants were recruited from the Chicago metropolitan area. Data from three participants were discarded (due to failure to complete one or more of the experimental conditions). This resulted in a final sample of 18 individuals (11 females; ages 21–47 years; mean age = 28.22 years). All participants were right handed, had normal or corrected-to-normal vision, did not report neurological or psychiatric disorders, and did not report the current use of psychoactive drugs. All participants gave written, informed consent and were remunerated for their participation. All participants were eligible for MRI and TMS procedures based on standard MRI safety screening as well as on their answers to a TMS safety-screening questionnaire (Rossi et al. 2009). The Institutional Review Board at Northwestern University approved the study protocol.

### Experiment Design

After acquiring a structural MRI scan, each participant completed three experiment sessions each separated by at least 1 day (range = 1–7 days, mean = 2.22 days), with different bilateral TBS targets used each session (frontopolar, dorsolateral, or control). After a delay of 5–20 min following bilateral TBS, each individual completed an associative memory task during which memory awareness was measured (see below). Six participants performed the experiment after a 5-min delay during which experiment instructions were given. The remaining 12 participants performed the experiment after a 20-min delay during which a different memory task involving different stimuli and task demands was performed (not reported here). No differences between these two delays were identified when entered as a factor in analyses, and the same pattern of findings was observed within each delay group when analyzed separately. Findings are thus reported collapsed across delay duration. The order of the stimulation locations across days was randomized across participants, with approximately the same number of participants receiving each stimulation condition for each of the three sessions (seven received frontopolar stimulation for session 1, five received dorsolateral stimulation for session 1, and six received control stimulation for session 1).

### MRI Parameters

MRI data were collected at the Northwestern University Center for Translational Imaging, supported by the Northwestern University Department of Radiology. A Siemens 3T TIM Trio whole-body scanner with a 32-channel head coil was used. Head movement was minimized with padding. A structural image was acquired to provide anatomical localization (MPRAGE T1-weighted scans, TR = 2400 ms, TE = 3.16 ms, voxel size = 1 mm3, FOV = 25.6 cm, flip angle = 8°, 176 sagittal slices). Structural MRI data were processed and stimulation targets were marked using AFNI (Cox 1996).

### Identification of Stimulation Locations and TBS Parameters

We identified bilateral stimulation locations for each participant (Fig. 1). The structural MRI for each individual was first transformed into stereotactic space using the MNI-305 template (Evans et al. 1993). The transformation matrix was stored to enable conversion between original MRI space and stereotactic space. After transformation, bilateral targets were identified and marked for frontopolar cortex (Brodmann area 10; MNI x = ±29, y = 66, z = 10), for dorsolateral prefrontal cortex (Brodmann area 46; MNI x = ±52, y = 15, z = 29), and for the control location (central fissure adjacent to paracentral lobule, MNI = ±4, −42, 73). The frontopolar and dorsolateral prefrontal locations were selected based on a balance of three factors: (1) location within Brodmann areas of interest (BA10 for frontopolar, BA46 for dorsolateral), (2) maximum absolute distance between frontopolar and dorsolateral locations, in order to reduce cross-contamination of stimulation effects, and (3) review of previous literature regarding key dorsolateral and frontopolar regions involved in recognition memory and memory awareness (e.g., Cabeza et al. 2003; Osaka et al. 2003; Fernandes et al. 2005; Burgess et al. 2007, 2010; Rosner et al. 2013). The dorsolateral locations were separated from the frontopolar locations by the greatest possible distance within the corresponding Brodmann's areas, and both sets of locations were activation maxima in the recognition memory fMRI studies listed above. The vertex control location was determined based on the approximate coordinates of Cz electrode from the international 10–20 EEG positioning system. Targets corresponding to stimulation locations were marked in MNI-305 space and then transformed into original MRI space. We then overlaid the targets onto the structural MRI to provide localization during TBS (which requires a structural MRI in original space). Thus, TBS was delivered to the same MNI coordinates in every individual despite individual differences in anatomy.

Figure 1.

Bilateral TBS locations. Locations are marked in MNI space on a model brain (ICBM-452; Rex et al. 2003). Frontopolar targets were x = ±29, y = +66, z = +10 mm. Dorsolateral targets were x = ±52, y = +15, z = +29 mm. Control (paracentral lobule) targets were x = ±4, y = −42, z = +73 mm.

Figure 1.

Bilateral TBS locations. Locations are marked in MNI space on a model brain (ICBM-452; Rex et al. 2003). Frontopolar targets were x = ±29, y = +66, z = +10 mm. Dorsolateral targets were x = ±52, y = +15, z = +29 mm. Control (paracentral lobule) targets were x = ±4, y = −42, z = +73 mm.

TBS was applied to the stimulation locations using a Nexstim eXimia NBS 4.3 air-cooled, MRI-guided system with a 70-mm figure-of-eight stimulation coil (Nexstim Ltd., Helsinki, Finland). MRI guidance was achieved using frameless stereotaxy, thus allowing for high accuracy in coil positioning and real-time monitoring of movement in order to adjust coil placement accordingly.

Resting motor threshold was determined during the first session as the minimum stimulation value necessary to generate contraction of the right abductor pollicis brevis muscle for at least 60% of pulses (as measured via electromyography using a contraction threshold of 50 mV). For the treatment conditions, TBS was applied at 80% motor threshold to the predefined locations. TBS was continuous, consisting of a series of three 50 Hz bursts separated by 160 ms (i.e., 50 Hz bursts at an ∼6 Hz carrier frequency). As TBS is applied to increasingly anterior prefrontal regions, it may become uncomfortable or even painful. Therefore, prior to TBS, a test pulse was delivered and participants were allowed to discontinue stimulation if they found the pulses aversive. During stimulation, participants were free to pull their heads away from the coil or to indicate their desire to stop the experiment at any time (the three excluded participants withdrew from at least one TBS condition). After indicating a willingness to proceed, TBS was administered continuously for 40 s to each bilateral location, first to the right and then to the left (600 pulses total on each side). Stimulation locations were targeted via MRI using a frameless infrared stereotactic system. The induced current field was oriented perpendicular/anterior to the long axis of the gyrus encompassing the stimulation location. The estimated induced voltage (V/m) at the stimulation location was calculated online based on a realistic head model (Ruohonen and Ilmoniemi 2002) and recorded for each session. The estimated intensity did not differ significantly for any two stimulation targets (pairwise P values > 0.12).

### Memory Testing

Objective memory performance and memory awareness were assessed using a novel fractal-object associative-recognition memory task (Fig. 2), administered using Presentation software (Neurobehavioral Systems). For each session, the task comprised two study-test blocks following bilateral TBS to one of the three stimulation locations, with different stimuli used for each session and block. For each of the two study periods, each of 15 unique color fractal images (used with permission from Harrington 2011) was paired with a unique color object (chosen from the BOSS database, Brodeur et al. 2010). The pairing of fractals to objects was randomized for each participant. Each fractal-object pair was presented centrally for 8 s, after which participants were prompted to make a judgment of learning rating (JOL) based on the likelihood of remembering the associated items on a later memory test. As is standard with most studies incorporating JOL ratings, participants were explicitly informed that their memory would later be tested. JOLs were self-paced and were made using a 1–9 Likert scale indicating low-to-high confidence, respectively (Nelson and Dunlosky 1991).

Figure 2.

Associative-recognition testing with memory awareness measures. For each study period, participants viewed 15 unique fractal-object pairs for 8 s each and provided JOLs after each pair to indicate likelihood that the studied pair would later be remembered. After study and before test, participants made global predictions of their upcoming test performance. The test included 15 associative-recognition trials. On each trial, participants were shown a studied (old) fractal and attempted to select the one object that was paired with the fractal at study of 20 objects (15 studied/old and 5 unstudied/new). Immediately after each selection, participants made RCJs to indicate likelihood that the selection was correct. After the test, participants provided global estimates (postdictions) of performance on the immediately preceding test. There were two study-test blocks for each of the three experiment sessions (with different TBS locations for each session), with different stimuli used for each block.

Figure 2.

Associative-recognition testing with memory awareness measures. For each study period, participants viewed 15 unique fractal-object pairs for 8 s each and provided JOLs after each pair to indicate likelihood that the studied pair would later be remembered. After study and before test, participants made global predictions of their upcoming test performance. The test included 15 associative-recognition trials. On each trial, participants were shown a studied (old) fractal and attempted to select the one object that was paired with the fractal at study of 20 objects (15 studied/old and 5 unstudied/new). Immediately after each selection, participants made RCJs to indicate likelihood that the selection was correct. After the test, participants provided global estimates (postdictions) of performance on the immediately preceding test. There were two study-test blocks for each of the three experiment sessions (with different TBS locations for each session), with different stimuli used for each block.

After each study period, participants were given a 60 s break prior to receiving test instructions and performing a corresponding memory test. For each test trial, participants viewed a fractal image in the upper central portion of the screen. The test was an associative-recognition memory test, such that below each studied fractal 20 color objects were presented arranged in a 4 × 5 grid (Fig. 2). Of the 20 objects, 15 were old (shown paired with fractals during study) and 5 were new (not presented previously). Participants used a mouse to select the object that they believed was previously paired with the fractal at the top of the screen. After selecting an object, participants were prompted to provide a retrospective confidence judgment (RCJ) based on whether or not they believed that they had chosen the correct object (Cosentino et al. 2011). RCJs were made using a 1–9 Likert scale indicating low-to-high confidence, as for JOLs. This procedure was repeated for two study-test blocks (with different stimuli per block), yielding a total of 30 trials for each TBS session/condition.

In addition to the trial-by-trial JOL and RCJ awareness ratings described above, participants also made global memory awareness judgments at two points during each study-test block of the experiment. After each study block and before each corresponding test block, participants made global predictions of performance, indicating how many correct answers, of 15 possible, they believed that they would provide on that given test block. Similarly, after completing each test block, participants provided a global postdiction of performance by indicating how many correct answers they believed they had provided. Comparisons among mean values for the three stimulation conditions used repeated-measures ANOVAs.

## Results

### Recognition Memory Performance

We first assessed item recognition memory performance. TBS location had no significant influence on the rate of selection of old objects over new objects irrespective of whether the old item selected was the correct associate or another old object (F2,34 = 0.08, ns), with no significant pairwise differences among stimulation locations (all P values > 0.72). Item recognition memory was near ceiling levels for all stimulation locations [mean item hit rates = 0.95 (SE = 0.06) for control, 0.95 (SE = 0.06) for dorsolateral, and 0.96 (SE = 0.01) for frontopolar], with only 0–4 item-based false alarm errors made by any given participant (mean false alarms = 0.09 across all conditions and blocks). Thus, all analyses of memory awareness that follow concern associative recognition rather than item recognition, as there was no performance variability for item recognition to support analyses of corresponding awareness variability. JOL and RCJ confidence measures therefore corresponded only to correct or incorrect associative responses (i.e., hits or misses) according to the instructions given to participants.

Associative recognition accuracy was computed as the proportion of correct associative-recognition hits during the test phase. The chance rate of performance (correct guessing) was 0.05 given the format of the test and all participants performed substantially above chance for all three TBS conditions, which was significant at the group level [mean hit rates = 0.52 (SE = 0.05) for control, 0.53 (SE = 0.06) for dorsolateral, and 0.57 (SE = 0.07) for frontopolar; all P values < 0.001 vs. chance]. Mean performance levels also did not vary by stimulation location (F2,34 = 0.03, ns), and there were no significant pairwise differences between any two TBS locations (all P values > 0.86). Thus, TBS location had no significant effect on associative-recognition accuracy.

It is possible that chance levels could have differed from 0.05 given that the associative-recognition test format included item sampling without replacement. That is, whenever participants made a correct response and were confident in the response, the selected item would effectively be removed from the pool of possible options and the chance rate would increase slightly (i.e., 1/20 chance rate before any correct responses were made, 1/19 after one correct response was made, 1/18 after two correct responses were made, etc.). Furthermore, it is possible that TBS location could have influenced participants' abilities to track correct responses and therefore TBS location could potentially have influenced accuracy if calculated using a chance rate that accounted for each individual's performance. To evaluate this possibility, we recomputed accuracy controlling for inflated chance accuracy with each correct response (see Supplementary Materials for a description of the correction procedure and analyses of corrected values). Accuracy remained significantly above chance for all TBS locations following this correction. Further, TBS location had no significant effect on corrected accuracy values, thus replicating the lack of effects of TBS location on associative-recognition accuracy for non-corrected values.

### Memory Awareness During Study

JOLs made during study were analyzed separately for subsequent correct versus incorrect associative-recognition responses in order to identify effects of TBS location on memory awareness during study. Higher memory awareness accuracy would be indicated by a better correspondence between JOL rating values and subsequent associative memory performance (i.e., higher JOLs for subsequent correct responses and/or lower JOLs for subsequent incorrect responses). As displayed in Figure 3A, JOL ratings for later correct associative-recognition trials (hits) did not vary by TBS location (F2,34 = 0.30, ns). In contrast, JOL ratings did differ for later incorrect associative-recognition trials (misses) (F2,34 = 3.68, MSE = 0.72, P = 0.03, $ηp2$ = 0.18; Fig. 3B). Post hoc paired-samples t-tests indicated that this effect for misses was driven by significantly lower (i.e., more accurate) JOL ratings for frontopolar TBS versus control (t17 = 3.77, P = 0.002, Cohen's d = 0.95). Frontopolar JOL ratings were also marginally lower for misses than those given for dorsolateral TBS (t17 = 1.87, P = 0.08, Cohen's d = 0.52). No significant differences were observed between dorsolateral and control. Thus, frontopolar stimulation was selective in enhancing memory awareness by reducing JOL ratings for associations that were later forgotten during test.

Figure 3.

Effects of TBS on memory awareness at study and test. (A) Mean study JOL values for trials that were subsequently correct during test (later-hit). (B) Mean study JOL values for trials that were subsequently incorrect during test (later-miss). (C) Mean RCJ values for trials with correct responses (Hit). (D) Mean RCJ values for trials with incorrect responses (Miss). (E) Mean trial-by-trial correspondence between JOL values during study and subsequent accuracy during test (γ correlation). (F) Mean trial-by-trial correspondence between RCJ values and accuracy during test (γ correlation). **P = 0.002; *P = 0.01; P = 0.08. Error bars indicate SE.

Figure 3.

Effects of TBS on memory awareness at study and test. (A) Mean study JOL values for trials that were subsequently correct during test (later-hit). (B) Mean study JOL values for trials that were subsequently incorrect during test (later-miss). (C) Mean RCJ values for trials with correct responses (Hit). (D) Mean RCJ values for trials with incorrect responses (Miss). (E) Mean trial-by-trial correspondence between JOL values during study and subsequent accuracy during test (γ correlation). (F) Mean trial-by-trial correspondence between RCJ values and accuracy during test (γ correlation). **P = 0.002; *P = 0.01; P = 0.08. Error bars indicate SE.

### Memory Awareness During Test

RCJs made during test were analyzed separately for correct versus incorrect associative-recognition responses. Higher memory awareness would be indicated by better correspondence between RCJ rating values and associative-recognition performance (i.e., higher RCJs for correct responses and/or lower RCJs for incorrect responses). For correct associative-recognition trials (hits), we observed a marginal main effect of RCJs according to stimulation location (F2,34 = 2.98, MSE = 0.56, P = 0.06, $ηp2$ = 0.15; Fig. 3C). Pairwise comparisons indicated that ratings were significantly higher for frontopolar TBS compared with control (t17 = 2.78, P = 0.01, Cohen's d = 0.48). RCJ ratings were also marginally higher for dorsolateral TBS compared with control (t17 = 1.86, P = 0.08, Cohen's d = 0.44) but did not differ significantly between frontopolar and dorsolateral TBS (t17 = 0.008, P = 0.99). A similar analysis for the incorrect recognition trials (misses) did not reveal a main effect of stimulation location (F2,34 = 0.66, ns; Fig. 3D), nor were any pairwise differences significant (all pairwise P values > 0.36). Thus, frontopolar stimulation was selective in enhancing memory awareness by increasing RCJs for correct associative-recognition responses.

### Trial-by-trial Correspondence of Performance and Awareness

To test effects of TBS on trial-by-trial correspondence between objective performance and subjective awareness ratings (i.e., resolution), we computed non-parametric Goodman–Kruskal γ correlations (Nelson 1984). Analyses were performed separately for study (trial-by-trial JOLs correlated with subsequent associative memory accuracy) and for test (trial-by-trial RCJs correlated with associative memory accuracy). There was a marginal main effect of stimulation location on γ correlations for JOLs (F2,34 = 3.02, MSE = 0.10, P = 0.06, $ηp2$ = 0.15) and a significant linear effect (F1,17 = 7.41, MSE = 0.08, P = 0.01, $ηp2$ = 0.47; Fig. 3E). This linear effect reflected increasing JOL-accuracy correspondence across the control, dorsolateral, and frontopolar TBS locations. The individual-participant variability of γ correlations for JOL-performance correspondence across treatment conditions is displayed in Figure 4. Whereas there was substantial individual variability in γ correlation values for the control location (Fig. 4, top panel), frontopolar TBS was very consistent across participants in increasing γ correlation values (Fig. 4, bottom panel), whereas the increase for dorsolateral TBS was less consistent (Fig. 4, middle panel).

Figure 4.

Individual variability of TBS effects on JOL-accuracy correspondence. The three panels show, for each participant, the correspondence between JOL and test accuracy (γ correlations) in the control condition, the change in γ correlations for dorsolateral TBS (vs. control), and the change in γ correlations for frontopolar TBS (vs. control).

Figure 4.

Individual variability of TBS effects on JOL-accuracy correspondence. The three panels show, for each participant, the correspondence between JOL and test accuracy (γ correlations) in the control condition, the change in γ correlations for dorsolateral TBS (vs. control), and the change in γ correlations for frontopolar TBS (vs. control).

Overall, mean RCJ γ correlations were substantially higher than those observed for JOLs (F1,17 = 75.56, MSE = 0.11, P < 0.001, $ηp2$ = 0.82), consistent with some previous research suggesting that RCJs can more accurately reflect performance than JOLs (e.g., Dougherty et al. 2005; Wattier and Collin 2011). However, TBS location did not significantly influence γ correlations for RCJs (F2,34 = 0.46, ns).

To complement these analyses of JOL and RCJ γ correlation, we also used an alternative Type-2 ROC computation that guards against the influence of response bias (i.e., or the degree to which an individual is prone to respond with high or low confidence regardless of actual judgment accuracy) on the relationship between subjective rating and response accuracy (see Maniscalco and Lau 2012; Fleming and Lau 2014). This additional analysis replicated our finding of a significant linear increase in JOL-performance accuracy across TBS conditions as well as no TBS effect on RCJ-performance accuracy (see Supplementary Materials for computational details and findings).

### Global Awareness Ratings

Calibration between recognition performance and global predictions and postdictions were analyzed using bias scores. These scores were calculated by subtracting the raw number of correctly identified associations from the raw number of associations they estimated that they would identify prior to beginning the test (predictions) or the number of associations that they believed they correctly identified after completing the test (postdictions). A bias score of zero would represent perfect calibration between global awareness and performance, a positive value would represent overconfidence, and a negative value would represent underconfidence. Mean bias scores did not vary significantly by TBS for either predictions or postdictions (F2,34 = 0.23, ns, and F2,34 = 1.03, ns, respectively). Mean bias values across all conditions were −2.07 items for predictions and −3.75 items for postdictions, suggesting a pattern of general underconfidence in global awareness. Finally, TBS location did not interact significantly with global judgment type (predictions: F2,34 = 0.75, ns; postdictions: F2,34 = 0.56, ns).

### No Disruptive Effects of TBS on Cross-Trial Interference/Integration

One interpretation of these effects is that increased memory awareness reflected an attenuation of the extent to which awareness responses were influenced by previous (i.e., interfering) trials. To test this possibility, we performed a control analysis of the relationship of JOL ratings on one trial to the next trial by correlating ratings made on trial n with ratings made on trial n+ 1. Given that ratings on the previous trial should have been irrelevant with respect to the rating made on the current trial, then a decrease in correlation would be expected with frontopolar TBS to the extent that the effects of TBS on accuracy of memory awareness were due to disruption of cross-trial interference/integration. The mean correlation values were near zero, indicating no reliable influence, and there was no effect of stimulation location on ratings (mean correlation for control TBS = −0.04, dorsolateral TBS = 0.02, and frontopolar TBS = −0.10; (F2,34 = 1.67, ns). Thus, a significant increase in JOL-accuracy correspondence (i.e., improved memory awareness accuracy) based on stimulation location shown in Figure 3E was not due to a reduction in the influence of one trial's ratings on the next.

We performed a similar control analysis for RCJ ratings during test. Interestingly, frontopolar TBS was selectively associated with an increase in the correlation between RCJ ratings made on one trial versus the next [i.e., increased correlation of RCJ on trial n with trial n+ 1; mean correlation for control TBS = −0.09, dorsolateral TBS = 0.02, and frontopolar TBS = 0.25; F2,34 = 3.96, MSE = 0.14, P = 0.03, $ηp2$ = 0.19]. Pairwise comparisons indicated that this increased correspondence was higher for frontopolar TBS compared with control TBS (t17 = 2.61, P = 0.02, Cohen's d = 0.85), and it was marginally higher for frontopolar TBS compared with dorsolateral TBS (t17 = 1.92, P = 0.07, Cohen's d = 0.58). However, no influence of dorsolateral TBS was found compared with control TBS (t17 = 0.90, ns). This is opposite to what would have been expected if frontopolar TBS inhibited the role of frontopolar cortex in integrating across trials and suggests that, alternatively, this integrative function may have been enhanced during test by frontopolar TBS.

Notably, RCJ ratings were more accurate following frontopolar TBS (Fig. 3C), not less accurate, as would be expected if the previous trial were interfering more with the current trial. We thus performed a similar analysis on accuracy values, given that correlated RCJ ratings from one trial to the next could have truly reflected increased correspondence in accuracy. For this analysis, accuracy was treated as a binary variable and the φ coefficient was used to measure correlation. This analysis identified a selective increase in the correlation of accuracy made on one trial compared with accuracy on the next trial due to frontopolar TBS (i.e., increased correlation of accuracy on trial n with trial n+ 1; mean correlation for control TBS = −0.02, dorsolateral TBS = −0.04, and frontopolar TBS = 0.07; (F2,34 = 3.16, MSE = 0.02, P = 0.05, $ηp2$ = 0.16). Pairwise comparisons indicated that frontopolar TBS was marginally higher than control TBS (t17 = 1.96, P = 0.07, Cohen's d = 0.45), and frontopolar TBS was significantly higher than dorsolateral TBS (t17 = 2.13, P = 0.05, Cohen's d = 0.50), yet dorsolateral TBS did not differ versus control TBS (t17 = 0.43, ns). Thus, frontopolar TBS increased the local structure of memory success, whereby successful performance on one trial was associated to a higher degree with successful performance on the subsequent trial after frontopolar TBS compared with the other TBS conditions. This pattern is opposite of what would have been expected had frontopolar TBS disrupted interference/integration from one trial to the next; in fact, this influence was increased during test. Notably, this finding is unrelated to the significant effects of TBS on JOL ratings during study, as the order of stimuli during test was irrelevant to ratings made during study.

## Discussion

Bilateral TBS of frontopolar cortex significantly improved awareness of associative-recognition memory during both study and test, relative to bilateral dorsolateral prefrontal TBS and to bilateral control TBS. During study, this improvement for frontopolar TBS versus both control TBS and dorsolateral TBS was indicated by better correspondence between JOL ratings and subsequent accuracy (lower JOL ratings for items that went on to be forgotten). In contrast, dorsolateral TBS did not significantly improve JOL ratings versus control TBS. These effects were further confirmed by analysis of trial-by-trial correspondence between JOL ratings and subsequent accuracy (γ correlation), which indicated a significant linear effect reflecting increasing JOL-performance correspondence across the control, dorsolateral, and frontopolar TBS locations. During test, frontopolar TBS improved correspondence between RCJ ratings and accuracy (higher RCJ ratings for correct items) versus control TBS. Dorsolateral TBS produced the same effect relative to control TBS, but with marginal significance. The trial-by-trial correspondence of awareness with objective performance (γ correlation) identified for JOLs at study was not evident for RCJs at test. This could have been due to the fact that γ correlation was much higher overall for RCJs than JOLs (e.g., Dougherty et al. 2005), thus obscuring possible enhancing effects of TBS (i.e., ceiling effects). In contrast to these effects of TBS on memory awareness, TBS had no significant effects on accuracy of associative-recognition responses (i.e., objective performance). This indicates that effects of TBS were specific to memory awareness, and thus did not reflect global/nonspecific influences of TBS on memory performance.

Collectively, these results suggest relatively greater involvement of frontopolar compared with dorsolateral prefrontal cortex in memory awareness, consistent with previous hypotheses of greater frontopolar involvement based on hierarchical rostrocaudal organization of prefrontal cortex (e.g., Fuster 1997; Koechlin and Hyafil 2007; Koechlin and Summerfield 2007; Badre 2008; Badre and D'Esposito 2009). By targeting frontopolar versus dorsolateral prefrontal cortex with TBS in healthy individuals, we provided anatomical specificity that extends inferences derived from lesion-deficit studies (e.g., Burgess et al. 2010) that frontopolar cortex makes necessary/causal contributions to memory awareness. Although some studies have identified lateralized material-specific effects of TBS to frontopolar cortex in visual perception tasks (e.g., right frontopolar TBS effects using images in Costa et al. 2013, and left frontopolar TBS effects using words in Costa et al. 2011), we used bilateral stimulation in order to more specifically focus on memory awareness rather than on material specificity. Future studies could test for laterality differences in frontopolar TBS effects on memory awareness.

Surprisingly, frontopolar TBS improved, rather than impaired, memory awareness for both JOL and RCJ responses. This is interesting in light of the proposal that frontopolar cortex is involved in memory awareness because it receives converging input from a variety of more posterior regions associated with memory processing, thus serving as the “capstone” of a rostrocaudal hierarchy for awareness, monitoring, and cognitive control (e.g., Friston 2005; Koechlin and Hyafil 2007; Badre 2008; Passingham and Wise 2012). To the extent that this converging input to frontopolar cortex is necessary for successful memory awareness, the increases in accuracy of memory awareness that we observed following frontopolar TBS could have been due to increased interactivity of frontopolar cortex with other memory-processing regions. Indeed, Gratton et al. (2013) identified increased functional connectivity of distributed prefrontal cortex networks following TBS. Although mechanisms for TBS-induced increases in functional connectivity of prefrontal cortex remain unclear, such increases provide parsimonious explanation for the results reported here that is consistent with previous theorizing on the importance of frontopolar connectivity for memory awareness. The current study used continuous TBS, as did those just summarized, rather than intermittent TBS. Future studies could examine effects of intermittent versus continuous TBS on prefrontal function, which could potentially lead to distinct effects (e.g., Dayan et al. 2013).

An alternative interpretation of our effects based on the notion that TBS has disruptive effects on function (Dayan et al. 2013) is that increased memory awareness after frontopolar TBS reflected reduction in the extent to which awareness responses were influenced by across-trial interference. That is, if TBS caused an inhibition-like disruption of memory for stimuli other than those present on the current trial, owing to a potential role of frontopolar cortex in integration across trials and similar working-memory functions (cf. Badre and Wagner 2004), then awareness ratings for the immediate trial could have been more accurate due to a reduction in disruptive levels of interference from competing representations. However, this possibility seems unlikely given that TBS had no influence on memory accuracy, which would have been expected if enhancing the accuracy of awareness responses were due to disruption of memory representations. Furthermore, results from control analyses were inconsistent with the interference explanation. Regarding JOLs, there was no effect of stimulation location on the magnitude of correlations between ratings of one trial with the subsequent trial. Had frontopolar TBS improved JOL-accuracy correspondence by producing disruptive effects on influence/interference from one trial to the next, then trial-to-trial correlations would have been expected to be decreased following frontopolar TBS compared with other TBS locations. Regarding RCJs, frontopolar TBS produced an increase in correlation of ratings from one trial to the next, not a decrease, which was likely due to the fact that accuracy was also more correlated from one trial to the next following frontopolar TBS compared with other TBS locations. Thus, frontopolar TBS actually increased, rather than decreased, the influence of information processing from one trial to the next. These patterns are opposite of what would have been expected based on the interpretation that TBS increased awareness accuracy by disrupting normal prefrontal involvement in working-memory maintenance and cross-trial integration of representations. These patterns are instead consistent with the facilitation hypothesis motivated by the fMRI connectivity findings of Gratton et al. (2013).

Our findings contribute to the understanding of proposed differential roles of rostral prefrontal cortex in imagining performance in the future (Addis et al. 2007; Buckner and Carroll 2007) and forming intentions toward future behavior (e.g., Okuda et al. 2007; Costa et al. 2013), versus lateral frontal cortex in confidence judgments based on immediately available mnemonic information (e.g., Pannu and Kaszniak 2005; Morgan et al. 2014). In contrast to these accounts, we found that frontopolar cortex stimulation had the most robust effect on both JOL and RCJ ratings. Indeed, there was a linear increase in JOL-accuracy correlations for control, dorsolateral, and frontopolar stimulation (Fig. 3E) and effects on RCJ confidence were robust only for frontopolar stimulation (Fig. 3C). Thus, our findings are consistent with the interpretation that frontopolar cortex is critical for prospective judgments but inconsistent with the view that this role is selective for prospection. Our findings suggest instead that frontopolar cortex is involved both in prospective judgments of future memory performance as well as in using immediately available mnemonic information to make confidence judgments. It is important to note however that prospective and RCJs are similar in the sense that both involve evaluation of the strength/quality of memory processing that occurred for the immediately preceding stimulus, and thus shared anatomical substrates are not unreasonable.

Some previous findings have emphasized frontopolar involvement in general awareness, including awareness of memory as well as of perception (Baird et al. 2013), whereas other findings have suggested unique roles of frontopolar cortex in perception awareness rather than memory awareness (Fleming et al. 2014). For instance, Fleming et al. (2014) found that patients with anterior prefrontal lesions have awareness deficits for visual perception but not for verbal recognition memory. However, it is unclear whether these relatively diffuse prefrontal lesions affect lateral versus medial sub-regions of frontopolar networks implicated in perceptual versus memory awareness (Baird et al. 2013). Furthermore, the nature of the memoranda (verbal vs. visual) might influence the loci of neural processing related to memory awareness. The current study did not attempt to distinguish frontopolar involvement in perception versus memory awareness, but findings nonetheless support conclusion that frontopolar cortex is not specific in supporting perception awareness versus memory awareness.

It should be noted that frontopolar TBS influenced JOL ratings for subsequent misses (rather than subsequent hits) and RCJ ratings for hits (rather than misses). Instead of proposing any specific relationship between JOL ratings to subsequent misses and RCJ ratings to hits, we speculate that whether frontopolar TBS influences hits versus misses should depend on the nature of the task and on the subjective experience of confidence for hits and misses given task parameters. Overall, individuals did not experience strong subjective distinctions between subsequent hits and misses as indicated by small differences in JOL levels for later-hits versus later-misses (e.g., Fig. 3A vs. B) and relatively low JOL-accuracy correlation (Fig. 3E). Indeed, many factors complicate the relationship between confidence at study and later performance (i.e., the variable success of retention, unpredictable levels of interference from forthcoming related stimuli, and uncertainty regarding the specific format of the subsequent test trial). Frontopolar TBS thus likely served to make individuals more aware of the type of encoding failure that robustly leads to poor later performance (i.e., because success prediction is more variable). In contrast, participants experienced strong and immediate differences in confidence between hits and misses during test, as indicated by RCJ ratings that highly distinguished these categories (e.g., Fig. 3C vs. D). In this circumstance, misses were likely readily categorized as such, and increases in hit RCJ ratings were likely due to heightened sensitivity to the type of retrieval success robustly related to correct performance. Thus, for both JOLs for later-misses and RCJs for hits, frontopolar TBS likely served to increase the sensitivity of the judgment, and different testing conditions could alter which type of responses (hits or misses) benefit from such increased sensitivity.

Under many circumstances, memory awareness is used to monitor the success of learning and can support various control functions that serve to improve learning (e.g., Koriat and Goldsmith 1996; Dunlosky and Bjork 2008; Metcalfe 2013). It is therefore noteworthy that frontopolar TBS did not improve memory performance in the current experiment, despite improved memory awareness. However, several features of our memory test likely served to limit participants' ability to effectively control learning based on memory awareness/monitoring. For instance, memoranda included novel and complex fractal stimuli arbitrarily paired with objects, and therefore most potential mnemonic strategies were reduced or eliminated (i.e., strategies based on semantic labeling, intentional rehearsal, or sustained mental imagery of stimuli). Further, each study stimulus pair was presented only once for a fixed duration, eliminating control strategies involving self-selection of study frequency and/or duration. Under self-controlled learning circumstances, improvement of memory awareness due to frontopolar TBS could potentially yield gains in memory performance. For instance, a region of frontopolar cortex overlapping the location stimulated in the current experiment was recently found to support judgments regarding what information to sample during associative learning, and these judgments enhanced later memory performance (Wang and Voss 2014).

Disruptions of memory awareness can occur in addition to poor objective performance in cases of clinical memory impairment (McGlynn and Kaszniak 1991; Boake et al. 1995; Perrotin et al. 2007; Salmon et al. 2008; Hainselin et al. 2012). Poor memory awareness could exacerbate problems with objective memory performance by hindering learning strategies normally supported by memory awareness, such as monitoring (Nelson and Narens 1980,1990; Chua et al. 2014). Impaired awareness of memory deficits could furthermore hinder identification of memory problems and consequently preclude intervention (Pannu and Kaszniak 2005; Souchay 2007; Cosentino 2014). Our finding of improved memory awareness due to frontopolar TBS thus holds promise as a potential treatment in cases of memory awareness impairment. However, future research will be needed to determine whether such improvement can occur in clinical populations, and whether the duration of improvement could be extended with clinically oriented stimulation regimens (e.g., Wang et al. 2014). Given the specificity of our continuous TBS stimulation to rostral prefrontal cortex, we provide evidence that frontopolar cortex has a causal role in memory awareness in healthy individuals and that memory awareness can be improved noninvasively. The current findings therefore motivate focus on this region as an intervention target in cases of impaired memory awareness.

## Supplementary Material

Supplementary material can be found at: http://www.cercor.oxfordjournals.org/

## Funding

This work was supported by award number P50-MH094263 from the National Institute of Mental Health and by award number T32-NS047987 from the National Institute of Neurological Disorders and Stroke. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

## Notes

We thank Molly Hermiller and Alyssa Cunningham for their assistance with data collection. We also thank Jane X. Wang and two anonymous reviewers for their helpful suggestions on an earlier version of this manuscript. Finally, we thank Doug Harrington (fractalarts.com) for his contribution of original fractal art for our study. Conflict of Interest: None declared.

## References

DR
Wong
AT
Schacter
DL
.
2007
.
Remembering the past and imagining the future: Common and distinct neural substrates during event construction and elaboration
.
Neuropsychologia
.
45
:
1363
1377
.
D
.
2008
.
Cognitive control, hierarchy, and the rostro-caudal organization of the frontal lobes
.
Trends Cogn Sci
.
12
:
193
200
.
D
D'Esposito
M
.
2009
.
Is the rostro-caudal axis of the frontal lobe hierarchical?
Nat Rev Neurosci
.
10
:
659
669
.
D
Wagner
.
2004
.
Selection, integration, and conflict monitoring: assessing the nature and generality of prefrontal cognitive control mechanisms
.
Neuron
.
41
:
473
487
.
Baird
B
Smallwood
J
Gorgolewski
KJ
Margulies
DS
.
2013
.
Medial and lateral networks in anterior prefrontal cortex support metacognitive ability for memory and perception
.
J Neurosci

33
:
16657
16665
.
Blumenfeld
RS
Lee
T
D'Esposito
M
.
2014
.
The effects of lateral prefrontal transcranial magnetic stimulation on item memory encoding
.
Neuropsychologia

53
:
197
202
.
Boake
C
Freeland
JC
Ringholz
GM
Nance
ML
Edwards
KE
.
1995
.
Awareness of memory loss after severe closed-head injury
.
Brain Inj
.
9
:
273
283
.
Braver
TS
Bongiolatti
SR
.
2002
.
The role of frontopolar cortex in subgoal processing during working memory
.
Neuroimage
.
15
:
523
536
.
Brodeur
MB
Dionne-Dostie
E
Montreuil
T
Lepage
M
.
2010
.
The Bank of Standardized Stimuli (BOSS), a new set of 480 normative photos of objects to be used as visual stimuli in cognitive research
.
PLoS ONE
.
5
:
e10773
.
Buckner
RL
Carroll
DC
.
2007
.
Self-projection and the brain
.
Trends Cogn Sci
.
11
:
49
57
.
Burgess
PW
Dumontheil
I
Gilbert
SJ
.
2007
.
The gateway hypothesis of rostral prefrontal cortex (area 10) function
.
Trends Cogn Sci
.
11
:
90
298
.
Burgess
PW
Simons
JS
Dumontheil
I
Gilbert
SJ
.
2010
.
The gateway hypothesis of rostral prefrontal cortex (area 10) function
. In:
Duncan
J
Phillips
L
McLeod
C
, editors.
Measuring the Mind: Speed, Control, and Age
.
Oxford
:
Oxford University Press
. p.
217
248
.
Cabeza
R
Dolcos
F
Prince
SE
Rice
HJ
Weissman
DH
Nyberg
L
.
2003
.
Attention-related activity during episodic memory retrieval: a cross-function fMRI study
.
Neuropsychologia
.
41
:
90
399
.
Christoff
K
Gabrieli
JDE
.
2000
.
The frontopolar cortex and human cognition: evidence for a rostrocaudal hierarchical organization within the human prefrontal cortex
.
Psychobiology
.
28
:
168
186
.
Christoff
K
Prabhakaran
V
Dorfman
J
Zhao
Z
Kroger
JK
Holyoak
KJ
Gabrieli
JDE
.
2001
.
Rostrocaudal prefrontal cortex involvement in relational integration during reasoning
.
Neuroimage
.
14
:
1136
1149
.
Chua
EF
Pergolizzi
D
Weintraub
RR
.
2014
.
Understanding metamemory processes, subjective levels expressed, and metacognitive accuracy
. In:
Fleming
SM
Frith
CD
, editors.
The Cognitive Neuroscience of Metacognition
.
Berlin
:
Springer-
, p.
267
292
.
Cosentino
SA
.
2014
.
Metacognition in Alzheimer's disease
. In:
Fleming
SM
Frith
CD
, editords.
The Cognitive Neuroscience of Metacognition
.
Berlin
:
Springer-
, p.
389
407
.
Cosentino
SA
Metcalfe
J
Holmes
B
Steffener
J
Stern
Y
.
2011
.
Finding the self in metacognitive evaluations: A study of metamemory and agency in non-demented elders
.
Neuropsychology
.
25
:
602
.
Costa
A
Oliveri
M
Barban
F
Bonnì
S
Koch
G
Caltagirone
C
Carlesimo
GA
.
2013
.
The right frontopolar cortex is involved in visual-spatial prospective memory
.
PLoS ONE
.
8
:
e56039
.
Costa
A
Oliveri
M
Barban
F
Torriero
S
Salerno
S
Gerfo
EL
Carlesimo
GA
.
2011
.
Keeping memory for intentions: a cTBS investigation of the frontopolar cortex
.
Cerebral Cortex
.
21
:
2696
2703
.
Cox
RW
.
1996
.
AFNI: software for analysis and visualization of functional magnetic resonance neuroimages
.
Comput Biomed Res
.
29
:
162
173
.
Dayan
E
Censor
N
Buch
ER
Sandrini
M
Cohen
LG
.
2013
.
Noninvasive brain stimulation: from physiology to network dynamics and back
.
Nat Neurosci
.
16
:
838
844
.
Dougherty
MR
Scheck
P
Nelson
TO
Narens
L
.
2005
.
Using the past to predict the future
.
Mem Cognit

33
:
1096
1115
.
Dreher
JC
Koechlin
E
Tierney
M
Grafman
J
.
2008
.
Damage to the fronto-polar cortex is associated with impaired multitasking
.
PLoS ONE
.
3.9
:
e3227
.
Dunlosky
J
Bjork
RA
, editors.
2008
.
A Handbook of Metamemory and Memory
.
Hillsdale
:
Psychology Press
.
Evans
AC
Collins
DL
Mills
SR
Brown
ED
Kelly
RL
Peters
TM
.
1993
.
3D statistical neuroanatomical models from 305 MRI volumes
.
Proceedings of IEEE-Nuclear Science Symposium and Medical Imaging Conference
.
1813
1817
.
Fernandes
MA
Moscovitch
M
Ziegler
M
C
.
2005
.
Brain regions associated with successful and unsuccessful retrieval of verbal episodic memory as revealed by divided attention
.
Neuropsychologia
.
43
:
1115
1127
.
Fleming
SM
Dolan
RJ
.
2012
.
The neural basis of accurate metacognition
.
Phil Trans R Soc B
.
367
:
338
349
.
Fleming
SM
Frith
CD
, editors.
2014
.
The Cognitive Neuroscience of Metacognition
.
Berlin
:
Springer-Verlag
.
Fleming
SM
Huijgen
J
Dolan
RJ
.
2012
.
Prefrontal contributions to metacognition in perceptual decision-making
.
J Neurosci
.
32
:
6117
6125
.
Fleming
SM
Lau
HC
.
2014
.
How to measure metacognition
.
Front Hum Neurosci
.
8
:
443
.
Fleming
SM
Ryu
J
Golfinos
JG
Blackmon
KE
.
2014
.
Domain-specific impairment in metacognitive accuracy following anterior prefrontal lesions
.
Brain

137
:
2811
2822
.
Fleming
SM
Weil
RS
Nagy
Z
Dolan
RJ
Rees
G
.
2010
.
Relating introspective accuracy to individual differences in brain structure
.
Science
.
329
:
1541
1543
.
Friston
K
.
2005
.
A theory of cortical responses
.
Phil Trans R Soc
.
B360
:
815
836
.
Fuster
J
.
1997
.
The Prefrontal Cortex: Anatomy, Physiology, and Neuropsychology of the Frontal Lobe
.
:
Lippincott-Raven
.
Gratton
C
Lee
TG
Nomura
EM
D'Esposito
M
.
2013
.
The effect of theta-burst TMS on cognitive control networks measured with resting state fMRI
.
Front Syst Neurosci
.
7
:
124
.
Hainselin
M
Quinette
P
Desgranges
B
Martinaud
O
de La Sayette
V
Hannequin
D
F
Eustache
F
.
2012
.
Awareness of disease state without explicit knowledge of memory failure in transient global amnesia
.
Cortex
.
48
:
1079
1084
.
Harrington
D
.
2011
.
FractalArts.com. Retrieved November 22, 2014, from
Available from: URL .
Lee
TG
Blumenfeld
RS
D'Esposito
M
.
2013
.
Disruption of dorsolateral but not ventrolateral prefrontal cortex improves unconscious perceptual memories
.
J Neurosci
.
33
:
13233
13237
.
Koechlin
E
Basso
G
Pietrini
P
Panzer
S
Grafman
J
.
1999
.
The role of anterior prefrontal cortex in human cognition
.
Nature
.
399
:
48
151
.
Koechlin
E
Hyafil
A
.
2007
.
Anterior prefrontal function and the limits of human decision-making
.
Science
.
318
:
594
598
.
Koechlin
E
Summerfield
C
.
2007
.
An information theoretical approach to prefrontal executive function
.
Trends Cogn Sci
.
11
:
229
235
.
Koriat
A
Goldsmith
M
.
1996
.
Monitoring and control processes in the strategic regulation of memory accuracy
.
Psych Rev
.
103
:
490
.
Maniscalco
B
Lau
H
.
2012
.
A signal detection theoretic approach for estimating metacognitive sensitivity from confidence ratings
.
Conscious Cogn

21
:
422
430
.
McGlynn
SM
Kaszniak
AW
.
1991
.
When metacognition fails: Impaired awareness of deficit in Alzheimer's disease
.
J Cogn Neurosci
.
3
:
183
187
.
Metcalfe
J
.
2013
.
Metacognitive control of study
. In:
Perfect
TJ
Lindsay
S
, editors.
The Sage Handbook of Applied Memory
.
London
:
Sage
, p.
461
485
.
Miller
EK
Cohen
JD
.
2001
.
An integrative theory of prefrontal cortex function
.
Annu Rev Neurosci
.
24
:
167
202
.
Morgan
G
Kornell
N
Kornblum
T
Terrace
HS
.
2014
.
Retrospective and prospective metacognitive judgments in rhesus macaques (Macaca mulatta)
.
Anim Cogn
.
17
:
249
257
.
Nelson
TO
.
1984
.
A comparison of current measures of accuracy of feeling-of-knowing predictions
.
Psych Bull
.
95
:
109
133
.
Nelson
TO
Dunlosky
J
.
1991
.
When people's judgments of learning (JOLs) are extremely accurate at predicting subsequent recall: the delayed-JOL effect
.
Psychol Sci
.
2
:
267
270
.
Nelson
T
Narens
L
.
1990
.
Metamemory: a theoretical framework and new findings
. In:
Bower
GH
, editor.
The Psychology of Learning and Motivation: Advances in Research and Theory
, Vol.
26
.
Waltham
:
, p.
125
169
.
Nelson
TO
Narens
L
.
1980
.
Norms of 300 general-information questions: accuracy of recall, latency of recall, and feeling-of-knowing ratings
.
J Verbal LearnVerbal Behav
.
19
:
338
368
.
Okuda
J
Fujii
T
Ohtake
H
Tsukiura
T
A
Frith
CD
Burgess
PW
.
2007
.
Differential involvement of regions of rostral prefrontal cortex (Brodmann area 10) in time-and event-based prospective memory
.
Int J Psychophysiol
.
64
:
233
246
.
Osaka
M
Osaka
N
Kondo
H
Morishita
M
Fukuyama
H
Aso
T
Shibasaki
H
.
2003
.
The neural basis of individual differences in working memory capacity: an fMRI study
.
NeuroImage
.
18
:
789
797
.
Pannu
JK
Kaszniak
AW
.
2005
.
Metamemory experiments in neurological populations: a review
.
Neuropsychol Rev
.
15
:
105
130
.
Passingham
RE
Wise
SP
.
2012
.
The Neurobiology of the Prefrontal Cortex. Anatomy, Evolution, and the Origin of Insight
.
Oxford
:
Oxford University Press
.
Perrotin
A
Belleville
S
Isingrini
M
.
2007
.
Metamemory monitoring in mild cognitive impairment: evidence of a less accurate episodic feeling-of knowing
.
Neuropsychologia
.
45
:
2811
2826
.
Ramnani
N
Owen
AM
.
2004
.
Anterior prefrontal cortex: insights into function from anatomy and neuroimaging
.
Nat Rev Neurosci
.
5
:
184
194
.
Rex
DE
Ma
JQ
Toga
AW
.
2003
.
The LONI pipeline processing environment
.
Neuroimage
.
19
:
1033
1048
.
Rosner
ZA
Elman
JA
Shimamura
AP
.
2013
.
The generation effect: activating broad neural circuits during memory encoding
.
Cortex
.
49
:
1901
1909
.
Rossi
S
Hallett
M
Rossini
PM
Pascual-Leone
A
.
2009
.
Safety, ethical considerations, and application guidelines for the use of transcranial magnetic stimulation in clinical practice and research
.
Clin Neurophysiol
.
120
:
2008
2039
.
Rounis
E
Maniscalco
B
Rothwell
JC
Passingham
RE
Lau
H
.
2010
.
Theta-burst transcranial magnetic stimulation to the prefrontal cortex impairs metacognitive visual awareness
.
Cog Neurosci

2011
:
165
175
.
Ruohonen
J
Ilmoniemi
RJ
.
2002
.
Physical principles for transcranial magnetic stimulation
. In:
Pascual-Leone
A
Davey
NJ
Rothwell
JC
Wassermann
EM
Puri
BK
, editors.
Handbook of Transcranial Magnetic Stimulation
.
New York
:
Oxford
. p.
18
30
.
Salmon
E
Perani
D
Collette
F
Feyers
D
Kalbe
E
Holthoff
V
Sorbi
S
Herholz
K
.
2008
.
A comparison of unawareness in frontotemporal dementia and Alzheimer's disease
.
J Neurol Neurosurg Psychiatry
.
79
:
176
179
.
Shimamura
AP
.
2002
.
Memory retrieval and executive control processes
. In:
Stuss
D
Knight
RT
, editors.
The Frontal Lobes
.
New York
:
Oxford University Press
. p.
210
220
.
Simons
JS
Henson
RNA
Gilbert
SJ
Fletcher
PC
.
2008
.
Separable forms of reality monitoring supported by anterior prefrontal cortex
.
J Cogn Neurosci
.
20
:
447
457
.
Souchay
C
.
2007
.
Metamemory in Alzheimer's disease
.
Cortex
.
43
:
987
1003
.
Voss
JL
Paller
KA
.
2009
.
An electrophysiological signature of unconscious recognition memory
.
Nat Neurosci
.
12
:
349
355
.
Wang
JX
Rogers
LM
Gross
EZ
Ryals
AJ
Dokucu
ME
Brandstatt
KL
Voss
JL
.
2014
.
Targeted enhancement of cortical-hippocampal brain networks and associative memory
.
Science
.
345
:
1054
1057
.
Wang
JX
Voss
JL
.
2014
.
Brain networks for exploration decisions utilizing distinct modeled information types during contextual learning
.
Neuron
.
82
:
1171
1182
.
Wattier
NW
Collin
CA
.
2011
.
Metamemory for faces, names, and common nouns
.
Acta Psychol
.
138
:
143
154
.