Abstract

Previous research has supported the use of percent retention scores in the neuropsychological assessment of memory, and many widely used memory measures provide for the calculation and normative comparison of these scores. The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS), an increasingly utilized assessment tool for cognitive impairment, provides normative data on delayed memory total raw scores only. The current study was aimed at determining the diagnostic accuracy of a novel percent retention score calculated from RBANS verbal memory subtests (delayed recall minus last learning trial) when distinguishing between normal controls, individuals diagnosed with Mild Cognitive Impairment, and individuals diagnosed with Alzheimer's disease. Results revealed excellent diagnostic accuracy of the RBANS percent retention scores when discriminating between the three groups. Findings suggest that RBANS percent retention scores provide excellent diagnostic accuracy offering supplementary information to clinicians and researchers alike.

Introduction

According to the World Health Organization (2006), latest estimates suggest that over 18 million people worldwide suffer from Alzheimer's disease (AD), a degenerative disease that manifests primarily in memory deficits while affecting other cognitive domains as well. Costs due to AD in the USA alone totaled more than 500 billion dollars in 2000 (World Health Organization, 2006). With the advent of pharmacological treatments that slow the progression of AD, as well as research efforts to prevent the development of this devastating disease, the importance of early and accurate diagnosis becomes clear. Furthermore, with a lack of ability to definitively diagnosis AD prior to autopsy, the diagnostic abilities of currently available instruments are of primary focus.

Recently, many researchers have focused on the concept of Mild Cognitive Impairment (MCI), defined as “a condition of intermediate symptomatology between the cognitive changes of aging and fully developed symptoms of dementia” (Petersen & Negash, 2008, p. 46). The criteria for MCI include complaints of cognitive decline in addition to evidence of cognitive impairment on neuropsychological testing with essentially intact activities of daily living; conversely, a diagnosis of fully developed dementia requires the demonstration of impairment in everyday functioning. Findings suggesting that individuals with MCI, particularly its amnestic variant, are at increased risk for progression to fully developed dementia underlie the importance of MCI detection (Petersen et al., 1999). Thus, MCI represents an opportunity for early intervention for the prevention of dementia, making the accurate detection and diagnosis of MCI imperative.

The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) was originally developed as a brief, easily administered measure for the detection of dementia (Randolph, 1998). This widely validated assessment tool (Aupperle, Beatty, Shelton, & Gontkovsky, 2002; Freilich & Hyer, 2007; Gold, Queern, Iannone, & Buchanan, 1999; Gontkovsky, Beatty, & Mold, 2004; Gontkovsky, Hillary, & Scott, 2002; Hobart, Goldberg, Bartko, & Gold, 1999; Larson, Kirschner, Bode, Heinemann, & Goodman, 2005; McKay, Casey, Wertheimer, & Fichtenberg, 2007; Pachet, 2007; Randolph, Tierney, Mohr, & Chase, 1998; Wilde, 2006) provides the measurement of cognitive abilities in several domains, including immediate and delayed memory, visuospatial/construction abilities, attention, and language, and requires approximately 30 min to complete administration. Index scores corrected for age are provided for each of the five cognitive domains and individual t-scores can be obtained from each of 12 subtests, which include immediate and delayed recall for listed information and stories, among others.

Despite the validity of the RBANS memory subtests, examining scores on immediate and delayed recall subtests separately can be misleading and can overestimate the examinee's memory performance. For example, a 80-year-old male who obtains a raw score of 21 (6 on trial 4) on the List Learning subtest (t-score = 45) and a raw score of 2 on List Recall (t-score = 42) is performing within the average range of functioning according to the normative data; however, if one examines the patient's percent retention from immediate to delayed recall, it appears that the patient does have some deficits in retention and retrieval of verbal information, remembering only 33% of the information he originally learned. Thus, percent retention scores may provide a more sensitive measure of memory dysfunction.

Normative data for percent retention scores exist for many of the most commonly used memory assessment instruments (Schoenberg et al., 2008), including the Wechsler Memory Scale (WMS-III; Wechsler, 1997). Several lines of research provide support for the use of percent retention scores in the neuropsychological assessment of memory. Percent retention scores have been shown to be helpful for differentiating AD from other dementia etiologies, such as frontotemporal dementia (Wicklund, Johnson, Rademaker, Weitner, & Weintraub, 2006) and dementia with Lewy bodies (Ferman et al., 2006). Griffith and colleagues (2006) successfully predicted conversion from amnestic MCI (aMCI) to AD within 2 years by utilizing percent retention scores from a visual memory task. Furthermore, unlike individual immediate and delayed recall total raw scores, which are highly influenced by age and gender, percent retention from immediate to delayed recall has been shown to be relatively independent of both age and gender (Gallagher & Burke, 2007).

In a recent article by Schoenberg and colleagues (2008), the authors examined differences in percent retention rates on the RBANS between controls and subjects with dementia. The authors computed percent retention scores by dividing delayed recall raw scores (i.e., List Recall) by total raw scores for the immediate recall subtests (i.e., List Learning total raw score). As expected, individuals previously diagnosed with some form of cognitive impairment, such as dementia or aMCI, obtained significantly lower percent retention scores than normal controls on all three subtests. Furthermore, 37% of patients in the clinical group who obtained scores within the range of normal functioning on the List Recall subtest were classified as impaired when retention rates were examined. This study provided evidence that RBANS percent retention scores may add clinical utility to this widely used screening instrument. The purpose of the current study was to expand the findings of Schoenberg and colleagues (2008) and examine the clinical utility of RBANS retention scores in diagnostic decision-making. In particular, the current study aimed to determine the diagnostic accuracy of RBANS retention scores when distinguishing between three specific diagnostic groups: Controls, aMCI, and AD. In addition, the methodology of the current study differed from the Schoenberg and colleagues (2008) study. Owing to the nature of the data available, the previous authors computed retention scores using an atypical mathematical formula where the total raw scores for List Learning, Story Memory, and Figure Copy were utilized rather than the scores from the last learning trail. Mathematically, this method creates a larger divisor and thus a smaller dividend. The current study utilized the more standard approach of dividing the delayed recall scores by scores obtained on the final learning trial (i.e., List Learning Trial 4 and Story Learning Trial 2).

Materials and Methods

The sample included 137 (49 men, 88 women) patients assessed within the memory disorders clinic (MDC) at a southwestern medical center. Mean age and education for the full sample was 75.12 (SD = 7.07) and 13.43 (3.36), respectively. Self-reported ethnicity was available for all but two patients: 125 (91.2%) non-Hispanic White, 2 (1.5%) African American, and 8 (5.8%) Hispanic. The ethnic make-up of the sample was representative of the population served by the clinic.

The MDC evaluations included three components: (a) neuropsychological testing that was conducted in English and included the RBANS (Randolph, 1998) among other standardized neuropsychological testing instruments; (b) neuropsychiatric interview and assessment which incorporated the Mini-Mental State Examination (MMSE; Folstein et al., 1975) and the CDR (Hughes, Berg, Danziger, Cohen, & Martin, 1982); and (c) a laboratory evaluation consisting of selected blood studies and magnetic resonance imaging (MRI) analyses. After all clinical data were gathered, information was presented to a weekly multidisciplinary MDC consensus diagnostic meeting, and diagnoses were assigned according to the published research criteria (APA, 2000; Chui et al., 1992; Crook, 1986; McKhann, 1984; Petersen et al., 1999). All participants provided informed consent prior to participation, and all procedures were approved by the local Institutional Review Board.

Control participants were all volunteers with no complaints of cognitive dysfunction who were judged to be clinically normal. The remaining participants presented to the MDC with cognitive complaints. Participants were classified as probable AD according to the criteria of the National Institute of Neurological and Communicative Disorders and Stroke-Alzheimer's Disease and Alzheimer's disease and related Disorders Association Work Group (NINCDS–ARDRDA; McKhann, 1984). Individuals were classified as aMCI or AD according to the existing criteria (Petersen et al., 1999). In order to be classified as aMCI, all participants had to complain of memory problems, have intact and general cognitive functioning (e.g., MMSE > 23), have memory performance 1.5 SD or more below age-appropriate norms, and have normal activities of daily living. In order to be classified as a control, the participant must have performed within normal limits on the psychometric assessment.

Of the 137 patients assessed, 20 (14.6%) were considered controls, 73 (53.3%) were given a diagnosis of AD, and 44 (32.1%) were diagnosed with aMCI. Demographic information for each of the three groups is provided in Table 1.

Table 1.

Demographics, neuropsychological testing scores, and RBANS percent retention scores across groups

 Controls aMCI AD 
n 20 44 73 
Age, mean (SD72.05 (7.10) 74.77 (6.95) 76.16 (6.97) 
Education, mean (SD14.60 (3.56) 13.91 (3.36) 12.82 (3.22) 
Gender (men/women) 7/13 15/29 27/46 
MMSE, mean (SD29.00 (1.21) 27.00 (2.01) 21.79 (4.47) 
CDR, mean (SD1.37 (1.24) 4.42 (2.71) 
Wechsler Memory Scales, mean (SD
 LM I 31.40 (9.56) 20.58 (10.29) 11.39 (9.36) 
 LM II 20.40 (7.92) 5.42 (6.74) 1.58 (2.72) 
AVLT, mean (SD
 Total Trials 1–5 34.75 (6.55) 24.65 (8.37) 17.74 (9.02) 
 Immediate Recall 7.50 (1.73) 2.29 (1.57) 1.21 (1.25) 
 Delayed Recall 7.25 (2.63) 1.12 (1.11) 0.33 (0.76) 
RBANS Subtests, mean (SD
 List Learning 26.25 (5.25) 20.84 (4.31) 14.56 (4.79) 
 List Learning Trial 4 7.85 (1.46) 5.86 (1.49) 4.49 (1.55) 
 Story Learning 19.10 (2.71) 12.47 (3.98) 7.87 (3.70) 
 Story Learning Trial 4 10.85 (1.46) 7.86 (2.45) 4.85 (2.46) 
 Figure Copy 18.75 (1.12) 17.43 (1.98) 15.10 (4.12) 
 Line Orientation 16.95 (2.91) 15.82 (3.32) 13.08 (4.15) 
 Picture Naming 9.85 (0.37) 9.36 (1.04) 8.89 (1.39) 
 Semantic Fluency 19.75 (3.55) 16.00 (4.29) 10.77 (3.57) 
 Digit Span 9.60 (2.28) 9.66 (1.84) 8.59 (2.30) 
 Coding 41.05 (8.02) 33.57 (8.72) 22.67 (11.25) 
 List Recall 6.00 (1.26) 2.61 (2.35) 0.38 (1.02) 
 List Recognition 19.35 (0.93) 17.43 (2.47) 14.81 (2.70) 
 Story Recall 9.55 (1.99) 4.68 (3.14) 1.25 (1.86) 
 Figure Recall 13.35 (4.21) 6.34 (5.22) 2.18 (3.58) 
RBANS Percent Retention, mean (SD
 List Retention 77.11 (12.01) 41.31 (32.45) 7.02 (17.51) 
 Story Retention 86.32 (12.51) 57.37 (31.25) 25.71 (30.65) 
 Controls aMCI AD 
n 20 44 73 
Age, mean (SD72.05 (7.10) 74.77 (6.95) 76.16 (6.97) 
Education, mean (SD14.60 (3.56) 13.91 (3.36) 12.82 (3.22) 
Gender (men/women) 7/13 15/29 27/46 
MMSE, mean (SD29.00 (1.21) 27.00 (2.01) 21.79 (4.47) 
CDR, mean (SD1.37 (1.24) 4.42 (2.71) 
Wechsler Memory Scales, mean (SD
 LM I 31.40 (9.56) 20.58 (10.29) 11.39 (9.36) 
 LM II 20.40 (7.92) 5.42 (6.74) 1.58 (2.72) 
AVLT, mean (SD
 Total Trials 1–5 34.75 (6.55) 24.65 (8.37) 17.74 (9.02) 
 Immediate Recall 7.50 (1.73) 2.29 (1.57) 1.21 (1.25) 
 Delayed Recall 7.25 (2.63) 1.12 (1.11) 0.33 (0.76) 
RBANS Subtests, mean (SD
 List Learning 26.25 (5.25) 20.84 (4.31) 14.56 (4.79) 
 List Learning Trial 4 7.85 (1.46) 5.86 (1.49) 4.49 (1.55) 
 Story Learning 19.10 (2.71) 12.47 (3.98) 7.87 (3.70) 
 Story Learning Trial 4 10.85 (1.46) 7.86 (2.45) 4.85 (2.46) 
 Figure Copy 18.75 (1.12) 17.43 (1.98) 15.10 (4.12) 
 Line Orientation 16.95 (2.91) 15.82 (3.32) 13.08 (4.15) 
 Picture Naming 9.85 (0.37) 9.36 (1.04) 8.89 (1.39) 
 Semantic Fluency 19.75 (3.55) 16.00 (4.29) 10.77 (3.57) 
 Digit Span 9.60 (2.28) 9.66 (1.84) 8.59 (2.30) 
 Coding 41.05 (8.02) 33.57 (8.72) 22.67 (11.25) 
 List Recall 6.00 (1.26) 2.61 (2.35) 0.38 (1.02) 
 List Recognition 19.35 (0.93) 17.43 (2.47) 14.81 (2.70) 
 Story Recall 9.55 (1.99) 4.68 (3.14) 1.25 (1.86) 
 Figure Recall 13.35 (4.21) 6.34 (5.22) 2.18 (3.58) 
RBANS Percent Retention, mean (SD
 List Retention 77.11 (12.01) 41.31 (32.45) 7.02 (17.51) 
 Story Retention 86.32 (12.51) 57.37 (31.25) 25.71 (30.65) 

Notes: MMSE = Mini-Mental State Examination; CDR = Clinical Dementia Rating scale sum of box scores; LM = Logical Memory; AVLT = Rey Auditory Verbal Learning Test; RBANS = Repeatable Battery for the Assessment of Neuropsychological Status; aMCI = amnestic mild cognitive impairment; AD = Alzheimer's disease.

Percent retention scores were computed for each of the memory subtests of the RBANS. Note that Figure Recall was excluded in the present study. The RBANS does not include a learning trial for Figure Copy. Because no indicator for initial encoding of the figure exists, the possibility that a subject's performance on Figure Recall is due to incidental learning rather than actual encoding and retention cannot be ruled out. Thus, the calculation of figure retention scores is prohibited by the format of the RBANS and the methodology utilized by the present study. Calculations for List and Story Retention scores were computed as follows: List Retention = (List Recall raw score/List Learning trial 4 raw score) × 100 and Story Retention = (Story Recall raw score/Story Memory trial 2 raw score) × 100. The diagnostic accuracy of these scores (i.e., List Retention and Story Retention) was examined in addition to mean group differences between controls, AD patients, and aMCI patients. Sensitivity (SN) and specificity (SP) indices, as well as likelihood ratios (LRs), were computed for a range of cutoff scores on both retention tasks. In addition, receiver operating characteristic (ROC) curves were plotted for each diagnostic group and resulting areas under the curve (AUCs), numerical indices of overall diagnostic accuracy, provided. Finally, in order to determine the clinical utility of retention rates over and above the currently utilized Delayed Memory Index of the RBANS, the AUCs of the retention rates were directly compared with that of the Delayed Memory Index via MedCalc software. MedCalc software utilizes a nonparametric approach to comparing the AUCs of two or more ROC curves as described by DeLong, DeLong, and Clarke-Pearson (1988).

Results

No significant differences were found between the three groups on age, F(2,134) = 2.80, p > .05, education, F(2,134) = 2.93, p > .05, or gender, F(2,134) = 0.05, p > .05. Therefore, further analyses were performed without demographic correction. One-way ANOVA revealed a significant difference between groups on List Retention, F(2,134) = 84.28, p < .01, and Story Retention, F(2,134) = 40.47, p < .01. According to post hoc analyses using the Bonferroni multiple comparisons, for each retention score, patients in the control group performed significantly better than patients in both the AD and aMCI groups. In addition, patients in the aMCI group obtained significantly higher retention scores than those in the AD group for both retention scores. Mean retention scores, in addition to raw RBANS subtest scores, for the three subject groups are presented in Table 1.

In order to examine the diagnostic accuracy of percent retention scores in distinguishing between controls, AD, and aMCI, indices of SN and SP were calculated for a range of cutoffs. In addition, LRs were calculated (see Tables 2–4). For distinguishing between controls and individuals with a diagnosis of AD, optimal cut scores were less than 60% on List Retention and less than 70% on Story Retention. These cut scores led to the correct classification of 94.6% (SN = 0.959, SP = 0.900) and 91.4% (SN = 0.904, SP = 0.950) of subjects, respectively. Examination of LRs indicate that an individual with a score of less than 60% on List Retention is almost 10 times more likely to have AD (LR+ = 9.59, 95% CI = 2.57–35.74). Furthermore, an individual with a score of 60% or above is 22 times more likely to not have this condition (LR− = 21.90, 95% CI = 7.16–66.96). LRs for a cutoff of 70% on Story Retention were 18.08 (95% CI = 2.67–122.33) for a positive test and 1.44 (4.86–20.18) for a negative test.

Table 2.

Diagnostic accuracy characteristics for retention scores: normal controls vs. AD

Ret. SN SP LR+ LR− 
List Learning Retention (List Recall/List Learning Trial 4) 
 <10% 0.808 (0.699–0.891) 1.000 — 5.21 (3.26–8.35) 
 <20% 0.849 (0.746–0.922) 1.000 — 6.64 (3.85–11.44) 
 <30% 0.904 (0.812–0.961) 1.000 — 10.43 (5.16–21.09) 
 <40% 0.918 (0.830–0.969) 1.000 — 12.17 (5.65–26.19) 
 <50% 0.945 (0.866–0.985) 1.000 — 18.25 (7.04–47.32) 
 <60% 0.959 (0.885–0.991) 0.900 (0.683–0.988) 9.59 (2.57–35.74) 21.90 (7.16–66.96) 
 <70% 0.973 (0.905–0.997) 0.750 (0.509–0.913) 3.89 (1.82–8.32) 27.38 (6.82–109.91) 
 <80% 0.986 (0.926–1.000) 0.650 (0.408–0.846) 2.82 (1.55–5.12) 47.45 (6.60–341.23) 
 <90% 1.000 0.100 (0.012–0.317) 1.11 (0.96–1.29) — 
Story Memory Retention (Story Recall/Story Memory Trial 2) 
 <10% 0.480 (0.361–0.600) 1.000 — 1.92 (1.54–2.39) 
 <20% 0.534 (0.414–0.652) 1.000 — 2.15 (1.68–2.75) 
 <30% 0.616 (0.495–0.728) 1.000 — 2.61 (1.95–3.49) 
 <40% 0.671 (0.551–0.777) 1.000 — 3.04 (2.19–4.22) 
 <50% 0.712 (0.595–0.812) 1.000 — 3.48 (2.42–4.99) 
 <60% 0.849 (0.746–0.922) 0.950 (0.751–0.999) 16.99 (2.51–115.03) 6.30 (3.62–10.97) 
 <70% 0.904 (0.812–0.961) 0.950 (0.751–0.999) 18.08 (2.67–122.33) 1.44 (4.86–20.18) 
 <80% 0.932 (0.847–0.977) 0.850 (0.621–0.968) 6.21 (2.18–17.66) 12.41 (5.22–29.50) 
 <90% 0.932 (0.847–0.977) 0.450 (0.231–0.685) 1.69 (1.13–2.53) 6.57 (2.48–17.42) 
 <100% 0.945 (0.866–0.985) 0.300 (0.119–0.543) 1.35 (1.01–1.81) 5.48 (1.71–17.54) 
Ret. SN SP LR+ LR− 
List Learning Retention (List Recall/List Learning Trial 4) 
 <10% 0.808 (0.699–0.891) 1.000 — 5.21 (3.26–8.35) 
 <20% 0.849 (0.746–0.922) 1.000 — 6.64 (3.85–11.44) 
 <30% 0.904 (0.812–0.961) 1.000 — 10.43 (5.16–21.09) 
 <40% 0.918 (0.830–0.969) 1.000 — 12.17 (5.65–26.19) 
 <50% 0.945 (0.866–0.985) 1.000 — 18.25 (7.04–47.32) 
 <60% 0.959 (0.885–0.991) 0.900 (0.683–0.988) 9.59 (2.57–35.74) 21.90 (7.16–66.96) 
 <70% 0.973 (0.905–0.997) 0.750 (0.509–0.913) 3.89 (1.82–8.32) 27.38 (6.82–109.91) 
 <80% 0.986 (0.926–1.000) 0.650 (0.408–0.846) 2.82 (1.55–5.12) 47.45 (6.60–341.23) 
 <90% 1.000 0.100 (0.012–0.317) 1.11 (0.96–1.29) — 
Story Memory Retention (Story Recall/Story Memory Trial 2) 
 <10% 0.480 (0.361–0.600) 1.000 — 1.92 (1.54–2.39) 
 <20% 0.534 (0.414–0.652) 1.000 — 2.15 (1.68–2.75) 
 <30% 0.616 (0.495–0.728) 1.000 — 2.61 (1.95–3.49) 
 <40% 0.671 (0.551–0.777) 1.000 — 3.04 (2.19–4.22) 
 <50% 0.712 (0.595–0.812) 1.000 — 3.48 (2.42–4.99) 
 <60% 0.849 (0.746–0.922) 0.950 (0.751–0.999) 16.99 (2.51–115.03) 6.30 (3.62–10.97) 
 <70% 0.904 (0.812–0.961) 0.950 (0.751–0.999) 18.08 (2.67–122.33) 1.44 (4.86–20.18) 
 <80% 0.932 (0.847–0.977) 0.850 (0.621–0.968) 6.21 (2.18–17.66) 12.41 (5.22–29.50) 
 <90% 0.932 (0.847–0.977) 0.450 (0.231–0.685) 1.69 (1.13–2.53) 6.57 (2.48–17.42) 
 <100% 0.945 (0.866–0.985) 0.300 (0.119–0.543) 1.35 (1.01–1.81) 5.48 (1.71–17.54) 

Notes: Ret. = % Retention; SN = sensitivity; SP = specificity; LR+ = likelihood ratio of a positive test; LR− = likelihood ratio of a negative test; AD = Alzheimer's disease; — = value could not be computed because the divisor was 0.

Table 3.

Diagnostic accuracy characteristics for retention scores: normal controls vs. aMCI

Ret. SN SP LR+ LR− 
List Learning Retention (List Recall/List Learning Trial 4) 
 <10% 0.227 (0.115–0.378) 1.000 — 1.29 (1.10–1.52) 
 <20% 0.273 (0.150–0.428) 1.000 — 1.38 (1.15–1.65) 
 <30% 0.409 (0.263–0.568) 1.000 — 1.69 (1.32–2.16) 
 <40% 0.500 (0.346–0.654) 1.000 — 2.00 (1.49–2.69) 
 <50% 0.568 (0.410–0.717) 1.000 — 2.32 (1.65–3.25) 
 <60% 0.636 (0.478–0.776) 0.900 (0.683–0.988) 6.36 (1.68–24.15) 2.48 (1.63–3.76) 
 <70% 0.818 (0.673–0.918) 0.878 (0.738–0.959) 6.71 (2.92–15.43) 4.83 (2.55–9.13) 
 <80% 0.864 (0.727–0.948) 0.650 (0.408–0.846) 2.47 (1.34–4.54) 4.77 (2.12–10.72) 
 <90% 0.909 (0.783–0.975) 0.100 (0.012–0.317) 1.01 (0.85–1.20) 1.10 (2.19–5.52) 
Story Memory Retention (Story Recall/Story Memory Trial 2) 
 <10% 0.114 (0.038–0.246) 1.000 — 1.13 (1.01–1.25) 
 <20% 0.136 (0.052–0.274) 1.000 — 1.16 (1.03–1.30) 
 <30% 0.182 (0.082–0.327) 1.000 — 1.22 (1.06–1.40) 
 <40% 0.296 (0.168–0.452) 1.000 — 1.42 (1.17–1.72) 
 <50% 0.364 (0.224–0.522) 1.000 — 1.57 (1.26–1.96) 
 <60% 0.455 (0.304–0.612) 0.950 (0.751–0.999) 9.09 (1.31–63.11) 1.74 (1.31–2.32) 
 <70% 0.568 (0.410–0.717) 0.950 (0.751–0.999) 11.36 (1.65–78.10) 2.20 (1.54–3.13) 
 <80% 0.750 (0.597–0.868) 0.850 (0.621–0.968) 5.00 (1.74–14.39) 3.40 (1.97–5.86) 
 <90% 0.796 (0.647–0.902) 0.450 (0.231–0.685) 1.45 (0.95–2.21) 2.20 (1.03–4.69) 
 <100% 0.864 (0.727–0.948) 0.300 (0.119–0.543) 1.23 (0.90–1.68) 2.20 (0.81–5.98) 
Ret. SN SP LR+ LR− 
List Learning Retention (List Recall/List Learning Trial 4) 
 <10% 0.227 (0.115–0.378) 1.000 — 1.29 (1.10–1.52) 
 <20% 0.273 (0.150–0.428) 1.000 — 1.38 (1.15–1.65) 
 <30% 0.409 (0.263–0.568) 1.000 — 1.69 (1.32–2.16) 
 <40% 0.500 (0.346–0.654) 1.000 — 2.00 (1.49–2.69) 
 <50% 0.568 (0.410–0.717) 1.000 — 2.32 (1.65–3.25) 
 <60% 0.636 (0.478–0.776) 0.900 (0.683–0.988) 6.36 (1.68–24.15) 2.48 (1.63–3.76) 
 <70% 0.818 (0.673–0.918) 0.878 (0.738–0.959) 6.71 (2.92–15.43) 4.83 (2.55–9.13) 
 <80% 0.864 (0.727–0.948) 0.650 (0.408–0.846) 2.47 (1.34–4.54) 4.77 (2.12–10.72) 
 <90% 0.909 (0.783–0.975) 0.100 (0.012–0.317) 1.01 (0.85–1.20) 1.10 (2.19–5.52) 
Story Memory Retention (Story Recall/Story Memory Trial 2) 
 <10% 0.114 (0.038–0.246) 1.000 — 1.13 (1.01–1.25) 
 <20% 0.136 (0.052–0.274) 1.000 — 1.16 (1.03–1.30) 
 <30% 0.182 (0.082–0.327) 1.000 — 1.22 (1.06–1.40) 
 <40% 0.296 (0.168–0.452) 1.000 — 1.42 (1.17–1.72) 
 <50% 0.364 (0.224–0.522) 1.000 — 1.57 (1.26–1.96) 
 <60% 0.455 (0.304–0.612) 0.950 (0.751–0.999) 9.09 (1.31–63.11) 1.74 (1.31–2.32) 
 <70% 0.568 (0.410–0.717) 0.950 (0.751–0.999) 11.36 (1.65–78.10) 2.20 (1.54–3.13) 
 <80% 0.750 (0.597–0.868) 0.850 (0.621–0.968) 5.00 (1.74–14.39) 3.40 (1.97–5.86) 
 <90% 0.796 (0.647–0.902) 0.450 (0.231–0.685) 1.45 (0.95–2.21) 2.20 (1.03–4.69) 
 <100% 0.864 (0.727–0.948) 0.300 (0.119–0.543) 1.23 (0.90–1.68) 2.20 (0.81–5.98) 

Notes: Ret. = % Retention; SN = sensitivity; SP = specificity; LR+ = likelihood ratio of a positive test; LR− = likelihood ratio of a negative test; aMCI = amnestic mild cognitive impairment; — = value could not be computed because the divisor was 0.

Table 4.

Diagnostic accuracy characteristics for retention scores: aMCI vs. AD

Ret. SN SP LR+ LR− 
List Learning Retention (List Recall/List Learning Trial 4) 
 <10% 0.808 (0.699–0.891) 0.773 (0.622–0.885) 3.56 (2.04–6.20) 4.03 (2.45–6.63) 
 <20% 0.849 (0.746–0.922) 0.727 (0.572–0.850) 3.11 (1.90–5.09) 4.83 (2.72–8.57) 
 <30% 0.904 (0.812–0.961) 0.718 (0.627–0.797) 1.53 (1.18–1.98) 4.27 (1.94–9.39) 
 <40% 0.918 (0.830–0.969) 0.500 (0.346–0.654) 1.84 (1.36–2.49) 6.08 (2.68–13.84) 
 <50% 0.945 (0.866–0.985) 0.432 (0.284–0.590) 1.66 (1.28–2.17) 7.88 (2.87–21.66) 
 <60% 0.959 (0.885–0.991) 0.364 (0.224–0.522) 1.51 (1.20–1.89) 8.85 (2.73–28.65) 
 <70% 0.973 (0.905–0.997) 0.182 (0.082–0.327) 1.19 (1.03–1.37) 6.64 (1.48–29.85) 
 <80% 0.986 (0.926–1.000) 0.136 (0.052–0.274) 1.14 (1.01–1.29) 9.95 (1.24–79.98) 
 <90% 1.000 0.091 (0.025–0.217) 1.10 (1.00–1.21) — 
Story Memory Retention (Story Recall/Story Memory Trial 2) 
 <10% 0.480 (0.361–0.600) 0.886 (0.754–0.962) 4.22 (1.79–9.96) 1.70 (1.33–2.17) 
 <20% 0.534 (0.414–0.652) 0.864 (0.727–0.948) 3.92 (1.81–8.49) 1.85 (1.41–2.43) 
 <30% 0.616 (0.495–0.728) 0.818 (0.673–0.918) 3.39 (1.77–6.51) 2.13 (1.55–2.94) 
 <40% 0.671 (0.551–0.777) 0.705 (0.548–0.832) 2.27 (1.40–3.69) 2.14 (1.47–3.13) 
 <50% 0.712 (0.595–0.812) 0.636 (0.478–0.776) 1.96 (1.29–2.97) 2.21 (1.45–3.38) 
 <60% 0.849 (0.746–0.922) 0.546 (0.389–0.696) 1.87 (1.33–2.62) 3.62 (1.97–6.65) 
 <70% 0.904 (0.812–0.961) 0.432 (0.284–0.590) 1.59 (1.22–2.08) 4.50 (2.06–9.84) 
 <80% 0.932 (0.847–0.977) 0.250 (0.132–0.403) 1.24 (1.04–1.49) 3.65 (1.36–9.81) 
 <90% 0.932 (0.847–0.977) 0.205 (0.098–0.353) 1.17 (0.996–1.38) 2.99 (1.07–8.34) 
 <100% 0.945 (0.866–0.985) 0.136 (0.052–0.274) 1.09 (0.96–1.25) 2.49 (0.74–8.33) 
Ret. SN SP LR+ LR− 
List Learning Retention (List Recall/List Learning Trial 4) 
 <10% 0.808 (0.699–0.891) 0.773 (0.622–0.885) 3.56 (2.04–6.20) 4.03 (2.45–6.63) 
 <20% 0.849 (0.746–0.922) 0.727 (0.572–0.850) 3.11 (1.90–5.09) 4.83 (2.72–8.57) 
 <30% 0.904 (0.812–0.961) 0.718 (0.627–0.797) 1.53 (1.18–1.98) 4.27 (1.94–9.39) 
 <40% 0.918 (0.830–0.969) 0.500 (0.346–0.654) 1.84 (1.36–2.49) 6.08 (2.68–13.84) 
 <50% 0.945 (0.866–0.985) 0.432 (0.284–0.590) 1.66 (1.28–2.17) 7.88 (2.87–21.66) 
 <60% 0.959 (0.885–0.991) 0.364 (0.224–0.522) 1.51 (1.20–1.89) 8.85 (2.73–28.65) 
 <70% 0.973 (0.905–0.997) 0.182 (0.082–0.327) 1.19 (1.03–1.37) 6.64 (1.48–29.85) 
 <80% 0.986 (0.926–1.000) 0.136 (0.052–0.274) 1.14 (1.01–1.29) 9.95 (1.24–79.98) 
 <90% 1.000 0.091 (0.025–0.217) 1.10 (1.00–1.21) — 
Story Memory Retention (Story Recall/Story Memory Trial 2) 
 <10% 0.480 (0.361–0.600) 0.886 (0.754–0.962) 4.22 (1.79–9.96) 1.70 (1.33–2.17) 
 <20% 0.534 (0.414–0.652) 0.864 (0.727–0.948) 3.92 (1.81–8.49) 1.85 (1.41–2.43) 
 <30% 0.616 (0.495–0.728) 0.818 (0.673–0.918) 3.39 (1.77–6.51) 2.13 (1.55–2.94) 
 <40% 0.671 (0.551–0.777) 0.705 (0.548–0.832) 2.27 (1.40–3.69) 2.14 (1.47–3.13) 
 <50% 0.712 (0.595–0.812) 0.636 (0.478–0.776) 1.96 (1.29–2.97) 2.21 (1.45–3.38) 
 <60% 0.849 (0.746–0.922) 0.546 (0.389–0.696) 1.87 (1.33–2.62) 3.62 (1.97–6.65) 
 <70% 0.904 (0.812–0.961) 0.432 (0.284–0.590) 1.59 (1.22–2.08) 4.50 (2.06–9.84) 
 <80% 0.932 (0.847–0.977) 0.250 (0.132–0.403) 1.24 (1.04–1.49) 3.65 (1.36–9.81) 
 <90% 0.932 (0.847–0.977) 0.205 (0.098–0.353) 1.17 (0.996–1.38) 2.99 (1.07–8.34) 
 <100% 0.945 (0.866–0.985) 0.136 (0.052–0.274) 1.09 (0.96–1.25) 2.49 (0.74–8.33) 

Notes: Ret. = % Retention; SN = sensitivity; SP = specificity; LR+ = likelihood ratio of a positive test; LR− = likelihood ratio of a negative test; aMCI = amnestic mild cognitive impairment; AD = Alzheimer's disease; — = value could not be computed because the divisor was 0.

Scores of less than 70% (84.7% correct classification rate, SN = 0.818, SP = 0.868) and less than 80% (78.1% correct classification rate, SN = 0.750, SP = 0.850) on List and Story Retention, respectively, provided optimal accuracy for distinguishing between cognitively normal controls and patients with aMCI. Participants with a score of less than 70% on List Retention were more than six times more likely to have aMCI (LR+ = 6.71; 95% CI = 2.92–15.43), whereas individuals with scores of 70% or more were more than four times more likely to not be diagnosed with aMCI (LR− = 4.83; 95% CI = 2.55–9.13), when compared with normal controls. When compared with normal controls, participants who obtained Story Retention scores of less than 80% were five times more likely to have aMCI (LR+ = 5.00; 95% CI = 1.74–14.39), whereas a score of 80% or more was associated with a three-fold increase in likelihood that the individual would not be diagnosed with aMCI (LR− = 3.40; 95% CI = 1.97–5.86).

Finally, optimal diagnostic accuracy indices for distinguishing between AD and aMCI were obtained with scores of less than 20% on List Retention and less than 50% on Story Retention. Utilization of these cut scores resulted in correct classification of 80.3% of subjects (SN = 0.849, SP = 0.727) and 68.4% of subjects (SN = 0.712, SP = 0.636), respectively. Individuals with a score of less than 20% on List Retention were three times more likely to have AD versus aMCI (LR+ = 3.11; 95% CI = 1.90–5.09), whereas those with scores of 20% or above were almost five times more likely to have aMCI rather than AD (LR− = 4.83, 95% CI = 2.72–8.57). On Story Retention, an individual with a score of less than 50% was approximately twice as likely to have AD versus aMCI (LR+ = 1.96; 95% CI = 1.29–2.97). Conversely, an individual with a score of 50% or more on Story Retention was twice as likely to have aMCI rather than AD (LR− = 2.21; 95% CI = 1.45–3.38).

Inspection of ROC curves and AUC statistics allows for estimation of overall diagnostic accuracy of a test. When distinguishing between controls and individuals with AD (see Fig. 1), the AUC of List Retention and Story Retention were 0.984 and 0.937, respectively, indicating excellent diagnostic accuracy. The AUCs of List Retention and Story Retention when distinguishing between controls and individuals with aMCI were 0.835 and 0.802, respectively (see Fig. 2), again demonstrating excellent diagnostic properties. Finally, when distinguishing between individuals with AD and those with aMCI, the AUCs for List Retention and Story Retention were 0.818 and 0.763, revealing adequate to excellent diagnostic accuracy (see Fig. 3). The AUCs for List Retention and Story Retention were compared with the diagnostic accuracy of the Delayed Memory Index, the standard score provided by RBANS normative data that serve as an overall indicator of delayed memory ability. Results showed that the AUCs for List Retention and Story Retention scores were not significantly different than those for the Delayed Memory Index scores when distinguishing between controls and individuals with AD, controls and individuals with aMCI, or individuals with aMCI and individuals with AD (ps > 0.05).

Fig. 1.

ROC curve: normal controls vs. AD. List Retention AUC = 0.984, Story Retention AUC = 0.937. AUC = area under the curve.

Fig. 1.

ROC curve: normal controls vs. AD. List Retention AUC = 0.984, Story Retention AUC = 0.937. AUC = area under the curve.

Fig. 2.

ROC curve: normal controls vs. aMCI. List Retention AUC = 0.835, Story Retention AUC = 0.802. AUC = area under the curve.

Fig. 2.

ROC curve: normal controls vs. aMCI. List Retention AUC = 0.835, Story Retention AUC = 0.802. AUC = area under the curve.

Fig. 3.

ROC curve: aMCI vs. AD. List Retention AUC = 0.818, Story Retention AUC = 0.763. AUC = area under the curve.

Fig. 3.

ROC curve: aMCI vs. AD. List Retention AUC = 0.818, Story Retention AUC = 0.763. AUC = area under the curve.

Discussion

The current study examined the diagnostic accuracy of RBANS percent retention scores using performance on the final trials of the List Learning and Story Memory subtests. This study expanded on previous research (Schoenberg et al., 2008) by providing specific information for clinicians regarding the diagnostic accuracy of RBANS retention when distinguishing between normal controls, individuals with aMCI, and individuals with AD. When compared with the recently published estimates of the diagnostic accuracy of standard RBANS index and subtests scores in detecting cognitive impairment associated with AD (Duff et al., 2008), the current estimates are comparable to or better than previously published estimates suggesting that these findings will provide valuable supplemental information to clinicians and researchers alike.

Results provide support for the use of percent retention scores when examining patient performance on RBANS memory subtests, particularly when detecting memory deficits associated with AD or MCI. As expected, individuals with AD had lower percent retention scores for both list and story memory than controls as well as individuals with a diagnosis of aMCI. In addition, individuals with aMCI obtained lower percent retention scores than did normal controls. SN and SP indices, as well as ROC curve analyses of the RBANS List Retention and Story Retention rate scores, suggest excellent diagnostic accuracy of these scores when distinguishing between controls, AD, and aMCI. The ability of RBANS retention scores to discriminate between patients with AD and those with aMCI is of particular interest. The diagnosis of MCI versus AD is made in the presence of cognitive impairment without significant impairment in daily functioning (Petersen et al., 1999). However, the boundary between these conditions is often blurred, as “impairment in daily functioning” is arguably difficult to define, thus complicating differential diagnosis. Furthermore, MCI, particularly aMCI, is associated with increased rates of conversion to AD, resulting in further complications in diagnostic decision-making. The current study, however, suggests that RBANS retention scores may add to the clinician's ability to distinguish cognitively between AD and aMCI.

The RBANS, developed as a brief for assessment and detection of dementia, has advantages over other commonly utilized neuropsychological screenings, including the ease and rapidity of use. Although retention scores did not provide more diagnostic accuracy than the commonly utilized Delayed Memory Index, the addition of retention rates extends the clinical and research utility of the RBANS by providing more sensitive data regarding acquisition and retrieval of verbal information. For example, retention rate information will most likely be beneficial for clinicians attempting to differentiate between different dementia syndromes as rapid decay of previously learned information is a hallmark of AD and not all other diseases. In addition to the common method of examining scores on the Immediate and Delayed Memory subtests separately, it is suggested that researchers and clinicians also utilize retention rates in order to formulate a more comprehensive clinical picture.

Several different limitations of the current study should be noted. First, the majority of the sample is Caucasian, and therefore results of the current study may not generalize to other ethnicities. Second, RBANS scores as presented here are not completely independent of aMCI and AD diagnoses, as these scores are considered by the consensus team when making diagnostic decisions. As described in Duff and colleagues (2008), however, other information, including clinical interview, lab results, neuroimaging, and multiple additional clinical neuropsychological measures, are also considered in the decision-making process and support the diagnoses of aMCI or AD. Subjects' performances on additional memory measures are provided in Table 1 for comparison. Finally, diagnoses in the current sample were determined by a consensus team, the gold standard in pre-mortem diagnosis of AD. However, retention rates for the RBANS can be further validated through well-defined patient samples via post-mortem autopsy and/or use of MRI. Future studies may also illuminate the utility of these scores in other diagnostic samples, such as among individuals with vascular dementia or traumatic brain injuries, and in distinguishing between different neurological disorders.

Despite these limitations, however, the current study adds to the clinical and research utility of the RBANS by providing clinicians and researchers with a more sensitive and informative tool for assessing verbal memory. Future large-scale studies should be conducted to provide normative data on RBANS retention scores among various populations.

Conflict of Interest

None declared.

References

American Psychiatric Association
Diagnostic and statistical manual of mental disorders
 , 
2000
Washington, DC
American Psychiatric Association
 
(4th ed., text revision)
Aupperle
R. L.
Beatty
W. W.
Shelton
F. N.
Gontkovsky
S. T.
Three screening batteries to detect cognitive impairment in multiple sclerosis
Multiple Sclerosis
 , 
2002
, vol. 
8
 (pg. 
382
-
389
)
Chui
H. C.
Victoroff
J. I.
Margolin
D.
Jagust
W.
Shankle
R.
Katzman
R.
Criteria for the diagnosis of ischemic vascular dementia proposed by the State of California Alzheimer's Disease Diagnostic and Treatment Centers
Neurology
 , 
1992
, vol. 
42
 (pg. 
473
-
480
)
Crook
T.
Bartus
R. T.
Ferris
S. H.
Whitehouse
P.
Cohen
G. D.
Gershon
S.
Age-associated memory impairment: Proposed diagnostic criteria and measures of clinical change. Report of a National Institute of Mental Health workgroup
Developmental Neuropsychology
 , 
1986
, vol. 
2
 (pg. 
261
-
276
)
DeLong
E. R.
DeLong
D. M.
Clarke-Pearson
D. L.
Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach
Biometrics
 , 
1988
, vol. 
44
 (pg. 
837
-
845
)
Duff
K.
Humphreys Clark
J. D.
O'Bryant
S. E.
Mold
J. W.
Schiffer
R. B.
Sutker
P. B.
Utility of the RBANS in detecting cognitive impairment associated with Alzheimer's disease: Sensitivity, specificity, and positive and negative predictive powers
Archives of Clinical Neuropsychology
 , 
2008
, vol. 
23
 (pg. 
603
-
612
)
Ferman
T. J.
Smith
G. E.
Boeve
B. F.
Graff-Radford
N. R.
Lucas
J. A.
Knopman
D. S.
, et al.  . 
Neuropsychological differentiation of dementia with Lewy bodies from normal aging and Alzheimer's disease
The Clinical Neuropsychologist
 , 
2006
, vol. 
20
 (pg. 
623
-
626
)
Folstein
M. F.
Folstein
S. E.
McHugh
P. R.
Mini-mental state
Journal of Psychiatric Research
 , 
1975
, vol. 
12
 (pg. 
189
-
198
)
Freilich
B. M.
Hyer
L. A.
Relation of the Repeatable Battery for Assessment of Neuropsychological Status to measures of daily functioning in dementia
Psychological Reports
 , 
2007
, vol. 
101
 (pg. 
119
-
129
)
Gallagher
C.
Burke
T.
Age, gender and IQ effects on the Rey–Osterrieth Complex Figure Test
British Journal of Clinical Psychology
 , 
2007
, vol. 
46
 (pg. 
35
-
45
)
Gold
J. M.
Queern
C.
Iannone
V. N.
Buchanan
R. W.
Repeatable Battery for the Assessment of Neuropsychological Status as a screening test in schizophrenia, I. Sensitivity, reliability, and validity
American Journal of Psychiatry
 , 
1999
, vol. 
156
 (pg. 
1944
-
1950
)
[PubMed]
Gontkovsky
S. T.
Beatty
W. W.
Mold
J. W.
Repeatable Battery for the Assessment of Neuropsychological Status in a normal, geriatric sample
Clinical Gerontologist
 , 
2004
, vol. 
27
 (pg. 
79
-
86
)
Gontkovsky
S. T.
Hillary
F. G.
Scott
J. G.
Cross-validation and test sensitivity of the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS)
Journal of Cognitive Rehabilitation
 , 
2002
, vol. 
20
 (pg. 
26
-
31
)
Griffith
H. R.
Netson
K. L.
Harrell
L. E.
Zamrini
E. Y.
Brockington
J. C.
Marson
D. C.
Amnestic mild cognitive impairment: Diagnostic outcomes and clinical prediction over a two-year time period
Journal of the International Neuropsychological Society
 , 
2006
, vol. 
12
 (pg. 
166
-
175
)
Hobart
M. P.
Goldberg
R.
Bartko
J. J.
Gold
J. M.
Repeatable Battery for the Assessment of Neuropsychological Status as a screening test in schizophrenia, II. Convergent/discriminant validity and diagnostic group comparisons
American Journal of Psychiatry
 , 
1999
, vol. 
156
 (pg. 
1951
-
1957
)
[PubMed]
Hughes
C. P.
Berg
L.
Danziger
W. L.
Cohen
L. A.
Martin
R. L.
A new clinical scale for the staging of dementia
British Journal of Psychiatry
 , 
1982
, vol. 
140
 (pg. 
566
-
572
)
Larson
E. B.
Kirschner
K.
Bode
R.
Heinemann
A.
Goodman
R.
Construct and predictive validity of the Repeatable Battery for the Assessment of Neuropsychological Status in the evaluation of stroke patients
Journal of Clinical and Experimental Neuropsychology
 , 
2005
, vol. 
27
 (pg. 
16
-
32
)
McKay
C.
Casey
J. E.
Wertheimer
J.
Fichtenberg
N. L.
Reliability and validity of the RBANS in a traumatic brain injured sample
Archives of Clinical Neuropsychology
 , 
2007
, vol. 
22
 (pg. 
91
-
98
)
McKhann
G.
Drachman
D.
Folstein
M.
Katzman
R.
Price
D.
Stadlan
E. M.
Clinical diagnosis of Alzheimer's disease: Report of the NINCDS-ADRDA Work Group under the auspices of Department of Health and Human Services Task Force on Alzheimer's Disease
Neurology
 , 
1984
, vol. 
34
 (pg. 
939
-
944
)
Pachet
A. K.
Construct validity of the Repeatable Battery of Neuropsychological Status (RBANS) with acquired brain injury patients
The Clinical Neuropsychologist
 , 
2007
, vol. 
21
 (pg. 
286
-
293
)
Petersen
R.
Negash
S.
Mild cognitive impairment: An overview
CNS Spectrums
 , 
2008
, vol. 
13
 (pg. 
45
-
53
)
[PubMed]
Petersen
R.
Smith
G.
Waring
S.
Ivnik
R.
Tangalos
E.
Kokmen
E.
Mild cognitive impairment: Clinical characterization and outcome
Archives of Neurology
 , 
1999
, vol. 
56
 (pg. 
303
-
308
)
Randolph
C.
RBANS: Repeatable Battery for the Assessment of Neuropsychological Status
 , 
1998
San Antonio, TX
The Psychological Corporation
Randolph
C.
Tierney
M. C.
Mohr
E.
Chase
T. N.
The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS): Preliminary clinical validity
Journal of Clinical and Experimental Neuropsychology
 , 
1998
, vol. 
20
 (pg. 
310
-
319
)
Schoenberg
M. R.
Duff
K.
Beglinger
L. J.
Moser
D. J.
Bayless
J. D.
Mold
J.
, et al.  . 
Retention rates on RBANS memory subtests in elderly adults
Journal of Geriatric Psychiatry and Neurology
 , 
2008
, vol. 
21
 (pg. 
26
-
33
)
Wechsler
D.
Wechsler Memory Scale—Third Edition.
 , 
1997
San Antonio, TX
The Psychological Corporation
Wicklund
A. H.
Johnson
N.
Rademaker
A.
Weitner
B. B.
Weintraub
S.
Word list versus story memory in Alzheimer disease and frontotemporal dementia
Alzheimer Disease and Related Disorders
 , 
2006
, vol. 
20
 (pg. 
86
-
92
)
Wilde
M. C.
The validity of the repeatable battery of neuropsychological status in acute stroke
The Clinical Neuropsychologist
 , 
2006
, vol. 
20
 (pg. 
702
-
715
)
World Health Organization
Alzheimer's disease: The brain killer
 , 
2006