Abstract

Using a sample of 44 clinically referred, non-litigating, older adults, we evaluated the Repeatable Battery for the Assessment of Neuropsychological Status Effort Index [RBANS EI; Silverberg, N. D., Wertheimer, J. C., & Fichtenberg, N. L. (2007). An effort index for the Repeatable Battery for the Assessment of Neurospcyhological Status (RBANS). The Clinical Neuropsychologist, 21, 841–854]. With the current RBANS EI cut-score guidelines, 31% of our sample was classified as putting forth suspect effort. With this, cognitive ability was significantly correlated with suspect effort scores. Thus, it appears that the current guidelines may not be useful in a cognitively impaired medically ill geriatric sample. Hence, further research on RBANS EI validation is warranted.

Introduction

It is crucial to understand the validity of the patient's responses when interpreting neuropsychological data (see Bush et al., 2005). Although most clinicians appreciate the importance of formally assessing effort when there is salient gain (e.g., financial reward or disability), it is also of assistance in other contexts (e.g., oppositional patients or medically referred patients). During the assessment, the onus is on the clinician to make the determination as to whether the patient is putting forth adequate effort, as it is the rare instance that the patient will volunteer this information.

There are a number of ways to evaluate effort during clinical assessment. For example, clinicians can subjectively decide whether individuals are putting forth adequate effort during an evaluation, though research suggests that this approach alone is at best questionable (Faust, Hart, Guilmette, & Arkes, 1988; Heaton, Smith, Lehman, & Vogt, 1978). There are also objective tests designed to only detect suboptimal effort (e.g., Tombaugh, 1996), as well as indices embedded within standard neuropsychological measures that identify those not putting forth their best effort (e.g., Root, Robbins, Chang, & van Gorp, 2006). Certainly, stand-alone measures of effort are useful, but in most cases these measures do not provide meaningful information regarding a patient's cognitive functioning. The appeal of embedded measures stems from a single measure being both sensitive to effort and able to provide well-validated information regarding cognitive functioning.

In-line with the latter approach, Silverberg, Wertheimer, & Fichtenberg (2007) developed a method to evaluate insufficient effort on a screening measure typically used for brief cognitive evaluation, the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS; Randolph, 1998). Silverberg and colleagues modeled their embedded Effort Index (EI) for the RBANS after forced-choice word list tasks and Digit Span tasks that have been shown to be proxy-markers of effort (e.g., Axelrod, Fichtenberg, Millis, & Wertheimer, 2006; Millis, Putnam, Adams, & Ricker, 1995). In the development of the RBANS EI, the authors first investigated the proportion of a heterogeneous clinical sample that scored below various raw score cut-points on RBANS Word List Recognition and Digit Span subtests. A scoring system was devised using the two subtests because many individuals scored below cut-scores on one of the subtests, but not the other. As a next step in validation of the RBANS EI, Silverberg and coworkers (2007) investigated five different groups' performances: (1) clinical traumatic brain injury; (2) clinical malingering; (3) simulated-naïve malingerers; (4) simulated-coached malingerers; and (5) controls. They reported that an RBANS EI cut-score greater than 3 should be considered suspicious. Noteworthy, the authors excluded persons with serious cognitive impairments in their initial investigation, a population in which the RBANS is frequently administered.

Initial RBANS EI research is promising, and the goal of this study was to investigate the frequency of RBANS EI scores in a diverse clinical sample of medically ill older adults with no known pending litigation. We had particular interest in this sample because the RBANS was originally developed and intended for use with older adults (Randolph, 1998). Although it was expected that our sample would obtain lower RBANS Word List Recognition and Digit Span scores given diminished cognitive abilities, we were curious whether these individuals would obtain scores suggestive of suspicious effort.

Materials and Methods

Participants

The present study is an archival review of patients referred for neuropsychological evaluation at an urban medical center between the years of 2004 and 2006. Patients were included in the study if they were 60 years of age or older, native English speakers, and completed the RBANS Form A as part of their clinical evaluation. Individuals suspected to be involved in litigation were excluded. Not all patients were administered each measure as part of their clinical battery, and therefore there are minor sample size differences across measures.

Forty-four patients met inclusion criteria for the study. Participants were between 61 and 88 years of age (M = 74.87 years; SD = 7.28), and had obtained between 8 and 21 years of formal education (M = 13.37 years; SD = 3.03). Most participants were right-handed (91%, n = 40), women (55%, n = 24), married (50%, n = 22), and Caucasian (64%, n = 28).

Most patients were referred for clinical neuropsychological assessment related to global mental status changes (46%, n = 20) or memory decline (25%, n = 11). On average, patients had four incoming medical diagnoses (SD = 2.87), and 50% of these participants had one to two incoming neurological diagnoses. The most common neurological diagnoses were cerebral vascular accident (25%, n = 11) and head injury (11%, n = 5). One patient reported current alcohol abuse and no patient reported current use of illicit substances.

Measures

Geriatric Depression Scale-15 items (GDS-15; Sheik & Yesavage, 1986)

The GDS-15 is a valid measure of depressive symptoms in older adults (for further review see Arthur, Jagger, Lindesay, Graham, & Clarke, 1999). This scale contains 15 yes/no questions, which can be self- or interviewer-administered in approximately 5 min. A score at or above 5 is commonly used to indicate depression (Meara et al., 1999).

Mini-Mental State Examination (MMSE; Folstein et al., 1975)

The MMSE is a commonly administered measure of general cognitive functioning. This 30-item measure includes tasks dependent on orientation, memory, attention, language functioning, and visual spatial skills. Scores below 23 indicate cognitive impairment (Cummings, 1993).

Repeatable Battery for the Assessment of Neuropsychological Status (RBANS; Randolph, 1998)

The RBANS was developed to evaluate functioning across multiple domains and, over the last decade there have been a number of encouraging publications that document the RBANS adequate psychometric properties and value in clinical settings (e.g., Duff, Humphreys Clark, O'Bryant, Mols, Schiffer, & Sutker, 2008). The RBANS is comprised of 12 subtests: List Learning, Story Memory, Digit Span, Coding, Figure Copy, Line Orientation, Picture Naming, Semantic Fluency, List Recall, List Recognition, Story Recall, and Figure Recall. These subtests yield index scores and a total rating score. The RBANS EI, as proposed by Silverberg and coworkers (2007) is based on the performance of individuals on RBANS Digit Span and Word List Recognition subtests. Silverberg and coworkers (2007) assigned weighted scores (ranging from 0 to 6) to different raw scores on these subtests (see Silverberg et al., 2007 for further information on weighted scores). The RBANS EI is computed by converting individual raw scores for the two aforementioned subtests to weighted scores, and summing the two weighted scores. Hence, theoretically the RBANS EI score for a given individual can range from 0 to 12. RBANS EI scores greater than 3 should raise suspicion regarding the level of effort put forth by a patient during testing.

Wechsler Test of Adult Reading (WTAR; The Psychological Corporation, 2001)

The WTAR is a 50-item word reading test that can be used with individuals aged 16–89. WTAR performance is a reasonable estimate of premorbid functioning because reading recognition is highly correlated with intellectual functioning and resilient to normal cognitive changes associated with aging.

Procedures

IRB approval was obtained prior to archival data collection. Inclusion criteria for the study are detailed in the Materials and Methods section. All evaluations were performed or supervised by clinical neuropsychology faculty. All tests were administered and scored by neuropsychology faculty, psychology pre-doctoral interns, or psychometrists in accordance with the standardized administration procedures.

Results

Premorbid intelligence for the majority of the sample was within the average range (WTAR Estimated Full Scale IQ: M = 97.76; SD = 12.80; range = 72–114; n = 37). The sample was generally not depressed (GDS-15: M = 3.44; SD = 2.87; range = 0–11; n = 36). Participants had a mean MMSE total score of 24.67 (SD = 3.70; range = 16–30; n = 31) and a mean RBANS Total Scaled Score of 68.30 (SD = 13.82; range = 45–100; n = 44).

RBANS EI scores for each participant were calculated following the procedure outlined in Silverberg and coworkers (2007). Forty-four patients completed the RBANS Digit Span (raw score M = 8.86; SD = 2.13; range = 5–16) and RBANS List Recognition (raw score M = 15.09; SD = 3.19; range = 9–20) subtests. In our sample, RBANS EI scores ranged from 0 to 8. Table 1 reports the absolute and cumulative percentage of RBANS EI scores in the sample. As illustrated in Table 1, 31% of our sample obtained RBANS EI scores greater than 3. This finding indicates that scores of this magnitude are relatively common in a clinical population of medically ill older adults, who were not known to be litigating.

Table 1

Percentage of participants by Effort Index Score on RBANS-A

RBANS EI score n Percentage of sample
 
  Absolute Cumulative 
19.0 19.0 
2.4 21.4 
13 31.0 52.4 
16.7 69.0 
16.7 85.7 
7.1 92.9 
2.4 95.2 
0.0 95.7 
4.8 100.0 
RBANS EI score n Percentage of sample
 
  Absolute Cumulative 
19.0 19.0 
2.4 21.4 
13 31.0 52.4 
16.7 69.0 
16.7 85.7 
7.1 92.9 
2.4 95.2 
0.0 95.7 
4.8 100.0 

Note: N = 44.

MMSE scores and RBANS Total Score were both significantly negatively correlated with the RBANS EI score (respectively, r = −.46, p = .017, n = 31; r = −.58, p < .0001, n = 44). Age at the time of testing, years of education, GDS scores, and number of medical diagnoses were not significantly correlated with the RBANS EI score.

Given the significant relationships between the RBANS EI score and both the RBANS total score and MMSE score, we wanted to further examine the effects of impaired cognitive functioning on RBANS EI score. Using extreme subsets of the participants, we grouped patients into a cognitively impaired group (n = 6) that was defined as those who obtained a RBANS Total Scaled Score <70 and a total MMSE Score <23 and a non-cognitively impaired group of patients (n = 9) who obtained a RBANS Total Scaled Score ≥70 and an MMSE Score ≥23. Table 2 reports frequencies of RBANS EI scores for each respective subsample. In the non-cognitively impaired group no one scored above the RBANS EI cut-score of 3, but in the cognitively impaired group half the sample scored above the cut-score. Although these subsamples are limited in size, results do suggest that cognitive impairment can at least partially explain observed extreme RBANS EI scores in this sample.

Table 2

RBANS EI scores in participants with and without cognitive impairment

RBANS EI score Non-cognitively impaired (n = 9)
 
Cognitively impaired (n = 6)
 
 n Percentage n Percentage 
33.3 16.7 
11.1 – – 
33.3 – – 
22.3 33.3 
– – 16.7 
– – – – 
– – 16.7 
– – – – 
– – 16.7 
RBANS EI score Non-cognitively impaired (n = 9)
 
Cognitively impaired (n = 6)
 
 n Percentage n Percentage 
33.3 16.7 
11.1 – – 
33.3 – – 
22.3 33.3 
– – 16.7 
– – – – 
– – 16.7 
– – – – 
– – 16.7 

Notes: Non-cognitively impaired, RBANS Total Scaled Score ≥ 70 and MMSE Total Score ≥ 23; Cognitively impaired, RBANS Total Scaled Score < 70 and MMSE Total Score < 23.

Discussion

Embedded effort indices are developed within standard neuropsychological tests to assess a patient's effort and motivation. Silverberg and coworkers (2007) were the first to develop such a measure for the RBANS. In an attempt to further understand the utility of the RBANS EI, we examined the frequency of RBANS EI scores in a medically ill older adult sample who were not known to be litigating. This sample differs from the study of Silverberg and coworkers, as our sample is generally older, more medically ill, and has greater cognitive dysfunction. Our findings show that 31% of our sample fall above the cut score of 3. With this in mind, general cognitive ability did significantly correlate with the RBANS EI score, but age, depression, and medical illness did not. Further, when we examined those with serious cognitive impairment relative to those with less cognitive difficulty, no one who obtained an RBANS Total Scaled Score ≥ 70 and an MMSE total score ≥ 23 obtained an RBANS EI score greater than 3. Although this subsample was limited in size, it potentially can assist clinicians in understanding why extreme RBANS EI scores are likely to be obtained in similar samples.

Similar to previous research (e.g., Howe, Anderson, Kaufman, Sachs, & Loring, 2007), this study illustrates that patients with severe cognitive impairment have an increased likelihood of performing in the suspicious range on effort testing. Given these findings, clinicians should be cautious interpreting the RBANS EI when an individual has significant cognitive impairment. This research does support recommended RBANS EI cut-score guidelines with medically ill older adults who do not exhibit severe cognitive impairments.

Certainly, additional research is needed to document the sensitivity, specificity, and predictive power of different RBANS EI cut-scores in similar samples, and to provide further recommendation regarding more appropriate cut-score guidelines. A logical next step in better understanding the clinical utility of the RBANS EI might include an investigation incorporating administration of both the RBANS and a frequently administered measure of effort in an older adult sample. Such a study would permit clinicians to observe whether there is convergent evidence that the RBANS EI is a useful embedded index of effort.

Conflict of Interest

None declared.

References

Arthur
A.
Jagger
C.
Lindesay
J.
Graham
C.
Clarke
M.
Using an annual over-75 health check to screen for depression: Validation of the short geriatric depression scale (GDS15) within general practice
International Journal of Geriatric Psychiatry
 , 
1999
, vol. 
14
 (pg. 
431
-
439
)
Axelrod
B. N.
Fichtenberg
N. L.
Millis
S. R.
Wertheimer
J. C.
Detecting incomplete effort with Digit Span from the Wechsler Adult Intelligence Scale—Third Edition
The Clinical Neuropsychologist
 , 
2006
, vol. 
20
 (pg. 
513
-
523
)
Bush
S. S.
Ruff
R. M.
Tröster
A. I.
Barth
J. T.
Koffler
S. P.
Pliskin
N. H.
, et al.  . 
Symptom validity assessment: Practice issues and medical necessity
Archives of Clinical Neuropsychology
 , 
2005
, vol. 
20
 (pg. 
419
-
426
)
Cummings
J. L.
Mini-Mental State Examination: Norms, normals, and numbers
Journal of the American Medial Association
 , 
1993
, vol. 
12
 (pg. 
2420
-
2421
)
Duff
K.
Humphreys Clark
J. D.
O'Bryant
S. E.
Mold
J. W.
Schiffer
R. B.
Sutker
P. B.
Utility of the RBANS in detecting cognitive impairment associated with Alzheimer's disease: Sensitivity, specificity, and positive and negative predictive powers
Archives of Clinical Neuropsychology
 , 
2008
, vol. 
23
 (pg. 
603
-
612
)
Faust
D.
Hart
K.
Guilmette
T.
Arkes
Neuropsychologist's capacity to detect adolescent malingering
Professional Psychology: Research and Practice
 , 
1988
, vol. 
19
 (pg. 
508
-
515
)
Folstein
M. F.
Folstein
S. E.
McHugh
R. R.
‘Mini-Mental State’: A practical method for grading the cognitive state of patients for the clinician
Journal of Psychiatric Research
 , 
1975
, vol. 
12
 (pg. 
189
-
198
)
Heaton
R. K.
Smith
H. H.
Lehman
A. W.
Vogt
A. T.
Prospects for faking believable deficits on neuropsychological testing
Journal of Consulting and Clinical Psychology
 , 
1978
, vol. 
46
 (pg. 
892
-
900
)
Howe
L. L. S.
Anderson
A. M.
Kaufman
D. A. S.
Sachs
B. C.
Loring
D. W.
Characterization of the Medical Symptom Validity Test in evaluation of clinically referred memory disorders clinic patients
Archives of Clinical Neuropsychology
 , 
2007
, vol. 
22
 (pg. 
753
-
761
)
Meara
J.
Mitchelmore
E.
Hobson
P.
Use of the GDS-15 geriatric depression scale as a screening instrument for depressive symptomatology in patients with Parkinson's disease and their carers in the community
Age and Ageing
 , 
1999
, vol. 
28
 (pg. 
35
-
38
)
Millis
S. R.
Putnam
S. H.
Adams
K. M.
Ricker
J. H.
The California Verbal Learning Test in the detection of incomplete effort in neuropsychological evaluation
Psychological Assessment
 , 
1995
, vol. 
7
 (pg. 
463
-
471
)
Randolph
C.
Repeatable Battery for the Assessment of Neuropsychological Status Manual
 , 
1998
San Antonio, TX
The Psychological Corporation
Root
J. C.
Robbins
R. N.
Chang
L.
van Gorp
W.
Detection of inadequate effort on the California Verbal Learning Test (2nd ed.): Forced choice recognition and critical item analysis
Journal of the International Neuropsychological Society
 , 
2006
, vol. 
12
 (pg. 
688
-
696
)
Sheik
J. I.
Yesavage
J. A.
Geriatric Depression Scale (GDS): Recent evidence and development of a shorter version
Clinical Gerontologist
 , 
1986
, vol. 
37
 (pg. 
819
-
820
)
Silverberg
N. D.
Wertheimer
J. C.
Fichtenberg
N. L.
An effort index for the Repeatable Battery for the Assessment of Neurospcyhological Status (RBANS)
The Clinical Neuropsychologist
 , 
2007
, vol. 
21
 (pg. 
841
-
854
)
The Psychological Corporation
Wechsler Test of Adult Reading
 , 
2001
San Antonio, TX
Harcourt Assessment
Tombaugh
T. N.
The Test of Memory Malingering
 , 
1996
Toronto, Ontario, Canada
Multi-Health Symptoms