Abstract

The current study examined two embedded response bias measures in the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS), the Effort Index (EI) and Effort Scale (ES), in relation to Malingered Neurocognitive Dysfunction criteria. We examined 105 individuals undergoing compensation-seeking disability evaluations. The results suggest the EI adequately differentiates the Probable/Definite Malingering group from the Incentive Only and Possible Malingering groups, while the ES does not, which is most likely representative of the current sample of disability litigants rather than its intended population of patients with amnesia. Classification accuracy statistics suggest that while the EI may not be an appropriate stand-alone measure in detecting neurocognitive malingering, it shows utility as a complementary or screening measure in forensic settings.

Introduction

All neuropsychological evaluations are susceptible to response bias in the form of poor effort. Consequently, the assessment of response bias and malingering should be incorporated into every evaluation. A particular area where this is especially important is in forensic settings, where secondary gains are often present. This crucial part of forensic neuropsychology is mandated by practice guidelines developed by the National Academy of Neuropsychology (Bush et al., 2005) and the American Academy of Clinical Neuropsychology (Heilbronner et al., 2009). In neuropsychological testing, good effort by the test takers is required to ascertain the extent of cognitive impairment (Bianchini, Mathias, & Greve, 2001). Impairment is established by testing the limits of an individual's ability. If the individual does not perform at her or his maximum ability, test results will confound functional deficits and poor effort, potentially resulting in an underestimate of cognitive functioning or an overestimate of cognitive impairment. The potential for skewed results in neuropsychological testing make it essential to measure the individual's performance and effort when taking these tests.

Several approaches have been developed to assess cognitive response bias, the most common being performance validity tests (PVTs). Another strategy for assessing cognitive response bias is the use of indices embedded in traditional neuropsychological measures, such as the Reliable Digit Span in the WAIS (Wechsler, 1981). This approach is efficient because it can provide retrospective measures of response bias calculated from previous neuropsychological testing.

The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS; Randolph, 1998) was originally developed as a screener to assist in assessing dementia, although it has also exhibited utility as a neuropsychological screening tool for Parkinson's disease (Beatty et al., 2003) and Huntington's disease (Randolph, Tierney, Mohr, & Chase, 1998).

More recently, two embedded validity indicators have been developed for the RBANS, the Effort Index (EI; Silverberg, Wertheimer, & Fichtenberg, 2007) and the Effort Scale (ES; Novitski, Steele, Karantzoulis, & Randolph, 2012). The EI utilizes two subtests of the RBANS, digit span and list recognition, which have been validated as measures of effort in other neuropsychological measures (e.g., Young, Baughman, & Roper, 2012). Weighted scores are calculated based on the combination of both the digit span and list recognition raw scores, because these measures of effort exhibit utility in discriminating between legitimate and feigned cognitive impairment in patients with mild traumatic brain injury (mTBI) (Silverberg et al., 2007). The EI has also been examined in relation to stand-alone PVTs in detecting suboptimal effort in a military sample, which resulted in excellent specificity and modest sensitivity, which produces few false positives at the expense of more false negatives (Armistead-Jehle & Hansen, 2011).

The ES utilizes the same subtests of the RBANS as the EI, but also implements the removal of the total free recall scores (the sum of List Recall, Story Recall, and Figure Recall) from the List Recognition score. The ES was developed to discriminate between poor effort and bona fide amnesia, with the notion that free recall performance would deteriorate at a faster rate than recognition memory in amnestic patients. Thus, removing the free recall scores would better discriminate between patients with amnesia and patients with mTBI who exhibited poor effort on collateral measures because, the amnestic patients' working memory would still be intact. The ES demonstrated excellent specificity in differentiating between patients with mTBI and patients with amnesia (Novitski et al., 2012).

The current study builds upon previous research by examining the ability of the EI and ES to detect poor effort in a sample of disability litigants who were classified using established criteria for Malingered Neurocognitive Dysfunction (MND; Slick, Sherman, & Iverson, 1999). The MND criteria were developed to assess MND and classify malingering dimensionally, with the possibilities of possible, probable, and definite malingering, depending on the amount of evidence in the various criteria. Our goal is to determine whether the EI and ES embedded in the RBANS can adequately differentiate probable and definite malingerers from individuals who show no signs of malingering, but nevertheless still possess the external incentive of receiving disability or worker's compensation. We expect to find that the probable and definite malingerers will exhibit poorer effort on both the EI and ES than the incentive only participants.

Method

Participants

The current sample consisted of 105 disability litigants undergoing evaluations with a forensic neuropsychiatrist and his psychology staff. The sample was predominantly Caucasian (98%), male (58%) with a mean age of 42.5 years (SD = 10.7) and mean education of 12.3 years (SD = 1.8). Seventy-two percent of the litigants were being evaluated for worker's compensation, 17% for personal injury, 7% for head injury, 2% for medical malpractice, and 2% were general disability evaluations. Sixty-two percent of the evaluations were conducted for the defense and 38% were conducted for the plaintiff. All participants gave permission to use their data anonymously for research purposes.

Measures

Repeatable Battery for the Assessment of Neuropsychological Status (RBANS; Randolph, 1998): The RBANS is a widely used neuropsychological measure originally developed to assess dementia. Consisting of 10 subtests, the RBANS provides a Total Score as well as index scores for immediate and delayed memory, attention, language, and visuospatial skills. More recently, two effort indices were constructed using scales embedded in the RBANS. These effort indices include the EI (Silverberg et al., 2007) and the ES (Novitski et al., 2012), which have exhibited good sensitivity and specificity in detecting poor effort on the RBANS.

Test of Memory Malingering (TOMM; Tombaugh, 1996). The TOMM is a forced-choice visual recognition task used for performance validity. The TOMM has exhibited excellent sensitivity and specificity in discerning people with legitimate neurocognitive impairments from those instructed to feign these impairments (Rees, Tombaugh, Gansler, & Moczynski, 1998).

Victoria Symptom Validity Test (VSVT; Slick, Hopp, Strauss, & Thompson, 1997). The VSVT is a computer administered number recognition task that determines performance classifications as valid, questionable, or invalid (i.e., below chance). The VSVT has displayed excellent sensitivity (i.e., 83%) and specificity (i.e., 100%) in differentiating healthy and non-litigating patients from those asked to feign post-concussion syndrome (Slick, Hopp, Strauss, Hunter, & Pinch, 1994).

Letter Memory Test (LMT; Inman, Vickery, Berry, Lamb, Edwards, & Smith, 1998). The LMT is a computer administered forced-choice letter recognition task. It has demonstrated excellent sensitivity among malingerers (i.e., 0.84) and TBI patients (i.e., 0.95) and specificity for non-litigating neurological patients (i.e., 1.0) (Inman et al., 1998).

Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF; Ben-Porath & Tellegen, 2008). The MMPI-2-RF is a restructured version of the MMPI-2 that was developed in line with contemporary views of personality and psychopathology. The measure is 338 items and utilizes a subset of the MMPI-2 normative sample. The current study utilized the five over-reporting validity scales. The MMPI-2-RF has been examined in relation to structured malingering criteria using these over-reporting validity scales, which have exhibited utility in discriminating between malingerers and non-malingerers (Tarescavage, Wygant, Gervais, & Ben-Porath, 2012).

Procedure

Participants gave permission for their data to be used in the study before they were administered the RBANS, VSVT, TOMM, LMT, and the five over-reporting validity scales on the MMPI-2-RF by masters level psychological associates under the supervision of a licensed doctoral psychologist. All of these measures were scored and interpreted using their respective manuals.

Malingering Classification. Utilizing these symptom validity measures, we classified the participants based on the MND (Slick, Sherman, & Iverson, 1999) criteria, which rates response bias dimensionally from Incentive Only, Possible Malingering, to Probable/Definite Malingering. Based on the criteria for preponderance of evidence in civil cases in our judicial system (>50% of weight of the evidence constitutes a ruling in favor of the party with the larger weight of supporting evidence), we combined the probable and definite malingering groups because the likelihood of malingering in these groups is greater than 50%, thus meeting the aforementioned criteria for preponderance of evidence in a civil case (i.e., disability determination). The MND criteria in this study included presence of a substantial external incentive (criteria A), as well as evidence from the three PVTs (VSVT, TOMM, and LMT).

The Incentive Only group (n = 52) contains individuals who met criterion A because of their involvement in disability litigation, but they showed no signs of malingering on the performance validity measures.

The Possible Malingering group (n = 25) contains individuals who met criterion A and criterion for at least one cognitive response bias indicator.

The Probable/Definite Malingering group (n = 28) contains individuals who met criterion A as well as performing below cutoff on at least two of the performance validity measures (criterion C2).

Results

First, we calculated zero-order correlations between the RBANS EI and ES with the MMPI 2-RF response bias scale (RBS), TOMM, VSVT, and LMT. The EI exhibited strong negative correlations with the TOMM, VSVT, and LMT (rs = −.33, −.42, and −.44, respectively, ps < .01), which indicated that poorer effort on the TOMM, VSVT, and LMT are related to poor effort on the RBANS. The EI also exhibited a strong-positive correlation with RBS (r = .28, p < .01), indicating that more response bias is related to poor effort on the RBANS. The ES exhibited a positive correlation with RBS (r = .24, p < .05). The ES was not significantly correlated with the TOMM (r = −.13, p = .31), VSVT (r = −.17, p = .09), or LMT (r = −.11, p = .55).

Next, we examined the EI and the ES in relation to the three MND classifications (Probable/Definite MND, Possible MND, and Incentive Only). We calculated ANOVAs for both the EI and the ES in relation to the three groups, which can be found in Table 1. ANOVAs were significant for both the EI, F(2, 102) = 6.43, p < .005, and ES, F(2, 102) = 4.30, p < .05. The Probable/Definite Malingering group exhibited significantly higher weighted scores (reflecting poor effort) on the EI than the Incentive Only group (d = 0.64). Unexpectedly, for the ES, the Probable/Definite Malingering group had higher scores (reflecting better effort) than the Incentive Only and Possible Malingering groups.

Table 1.

Repeatable Battery for the Assessment of Neuropsychological Status effort index and effort scale in relation to criteria for malingered neurocognitive dysfunction (MND; N = 105)

MND classification
 
 Incentive Only (n = 52)
 
Possible MND (n = 25)
 
Prob./Def. MND (n = 28)
 
ANOVA Effect size 
M SD M SD M SD F(2, 102) d 
Effort Index 0.85a 1.8 0.48a 1.0 2.14b 2.4 6.43** 0.64 
Effort Scale 3.58a 7.5 5.0ab 7.5 8.36b 5.1 4.30* 0.71 
MND classification
 
 Incentive Only (n = 52)
 
Possible MND (n = 25)
 
Prob./Def. MND (n = 28)
 
ANOVA Effect size 
M SD M SD M SD F(2, 102) d 
Effort Index 0.85a 1.8 0.48a 1.0 2.14b 2.4 6.43** 0.64 
Effort Scale 3.58a 7.5 5.0ab 7.5 8.36b 5.1 4.30* 0.71 

Note: *p < .05, **p < .005. Means with different subtexts are significantly different (Tukey HSD). Cohen's d calculated: Probable/Definite Malingering group versus Incentive Only group.

Given the unexpected results for the ES, we only calculated classification statistics for the EI, which can be found in Table 2. The four EI cutoff scores of 2, 3, 5, and 6 that were previously examined in the literature exhibited excellent specificity (0.81, 0.85, 0.92, 0.96, respectively); however, this was at the expense of sensitivity (0.50, 0.36, 0.21, 0.11, respectively). We also calculated predictive powers, because they are useful in assisting clinicians with their diagnostic determinations. However, since they are highly influenced by base rates, we calculated Positive and Negative predictive power estimates (PPP and NPP) for a range of hypothetical base rates (0.10 at the low end to 0.50), which maximizes sensitivity and specificity, resulting in fewer classification errors.

Table 2.

Classification accuracy of effort index

     Predictive power: positive/negative
 
Cutoff SENS SPEC BR 0.10 0.20 0.30 0.40 0.50 
Effort Index 0.50 0.81  0.23/0.94 0.40/0.87 0.53/0.79 0.64/0.71 0.72/0.62 
 0.36 0.85  0.21/0.92 0.38/0.84 0.51/0.76 0.62/0.67 0.71/0.57 
 0.21 0.92  0.23/0.91 0.40/0.82 0.53/0.73 0.64/0.64 0.72/0.54 
 0.11 0.96  0.23/0.91 0.41/0.81 0.54/0.72 0.65/0.62 0.73/0.52 
     Predictive power: positive/negative
 
Cutoff SENS SPEC BR 0.10 0.20 0.30 0.40 0.50 
Effort Index 0.50 0.81  0.23/0.94 0.40/0.87 0.53/0.79 0.64/0.71 0.72/0.62 
 0.36 0.85  0.21/0.92 0.38/0.84 0.51/0.76 0.62/0.67 0.71/0.57 
 0.21 0.92  0.23/0.91 0.40/0.82 0.53/0.73 0.64/0.64 0.72/0.54 
 0.11 0.96  0.23/0.91 0.41/0.81 0.54/0.72 0.65/0.62 0.73/0.52 

Notes: Base rate was 35%. Incentive Only litigants were compared with the Probable/Definite Malingering group.

SENS = sensitivity, SPEC = specificity, BR = hypothetical base rates ranging from 10% to 50%.

Discussion

The current study examined the RBANS embedded effort indices in relation to the MND Criteria (Slick et al., 1999). Our study represents the first examination of both of the RBANS effort indices in a forensic disability setting using structured malingering criteria. Our results suggest that the EI is able to detect poor cognitive performance commensurate with other cognitive response bias indicators and exhibited good specificity (false-positive rates of 19% or less at selected cutoffs). However, the EI also exhibited poor sensitivity, which suggests that it might confabulate malingering and other conditions.

Unexpectedly, the ES worked in the opposite direction with malingering patients performing better than non-malingering litigants. This is most likely due to the removal of the free recall scores from the list recognition scores, a method specifically designed to assess effort in patients with amnesia (Novitski et al., 2012). Individuals in the current sample who were expected to provide good effort (based on MND criteria) could potentially be misclassified on the ES because their list recognition and recall memory scores would both be high, resulting in a lower ES score (indicating poor effort) after subtracting their recall memory scores from their list recognition scores. Conversely, individuals who were expected to exhibit poor effort could have potentially exhibited higher scores on the ES (indicating good effort) because of lower scores on recall memory, which would have less of an impact on their ES scores. Furthermore, malingerers may not realize which types of memory are potentially affected by TBI, resulting in poorer performance on all tests. This would lower legitimate scores when compared with poorer performance by malingerers on free recall performance tasks.

One possible limitation to the current study is that the majority of participants were Caucasian, which affects the generalizability of the data. Future research should include a more diverse sample to improve the external validity of the results. Another possible limitation was the small sample size. By incorporating a larger sample size, future research may yield more robust findings. Despite these limitations, the current study nevertheless provides useful data regarding the performance of the embedded RBANS effort indices in relation structured malingering criteria, which up to this point has not been examined.

In conclusion, the EI exhibited utility in detecting inadequate effort in a forensic setting. Although the EI exhibited weaker sensitivity (lower true positive rates) at the selected cutoffs (0.11–0.50), it exhibited excellent specificity (0.81–0.96), which produces false-positive rates between 4% and 19%. While the EI may not be an appropriate stand-alone measure used to detect neurocognitive malingering, it shows utility as a screening or complementary measure in forensic settings. The ES was not associated with stand-alone symptom validity measures in our sample and exhibited the opposite effects in our analyses with malingering participants performing better than non-malingering participants on this measure. However, this was most likely a result of the sample being used (disability litigants) rather than a sample of patients with amnesia, which is the ES's intended use. Therefore, it does not appear to be appropriate to use the ES to detect malingering in a sample of disability litigants, as the scale was designed to assess effort in individuals with amnesia.

Conflict of Interest

None declared.

References

Armistead-Jehle
P.
Hansen
C. L.
(
2011
).
Comparison of the Repeatable Battery for the Assessment of Neuropsychological Status effort index and stand-alone symptom validity tests in a military sample
.
Archives of Clinical Neuropsychology
 ,
26
,
592
601
.
Beatty
W. W.
Ryder
K. A.
Gontkovsky
S. T.
Scott
J. G.
McSwan
K. L.
Bharucha
K. J.
(
2003
).
Analyzing the subcortical dementia syndrome of Parkinson's disease using the RBANS
.
Archives of Clinical Neuropsychology
 ,
18
,
509
520
.
Ben-Porath
Y. S.
Tellegen
A.
(
2008
).
The minnesota multiphasic personality inventory -2 restructured form: Manual for administration, scoring, and interpretation
 .
Minneapolis, MN
:
University of Minnesota Press
.
Bianchini
K. J.
Mathias
C. W.
Greve
K. W.
(
2001
).
Symptom validity testing: A critical review
.
The Clinical Neuropsychologist
 ,
15
,
19
45
.
Bush
S. S.
Ruff
R. M.
Troster
A. L.
Barth
J. T.
Koffler
S. B.
Pliskin
N. H.
et al
(NAN Policy and Planning Committee)
. (
2005
).
Symptom validity assessment: Practical issues and medical necessity
.
NAN position paper
.
Archives of Clinical Neuropsychology
 ,
20
,
419
426
.
Heilbronner
R. L.
Sweet
J. J.
Morgan
J. E.
Larrabee
G. J.
Millis
S. R.
, &
Conference Participants
(
2009
).
American Academy of Clinical Neuropsychology consensus conference statement on the neuropsychological assessment of effort, response bias, and malingering
.
The Clinical Neuropsychologist
 ,
23
,
1093
1129
.
Inman
T. H.
Vickery
C. D.
Berry
D. T. R.
Lamb
D. G.
Edwards
C. L.
Smith
G. T.
(
1998
).
Development and initial validation of a new procedure for evaluating adequacy of effort given during neuropsychological testing: The Letter Memory Test
.
Psychological Assessment
 ,
10
,
128
139
.
Novitski
J.
Steele
S.
Karantzoulis
S.
Randolph
C.
(
2012
).
The Repeatable Battery for the Assessment of Neuropsychological Status effort scale
.
Archives of Clinical Neuropsychology
 ,
27
,
190
195
.
Randolph
C.
(
1998
).
Repeatable battery for the assessment of neuropsychological status
 .
San Antonio, TX
:
The Psychological Corporation
.
Randolph
C.
Tierney
M. C.
Mohr
E.
Chase
T. N.
(
1998
).
The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS): Preliminary clinical validity
.
Journal of Clinical and Experimental Neuropsychology
 ,
20
,
310
319
.
Rees
L. M.
Tombaugh
T. N.
Gansler
D. A.
Moczynski
N. P.
(
1998
).
Five validation experiments of the Test of Memory Malingering (TOMM)
.
Psychological Assessment
 ,
10
,
10
20
.
Silverberg
N. D.
Wertheimer
J. C.
Fichtenberg
N. L.
(
2007
).
An effort index for the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS)
.
Clinical Neuropsychology
 ,
21
,
841
854
.
Slick
D.
Hopp
G.
Strauss
E.
Hunter
M.
Pinch
D.
(
1994
).
Detecting dissimulation: Profiles of simulated malingerers, traumatic brain injury patients, and normal controls on a revised version of Hiscock and Hiscock's forced choice memory test
.
Journal of Clinical and Experimental Neuropsychology
 ,
16
,
472
481
.
Slick
D.
Hopp
G.
Strauss
E.
Thompson
G. B.
(
1997
).
Victoria symptom validity test version 1.0 professional manual
 .
Odessa, FL
:
Psychological Assessment Resources
.
Slick
D. J.
Sherman
E. M. S.
Iverson
G. L.
(
1999
).
Forum: Diagnostic criteria for malingered neurocognitive dysfunction: Proposed standards for clinical practice and research
.
The Clinical Neuropsychologist
 ,
13
,
545
561
.
Tarescavage
A. M.
Wygant
D. B.
Gervais
R. O.
Ben-Porath
Y. B.
(
2012
).
Association between the MMPI-2 Restructured Form (MMPI-2-RF) and malingered neurocognitive dysfunction among non-head injury disability claimants
.
The Clinical Neuropsychologist
 ,
27
,
313
335
.
Tombaugh
T. N.
(
1996
).
Test of memory malingering: TOMM
 .
North Towanda, NY
:
Multi-Health Systems
.
Wechsler
D.
(
1981
).
Wechsler adult intelligence scale-revised
 .
San Antonio, TX
:
The Psychological Corporation
.
Young
J. C.
Baughman
B. C.
Roper
B. L.
(
2012
).
Validation of the Repeatable Battery for the Assessment of Neuropsychological Status – Effort index in a veteran sample
.
The Clinical Neuropsychologist
 ,
26
,
688
699
.