The aim of the current study was to compare two embedded measures of effort for the repeatable battery for the assessment of neuropsychological status (RBANS). Sensitivity and specificity of the Effort Index (EI) and Effort Scale (ES) were compared in a sample of individuals with genuine memory impairment (MI) and individuals coached to simulate MI. Overall, the EI yielded a sensitivity of 0.89 and specificity of 0.41, while the ES yielded a sensitivity of 0.88 and specificity of 0.81. When those in the MI group were separated by level of impairment, the EI was more specific when RBANS Total Scores were in the average or mildly impaired range, and the ES had greater specificity when RBANS Total Scores were in the severely impaired range. These results suggest that the embedded measure should be selected based on the level of impairment on the RBANS.

Introduction

Neuropsychological evaluations are frequently conducted to assess and diagnose memory disorders (McKhann et al., 2011; Sperling et al., 2011), but the previous literature has not adequately addressed effort assessment in older adult populations. Inclusion of stand-alone and embedded performance validity tests (PVTs) is considered a standard of practice to assess for the proper motivation and effort. Embedded measures, particularly, have recently garnered interest (Bush et al., 2005; Heilbronner, Sweet, Morgan, Larrabee, & Millis, 2009). Failures of PVTs during forensic evaluations (Ardolf, Denney, & Houston, 2007; Larrabee, 2003) and social security disability evaluations (Chafetz, Abrahams, & Kohlmaier, 2007; Chafetz, 2008) may suggest intentional suppression of effort; however, older adults may fail PVTs for a myriad of reasons. These reasons include poor task-engagement, genuine impairment, emotional factors (e.g., depression), fatigue, and processing speed deficits (Storandt, 1994). Embedded measures are gaining popularity in clinical practice, but research on embedded measures of effort has failed to address non-malingered PVT failure in older adult populations.

Embedded PVTs may be particularly beneficial in clinical practice, especially when the measures are included in brief screening instruments. While stand-alone PVTs require extra administration time, embedded indices are derived from data obtained from the neuropsychological testing battery. Further, embedded indices of effort are less-easily detected by those feigning impairment (Greve & Bianchini, 2004; Hook, Marquine, & Hoelze, 2009) and can be used to ensure consistent effort throughout the battery of several neuropsychological tests.

The repeatable battery for the assessment of neuropsychological status (RBANS; Randolph, 1998) is a brief and commonly used neuropsychological battery which can be used in clinical samples with memory disorders (see Duff et al., 2008). The RBANS allows for assessment of cognitive functioning within a brief administration time, making it an excellent candidate for embedded measures. To date, two embedded indices of performance validity have been developed for the RBANS; the Effort Index (EI; Silverberg, Wertheimer, & Fichtenberg, 2007) and the Effort Scale (ES; Novitski, Steele, Karantzoulis, & Randolph, 2012). However, significant issues, outlined below, exist in the available research on both indices.

The EI (Silverberg et al., 2007) was developed with participants from an outpatient neurorehabilitation clinic, but those with advanced Alzheimer's disease were excluded from the sample. All participants had evidence of a neurologic disorder, a clinically significant psychiatric illness, or both. Following the development of the EI, the researchers proceeded with validation by assessing the ability of the EI to distinguish between a clinical TBI sample, a clinical malingering sample, and three simulator groups (control group, simulated-naive malingerers, and simulated-coached malingerers). The EI was found to be highly sensitive in discriminating between the groups. A cut score was developed in which scores >3 were considered suspicious of poor effort. However, the researchers stated a cut score of one would provide greater ability to discriminate between those with true mild TBI (mTBI) and those who were attempting to appear as though they had suffered a mTBI.

Other researchers have examined the use of the EI in various samples. The EI was supported in a sample of 44 medically ill, non-litigating older adults, with the caveat that the index is not appropriate for those with severe cognitive impairment (Hook et al., 2009). When the EI was investigated in a sample of 303 older adult veterans referred for neuropsychological evaluations at a memory disorders clinic, the EI had a sensitivity of 64% and a specificity of ∼85% when effort classification was determined by the Test of Memory Malingering (Barker, Horner, & Bachman, 2010). Further, the EI showed a specificity of 94% and sensitivity of 31% when compared with the Word Memory Test classifications of effort in a sample of 85 veterans under the age of 65 (Young, Baughman, & Roper, 2012). Armistead-Jehle and Hansen (2011) compared the EI with established stand-alone measures of effort and found the EI had high specificity, but weak sensitivity compared with the Medical Symptom Validity Test (MSVT; Green, 2004), Non-Verbal Medical Symptom Validity Test (Green, 2008), and Test of Memory Malingering (Tombaugh, 1996).

The ES was developed to discriminate between poor effort and true amnestic disorders (Novitski et al., 2012), but effort was not independently assessed using a stand-alone measure in the probable Alzheimer's disease group or the amnestic Mild Cognitive Impairment group. The development of the ES used existing RBANS data for a sample of mTBI patients and a group sample clinical diagnoses of amnestic Mild Cognitive Impairment or probable Alzheimer's disease. The mTBI patients were administered Green's Word Memory Test (Green, 2003, revised 2005) and had scored below cutoffs, suggesting questionable effort. However, the authors cautioned that high-false-positive rates may be found if the ES is used in a population without genuine cognitive impairment. The high-false-positive rates were hypothesized to be the result of a ceiling effect that is inherent in the List Recognition subtest. The development suggested that the ES cannot be calculated unless Digit Span raw scores are <9, List Recognition raw scores are <19, or when Digit Span + List Recognition raw scores are <28. The researchers found that ES was better able to discriminate between amnestic groups and mTBI groups compared with the EI. The ES is a recent creation, and to the authors' knowledge, other studies have yet to validate the ES.

To address the limitations of the literature, the present study aimed to examine the sensitivity and specificity of the EI and ES in a coached-simulating sample (CS) and in individuals previously diagnosed with memory impairment (MI) by a physician or geropsychologist. The MSVT was used as a stand-alone PVT and a hybrid known groups/simulation design was implemented in order to ensure group membership and validity of effort within the two samples. Previous research has offered support for the MSVT's use with individuals with severe dementia (Axelrod & Schutte, 2010; Dean, Victor, Boone, Philpott, & Hess, 2009; Howe & Loring, 2009; Singhal, Green, Ashaye, Shankar, & Gill, 2009). Simulation designs have excellent internal validity (Rogers, 2008) and this design will better ensure that individuals in the CS perform with poor effort. It was expected that first, the EI and ES would show high sensitivity but unacceptable specificity compared with the known group membership in the whole sample and second, the EI and ES would demonstrate greater specificity depending on level of impairment on the RBANS.

Methods

Participants

The sample was compromised of two groups of participants: a CS and a MI sample. The final sample included 90 individuals with 44 in the CS and 46 in the MI. The MI was recruited from five skilled nursing facilities specializing in long-term care for MI. Inclusion criteria included consent from legal guardian, assent from participant, and previous diagnosis of a memory disorder (i.e., Alzheimer's disease, vascular dementia, dementia not otherwise specified, or mild cognitive impairment). Participants were excluded if the individual was unable to give their assent due to a comprised mental status (e.g., delirium), had a medical condition (i.e., broken writing hand) that would adversely impact test performance, or had suspicious scores on the MSVT. Suspicious MSVT scores included falling below random on any of the subtests or obtaining the poor effort profile as suggested by the MSVT's test publisher (Green 2004, 2011). Diagnosis of dementia or MI was made by the participant's treating physician or geropsychologist based on the criteria of the International Classification of Diseases-10 (ICD-10; World Health Organization, 1990). Dementia or MI diagnoses included probable Alzheimer's disease (n = 7), Vascular Dementia (n = 21), Mild Cognitive Impairment (n = 1), and Dementia Not Otherwise Specified (n = 17). Within the MI, each participant had a minimum of 1 medical diagnosis, with a mean of 5.04 (SD = 1.74) medical diagnoses. Additionally, within the MI, there was an average of 1.06 (SD = 0.89) psychiatric diagnoses. A wide range of medical and psychiatric diagnoses were seen within the MI (Tables 1 and 2). Forty-six individuals met inclusion criteria for the study. Participants were between age 52 and 89 (M = 76.44 years; SD = 10.49) and had obtained between 6 and 18 years of education (M = 11.17 years; SD = 2.82). These individuals were given the RBANS and MSVT for research purposes alone. The results of these assessments were not used for clinical purposes.

Table 1.

Co-occurring medical diagnoses with the memory impairment group (MI)

Condition Number Percentage 
Hypertension 29 63.04 
Diabetes (Type I or Type II) 15 32.61 
Hyperlipidemia 14 30.43 
Hypothyroidism 17.39 
Anemia 15.22 
Reflux esophagitis 13.04 
Urinary tract infection 13.04 
Osteoporosis 10.87 
Cerebrovascular accident 10.87 
Atrial fibrillation 10.87 
Coronary atherosclerosis 10.87 
Insomnia 8.70 
Chronic obstructive pulmonary disease 8.70 
Constipation 8.70 
Chronic airway obstruction 8.70 
Pneumonia 6.52 
Paralysis agitans 6.52 
Restless leg syndrome 6.52 
Renal disease or failure 4.35 
Hypercholesterolemia 4.35 
Upper respiratory disease 4.35 
Cerebrovascular disease 4.35 
Peripheral vascular disease 4.35 
Amputation 4.35 
Hypoxia 4.35 
Muscle weakness 4.35 
Fracture of bone 4.35 
Generalized pain disorder 4.35 
Old myocardial infarct 4.35 
Epilepsy 4.35 
Angina pectoris 4.35 
Condition Number Percentage 
Hypertension 29 63.04 
Diabetes (Type I or Type II) 15 32.61 
Hyperlipidemia 14 30.43 
Hypothyroidism 17.39 
Anemia 15.22 
Reflux esophagitis 13.04 
Urinary tract infection 13.04 
Osteoporosis 10.87 
Cerebrovascular accident 10.87 
Atrial fibrillation 10.87 
Coronary atherosclerosis 10.87 
Insomnia 8.70 
Chronic obstructive pulmonary disease 8.70 
Constipation 8.70 
Chronic airway obstruction 8.70 
Pneumonia 6.52 
Paralysis agitans 6.52 
Restless leg syndrome 6.52 
Renal disease or failure 4.35 
Hypercholesterolemia 4.35 
Upper respiratory disease 4.35 
Cerebrovascular disease 4.35 
Peripheral vascular disease 4.35 
Amputation 4.35 
Hypoxia 4.35 
Muscle weakness 4.35 
Fracture of bone 4.35 
Generalized pain disorder 4.35 
Old myocardial infarct 4.35 
Epilepsy 4.35 
Angina pectoris 4.35 

Notes: Each participant had multiple medical diagnoses. Diagnoses were included in the chart if there were two or more participants from the MI with the diagnosis. For an exhaustive list, contact the corresponding author.

Table 2.

Co-occurring psychiatric condition within the MI

Condition Number Percentage 
Depression NOS 18 39.13 
Anxiety NOS 11 23.91 
Bipolar disorder NOS 8.70 
Psychosis NOS 6.52 
Schizophrenia (all types) 6.52 
Drug Abuse/dependence 4.35 
Condition Number Percentage 
Depression NOS 18 39.13 
Anxiety NOS 11 23.91 
Bipolar disorder NOS 8.70 
Psychosis NOS 6.52 
Schizophrenia (all types) 6.52 
Drug Abuse/dependence 4.35 

Notes: Each participant had multiple medical diagnoses. Diagnoses were only included in this chart if there were two or more participants from the MI that had the diagnosis. For an exhaustive list, contact the corresponding author.

Individuals in the CS were recruited from The School of Professional Psychology at Forest Institute (N = 44). Inclusion criteria for participating in the CS was enrollment in the first 2 years of the Master's or Doctoral programs in psychology at the time of testing (n = 33) or a staff member of Forest Institute (n = 11); IR, DR, or CNS scores <85% on the MSVT; and no prior knowledge of neuropsychological measures being given. Participants for the CS were between age 22 and 61 (M = 27.82 years; SD = 9.01) and had obtained between 14 and 18 years of education (M = 16.41 years; SD = 0.82).

Measures

Repeatable battery for the assessment of neuropsychological status

The RBANS (Randolph, 1998) was developed as a brief cognitive screening instrument to assess for dementia in older adult populations. The RBANS consists of 12 subtests measuring the following domains: Immediate Memory, Visuospatial/Constructional, Language, Attention, and Delayed Memory. A Total Score can be derived from the index scores. Administration of the RBANS takes ∼20–40 min. The RBANS has a mean score of 100 with a SD of 15. Scores can range from 40 to 160.

Medical symptom validity test

The MSVT is a stand-alone symptom validity test introduced by Green (2004) which consists of 10 semantically linked words pairs. The word pairs are presented during two learning trials. The test contains four subtests: Immediate Recall (IR), Delayed Recall (DR), Paired Associates (PA), and Free Recall (FR). A fifth score is developed based on the agreement within IR and DR and is labeled the Consistency (CNS) score. Effort profiles are based on the four subtest scores and CNS.

Profile analyses for this study were conducted according to the Advanced Interpretation (Green, 2011) program produced by the test publisher. Good effort was determined in the MI by obtaining scores classified as “normal memory,” “weak memory,” or “genuine memory impairment profile” (GMIP) based on the Advanced Interpretation (Green, 2011) program output. Individuals in the MI were excluded from the analysis if they achieved the “poor effort profile.” Suppressed effort was ensured in the CS by obtaining scores <85% on IR, DR, or CNS. Individuals in the CS were excluded if the obtained a “weak memory” or “normal memory” profile. It is important to note that within the MSVT profile analysis, individuals that are able to achieve “normal memory” are not necessarily absent of MI. Rather, individuals with severe MI are able to pass the MSVT and the profile labels can be viewed as labels for effort not MI (see Axelrod & Schutte, 2010; Singhal et al., 2009 for review).

Procedures

IRB approval from Forest Institute was obtained prior to collection of data. The RBANS and MSVT were administered to those in the MI in their room or a private space at the long-term care facility based on the participant's request. The RBANS was administered first, followed by a break and then the MSVT. Delays between the RBANS and MSVT ranged from 15 min to 5 days (M = 195.60 min; SD = 310.50). A delay was provided for the MI based on individual's fatigue from testing. Each of the measures was given according to their standardized procedures with deviations made to accommodate examinees. Deviations included that individuals were assisted in making selections with the computer mouse on the MSVT if they noted difficulty or uncertainty regarding the use of the mouse, and the Coding and Line Orientation subtests from the RBANS were discontinued if the individual could not grasp the concept of the task during the sample items and the subtest was scored as a zero. The deviation with the MSVT is consistent with deviations used in previous studies utilizing the MSVT with older adult populations and is not thought to affect scores (Howe & Loring, 2009; Singhal, Green, Ashaye, Shankar, & Gill, 2009). Individuals in the MI were not compensated for their participation.

Individuals in the CS were read a script (Appendix) introducing the concept of suppressing effort for secondary gain. The script instructed participants to simulate dementia in order to obtain disability assistance to help with everyday living expenses. Additionally, the script instructed participants to engage in testing as though they were 75 years of age and they were provided a sheet that listed early signs of dementia from Epigee Women's Health (Dementia: Symptoms and Early Signs, n.d.). The CS was then given the RBANS and MSVT in a single session. There was no deviation from standard protocol made for the CS. The participants were asked to complete a demographic sheet and a questionnaire at the conclusion of the study. The questionnaire asked participants to identify their understanding of the instructions and to affirm they had attempted to complete the tests as though they were 75 years old with dementia. The demographic sheet collected information on ethnicity, age, and education. Individuals in the CS were entered in a drawing for one of four $25 VISA gift cards for completing the testing.

Data Analysis Strategy

The EI (Silverberg et al., 2007) and ES (Novitski et al., 2012) were calculated according to instructions by the respective authors in the original articles. The ES was only calculated when the sum of List Recognition and Digit Span subtests raw score was <28 per the authors' suggestion in the discussion. A cutoff score of <12 for the ES and >3 for the EI was used to indicate a positive result for suboptimal effort. Descriptive statistics were calculated for the demographic and clinical variables for both samples. Non-parametric statistics were used because scores violated the assumption of a normal distribution. Receiver–Operator characteristic curves were constructed for the EI and ES. Classification statistics were calculated for the EI and ES. Participants in the MI were divided into severity groups based on RBANS Total Score to calculate EI and ES pass/failure rates based on impairment. Severity was divided into three groups based on interpretative cut points suggested by Heaton, Miller, Taylor, and Grant (2004) and Taylor and Heaton (2001). Scores were classified based on the following criteria for RBANS Total Score: Average/Mild Impairment (standard scores of 70 or above), Moderate Impairment (standard scores from 55 to 69), and Severe Impairment (standard scores <54).

Results

RBANS, EI, and ES scores were not normally distributed. Independent samples Mann–Whitney U-tests, a non-parametric statistic, were used to compare means. Statistically significant group differences for the CS and MI were found for age, years of education, RBANS Total Score, RBANS Language Index, RBANS Attention Index, RBANS Delayed Memory Index, total EI score, and total ES score (Table 3).

Table 3.

Descriptive statistics for sample based on independent samples Mann–Whitney U-test results

 MI
 
CS
 
Z p R 
Mean SD Mean SD 
Age (years) 76.44 10.49 27.82 9.01 8.18 <.001 .86 
Education (years) 11.17 2.82 16.41 0.82 −7.50 <.001 −.79 
RBANS TS 57.48 11.70 48.52 8.66 4.98 <.001 .52 
RBANS IMI 60.26 14.23 59.50 14.98 .264 .792 .03 
RBANS VCI 65.76 17.07 64.57 12.56 −.288 .773 −.03 
RBANS LGI 71.59 14.14 47.50 12.21 122.45 <.001 12.91 
RBANS ATI 65.57 14.42 45.89 10.03 6.45 <.001 .68 
RBANS DMI 58.83 17.72 47.66 12.43 4.42 <.001 .47 
EI raw score 3.70 2.65 7.98 3.18 −5.62 <.001 −.59 
ES raw score 15.59 6.32 5.41 3.95 6.69 <.001 .71 
 MI
 
CS
 
Z p R 
Mean SD Mean SD 
Age (years) 76.44 10.49 27.82 9.01 8.18 <.001 .86 
Education (years) 11.17 2.82 16.41 0.82 −7.50 <.001 −.79 
RBANS TS 57.48 11.70 48.52 8.66 4.98 <.001 .52 
RBANS IMI 60.26 14.23 59.50 14.98 .264 .792 .03 
RBANS VCI 65.76 17.07 64.57 12.56 −.288 .773 −.03 
RBANS LGI 71.59 14.14 47.50 12.21 122.45 <.001 12.91 
RBANS ATI 65.57 14.42 45.89 10.03 6.45 <.001 .68 
RBANS DMI 58.83 17.72 47.66 12.43 4.42 <.001 .47 
EI raw score 3.70 2.65 7.98 3.18 −5.62 <.001 −.59 
ES raw score 15.59 6.32 5.41 3.95 6.69 <.001 .71 

Notes: MI = Memory Impairment Group; CS = Coached Simulator Group; RBANS = Repeatable Battery for the Assessment of Neuropsychological Status; TSTotal Score; IMI = Immediate Memory Index; VCI = Visuospatial/Construction Index; LGI = Language Index, ATI = Attention Index; DMI = Delayed Memory Index; EI = Effort Index, based on the cutoff score of >3 (Silverberg et al., 2007), ES = Effort Scale, based on a cutoff score of <12 (Novitski et al., 2012). All RBANS scores are shown as standard scores.

The ES and EI had similar sensitivities to simulated MI, but the ES showed greater specificity to genuine dementia (Table 4). Twenty-seven individuals in the MI were misclassified by the EI (Table 5), while eight where misclassified by the ES (Table 6). The ES was not calculated for one individual in the CS and three in the MI because the sum of List Recognition and Digit Span scores was >28.

Table 4.

Sensitivity and specificity of Effort Index and Effort Scale

 EI ES 
Sensitivity 0.89 0.88 
Specificity 0.41 0.81 
 EI ES 
Sensitivity 0.89 0.88 
Specificity 0.41 0.81 

Notes: EI = Effort Index, based on the cutoff score of >3 (Silverberg et al., 2007); ES = Effort Scale, based on a cutoff score of <12 (Novitski et al., 2012).

Table 5.

Effort Index pass and failure based on subsamples

 MI CS 
Pass 19 
Fail 27 39 
 MI CS 
Pass 19 
Fail 27 39 

Notes: MI = Memory Impairment Group; CS = Coached Simulation Group.

Table 6.

Effort Scale pass and failure based on subsamples

 MI CS 
Pass 35 
Fail 38 
Not calculated 
 MI CS 
Pass 35 
Fail 38 
Not calculated 

Notes: MI = Memory Impairment Group; CS = Coached Simulation Group. The Effort Scale was not calculated if List Recognition and Digit Span raw score sum was >28.

ROC curves were calculated for both the EI (Fig. 1) and ES (Fig. 2). The EI produced an area under the curve (AUC) of 0.839, while the ES had an AUC of 0.920. The ROC curves suggest that the EI and ES scores have a high ability to discriminate the MI and CS.

Fig. 1.

Receiver-operating curve for the repeatable battery for the assessment of neuropsychological status Effort Index (Silverberg et al., 2007).

Fig. 1.

Receiver-operating curve for the repeatable battery for the assessment of neuropsychological status Effort Index (Silverberg et al., 2007).

Fig. 2.

Receiver-operating curve for the RBANS ES (Novitski et al., 2012).

Fig. 2.

Receiver-operating curve for the RBANS ES (Novitski et al., 2012).

Specificity and classification rates were calculated for the EI and ES based on severity rankings on the RBANS Total Score (Table 7). The EI demonstrated a specificity of 0.75 when the RBANS Total Score was average or mildly impaired. The ES had a specificity of 0.96 when the RBANS Total Score was severely impaired.

Table 7.

Effort Index and Effort Scale performance based on severity of MI

Impairment level n RBANS TS (SDEI specificity ES specificity 
Average/mild 78.50 (10.45) 0.75 0.40* 
Moderate 16 58.96 (3.55) 0.60 0.73 
Severe 22 49.41 (3.35) 0.17 0.96 
Impairment level n RBANS TS (SDEI specificity ES specificity 
Average/mild 78.50 (10.45) 0.75 0.40* 
Moderate 16 58.96 (3.55) 0.60 0.73 
Severe 22 49.41 (3.35) 0.17 0.96 

Notes: RBANS = Repeatable Battery for the Assessment of Neuropsychological Status; TS = Total Score; EI = Effort Index; ES = Effort Scale. Scores were classified as Average or Mild if the standard score was 70 or above, Moderate was between 55 and 69, and Severe was <54. *ES was calculated for the Average/Mild group based on five individuals, three were excluded based on rules of calculating ES.

Discussion

PVT is considered standard in the field of clinical neuropsychology, but research has not adequately addressed PVT in older adult populations. Two embedded performance validity measures have been developed in the RBANS: EI (Silverberg et al., 2007) and ES (Novitski et al., 2012). No previous studies have simultaneously compared the sensitivity and specificity of these embedded measures using an external measure of effort. The current study aimed to address this limitation in past research.

The results of this study indicate the EI and ES have vastly different specificity levels in a cognitively impaired older adult sample, while maintaining grossly comparable sensitivity in a simulation sample. The ES demonstrated higher levels of specificity, while the EI demonstrated relatively low specificity. The low specificity found with the EI is consistent with previous research conducted (see Barker et al., 2010).

Further analysis revealed that the total level of impairment may be considered when selecting the embedded measure of effort on the RBANS. Specifically, when the RBANS Total Score was used to classify level of cognitive impairment, the EI demonstrated higher specificity among individuals with average or mildly impaired scores, while the ES showed high specificity among the severely impaired individuals. The EI and ES showed similar specificity among the moderately impaired group. Overall, the classification rates of the ES were relatively stable, while the EI classification rates declined as cognitive functioning decreased. For clinical purposes, it appears the ES may have more clinical utility when an individual has moderate or severe cognitive impairment and the EI may have more clinical utility when an individual has mild impairment in cognitive functioning.

The MI included a wide range of MI, medical diagnoses, and psychiatric diagnoses. Although this fact may pose some difficulty with understanding exactly the impact of specific memory diagnoses, the broad range of impairment and diagnoses in this study most likely represent common clinical referrals for MI. The same is true in regard to medical and psychiatric diagnoses.

This study had two major limitations: the use of a simulation sample and the time delay between the MSVT and RBANS. Simulation samples do not necessarily simulate in the same manner as those who are truly malingering. Thus, there may be some difficulties with extrapolating the sensitivity related findings based on this simulation group to clinical referrals performing with suboptimal effort.

The delay between the MSVT and RBANS within the MI may be a concern. However, a post hoc analysis was done to compare the MI sample based on whether time delay was <2 h (n = 22) or >2 h (n = 24). Independent samples Mann–Whitney U-tests revealed no significant differences for individuals in the two time delay groups based on the MSVT subtests IR (Z = .689; p = .49), DR (Z = −.166; p = .87), CNS (Z = 1.206; p = .23), PA (Z = −.022; p = .98), and FR (Z = −.656; p = .51) and RBANS Total Score (Z = .794; p = .43). Furthermore, only eight of the individuals from the MI had a delay of >200 min between the RBANS and MSVT, with five being >1 day. If those eight individuals are excluded, the average time between the RBANS and MSVT is 96.95 min (SD = 53.65), which is a reasonable delay that might occur in traditional testing between a stand-alone PVT and other neuropsychological measures. Additionally, if those eight individuals are excluded from the analysis all together, specificity is 0.42 for EI and 0.82 for ES. The individuals were retained in the analysis because such a delay may reflect the realities of clinical practice and no significant differences were found in the performance on the testing measures based on delay time.

Future research should focus on the convergent validity of the ES and EI with other stand-alone PVTs. Furthermore, research should examine the EI and ES in a genuine patient sample that has been determined to be feigning or exaggerating effort based on criterion indices. This study utilized the MSVT as the criterion for effort and further research should examine the possibility of validation of the ES with other measures of validity; specifically, validation with multiple measures may be beneficial.

Funding

This work was supported by The School of Professional Psychology at Forest Institute through the Student Research Grants to help pay for test credits and protocols; and Pearson Assessment through the Research Assistance Program which RBANS protocols were obtained at a 50% discounted rate.

Conflict of Interest

None declared.

Acknowledgements

The researchers acknowledge Shelia Hutson, PsyD, Michael Huffman, BSW, Debra Harlan, MSW, Annie Grant, BS, and Mark Kilgore MSIE, for their help with coordinating participants for the MI sample.

Appendix

Script for Coached Participants

Many times individuals will attempt to gain resources to help with their cost of living through faking symptoms. An example of this may be that an individual would claim to have dementia-like symptoms in order to get disability or other monetary benefits. For this study, we will ask you to take these tests as though you are pretending to have dementia and are ∼75 years of age. Pretend that if you do not successfully take these tests in a manner similar to that of someone with dementia, you will be forced to sell your home, car, and other valuables in order to buy food and other daily living supplies. After consulting with your lawyer, who is helping you to file for disability, you discover that often times during the psychological evaluation you will be required to take tests to determine if you have dementia and if you are performing with sufficient effort. Thus, you decide that you will try not to come across as severely demented. The goal for you will be to provide enough effort to pass the tests that may measure your effort, while still looking as though you have difficulties with memory on the measures testing for dementia. Before going to take the psychological measures, you did a web search and discovered the following passage to help you prepare how you should perform on the all of the tests (Hand the participant the Epigee Women's Health (Dementia: Symptoms and Early Signs, n.d. sheet.).

References

Ardolf
B. R.
Denney
R. L.
Houston
C. M.
Base rates of negative response bias and malingered neurocognitive dysfunction among criminal defendants referred for neuropsychological evaluation
The Clinical Neuropsychologist
 
2007
21
899
916
Armistead-Jehle
P.
Hansen
C. L.
Comparison of the repeatable battery for the assessment of neuropsychological status Effort Index and stand-alone symptom validity tests in a military sample
Archives of Clinical Neuropsychology
 
2011
26
592
601
Axelrod
B. N.
Schutte
C.
Analysis of the dementia profile on the medical symptom validity test
The Clinical Neuropsychologist
 
2010
24
873
881
Barker
M. D.
Horner
M. D.
Bachman
D. L.
Embedded indices of effort in the repeatable battery for the assessment of neuropsychological status (RBANS) in a geriatric sample
The Clinical Neuropsychologist
 
2010
21
1061
1077
Bush
S. S.
Ruff
R. M.
Troster
A. I.
Barth
J. T.
Koffler
S. P.
Pliskin
N. H.
et al.  
Symptom validity assessment: Practice issues and medical necessity
Archives of Clinical Neuropsychology
 
2005
20
419
246
Chafetz
M. D.
Malingering on the social security disability consultative exam: Predictors and base rates
The Clinical Neuropsychologist
 
2008
22
529
546
Chafetz
M. D.
Abrahams
J. P.
Kohlmaier
J.
Malingering on the social security disability consultatitve exam: A new rating scale
Archives of Clinical Neuropsychology
 
2007
22
1
14
Dean
A. C.
Victor
T. L.
Boone
K. B.
Philpott
L. M.
Hess
R. A.
Dementia and effort test performance
The Clinical Neuropsychologist
 
2009
23
133
152
Dementia: Symptoms and Early Signs
n.d.
Epigee Women's Health. Retrieved August 21, 2011, from http://www.epigee.org/mental/dementia.html
Duff
K.
Humphreys Clark
J. D.
O'Bryant
S. E.
Mold
J. W.
Schiffer
R. B.
Sutker
P. B.
Utility of the RBANS in detecting cognitive impairment associated with Alzheimer's disease: Sensitivity, specificity, and positive and negative predictive powers
Archives of Clinical Neuropsychology
 
2008
23
603
612
Green
P.
Word memory test for windows: User's manual and program
 
2003
Edmonton, Canada
Green's Publishing
revised 2005
Green
P.
Green's Medical Symptom Validity Test (MSVT) for microsoft windows: User's manual
 
2004
Edmonton, Canada
Green's Publishing
Green
P.
Test manual for the nonverbal Medical Symptom Validity Test
 
2008
Edmonton
Green's Publishing
Green
P.
Advanced interpretation (AI) program
 
2011
Edmonton
Green's Publishing
Greve
K.
Bianchini
K.
Setting empirical cut-offs on psychometric indicators of negative response bias: A methodological commentary with recommendations
Archives of Clinical Neuropsychology
 
2004
19
533
541
Heaton
R. K.
Miller
W.
Taylor
M. J.
Grant
I.
Revised comprehensive norms for an expanded Halstead-Reitan battery: Demographically adjusted neuropsychological norms for African-American and caucasian adults
 
2004
Lutz, FL
Psychological Assessment Resources
Heilbronner
R. L.
Sweet
J. J.
Morgan
J. E.
Larrabee
G. J.
Millis
S. R.
American Academy of clinical neuropsychology consensus conference statement on the neuropsychological assessment of effort, response bias, and malinger
The Clinical Neuropsychologist
 
2009
23
1093
1129
Hook
J. N.
Marquine
M. J.
Hoelzle
J. B.
Repeatable battery for the assessment of neuropsychological status Effort Index performance in a medically ill geriatric sample
Archives of Clinical Neuropsychology
 
2009
24
231
235
Howe
L. L. S.
Loring
D. W.
Classification accuracy and predictive ability of the Medical Symptoms Validity Test's dementia profile and general memory impairment profile
The Clinical Neuropsychologist
 
2009
23
329
342
Larrabee
G. J.
Detection of malingering using atypical performance patterns on standard neuropsychological tests
The Clinical Neuropsychologist
 
2003
17
410
425
McKhann
G. M.
Knopman
D. S.
Chertkow
H.
Hyman
B. T.
Jack
C. R.
Kawas
C. H.
et al.  
The diagnosis of dementia due to Alzheimer's disease: Recommendations from the National Institute on Aging and the Alzheimer's Association work group
Alzheimer's & Dementia
 
2011
7
1
7
Novitski
J.
Steele
S.
Karantzoulis
S.
Randolph
C.
The repeatable battery for the assessment of neuropsychological status effort scale
Archives of Clinical Neuropsychology
 
2012
27
190
195
Randolph
C.
The repeatable battery for the assessment of neuropsychological status: Manual
 
1998
San Antonio, TX
The Psychological Corporation
Rogers
R.
Clinical assessment of malingering and deception, third edition
 
2008
New York
The Guildford Press
Silverberg
N. D.
Wertheimer
J. C.
Fichtenberg
N. L.
An Effort Index for the repeatable battery for the assessment of neuropsychological status (RBANS)
The Clinical Neuropsychologist
 
2007
21
841
854
Singhal
A.
Green
P.
Ashaye
K.
Shankar
K.
Gill
D.
High specificity of the Medical Symptom Validity Test in patients with very severe memory impairment
Archives of Clinical Neuropsychology
 
2009
24
721
728
Sperling
R. A.
Aisen
P. S.
Beckett
L. A.
Bennett
D. A.
Craft
S.
Fagan
A. M.
et al.  
Toward defining the preclinical stages of Alzheimer's disease: Recommendations from the National Institute on Aging and the Alzheimer's Association work group
Alzheimer's & Dementia
 
2011
7
280
292
Storandt
M.
Storandt
M.
VandenBos
G. R.
General principles of assessment of older adults
Neuropsychological assessment of dementia and depression
 
1994
Washington, DC
American Psychological Association
Taylor
M. J.
Heaton
R. K.
Sensitivity and specificity of WAIS-III/WMS-III demographically corrected factor scores in neuropsychological assessment
Journal of the International Neuropsychological Society
 
2001
7
867
874
Tombaugh
T. N.
TOMM: Test of memory malingering manual
 
1996
North Tonawanda, NY
Multi-Health System, Inc
World Health Organization
1990
International statistical classification of diseases and related health problems (10th ed.). Retrived August 30, 2011, from http://apps.who.int/classifications/icd10/browse/2010/en#/F32
Young
J. C.
Baughman
B. C.
Roper
B. L.
Validation of the repeatable battery for the assessment of neuropsychological status—Effort Index in a veteran sample
The Clinical Neuropsychologist
 
2012
26
688
699

Author notes

The pilot portion of this research data was originally presented at the 32nd Annual Conference of the National Academy of Neuropsychology in Nashville, TN, USA (2012).