Abstract

Symptom validity tests (SVTs) are often used in neuropsychological assessment; however, recent studies indicate that cognitive impairment/dementia may contribute to failing scores on some effort tests. The purpose of this study was to characterize how individuals with Huntington disease (HD) perform on three SVTs and to examine the relationship between SVT performance and demographic and clinical variables. Results indicate that while the majority of HD patients passed the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) Effort Index (EI; 82% of n = 121) and the Test of Memory Malingering (92% of n = 36), failure of these SVTs was associated with poorer cognitive and adaptive functioning, and greater motor impairment. Results showed that less than one-third passed the RBANS Effort Scale (ES; 30% of n = 43) and few clinical and demographic variables were correlated with this SVT performance. Although some SVTs may be better suited to HD, cognitive ability should be considered when evaluating effort in HD.

Introduction

Symptom validity tests (SVTs), including stand-alone tests that exclusively measure effort and embedded indices in tests of general neuropsychological abilities (Hook, Marquine, & Hoelzle, 2009), are increasingly considered a necessary component of neuropsychological assessment (Bush et al., 2005). A National Academy of Neuropsychology position paper referred to such testing as “essential” and stated, “The clinician should be prepared to justify a decision not to assess symptom validity as part of a neuropsychological evaluation” (Bush et al., 2005, p. 421). The utility of an SVT, however, is only as strong as its capacity to measure effort in the context of an individual's actual cognitive abilities. Recent studies have suggested that some of the most frequently used SVTs may incorrectly identify individuals as putting forth poor effort when their scores are more likely reflective of genuine severe cognitive impairment or dementia (Teichner & Wagner, 2004; Merten, Bossink, & Schmand, 2007; Dean, Victor, Boone, Philpott, & Hess, 2009; Hook et al., 2009; Duff et al, 2011; Willis, Farrer, & Bigler, 2011; O'Maher et al., 2012). While the issue of false positives may be mitigated by comparing test results with the score profiles and performance base rates for individuals with similar impairments and considering the broader clinical context of a subject's performance (Bush et al., 2005; Howe, Anderson, Kaufman, Sachs, & Loring, 2007; Howe & Loring, 2009), the implications of incorrectly conflating cognitive abilities with effort remain significant.

The Test of Memory Malingering (TOMM), a forced-choice recognition test, is a stand-alone measure to determine adequacy of effort (Tombaugh, 1996). The TOMM was initially validated on cognitively intact individuals, as well as those with cognitive impairment, traumatic brain injury, and dementia (Tombaugh, 1997). Although the TOMM is used in dementia samples, Tombaugh notes in the manual that patients with dementia scored below the rest of the clinical sample (92% correct on trial 2 vs 97% correct) and 27% of their dementia sample failed (scored <45 on trial 2) in the validation study. Subsequent research has found that some manifestations of cognitive impairment result in false positives on the test. For instance, recent studies have found that the TOMM may particularly overestimate poor effort in individuals with moderate to severe dementia (Teichner & Wagner, 2004; Merten et al., 2007; Rudman, Oyebode, Jones, & Bentham, 2011).

The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) is a brief cognitive screening measure that is frequently used to evaluate individuals with a range of neuropsychological disorders, including dementia (Randolph, Tierney, Mohr, & Chase, 1998). Two embedded validity indicators for the RBANS have been proposed. The first, the EI, utilizes weighted scores from two subtests. While the EI has been found to be effective at differentiating between individuals with genuine and feigned traumatic brain injury (Silverberg, Wertheimer, & Fichtenberg, 2007), studies have found that the EI may result in an increased number of false positives in cognitively impaired individuals, including those diagnosed with probable Alzheimer's disease (Duff et al., 2011), cognitively impaired medically ill older adults (Hook et al., 2009), individuals with less education (Duff et al., 2011), and lower baseline cognitive status (O'Mahar et al., 2012).

Novitski, Steele, Karantzoulis, and Randolph (2012) recently reported on a new RBANS embedded symptom validity indicator, the Effort Scale (ES), which takes into account five RBANS subtests. The ES was developed to help identify poor effort among patients with “true amnesia” (i.e., Alzheimer's disease and other amnestic disorders), thus attempting to correct the problems identified with the EI. Novitski and colleagues assert that the ES results in fewer false positives than the EI, but the scale can only be appropriately applied to individuals with true cognitive impairment as defined by poor performance on the List Recognition or Digit Span subtests.

The question of whether it is appropriate to utilize SVTs with individuals who have dementia is salient for those suffering from Huntington disease (HD), an autosomal-dominant neurodegenerative condition resulting in motor, psychiatric, and cognitive impairment (Ross & Tabrizi, 2011). Cognitive decline is often the first symptom to emerge in HD, sometimes years prior to an official diagnosis, with the affected domains including executive functioning, attention, memory, and psychomotor skills (Ho et al., 2003; Paulsen et al., 2006; Beglinger et al., 2010; Duff, Paulsen, et al., 2010). Because HD has typical age of onset in middle adulthood (i.e., 40's), patients are often still working when they become ill. The disease leads to disability, thus impairments may be examined for validity during cognitive assessments, particularly when disability insurance benefits are at stake. To our knowledge, there are no published reports on SVT in an HD population. The purpose of this study is to characterize SVT performance in HD as a first step in developing profiles of cognitively impaired individuals with HD, and to examine the relationship between SVT performance and demographic and clinical variables. We hypothesize that cognitive impairment, lower education, and lower estimated premorbid functioning will be associated with worse performance on SVTs.

Methods

Participants and Procedure

Individuals seen for a clinical visit between 2002 and 2012 at the University of Iowa Huntington's Disease Society of America Center of Excellence (UI HDSA COE) served as participants for this study. All procedures were approved by the University of Iowa Institutional Review Board. Prior to data collection, participants signed informed consent documents notifying them of study risks and explaining that the data would be used for research purposes. Participants were not financially compensated. Participants were seen by a multi-disciplinary team, including a board-certified neurologist specializing in movement disorders, psychiatrist, and clinical neuropsychologist. All participants had a positive family history of HD; patients also had a clinical diagnosis of HD as determined by a movement disorder specialist and/or a positive test for the HD gene expansion. Additionally, each patient completed the Unified Huntington's Disease Rating Scale (UHDRS) battery (HSG, 1996), which is used for both clinical and research purposes, and were classified by the motor rater with a confidence rating that the motor signs were indicative of HD. Participants were included in these analyses if they completed a neuropsychological battery that included at least one SVT (the TOMM, the EI, and/or the ES) administered by a trained research assistant or clinical or counseling psychology doctoral student. Participants were included in each subsample for which they completed an SVT, thus individuals may be included in one, two, or three of the subsamples.

Measures

The RBANS is a brief cognitive screening measure that has 12 subtests, resulting in five index scores (attention, language, visuospatial/constructional abilities, immediate memory, and delayed memory) and a total score (Randolph et al., 1998). The index and total scores are age-corrected standard scores. For this study, all subtests were scored using the RBANS manual instructions except Figure Copy and Figure Recall, which were scored using revised criteria that allow more leniency in scoring minor drawing deviations (Duff et al., 2007). There are currently two established embedded symptom validity indices in the RBANS: the EI and the ES.

In this investigation, participants whose TOMM, EI, and ES scores suggest good effort will be referred to as having “passed,” while those whose SVT scores suggest poor effort will be referred to as having “failed” the indices. The EI converts the raw scores for two subtests, Digit Span and List Recognition, to weighted scores. These weighted scores are then summed, resulting in a number from 0 through 12, with higher scores indicating poorer effort (Silverberg et al., 2007). The generally accepted cut-off score to indicate poor effort of >3 was used with this sample (Silverberg et al., 2007; Hook et al., 2009; Duff et al., 2011). As described in Novitski et al. (2012), the formula for the ES includes five RBANS subtests (List Recognition, List Recall, Story Recall, Figure Recall, and Digit Span) and results in a number ranging from −8 through 28, with lower scores indicating poorer effort (Novitski et al., 2012). Novitski and colleagues (2012) proposed a cut-off score of <12, which was used with this sample. As suggested by Novitski, the scale is only applied to individuals who score poorly on List Recognition (raw < 19) and Digit Span (raw < 9). The TOMM, a forced-choice recognition test, is comprised of two required trials, as well as an optional retention trial to be used if the individual scores <45 of the 50 items on the second trial (Tombaugh, 1996). The manual recommended cut-off score of <45 on Trial 2 was used with this sample.

The UHDRS, a standardized assessment tool for HD, includes a motor examination, an assessment of functional and psychiatric abilities, and measures of cognitive ability (HSG, 1996). The Total Motor Score (TMS), which ranges from 0 to 124 with higher scores suggesting more impairment, considers individual motor signs, including finger tapping, chorea, rigidity, and dysarthria. The movement disorder specialist then assigns each participant a Diagnostic Confidence Level (DCL), a rating scale ranging from 0 (normal) to 4 (unquestionable HD, ≥99% confidence). The Total Functional Capacity (TFC), which is based on patient and companion report, considers ability to complete activities of daily living, occupation, finances, and other domains. The TFC ranges from 0 to 13, with greater scores signifying higher functioning. The UHDRS also includes three cognitive tests: the Controlled Oral Word Association Test (COWAT; Benton, Hamsher, & Sivan, 1994), which measures verbal fluency, the Symbol Digit Modalities Test—Written Form (SDMT; Smith, 1982), which assesses attention and psychomotor speed, and the three-part Stroop Color and Word Test (Stroop, 1935), which evaluates processing speed and verbal inhibition.

The Barona (Barona, Reynolds, Chastain, 1984) demographic equation is a regression formula that predicts premorbid IQ using demographic factors such as age, race, gender, education, occupation, and geographic region. The Beck Depression Inventory—II (BDI-II; Beck, Steer, & Brown, 1996) is a widely used, self-rated questionnaire that measures several symptoms associated with depression, including hopelessness, guilt, irritability, and fatigue.

Data Analytic Strategy

This paper describes three subsamples. The first subsample, for which we calculated the EI, includes all study participants (n = 121). The second subsample includes only those participants who met the ES parameters (i.e., below specific List Recognition and Digit Span cut scores) set by Novitski and colleagues (2012; n = 43). The TOMM was added to the battery more recently, with a resulting smaller sample (n = 36).

The EI and ES were calculated using the formulas described by the index authors (Silverberg et al., 2007; Novitski et al., 2012). Descriptive statistics were calculated for the demographic (age, gender, education) and clinical variables (DCL, TFC, TMS, RBANS Total Score, the five RBANS index scores, BDI-II, and Barona) for the three subsamples. To further examine the relationship between demographic and clinical variables and EI and ES performance, descriptive statistics and independent t-tests were performed for participants who both passed (EI ≤ 3; ES ≥ 12) and failed (EI > 3; ES < 12) the indices. (They were not performed for the TOMM due to smaller sample size.) The three SVTs were examined for associations with age, education, UHDRS variables (motor, functional, and cognitive abilities), premorbid IQ, and depression using Spearman's ρ correlation. Nonparametric statistics were used because scores were not normally distributed.

Participants in each subsample were divided into stages of HD, which is determined using the Total Functional Capacity scale from the UHDRS rating patients' capacity to perform everyday tasks (e.g., ambulate, manage finances, perform ADLs). Higher stages suggest poorer functioning (Stage 1: TFC = 11 to 13; Stage 2: TFC = 7 to 10; Stage 3: TFC = 3 to 6; Stage 4: TFC = 1 to 2). EI, ES, and TOMM pass/fail rates were determined for each stage. Due to small sample sizes for stage 4 on the EI and ES, stages 3 and 4 were combined.

Results

Descriptive statistics for demographic and clinical characteristics are provided in Table 1 for all participants, as well as for each subsample: EI, ES, and TOMM. The overall sample was 53% female and participants had an average age of 46 years (SD = 12.9), 13 years of education, and an average estimated premorbid IQ (Barona M = 106). UHDRS and cognitive scores indicate that the sample had abnormalities in motor functions, cognition and functional capacity (44% stage 1, 36% stage 2, 17% stage 3, 3% stage 4), consistent with mild to moderate HD. Some participants are included in more than one subsample.

Table 1.

Demographic and clinical characteristics of Total Sample/EI, ES, and TOMM participants

  All/EI participants
 
ES participants
 
TOMM participants
 
n M (SD)/range n M (SD)/range n M (SD)/range 
Age (years) 121 46.41 (12.92) 18–80 43 47.98/24–75 36 46.11 (12.59) 19–73 
Sex (% female) 121 53% 43 41.9% 36 42.1% 
Education (years) 121 13.31 (2.27) 8–20 43 12.56 (1.96) 8–17 36 13.58 (2.19) 8–20 
Baronaa 38 105.99 (7.77) 85.84–119 12 105.04 (6.63) 99.2–118 23 106.65 (6.81) 92–119 
TFC 107 9.32 (3.43) 1–15 37 7.7 (3.41) 1–15 33 9.88 (3.17) 3–15 
DCL 117 3.26 (1.05) 0–4 41 3.56 (.67) 2–4 34 3.32 (1.12) 0–4 
TMS 111 31.75 (18.45) 0–86 39 39.33 (16.61) 10–71 33 30.12 (21.18) 0–74 
RBANS Totala 116 69.38 (16.66) 43–114 42 54.24 (7.6) 43–80 36 70.19 (17.45) 41–114 
RBANS IMa 121 70.01 (20.17) 40–126 43 53.58 (11.87) 40–85 36 73.17 (22.10) 40–114 
RBANS VCa 118 80.14 (17.85) 50–121 43 73.05 (16.31) 50–112 36 89.19 (17.57) 56–121 
RBANS La 121 81.88 (12.25) 40–122 43 75.28 (11.35) 40–95 36 83.03 (9.60) 64–112 
RBANS Aa 118 68.31 (17.50) 40–122 42 53.43 (8.37) 40–75 36 71.97 (16.72) 40–106 
RBANS DMa 118 74.37 (21.60) 40–131 43 53.30 (10.23) 40–81 36 78.17 (24.02) 40–131 
BDI-II 60 15.48 (12.16) 0–43 20 18.7% (14.01) 1–43 36 12.89 (9.83) 0–35 
SVT 121 EI = 1.91 (2.29) 0–9 43 ES = 10.53 (4.51) −4–20 36 47.92 (4.81) 29–50 
SVT % Pass 121 81.8% 44 30.2% 36 91.8% 
  All/EI participants
 
ES participants
 
TOMM participants
 
n M (SD)/range n M (SD)/range n M (SD)/range 
Age (years) 121 46.41 (12.92) 18–80 43 47.98/24–75 36 46.11 (12.59) 19–73 
Sex (% female) 121 53% 43 41.9% 36 42.1% 
Education (years) 121 13.31 (2.27) 8–20 43 12.56 (1.96) 8–17 36 13.58 (2.19) 8–20 
Baronaa 38 105.99 (7.77) 85.84–119 12 105.04 (6.63) 99.2–118 23 106.65 (6.81) 92–119 
TFC 107 9.32 (3.43) 1–15 37 7.7 (3.41) 1–15 33 9.88 (3.17) 3–15 
DCL 117 3.26 (1.05) 0–4 41 3.56 (.67) 2–4 34 3.32 (1.12) 0–4 
TMS 111 31.75 (18.45) 0–86 39 39.33 (16.61) 10–71 33 30.12 (21.18) 0–74 
RBANS Totala 116 69.38 (16.66) 43–114 42 54.24 (7.6) 43–80 36 70.19 (17.45) 41–114 
RBANS IMa 121 70.01 (20.17) 40–126 43 53.58 (11.87) 40–85 36 73.17 (22.10) 40–114 
RBANS VCa 118 80.14 (17.85) 50–121 43 73.05 (16.31) 50–112 36 89.19 (17.57) 56–121 
RBANS La 121 81.88 (12.25) 40–122 43 75.28 (11.35) 40–95 36 83.03 (9.60) 64–112 
RBANS Aa 118 68.31 (17.50) 40–122 42 53.43 (8.37) 40–75 36 71.97 (16.72) 40–106 
RBANS DMa 118 74.37 (21.60) 40–131 43 53.30 (10.23) 40–81 36 78.17 (24.02) 40–131 
BDI-II 60 15.48 (12.16) 0–43 20 18.7% (14.01) 1–43 36 12.89 (9.83) 0–35 
SVT 121 EI = 1.91 (2.29) 0–9 43 ES = 10.53 (4.51) −4–20 36 47.92 (4.81) 29–50 
SVT % Pass 121 81.8% 44 30.2% 36 91.8% 

Notes: TFC = Total Functional Capacity; DCL = Diagnostic Confidence Level; TMS = Total Motor Scale; RBANS = Repeatable Battery for the Assessment of Neuropsychological Status; RBANS IM: Immediate Memory Index; RBANS VC: Visuospatial/Constructional Index; RBANS L: Language Index; RBANS A: Attention Index; RBANS DM: Delayed Memory Index; BDI-II = Beck Depression Inventory-II; SVT = Symptom Validity Test.

SVT cut-scores: EI Pass ≤ 3; ES Pass ≥ 12; TOMM Pass ≥ 45. Higher scores indicate better performance/less impairment on the RBANS and TFC. Lower scores indicate better performance/less impairment on the TMS, DCL, and BDI-II.

aStandard scores with M = 100; SD = 15.

Effort Index

Descriptive statistics for demographic and clinical variables are provided in Table 1 for the EI (n = 121) participants. The vast majority (81.8%) received a passing score of 3 or lower on the EI. Independent sample t-tests revealed that the EI Pass group performed significantly better than the EI Fail Group on the following: RBANS Total (M = 73.5, SD = 15.7 vs. M = 51.77, SD = 4.77; t(107) = 11.34, p < .01), Barona (M = 106.79, SD = 8.02 vs. M = 100.68, SD = 1.99; t(27) = 3.68, p < .01) and TFC (M = 9.4, SD = 3.40 vs. M = 7.53, SD = 3.06; t(105) = 2.58, p < .05). The EI Pass group also was less impaired on the following: TMS (M = 29.98, SD = 17.80 vs. M = 40.89, SD = 19.57; t(109) = −2.34, p < .05) and DCL (M = 3.2, SD = 1.11 vs. M = 3.6, SD = .68; t(43) = − 2.14, p < .05). The pass and fail groups were not statistically different on the BDI-II.

Bivariate correlations revealed that fewer years of education (ρ = −0.25, p < .05), lower scores on several cognitive measures (RBANS Total, ρ = −0.76, p < .01; COWAT, ρ = −0.56, p < .01; SDMT, ρ = −0.60, p < .01; Stroop-Color, ρ = −0.56, p < .01; Stroop-Word, ρ = −0.49, p < .01; Stroop-Interference, ρ = −0.54, p < .01), lower scores on adaptive functioning (TFC, ρ = −0.39, p < .01) and increased motor impairment (TMS, ρ = 0.36, p < .01; DCL, ρ = 0.27, p < .01) were related to higher EI scores, with higher EI scores suggesting poorer effort (Table 2).

Table 2.

Spearman's correlations between demographic and clinical characteristics and EI, ES, and TOMM scores

 n EI n ES n TOMM 
Age 121 0.07 43 0.28 36 −0.23 
Education 121 −0.22* 43 −0.13 33 0.26 
Barona 38 −0.07 12 0.05 23 0.40 
TFC 107 −0.39** 37 −0.41* 33 0.46** 
DCL 117 0.27** 42 0.17 34 −0.24 
TMS 111 0.36** 39 0.16 33 −0.39* 
RBANS Total 116 −0.76*** 42 −0.17 36 0.59** 
BDI-II 60 0.12 20 0.09 36 0.06 
COWAT 113 −0.56** 38 −0.43** 35 0.66** 
SDMT 109 −0.60** 36 −0.34* 35 0.48* 
Stroop - Color 112 −0.56** 38 −0.35* 35 0.41* 
Stroop - Word 111 −0.49** 37 −0.32 35 0.44** 
Stroop - Interference 111 −0.54** 37 −0.32 35 0.35* 
 n EI n ES n TOMM 
Age 121 0.07 43 0.28 36 −0.23 
Education 121 −0.22* 43 −0.13 33 0.26 
Barona 38 −0.07 12 0.05 23 0.40 
TFC 107 −0.39** 37 −0.41* 33 0.46** 
DCL 117 0.27** 42 0.17 34 −0.24 
TMS 111 0.36** 39 0.16 33 −0.39* 
RBANS Total 116 −0.76*** 42 −0.17 36 0.59** 
BDI-II 60 0.12 20 0.09 36 0.06 
COWAT 113 −0.56** 38 −0.43** 35 0.66** 
SDMT 109 −0.60** 36 −0.34* 35 0.48* 
Stroop - Color 112 −0.56** 38 −0.35* 35 0.41* 
Stroop - Word 111 −0.49** 37 −0.32 35 0.44** 
Stroop - Interference 111 −0.54** 37 −0.32 35 0.35* 

Notes: TFC = Total Functional Capacity; DCL = Diagnostic Confidence Level; TMS = Total Motor Scale; RBANS = Repeatable Battery for the Assessment of Neuropsychological Status; BDI-II = Beck Depression Inventory-II; COWAT = Controlled Oral Word Association Test; SDMT = Symbol Digit Modalities Test.

*p < .05.

**p < .01.

***p < .001.

When participants were divided by HD stage, 91.7% of participants in stage 1 (n = 44/48), 78.9% of those in stage 2 (n = 30/38), and 66.6%% in stages 3 and 4 (n = 14/21) passed the EI.

Effort Scale

Forty-three participants met criteria to apply an ES score (i.e., List Recognition raw scores of <19 and Digit Span raw scores of <9, see Table 1 for descriptive statistics). On average, the group was characteristic of patients with mild to moderate HD (27% stage 1, 39% stage 2, 31% stage 3, 3% stage 4). Less than one-third of participants (30.2%) received a passing score of 12 or higher on the Effort Scale. There were no significant differences between the ES Pass and Fail groups on any of the demographic and clinical variables measured. Bivariate correlations revealed that higher levels of daily functioning (TFC, ρ = −0.41, p < .05), higher verbal fluency (COWAT, ρ = −0.43, p < .01), attention and psychomotor speed (SDMT, ρ = −0.34, p < .05), and processing speed (Stroop-Color, ρ = −0.35, p < .05) were related to lower ES scores, with lower ES scores suggesting poorer effort (Table 2).

When participants were divided by HD stage, 11.1% of participants in stage 1 (n = 1/10), 13.3% of those in stage 2 (n = 2/15), and 46.2% in stages 3 and 4 (n = 6/13) passed the ES.

Test of Memory Malingering

Descriptive statistics for demographic and clinical variables are provided in Table 1 for the TOMM (n = 36) participants. As expected, the group was characterized by patients with mild to moderate HD (45% stage 1, 43% stage 2, 12% stage 3). The vast majority of participants (91.8%) received a passing score of 45 or higher on the second trial of the TOMM. Scores ranged from 29 to 50. All of the participants who failed Trial 2 (8.2%) also failed the Retention trial. Bivariate correlations revealed that higher levels of functioning (TFC, ρ = 0.46, p = .01) and lower motor impairment (TMS, ρ = −0.39, p < .05) were related to passing TOMM scores, as were higher scores on several cognitive measures (RBANS Total, ρ = 0.59, p < .01; COWAT, ρ = 0.66, p < .01; SDMT, ρ = 0.48, p < .05; Stroop-Color, ρ = 0.41, p < .05; ρ = Stroop-Word, ρ = 0.44, p < .01; Stroop-Interference, ρ = 0.35, p < 0.05).

When participants were divided by HD stage, 86.7% of participants in stage 1 (n = 13/15), 100% of those in stage 2 (n = 14), and 75% in stage 3 (n = 3/4) passed the TOMM.

Discussion

Recent studies have found that performance on some commonly used SVTs may be affected by cognitive impairment or dementia, yet the impact of HD on SVT performance has been previously unreported. The present study sought to characterize how a sample of HD patients that presented for a routine clinical visit performed on three SVTs and to examine the relationship between SVT performance and demographic and clinical variables. These findings set the foundation for the development of score profiles for individuals diagnosed with HD at different levels of cognitive impairment, which may bolster the detection of false positives in future samples (Howe et al., 2007; Howe & Loring, 2009).

The vast majority of participants passed the EI and the TOMM. This is consistent with Mittenberg, Patton, Canyock, & Condit (2002) which, in a survey of American Board of Clinical Neuropsychology members, found that only 8% of medical cases resulted in questionable symptom validity. In our study, most (though not all) clinical and demographic variables were correlated with EI and TOMM performance and those participants who failed the EI and TOMM tended to have greater cognitive impairment and higher disease-related symptoms. However, less than half of the participants passed the ES and few clinical and demographic variables were correlated with SVT performance, suggesting that the EI and TOMM may be more useful measures of symptom validity in the HD population, whereas the ES may not be suited to this population.

Although the appropriateness of traditional SVT cutoff scores in patients with cognitive impairment has been questioned, most of our samples passed the EI (81.8%) and the TOMM (91.8%), using the typical cutoff scores (EI Pass ≤ 3; TOMM Pass ≥ 45). As illustrated in Fig. 1, all participants who passed the measures had RBANS Total scores above 65. However, cognitive impairment was associated with failing SVT scores. This finding is consistent with current literature that suggests that the EI and TOMM may result in more false positives among individuals with cognitive impairment (Teichner & Wagner, 2004; Hook et al., 2009; Duff et al., 2011; O'Mahar et al., 2012). Our hypothesis that poorer cognitive performance would be associated with worse performance on the TOMM and EI was supported by a number of cognitive measures, including the RBANS, COWAT, SDMT, and Stroop Color and Word Test. Additionally, as disease stage worsened, the number of those failing the EI steadily rose. There were also associations between poor SVT scores and lower adaptive functioning and greater motor impairment, indicating that disease progression is associated with worse performance. However, the level of education was only associated with performance on the EI, and premorbid IQ was not associated with performance on either SVT measure, although low sample size for the Barona may have affected results. Depression was also not associated with SVT performance.

Fig. 1.

Relationship between SVT scores and RBANS Total score. The three panels illustrate the relationship between each SVT score and RBANS Total score. The range of possible scores for each SVT is shown on the x-axis. RBANS Total scores are shown on the y-axis. Published SVT cut-scores are as follows: EI Pass ≤ 3 (lower scores suggest better effort); ES Pass ≥ 12 (higher scores suggest better effort); TOMM Pass ≥ 45 (higher scores suggest better effort).

Fig. 1.

Relationship between SVT scores and RBANS Total score. The three panels illustrate the relationship between each SVT score and RBANS Total score. The range of possible scores for each SVT is shown on the x-axis. RBANS Total scores are shown on the y-axis. Published SVT cut-scores are as follows: EI Pass ≤ 3 (lower scores suggest better effort); ES Pass ≥ 12 (higher scores suggest better effort); TOMM Pass ≥ 45 (higher scores suggest better effort).

Results for the ES were somewhat unexpected, with less than a third of the sample passing when using the proposed cut-off score (ES Pass ≥ 12). Furthermore, among the few significant associations between demographic and clinical variables and the ES, the findings ran counter to our hypothesis that passing ES scores would be related to higher cognitive and adaptive functioning. Although we used the Novitski and colleagues (2012) parameters of low scores on List Recognition and/or Digit Span to compile our ES sample (resulting in a more cognitively impaired sample compared with the other two groups), our results were not consistent with the previous publication, which found fewer false positives when evaluating effort in their cognitively impaired sample. This may be explained by the difference in samples, with Novitski and colleagues developing the ES to discriminate individuals with “true amnesia” from those with suspect effort, which they define as patients with disproportionately poor recognition relative to free recall and/or poor digit span. Although patients with HD do show memory impairment, the pattern of deficits on the RBANS may be different enough from those with cortical dementias as to render the ES ineffective in patients with subcortical patterns of impairment. In particular, patients with HD have more prominent attention deficits. In our sample, the RBANS Attention Index was the most impaired Index score in all three subsamples. The Novitski parameters were developed for patients in which “simple working memory as measured by the Digit Span subtest would still be expected to remain relatively intact …” (p. 191). Our patients showed more impaired attention with less impaired free recall than the Novitski parameters were designed for. Previous research has shown the RBANS to discriminate cortical from subcortical dementias (Randolph et al., 1998; Duff, Beglinger, et al., 2010). Moreover, in our sample, the ES could not be calculated for 64% of HD patients with cognitive impairment because they performed above the recommended cut-off (i.e., too well) raising concerns about the usefulness in this population.

Although the vast majority of participants in the EI and TOMM samples passed the SVT indices, performance was associated with cognitive impairment and symptom level (i.e., motor impairment and functional status) raising concern about administering such measures to more severely cognitively and motorically impaired populations. Patients who performed in the borderline to normal range on the RBANS, and even patients with impaired performance on the RBANS (i.e., with RBANS Total Scores down into the mid 60s), were able to pass the EI and TOMM reliably. This finding is further evidence that patients with milder impairments should be able to pass the traditional cutoff scores on the EI and TOMM. More work is needed among those with moderate to severe cognitive impairments to address how to assess level of effort. Future studies should consider whether more lenient cut scores or use of SVTs with easier items, such as those developed for patients with dementia (e.g., Medical Symptom Validity Test; Howe & Loring, 2009), would be more appropriate for patients past the mild stages of dementia. Research may also build on the current results in order to create score profiles for individuals with HD and cognitive impairment so that future clinicians may make more informed determinations of effort. Additionally, it will be important to use the ES in other populations suffering from non-amnestic types of dementia (such as Parkinson's disease) to see if it is generalizable to disorders with similar memory profiles to HD, or if it is best used with individuals with suspected Alzheimer's disease.

This study contributes to the literature on the impact of cognitive impairment on SVTs in HD; however, some limitations should be noted. We do not know the extent of each patient's effort and motivation. While we assume reasonable effort given the context of the evaluation (i.e., non-forensic), this remains an open question in our sample as no gold standard yet exists for the assessment of effort testing in HD. We cannot definitively rule out that some of the patients in the sample did not give full effort. Additionally, two of our three SVTs (the EI and the ES) were embedded measures of one of our main clinical variables, the RBANS, resulting in a risk of circularity. To minimize the impact of this, we supplemented our analyses with additional cognitive tests to the extent available. Finally, since this is the first study to address the issue of SVTs in individuals with HD, these results should be considered preliminary and are in need of replication. Although the sample size is quite reasonable by HD research standards, we do acknowledge that the TOMM and ES subsamples are modest and statistical power may be an issue. Future studies are needed to evaluate SVT performance in HD with larger samples and other commonly used SVTs.

This study indicates that traditional symptom validity cut scores are effective in the majority of individuals with mild to moderate cognitive impairment; however, in more impaired patients there may be a higher percentage of SVT failures. All subjects with RBANS Total scores above 65 passed the EI and TOMM, suggesting that caution should be used in evaluating SVT results in individuals with RBANS Totals below that score. This is a salient issue for individuals with HD who apply for disability, especially those with greater cognitive impairment. Validity of SVTs may be linked to disease stage, with a higher tendency to fail among those who are later in the progression of HD.

Funding

This work was supported by The University of Iowa Huntington's Disease Society of America Center of Excellence.

Conflict of Interest

None declared.

Acknowledgements

We would like to thank Anne Leserman and the staff of the clinic at the University of Iowa Huntington's Disease Society of America Center of Excellence. We are grateful to individuals with HD and their families who have contributed their time to participate in clinical research.

References

Barona
A.
Reynolds
C. R.
Chastain
R.
A demographically based index of premorbid intelligence test for the WAIS-R
Journal of Consulting and Clinical Psychology
 , 
1984
, vol. 
52
 (pg. 
885
-
887
)
Beck
A. T.
Steer
R. A.
Brown
G.
BDI-II: Beck Depression Inventory-II manual
 , 
1996
New York
Psychological Corporation
Beglinger
L. J.
Duff
K.
Allison
J.
Theriault
D.
O'Rourke
J. J.
Leserman
A
, et al.  . 
Cognitive change in patients with Huntington disease on the Repeatable Battery for the Assessment of Neuropsychological Status
Journal of Clinical and Experimental Neuropsychology
 , 
2010
, vol. 
32
 (pg. 
573
-
587
)
Benton
A. L.
Hamsher
K. d. S.
Sivan
A. B.
Multilingual Aphasia Examination
 , 
1994
3rd ed.
San Antonio, TX
Psychological Corporation
Bush
S. S.
Ruff
R. M.
Troster
A. I.
Barth
J. T.
Koffler
S. P.
Pliskin
N. H.
, et al.  . 
Symptom validity assessment: Practice issues and medical necessity NAN policy & planning committee
Archives of Clinical Neuropsychology
 , 
2005
, vol. 
20
 (pg. 
419
-
426
)
Dean
A. C.
Victor
T. L.
Boone
K. B.
Philpott
L. M.
Hess
R. A.
Dementia and effort test performance
The Clinical Neuropsychologist
 , 
2009
, vol. 
2
 (pg. 
133
-
152
)
Duff
K.
Beglinger
L. J.
Theriault
D.
Allison
J.
Paulsen
J. S.
Cognitive deficits in Huntington's disease on the Repeatable Battery for the Assessment of Neuropsychological Status
Journal of Clinical and Experimental Neuropsychology
 , 
2010
, vol. 
32
 (pg. 
231
-
238
)
Duff
K.
Leber
W. R.
Patton
D. E.
Schoenberg
M. R.
Mold
J. W.
Scott
J. G.
, et al.  . 
Modified scoring criteria for the RBANS figures
Applied Neuropscyhology
 , 
2007
, vol. 
14
 (pg. 
73
-
83
)
Duff
K.
Paulsen
J.
Mills
J.
Beglinger
L. J.
Moser
D. J.
Smith
M. M.
, et al.  . 
PREDICT-HD Investigators and Coordinators of the Huntington Study Group
Mild cognitive impairment in prediagnosed Huntington disease
Neurology
 , 
2010
, vol. 
75
 (pg. 
500
-
507
)
Duff
K.
Spering
C. C.
O'Bryant
S. E.
Beglinger
L. J.
Moser
D. J.
Bayless
J. D.
, et al.  . 
The RBANS Effort Index: Base rates in geriatric samples
Applied Neuropsychology
 , 
2011
, vol. 
18
 (pg. 
11
-
17
)
Ho
A. K.
Sahakian
B. J.
Brown
R. G.
Barker
R. A.
Hodges
J. R.
Ané
M. N.
, et al.  . 
NEST-HD Consortium
Profile of cognitive progression in early Huntington's disease
Neurology
 , 
2003
, vol. 
61
 (pg. 
1702
-
1706
)
Hook
J. N.
Marquine
M. J.
Hoelzle
J. B.
Repeatable battery for the assessment of neuropsychological status effort index performance in a medically ill geriatric sample
Archives of Clinical Neuropsychology
 , 
2009
, vol. 
24
 (pg. 
231
-
235
)
Howe
L. L. S.
Anderson
A. M.
Kaufman
D. A. S.
Sachsa
B. C.
Loring
D. W.
Characterization of the Medical Symptom Validity Test in evaluation of clinically referred memory disorders clinic patients
Archives of Clinical Neuropsychology
 , 
2007
, vol. 
22
 (pg. 
753
-
761
)
Howe
L. L. S.
Loring
D. W.
Classification accuracy and predictive ability of the Medical Symptom Validity Test's dementia profile and general memory impairment profile
The Clinical Neuropsychologist
 , 
2009
, vol. 
23
 (pg. 
329
-
342
)
HSG
Unified Huntington's Disease Rating Scale: Reliability and consistency
Movement Disorders
 , 
1996
, vol. 
11
 (pg. 
136
-
142
Merten
T.
Bossink
L.
Schmand
B.
On the limits of effort testing: Symptom validity tests and severity of neurocognitive symptoms in nonlitigant patients
Journal of Clinical and Experimental Neuropsychology
 , 
2007
, vol. 
29
 (pg. 
308
-
318
)
Mittenberg
W.
Patton
C.
Canyock
E. M.
Condit
D. C.
Base rates of malingering and symptom exaggeration
Journal of Clinical and Experimental Neuropsychology
 , 
2002
, vol. 
24
 (pg. 
1094
-
1102
)
Novitski
J.
Steele
S.
Karantzoulis
S.
Randolph
C.
The Repeatable Battery for the Assessment of Neuropsychological Status Effort Scale
Archives of Clinical Neuropsychology
 , 
2012
, vol. 
27
 (pg. 
190
-
195
)
O'Mahar
K. M.
Duff
K.
Scott
J. G.
Linck
J. F.
Adams
R. L.
Mold
J. W.
Brief report: The temporal stability of the Repeatable Battery for the Assessment of Neuropsychological Status Effort Index in geriatric samples
Archives of Clinical Neuropsychology
 , 
2012
, vol. 
27
 (pg. 
114
-
118
)
Paulsen
J. S.
Hayden
M.
Stout
J. C.
Langbehn
D. R.
Aylward
E.
Ross
C. A.
, et al.  . 
the PREDICT-HD Investigators of the Huntington Study Group
Preparing for preventive clinical trials: The Predict-HD Study
Archives of Neurology
 , 
2006
, vol. 
63
 (pg. 
883
-
890
)
Randolph
C.
Tierney
M. C.
Mohr
E.
Chase
T. N.
The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS): Preliminary Clinical Validity
Journal of Clinical and Experimental Neuropsychology
 , 
1998
, vol. 
20
 (pg. 
310
-
319
)
Ross
C. A.
Tabrizi
S. J.
Huntington's disease: From molecular pathogenesis to clinical treatment
The Lancet Neurology
 , 
2011
, vol. 
10
 (pg. 
83
-
98
)
Rudman
N.
Oyebode
J. R.
Jones
C. A.
Bentham
P.
An investigation into the validity of effort tests in a working age dementia population
Aging & Mental Health
 , 
2011
, vol. 
15
 (pg. 
47
-
57
)
Silverberg
N. D.
Wertheimer
J. C.
Fichtenberg
N. L.
An effort index for the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS)
Clinical Neuropsychology
 , 
2007
, vol. 
21
 (pg. 
841
-
854
)
Smith
A.
Symbol Digit Modalities Test (SDMT) Manual (Revised)
 , 
1982
Los Angeles
Western Psychological Services
Stroop
J. R.
Studies of interference in serial verbal reactions
Journal of Experimental Psychology
 , 
1935
, vol. 
18
 (pg. 
643
-
662
)
Teichner
G.
Wagner
M.
The test of memory malingering (TOMM): Normative data for cognitively intact, cognitively impaired, and elderly patients with dementia
Archives of Clinical Neuropsychology
 , 
2004
, vol. 
19
 (pg. 
455
-
464
)
Tombaugh
T. N.
The Test of Memory Malingering
 , 
1996
Toronto, Canada
Multi-Health Systems
Tombaugh
T. N.
The test of memory malingering (TOMM): Normative data from cognitively intact and cognitively impaired individuals
Psychological Assessment
 , 
1997
, vol. 
9
 (pg. 
260
-
268
)
Willis
P. F.
Farrer
T. J.
Bigler
E. D.
Are effort measures sensitive to cognitive impairment
Military Medicine
 , 
2011
, vol. 
176
 (pg. 
1426
-
1431