Abstract

Previous work in a disability-seeking sample has demonstrated that as symptom validity test (SVT) scores decline, there is a corresponding increase in subjective reports of memory problems as measured by the Memory Complaints Inventory (MCI). The current archival study examined this relationship in a clinical sample of active and retired military service members and their adult family members without overt potential for secondary gain (n= 191). General support for the previously evidenced relationship between SVT performances and MCI responses was found. Select MCI subscales (i.e., Amnesia for Complex Behavior and Amnesia for Antisocial Behavior) were not as strongly correlated with SVT scores as in the previously studied disability-seeking groups. The relationship between performances on an embedded effort measure and MCI scores was not nearly as robust as the relationship between MCI profiles and stand-alone SVTs. The potential clinical implications for these findings are discussed.

Introduction

Among patients presenting for neuropsychological assessment, subjective reports of memory deficits are among the most common complaints (Lezak, Howieson, & Loring, 2004). However, across a wide variety of diagnostic groups, the existing literature reveals little correlation between subjective complaints of poor memory and objective findings on standardized measures of memory functioning (Ford, Slade, & Butler, 2004; Landrø, Sletvold, & Celius, 2000; Matotek, Saling, Gates, & Sedal, 2001; Mol, van Boxtel, Willems, & Jolles, 2006; Pearman, 2009; Suhr, 2003). Historically, common explanations for the limited correlation between subjective memory complaints and objective test findings have included psychological, somatic, and personality factors such as depression (Chamelian & Feinstein, 2006), emotional distress (Munoz & Esteve, 2005), pain (McCracken & Iverson, 2001), and neuroticism (Pearman & Storandt, 2004).

Response bias is another contributing factor to consider in this relationship. Gervais, Ben-Porath, Wygant, and Green (2008) showed that the Response Bias Scale, an Minnesota Multiphasic Personalty Inventory -2 (MMPI-2) scale designed to detect cognitive response bias (Gervais, Ben-Porath, Wygant, & Green, 2007), was related to increased Memory Complaints Inventory (MCI; Green, 2004a) scores in a large sample of non-head injured disability referrals. MCI scores were also significantly higher in those fibromyalgia patients who failed effort tests than in those who passed effort tests, indicating that symptom exaggeration can affect self-reports of memory problems (Gervais et al., 2001). An increasing number of studies demonstrate that, within the confines of neuropsychological testing, effort accounts for more variance in test scores than does neurological insult (Constantinou, Bauer, Ashendorf, Fisher, & McCaffrey, 2005; Fox, 2011; Green, 2007; Green, Rohling, Lees-Haley, & Allen, 2001; Meyers, Volbrecht, Axelrod, & Reinsch-Boothby, 2011; Moss, Jones, Forkis, & Quinn, 2003; Stevens, Friedel, Mehren, & Merten, 2008; West, Curtis, Greve, & Bianchini, 2010). As such, it appears important to consider whether impression management strategies are contributing to elevated reports of memory complaints in neuropsychological evaluations.

A recent study by Armistead-Jehle, Gervais, and Green (2012) demonstrated that as the symptom validity test (SVT) performances of individuals involved in disability examinations worsened, there was a corresponding increase in subjective reports of memory complaints across all MCI subscales. These data were taken to suggest that an underlying dimension of symptom exaggeration served to influence self-reported memory complaints and also effort on SVTs. Although this study included two independent samples of notable size, all those studied were involved in some sort of disability or compensation evaluation and, as such, the study's external validity was limited. The aim of the current study was to evaluate the relationship between subjective memory complaints and performance on various SVTs in an outpatient sample not known to be involved in compensation or disability claims. The authors hypothesized that the relationship between MCI item endorsement and symptom validity testing demonstrated in the original Armistead-Jehle and colleagues (2012) publication would be maintained. However, given the non-disability seeking nature of the current sample, it was further hypothesized that this relationship would be somewhat attenuated.

Method

The study included a total of 191 adults evaluated by the first author in an outpatient neuropsychology clinic located at Midwest U.S. Army Health Center between June 2009 and December 2011. The sample included military service members as well as adult family members and military retirees referred for neuropsychological evaluation from either primary care or behavioral health. Referral questions typically related to the patient's reported cognitive complaints hypothesized by the referring provider to be a result of neurologic insult and/or possible psychiatric conditions. All participants spoke English fluently and the testing was conducted in English. The average age of the sample was 36.4 years (SD = 8.7), with an average education of 15.5 years (SD = 2.1). Predicted Wechsler Adult Intelligence Scale - 3rd Edition Full Scale Intelligence Quotient (uncorrected for effort) was determined using demographically corrected performance on the Wechsler Test of Adult Reading (Pearson, 2001) and averaged 108.3 (SD = 9.2). Eighty-four percent of the sample was male. With regard to neurological diagnosis, the majority of the sample (n= 140, 73%) consisted of people with a remote history of at least one mild Traumatic Brain Injury (TBI) as defined by the American Congress of Rehabilitation Medicine criteria (ACRM, 1993). Two had a remote history of moderate or severe TBI (1%). At the time of evaluation, all patients with a history of TBI were at least 2 months out from their injury, with most patients experiencing their injuries more than a year before the assessment. Other neurologic diagnoses included a history of cerebral vascular accident (1%), carbon monoxide poisoning (0.5%), narcolepsy (0.5%), a history of radiation and chemotherapy secondary to throat cancer (0.5%), history of hypoxia (0.5%), chronic fatigue syndrome (0.5%), sleep apnea (0.5%), and mild cognitive impairment secondary to Parkinson's disease (0.5%). Forty individuals (21%) did not present with any history of neurological diagnoses. Non-mutually exclusive psychiatric diagnoses consisted of Posttraumatic Stress Disorder (n= 41, 21%), other Anxiety Disorder (n= 64, 34%), Unipolar Depressive Disorder (n= 46, 24%), Substance Use Disorder (n= 3, 2%), Adjustment Disorder (n= 7, 4%), and Attention Deficit Hyperactivity Disorder (n = 2, 1%). Ethnic breakdown was as follows: Caucasian (78.0%), African American (15.3%), Hispanic (4.7%), Asian American (1.0%), Pacific Islander (0.5%), and Native American (0.5%). The majority of the sample consisted of active duty service members (78%), with smaller percentages of U.S. Army Reserve or National Guard Members (7.3%), adult family members (10.5%), and military veterans (4.2%). No participants were known to be pending any litigation. Those service members whose evaluations were conducted in the context of a Medical Evaluation Board or Temporary Disability Retirement List evaluations were considered to be involved in a disability type evaluation. Consequently, they were never considered for analysis and are not included in the descriptive statistics reported above. This retrospective analysis of clinical data was approved by the Institutional Review Board at Madigan Army Medical Center.

Procedures

All patients were tested by the first author or a trained neuropsychology technician under the supervision of the first author. Prior to all evaluations, the patients gave consent for the assessment and were instructed to provide their best effort across the tests administered. As part of a neuropsychological test battery, all patients were administered the MCI (Green, 2004a) as well as a range of SVTs and cognitive measures. Consecutive referrals were reviewed for cases that were administered the MCI and at least one of the following: the Medical Symptom Validity Test (MSVT; Green, 2004b), Non-Verbal MSVT (NV-MSVT; Green, 2008), Test of Memory Malingering (TOMM: Tombaugh, 1996), or Repeatable Battery for the Assessment of Neuropsychological Status (RBANS; Randolph, 1998) Effort Index (EI; Silverberg, Wertheimer, & Fichtenberg, 2007). Not every patient was administered the MCI and every SVT noted above. The patients were not alerted to the types of SVTs employed or the order within test administration. SVT administration was scattered throughout the neuropsychological battery with the MSVT administered as the first SVT followed by the NV-MSVT, TOMM, and RBANS EI. The examiner remained in the room at all times throughout the evaluations.

Measures

The MCI is a 58-item computer administered self-report measure of potential memory problems. The measure consists of nine scales rationally designed to tap specific types of reported memory problems: General Memory Problems (GMP), Numeric Information Problems (NIP), Visuospatial Memory Problems (VSMP), Verbal Memory Problems (VMP), Pain Interferes with Memory (PIM), Memory Interferes with Work (MIW), Impairment of Remote Memory (IRM), Amnesia for Complex Behavior (ACB), and Amnesia for Antisocial Behavior (AAB). The first six scales include plausible memory complaints. In contrast, the latter three scales were intentionally designed to consist of items that would be considered implausible for most individuals with memory problems secondary to an organic etiology (e.g., “There are large gaps in my memory of my childhood” [IRM]; “I have hit people and had no memory of doing it” [AAB]; “Minutes or hours can go by and I have no idea what I have just done” [ACB]). The endorsement of such symptoms in clinical practice is typically thought to reflect either psychiatric origins or exaggerated/feigned memory complaints. The MCI has been employed as an external criterion for the validation of several MMPI-2 and Minnesota Multiphasic Personalty Inventory -2-Restructred Form subscales, including Infrequency, Back Infrequency, Infrequency Psychopathy, Symptom Validity Scale (FBS), Response Bias Scale, Infrequent Responses, Infrequent Psychopathology Responses, and Symptom Validity (Gervais, Ben-Porath, Wygant, & Sellbom, 2010; Gervais et al., 2008).

The MSVT (Green, 2004b) is a brief automated verbal memory screening with several subtests designed to measure verbal memory and response consistency. Ten easy-to-remember word pairs representing a single common object (e.g., ballpoint pen) are shown across two trials. Afterwards, two forced-choice recognition subtests are administered and a response consistency score between these subtests in generated. The measure is concluded with Paired Associates and Free Recall trials. In addition to data presented in the MSVT manual, a number of studies have demonstrated the utility of this measure in the discrimination between those with genuine memory impairment and those simulating impairment in a range of patient samples (e.g., Chafetz, 2008; Howe & Loring, 2009; Singhal, Green, Ashaye, Shankar, & Gill, 2009; see Carone, 2009, for review). For instance, the specificity of the MSVT in patients with possible dementia have been reported as 91 and 100% (Howe & Loring, 2009; Singhal et al., 2009). Sensitivity has been reported to approximately 97% in simulator studies (Green, 2004b).

The NV-MSVT (Green, 2008) is a brief automated non-verbal memory screening with several subtests designed to measure non-verbal memory and response consistency. Ten artist-drawn colored images representing an intuitive pair of items (e.g., a baseball and a baseball bat) are shown across two trials. Afterwards, a series of forced-choice trials with varying degrees of difficulty are presented. These are followed by a Paired Associates recall subtest and the measure is concluded with a Free Recall task. According to the manual, the NV-MSVT was reported to have 100% specificity in good effort volunteers, 95% specificity in patients diagnosed with dementia, and 72.5% sensitivity to poor effort in simulators. The sensitivity and specificity of the NV-MSVT as a measure of symptom validity has been further validated in a number of other studies (e.g., Green, Flaro, Brockhaus, & Montijo, 2010; Henry, Merten, Wolf, & Harth, 2009; Singhal et al., 2009; see Wager & Howe, 2010, for review).

The TOMM (Tombaugh, 1996) is a forced-choice recognition task involving the presentation of 50 line drawings across two trials. After each trial, the examinee is shown a target picture paired with a foil and asked to select the target item. According to the TOMM manual (Tombaugh, 1996), the measure showed 100% specificity and 82% sensitivity in a simulator study (p. 15). Validation within a clinical sample, however, demonstrated a 27% false-positive rate and 73% specificity in patients with dementia (p. 13). More recent data provide evidence that the TOMM is less sensitive to effort when compared with the NV-MSVT (Armistead-Jehle & Gervais, 2011; Green, 2011). Other psychometric properties of the TOMM are outlined in the test manual (Tombaugh, 1996). While the TOMM retention trial has demonstrated increased sensitivity over trials 1 and 2 (Greve & Bianchini, 2006; Wisdom, Brown, Chen, & Collins, 2012), the optional retention trial was not administered in the current study secondary to the time demands present in the clinical environment from which these data were collected.

The RBANS (Randolph, 1998) is a brief battery comprised of 12 subtests that yields five index scores and a total summary score. The psychometric properties and clinical utility of this measure are well established (e.g., McKay, Casey, Wertheimer, & Fichtenberg, 2007; Moser & Shatz, 2002). Silverberg and colleagues (2007) constructed the RBANS EI by converting raw scores from the Digit Span and Word List Recognition subtests to weighted scores, each ranging from 0 to 6. These two weighted scores are then summed to arrive at the EI score that can then range from 0 to 12. Silverberg and colleagues determined that a cut score of 1 should be used in patients with post-acute mild TBI. Although others have questioned the sensitivity and specificity of this measure across various groups (i.e., Armistead-Jehle & Hansen, 2011; Barker, Horner, & Bachman, 2010; Hook, Marquine, & Hoelzle, 2009), it was included in the analysis as an embedded index of effort.

Data analysis

In order to examine the relationship between MCI scores and various SVT performances, SVT scores were categorized into ranges reflecting levels of performance on the measure (e.g., see ranges of scores on the MSVT effort measures in Table 1). As in the earlier Armistead-Jehle and colleagues (2012) study, in an effort to examine the entire range of variability, the SVTs were not treated as dichotomous pass/fail measures, but rather as continuous variables. The MSVT was categorized as a function of the average score (percent correct) across the easy subtests (Immediate Recognition [IR], Delayed Recognition [DR], and Consistency [CNS]). The NV-MSVT was categorized as a function of the average score (percent correct) on DR, CNS, DR Archetypes (DRA), and DR Variation (DRV) subtests. TOMM scores were categorized by the number correct on the second trial. RBANS EI scores were categorized as a sum of their weighted score as outlined in Silverberg and colleagues (2007). Secondary to the smaller sample sizes relative to the Armistead-Jehle and colleagues (2012) study, the ranges across the stand alone SVTs were truncated so that statistical comparisons between the various MCI subtests remained meaningful.

Table 1.

MCI scores as a function of MSVT performance (n= 185)

  MSVT score range
 
ANOVA
 
ES
 
91%–100% (n = 153) 85%–90% (n = 15) <85% (n = 17) F-value p-value η2 d1 d2 
GMP 27.6 (18.5)a 51.7 (20.6)b 48.5 (14.6)b 19.0 <.001 0.17 1.23 0.57 
NIP 32.5 (21.6)a 46.2 (20.5)b 48.8 (16.7)b 7.12 .001 0.07 0.65 0.84 
VSMP 22.5 (19.0)a 39.7 (20.9)b 47.9 (21.4)b 17.1 <.001 0.16 0.86 1.26 
VMP 40.4 (24.9)a 58.0 (21.6)b 64.4 (16.9)b 10.5 <.001 0.10 0.76 1.13 
PIM 14.8 (22.4)a 30.9 (32.4)b 23.5 (30.9)b 3.7 .026 (ns) 0.04 0.58 0.32 
MIW 25.8 (23.5)a 40.0 (25.4)ab 45.9 (25.6)b 7.2 .001 0.07 0.58 0.82 
IRM 14.4 (11.7)a 30.5 (20.0)b 28.5 (19.9)b 16.6 <.001 0.15 0.98 0.86 
ACB 16.9 (15.7)a 30.1 (22.7)b 32.3 (21.0)b 10.2 <.001 0.10 0.68 0.83 
AAB 7.3 (10.3) 11.1 (14.5) 11.9 (14.5) 1.9 .148 (ns) 0.02 0.30 0.37 
Mean MCI 22.5 (14.8)a 37.5 (17.9)b 39.1 (14.4)b 14.8 <.001 0.14 0.91 1.14 
  MSVT score range
 
ANOVA
 
ES
 
91%–100% (n = 153) 85%–90% (n = 15) <85% (n = 17) F-value p-value η2 d1 d2 
GMP 27.6 (18.5)a 51.7 (20.6)b 48.5 (14.6)b 19.0 <.001 0.17 1.23 0.57 
NIP 32.5 (21.6)a 46.2 (20.5)b 48.8 (16.7)b 7.12 .001 0.07 0.65 0.84 
VSMP 22.5 (19.0)a 39.7 (20.9)b 47.9 (21.4)b 17.1 <.001 0.16 0.86 1.26 
VMP 40.4 (24.9)a 58.0 (21.6)b 64.4 (16.9)b 10.5 <.001 0.10 0.76 1.13 
PIM 14.8 (22.4)a 30.9 (32.4)b 23.5 (30.9)b 3.7 .026 (ns) 0.04 0.58 0.32 
MIW 25.8 (23.5)a 40.0 (25.4)ab 45.9 (25.6)b 7.2 .001 0.07 0.58 0.82 
IRM 14.4 (11.7)a 30.5 (20.0)b 28.5 (19.9)b 16.6 <.001 0.15 0.98 0.86 
ACB 16.9 (15.7)a 30.1 (22.7)b 32.3 (21.0)b 10.2 <.001 0.10 0.68 0.83 
AAB 7.3 (10.3) 11.1 (14.5) 11.9 (14.5) 1.9 .148 (ns) 0.02 0.30 0.37 
Mean MCI 22.5 (14.8)a 37.5 (17.9)b 39.1 (14.4)b 14.8 <.001 0.14 0.91 1.14 

Notes: Means with different superscript letters are significantly different (Tukey HSD). df for all ANOVA cells = 2, 182. MSVT = Medical Symptom Validity Test; GMP = General Memory Problems; NIP = Numeric Information Problems; VSMP = Visuospatial Memory Problems; VMP = Verbal Memory Problems; PIM = Pain Interferes with Memory; MIW = Memory Interferes with Work; IRM = Impairment of Remote Memory; ACB = Amnesia for Complex Behavior; AAB = Amnesia for Antisocial Behavior; MCI = Memory Complaints Inventory; d1 = MSVT score range 91%–100% versus 85%–90%; d2 = MSVT score range 91%–100% versus <85%; ns = non-significant finding after the Bonferroni correction.

Results

Tables 1–4 show the means and standard deviations of each MCI subtest as a function of overall scores on the three stand-alone SVTs and the one embedded SVT. Table 5 outlines the correlations between SVT performance and each MCI subscale. In general, across each of the nine MCI subscales, there was an increase in the average score as SVT performances worsened. With each SVT, a one-way ANOVA was conducted to examine if there were statistically significance differences on the various MCI subscale scores or the overall MCI average as a function of SVT performance. Given the multiple one-way ANOVAs conducted a Bonferoni correction was applied, with the significance level set at 0.005. As shown in Tables 1–3, the vast majority of ANOVAs were significant on the stand-alone SVTs (i.e., MSVT, NV-MSVT, and TOMM). Less than half of the MCI subtests were statistically significant with the RBANS EI comparisons (Table 4). While the results of post hoc analyses showed some minor variations, the overwhelming pattern was for memory complaints to increase as stand-alone SVT performance declined. For each stand-alone SVT, Cohen's d effect size estimates were calculated comparing the top performing SVT groups to lower performing groups. As stand-alone SVT performance declined the effect sizes increased, serving to support the hypothesis that effort in neurocognitive testing is related to self-report of memory deficits. Figs 1–3 depict in graph form MCI responses as a function of MSVT, NV-MSVT, and TOMM performances. As the reader will see higher scores are generally seen across MCI subtests for increasingly worse SVT performance.

Table 2.

MCI scores as a function of NV-MSVT performance sample 1 (n= 175)

  NV-MSVT score range
 
ANOVA
 
ES
 
91%–100% (n = 144) 85%–90% (n = 17) <85% (n = 14) F-value p-value η2 d1 d2 
GMP 27.7 (19.2)a 41.1 (15.9)b 54.7 (16.2)b 15.6 <.001 0.15 0.19 0.96 
NIP 39.9 (22.0)a 38.1 (13.1)ab 53.3 (15.5)b 6.3 .002 0.07 0.10 0.70 
VSMP 22.9 (19.7)a 32.4 (15.9)a 50.0 (23.9)b 13.1 <.001 0.13 0.53 1.24 
VMP 39.9 (24.9)a 53.5 (20.3)ab 69.6 (13.9)b 11.6 <.001 0.12 0.60 1.47 
PIM 14.6 (22.2)a 14.7 (13.3)a 47.0 (36.4)b 12.8 <.001 0.13 0.01 1.07 
MIW 25.9 (23.3)a 32.4 (16.2)a 53.2 (30.2)b 9.0 <.001 0.10 0.32 1.01 
IRM 15.6 (13.4)a 22.4 (14.4)ab 30.0 (21.7)a 7.6 .001 0.09 0.49 0.80 
ACB 17.2 (15.1)a 21.0 (16.3)a 42.6 (23.7)b 16.2 <.001 0.16 0.24 1.28 
AAB 6.8 (8.9)a 11.4 (14.2)ab 18.8 (20.8)b 8.6 <.001 0.10 0.39 0.75 
Mean MCI 22.6 (15.0)a 29.7 (11.0)a 46.5 (17.3)b 17.4 <.001 0.17 0.54 1.48 
  NV-MSVT score range
 
ANOVA
 
ES
 
91%–100% (n = 144) 85%–90% (n = 17) <85% (n = 14) F-value p-value η2 d1 d2 
GMP 27.7 (19.2)a 41.1 (15.9)b 54.7 (16.2)b 15.6 <.001 0.15 0.19 0.96 
NIP 39.9 (22.0)a 38.1 (13.1)ab 53.3 (15.5)b 6.3 .002 0.07 0.10 0.70 
VSMP 22.9 (19.7)a 32.4 (15.9)a 50.0 (23.9)b 13.1 <.001 0.13 0.53 1.24 
VMP 39.9 (24.9)a 53.5 (20.3)ab 69.6 (13.9)b 11.6 <.001 0.12 0.60 1.47 
PIM 14.6 (22.2)a 14.7 (13.3)a 47.0 (36.4)b 12.8 <.001 0.13 0.01 1.07 
MIW 25.9 (23.3)a 32.4 (16.2)a 53.2 (30.2)b 9.0 <.001 0.10 0.32 1.01 
IRM 15.6 (13.4)a 22.4 (14.4)ab 30.0 (21.7)a 7.6 .001 0.09 0.49 0.80 
ACB 17.2 (15.1)a 21.0 (16.3)a 42.6 (23.7)b 16.2 <.001 0.16 0.24 1.28 
AAB 6.8 (8.9)a 11.4 (14.2)ab 18.8 (20.8)b 8.6 <.001 0.10 0.39 0.75 
Mean MCI 22.6 (15.0)a 29.7 (11.0)a 46.5 (17.3)b 17.4 <.001 0.17 0.54 1.48 

Notes: Means with different superscript letters are significantly different (Tukey HSD); df for all ANOVA cells = 2, 172; NV-MSVT = Nonverbal Medical Symptom Validity Test; GMP = General Memory Problems; NIP = Numeric Information Problems; VSMP = Visuospatial Memory Problems; VMP = Verbal Memory Problems; PIM = Pain Interferes with Memory; MIW = Memory Interferes with Work; IRM = Impairment of Remote Memory; ACB = Amnesia for Complex Behavior; AAB = Amnesia for Antisocial Behavior; MCI = Memory Complaints Inventory; d1 = NV-MSVT score range 91%–100% versus 85%–90%; d2 = NV-MSVT score range 91%–100% versus <85%; ns = non-significant finding after the Bonferroni correction.

Table 3.

MCI scores as a function of TOMM performance sample 1 (n= 162)

  TOMM score range
 
ANOVA
 
ES
 
48–50 (n = 143) 42–47 (n = 11) <42 (n = 8) F-value p-value η2 d1 d2 
GMP 28.3 (18.6)a 50.0 (14.1)b 52.8 (14.0)b 13.1 <.001 0.14 1.31 1.49 
NIP 33.2 (21.1)a 42.1 (15.6)ab 57.4 (17.2)b 6.6 .002 0.08 0.48 1.26 
VSMP 23.2 (19.4)a 39.6 (22.4)b 53.1 (20.4)b 11.7 <.001 0.13 0.78 1.50 
VMP 40.0 (24.0)a 70.5 (19.2)b 70.3 (9.0)b 14.4 <.001 0.15 1.40 1.67 
PIM 13.5 (19.7)a 18.9 (21.8)a 51.0 (35.5)b 12.4 <.001 0.14 0.26 1.31 
MIW 25.5 (22.2)a 48.2 (25.1)b 46.9 (29.6)b 8.0 .001 0.09 0.96 0.82 
IRM 15.8 (13.7)a 21.6 (11.2)a 39.6 (24.3)b 11.3 <.001 0.13 0.46 1.21 
ACB 17.2 (15.3)a 30.0 (18.1)b 37.1 (24.4)b 8.6 <.001 0.10 0.76 0.98 
AAB 7.1 (9.8)a 14.2 (15.5)a 15.1 (20.0)a 3.9 .021 (ns) 0.05 0.55 0.51 
Mean MCI 22.6 (14.3)a 37.6 (15.0)b 47.1 (14.7)b 15.5 <.001 0.16 1.02 1.69 
  TOMM score range
 
ANOVA
 
ES
 
48–50 (n = 143) 42–47 (n = 11) <42 (n = 8) F-value p-value η2 d1 d2 
GMP 28.3 (18.6)a 50.0 (14.1)b 52.8 (14.0)b 13.1 <.001 0.14 1.31 1.49 
NIP 33.2 (21.1)a 42.1 (15.6)ab 57.4 (17.2)b 6.6 .002 0.08 0.48 1.26 
VSMP 23.2 (19.4)a 39.6 (22.4)b 53.1 (20.4)b 11.7 <.001 0.13 0.78 1.50 
VMP 40.0 (24.0)a 70.5 (19.2)b 70.3 (9.0)b 14.4 <.001 0.15 1.40 1.67 
PIM 13.5 (19.7)a 18.9 (21.8)a 51.0 (35.5)b 12.4 <.001 0.14 0.26 1.31 
MIW 25.5 (22.2)a 48.2 (25.1)b 46.9 (29.6)b 8.0 .001 0.09 0.96 0.82 
IRM 15.8 (13.7)a 21.6 (11.2)a 39.6 (24.3)b 11.3 <.001 0.13 0.46 1.21 
ACB 17.2 (15.3)a 30.0 (18.1)b 37.1 (24.4)b 8.6 <.001 0.10 0.76 0.98 
AAB 7.1 (9.8)a 14.2 (15.5)a 15.1 (20.0)a 3.9 .021 (ns) 0.05 0.55 0.51 
Mean MCI 22.6 (14.3)a 37.6 (15.0)b 47.1 (14.7)b 15.5 <.001 0.16 1.02 1.69 

Notes: Means with different superscript letters are significantly different (Tukey HSD); df for all ANOVA cells = 2, 159; TOMM = Test of Memory Malingering; GMP = General Memory Problems; NIP = Numeric Information Problems; VSP = Visuospatial Memory Problems; VMP = Verbal Memory Problems; PIM = Pain Interferes with Memory; MIW = Memory Interferes with Work; IRM = Impairment of Remote Memory; ACB = Amnesia for Complex Behavior; AAB = Amnesia for Antisocial Behavior; MCI = Memory Complaints Inventory; d1 = TOMM score range 48–50 versus 42–47; d2 = TOMM score range 48–50 versus <42; ns = non-significant finding after the Bonferroni correction.

Table 4.

MCI scores as a function of RBANS EI (n= 158)

  RBANS EI score range
 
ANOVA
 
ES
 
0 (n = 133) 1 (n = 12) >1 (n = 13) F-value p-value η2 d1 d2 
GMP 27.9 (18.0)a 42.8 (20.8)b 42.4 (21.7)b 6.6 .002 0.08 0.77 0.73 
NIP 32.8 (20.8)a 51.4 (21.7)b 38.9 (19.1)b 4.7 .010 (ns) 0.06 0.88 0.31 
VSMP 22.1 (18.6)a 47.1 (24.8)b 35.0 (24.0)b 10.8 <.001 0.12 1.14 0.60 
VMP 40.5 (24.0)a 58.3 (17.4)b 56.9 (30.2)b 5.3 .006 (ns) 0.06 0.85 0.60 
PIM 14.7 (21.1)a 37.5 (31.4)b 19.2 (29.5)b 5.6 .004 0.07 0.85 0.18 
MIW 25.6 (22.0)a 36.7 (24.9)a 41.2 (31.6)a 3.7 .027 (ns) 0.05 0.47 0.57 
IRM 15.8 (14.1)a 26.9 (21.3)b 21.2 (14.7)b 3.7 .028 (ns) 0.05 0.61 0.37 
ACB 17.1 (14.8)a 29.4 (19.6)b 25.3 (23.1)b 4.5 .013 (ns) 0.05 0.71 0.42 
AAB 7.4 (9.8) 8.6 (15.1) 13.2 (19.1) 1.6 .211 (ns) 0.02 0.09 0.38 
Mean MCI 22.7 (14.3)ab 37.6 (15.7)b 32.6 (19.5)ab 7.6 .001 0.09 0.99 0.58 
  RBANS EI score range
 
ANOVA
 
ES
 
0 (n = 133) 1 (n = 12) >1 (n = 13) F-value p-value η2 d1 d2 
GMP 27.9 (18.0)a 42.8 (20.8)b 42.4 (21.7)b 6.6 .002 0.08 0.77 0.73 
NIP 32.8 (20.8)a 51.4 (21.7)b 38.9 (19.1)b 4.7 .010 (ns) 0.06 0.88 0.31 
VSMP 22.1 (18.6)a 47.1 (24.8)b 35.0 (24.0)b 10.8 <.001 0.12 1.14 0.60 
VMP 40.5 (24.0)a 58.3 (17.4)b 56.9 (30.2)b 5.3 .006 (ns) 0.06 0.85 0.60 
PIM 14.7 (21.1)a 37.5 (31.4)b 19.2 (29.5)b 5.6 .004 0.07 0.85 0.18 
MIW 25.6 (22.0)a 36.7 (24.9)a 41.2 (31.6)a 3.7 .027 (ns) 0.05 0.47 0.57 
IRM 15.8 (14.1)a 26.9 (21.3)b 21.2 (14.7)b 3.7 .028 (ns) 0.05 0.61 0.37 
ACB 17.1 (14.8)a 29.4 (19.6)b 25.3 (23.1)b 4.5 .013 (ns) 0.05 0.71 0.42 
AAB 7.4 (9.8) 8.6 (15.1) 13.2 (19.1) 1.6 .211 (ns) 0.02 0.09 0.38 
Mean MCI 22.7 (14.3)ab 37.6 (15.7)b 32.6 (19.5)ab 7.6 .001 0.09 0.99 0.58 

Notes: Means with different superscript letters are significantly different (Tukey HSD); df for all ANOVA cells = 2, 155; RBANS EI = Repeatable Battery for the Assessment of Neuropsychological Status Effort Index; GMP = General Memory Problems; NIP = Numeric Information Problems; VSMP = Visuospatial Memory Problems; VMP = Verbal Memory Problems; PIM = Pain Interferes with Memory; MIW = Memory Interferes with Work; IRM = Impairment of Remote Memory; ACB = Amnesia for Complex Behavior; AAB = Amnesia for Antisocial Behavior; MCI = Memory Complaints Inventory; d1 = EI score of 0 versus 1; d2 = EI score of 0 versus >1; ns = non-significant finding after the Bonferroni correction.

Table 5.

Correlations between MCI subscales and SVTs

 MSVT (n= 185) NV-MSVT (n= 175) TOMM (n= 162) RBANS EI (n= 158) 
GMP −0.40*** −0.45*** −0.36*** 0.20* 
NIP −0.31*** −0.31*** −0.28*** 0.12 
VSMP −0.43*** −0.38*** −0.37*** 0.23** 
VMP −0.35*** −0.37*** −0.35*** 0.20* 
PIM −0.20** −0.33*** −0.36*** 0.05 
MIW −0.25*** −0.35*** −0.26** 0.16 
IRM −0.41*** −0.32*** −0.36*** 0.14 
ACB −0.37*** −0.40*** −0.32*** 0.11 
AAB −0.21** −0.30*** −0.21** 0.05 
Mean MCI −0.40*** −0.44*** −0.40*** 0.18* 
 MSVT (n= 185) NV-MSVT (n= 175) TOMM (n= 162) RBANS EI (n= 158) 
GMP −0.40*** −0.45*** −0.36*** 0.20* 
NIP −0.31*** −0.31*** −0.28*** 0.12 
VSMP −0.43*** −0.38*** −0.37*** 0.23** 
VMP −0.35*** −0.37*** −0.35*** 0.20* 
PIM −0.20** −0.33*** −0.36*** 0.05 
MIW −0.25*** −0.35*** −0.26** 0.16 
IRM −0.41*** −0.32*** −0.36*** 0.14 
ACB −0.37*** −0.40*** −0.32*** 0.11 
AAB −0.21** −0.30*** −0.21** 0.05 
Mean MCI −0.40*** −0.44*** −0.40*** 0.18* 

Notes: MSVT = Medical Symptom Validity Test Easy subtest Average; NV-MSVT = Nonverbal Medical Symptom Validity Test Easy Subtest Average (DR, CNS, DRV, and DRA); TOMM = Test of Memory Malingering Trial 2 Score; RBANS EI = Repeatable Battery for the Assessment of Neuropsychological Status Effort Index; GMP = General Memory Problems; NIP = Numeric Information Problems; VSMP = Visuospatial Memory Problems; VMP = Verbal Memory Problems; PIM = Pain Interferes with Memory; MIW = Memory Interferes with Work; IRM = Impairment of Remote Memory; ACB = Amnesia for Complex Behavior; AAB = Amnesia for Antisocial Behavior; MCI = Memory Complaints Inventory.

*p< .05.

**p< .01.

***p< .001.

Fig. 1.

MCI scores as a function of MSVT performance (n= 185).

Fig. 1.

MCI scores as a function of MSVT performance (n= 185).

Fig. 2.

MCI scores as a function of NV-MSVT performance (n= 175).

Fig. 2.

MCI scores as a function of NV-MSVT performance (n= 175).

Fig. 3.

MCI scores as a function of TOMM performance (n= 162).

Fig. 3.

MCI scores as a function of TOMM performance (n= 162).

Discussion

The current study examined the relationship between subjective memory complaints and performances on various SVTs in a clinical sample not overtly involved in disability evaluations. The investigation sought to expand the findings of Armistead-Jehle and colleagues (2012) who demonstrated that as the SVT performances of people involved in disability examinations worsened, there was a corresponding increase in subjective reports of memory complaints across all MCI subscales. The current findings largely echoed these results and again showed that as patients' performances across a variety of stand-alone SVTs declined, there was a corresponding increase in subjective memory complaints depicted on the MCI.

Despite the general replication of the previous Armistead-Jehle and colleagues (2012) study, there were some differences in results between the two investigations. Within the disability-seeking samples evaluated in the original study, all MCI subscales were impacted by SVT performances; however, in the current study, select MCI subscales appeared less consistently influenced. For instance, the AAB subscale was only variably impacted by SVT performance and effect sizes on this subscale were generally lower than in the original study. The ACB subscale also evidenced somewhat lower effect sizes relative to the original study. As noted above, the AAB and ACB subscales consist of items designed to represent implausible memory complaints in neurological groups and are rarely endorsed by any patient group. As such, if a respondent does not endorse the item, it tells us little about that individual. However, if the item is endorsed the respondent is affirming symptoms that are very unlikely to be true, but may be viewed by the respondent as suggestive of severe pathology. This difference then between AAB and ACB subscale elevations between the two studies could potentially relate to the nature of the samples investigated. Relative to the disability-seeking sample of the original study, the current clinical sample appeared to have much less external incentive to endorse the exceptionally pathological appearing items that comprise the AAB and ACB subscales. In other words, it seems rational to conclude that the AAB and ACB subscales did not cleanly discriminate among a sample that, as a whole, did not appear to have incentive to dissimulate their respective presentations, as the samples in the original article might have.

In the current study, performances on the stand-alone SVTs showed a robust relationship with MCI profiles, while performances on the embedded measure (RBANS EI) failed to evidence a relationship of similar magnitude. It has been shown that embedded SVTs are not as sensitive to suboptimal effort relative to stand-alone measures (Miele, Gunner, Lynch, & McCaffrey, 2012) and the RBANS EI has been shown to have only moderate sensitivity to poor effort when compared with a range of stand-alone SVTs (Armistead-Jehle & Hansen, 2011). Consequently, this limited sensitivity may explain the current relationship between RBANS EI performances and MCI profiles.

The current study partially addresses a cited weakness in the original Armistead-Jehle and colleagues (2012) work that looked exclusively at a known disability-seeking group. Participants in the current study were seen as part of a clinical referral and those with an obvious incentive to exaggerate symptomatology were removed from analysis. However, it is noteworthy that the current sample was composed primarily of active duty military service members. While the majority of the sample did not fail SVTs, the possibility of implicit secondary gain for symptom over-report is arguably present in members of this sample. Sources of potential secondary gain could include restrictions in current duty assignments, future disability claims to be discharged from military service, or veteran disability claims after discharge. While it is thought that such motivation to under-achieve on SVTs or to over-report on self-report measures is not present in the majority of those in the current sample, the possibility in this clinical environment exists and as such must be addressed. Future research outside of the military or veteran context may provide a cleaner sample of non-disability seeking individuals.

Nevertheless, as in the original study, it is reasonable to assert that symptom exaggeration explains at least a portion of the association between low SVT performances and heightened subjective memory complaints and that those who exaggerate their cognitive problems on self-report scales like the MCI may also be apt to exaggerate their cognitive difficulties by underachieving on cognitive tests as reflected in diminished SVT scores. Although psychological, somatic, and personality factors may also play a role in this relationship, the clinical implication cited in the earlier Armistead-Jehle and colleagues (2012) study would appear to have continued application. That is, when a patient presents with elevated memory complaints, symptom exaggeration or dissimulation should be considered and ruled out before other medical or psychiatric etiologies are concluded.

While the current work is thought to extend the generalizability of the original Armistead-Jehle and colleagues (2012) investigation, the study is not without its limitations. The sample size of the current study, while reasonable, was notably smaller than the original study and as a result SVT performances needed to be categorized in a narrower manner to achieve reasonable cell sizes. Such categorization precluded an exact comparison with the original study. As noted above, another potential limitation lies with the environment from which the sample was drawn. The vast majority of the current sample was active duty military service members, which may demonstrate differences from patients seeking neuropsychological evaluation in the civilian context. Replication in a non-disability seeking civilian sample would further bolster the external validity of these findings.

In sum, the general findings of the earlier Armistead-Jehle and colleagues (2012) study demonstrating that as SVT performances worsened, there is a corresponding increase in subjective reports of memory complaints across MCI subscales were replicated. However, within the current clinical sample, a handful of MCI subscales designed to represent implausible items failed to show discrimination to the same extent as in the original study. Such a finding may suggest that within clinical samples these items are not as readily endorsed relative to disability-seeking individuals. Taken together with the original Armistead-Jehle and colleagues study, the current data suggest that when evaluating patients with notable self-reported memory complaints, objective measures of effort should be included in an attempt to rule out suboptimal effort and/or conscious attempts at symptom exaggeration affecting the individual's presentation across both clinical and disability samples.

Conflict of Interest

PG is the inventor of the MCI, MSVT, and NV-MSVT and he partly owns Green's Publishing, which sells each of these instruments commercially. PA-J and ROG have no financial interest in any of the instruments used in this study.

Acknowledgements

The views, opinions, and/or findings contained in this article are those of the authors and should not be construed as an official Department of the Army position, policy, or decision unless so designated by other official documentation.

References

American College of Rehabilitation Medicine
Definition of mild traumatic brain injury
Journal of Head Trauma Rehabilitation
 , 
1993
, vol. 
8
 (pg. 
86
-
87
)
Armistead-Jehle
P.
Gervais
R. O.
Sensitivity of the Test of Memory Malingering and the Nonverbal Medical Symptom Validity Test: A replication study
Applied Neuropsychology
 , 
2011
, vol. 
18
 (pg. 
284
-
290
)
Armistead-Jehle
P.
Gervais
R. O.
Green
P.
Memory Complaints Inventory results as a function of symptom validity test performance
Archives of Clinical Neuropsychology
 , 
2012
, vol. 
27
 (pg. 
101
-
113
)
Armistead-Jehle
P.
Hansen
C. L.
Comparison of the Repeatable Battery for the Assessment of Neuropsychological Status Effort Index and Stand-Alone Symptom Validity Tests in a Military Sample
Archives of Clinical Neuropsychology
 , 
2011
, vol. 
26
 (pg. 
592
-
601
)
Barker
M. D.
Horner
M. D.
Bachman
D. L.
Embedded Indices of Effort in the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) in a geriatric sample
The Clinical Neuropsychologist
 , 
2010
, vol. 
24
 (pg. 
1064
-
1077
)
Carone
D. A.
Test review of the Medical Symptom Validity Test (2009)
Applied Neuropsychology
 , 
2009
, vol. 
16
 (pg. 
309
-
311
)
Chafetz
M.
Malingering on the social security disability consultative exam: Predictors and base rates
The Clinical Neuropsychologist
 , 
2008
, vol. 
22
 (pg. 
529
-
546
)
Chamelian
L.
Feinstein
A.
The effect of major depression on subjective and objective cognitive deficits in mild to moderate traumatic brain injury
Journal of Neuropsychiatry and Clinical Neurosciences
 , 
2006
, vol. 
18
 (pg. 
33
-
38
)
Constantinou
M.
Bauer
L.
Ashendorf
L.
Fisher
J. M.
McCaffrey
R. J.
Is poor performance on recognition memory effort measures indicative of generalized poor performance on neuropsychological tests
Archives of Clinical Neuropsychology
 , 
2005
, vol. 
20
 (pg. 
191
-
198
)
Ford
N.
Slade
P.
Butler
G.
An absence of evidence linking perceived memory problems to the menopause
The British Journal of General Practice
 , 
2004
, vol. 
54
 (pg. 
434
-
438
)
Fox
D. D.
Symptom validity test failure indicates invalidity of neuropsychological tests
The Clinical Neuropsychologist
 , 
2011
, vol. 
25
 (pg. 
488
-
495
)
Gervais
R. O.
Ben-Porath
Y. S.
Wygant
D. B.
Green
P.
Development and validation of the Response Bias Scale (RBS) for the MMPI-2
Assessment
 , 
2007
, vol. 
14
 (pg. 
196
-
208
)
Gervais
R. O.
Ben-Porath
Y. S.
Wygant
D. B.
Green
P.
Differential sensitivity of the Response Bias Scale (RBS) and MMPI-2 validity scales to memory impairment
The Clinical Neuropsychologist
 , 
2008
, vol. 
22
 (pg. 
1061
-
1079
)
Gervais
R. O.
Ben-Porath
Y. S.
Wygant
D. B.
Sellbom
M.
Incremental validity of the MMPI-2-RF over-reporting scales and RBS in assessing the veracity of memory complaints
Archives of Clinical Neuropsychology
 , 
2010
, vol. 
25
 (pg. 
274
-
284
)
Gervais
R. O.
Russell
A. S.
Green
P.
Allen
L. M.
III
Ferrari
R.
Pieschel
S. D.
Effort testing in patients with fibromyalgia and disability incentives
Journal of Rheumatology
 , 
2001
, vol. 
28
 (pg. 
1892
-
1899
)
Green
P.
Memory Complaints Inventory
 , 
2004
Edmonton, Canada
Green's Publishing
Green
P.
Green's Medical Symptom Validity Test (MSVT) for Microsoft Windows: User's manual
 , 
2004
Edmonton, Canada
Green's Publishing
Green
P.
The pervasive influence of effort on neuropsychological tests
Physical Medicine and Rehabilitation Clinics of North America
 , 
2007
, vol. 
18
 (pg. 
43
-
68
)
Green
P.
Manual for the Nonverbal Medical Symptom Validity Test
 , 
2008
Edmonton, Canada
Green's Publishing
Green
P.
Comparison between the Test of Memory Malingering (TOMM) and the Nonverbal Medical Symptom Validity Test (NV-MSVT) in adults with disability claims
Applied Neuropsychology
 , 
2011
, vol. 
18
 (pg. 
18
-
26
)
Green
P.
Flaro
L.
Brockhaus
R.
Montijo
J.
Reynolds
C. R.
Horton
A.
Performance on the WMT, MSVT, & NV-MSVT in children with developmental disabilities and in adults with mild traumatic brain injury
Detection of malingering during head injury litigation – 2nd Edition
 , 
2010
New York
Plenum Press
Green
P.
Rohling
M. L.
Lees-Haley
P. R.
Allen
L. M.
III.
Effort has a greater effect on test scores then severe brain injury in compensation claimants
Brain Injury
 , 
2001
, vol. 
15
 (pg. 
1045
-
1060
)
Greve
K. W.
Bianchini
K. J.
Should the retention trial of the Test of Memory Malingering be optional?
Archives of Clinical Neuropsychology
 , 
2006
, vol. 
21
 (pg. 
117
-
119
)
Henry
M.
Merten
T.
Wolf
S.
Harth
S.
Nonverbal Medical Symptom Validity Test performance in elderly health adults and clinical neurology patients
Journal of Clinical and Experimental Neuropsychology
 , 
2009
, vol. 
8
 (pg. 
1
-
10
)
Hook
J. N.
Marquine
M. J.
Hoelzle
J. B.
Repeatable Battery for the Assessment of Neuropsychological Status effort index performance in a medically ill geriatric sample
Archives of Clinical Neuropsychology
 , 
2009
, vol. 
24
 (pg. 
231
-
235
)
Howe
L. L.
Loring
D. W.
Classification accuracy and predictive ability of the Medical Symptom Validity Test's dementia profile and general memory impairment profile
The Clinical Neuropsychologist
 , 
2009
, vol. 
23
 (pg. 
329
-
342
)
Landrø
N. I.
Sletvold
H.
Celius
E. G.
Memory functioning and emotional changes in early phase multiple sclerosis
Archives Clinical Neuropsychology
 , 
2000
, vol. 
15
 (pg. 
37
-
46
)
Lezak
M. D.
Howieson
D. B.
Loring
D.
Neuropsychological assessment
 , 
2004
4th ed.
New York
Oxford University Press
Matotek
K.
Saling
M. M.
Gates
P.
Sedal
L.
Subjective complaints, verbal fluency, and working memory in mild multiple sclerosis
Applied Neuropsychology
 , 
2001
, vol. 
8
 (pg. 
204
-
210
)
McCracken
L.
Iverson
G. L.
Predicting complaints of impaired cognitive functioning in patients with chronic pain
Journal of Pain and Symptom Management
 , 
2001
, vol. 
21
 (pg. 
392
-
396
)
McKay
C.
Casey
J. E.
Wertheimer
J.
Fichtenberg
N. L.
Reliability and validity of the RBANS in a traumatic brain injured sample
Archives of Clinical Neuropsychology
 , 
2007
, vol. 
22
 (pg. 
91
-
98
)
Meyers
J. E.
Volbrecht
M.
Axelrod
B. N.
Reinsch-Boothby
L.
Embedded symptom validity tests and overall neuropsychological test performance
Archives of Clinical Neuropsychology
 , 
2011
, vol. 
26
 (pg. 
8
-
15
)
Miele
A. S.
Gunner
J. H.
Lynch
J. K.
McCaffrey
R. J.
Are embedded validity indices equivalent to free-standing symptom validity tests?
Archives of Clinical Neuropsychology
 , 
2012
, vol. 
27
 (pg. 
10
-
22
)
Mol
M. E.
van Boxtel
M. P. J.
Willems
D.
Jolles
J.
Do subjective memory complaints predict cognitive dysfunction over time? A six-year follow-up of the Maastricht Aging Study
International Journal of Geriatric Psychiatry
 , 
2006
, vol. 
21
 (pg. 
432
-
441
)
Moser
R. S.
Shatz
P.
Enduring effects of concussion in young athletes
Archives of Clinical Neuropsychology
 , 
2002
, vol. 
17
 (pg. 
91
-
100
)
Moss
A.
Jones
C.
Fokias
D.
Quinn
D.
The mediating effects of effort upon the relationship between head injury severity and cognitive functioning
Brain Injury
 , 
2003
, vol. 
17
 (pg. 
377
-
387
)
Munoz
M.
Esteve
R.
Reports of memory functioning by patients with chronic pain
Clinical Journal of Pain
 , 
2005
, vol. 
21
 (pg. 
287
-
291
)
Pearman
A.
Predictors of subjective memory in young adults
Journal of Adult Development
 , 
2009
, vol. 
16
 (pg. 
101
-
107
)
Pearman
A.
Storandt
M.
Predictors of subjective memory in older adults
Journal of Gerontology
 , 
2004
, vol. 
59B
 (pg. 
4
-
6
)
Pearson
Wechsler Test of Adult Reading: Manual
 , 
2001
San Antonio, TX
Pearson
Randolph
C.
Repeatable Battery for the Assessment of Neuropsychological Status Manual
 , 
1998
San Antonio, TX
The Psychological Corporation
Silverberg
N. D.
Wertheimer
J. C.
Fichtenberg
N. L.
An effort index for the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS)
The Clinical Neuropsychologist
 , 
2007
, vol. 
21
 (pg. 
841
-
854
)
Singhal
A.
Green
P.
Ashaye
K.
Shankar
K.
Gill
D.
High specificity of the medical symptom validity test in patients with very severe memory impairment
Archives of Clinical Neuropsychology
 , 
2009
, vol. 
24
 (pg. 
721
-
728
)
Stevens
A.
Friedel
E.
Mehren
G.
Merten
T.
Malingering and uncooperativeness in psychiatric and psychological assessment: Prevalence and effects in a German sample of claimants
Psychiatry Research
 , 
2008
, vol. 
157
 (pg. 
191
-
200
)
Suhr
J. A.
Neuropsychological impairment in fibromyalgia: Relations to depression, fatigue, and pain
Journal of Psychosomatic Research
 , 
2003
, vol. 
55
 (pg. 
321
-
329
)
Tombaugh
T. N.
The Test of Memory Malingering
 , 
1996
Los Angeles
Western Psychology Corporation
Wager
J. G.
Howe
L. L. S.
Nonverbal Medical Symptom Validity Test: Try faking now!
Applied Neuropsychology
 , 
2010
, vol. 
17
 (pg. 
305
-
309
)
West
L. K.
Curtis
K. L.
Greve
K. W.
Bianchini
K. J.
Memory in traumatic brain injury: The effects of severity and effort on the Wechsler Memory Scale-III
Journal of Neuropsychology
 , 
2010
, vol. 
5
 (pg. 
114
-
125
)
Wisdom
N. M.
Brown
W. L.
Chen
D. K.
Collins
R. L.
The use of all three Test of Memory Malingering trials in establishing the level of effort
Archives of Clinical Neuropsychology
 , 
2012
, vol. 
27
 (pg. 
208
-
2012
)