Abstract

Although soldiers of Operations Iraqi Freedom (OIF) and Enduring Freedom (OEF) encounter combat-related concussion at an unprecedented rate, relatively few studies have examined how evaluation context, insufficient effort, and concussion history impact neuropsychological performances in the years following injury. The current study explores these issues in a sample of 119 U.S. veterans (OEF/OIF forensic concussion, n = 24; non-OEF/OIF forensic concussion, n = 20; OEF/OIF research concussion, n = 38; OEF/OIF research without concussion, n = 37). The OEF/OIF forensic concussion group exhibited significantly higher rates of insufficient effort relative to the OEF/OIF research concussion group, but a comparable rate of insufficient effort relative to the non-OEF/OIF forensic concussion group. After controlling for effort, the research concussion and the research non-concussion groups demonstrated comparable neuropsychological performance. Results highlight the importance of effort assessment among OEF/OIF and other veterans with concussion history, particularly in forensic contexts.

Introduction

Most studies examining the trajectory of cognitive recovery following mild traumatic brain injury (MTBI or concussion) include civilian samples, such as athletes who sustain sports-related injuries (e.g., McCrea, 2001). Although uncomplicated concussion frequently contributes to objective cognitive impairment within the acute stage of injury, the overwhelming majority of individuals attain a baseline level of functioning within days, weeks, to no more than a few months post-injury. This favorable course of recovery has been documented by several meta-analytic investigations (Belanger, Curtiss, Demery, Lebowitz, & Vanderploeg, 2005; Binder, Rohling, & Larrabee, 1997; Frencham, Fox, & Maybery, 2005; Iverson, 2005; Schretlen & Shapiro, 2003). Iverson aptly concludes, “under most circumstances, we should anticipate good recovery following MTBI” (p. 311).

Nevertheless, a minority of individuals experience persisting cognitive limitations and other post-concussive symptoms (PCS) beyond the usual course of recovery. Researchers have discussed various factors that may complicate recovery, such as premorbid psychological disturbance (Greiffenstein & Baker, 2001), co-morbid emotional difficulty (Moore, Terryberry-Spohr, & Hope, 2006), and chronic pain (Nicholson, 2000). Evaluation context has also been identified as a significant moderator of cognitive functioning following concussion (Belanger et al., 2005; Binder & Rohling, 1996). Belanger and colleagues (2005) illustrated that litigants and population-based samples showed similar cognitive limitations within the first 3 months post-injury (d = 0.52 vs. 0.63, respectively). However, whereas population-based samples showed essentially no cognitive limitations after 3 months (d = 0.04), litigants exhibited stable, or further diminished, cognitive performances (d = 0.78). In other words, individuals evaluated in a secondary gain context are more likely to exhibit ongoing cognitive limitations.

Rates of symptom exaggeration (e.g., insufficient effort) are known to increase dramatically in secondary gain contexts (AACN, 2007; Bush et al., 2005; Heilbronner et al., 2009). It follows that symptom exaggeration may partially explain why concussion groups evaluated in secondary gain contexts exhibit stable cognitive deficits beyond the typical recovery period. American Board of Clinical Neuropsychology members were surveyed to investigate rates of insufficient effort across clinical practices (Mittenberg, Patton, Canyock, & Condit, 2002). Respondents estimated that approximately 30% of personal injury and disability claimant groups exaggerate cognitive symptoms. Specific to head injury, respondents estimated that a much greater proportion of concussion groups exaggerate their symptoms (∼39%) relative to moderate/severe brain injury groups (∼9%).

Additional research illustrates the meaningful impact of insufficient effort on neuropsychological performance in forensic concussion samples. Green, Rohling, Lees-Haley, and Allen (2001) examined neuropsychological performance of a large forensic sample, 470 (52%) of whom had a history of concussion. The authors reported that concussion claimants with poor effort demonstrated significantly worse cognitive performances than claimants who had sustained moderate–severe brain injuries and exhibited sufficient effort on testing. Effort correlated highly (r = 0.73) with cognitive performance and accounted for 4.5 times more variance in overall cognitive performance than brain injury severity itself. In sum, there is clear evidence that evaluation of effort is crucial to the neuropsychological evaluation of late concussion, especially in secondary gain contexts.

Soldiers of Operations Iraqi Freedom (OIF) and Enduring Freedom (OEF) sustain concussion at an unprecedented rate relative to previous conflicts (McCrea et al., 2008). Survey data suggest that between 12% and 23% of returning OEF/OIF personnel report a history of concussion (Hoge et al., 2008; Schneiderman, Braver, & Kang, 2008; Terrio et al., 2009). Increased report of concussion appears to reflect the novel features of modern warfare, such as frequent exposure to explosive blast. Although the civilian literature would suggest a favorable course of recovery for veterans with remote histories of concussion, a meaningful proportion of OEF/OIF veterans continue to report cognitive limitations and other PCS upon their return from deployment (Hoge et al., 2008). Few studies have examined how evaluation context and effort impact neuropsychological performances in OEF/OIF veterans with histories of concussion.

Regarding evaluation context, veterans with histories of concussion may initiate a service-connection claim and have incentive to embellish or exaggerate symptoms (Howe, 2009), even though this does not necessarily imply that they are malingering. Service connection is established through the compensation and pension (CP) process, which is similar to an independent medical examination or other civilian disability assessment. The CP process involves a claim of disability attributed to service-related injury (e.g., concussion). Neuropsychological evaluations conducted in the CP context may determine whether an OEF/OIF veteran's claim of concussion is associated with cognitive limitations. We are not aware of previous research examining neuropsychological profiles of OEF/OIF veterans in the CP context.

Regarding effort, most studies inclusive of OEF/OIF samples treat effort as a confounding variable without necessarily describing the effect on cognitive performance (Belanger, Kretzmer, Yoash-Gantz, Pickett, & Tupler, 2009; Brenner et al., 2010; Vasterling et al., 2006). By including only those participants who demonstrate sufficient effort a priori, it is possible to explore research questions more effectively since the variance that would be attributed to poor effort is eliminated. For instance, in OEF/OIF samples with sufficient effort, researchers have examined whether blast and non-blast mechanisms of injury (Belanger et al., 2009), persisting symptoms following concussion (Brenner et al., 2010), or the deployment process (Vasterling et al., 2006) are associated with neuropsychological performance.

Only two published papers have focused explicitly on symptom validity testing and effort assessment in OEF/OIF samples. Armistead-Jehle (2010) examined Medical Symptom Validity Test (MSVT; Green, 2004) performance in a group of 45 OEF/OIF veterans with a history of concussion who underwent neuropsychological evaluation in a clinical context. Nearly 58% of the sample demonstrated insufficient effort on at least one MSVT indicator. Notably, the study did not summarize how effort impacted neuropsychological performance. A second study also focused on MSVT performance in 23 OEF/OIF veterans with historical concussion (Whitney, Shepard, Williams, Davis, & Adams, 2009). A significantly smaller percentage of individuals (17%) demonstrated insufficient effort on the MSVT and at least one other symptom validity test (e.g., Test of Memory Malingering; Tombaugh, 1996) or embedded effort indicator. Additional research with larger samples and a broader scope of effort measures is needed to determine the prevalence of insufficient effort among OEF/OIF veterans evaluated across various contexts.

The aim of the current study was to extend the symptom validity literature in OEF/OIF samples and clarify the impact of evaluation context, insufficient effort, and concussion on neuropsychological performances in the late stage of recovery (i.e., years post-concussion). The study explored three primary hypotheses. First, it was hypothesized that an OEF/OIF forensic concussion sample would demonstrate a higher rate of insufficient effort than an OEF/OIF research concussion sample, but a comparable rate of insufficient effort relative to a non-OEF/OIF forensic concussion sample. Second, regardless of context (forensic, research) or cohort (OEF/OIF, non-OEF/OIF), it was expected that insufficient effort would correlate significantly with overall cognitive performance. Third, when controlling for effort across OEF/OIF groups, it was predicted that individuals with a history of concussion would demonstrate comparable neuropsychological performances relative to those without concussion history.

Method

Participants

One-hundred and nineteen U.S. veterans participated in the current study (Table 1). All spoke English as a primary language and resided within the Midwestern region of the USA/Veterans Integrated Service Network (VISN) 23. Consistent with the general demographic makeup of this region, 111 (93.3%) were male and 111 (93.3%) were Caucasian. The mean age was 35.5 (SD = 10.2; range: 21–61 years). Most individuals obtained at least 12 years of formal education (M = 13.7; SD = 2.3; range: 6–21 years).

Table 1.

Demographics by evaluation context

Variable Full sample (N = 119)
 
Evaluation context (M [SD])
 
F-value 
   Forensic
 
Research
 
 
 Range M (SDOEF/OIF concussion (n = 24) Non-OEF/OIF concussion (n = 20) OEF/OIF concussion (n = 38) OEF/OIF no concussion (n = 37)  
Age (years) 21–61 35.5 (10.2) 33.1 (9.9) 43.6 (12.6)a,b,c 34.4 (7.8) 33.9 (9.3) 5.72*** 
Education (years) 6–21 13.7 (2.3) 13.0 (2.1) 12.3 (1.8)b,c 14.5 (1.9) 14.1 (2.6) 5.54*** 
Concussions (number) 1–101 7.7 (16.9) 1.6 (1.0)b 2.2 (3.2) 14.5 (23.2) n/a 6.37** 
Most recent Concussion (weeks) 32–2,178 327.0 (425.6) 173.0 (83.6) 841.2 (614.1)a,b 149.1 (62.9) n/a 36.70*** 
Variable Full sample (N = 119)
 
Evaluation context (M [SD])
 
F-value 
   Forensic
 
Research
 
 
 Range M (SDOEF/OIF concussion (n = 24) Non-OEF/OIF concussion (n = 20) OEF/OIF concussion (n = 38) OEF/OIF no concussion (n = 37)  
Age (years) 21–61 35.5 (10.2) 33.1 (9.9) 43.6 (12.6)a,b,c 34.4 (7.8) 33.9 (9.3) 5.72*** 
Education (years) 6–21 13.7 (2.3) 13.0 (2.1) 12.3 (1.8)b,c 14.5 (1.9) 14.1 (2.6) 5.54*** 
Concussions (number) 1–101 7.7 (16.9) 1.6 (1.0)b 2.2 (3.2) 14.5 (23.2) n/a 6.37** 
Most recent Concussion (weeks) 32–2,178 327.0 (425.6) 173.0 (83.6) 841.2 (614.1)a,b 149.1 (62.9) n/a 36.70*** 

Notes: OEF = Operations Enduring Freedom; OIF = Operations Iraqi Freedom.

aSignificantly different from Forensic OEF/OIF concussion sample.

bSignificantly different from Research OEF/OIF concussion sample.

cSignificantly different from Research OEF/OIF non-concussion sample.

**p < .01

***p < .001

Participants were assessed in either a forensic CP (n = 44, 37%) or research (n = 75, 63%) context. Within the CP context, participants were consecutively referred for neuropsychological evaluation related to claim of concussion(s) sustained in service. Neuropsychological evaluation of these individuals was conducted by the first author. Research participants were OEF/OIF veterans recruited to participate in ongoing research activities at the Minneapolis VA Medical Center. Participants underwent neuropsychological assessment under the direction of the first, second, and third authors. The research sample was a subgroup of approximately 2,600 National Guard personnel deployed in 2005 for a 15-month tour in Iraq.

MTBI for the current sample was operationalized according to the criteria set forth by the American Congress of Rehabilitation Medicine (ACRM; Kay et al., 1993). Under this scheme, concussion is defined by the following criteria: (a) any period of loss of consciousness (LOC), (b) any loss of memory for events surrounding the event, (c) any alteration in mental state at the time of the accident, including feeling dazed, disoriented, or confused, and (d) focal neurologic deficits. By definition, LOC cannot persist beyond 30 min, and post-traumatic amnesia (PTA) cannot extend beyond 24 h.

Forensic context: OEF/OIF concussion group (n = 24)

Nineteen (79.1%) of the OEF/OIF forensic participants endorsed a history of blast-related concussion, whereas the remaining five (20.9%) described concussions sustained by non-blast-related events (motor vehicle accident, fall-related). The mean number of reported concussions was 1.67 (SD = 1.05), with a median of 1.00 and a range of 1.00–4.00. Fifteen (62.5%) participants indicated that previous concussion resulted in alteration of consciousness, but denied that any of the concussions resulted in complete LOC or extended periods of PTA. Five (20.8%) endorsed neurologic signs, but denied that concussions resulted in any alteration of consciousness or PTA. Four (16%) indicated that concussions resulted in definite but brief LOC with minimal duration of PTA. A review of available medical records confirmed a psychiatric history that included post-traumatic stress disorder (PTSD) and other anxiety disorders (n = 20; 83.3%), major depressive disorder or depression not otherwise specified (n = 9; 37.5%), and/or alcohol abuse/dependence (n = 8; 33.3%). One (4.2%) participant indicated a history of premorbid learning disability. Physical symptoms included ongoing headaches and/or other diffuse body pain attributed to previous military activities (n = 22; 91.7%). Three (12.5%) participants experienced ongoing tinnitus.

Forensic context: Non-OEF/OIF concussion group (n = 20)

For the non-OEF/OIF concussion sample, seven (35%) participants completed tours from the Vietnam War era, and five (25%) completed tours during the Gulf War era. Eight (40%) participants sustained concussions as a result of military activities subsequent to the year 2000, but not in support of OEF/OIF. One non-OEF/OIF claimant (5%) endorsed a history of blast-related concussion during the time of service in Vietnam. The remainder of the non-OEF/OIF group described a history of non-blast-related concussions, such as motor vehicle accidents, boxing-related injuries, and concussions sustained as a result of falling. The mean number of reported concussions was 2.15 (SD = 3.23), with a median of 1.00 and a range of 1.00–15.00. Ten (50%) participants endorsed a history of definite LOC of brief duration with minimal PTA. Eight (40%) denied LOC, but indicated that concussions resulted in brief alteration of consciousness with minimal PTA. Two (10%) participants endorsed neurologic signs, but denied that concussions resulted in any alteration of consciousness or PTA. Medical records confirmed a psychiatric history of depression (n = 10; 50%), PTSD/other anxiety disorder (n = 8; 40%), and/or alcohol abuse/dependence (n = 3; 15%). Sixteen (80%) participants endorsed physical symptoms in the form of headaches or orthopedic pain.

Research context: OEF/OIF concussion group (n = 38)

Thirty-eight research participants endorsed a history of blast-related concussion during service in Iraq or Afghanistan. The mean number of blast exposures was 14.5 (SD = 23.20), with a median of 4.00 and a range of 1.00–100. Sixteen (42.1%) denied that concussions were associated with LOC or PTA, but did describe neurologic signs. Twelve (31.6%) participants indicated definite but brief LOC associated with prior concussions, with brief PTA. Ten (26.3%) participants denied LOC, but indicated that concussions resulted in brief alteration of consciousness with minimal PTA and neurologic signs. According to the structured clinical interview for DSM-IV-TR Axis I Disorders (SCID; First, Spitzer, Gibbon, & Williams, 2007), psychiatric history for the research concussion group included major depressive disorder or depression not otherwise specified (n = 17; 44.7%), alcohol abuse/dependence (n = 12; 31.6%), and/or PTSD/other anxiety disorder (n = 12; 31.6%). Seventeen (44.7%) participants endorsed ongoing physical symptoms of headaches or other pain.

Research context: OEF/OIF non-concussion group (n = 37)

Thirty-seven OEF/OIF research participants denied any history of combat-related concussion. SCID-confirmed psychiatric history included alcohol abuse/dependence (n = 10; 27.0%), major depressive disorder or depression not otherwise specified (n = 8; 21.6%), and/or PTSD/other anxiety disorder (n = 9; 24.3%). One participant (2.7%) endorsed history of possible learning disorder. Nine (24.3%) participants endorsed history of headaches and other physical pain (n = 9; 24.3%). Eight (21.6%) participants endorsed physical symptoms of headaches or other bodily pain.

Measures and Procedure

Neuropsychologists have developed a variety of symptom validity tests and symptom validity indices to evaluate response validity (Boone, 2007). Symptom validity tests, either forced-choice or non-forced-choice measures, are developed prospectively to assess effort and task engagement. Symptom validity indicators, either forced-choice or non-forced-choice, are derived from standard measures of cognitive ability.

Four effort measures were administered. Symptom validity tests included the Victoria Symptom Validity Test (VSVT; Slick, Hopp, Strauss, & Spellacy, 1996) and Rey-15 Item and Recognition Test (FIT; |Boone, Salazar, Lu, Warner-Chacon, & Razani, 2002). Symptom validity indicators included the Wechsler Adult Intelligence Scale-Third Edition (WAIS-III) Digit-Span subtest (Wechsler, 1997) and the California Verbal Learning Test-Second Edition (CVLT-II) Forced-Choice Recognition Trial (CVLT-II; Delis, Kaplan, Kramer, & Ober, 2000). “Insufficient effort” for each of these indicators was defined as follows. VSVT: Total Recognition <43 or Difficult Item Recognition <20 (Grote et al., 2000); FIT: Combination Score <21 (Boone et al., 2002); WAIS-III Digit Span: Reliable Digit Span <8 (Greiffenstein, Baker, & Gola, 1994); and CVLT-II: Forced Choice Recognition <15 (Delis et al., 2000).

Research participants completed a test battery to assess multiple domains of cognitive function (Table 2). General intelligence was assessed with the Information subtest of the WAIS-III. Attention was assessed with the WAIS-III Digit-Span subtest. Language and verbal fluency were assessed with Controlled Oral Word Association Test (COWAT; Gladsjo et al., 1999). Verbal learning/memory was assessed with the CVLT-II. Executive functioning and cognitive efficiency was measured by the Trail Making Tests A and B (TMT A and B; Heaton et al., 1991), Stroop Color Word Test (Golden, 1978), and the Digit-Symbol Coding subtest from the WAIS-III. Although individuals evaluated in the forensic context completed a more extended neuropsychological test battery, analyses were conducted only on those measures that were administered uniformly across groups.

Table 2.

Neuropsychological performances by evaluation context

Measure Evaluation context (M [SD])
 
F-value η2 
 Forensic
 
Research
 
  
 OEF/OIF concussion (n = 24) Non-OEF/OIF concussion (n = 20) OEF/OIF concussion (n = 38) OEF/OIF no concussion (n = 37)   
Effort/motivation 
 VSVT       
  Easy items 22.92 (2.00)b,c 23.55 (0.89) 23.90 (0.31) 23.89 (0.32) 5.90*** .13 
  Difficult items 15.63 (6.25)b,c 16.90 (5.51)b,c 22.11 (2.87) 23.24 (1.34) 24.95*** .39 
  Total items 38.54 (7.43)b,c 40.45 (5.94)b,c 46.00 (3.08) 47.14 (1.44) 24.02*** .39 
 Rey FIT       
  Combination 26.13 (3.72)b,c 25.65 (5.26)b,c 28.84 (2.06) 29.35 (1.57) 10.14*** .21 
 CVLT-II       
  Forced-choice 15.50 (1.18) 14.90 (1.97)b,c 15.87 (0.41) 15.92 (0.36) 5.34** .12 
 WAIS-III       
  RDS+ 8.38 (1.71) 8.65 (1.60) 9.63 (1.98) 9.68 (2.06) 3.50 .08 
Intellectual function       
 WAIS-III       
  Information (SS) 11.58 (1.82) 10.50 (3.46) 11.97 (1.92) 11.95 (2.30) 2.06 .05 
Attention       
 WAIS-III       
  Digit Span (SS) 8.42 (2.77) 8.30 (2.34) 9.61 (2.55) 10.03 (2.67) 3.04 .07 
Language       
 COWAT (T42.83 (8.52) 45.60 (6.47) 44.79 (10.43) 45.97 (9.54) 0.61 .02 
Verbal Memory       
 CVLT-II       
  Trials 1–5 (T46.33 (9.23)c 48.25 (10.39) 52.82 (8.61) 55.60 (8.57) 6.27*** .14 
  Long Free (z−0.73 (1.12)c −0.48 (0.95) 0.03 (0.99) 0.30 (0.89) 6.84*** .15 
Executive function 
 TMT A (T39.96 (9.78)b,c 46.30 (9.72) 48.71 (10.44) 49.76 (11.25) 4.84** .11 
 TMT B (T44.71 (10.22)c 47.45 (12.62) 51.45 (10.25) 52.84 (6.12) 4.23**  
 Digit-Symbol (SS) 7.12 (2.53)b,c 6.95 (1.64)b,c 9.92 (2.53) 10.19 (2.61) 14.23*** .10 
 Stroop Word (T39.12 (7.54)c 39.40 (8.04)c 45.47 (9.01) 47.16 (8.97) 6.52*** .15 
 Stroop Color (T40.04 (7.00)c 37.45 (7.00)b,c 45.29 (8.60) 46.84 (7.10) 8.93*** .19 
 Stroop C-W (T43.78 (8.10) 41.15 (7.90)c 46.95 (9.09) 50.03 (8.93) 5.36** .12 
 Effort Failures (Raw) 1.17 (0.87)b,c 1.15 (1.31)b,c 0.32 (0.58) 0.19 (0.46) 12.67*** .25 
 OTBM (z−0.75 (0.52)b,c −0.64 (0.56)b,c −0.15 (0.55) 0.02 (0.47) 14.72*** .28 
Measure Evaluation context (M [SD])
 
F-value η2 
 Forensic
 
Research
 
  
 OEF/OIF concussion (n = 24) Non-OEF/OIF concussion (n = 20) OEF/OIF concussion (n = 38) OEF/OIF no concussion (n = 37)   
Effort/motivation 
 VSVT       
  Easy items 22.92 (2.00)b,c 23.55 (0.89) 23.90 (0.31) 23.89 (0.32) 5.90*** .13 
  Difficult items 15.63 (6.25)b,c 16.90 (5.51)b,c 22.11 (2.87) 23.24 (1.34) 24.95*** .39 
  Total items 38.54 (7.43)b,c 40.45 (5.94)b,c 46.00 (3.08) 47.14 (1.44) 24.02*** .39 
 Rey FIT       
  Combination 26.13 (3.72)b,c 25.65 (5.26)b,c 28.84 (2.06) 29.35 (1.57) 10.14*** .21 
 CVLT-II       
  Forced-choice 15.50 (1.18) 14.90 (1.97)b,c 15.87 (0.41) 15.92 (0.36) 5.34** .12 
 WAIS-III       
  RDS+ 8.38 (1.71) 8.65 (1.60) 9.63 (1.98) 9.68 (2.06) 3.50 .08 
Intellectual function       
 WAIS-III       
  Information (SS) 11.58 (1.82) 10.50 (3.46) 11.97 (1.92) 11.95 (2.30) 2.06 .05 
Attention       
 WAIS-III       
  Digit Span (SS) 8.42 (2.77) 8.30 (2.34) 9.61 (2.55) 10.03 (2.67) 3.04 .07 
Language       
 COWAT (T42.83 (8.52) 45.60 (6.47) 44.79 (10.43) 45.97 (9.54) 0.61 .02 
Verbal Memory       
 CVLT-II       
  Trials 1–5 (T46.33 (9.23)c 48.25 (10.39) 52.82 (8.61) 55.60 (8.57) 6.27*** .14 
  Long Free (z−0.73 (1.12)c −0.48 (0.95) 0.03 (0.99) 0.30 (0.89) 6.84*** .15 
Executive function 
 TMT A (T39.96 (9.78)b,c 46.30 (9.72) 48.71 (10.44) 49.76 (11.25) 4.84** .11 
 TMT B (T44.71 (10.22)c 47.45 (12.62) 51.45 (10.25) 52.84 (6.12) 4.23**  
 Digit-Symbol (SS) 7.12 (2.53)b,c 6.95 (1.64)b,c 9.92 (2.53) 10.19 (2.61) 14.23*** .10 
 Stroop Word (T39.12 (7.54)c 39.40 (8.04)c 45.47 (9.01) 47.16 (8.97) 6.52*** .15 
 Stroop Color (T40.04 (7.00)c 37.45 (7.00)b,c 45.29 (8.60) 46.84 (7.10) 8.93*** .19 
 Stroop C-W (T43.78 (8.10) 41.15 (7.90)c 46.95 (9.09) 50.03 (8.93) 5.36** .12 
 Effort Failures (Raw) 1.17 (0.87)b,c 1.15 (1.31)b,c 0.32 (0.58) 0.19 (0.46) 12.67*** .25 
 OTBM (z−0.75 (0.52)b,c −0.64 (0.56)b,c −0.15 (0.55) 0.02 (0.47) 14.72*** .28 

Notes: OEF = Operations Enduring Freedom; OIF = Operations Iraqi Freedom; VSVT = Victoria Symptom Validity Test (Slick et al., 1996); Rey FIT = Rey 15-Item Test (Boone et al., 2002); WAIS-III = Wechsler Adult Intelligence Scale-Third Edition (Wechsler, 1997); COWAT = Controlled Oral Word Association Test (Gladsjo et al., 1999); CVLT-II = California Verbal Learning Test, Second Edition (Delis et al., 2000); Stroop = Stroop Color and Word Test (Golden, 1978); TMT = Trail Making Test (Heaton, Grant, & Matthews, 1991); OTBM = overall test battery mean performance (Green et al., 2001).

bSignificantly different from Research OEF/OIF concussion sample.

cSignificantly different from Research OEF/OIF non-concussion sample.

**p < .01

***p < .001

Similar to Green and colleagues (2001), an overall test battery mean (OTBM) of neuropsychological performances was generated. This was achieved by first transforming standard score performances (i.e., age-corrected scaled scores and T-scores) from 10 of the neuropsychological measures into z-scores, and then generating an average z-score across the measures. The WAIS-III Information subtest was used as an indicator of premorbid ability and did not contribute to the OTBM. OTBM was based on: WAIS-III Digit-Span and Digit-Symbol Coding (Scaled Scores), COWAT (T-score), CVLT-II Trials 1–5 (T-score), CVLT-II Long Free Recall (z-score), TMT A and B (T-scores), and Stroop Color Word Test (T-scores for Word, Color, and Interference trial) performances.

Results

Preliminary analyses revealed significant differences for age, F(3,115) = 5.72, p = .001, and education, F(3,115) = 5.54, p = .001, across the four samples (Table 1). Tukey's HSD post hoc analyses revealed that the non-OEF/OIF forensic group was significantly older than the remaining three groups (p < .005). The non-OEF/OIF forensic group also completed significantly fewer years of education than the two research groups, p < .05. Neither age nor education was significantly correlated with overall number of effort failures. Education (r = .23, p = .01), but not age (r = .11, p = .25), was significantly correlated with overall neuropsychological performance.

Weeks since most recent concussion, F(2,78) = 36.70, p < .0001, and number of concussive injuries, F(2,78) = 6.37, p < .01, were significantly different across the three concussion groups. Tukey's HSD post hoc analyses revealed that the non-OEF/OIF concussion group reported a greater duration of time since most recent concussion relative to the OEF/OIF forensic and OEF/OIF research concussion groups (p < .0001). Additionally, the OEF/OIF forensic concussion group reported fewer previous concussions than the OEF/OIF research concussion group (p < .01).

To elucidate whether these factors were meaningful with regard to effort and neuropsychological performances, weeks since injury and number of concussions were correlated with the number of effort failures and the OTBM. These analyses revealed that weeks since most recent concussion was not significantly correlated with the number of diminished effort performances (r = .19, p = .08) or the OTBM (r = −.11, p = .32). The number of self-reported concussions was not significantly associated with the number of diminished effort performances (r = −.20, p = .07) or OTBM performance (r = .15, p = .17).

Table 2 presents effort and neuropsychological performances across the four samples. One-way analysis of variance (ANOVA) with group membership as the independent variable and effort as the dependent variable revealed that number of diminished effort performances was significantly different across the four groups—F(3,115) = 12.67, p < .0001. Tukey's HSD post hoc analyses revealed that both forensic groups demonstrated a greater number of diminished effort failures than research groups (p < .0001). Effort failures were not significantly different within contexts (p > .05).

On the individual effort measure level, ANOVA with group membership as the independent variable and performances on each of the effort measures revealed significant between-group differences on the VSVT Total items, F(3,115) = 24.02, p < .0001, VSVT Easy items, F(3,115) = 5.90, p < .0001, VSVT Difficult items, F(3,115) = 24.95, p < .0001, Rey FIT Combination score, F(3,115) = 10.14, p < .0001, and CVLT-II FC, F(3,115) = 5.34, p < .01). After correcting for multiple comparisons and assigning p-value to .01, RDS was not significantly different across groups, F(3,115) = 3.50, p = .02 and was not further investigated.

Tukey's HSD post hoc analyses revealed that both forensic groups showed significantly worse VSVT Total and Difficult item performances than both the research groups (p < .0001). The OEF/OIF forensic group showed significantly worse VSVT Easy item performance relative to both the research groups (p < .0001), whereas the non-OEF/OIF forensic group did not show these differences. VSVT performances were not meaningfully different between the OEF/OIF forensic and the non-OEF/OIF forensic groups (p > .05). Post hoc analyses revealed that both forensic concussion groups showed significantly worse Rey FIT Combination performance relative to both research groups, p < .005. Rey FIT Combination performances were not meaningfully different within contexts. Post hoc analyses revealed that the non-OEF/OIF forensic group showed significantly worse CVLT-FC performance relative to both research groups. CVLT-FC performance was not significantly different between the OEF/OIF forensic group and the two research groups, p > .05. CVLT-FC performance was not meaningfully different within contexts.

To further understand the impact of evaluation context on effort, rates of insufficient effort for each of three indicators (excluding RDS) were observed by evaluation context. Effort performances were comparable within contexts and justified creation of combined forensic (n = 44) and research (n = 75) groups. As shown in Fig. 1, 26 of 44 forensic participants (59.1%) showed at least one indication of insufficient effort, 7 of 44 (15.9%) showed two indications, and 4 of 44 (9.1%) showed three indications. In contrast, 8 of 75 (10.7%) research participants showed one indication of insufficient effort and 0% showed greater than one indication of insufficient effort.

Fig. 1.

Frequency of insufficient effort indicators by context. N = 119, including combined forensic (n = 44) and research (n = 75) samples. Effort indicators included VSVT (Slick, Hopp, Strauss, & Spellacy, 1996), Rey-15 FIT combination (Boone et al., 2002), and CVLT-II FC (Delis et al., 2000).

Fig. 1.

Frequency of insufficient effort indicators by context. N = 119, including combined forensic (n = 44) and research (n = 75) samples. Effort indicators included VSVT (Slick, Hopp, Strauss, & Spellacy, 1996), Rey-15 FIT combination (Boone et al., 2002), and CVLT-II FC (Delis et al., 2000).

Effort performances were also examined to identify the rates of insufficient effort on each individual measure. Of the 44 forensic participants, 26 (59.1%) demonstrated insufficient effort on the VSVT, whereas only six (8%) OEF/OIF research participants demonstrated insufficient effort on the VSVT. Seven (15.9%) forensic participants and two (2.7%) research participants showed insufficient CVLT-II FC performance. Four (9.1%) forensic participants demonstrated poor effort on the Rey FIT relative to 0% of the research sample.

Table 3 illustrates the magnitude of relationship between effort indicators and overall cognitive performance in the combined forensic and combined research groups, respectively. Effort and overall cognitive performance correlates were generally higher in the forensic group (r = .45–.58) relative to the research group (r = .10–.29). In other words, effort accounted for 20.3%–33.6% of the performance variance in the forensic group, compared with 1.0%–8.4% in the research group.

Table 3.

Correlates of three effort measures and overall neuropsychological performance

Measure VSVT Rey FIT CVLT-II OTBM 
VSVT — −.09 −.06 .29 
Rey FIT .57** — .28 .17 
CVLT-II .48** .62** — .10 
OTBM .45* .58** .48** — 
Measure VSVT Rey FIT CVLT-II OTBM 
VSVT — −.09 −.06 .29 
Rey FIT .57** — .28 .17 
CVLT-II .48** .62** — .10 
OTBM .45* .58** .48** — 

Notes: Combined research sample (n = 75) depicted above the diagonal; Combined forensic sample (n = 44) correlations depicted below the diagonal. VSVT = Victoria Symptom Validity Test (Slick et al., 1996); Rey FIT = Rey 15-Item Test (Boone et al., 2002); CVLT-II = California Verbal Learning Test, Second Edition (Delis et al., 2000); OTBM = overall test battery mean performance (Green et al., 2001).

*p < .01

**p < .001.

Excluding OEF/OIF participants who displayed any indication of insufficient effort permitted unbiased assessment of whether concussion history impacted neuropsychological performance years after injury (Hypothesis 3). This resulted in 33 OEF/OIF participants with a history of concussion and 31 OEF/OIF participants without concussion with uniformly intact effort. Overall cognitive performance between these two groups was not significantly different—t(62) = 1.63, p = .11. However, it was noted that only five of the participants with concussion were derived from the forensic sample. Given concern that this limited number might not adequately represent OEF/OIF concussion samples evaluated in forensic contexts, neuropsychological performance comparisons were re-examined between the research concussion and the non-concussion groups. This resulted in identification of 28 research concussion participants and 31 non-concussion participants who demonstrated uniformly intact effort. As shown in Table 4, analyses did not yield any significant between-group differences across the neuropsychological measures examined nor on the OTBM performance.

Table 4.

Neuropsychological performances for OEF/OIF research concussion and non-concussion samples with uniformly sufficient effort

Measure OEF/OIF research sample (M [SD])
 
t d 
 Concussiona Non-Concussionb   
Intellectual function 
 WAIS-III     
  Information (SS) 12.21 (2.01) 12.26 (2.25) 0.08 0.22 
Attention 
 WAIS-III     
  Digit Span (SS) 10.50 (2.29) 10.65 (2.37) 0.24 0.06 
Language 
 COWAT (T47.04 (8.94) 46.97 (9.69) 0.03 0.01 
Verbal Memory 
 CVLT-II     
  Trials 1–5 (T54.43 (8.95) 57.16 (8.04) 1.24 0.32 
  Long Free (z0.27 (0.91) 0.45 (0.81) 0.82 0.21 
Executive Function 
 TMT A (T48.89 (9.89) 49.61 (11.66) 0.25 0.10 
 TMT B (T52.21 (10.20) 52.77 (5.83) 0.26 0.07 
 Digit-Symbol (SS) 10.46 (2.52) 10.35 (2.47) 0.17 0.04 
 Stroop Word (T46.29 (9.10) 48.58 (8.64) 0.99 0.26 
 Stroop Color (T46.00 (8.88) 47.26 (6.90) 0.61 0.16 
 Stroop Color-Word (T48.86 (9.35) 50.52 (9.22) 0.69 0.18 
OTBM (z0.00 (0.55) 0.11 (0.42) 0.95 0.22 
Measure OEF/OIF research sample (M [SD])
 
t d 
 Concussiona Non-Concussionb   
Intellectual function 
 WAIS-III     
  Information (SS) 12.21 (2.01) 12.26 (2.25) 0.08 0.22 
Attention 
 WAIS-III     
  Digit Span (SS) 10.50 (2.29) 10.65 (2.37) 0.24 0.06 
Language 
 COWAT (T47.04 (8.94) 46.97 (9.69) 0.03 0.01 
Verbal Memory 
 CVLT-II     
  Trials 1–5 (T54.43 (8.95) 57.16 (8.04) 1.24 0.32 
  Long Free (z0.27 (0.91) 0.45 (0.81) 0.82 0.21 
Executive Function 
 TMT A (T48.89 (9.89) 49.61 (11.66) 0.25 0.10 
 TMT B (T52.21 (10.20) 52.77 (5.83) 0.26 0.07 
 Digit-Symbol (SS) 10.46 (2.52) 10.35 (2.47) 0.17 0.04 
 Stroop Word (T46.29 (9.10) 48.58 (8.64) 0.99 0.26 
 Stroop Color (T46.00 (8.88) 47.26 (6.90) 0.61 0.16 
 Stroop Color-Word (T48.86 (9.35) 50.52 (9.22) 0.69 0.18 
OTBM (z0.00 (0.55) 0.11 (0.42) 0.95 0.22 

Notes: OEF = Operations Enduring Freedom; OIF = Operations Iraqi Freedom;WAIS-III = Wechsler Adult Intelligence Scale-Third Edition; COWAT = Controlled Oral Word Association Test; CVLT-II = California Verbal Learning Test, Second Edition; TMT = Trail Making Test; OTBM = Overall test battery mean (across 10 cognitive measures). All t-tests were non-significant.

an = 28

bn = 31.

Discussion

The aim of the current study was to explore the impact of evaluation context, insufficient effort, and concussion on neuropsychological performances in OEF/OIF veterans with varied concussion histories evaluated years following injury. As expected, in comparison to the research sample, OEF/OIF veterans with histories of concussion showed much higher rates of insufficient effort when evaluated in a forensic context. In contrast, the OEF/OIF forensic concussion sample demonstrated a similar rate of insufficient effort relative to a non-OEF/OIF forensic concussion sample. Insufficient effort, in turn, accounted for a significant proportion of variance in overall cognitive test performance in the forensic sample (from 20.3% to 33.6%), consistent with findings from civilian forensic concussion literature (Green et al., 2001; Rohling & Demakis, 2010). Insufficient effort was clearly less relevant to the research samples, and consequently did not show a meaningful association with overall cognitive performance (1.0%–8.4% variance accounted for by effort). After controlling for effort, OEF/OIF concussion and non-concussion groups did not show significant neuropsychological performance differences. The latter finding seems consistent with an extended civilian literature (Belanger et al., 2005; Binder et al., 1997; Frencham, Fox, & Maybery, 2005; Iverson, 2005; Schretlen & Shapiro, 2003) and a burgeoning OEF/OIF literature (Brenner et al., 2010). After controlling for evaluation context and relevant non-concussion-related factors (e.g., insufficient effort), remote history of concussion does not typically contribute to meaningful neuropsychological impairment in the late stage of recovery.

Merging current with previous OEF/OIF symptom validity research (Armistead-Jehle, 2010; Whitney et al., 2009), findings suggest that rates of insufficient effort clearly vary according to the setting in which veterans undergo assessment. In the current analysis, 22 of 99 (22.2%) OEF/OIF veterans demonstrated insufficient effort on the VSVT, a forced-choice effort measure. Six (8%) of these were research participants and 16 (66.7%) were forensic participants. In previously published investigations that include clinical OEF/OIF samples, results of forced-choice effort testing with the MSVT suggest that rates of diminished effort ranged from 17% (Whitney et al., 2009) to nearly 58% (Armistead-Jehle, 2010). Although it is acknowledged that classification accuracies may vary between the VSVT and the MSVT, available results suggest that 8% of OEF/OIF research participants, 17%–58% of clinical patients, and 67% of forensic claimants show insufficient effort on forced-choice measures. Cumulative findings suggest that response validity assessment is crucial in any evaluation context (including research), but particularly in forensic and clinical settings.

To be clear, these concerning rates of insufficient effort do not necessarily imply intentional subversion of performance (malingering). In fact, few of the current OEF/OIF forensic concussion participants demonstrated statistically less-than-chance level performance on forced-choice effort testing that would support a formal diagnosis of malingered neurocognitive dysfunction (Slick, Sherman, & Iverson, 1999). Among the 24 OEF/OIF veterans evaluated in the current forensic context, only three (6.8%) participants showed statistically less-than-chance level (p < .05) performance on the VSVT. For some OEF/OIF veterans, subtle variations in task engagement may reflect alternate explanations, such as emotional difficulty, pain, or fatigue (Vanderploeg & Belanger, 2009). At the same time, clinicians should recognize that most individuals with clinical conditions such as depression (Ashendorf, Constantinou, & McCaffrey, 2004) and pain (e.g., Etherton, Bianchini, Greve, & Ciota, 2005) perform very well on symptom validity testing. Confidence in assigning a diagnosis of malingering should increase in the presence of multiple indications of insufficient effort that are below established cut-scores (Larrabee, 2008).

It is also important to note that rates of insufficient effort were comparable in the non-OEF/OIF concussion sample and the OEF/OIF forensic concussion sample. Although additional data from larger samples are needed, current findings would suggest that cognitive response invalidity is a context-specific, not cohort-specific, phenomenon in veteran concussion samples. Veterans involved in disability claim related to a previous concussive injury are more likely to subvert performance, regardless of background, than those who undergo neuropsychological evaluation in other contexts.

Findings also illustrate the differential utility of symptom validity tests. The OEF/OIF concussion groups in the current analysis were most likely to demonstrate insufficient effort on the VSVT, particularly on the difficult items. OEF/OIF veterans were far less likely to show diminished effort on the Rey FIT, CVLT-II FC, or RDS. In fact, RDS was not meaningfully different across groups, suggesting that it may have limited utility relative to other effort indicators. Clinicians should strongly consider employing the VSVT or other forced-choice effort measures during neuropsychological assessment of OEF/OIF concussion groups, particularly in forensic settings.

The current study is among the first to present relationships of multiple effort measures in the OEF/OIF forensic samples. Administration of multiple indicators essentially allows for a “continuous sampling” of effort throughout testing (Boone, 2009), and ultimately improves accuracy in the differential diagnosis of malingering (Bush et al., 2005; Larrabee, 2008; Nelson et al., 2003; Slick et al., 1999). This practice is only justified to the extent that effort measures are not redundantly correlated with one another (Rosenfeld, Sands, & van Gorp, 2000). As the VSVT, Rey FIT, and CVLT-II FC performances were significantly, but only moderately correlated with one another (Table 2), they may provide unique information with regard to task engagement. Future research could investigate incremental validity of multiple effort measures in veteran samples (e.g., Larrabee, 2008).

Present findings illustrate the non-specificity of PCS, which are common in healthy community samples (Paniak et al., 2002) as well as non-TBI clinical samples (Radanov, Dvorak, & Valach, 1992). For example, 21% of the current non-concussion research sample reported ongoing headaches and other forms of physical pain in spite of no reported history of concussion. This result would suggest that clinicians use caution in determining concussion severity on the basis of current PCS in OEF/OIF samples. Continued research is needed to identify factors that account for endorsement of persisting PCS (cf. Iverson, Lange, Brooks, & Rennison, 2010; Tsanadis et al., 2008) and PTSD (cf. Demakis, Gervais, & Rohling, 2008) in clinical neuropsychological evaluation settings. Greater understanding of co-morbid physical and psychological injuries may assist in conceptualization of subjective cognitive complaints and ultimately promote a “biopsychosocial approach” to treatment of persisting PCS (McCrea et al., 2009).

A limitation of the current study is that most OEF/OIF veterans sustained relatively few concussions, with most participants reporting only a single concussion as a result of combat activity. It is possible that samples sustaining more recurrent concussions would show greater cognitive compromise relative to non-concussion samples. Accumulating data suggest that recurrent concussion may be associated with a less favorable course of recovery relative to a single concussion and contribute to persisting cognitive limitation (Belanger, Spiegel, & Vanderploeg, 2010).

An additional limitation of this research is that we did not explore the impact of substance abuse/dependence, emotional factors, and other non-concussion-related variables on neuropsychological performances. This is significant because alcohol and other substances are known to have an untoward effect on cognitive functioning, and emotional distress in itself contributes to attention and other cognitive limitations independent of concussion history (e.g., Vasterling, Brailey, Constans, & Sutker, 1998; Vasterling et al., 2002). Future research is needed to identify how substance use, PTS, chronic pain, and other non-concussion-related factors impact cognitive performances across evaluation contexts. Additionally, the current study did not thoroughly explore the possibility that PTSD and concussion interact with one another to negatively impact cognitive outcomes. Future longitudinal research might document whether individuals with co-morbid PTSD and concussion demonstrate cognitive improvement with greater management of psychological difficulties.

In summary, this research suggests that rates of insufficient effort vary by evaluation context in OEF/OIF concussion samples. Consistent with civilian literature, insufficient effort accounts for a significant proportion of cognitive performance variance in forensic contexts. After controlling for evaluation context and insufficient effort, OEF/OIF veterans with remote history of concussion demonstrate similar cognitive performances relative to non-concussed veterans.

Funding

This study was supported in part by Grants funded by the Congressionally Directed Medical Research Programs (number PT074550, contract W81XWH-08-2-0038) to SRS and the Minnesota Veterans Research Institute (MVRI) to NWN.

Conflict of interest

None declared.

References

American Academy of Clinical Neuropsychology (AACN) Board of Directors.
American Academy of Clinical Neuropsychology (AACN) practice guidelines for neuropsychological assessment and consultation
The Clinical Neuropsychologist
 , 
2007
, vol. 
21
 (pg. 
209
-
231
)
Armistead-Jehle
P.
Symptom validity test performance in U.S. veterans referred for evaluation of mild TBI
Applied Neuropsychology
 , 
2010
, vol. 
17
 (pg. 
52
-
59
)
Ashendorf
L.
Constantinou
M.
McCaffrey
R. J.
The effect of depression and anxiety on the TOMM in community-dwelling older adults
Archives of Clinical Neuropsychology
 , 
2004
, vol. 
19
 (pg. 
125
-
130
)
Belanger
H. G.
Curtiss
G.
Demery
J. A.
Lebowitz
B. K.
Vanderploeg
R. D.
Factors moderating neuropsychological outcomes following mild traumatic brain injury: A meta-analysis
Journal of the International Neuropsychological Society
 , 
2005
, vol. 
11
 (pg. 
215
-
227
)
Belanger
H. G.
Kretzmer
T.
Yoash-Gantz
R.
Pickett
T.
Tupler
L. A.
Cognitive sequelae of blast-related versus other mechanisms of brain trauma
Journal of the International Neuropsychological Society
 , 
2009
, vol. 
15
 (pg. 
1
-
8
)
Belanger
H. G.
Spiegel
E.
Vanderploeg
R. D.
Neuropsychological performance following a history of multiple self-reported concussions: A meta-analysis
Journal of the International Neuropsychological Society
 , 
2010
, vol. 
16
 (pg. 
262
-
267
)
Binder
L. M.
Rohling
M. L.
Money matters: A meta-analytic review of the effects of financial incentives on recovery after closed-head injury
American Journal of Psychiatry
 , 
1996
, vol. 
153
 (pg. 
7
-
10
)
Binder
L. M.
Rohling
M. L.
Larrabee
G. J.
A review of mild head trauma. Part I: Meta-analytic review of neuropsychological studies
Journal of Clinical and Experimental Neuropsychology
 , 
1997
, vol. 
19
 (pg. 
421
-
431
)
Boone
K. B.
Assessment of feigned cognitive impairment: A neuropsychological perspective
 , 
2007
New York, NY
Guilford Press
Boone
K. B.
The need for continuous and comprehensive sampling of effort/response bias during neuropsychological examinations
The Clinical Neuropsychologist
 , 
2009
, vol. 
23
 (pg. 
729
-
741
)
Boone
K. B.
Salazar
X.
Lu
P.
Warner-Chacon
K.
Razani
J.
The Rey 15-Item Recognition Trial: A technique to enhance sensitivity of the Rey 15-Item Memorization Test
Journal of Clinical and Experimental Neuropsychology
 , 
2002
, vol. 
24
 (pg. 
561
-
573
)
Brenner
L. A.
Terrio
H.
Homaifar
B. Y.
Gutierrez
P. M.
Staves
P. J.
Harwood
J. E. F.
, et al.  . 
Neuropsychological test performance in soldiers with blast-related mild TBI
Neuropsychology
 , 
2010
, vol. 
24
 (pg. 
160
-
167
)
Bush
S. S.
Ruff
R. M.
Troster
A. I.
Barth
J. T.
Koffler
S. P.
Pliskin
N. H.
, et al.  . 
NAN Position Paper: Symptom validity assessment: Practice issues and medical necessity
Archives of Clinical Neuropsychology
 , 
2005
, vol. 
20
 (pg. 
419
-
426
)
Delis
D. C.
Kaplan
E.
Kramer
J.
Ober
B. A.
California Verbal Learning Test – Second Edition (CVLT-II)
 , 
2000
San Antonio, TX
The Psychological Corporation
Demakis
G. J.
Gervais
R. O.
Rohling
M. L.
The effect of failure on cognitive and psychological symptom validity tests in litigants with symptoms of post-traumatic stress disorder
The Clinical Neuropsychologist
 , 
2008
, vol. 
22
 (pg. 
879
-
895
)
Etherton
J. L.
Bianchini
K. J.
Greve
K. W.
Ciota
M. A.
Test of Memory Malingering performance is unaffected by laboratory-induced pain: Implications for clinical use
Archives of Clinical Neuropsychology
 , 
2005
, vol. 
20
 (pg. 
375
-
384
)
First
M. B.
Spitzer
R. L.
Gibbon
M.
Williams
J. S. W.
Structured clinical interview for DSM-IV-TR Axis I Disorders – Non-patient Edition (SCID-I/NP, 1/2007 revision)
 , 
2007
New York
Biometrics Research Department
Frencham
K. A.
Fox
A. M.
Maybery
M. T.
Neuropsychological studies of mild traumatic brain injury: A meta-analytic review of research since 1995
Journal of Clinical and Experimental Neuropsychology
 , 
2005
, vol. 
27
 (pg. 
334
-
351
)
Gladsjo
J. A.
Schuman
C. C.
Evans
J. D.
Peavy
G. M.
Miller
S. W.
Heaton
R. K.
Norms for letter and category fluency: Demographic corrections for age, education, and ethnicity
Assessment
 , 
1999
, vol. 
6
 (pg. 
147
-
178
)
Golden
C. J.
Stroop Color and Word Test: A manual for clinical and experimental uses
 , 
1978
Chicago
Stoelting
Green
P.
Green's Medical Symptom Validity Test (MSVT) for Windows: User's manual
 , 
2004
Edmonton, Canada
Green's Publishing
Green
P.
Rohling
M. L.
Lees-Haley
P. R.
Allen
L. M.
Effort has a greater effect on test scores than severe brain injury in compensation claimants
Brain Injury
 , 
2001
, vol. 
15
 (pg. 
1045
-
1060
)
Greiffenstein
M. F.
Baker
J. W.
Comparison of premorbid and post-injury MMPI-2 profiles in late postconcussion claimants
The Clinical Neuropsychologist
 , 
2001
, vol. 
15
 (pg. 
162
-
170
)
Greiffenstein
M. F.
Baker
W. J.
Gola
T.
Validation of malingered amnesic measures with a large clinical sample
Psychological Assessment
 , 
1994
, vol. 
6
 (pg. 
218
-
224
)
Grote
C.L.
Kooker
E.K.
Garron
D.C.
Nyenhuis
D.L.
Smith
C.A.
Mattingly
M.L.
Performance of compensation seeking and non-compensation seeking samples on the Victoria Symptom Validity Test: Cross-validation and extension of a standardization study
Journal of Clinical and Experimental Neuropsychology
 , 
2000
, vol. 
22
 (pg. 
709
-
719
)
Heaton
R. K.
Grant
I.
Matthews
C. G.
Comprehensive norms for an Expanded Halstead–Reitan Battery: Demographic corrections, research findings, and clinical applications
 , 
1991
Odessa, FL
Psychological Assessment Resources
Heilbronner
R. L.
Sweet
J. J.
Morgan
J. E.
Larrabee
G. J.
Millis
S. R.
Conference Participants
American Academy of Clinical Neuropsychology consensus conference statement on the neuropsychological assessment of effort, response bias, and malingering
The Clinical Neuropsychologist
 , 
2009
, vol. 
23
 (pg. 
1093
-
1129
)
Hoge
C. W.
McGurk
D.
Thomas
J. L.
Cox
A. L.
Engel
C. C.
, et al.  . 
Mild traumatic brain injury in U.S. soldiers returning from Iraq
New England Journal of Medicine
 , 
2008
, vol. 
358
 (pg. 
453
-
463
)
Howe
L. L. S.
Giving context to post-deployment post-concussive-like symptoms: Blast related potential mild traumatic brain injury and comorbidities
The Clinical Neuropsychologist
 , 
2009
, vol. 
23
 (pg. 
1315
-
1337
)
Iverson
G. L.
Outcome from mild traumatic brain injury
Current Opinion in Psychiatry
 , 
2005
, vol. 
18
 (pg. 
301
-
317
)
Iverson
G. L.
Lange
R. T.
Brooks
B. L.
Rennison
V. L. A.
‘Good old days’ bias following mild traumatic brain injury
The Clinical Neuropsychologist
 , 
2010
, vol. 
24
 (pg. 
17
-
37
)
Iverson
G. L.
Langlois
J. A.
McCrea
M. A.
Kelly
J. P.
Challenges associated with post-deployment screening for mild traumatic brain injury in military personnel
The Clinical Neuropsychologist
 , 
2010
, vol. 
23
 (pg. 
1299
-
1314
)
Kay
T.
Harrington
D. E.
Adams
R.
Anderson
T.
Berrol
S.
Cicerone
K.
, et al.  . 
Definition of mild traumatic brain injury
Journal of Head Trauma Rehabilitation
 , 
1993
, vol. 
8
 (pg. 
86
-
87
)
Larrabee
G. J.
Aggregation across multiple indicators improves the detection of malingering: Relationship to likelihood ratios
The Clinical Neuropsychologist
 , 
2008
, vol. 
22
 (pg. 
666
-
679
)
McCrea
M.
Standardized mental status testing on the sideline after sport-related concussion
Journal of Athletic Training
 , 
2001
, vol. 
36
 (pg. 
274
-
279
)
McCrea
M.
Iverson
G. L.
McAllister
T. W.
Hammeke
T. A.
Powell
M. R.
Barr
W. B.
, et al.  . 
An integrated review of recovery after mild traumatic brain injury (MTBI): Implications for clinical management
The Clinical Neuropsychologist
 , 
2009
, vol. 
23
 (pg. 
1368
-
1390
)
McCrea
M.
Pliskin
N.
Barth
J.
Cox
D.
Fink
J.
French
L.
, et al.  . 
Official position of the military TBI task force on the role of neuropsychology and rehabilitation psychology in the evaluation, management, and research of military veterans with traumatic brain injury
The Clinical Neuropsychologist
 , 
2008
, vol. 
22
 (pg. 
10
-
26
)
Mittenberg
W.
Patton
C.
Canyock
E.M.
Condit
D. C.
Base rates of malingering and symptom exaggeration
Journal of Clinical and Experimental Neuropsychology
 , 
2002
, vol. 
24
 (pg. 
1094
-
1102
)
Moore
E. L.
Terryberry-Spohr
L.
Hope
D. A.
Mild traumatic brain injury and anxiety sequelae: A review of the literature
Brain Injury
 , 
2006
, vol. 
20
 (pg. 
117
-
132
)
Nelson
N. W.
Boone
K. B.
Dueck
A.
Wagener
L.
Lu
P.
Grills
C.
Relationships between eight measures of suspect effort
The Clinical Neuropsychologist
 , 
2003
, vol. 
17
 (pg. 
263
-
272
)
Nicholson
K.
Pain, cognition and traumatic brain injury
NeuroRehabilitation
 , 
2000
, vol. 
14
 (pg. 
95
-
103
)
Paniak
C.
Reynolds
S.
Phillips
K.
Toller-Lobe
G.
Melnyk
A.
Nagy
J.
Patient complaints within 1 month of traumatic brain injury: A controlled study
Archives of Clinical Neuropsychology
 , 
2002
, vol. 
17
 (pg. 
319
-
334
)
Radanov
B. P.
Dvorak
J.
Valach
L.
Cognitive deficits in patients after soft tissue injury of the cervical spine
Spine
 , 
1992
, vol. 
17
 (pg. 
127
-
131
)
Rohling
M. L.
Demakis
G. J.
Bowden, Shores, & Mathias (2006): Failure to replicate or just failure to notice. Does effort still account for more variance in neuropsychological test scores than TBI severity?
The Clinical Neuropsychologist
 , 
2010
, vol. 
24
 (pg. 
119
-
136
)
Rosenfeld
B.
Sands
S. A.
van Gorp
W. G.
Have we forgotten the base rate problem? Methodological issues in the detection of distortion
Archives of Clinical Neuropsychology
 , 
2000
, vol. 
15
 (pg. 
349
-
359
)
Schneiderman
A. I.
Braver
E. R.
Kang
H. K.
Understanding sequelae of injury mechanisms and mild traumatic brain injury incurred during the conflicts in Iraq and Afghanistan: Persistent postconcussive symptoms and posttraumatic stress disorder
American Journal of Epidemiology
 , 
2008
, vol. 
167
 (pg. 
1446
-
1452
)
Schretlen
D.J.
Shapiro
A.M.
A quantitative review of the effects of traumatic brain injury on cognitive functioning
International Review of Psychiatry
 , 
2003
, vol. 
15
 (pg. 
341
-
349
)
Slick
D. J.
Hopp
G.
Strauss
E.
Spellacy
F. J.
Victoria Symptom Validity Test: Efficiency for detecting feigned memory impairment and relationship to neuropsychological tests and MMPI-2 validity scales
Journal of Clinical and Experimental Neuropsychology
 , 
1996
, vol. 
18
 (pg. 
911
-
922
)
Slick
D. J.
Sherman
E.
Iverson
G. L.
Diagnostic criteria for malingered neurocognitive dysfunction: Proposed standards for clinical practice and research
The Clinical Neuropsychologist
 , 
1999
, vol. 
13
 (pg. 
545
-
561
)
Terrio
H.
Brenner
L. A.
Ivins
B. J.
Cho
J. M.
Schwab
K.
Scally
K.
, et al.  . 
Traumatic brain injury screening: Preliminary findings in a U.S. Army Brigade combat team
Journal of Head Trauma Rehabilitation
 , 
2009
, vol. 
24
 (pg. 
14
-
23
)
Tombaugh
T. N.
Test of Memory Malingering
 , 
1996
Toronto, ON
Multi Health Systems
Tsanadis
J.
Montoya
E.
Hanks
R. A.
Millis
S. R.
Fichtenberg
N. L.
Axelrod
B. N.
Brain injury severity, litigation status, and self-report of postconcussive symptoms
The Clinical Neuropsychologist
 , 
2008
, vol. 
22
 (pg. 
1080
-
1092
)
Vanderploeg
R. D.
Belanger
H. G.
Morgan
J.
Sweet
J.
Multifactorial contributions to questionable effort and test performance within a military context
Neuropsychology of malingering casebook
 , 
2009
New York, NY
Taylor & Francis
Vasterling
J. J.
Brailey
K.
Constans
J. I.
Sutker
P. B.
Attention and memory dysfunction in posttraumatic stress disorder
Neuropsychology
 , 
1998
, vol. 
12
 (pg. 
125
-
133
)
Vasterling
J. J.
Duke
L. M.
Brailey
K.
Constans
J. I.
Allain
A. N.
Sutker
P. B.
Attention, learning, and memory performances and intellectual resources in Vietnam veterans: PTSD and no disorder comparisons
Neuropsychology
 , 
2002
, vol. 
16
 (pg. 
5
-
14
)
Vasterling
J. J.
Proctor
S. P.
Amoroso
P.
Kane
R.
Heeren
T.
White
R. F.
Neuropsychological outcomes of Army personnel following deployment to the Iraq War
Journal of the American Medical Association
 , 
2006
, vol. 
296
 (pg. 
519
-
529
)
Wechsler
D. A.
Wechsler Adult Intelligence Scale – III (WAIS-III)
 , 
1997
New York
The Psychological Corporation
Whitney
K. A.
Shepard
P. H.
Williams
A. L.
Davis
J. J.
Adams
K. M.
The Medical Symptom Validity Test in the evaluation of Operation Iraqi Freedom/Operation Enduring Freedom soldiers: A preliminary study
Archives of Clinical Neuropsychology
 , 
2009
, vol. 
24
 (pg. 
145
-
152
)