Abstract

Operation Enduring Freedom and Operation Iraqi Freedom combat veterans given definite diagnoses of mild Traumatic Brain Injury (TBI) during the Veteran Health Administration (VHA) Comprehensive TBI evaluation and reporting no post-deployment head injury were examined to assess (a) consistency of self-reported memory impairment and (b) symptom validity test (SVT) performance via a two-part study. Study 1 found that while 49 of 50 veterans reported moderate to very severe memory impairment during the VHA Comprehensive TBI evaluation, only 7 had reported any memory problem at the time of their Department of Defense (DOD) post-deployment health assessment. Study 2 found that of 38 veterans referred for neuropsychological evaluations following a positive VHA Comprehensive TBI evaluation, 68.4% failed the Word Memory Test, a forced choice memory recognition symptom validity task. Together, these studies raise questions concerning the use of veteran symptom self-report for TBI assessments and argue for the inclusion of SVTs and the expanded use of contemporaneous DOD records to improve the diagnostic accuracy of the VHA Comprehensive TBI evaluation.

Introduction

Traumatic Brain Injury (TBI) is regarded as a leading injury among military personnel serving in the Operation Enduring Freedom (OEF) and Operation Iraqi Freedom (OIF) combat theaters due in large part to improvised explosive devices. Among TBIs, the majority are considered concussions or mild TBIs (mTBIs; Defense and Veterans Brain Injury Center, 2012).

mTBI is particularly difficult to diagnose. The cognitive and physical complaints associated with a history of mTBI are common to other psychological and medical disorders (Benge, Pastorek, & Thornton, 2009). For example, mTBI symptoms can include etiologically non-specific symptoms such as anxiety, depression, fatigue, irritability, and sleep difficulties (Spencer, Drag, Walker, & Bieliauskas, 2010). Although mTBI symptoms are temporary for most people, some continue to report persistent cognitive problems in need of intervention (McCrea, 2008).

In an effort to respond to the needs of veterans serving in OEF/OIF, the Department of Defense (DOD) and the Veterans Health Administration (VHA) have instituted a system of screenings and evaluations based in large part on self-reported symptoms. The DOD Post-Deployment Health Assessment (PDHA) consists of both self-reported symptoms and a face-to-face post-deployment assessment interview with a credentialed healthcare provider. It is required for anyone deployed at least 30 days (DOD, 2002, 2003, 2006). Although the DOD PDHA is designed to provide comprehensive health surveillance for service members affected by deployment, it also assesses for more specific TBI type symptoms such as self-reported memory difficulties.

The VHA screens every OEF/OIF veteran entering the VHA system with a short TBI screen consisting of four sets of questions assessing for possible brain injury. Veterans scoring positive on the TBI screen are contacted to complete a more detailed VHA Comprehensive TBI evaluation (VHA, 2010a). This Comprehensive TBI evaluation includes a determination of the origin or etiology of the patient's injury based on veteran self-report, an assessment for neurobehavioral symptoms based on veteran self-report using the 22-item Neurobehavioral Symptom Inventory (NBI), a physical evaluation, and a follow-up treatment plan. Depending on the findings, additional consultations can be generated to include neuropsychology, physical therapy, audiology, and other services.

While reliance on self-reported symptoms allows for rapid administration, Cooper, Nelson, Armistead-Jehle, and Bowles (2011) caution that many self-report instruments lack any means of assessing response bias, leaving them vulnerable to manipulation by the responder. This is consistent with numerous studies that have raised concerns about the use of self-report to diagnose trauma-related injury (Cooper et al., 2011; Roemer, Litz, & Orsillo, 1997), including mTBI (Benge et al., 2009; Spencer et al., 2010; Xydakis, Robbins, & Grant, 2008), especially when obtained months, if not years, after the purported TBI event.

Research consistently finds that reports of trauma symptoms and events are inconsistent over time both in general population studies (Hepp et al., 2006) and in military samples (Frueh et al., 2005; Roca & Freeman, 2001; Southwick, Morgan, Nicolaou, & Charney, 1997). Carlson and colleagues (2011) note in their systematic review of the mTBI and Post Traumatic Stress Disorder (PTSD) literature that screening for mTBI based on retrospective self-report may be “invalid or inaccurate, in part, due to recall error” (p. 104) and may lead to “overestimation of TBI and PTSD” (p. 110).

Self-report of cognitive impairment has been shown to correlate poorly with performance on objective neuropsychological testing across multiple clinical populations, including mild-to-moderate head injury (Branca, Giordani, Lutz, & Saper, 1996). Within combat veteran populations, self-reported cognitive functioning is often as strongly associated with anxiety, depression, and PTSD as with objective neuropsychological measures of cognitive impairment (Axelrod & Milner, 2000; Belenger, Kretzmer, Vanderploeg, & French, 2010; Chamelian & Feinstein, 2006; Gass & Apple, 1997; Hoge et al., 2008; Spencer et al., 2010). Hoge (2008) has argued that the use of multiple TBI screens can result in the unintended iatrogenic effect of veterans falsely attributing normal or stress-related symptoms to TBI due to repeated exposure to TBI screenings and assessments.

Armistead-Jehle (2010) noted the lack of symptom validity measures on the DOD and VHA TBI assessment procedures. Studies consistently find that effort has a disproportionate impact on neuropsychological test performance, especially in cases of suspected TBI (see, e.g., Fox, 2011; Green, 2007; Green, Rohling, Lees-Haley, & Allen, 2001; Lange, Pancholi, Bhagwat, Anderson-Barnes, & French, 2012; Rohling, 2000; West, Curtis, Greve, & Bianchini, 2010). For example, Green and colleagues (2001) found effort accounted for more variance than severity of head injury on neuropsychological test performance in a sample of individuals seeking compensation. Various studies found that effort as measured by symptom validity tests (SVTs) accounted for half the variance in neuropsychological test performance in a sample of mild head injury patients (e.g., Constantinou, Bauer, Ashendorf, Fisher, & McCaffrey, 2005; Meyers, Volbrecht, Axelrod, & Reinsh-Boothby, 2011). In a study of OEF/OIF combat veterans with and without a history of concussion, Nelson and colleagues (2010) found no difference in neuropsychological test performance once poor effort and compensation issues were controlled for.

Because of the need to control for effort, the use of symptom validity testing in neuropsychology has gained increasing acceptance in recent years (Bush et al., 2005). The 2009 American Academy of Neuropsychology consensus conference (Heilbronner, Sweet, Morgan, Larrabee, & Millis, 2009) recently identified the use of effort tests as a hallmark of ethical practice stating “for a clinician to choose not to use effort tests and embedded validity indicators requires a solid justification” (p. 1105).

To date, neither the DOD PDHA nor the VHA TBI screens or Comprehensive TBI evaluations routinely employ symptom validity measures. Neither controls for inconsistent reporting of neuro-cognitive symptoms. Without controlling for symptom validity and self-report symptom inconsistency, the emphasis on symptom self-report on the DOD and VHA assessment instruments is suspect and may lead to inaccurate diagnoses.

A review of the literature failed to find any studies examining consistency of self-reported memory impairment from DOD PDHA to VHA Comprehensive TBI evaluation for OEF/OIF veterans. A study by Donnelly and colleagues (2011) examined the consistency of OEF/OIF veteran self-report on the VHA TBI screen over a two week period and noted that the least consistent items were complaints of balance problems and memory, with a 72% rate of consistency. They also found that these veterans were more likely to endorse more complaints of memory problems during the second screen than they reported during the first screen.

To date, a review of the literature found three studies examining symptom validity performance in OEF/OIF veterans (Armistead-Jehle, 2010; Nelson et al., 2010; Whitney, Shepherd, Williams, Davis, & Adams, 2009). Only Armistead-Jehle (2010) examined symptom test performance specifically in relation to the VHA Comprehensive TBI evaluation. Using the Medical Symptom Test (MST), he found a 58% SVT failure rate in 45 OEF/OIF combat veterans referred for a neuropsychological evaluation following a positive TBI diagnosis on the VHA Comprehensive TBI evaluation. Nelson and colleagues (2010) examined the symptom validity performance of 119 OEF/OIF veterans referred for neuropsychological evaluation to examine the impact of context on failure rate. The sample consisted of 44 individuals seen within a compensation and pension forensic context and 75 seen within a research context. The authors found a 29% SVT failure rate overall, but with significant differences depending on assessment context. Of those veterans assessed within a compensation and pension context, 59.1% showed symptom validity failure on one or more measures, whereas only 9.4% of the veterans seen within a research context showed symptom validity failure. Whitney and colleagues (2009) examined the MST performance of 23 OEF/OIF veterans reporting mTBI and referred for neuropsychological testing within a VA polytrauma network site. The sample consisted of nine individuals enrolled in active duty, and 14 who had been recently discharged. The authors found a 17% failure rate overall, but with differences depending on discharge status. Veterans recently discharged show a 0.0% symptom validity failure rate, whereas those enrolled in active duty showed a 44.4% failure rate.

The purposes of these studies were (a) to ascertain the consistency of self-reported memory impairment from DOD PDHA to VHA Comprehensive TBI evaluation and (b) to assess SVT performance near the time of the Comprehensive TBI evaluation in OEF/OIF combat veterans diagnosed positive for TBI during the VHA Comprehensive TBI evaluation.

These archival studies were approved by the Veteran Affairs New York Healthcare System's Institutional Review Board.

Study 1: Consistency of Self-Reported Memory Impairment

Methods

Subjects

Study 1 used a retrospective chart review of the VHA computerized patient record system to identify 50 OEF/OIF combat veterans consecutively seen by Physical Medicine and Rehabilitation, who had been given a definite diagnosis of TBI on the VHA Comprehensive TBI evaluation and who had their DOD PDHA available in the VHA computerized patient record system. Subjects were excluded from this study if, during the time of their VHA Comprehensive TBI evaluation, they reported any head injury or concussion following their deployment. Subjects assessed as part of compensation and pension evaluations were also excluded.

Measures

Self-reports of memory impairment at the end of deployment were obtained from the DOD PDHA. The DOD PDHA specifically asks veterans to report “forgetful or trouble remembering things” during the time of deployment and/or at the time of the PDHA. For purposes of this study, a report of yes to either was coded as a positive report of memory impairment at the time of deployment and a report of no was coded as a negative report of memory impairment.

Self-reports of total symptom impairment and of memory impairment at the time of the VHA Comprehensive TBI evaluation were measured by veteran response to the NBI. Developed by Cicerone and Kalmar (1995), the NBI is a self-report rating scale that addresses 22 symptoms across four symptom clusters, including affective symptoms such as anxiety, somatic symptoms such as nausea, sensory symptoms such as sensitivity to light, and cognitive symptoms such as forgetfulness. Veterans rated each symptom on a five-point scale that assesses both the degree of symptom presence and the level of functional impairment using the following key: The sum of ratings for all 22 items determined the total symptom impairment score. Ratings for “15n: Forgetfulness, can't remember things” determined the memory impairment score.

  • None 0: rarely if ever present; not a problem at all.

  • Mild 1: occasionally present, but it does not disrupt activities; I can usually continue what I'm doing; doesn't really concern me.

  • Moderate 2: often present, occasionally disrupts my activities; I can usually continue what I'm doing with some effort; I am somewhat concerned.

  • Severe 3: frequently present and disrupts activities; I can only do things that are fairly simple or take little effort; I feel like I need help.

  • Very Severe 4: almost always present and I have been unable to perform at work, school, or home due to this problem; I probably cannot function without help.

Results

Table 1 presents the demographic information and levels of self-reported cognitive impairment for this sample as reported at the time of the VHA Comprehensive TBI evaluation. The mean time from DOD PDHA to VHA Comprehensive TBI evaluation was approximately 4 years. The mean age was 32.2 years (SD = 7.9), with all subjects between the ages of 22 and 53. The vast majority of subjects were men (94%). Ethnic breakdown was as follows: 46% Hispanic, 34% Caucasian, 16% African American, and 4% other. All but one had at least high-school education, with 25 reporting college-level schooling.

Table 1.

Study 1: Consistency of self-report: demographic and clinical findings at the time of VHA Comprehensive TBI evaluation

 Mean (SDMin Max 
DOD PDHA to VHA Comprehensive TBI evaluation (years) 3.88 (1.81) 0.5 7.7 
Age (years) 32.2 (7.9) 22.4 53.9 
Gender (% men) 94.0   
Percent with service connection 70.0   
Percent of service connection 36.0 (33.9) 0.0 100.0 
Percent with service connected for PTSD 48.0   
Percent with service connected for TBI 10.0   
 Median (QaMin Max 
Reported VHA memory impairment 3.0b (0.5) 
Total reported VHA symptom impairment 38.5c (12.3) 11.0 85.0 
 Mean (SDMin Max 
DOD PDHA to VHA Comprehensive TBI evaluation (years) 3.88 (1.81) 0.5 7.7 
Age (years) 32.2 (7.9) 22.4 53.9 
Gender (% men) 94.0   
Percent with service connection 70.0   
Percent of service connection 36.0 (33.9) 0.0 100.0 
Percent with service connected for PTSD 48.0   
Percent with service connected for TBI 10.0   
 Median (QaMin Max 
Reported VHA memory impairment 3.0b (0.5) 
Total reported VHA symptom impairment 38.5c (12.3) 11.0 85.0 

Notes: DOD = Department of Defense; PDHA = Post-Deployment Health Assessment; VHA = Veteran Health Administration; TBI = Traumatic Brain Injury.

aSemi-interquartile range.

bMemory impairment possible range is 0–4, with higher scores indicating greater impairment.

cTotal symptom impairment possible range is 0–88, with higher scores indicating greater impairment.

The majority (70%) had a service connected rating. The Veteran Benefit Administration provides a service connected rating to veterans for conditions that were caused by or exacerbated by their military service. Approximately half (48%) of the subjects were service connected for PTSD and another 10% were service connected for a brain-related issue. The median total symptom impairment on the NBI was 38.5 (Q = 12.3), with a range from 11.0 to 85.0. All but one reported some level of memory impairment with consequent impairment in functioning, with 90% rating their level of forgetfulness as moderate or higher and 68% rating it as severe or very severe.

Table 2 presents the consistency of self-reported memory impairment from the time of the DOD PDHA to the time of VHA Comprehensive TBI evaluation. While 49 of 50 subjects reported memory impairment with consequent impairment in functioning at the time of the VHA Comprehensive TBI evaluation, only 7 reported problematic memory at the time of the DOD PDHA. This is the opposite pattern expected of mTBI sequelae where course of recovery is characterized by improvement rather than deterioration (e.g., seeMcCrea, 2008; VHA, 2010b). A pairwise comparison using a continuity-corrected McNemar test found this difference between reported memory impairment at the time of the VHA Comprehensive TBI evaluation and at the time of the DOD PDHA to be highly significant (p < .0001).

Table 2.

Study 1: Consistency of self-report: self-reported memory impairment at DOD PDHA and VHA Comprehensive TBI evaluation

At the time of VHA Comprehensive TBI evaluation At the time of DOD PDHA
 
No Yes 
None 
Mild 
Moderate 10 
Severe 18 
Very Severe 11 
At the time of VHA Comprehensive TBI evaluation At the time of DOD PDHA
 
No Yes 
None 
Mild 
Moderate 10 
Severe 18 
Very Severe 11 

Notes: DOD = Department of Defense; PDHA = Post-Deployment Health Assessment; VHA = Veteran Health Administration; TBI = Traumatic Brain Injury. The McNemar test with continuity correction p < .0001.

Study 2: Symptom Validity Performance near Time of the VHA Comprehensive TBI Evaluation

Methods

Subjects

Subjects were 38 OEF/OIF combat veterans consecutively referred to the author for neuropsychological assessment following a definite diagnosis of TBI on the VHA Comprehensive TBI evaluation and who were given a SVT as part of their follow-up neuropsychological evaluations. Subjects reporting post-deployment head injury, having a dementia profile as detailed below, and assessed as part of compensation and pension evaluations were excluded.

The mean age was 33.3 years (SD = 8.6), with all subjects between the ages of 21 and 53. The vast majority of subjects were men (89.5%). Ethnic breakdown was as follows: 50% Hispanic, 26.3% African American, and 23.7% Caucasian. All but one had at least high-school education, with the mean years schooling at 13.4 (SD = 2.0).

Materials

SVT performance was assessed using the Word Memory Test (WMT; Green, 2005). The WMT is a computerized SVT that assesses effort and verbal memory across multiple subtests. Numerous studies have demonstrated that this measure has a high sensitivity to the exaggeration of cognitive impairment (e.g., Green, 2005; Iverson, Green, & Gervais, 1999; Tan, Slick, Strauss, & Hultsch, 2002).

The WMT is a computer administered, paired associative verbal memory test that uses semantically related (e.g., pen, paper) word pairs. The word pairs are presented twice. Immediately after the second presentation, Immediate Recognition (IR) is tested via a force choice format in which a word from the list, called the target word, is presented with a word that was not in the list, called a foil word. The subject is asked to identify the word that was in the list. The foil words are less semantically related than the 20 target word pairs (e.g., pen, notebook). Delayed Recognition (DR) is tested similarly 30 min later with a different foil word. The DR trial is followed by a Multiple Choice task, Paired Associate task, and a Free Recall task. In addition to these subtests, Consistency scale (CNS) is calculated to assess recognition consistency between IR and DR trials. The IR, DR, and CNS subtests are considered tests of effort, since research shows that numerous adult control and clinical groups (e.g., mTBI, moderate TBI, severe TBI, PTSD, depression, fibromyalgia) as well as numerous children groups (e.g., mental retardation, fetal alcohol syndrome, etc.) can obtain a perfect or near perfect score if motivated to do so (Green, 2005). Pass and fail were scored consistent with the instructions in the test manual and precise criteria are not reported here to protect test security (Green, 2005). Since the literature shows that the WMT requires little effort or ability across clinical groups (with the exception of advanced dementia) to obtain a near perfect score, failure indicates that the examinee did not expend enough effort in the direction of capable performance, so that their symptom presentation is not a valid reflection of their capabilities (Green, 2005).

Procedures

All subjects were tested by the author or a pre-doctoral intern under the direct supervision of the author after subjects received a diagnosis of definite TBI during the VHA Comprehensive TBI evaluation. Subjects were administered a flexible neuropsychological battery with the WMT computer administered by the author among the first neuropsychological measures. Prior to all evaluations, the subjects gave written consent for the assessment, and at post-testing completed a feedback form, both of which included a check box attesting, respectively, that they would give or had given their “best and most honest effort.” All subjects responded yes to both pre- and post-assessment check boxes.

Since dementia could account for failing scores on the IR, DR, and/or CNS parts of the WMT (Green, 2005), dementia was ruled out if the following three conditions applied: (a) there was no indication of a dementing condition on the VHA Comprehensive TBI evaluation, (2) there was no evidence of a dementing condition during the neuropsychological assessment, and (3) all subjects were showing at least adequate functioning by working full time, attending college full time, or doing both. One veteran was excluded from the study after leaving the VA before his occupational and educational status could be determined.

Self-reports of total symptom impairment and of memory impairment at the time of the VHA Comprehensive TBI evaluation were derived as detailed above. Where available, self-report of memory impairment at the time of return from deployment as recorded on the DOD PDHA was also tallied, as described above. Data analyses were conducted with Statistical Package for the Social Sciences (SPSS) 14.0.

Results

Table 3 presents the descriptive statistics of subject performance on the WMT. Sixty eight point four percent reached failure threshold on IR, DR, and/or CNS, as per the manual (Green, 2005). Of the 26 subjects failing the WMT, all failed at least two of the three effort subtests (IR, DR, CNS) with 44.8% failing all the three. As seen in Table 4, there were no significant differences between pass/fail rates based for gender, education, percent with service connection, percent of service connection, percent with either service connection for PTSD or a brain-related condition, self-reported memory impairment, or total self-reported symptom impairment. There was no significant difference based on ethnicity—χ2 = 0.2 (2), p = .9. There was a significant difference (p < .01) based on age, with a larger number of older subjects failing the WMT.

Table 3.

Study 2: Symptom validity: WMT scores in 38 OEF/OIF veterans referred for neuropsychological testing after obtaining a definite diagnosis of mTBI on the VHA Comprehensive TBI evaluation

 Subtest Median (QaMean (SDRange 
Pass WMT (n = 12) IR 97.5 (2.5) 96.7 (3.7) 88–100 
DR 97.5 (1.3) 98.1 (1.9) 95–100 
CNS 96.3 (3.8) 95.6 (4.3) 88–100 
MC 95.0 (5.0) 91.3 (10.9) 65–100 
PA 90.0 (8.1) 88.3 (12.9) 60–100 
FR 53.8 (9.1) 57.1 (16.0) 33–90 
Fail WMT (n = 26) IR 73.8 (8.1) 73..4 (14.0) 43–100 
DR 70.0 (4.7) 67.1 (12.7) 40–85 
CNS 65.0 (4.7) 64.1 (8.9) 45–80 
MC 45.0 (10.0) 45.2 (17.7) 5–75 
PA 45.0 (11.9) 44.0 (18.3) 10–70 
FR 28.8 (4.7) 28.4 (10.4) 8–48 
 Subtest Median (QaMean (SDRange 
Pass WMT (n = 12) IR 97.5 (2.5) 96.7 (3.7) 88–100 
DR 97.5 (1.3) 98.1 (1.9) 95–100 
CNS 96.3 (3.8) 95.6 (4.3) 88–100 
MC 95.0 (5.0) 91.3 (10.9) 65–100 
PA 90.0 (8.1) 88.3 (12.9) 60–100 
FR 53.8 (9.1) 57.1 (16.0) 33–90 
Fail WMT (n = 26) IR 73.8 (8.1) 73..4 (14.0) 43–100 
DR 70.0 (4.7) 67.1 (12.7) 40–85 
CNS 65.0 (4.7) 64.1 (8.9) 45–80 
MC 45.0 (10.0) 45.2 (17.7) 5–75 
PA 45.0 (11.9) 44.0 (18.3) 10–70 
FR 28.8 (4.7) 28.4 (10.4) 8–48 

Notes: WMT = Word Memory Test; IR = Immediate Recognition; DR = Delayed Recognition; CNS = Consistency scale; MC = Multiple Choice; PA = Paired Associate; FR = Free Recall.

aSemi-interquartile range.

Table 4.

Study 2: Symptom validity: demographic and clinical comparisons as a function of WMT performance (means [SD])

 Pass WMT Fail WMT Test value (dfp-value 
Age (years) 28.5 (5.0) 35.5 (9.1) 3.0 (36)a <.01 
Gender (% men) 100.0 84.6 0.3b ns 
Education (years) 12.9 (1.2) 13.6 (2.3) 1.0 (36)a ns 
Percent with service connection 66.7 65.4 1.0b ns 
Percent of service connection 40.0 (39.1) 41.2 (38.0) 0.1(36)a ns 
Percent PTSD service connected 50.0 34.6 0.5b ns 
Percent TBI service connected 0.0 15.4 0.3b ns 
Reported memory impairment 2.7 (1.2) 2.9 (1.0) 0.7(36)a ns 
Reported symptom impairment 40.2 (13.7) 46.4 (21.1) 0.9 (36)a ns 
 Pass WMT Fail WMT Test value (dfp-value 
Age (years) 28.5 (5.0) 35.5 (9.1) 3.0 (36)a <.01 
Gender (% men) 100.0 84.6 0.3b ns 
Education (years) 12.9 (1.2) 13.6 (2.3) 1.0 (36)a ns 
Percent with service connection 66.7 65.4 1.0b ns 
Percent of service connection 40.0 (39.1) 41.2 (38.0) 0.1(36)a ns 
Percent PTSD service connected 50.0 34.6 0.5b ns 
Percent TBI service connected 0.0 15.4 0.3b ns 
Reported memory impairment 2.7 (1.2) 2.9 (1.0) 0.7(36)a ns 
Reported symptom impairment 40.2 (13.7) 46.4 (21.1) 0.9 (36)a ns 

Notes: WMT = Word Memory Test; TBI = Traumatic Brain Injury.

at-test.

bFisher's exact test.

Of the 38 subjects, 20 had their DOD PDHA available on the VHA's computerized patient record system. Of these 20, one reported no memory impairment at the time of the VHA Comprehensive TBI evaluation. Nineteen reported some level of memory impairment with consequent impairment in function, with three rating their level of memory impairment as moderate and 16 as severe or very severe. Of the 19 with self-reported moderate to very severe memory impairment, only two reported problematic memory at the time of the DOD PDHA. As noted above, this is the opposite pattern expected of mTBI sequelae. A pairwise comparison using a continuity-corrected McNemar test found this difference between reported memory impairment at the time of the VHA Comprehensive TBI evaluation and at the time of the DOD PDHA to be highly significant (p < .0001).

General Discussion

The main finding of the first study is the marked lack of consistency in self-reported memory problems from time of DOD PDHA to VHA Comprehensive TBI evaluation. The expected pattern with mTBI is for a quick return to premorbid levels of cognitive functioning weeks after injury for most, with a small minority reporting persistent symptoms. This study found OEF/OEF combat veterans reporting the opposite pattern. All but 1 of 50 subjects reported moderate to very severe levels of memory impairment with consequent impairment in functioning at the time of their VHA Comprehensive TBI evaluation, whereas only 7 of these 49, or <15%, reported problematic memory during their DOD PDHA. Since all subjects denied any head injury or concussion post-deployment, there was no known intervening head insult to account for this.

It is difficult to reconcile this by arguing that the veterans simply did not notice their memory problems until they returned home because the military structure somehow compensated for the memory deficits. First, of these 50 subjects, 20 were assessed by the author as part of the second study, so their qualitative self-reports were available for review. All reported steadily deteriorating memory, with intact memory prior to the purported etiological event, a mild decrease in memory functioning immediately after, and steady ongoing deterioration in memory with consequent impairment in functioning continuing to time of assessment. Second, the majority of veterans reporting memory impairment reported severe to very severe memory impairment with consequent serious impairment in functioning ranging from the ability to “only do things that are fairly simple” to “I have been unable to perform at work.” Given this level of functional impairment, it seems likely that others would have noticed problems such as an inability to perform at work even if the veterans did not. Third, a minority reported TBI events that were explicitly contradicted by their contemporaneous DOD medical records. Finally, Donnelly and colleagues (2011) examined veterans post-discharge and found an increase in reported memory impairment on a TBI screen administered twice over a two week period.

A more promising hypothesis for future research would be examining the impact of attribution error and the iatrogenic impact of TBI screens on self-reported memory impairment. Hoge (2008, 2010) has argued that symptoms related to PTSD and depression can be misattributed to TBI and that an unintended iatrogenic effect of mTBI screens is “widespread screening for concussion/mTBI may actually be causing warriors who are not brained injured to believe they are” (Hoge, 2010; p. 44). Veterans then falsely believing that they were members of the “TBI” diagnostic category would be more likely to show the symptoms associated with that category—the phenomenon of “diagnosis threat” (Suhr & Gunstad, 2005).

The second study is consistent with that of Armistead-Jehle (2010), who also studied OEF/OIF veterans referred for assessment following a positive VHA Comprehensive TBI evaluation. Both Armistead-Jehle (2010) and the current study found high rates of SVT failure in OEF/OIF veterans referred for neuropsychological assessment following a definite diagnosis of TBI on the VHA Comprehensive TBI evaluation. Using the Medical SVT, a briefer version of the WMT used in this study, Armistead-Jehle (2010) found a 58% rate of failure with no difference between pass/fail groups on age, gender, education, or ethnicity. This study employing the WMT found a higher rate of SVT failure at 68% in an OEF/OIF subject sample. Although there was no difference between pass/fail groups in the current study in terms of gender, education, or ethnicity, there was a significant difference in age, with older subjects failing the WMT at a significantly higher rate than younger subjects. Whether this difference in failure rates was due to differences in test sensitivity or sample variables is unclear and points to the need for further research. Since Nelson and colleagues (2010) and Whitney and colleagues (2009) employed subjects other than those referred after a definite diagnosis of TBI on the VHA Comprehensive TBI evaluation, they were excluded from the present discussion.

Although the first study found a lack of consistency between veteran self-report of memory functioning post-deployment to VHA Comprehensive TBI evaluation, the second study found an unexpected lack of consistency between veteran self-report of memory functioning and actual functioning. As part of the dementia exclusion criteria, only veterans showing at least adequate functioning defined as working and/or attending college full time were included. Of these 38 subjects, 12 described their memory impairment as “very severe,” endorsing the statement “I have been unable to perform at work, school or at home,” with another 12 rating their memory impairment as “severe,” and endorsing the statement “I can only do things that are fairly simple or take little effort … ”, at the time they were actually working full time and/or attending college “full time.” There was no evidence in the VHA Comprehensive TBI evaluations of these subjects that this inconsistency had been noted or taken into account in formulating the positive diagnosis of TBI.

Limitations of the present two studies include the fact that the sample sizes were small and that the samples were drawn from one geographic area in the Mid-Atlantic States. Although Armistead-Jehle (2010) found similar findings using a sample drawn from a very different geographic area, Pacific Island, a future research program with a large sample size from multiple sites would help determine the extent to which these findings are generalizable. The use of only one SVT in the second study is a limitation, given that effort may fluctuate over the course of an assessment (e.g., Boone, 2009). Further research employing multiple SVT measures would address this. Finally, this research is descriptive and does not address issues of causation. Failure on forced choice recognition SVTs reflects poor effort, but not intention. Although Armistead-Jehle (2010) detailed how issues of secondary gain are ever present in the VHA, providing incentive to appear more compromised than is the case; others have suggested that SVT failure can be due to other conditions, for example, factitious disorder or somatoform illness (Boone, 2007; Cooper et al., 2011).

The current study is consistent with the literature questioning the emphasis on self-report for TBI evaluations following distant etiological events (Hoge, 2008, Hoge, Goldberg, & Castro, 2009). In particular, this study questions the wisdom of assuming that one can expect accurate self-report of distant etiological events in a sample with self-reported marked memory impairment. The current studies are consistent with the available, though limited research in arguing for the greater use of contemporaneous records and symptom validity testing to increase the diagnostic accuracy of the VHA Comprehensive TBI evaluation (e.g., Armistead-Jehle, 2010; Ruff et al., 2009).

The cost of misdiagnosing a veteran with TBI is considerable, even when issues of monetary compensation are not involved. Taylor and colleagues (2012) examined the prevalence and healthcare cost of veterans diagnosed with TBI compared with those without a TBI diagnosis by examining all OEF/OIF veterans using VHA inpatient or outpatient care services in 2009. They found that the median annual cost per patient was nearly four times higher for TBI-diagnosed veterans when compared with those without TBI. The personal cost to our veterans is also considerable. For example, veterans misdiagnosed with TBI may develop negative expectations for individual performance, leading to lessened self-esteem and worsened performance (Suhr & Gunstad, 2005). Those with symptoms erroneously attributed to TBI rather than their true source, such as psychiatric illness or life stress, may go untreated.

Conflict of Interest

None declared.

Acknowledgements

This paper is the result of work supported in part with resources and the use of facilities at the Department of Veterans Affairs New York Harbor Healthcare System; as such it is in the public domain. Contents do not necessarily reflect the views of the Department of Veterans Affairs or U.S. Government.

References

Armistead-Jehle
P.
Symptom validity test performance in U.S. veterans referred for evaluation of mild TBI
Applied Neuropsychology
 , 
2010
, vol. 
17
 (pg. 
32
-
59
)
Axelrod
B.
Milner
I.
Gulf War illness research: separating the wheat from the chaff
The Clinical Psychologist
 , 
2000
, vol. 
14
 
3
(pg. 
344
-
348
)
Belenger
H.
Kretzmer
T.
Vanderploeg
R.
French
L.
Symptom complaints following combat-related traumatic brain injury: relationship to traumatic brain injury severity and posttraumatic stress disorder
Journal of the International Neuropsychological Society
 , 
2010
, vol. 
16
 (pg. 
194
-
199
)
Benge
J.
Pastorek
N.
Thornton
G.
Postconcussive symptoms in OEF-OIF veterans: Factor structure and impact of posttraumatic stress
Rehabilitation Psychology
 , 
2009
, vol. 
54
 
3
(pg. 
270
-
278
)
Boone
K. B.
Boone
K.
A reconsideration of the Slick et al. criteria (1999) for malingered neuro-cognitive deficit
Assessment of feigned cognitive impairment
 , 
2007
New York
Guilford Press
Boone
K. B.
The need for continuous and comprehensive sampling of effort/response bias during neuropsychological examinations
The Clinical Neuropsychologist
 , 
2009
, vol. 
23
 (pg. 
729
-
741
)
Branca
B.
Giordani
B.
Lutz
T.
Saper
J.
Self-report of cognition and objective test performance in posttraumatic headache
Headache
 , 
1996
, vol. 
36
 
5
(pg. 
300
-
306
)
Bush
S.
Ruff
R.
Troster
A.
Barth
J.
Koffler
S.
Pliskin
N.
, et al.  . 
NAN position paper: Symptom validity assessment: Practice issues and medical necessity
Archives of Clinical Neuropsychology
 , 
2005
, vol. 
20
 (pg. 
419
-
426
)
Carlson
K.
Kehle
S.
Meis
L.
Greer
N.
MacDonald
R.
, et al.  . 
Prevalence, assessment and treatment of mild traumatic brain injury and posttraumatic stress disorder: A systematic review of the evidence
Journal of Head Trauma
 , 
2011
, vol. 
26
 
2
(pg. 
103
-
115
)
Chamelian
L.
Feinstein
A.
The effect of major depression on subjective and objective cognitive deficits in mild to moderate traumatic brain injury
Journal of Neuropsychiatry and Clinical Neurosciences
 , 
2006
, vol. 
18
 (pg. 
33
-
38
)
Cicerone
K.
Kalmar
K.
Persistent postconcussion syndrome: The structure of subjective complaints after mild traumatic brain injury
Journal of Head Trauma
 , 
1995
, vol. 
10
 
3
(pg. 
1
-
17
)
Constantinou
M.
Bauer
L.
Ashendorf
L.
Fisher
J.
McCaffrey
J.
Is poor memory on recognition memory effort measures indicative of generalized poor performance on neuropsychological tests
Archives of Clinical Neuropsychology
 , 
2005
, vol. 
20
 (pg. 
191
-
198
)
Cooper
D.
Nelson
L.
Armistead-Jehle
P.
Bowles
A.
Utility of the Mild Brain Injury Symptoms Scale as a screening measure for symptom over-reporting in Operation Enduring Freedom/Operation Iraqi Freedom service members with post-concussive complaints
Archives of Clinical Neuropsychology
 , 
2011
, vol. 
26
 (pg. 
718
-
827
)
Defense and Veterans Brain Injury Center
2012
 
DoD Worldwide Numbers for TBI. Retrieved September 28, 2012, from http://www.dvbic.org/dod-worldwide-numbers-tbi
Department of Defense
Updated Procedures for Deployment Health Surveillance and Readiness
2002
 
Joint Chief of Staff Memorandum MCM-0006-02, 1 February 2002
Department of Defense
Enhanced Post Post-Deployment Health Assessments
2003
Undersecretary of Defense Memorandum for Secretaries of the Military Departments. Combatant Commanders and Director of the Joint Staff
 
, 22 April 2003
Department of Defense
Instruction: Deployment Health
2006
 
Number 6490.03, 11 August 2006
Donnelly
K.
Donnelly
J.
Dunnam
M.
Warner
G.
Kittleson
C.
Constance
J.
, et al.  . 
Reliability, sensitivity, and specificity of the VA traumatic brain injury screening tool
Journal of Head Trauma Rehabilitation
 , 
2011
, vol. 
26
 
6
(pg. 
439
-
453
)
Fox
D.
Symptom validity test failure indicates invalidity of neuropsychological tests
The Clinical Neuropsychologist
 , 
2011
, vol. 
25
 (pg. 
488
-
495
)
Frueh
B.
Elhai
J.
Grubaugh
A.
Monnier
J.
Kashdan
T.
, et al.  . 
Documented combat exposure of US veterans seeking treatment for combat-related post-traumatic stress disorder
British Journal of Psychiatry
 , 
2005
, vol. 
186
 (pg. 
467
-
472
)
Gass
C.
Apple
C.
Cognitive complaints in closed-head injury: relationship to memory test performance and emotional disturbance
Journal of Clinical Experimental Neuropsychology
 , 
1997
, vol. 
19
 
2
(pg. 
290
-
299
)
Green
P.
Green's Word Memory Test for Microsoft Windows: User's Manual
 , 
2005
Edmonton
Green's Publishing Company
Green
P.
The pervasive influence of effort on neuropsychological tests
Physical Medicine and Rehabilitation Clinics of North America
 , 
2007
, vol. 
18
 (pg. 
43
-
68
)
Green
P.
Rohling
M.
Lees-Haley
P.
Allen
L.
Effort has a greater impact on test scores than severe brain injury in compensation claimants
Brain Injury
 , 
2001
, vol. 
15
 
12
(pg. 
1045
-
1060
)
Heilbronner
R.
Sweet
J.
Morgan
J.
Larrabee
G.
Millis
S.
American Academy of Clinical Neuropsychology Consensus Conference Statement on the neuropsychological assessment of effort, response bias, and malingering
The Clinical Neuropsychologist
 , 
2009
, vol. 
23
 (pg. 
1093
-
1129
)
Hepp
U.
Gamma
A.
Milos
G.
Eich
D.
Ajdacic-Gross
V.
Rossler
W.
Inconsistency in reporting potentially traumatic events
British Journal of Psychiatry
 , 
2006
, vol. 
188
 (pg. 
278
-
283
)
Hoge
C.
Correspondence: Mild traumatic brain injury in U.S. soldiers returning from Iraq
New England Journal of Medicine
 , 
2008
, vol. 
358
 (pg. 
2177
-
2179
)
Hoge
C.
Once a warrior, always a warrior
 , 
2010
Connecticut
Globe Pequot Press
Hoge
C.
Goldberg
H.
Castro
C.
Care of war veterans with mild traumatic brain injury—flawed perspectives
New England Journal of Medicine
 , 
2009
, vol. 
360
 (pg. 
1588
-
1591
)
Hoge
C.
McGurk
D.
Thomas
J.
Cox
A.
Engel
C.
Castro
C.
Mild traumatic brain injury in U.S. soldiers returning from Iraq
New England Journal of Medicine
 , 
2008
, vol. 
358
 (pg. 
453
-
463
)
Iverson
G.
Green
P.
Gervais
R.
Using the Word Memory Test to detect biased responding in head injury litigation
Journal of Cognitive Rehabilitation
 , 
1999
, vol. 
March/April
 (pg. 
2
-
6
)
Lange
R.
Pancholi
S.
Bhagwat
A.
Anderson-Barnes
V.
French
L.
Influence of poor effort on neuropsychological test performance in U.S. military personnel following mild traumatic brain injury
Journal of Clinical and Experimental Neuropsychology
 , 
2012
, vol. 
34
 
5
(pg. 
453
-
466
)
McCrea
M.
Mild Traumatic Brain Injury and Postconcussion Syndrome: The new evidence base for diagnosis and treatment
 , 
2008
New York
Oxford University Press
Meyers
J.
Volbrecht
M.
Axelrod
B.
Reinsh-Boothby
L.
Embedded symptom validity tests and overall neuropsychological test performance
Archives of Clinical Neuropsychology
 , 
2011
, vol. 
26
 (pg. 
8
-
15
)
Nelson
N.
Hoezle
J.
McGuire
K.
Ferrier-Aurbach
A.
Charlesworth
M.
Sponheim
S.
Evaluation context impacts neuropsychological performance of OEF/OIF veterans with reported combat-related concussion
Archives of Clinical Neuropsychology
 , 
2010
, vol. 
25
 (pg. 
713
-
723
)
Roca
V.
Freeman
T.
Complaints of impaired memory in veterans with PTSD
American Journal of Psychiatry
 , 
2001
, vol. 
158
 
10
(pg. 
1738
-
1739
)
Roemer
L.
Litz
B.
Orsillo
S.
Consistency of traumatic memories
American Journal of Psychiatry
 , 
1997
, vol. 
154
 pg. 
1628
 
Rohling
M.
Effect size of impairment associated with symptom exaggeration versus definite traumatic brain injury
Archives of Clinical Neuropsychology
 , 
2000
, vol. 
15
 
8
pg. 
843
 
Ruff
R.
Iverson
G.
Barth
J.
Bush
S.
Broshek
D.
NAN Policy and Planning Committee
Recommendations for diagnosing a mild traumatic brain injury: A National Academy of Neuropsychology education paper
Archive of Clinical Neuropsychology
 , 
2009
, vol. 
24
 
1
(pg. 
3
-
10
)
Southwick
S.
Morgan
C.
Nicolaou
A.
Charney
D.
Consistency of memory for combat-related traumatic events in veterans of Operation Desert Storm
American Journal of Psychiatry
 , 
1997
, vol. 
154
 (pg. 
173
-
177
)
Spencer
R.
Drag
L.
Walker
S.
Bieliauskas
L.
Self-reported cognitive symptoms following mild traumatic brain injury are poorly associated with neuropsychological performance in OIF/OEF veterans
Journal of Rehabilitation Research and Development
 , 
2010
, vol. 
47
 
6
(pg. 
521
-
530
)
Suhr
J.
Gunstad
J.
Further explorations of the effect of “diagnosis threat” on cognitive performance in individuals with mild brain injury
Journal of the International Neuropsychological Society
 , 
2005
, vol. 
11
 (pg. 
23
-
29
)
Tan
J.
Slick
D.
Strauss
E.
Hultsch
D.
Malingering strategies on symptom validity tests
The Clinical Psychologist
 , 
2002
, vol. 
16
 
4
(pg. 
495
-
505
)
Taylor
B.
Hagel
E.
Carlson
K.
Cifu
D.
Cutting
A.
Bidelspach
D.
Prevalence and costs of co-occurring Traumatic Brain Injury with and without psychiatric disturbance and pain among Afghanistan and Iraq War veteran VA users
Medical Care
 , 
2012
, vol. 
50
 
4
(pg. 
342
-
346
)
Veterans Health Administration
Screening and evaluation of possible traumatic brain injury in Operation Enduring Freedom (OEF) and Operation Iraqi Freedom (OIF) veterans
 , 
2010
Washington, DC
Department of Veterans Affairs
 
VHA Directive 2010
Traumatic Brain Injury
 , 
2010
Washington, DC
Department of Veterans Affairs
 
West
L.
Curtis
K.
Greve
K.
Bianchini
K.
Memory in traumatic brain injury: The effects of severity and effort on the Wechsler Memory Scale III
Journal of Neuropsychology
 , 
2010
, vol. 
5
 (pg. 
114
-
125
)
Whitney
A.
Shepherd
P.
Williams
A.
Davis
J.
Adams
K.
The Medical Symptom Validity Test in the evaluation of Operation Iraqi Freedom/Operation Enduring Freedom soldiers: A preliminary study
Archives of Clinical Neuropsychology
 , 
2009
, vol. 
24
 (pg. 
145
-
152
)
Xydakis
M.
Robbins
A.
Grant
G.
Mild traumatic brain injury in U.S. soldiers returning from Iraq
New England Journal of Medicine
 , 
2008
, vol. 
358
 
20
pg. 
2177