Abstract

Evaluation of post-deployment conditions such as post-concussive syndrome (PCS) and posttraumatic stress disorder (PTSD) frequently relies upon brief, self-report checklists which are face valid and highly susceptible to potential symptom validity issues such as symptom exaggeration. We investigated the psychometric prope1rties of a 5-item measure of symptom exaggeration (mild brain injury atypical symptoms [mBIAS] scale) embedded in commonly used PCS and PTSD screening instruments in a sample of 403 patients seen in a brain injury clinic at a large military medical center. Exploratory factor analysis, examining measures of posttraumatic stress, post-concussive symptoms, and symptom over-reporting revealed a 6-factor model with the mBIAS scale items representing a unique factor. Analysis of psychometric properties demonstrated that a score of 8 on the mBIAS was optimal for the detection of symptom over-reporting (sensitivity = 0.94, specificity = 0.92) and appears to be the most favorable cut score for interpretive use. The findings provide a strong initial support for the use of the mBIAS in post-deployment populations.

Introduction

Among Operation Enduring Freedom (OEF) and Operation Iraqi Freedom (OIF) combat veterans, traumatic brain injury (TBI) and post-concussive syndrome (PCS) have been referred to as signature injuries and have been the subject of considerable clinical, research, and administrative attention (Warden, 2006). The majority of brain injuries sustained in these combat operations (>85%) meet criteria for the mild variant of this condition (DVBIC, 2008). Although the clinical literature strongly suggests a rapid and complete return to baseline functioning in the vast majority of those who experience a mild TBI (mTBI), there are indications that a small minority of individuals (<5% in unselected samples) develop lingering symptoms as a result of the neurological insult (McCrea, 2008). Estimating prevalence rates of these conditions in post-deployment samples is often times an onerous endeavor, complicated in large part by the non-specificity of symptoms, absence of biomarkers, and the self-report method of symptom identification generally employed as means of assessment. Taking into account these difficulties, it has been reported that between 15% and 20% of deployed service members endorse experiences associated with mTBI (Hoge et al., 2008; Tanielian & Jaycox, 2008) beyond the normal period of expected recovery.

One of the greatest challenges facing the Military and Veterans Affairs Healthcare Systems (VHS) at this time is the identification and treatment of post-deployment conditions, including mTBI. Assessment of mTBI is common in post-deployment clinical environments throughout the Department of Defense (DoD) and VHS. Clinicians often rely upon self-report data and face-valid measures of post-concussive, adaptive, and psychological functioning in these settings to assist with differential diagnosis and treatment planning, as well as with more disability-based evaluations like medical evaluation boards and compensation and pension (C&P) assessments. While self-report data have the advantage of rapid and patient-directed administration, reliance upon such measures can be problematic as many of these instruments are devoid of mechanisms to assess response bias and has the potential to be manipulated by the respondent for a number of reasons.

Evaluation of response bias in veterans and military personnel with self-reported histories of mTBI has been the subject of three recent publications. Armistead-Jehle (2010) found that 58% of veterans referred for clinical evaluation of mTBI failed a well-validated measure of symptom validity (i.e., Medical Symptom Validity Test [MSVT]; Green, 2004), suggesting suboptimal effort and potential exaggeration of cognitive symptoms in the sample. Nelson and colleagues (2010), in their sample of veterans presenting for C&P evaluations, revealed a 59% failure rate on the Victoria Symptom Validity Test (VSVT; Slick, Hopp, Strauss, & Spellacy, 1996). Whitney, Shepard, Williams, Davis, and Adams (2009) administered the MSVT to a sample of 23 combat veterans reporting a history of mTBI referred for neuropsychological testing within a Veterans Affairs Medical Center. The sample was composed of 9 individuals still enrolled in active duty service and 14 who had been recently discharged. These authors observed a 17% failure rate on the MSVT, with all of those failing among the participants still on active duty service.

Methods also exist to detect the exaggeration of self-reported cognitive symptoms in one actuarial measure of psychological functioning and personality. Specifically, the Minnesota Multiphasic Personality Inventory-Second Edition (MMPI-2; Butcher et al., 2001) and MMPI-2-Restructured Form (MMPI-2-RF; Ben-Porath & Tellegen, 2008) Response Bias Scale (RBS) has demonstrated utility in identifying individuals who have failed neurocognitive effort measures (Gervais, Ben-Porath, Wygant, & Green, 2007; Gervais, Ben-Porath, Wygant, & Sellborn, 2010). Whitney, Davis, Shepard, and Herman (2008) examined the MMPI-2 RBS in a sample of 45 veterans involved in outpatient neuropsychological evaluations conducted for clinical purposes. The authors compared RBS scores in veterans who passed and failed a commonly administered measure of neurocognitive symptom validity, the Test of Memory Malingering (TOMM; Tombaugh, 1996). Although the RBS raw score was significantly higher in those that failed the TOMM, both groups showed elevated RBS scores suggestive of exaggerated memory complaints. Apart from this study, the authors are aware of no other published studies examining the relationship between self-report symptom complaints and validated measures of response bias, such as the RBS, within an OEF/OIF military or veteran sample.

Although over-reporting of possible PCS symptoms secondary to a conscious attempt to manage symptom presentation can potentially explain the response bias noted in the above studies, somatoform-spectrum conditions are also of significant concern in the evaluation of exaggerated PCS self-report. Individuals with somatoform-spectrum conditions can overwhelm medical resources through high healthcare utilization and referrals to specialty care. Furthermore, somatoform-spectrum conditions are associated with significant limitations in patient functioning. Somatization disorders have been hypothesized for individuals with idiopathic cognitive complaints (Delis & Wetter, 2007), particularly in situations in which there is no clear external incentive or secondary gain. Additionally, somatoform-spectrum disorders have been either demonstrated or postulated as potential etiologies in a number of post-deployment conditions including trench warfare (Jones, Fear, & Wessley, 2007) and Gulf War Illness (Engel, 2006; Labbate, Cardena, Dimitreva, Roy, & Engel, 1998). Several promising studies have examined the utility of screening measures for psychopathology (including somatoform-spectrum conditions) in primary care settings in which medically unexplained symptoms account for more than one third of clinic visits. Interian and colleagues (2004) demonstrated that individuals who endorsed an elevated number of pseudoneurological symptoms were more likely to present with more severe levels of anxiety, depression, somatic complaints, and physical dysfunction. Given the non-specific nature of post-concussive complaints, a similar screening measure would appear to hold promise in the detection of individuals with somatoform-spectrum disorders presenting for care in TBI clinics.

Another challenge facing healthcare providers tasked with identifying mTBI and PCS is discrimination between PCS and posttraumatic stress disorder (PTSD); two conditions with a notable degree of overlap between symptom clusters (Benge, Pastorek, & Thornton, 2009; Caplan et al., 2010; Cooper et al., 2011), especially in post-deployment settings. Compounding this challenge is the potential for examinee exaggeration of PTSD symptoms in select veteran and military samples. Freeman, Powell, and Kimbrell (2008) examined psychological symptom exaggeration in a Vietnam veteran sample and demonstrated that 53% of the participants evidenced clear symptom exaggeration on the Structured Interview of Reported Symptoms (Rogers, Bagby, & Dickens, 1992). Smith and Frueh (1996) identified 37% of veterans diagnosed with PTSD as symptom exaggerators based on MMPI-2 F-K index scores. They further found that, relative to participants identified as non-exaggerators, exaggerators scored higher on self-report instruments of various psychological symptoms. Other authors have noted symptom exaggeration in combat veterans evaluated for PTSD and corresponding elevations on self-report instruments of psychological functioning (e.g., DeViva & Bloem, 2003; Frueh, Hamner, Cahill, Gold, & Hamlin, 2000; Gold & Frueh, 1999).

Although the literature on neuropsychological effort testing in military and veteran personnel continues to evolve, the initial indications suggest that assessment of response bias in these samples is a factor in need of consideration. The literature further suggests that measures are available to detect cognitive impression management in individuals presenting for evaluation of mTBI; however, these measures either require administration by a professional or paraprofessional (i.e., MSVT, VSVT, TOMM) or are too lengthy to be employed as a screening instrument (i.e., MMPI-2, MMPI-2-RF). As such, there appears to be a need for a rapidly administered self-report screening instrument of symptom exaggeration for mTBI. Given the overlap between the symptoms of PCS and PTSD briefly described above, any measure designed to detect exaggeration of mTBI sequelae would be improved by demonstrating divergence from PTSD symptoms checklists.

The aim of the current study was to investigate the psychometric properties of a short self-report screening measure of symptom exaggeration for PCS. We hypothesized that the items of the mild brain injury atypical symptom (mBIAS) scale would represent a unique factor when analyzed with other measures of neurological symptoms and PTSD. It was further hypothesized that endorsement of mBIAS items would correlate positively with elevated scores on validated self-report measures of neurological symptoms and PTSD.

Methods

Participants

Subjects were consecutive referrals to a brain injury clinic at a large military medical center in San Antonio (Brooke Army Medical Center; BAMC). The sample was primarily composed of active duty service members (including activated reservists and members of the National Guard). This archival study was approved by the BAMC Institutional Review Board. As part of standard operating procedures, all individuals referred to the clinic completed self-report symptom questionnaires on a computer kiosk prior to their initial encounter with a medical provider. Only subjects completing all three self-report questionnaires (as described below) were included in the final sample. From an initial archival set of 443 subjects, 40 subjects were excluded from analysis for incomplete data on one or more measures of interest. The final resulting sample consisted of 403 subjects.

Subjects referred to the clinic can be broadly divided into two patient categories: (a) Service Members with documented or suspected mTBI and (b) neurorehabilitation patients. For the purpose of this study, these groups were further divided into the following four categories for analysis: (a) no TBI, n = 73; (b) mTBI, n = 268; (c) moderate/severe TBI and penetrating BI, n = 39; and (d) neurologic patients referred for a variety of rehabilitation needs, n = 23. All the individuals in the no TBI, mTBI, and moderate/severe TBI and penetrating BI groups were on active duty. The neurologic patient group included 23 patients with the following diagnoses: tumor (n = 7), stroke (n = 6), electrical (n = 3), subarachnoid hemorrhage (n = 2), anoxia (n = 2), encephalitis (n = 1), Parkinson's disease (n = 1), and fronto-temporal dementia (n = 1). The neurologic group included one retiree and one military dependent. Although the focus of the study was to examine the utility of the mBIAS scale on individuals with mTBI, neurologic and moderate/severe TBI patients were included in these analyses for the purposes of comparing endorsement rates on the mBIAS in Service Members with mTBI to a sample of individuals with known neurologic signs/symptoms. The purpose of this inclusion was to ensure that the items on the mBIAS were not commonly endorsed by individuals with severe diffuse impairments (e.g., severe TBI) or focal neurologic lesions (e.g., stroke), minimizing the potential for false-positive errors.

Diagnosis of TBI was made through semi-structured clinical interview and record review by a physiatrist, nurse practitioner, or physician assistant. Consistent with the American Congress of Rehabilitation Medicine criteria (ACRM, 1993), mTBI (also referred to as concussion) was operationally defined as one or more of the following: loss of consciousness (LOC) <30 min; loss of memory for events immediately before (retrograde amnesia) or after the injury event (posttraumatic amnesia; PTA < 24 h); any alteration in mental state at the time of the injury (dazed, disoriented, confused); presence of focal neurological deficits; or a Glasgow Coma Scale (GCS) score ≥ 13. Moderate TBI was defined as GCS score between 9 and 12, LOC >30 min but <24 h, and/or duration of PTA > 24 h but <7 days. Consistent with the current DoD guidance (Casscells, 2007), individuals with positive neuroimaging findings, who otherwise met criteria for mTBI, were classified as moderate TBI. Severe TBI was defined as GCS score ≤ 8, duration of PTA > 7 days, and/or LOC > 24 h. GCS scores, when present, were determined by a review of available medical records and represent the lowest post-resuscitation GCS score obtained by trauma personnel. LOC and alteration of consciousness were determined by semi-structured clinical interview and were based on retrospective self-report. Since LOC and GCS scores could not be obtained for every subject, these variables were used solely in diagnostic determination and were not analyzed separately. Demographic and injury data were also collected as part of the archival design of the current study.

Measures

Symptoms of posttraumatic stress

The Posttraumatic Checklist-Military version (PCL-M; Blanchard, Jones-Alexander, Buckley, & Forneris, 1996) is a self-rated interval-level rating scale used to screen for PTSD. The PCL-M requires the identification of a specific traumatic event or occurrence from which symptoms are thought to be triggered, designated the “reference trauma.” The PCL-M consists of 17 items, each designed to capture one of three distinct clusters of symptoms representing the B, C, or D diagnostic criteria described for PTSD in the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR), revised fourth (American Psychiatric Association, 2000) edition. These three clusters are labeled re-experiencing (“B” items, 1–5), avoidance or numbing (“C” items, 6–12), and hyper-arousal (“D” items, 13–17). The frequency of occurrence of each symptom for the past month is marked using a 1 (not at all) to 5 (extremely) Likert-scale scoring. Scores are derived by summing the weighted frequencies for all items marked and can range from 17 to 85. The PCL-M has been validated for use in samples of military Veterans, with a total score of ≥60 demonstrating the most diagnostically useful cut score in indicating the presence of PTSD (Keen, Kutter, Niles, & Krinsley, 2008).

Post-concussive symptoms

The Neurobehavioral Symptom Inventory (NSI; Ciccerone & Kalmar, 1995) is a 22-item self-report inventory of common post-concussive sequelae. Subjects were instructed to rate the presence of each symptom within the past 2 weeks on a 5-item Likert scale ranging from 0 (None) to 4 (Very Severe). The original cluster analysis of patients with mTBI revealed a 4-factor solution, consisting of an affective cluster, cognitive cluster, somatic cluster, and sensory cluster (Ciccerone & Kalmar, 1995), although it should be noted that this may not be representative of typical mTBI populations since all of the individuals in their sample were involved in litigation. A more recent factor analysis of the NSI in an OEF/OIF Veteran population yielded a difficult to interpret factor structure (Benge et al., 2009); however, symptoms of posttraumatic stress accounted for a significant proportion of the variance in individual post-concussive symptoms in this factor analysis. After controlling for posttraumatic stress, a 4-factor solution (as above) emerged. Similarly, a recent factor analysis of the NSI in three large military OEF/OIF samples revealed a 3-factor solution, composed of cognitive, affective, and somatic/sensory clusters (Caplan et al., 2010).

Symptom over-reporting

The mBIAS scale consisted of five rationally developed items (Fig. 1), which were selected from a larger pool of items felt to be uncommonly endorsed symptoms following mTBI. The five items were selected by a panel consisting of a board-certified neurologist, board-certified physiatrist, and senior neuropsychologist each with extensive experience working with mTBI. To reduce the transparency of these items, they were written in a style consistent with the NSI and PCL-M, using a Likert-scale which closely resembled the structure of the other self-report measures. Since subjects were administered the items on a computer kiosk which did not indicate the name of the scale (e.g., PCL-M), the mBIAS items were interspersed within the content of the PCL-M and NSI during the computer administration to further minimize the likelihood of detection.

Fig. 1.

Individual mBIAS items.

Fig. 1.

Individual mBIAS items.

Study Design

The final sample for analysis consisted of data from 403 patients. Data were analyzed using PASW for Windows, Version 18 (statistical package originally known as SPSS; IBM Corp., 2010). Patients were compared on demographic variables including age, gender, and rank, as well as the number of deployments and time post-injury. Time post-injury was calculated as the difference between the date of evaluation (i.e., clinical interview to establish the diagnosis) and the date of injury in months. Our statistical approach included both individual item analysis and symptom cluster analysis to explore our hypothesis that positive responses on the mBIAS would be associated with elevated scores on other self-report symptom measures and may represent instances of over-reporting of symptoms.

Statistical Analysis

Descriptives

Descriptive analyses were performed for demographic variables, and means and standard deviations (or percentage/frequencies for categorical variables) were obtained by patient/case type. Categorization of cases was as follows: (a) no TBI, these patients were determined not to have sustained a TBI; (b) mTBI, these patients were determined to have sustained a TBI of mild severity; (c) moderate/severe TBI and penetrating BI; and (d) neurologic diagnoses, these patients were seen in the clinic for other neurological concerns (e.g., status post-tumor resection). Demographic and descriptive information obtained in this study included age, gender, rank, number of deployments, and time since injury.

For the purpose of analysis, military rank was categorized into three levels, consistent with traditional military classification, enlisted (including E-1 through E-4), non-commissioned officers (E-5 through E-9), and officers (O-1 though O-6). The following service branches of the United States Military were represented in the full initial sample (N = 443) with 390 (90.3%) Army, 13 (2.9%) Marine Corps, 24 (5.4%) Air Force, and 5 (1.2%) Navy. Eleven cases were of unknown rank and branch of service.

Age was evaluated using a one-way analysis of variance (ANOVA), and all other categorical variables were evaluated using χ2. Due to variability across case types in the demographic and descriptive variables described above, these were used in an ANCOVA as covariates to determine whether the primary clinical measures (NSI, PCL-M, and mBIAS) varied across injury severity/type independently of these factors. Partial eta-squared (η2) statistics were calculated to provide a measure of effect size for each of the covariates.

Factor analysis of instrument items

Exploratory factor analysis was performed on all items from the mBIAS, NSI, and PCL-M, in order to determine the degree to which the mBIAS may be reflecting variance associated with symptom complexes related to either PCS (NSI) or PTSD (PCL-M). Factors were extracted using principal components analysis. We hypothesized that items from each instrument would share more variance with other items from that instrument than those from other instruments. Due to a priori expectations of some degree of intercorrelation between these measures, oblimin rotation with Kaiser normalization was used to generate the factor structure allowing for oblique solutions. All Eigen values were >1.

mBIAS sensitivity, specificity, and positive and negative predictive values

Given that endorsement of items on the mBIAS represented symptoms that were considered “extremely unlikely” to be caused by any non-penetrating or non-focal TBI, item endorsement was used as a marker for symptom over-reporting (State = True). The clinical criterion for symptom over-reporting was selected as the endorsement of symptoms on both the NSI and the PCL-M sufficient to score 1.5 SD above the mean for the sample on both of the scales. To identify the most favorable mBIAS cutoff for the indication of “True Over-Reporting,” we conducted a series of calculations examining the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for a series of mBIAS cut scores. Analyses were also conducted using incrementally increasing hypothetical base rates of symptom exaggeration for the purposes of estimating differing PPVs and NPVs in differing clinically relevant environments with differing rates of individuals who may be over-reporting symptoms.

Although it is acknowledged that such high scores (+1.5 SD on both measures) could reflect severe neurological and psychiatric symptoms in the respondent, we felt that responses in this range across both measures also warranted some degree of scrutiny as they could reflect a response bias. This reasoning is in line with the projected use of the mBIAS as a screening instrument designed to alert the provider to the potential of possible over-reporting which would in turn result in a specialty referral where further assessment would be completed. The selected cut point allowed for a modest sample size which met the criterion (N = 18) and preserved a very low probability of this level of symptom experience. (i.e., since a Z-score of 1.5 has a low probability [p = .067] and multiplying this by 2, yields an extremely low probability [p = .0045], this level of endorsement was believed to serve as a reasonable proxy for “over-endorsement.”)

Results

Demographic and Descriptive Analyses

Significant differences (p < .05) across case type were found for age, gender, rank, number of deployments, and time since injury (Table 1). Due to this variability, these variables were used as covariates in generating estimated marginal means for the case type groups on each of the clinical measures of interest, these values are listed in Table 1. Individuals with missing demographic or injury-related data were excluded from the ANCOVA.

Table 1.

Sample characteristics by case type

Demographics No injury (N = 73) (mean [SD] or N [%])a Mild (N = 268) (mean [SD] or N [%])a Moderate/severe (N = 39) (mean [SD] or N [%])a Neurologic (N = 23) (mean [SD] or N [%])a F or χ2 (df) p-value 
Age (years) 30 (9) 32 (9) 28 (7.5) 37 (11) 6.4 (3) .001 
Mena 63 (86) 248 (93) 37 (95) 18 (78) 7.9 (3) .049 
# Deployments 1.6 (1.0) 2.1 (1.5) 1.3 (1.3) 2.1 (2.0) 2.6 (3) .052 
Ranka     27 (15) .028 
 Enlisted 40 (58) 114 (43) 22 (56) 9 (43)  
 NCO 22 (32) 125 (47) 14 (36) 7 (33)  
 Officer 7 (10) 25 (10) 3 (8) 5 (24)  
Time since injury (years)     17.6 (6) .007 
 1 43 (86) 148 (62) 28 (72) 17 (85)  
 1–3 6 (12) 53 (22) 9 (23) 3 (15)  
 >3 1 (2) 36 (15) 2 (5) 0 (0)  
Clinical measuresb No injury (N = 18) (M [SE]) Mild (N = 179) (M [SE]) Moderate/severe (N = 25) (M [SE]) Neurologic (N = 11) (M [SE]) F (p-value), part. η2 
PCL-M 31 (4) 42 (1) 37 (3) 35 (5) 3.2 (.025), 0.041 
NSI 17 (4) 32 (1) 34 (3) 31 (5) 4.9 (.003), 0.061 
mBIAS 5 (0.4) 6 (0.1) 6 (0.3) 6 (0.5) 0.55 (.649), 0.007 
Demographics No injury (N = 73) (mean [SD] or N [%])a Mild (N = 268) (mean [SD] or N [%])a Moderate/severe (N = 39) (mean [SD] or N [%])a Neurologic (N = 23) (mean [SD] or N [%])a F or χ2 (df) p-value 
Age (years) 30 (9) 32 (9) 28 (7.5) 37 (11) 6.4 (3) .001 
Mena 63 (86) 248 (93) 37 (95) 18 (78) 7.9 (3) .049 
# Deployments 1.6 (1.0) 2.1 (1.5) 1.3 (1.3) 2.1 (2.0) 2.6 (3) .052 
Ranka     27 (15) .028 
 Enlisted 40 (58) 114 (43) 22 (56) 9 (43)  
 NCO 22 (32) 125 (47) 14 (36) 7 (33)  
 Officer 7 (10) 25 (10) 3 (8) 5 (24)  
Time since injury (years)     17.6 (6) .007 
 1 43 (86) 148 (62) 28 (72) 17 (85)  
 1–3 6 (12) 53 (22) 9 (23) 3 (15)  
 >3 1 (2) 36 (15) 2 (5) 0 (0)  
Clinical measuresb No injury (N = 18) (M [SE]) Mild (N = 179) (M [SE]) Moderate/severe (N = 25) (M [SE]) Neurologic (N = 11) (M [SE]) F (p-value), part. η2 
PCL-M 31 (4) 42 (1) 37 (3) 35 (5) 3.2 (.025), 0.041 
NSI 17 (4) 32 (1) 34 (3) 31 (5) 4.9 (.003), 0.061 
mBIAS 5 (0.4) 6 (0.1) 6 (0.3) 6 (0.5) 0.55 (.649), 0.007 

Notes:M = estimated marginal means based on the effects of demographic covariates; SE = standard error of the estimate; NCO = non-commissioned officers; PCL-M = Posttraumatic Checklist-Military version; NSI = Neurobehavioral Symptom Inventory; mBIAS = mild brain injury atypical symptoms.

aPercentages are column percents.

bF-statistic and associated p-values determined for univariate effect of Injury Type, controlling for demographic covariates by ANCOVA, cases deleted based on missing data analysis by analysis.

ANCOVA to evaluate the PCL-M across case types indicated that number of deployments (F = 18.2, df= 1, p < .05, partial η2= 0.08) and time since injury (F = 36.3, df= 1, p < .05, partial η2= 0.139) were significant covariates for the total score.

This pattern was repeated with the NSI, where ANCOVA showed number of deployments (F = 6.6, df= 1, p < .05, partial η2= 0.03), time since injury (F = 4.9, df= 1, p < .05, partial η2= 0.09), and age (F = 3.9, df= 1, p < .05, partial η2= 0.017) were significant covariates across injury/case types.

ANCOVA showed that significant covariates of mBIAS item endorsement were age (F = 15.1, df= 1, p < .05, partial η2= 0.06) and number of deployments (F = 8.1, df= 1, p < .05, partial η2= 0.04), with a trend toward significance for rank (F = 3.6, df= 1, p = .06, partial η2= 0.02).

Factor Analysis

The 6-factor solution accounted for 65% of the total variance. Factor 1 accounted for 45% of the variance and consisted of items from both the NSI and the PCL-M. Factor 2 accounted for 7% of the total variance and consisted of items from the NSI. Factor 3 accounted for 4% of the total variance and consisted only of items from the PCL-M. All mBIAS items fell on the fourth factor which accounted for only 3% of the total variance. This factor had only one item from the NSI (item 11) with a factor score over 0.40, and only item 1 from the mBIAS loaded on any other factors with an absolute score >0.40 (Table 2). Factor 5 accounted for 3% of the variance and consisted of negative loadings of items from both the NSI and the PCL-M. Factor 6 also accounted for 3% of the total variance with 3 of the four items coming from the NSI and the remaining item from the PCL-M. Factor labels were rationally developed based on item content and are reported in Tables 2 and 3.

Table 2.

Factor loadings for PCL-M, NSI, and mBIAS items

Source Item content Factors
 
Mood Vestibular Trauma mBIAS Cog. Sleep 
PCL-M-10 Distant and Cut off 0.709      
NSI-20 Depressed or Sad 0.663      
PCL-M-9 Loss of Interest 0.659      
PCL-M-11 Emotionally Numb 0.626      
NSI-22 Easily Frustrated 0.604      
PCL-M-14 Feeling Irritable 0.548      
NSI-21 Irritability 0.539      
PCL-M-12 Foreshortened Future 0.530      
NSI-19 Anxious or Tense 0.475      
NSI-17 Fatigue 0.468      
NSI-12 Appetite Change 0.421      
NSI-2 Loss of Balance  0.823     
NSI-1 Dizziness  0.802     
NSI-5 Nausea  0.636     
NSI-6 Vision Problems  0.604     
NSI-3 Clumsiness  0.602     
NSI-10 Numbness or Tingling — — — — — — 
PCL-M-1 Intrusive Thoughts   0.928    
PCL-M-3 Re-Experiencing   0.896    
PCL-M-4 Triggered Distress   0.882    
PCL-M-2 Nightmares   0.880    
PCL-M-5 Triggered Arousal   0.866    
PCL-M-6 Avoidance (Cognitive)   0.853    
PCL-M-7 Avoidance (Behavioral)   0.788    
PCL-M-8 Poor Trauma Memory   0.643    
PCL-M-17 Exaggerated Startle   0.611    
PCL-M-16 Hypervigilance   0.598    
mBIAS-3 Temporary Mutism    0.677   
mBIAS-2 Monochromatic Vision    0.587   
mBIAS-1 Temporary Deafness    0.463 −0.437  
mBIAS-4 Bilateral Arm Numbness    0.449   
NSI-11 Smell/Taste Changes    0.438   
mBIAS-5 Difficulty Swallowing    0.433   
NSI-16 Executive Dysfunction     −0.596  
NSI-14 Forgetfulness     −0.591  
NSI-8 Hearing Difficulty     −0.583  
NSI-13 Poor Concentration     −0.567  
PCL-M-15 Difficulty Concentrating 0.426    −0.494  
NSI-15 Decision Making 0.415    −0.465  
NSI-18 Sleep Difficulties      0.687 
PCL-M-13 Sleep Difficulties      0.673 
NSI-7 Light Sensitivity      0.489 
NSI-4 Headaches      0.464 
NSI-9 Noise Sensitivity — — — — — — 
Source Item content Factors
 
Mood Vestibular Trauma mBIAS Cog. Sleep 
PCL-M-10 Distant and Cut off 0.709      
NSI-20 Depressed or Sad 0.663      
PCL-M-9 Loss of Interest 0.659      
PCL-M-11 Emotionally Numb 0.626      
NSI-22 Easily Frustrated 0.604      
PCL-M-14 Feeling Irritable 0.548      
NSI-21 Irritability 0.539      
PCL-M-12 Foreshortened Future 0.530      
NSI-19 Anxious or Tense 0.475      
NSI-17 Fatigue 0.468      
NSI-12 Appetite Change 0.421      
NSI-2 Loss of Balance  0.823     
NSI-1 Dizziness  0.802     
NSI-5 Nausea  0.636     
NSI-6 Vision Problems  0.604     
NSI-3 Clumsiness  0.602     
NSI-10 Numbness or Tingling — — — — — — 
PCL-M-1 Intrusive Thoughts   0.928    
PCL-M-3 Re-Experiencing   0.896    
PCL-M-4 Triggered Distress   0.882    
PCL-M-2 Nightmares   0.880    
PCL-M-5 Triggered Arousal   0.866    
PCL-M-6 Avoidance (Cognitive)   0.853    
PCL-M-7 Avoidance (Behavioral)   0.788    
PCL-M-8 Poor Trauma Memory   0.643    
PCL-M-17 Exaggerated Startle   0.611    
PCL-M-16 Hypervigilance   0.598    
mBIAS-3 Temporary Mutism    0.677   
mBIAS-2 Monochromatic Vision    0.587   
mBIAS-1 Temporary Deafness    0.463 −0.437  
mBIAS-4 Bilateral Arm Numbness    0.449   
NSI-11 Smell/Taste Changes    0.438   
mBIAS-5 Difficulty Swallowing    0.433   
NSI-16 Executive Dysfunction     −0.596  
NSI-14 Forgetfulness     −0.591  
NSI-8 Hearing Difficulty     −0.583  
NSI-13 Poor Concentration     −0.567  
PCL-M-15 Difficulty Concentrating 0.426    −0.494  
NSI-15 Decision Making 0.415    −0.465  
NSI-18 Sleep Difficulties      0.687 
PCL-M-13 Sleep Difficulties      0.673 
NSI-7 Light Sensitivity      0.489 
NSI-4 Headaches      0.464 
NSI-9 Noise Sensitivity — — — — — — 

Notes: PCL-M = Posttraumatic Checklist-Military version; NSI = Neurobehavioral Symptom Inventory; mBIAS = mild brain injury atypical symptoms.

Table 3.

Component correlation matrix

Factor Mood Vestibular Trauma mBIAS Cog. Sleep 
Mood 1.000      
Vestibular .363 1.000     
Trauma .535 .380 1.000    
mBIAS .201 .294 .283 1.000   
Cog. −.289 −.385 −.373 −.226 1.000  
Sleep .339 .368 .412 .188 −.245 1.000 
Factor Mood Vestibular Trauma mBIAS Cog. Sleep 
Mood 1.000      
Vestibular .363 1.000     
Trauma .535 .380 1.000    
mBIAS .201 .294 .283 1.000   
Cog. −.289 −.385 −.373 −.226 1.000  
Sleep .339 .368 .412 .188 −.245 1.000 

Note: mBIAS = mild brain injury atypical symptoms.

The correlation matrix between the six extracted factors shows high correlation between the factors associated with the traditional symptom measures, with the lowest overall correlations to the mBIAS factor (Table 3).

Sensitivity, Specificity, PPV and NPV

As is shown in Table 4, the cutoff with the best balance of sensitivity and specificity, PPV, and NPV across a range of likely base rates of over-reporting appears to be an mBIAS score of ≥8. There appear to be appreciable losses in performance of the tool with cut scores below or above this point, particularly among populations with lower base rates of over-reporting.

Table 4.

Sensitivity, specificity, and predictive values of cut points for mBIAS scores in detecting >1.5 SD scores on both Neurobehavioral Symptom Inventory and Posttraumatic Checklist-Military version

mBIAS cutoff score Obtained values
 
PPV: Hypothetical base rates
 
NPV: Hypothetical base rates
 
Sensitivity Specificity 0.20 0.30 0.40 0.50 0.20 0.30 0.40 0.50 
≥7 0.94 0.84 0.59 0.72 0.80 0.85 0.98 0.97 0.95 0.93 
≥8 0.94 0.92 0.75 0.83 0.89 0.92 0.98 0.97 0.96 0.94 
≥9 0.50 0.97 0.81 0.88 0.92 0.94 0.89 0.82 0.74 0.66 
mBIAS cutoff score Obtained values
 
PPV: Hypothetical base rates
 
NPV: Hypothetical base rates
 
Sensitivity Specificity 0.20 0.30 0.40 0.50 0.20 0.30 0.40 0.50 
≥7 0.94 0.84 0.59 0.72 0.80 0.85 0.98 0.97 0.95 0.93 
≥8 0.94 0.92 0.75 0.83 0.89 0.92 0.98 0.97 0.96 0.94 
≥9 0.50 0.97 0.81 0.88 0.92 0.94 0.89 0.82 0.74 0.66 

Notes: mBIAS = mild brain injury atypical symptoms; PPV = positive predictive value; NPV = negative predictive value.

Frequency of mBIAS Scores ≥ 8 in Each Patient Category

To determine the likelihood of elevation of mBIAS scores in individuals with more severe brain injuries, or those with known neurological disorders, we calculated the number of individuals in each diagnostic category who scored ≥8 on the mBIAS items (Table 5). It should be noted that the resulting Pearson χ2 value (3.06, df = 3, p = .38) indicates no significant difference across groups.

Table 5.

Frequency of mBIAS scores ≥8 by diagnostic group

 No Injury (N = 73; N [%]) Mild (N = 268; N [%]) Moderate/severe (N = 39; N [%]) Neurologic (N = 23; N [%]) χ2 (df), p-value 
mBIAS ≥ 8 5 (7*) 36 (13*) 4 (10*) 4 (17*) 3.06 (3), .383 
 No Injury (N = 73; N [%]) Mild (N = 268; N [%]) Moderate/severe (N = 39; N [%]) Neurologic (N = 23; N [%]) χ2 (df), p-value 
mBIAS ≥ 8 5 (7*) 36 (13*) 4 (10*) 4 (17*) 3.06 (3), .383 

Notes: PCL-M = Posttraumatic Checklist-Military version; mBIAS = mild brain injury atypical symptoms.

*Percentages are Column Percents.

Discussion

Given the existing clinical climates in the Departments of Defense and Veterans Affairs, there has been a substantial increase in the need to evaluate service members and veterans who present with reported histories of mTBI and possible PCS. Previous literature has suggested that factors other than neurological complications, such as conscious attempts at impression management and somatoform-type disorders, can result in potentially exaggerated symptom presentations. While extant measures have been designed to detect such presentations (e.g., MSVT, MMPI-2-RF), they are typically employed in the contexts of neuropsychological and/or psychological specialty assessments and are thus not well suited for screening evaluations. The current study attempted to initially validate a recently constructed, rapidly administered, self-report measure designed to detect possible exaggeration of post-concussive symptoms.

The reported factor analytic data suggest that the mBIAS items measure a unique dimension in symptom reporting among Service Members presenting to a Neurology/TBI specialty clinic. mBIAS items composed a factor independent of other items from the included self-report measures of neurological and PTSD symptoms. As such, mBIAS items do not appear to be tapping the same symptom complexes associated with PTSD or mTBI sequelae. It then follows that mBIAS responses should not be confounded by the presence or severity of PTSD or PCS symptoms. Such an inventory is thought to have utility in the current DoD and Veterans Health Administration clinical environments where the possibility for exaggeration of symptoms following a history of mTBI has been suggested (i.e., Armistead-Jehle, 2010; Nelson et al., 2010).

The current findings also suggest that while any positive response across mBIAS items may be indicative of symptom over-reporting, an appropriate level for increased attention appears to be any combination of responses resulting in a score of ≥8. The current findings are in line with the extant literature that suggest over-reporting of pseudo-neurological symptoms is correlated with elevated scores on self-report measures of psychological and somatic symptoms (e.g., Interian et al., 2004). mBIAS responses of this magnitude should then be interpreted as a directive to assess for potential conscious over-reporting of symptoms (and possible malingering) or the potential for somataform-spectrum disorders. Within the primary care or non-behavioral health specialty setting this could manifest in a referral to neuropsychology and/or psychology for further evaluation. Additionally, consideration should be given altering the approach of the clinical interview in individuals with elevated mBIAS scores including greater reliance on open-ended versus close-ended questions (e.g., “What is the symptom that is of most concern to you right now?”) to reduce the likelihood of iatrogenic symptoms.

Given that this investigation is a preliminary examination of the mBIAS scale, there are several limitations of the current study. First, the sample composed of active duty service members and while the authors can find no compelling reason to limit the generalizability of the current results, future research with veterans will be necessary to ensure external validity. Second, since the current study relied upon an archival clinical sample, several methodological approaches could not be employed which may have helped to better explain the results. Specifically, we did not employ a simulator design, which would allow for a higher degree of confidence that over-reporting is actually a function of conscious attempts to exaggerate symptoms. Although the mBIAS items were specifically chosen for their perceived low probability following mTBI, the possibility remains that these five items may be endorsed in patients with co-occurring physical or sensory conditions other than mTBI (e.g., acoustic trauma). Future investigations should examine the base rates of mBIAS item endorsement in post-deployment controls and specific medical populations where there is increased potential for false-positive errors. Our preliminary analysis using a sample of brain-injured patients at the severe end of the spectrum and with neurologic patients did not yield a high rate of false-positive errors in these known neurologic groups. Additionally, although several patients in the sample carried working clinical diagnoses of somatoform-spectrum disorders, no empirically rigorous approach (e.g., MMPI-2; Structured Clinical Interview for DSM-IV) was used to validate this diagnosis. Findings from a known sample of individuals with somatoform-spectrum conditions would bolster the internal validity of the measure. Future research studies comparing the mBIAS to cognitive SVTs (i.e., Word Memory Test [Green, 2003]; MSVT, etc.) and the RBS of the MMPI-2 and/or MMPI-2-RF will also be of importance in the further validation of this measure.

Given the study limitations described above, the current data cannot be used to determine precisely why an individual demonstrates elevated scores on the mBIAS. Elevated scores could potentially reflect conscious efforts on behalf of the patient to exaggerate post-concussive symptoms or a somatoform-type response suggesting unconscious conversion of psychological symptoms to a physical (in this case neurological) presentation. However, the aim of the current study was to explore the psychometric properties of a short, self-report measure to detect potential over-reporting of post-concussive symptomatology for use with other face-valid screening instruments in post-deployment settings. Factor analysis and basic psychometric analysis provide a strong support for its use in the post-deployment military population. The current data also suggest that the mBIAS may hold promise in other forensic settings which require differentiation of post-concussive, posttraumatic, and non-credible symptoms.

Conflict of Interest

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper. The opinions and assertions contained in this article are solely the authors' private ones and are not to be construed as official or reflecting the views of the United States Army or the DoD. This manuscript was prepared by United States Government employees and therefore cannot be copyrighted and may be copied without restriction.

Acknowledgements

The authors wish to thank Hannah Castellaw, senior programmer/analyst, for her expertise in developing the TBI clinical database which has greatly enhanced our ability to conduct clinically relevant research to better serve our patients. We also wish to thank Glenn Larrabee, PhD, for his thoughtful critique of the manuscript and suggestions for future research, and Rael Lange, PhD, for his assistance with statistical issues.

References

American College of Rehabilitation Medicine.
Definition of mild traumatic brain injury
Journal of Head Trauma Rehabilitation
 , 
1993
, vol. 
8
 (pg. 
86
-
87
)
American Psychiatric Association.
Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition
 , 
2000
Washington, DC
Author
 
Text Revision
Armistead-Jehle
P.
Symptom validity test performance in U.S. veterans referred for evaluation of mild TBI
Applied Neuropsychology
 , 
2010
, vol. 
17
 (pg. 
52
-
59
)
Benge
J. F.
Pastorek
N. J.
Thornton
G. M.
Postconcussive symptoms in OEF/OIF veterans: Factor structure and impact of posttraumatic stress
Rehabilitation Psychology
 , 
2009
, vol. 
54
 
3
(pg. 
270
-
278
)
Ben-Porath
Y. S.
Tellegen
A.
MMPI-2-RF (Minnesota Multiphasic Personality Inventory-2-Restructured Form). Manual for administration, scoring, and interpretation
 , 
2008
Minneapolis
The University of Minnesota Press
Blanchard
E.
Jones-Alexander
J.
Buckley
T.
Forneris
C. A.
Psychometric properties of the PTSD checklist
Behavioral Research and Therapy
 , 
1996
, vol. 
34
 (pg. 
669
-
673
)
Butcher
J. N.
Graham
J. R.
Ben-Porath
Y. S.
Tellegen
A.
Fahlstrom
W. G.
Kaemmer
B.
MMPI-2: Manual for administration and scoring
 , 
2001
Rev. ed.
Minneapolis, MN
The University of Minnesota Press
Caplan
L. J.
Ivins
B.
Poole
J. H.
Vanderplueg
R. D.
Jaffee
M. S.
Schwab
K.
The structure of postconcussive symptoms in 3 US military samples
Journal of Head Trauma Rehabilitation
 , 
2010
, vol. 
25
 
6
(pg. 
447
-
458
)
Casscells
S. Ward
Consolidation of Traumatic Brain Injury Initiatives in the Department of Defense
2007
 
(Health Affairs Memorandum, dated March 23, 2007)
Ciccerone
K.
Kalmar
K.
Persistent postconcussion syndrome: The structure of subjective complaints after mild traumatic brain injury
Journal of Head Trauma Rehabilitation
 , 
1995
, vol. 
10
 
3
(pg. 
1
-
17
)
Cooper
D. B.
Kennedy
J. E.
Cullen
M. A.
Critchfield
E.
Amador
R. R.
Bowles
A. O.
Association between combat stress and post-concussive symptom reporting in OEF/OIF service members with mild traumatic brain injuries
Brain Injury
 , 
2011
, vol. 
25
 
1
(pg. 
1
-
7
)
Defense and Veterans Brain Injury Center.
Operation Iraqi Freedom (OIF)/Operation Enduring Freedom (OEF) Fact Sheet: November
2008
Delis
D. C.
Wetter
S. R.
Cogniform disorder and cogniform condition: Proposed diagnoses for excessive cognitive symptoms
Archives of Clinical Neuropsychology
 , 
2007
, vol. 
24
 (pg. 
145
-
152
)
Deviva
J. C.
Bloem
W. D.
Symptom exaggeration and compensation seeking among combat veterans with posttraumatic stress disorder
Journal of Traumatic Stress
 , 
2003
, vol. 
16
 (pg. 
503
-
507
)
Engel
C. C.
“Mirror, mirror…” whose explanation is the fairest? Evolving notions of somatization and idiopathic physical symptoms
CNS Spectrums
 , 
2006
, vol. 
11
 
3
(pg. 
187
-
188
)
Freeman
T.
Powell
M.
Kimbrell
T.
Measuring symptom exaggeration in veterans with chronic post traumatic stress disorder
Psychiatry Research
 , 
2008
, vol. 
158
 (pg. 
374
-
380
)
Frueh
B. C.
Hamner
M. B.
Cahill
S. P.
Gold
P. B.
Hamlin
K.
Apparent symptom over reporting among combat veterans evaluated for PTSD
Clinical Psychology Review
 , 
2000
, vol. 
20
 (pg. 
853
-
885
)
Gervais
R. O.
Ben-Porath
Y. S.
Wygant
D. B.
Green
P.
Development and validation of a response bias scale (RBS) for the MMPI-2. Assessment
2007
, vol. 
14
 (pg. 
196
-
208
)
Gervais
R. O.
Ben-Porath
Y. S.
Wygant
D. B.
Sellborn
M.
Incremental validity of the MMPI-2-RF over-reporting scales and RBS in assessing the veracity or memory complaints
Archives of Clinical Neuropsychology
 , 
2010
, vol. 
25
 (pg. 
274
-
284
)
Gold
P. B.
Frueh
B. C.
Compensation-seeking and extreme exaggeration of psychopathology among combat veterans evaluated for posttraumatic stress disorder
Journal of Nervous and Mental Disease
 , 
1999
, vol. 
187
 (pg. 
680
-
684
)
Green
P.
Green's Word Memory Test for Windows: User's manual
 , 
2003
Edmonton, Canada
Green's Publishing
Green
P.
Green's Medical Symptom Validity Test (MSVT) for Microsoft Windows: User's manual
 , 
2004
Edmonton, Canada
Green's Publishing
Hoge
C. W.
McGurk
D.
Thomas
J. I.
Cox
A. L.
Engel
C. C.
Castro
C. A.
Mild traumatic brain injury in U.S. soldiers returning from Iraq
The New England Journal of Medicine
 , 
2008
, vol. 
358
 
5
(pg. 
453
-
463
)
Interian
A.
Gara
M. A.
Diaz-Martinez
A. M.
Warman
M. J.
Escobar
J. I.
Allen
L. A.
, et al.  . 
The value of pseudoneurological symptoms for assessing psychopathology in primary care
Psychosomatic Medicine
 , 
2004
, vol. 
66
 (pg. 
141
-
146
)
Jones
E.
Fear
N. T.
Wessley
S.
Shell shock and mild traumatic brain injury: A historical review
American Journal of Psychiatry
 , 
2007
, vol. 
164
 
11
(pg. 
1641
-
1645
)
Keen
S. M.
Kutter
C. J.
Niles
B. L.
Krinsley
K. E.
Psychometric properties of PTSD checklist in sample of male veterans
Journal of Rehabilitation Research and Development
 , 
2008
, vol. 
45
 
3
(pg. 
465
-
474
)
Labbate
L. A.
Cardena
E.
Dimitreva
J.
Roy
M.
Engel
C. C.
Psychiatric syndromes in Persian Gulf War veterans: An association of handling dead bodies with somatoform disorders
Psychotherapy and Psychosomatics
 , 
1998
, vol. 
67
 
4
(pg. 
275
-
279
)
McCrea
M.
Mild Traumatic Brain Injury and Postconcussion Syndrome: The new evidence base for diagnosis and treatment
 , 
2008
New York
Oxford University Press
Nelson
N. W.
Hoelzle
J. B.
McGuire
K. A.
Ferrier-Auerbach
A. G.
Choarlesworht
M. J.
Sponheim
S. R.
Evaluation context impacts neuropsychological performance of OEF/OIF veterans with reported combat-related concussion
Archives of Clinical Neuropsychology
 , 
2010
, vol. 
25
 (pg. 
713
-
723
)
PASW (Version 18.0, 2010) [Computer software]
 
Chicago, IL
SPSS, IBM Corp
Rogers
R.
Bagby
R.
Dickens
S.
Structured Interview of Reported Symptoms professional manual
 , 
1992
Florida
Psychological Assessment Resources, Inc
Slick
D. J.
Hopp
G.
Strauss
E.
Spellacy
F. J.
Victoria Symptom Validity Test: Efficiency for detection feigned memory impairment and relationship to neuropsychological tests and MMPI-2 validity scales
Journal of Clinical and Experimental Neuropsychology
 , 
1996
, vol. 
18
 (pg. 
911
-
922
)
Smith
D. W.
Frueh
B. C.
Compensation seeking, comorbidity, and apparent exaggeration of PTSD symptom among Vietnam combat veterans
Psychological Assessment
 , 
1996
, vol. 
8
 (pg. 
3
-
6
)
Tanielian
T.
Jaycox
L. H.
Invisible wounds of war: Psychological and cognitive injuries, their consequences, and services to assist recovery
 , 
2008
Santa Monica, CA
RAND Corp
Tombaugh
T. N.
The Test of Memory Malingering
 , 
1996
Los Angeles
Western Psychology Corporation
Warden
D.
Military TBI during the Iraq and Afghanistan Wars
Journal of Head Trauma Rehabilitation
 , 
2006
, vol. 
21
 (pg. 
398
-
402
)
Whitney
K. A.
Davis
J. J.
Shepard
P. H.
Herman
S. M.
Utility of the Response Bias Scale (RBS) and other MMPI-2 validity scales in predicting TOMM performance
Archives of Clinical Neuropsychology
 , 
2008
, vol. 
23
 (pg. 
777
-
786
)
Whitney
K. A.
Shepard
P. H.
Williams
A. L.
Davis
J. L.
Adams
K. M.
The medical symptom validity test in the evaluation of operation Iraqi freedom/operation enduring freedom soldiers: A preliminary study
Archives of Clinical Neuropsychology
 , 
2009
, vol. 
24
 (pg. 
145
-
152
)