The current study sought to characterize the psychological architecture of individuals who put forth inadequate effort. The Minnesota Multiphasic Personality Inventory, 2nd Edition-Restructured Form was used to identify dimensions of psychological functioning in a mixed outpatient sample of U.S. Veterans referred for neuropsychological evaluation as part of their clinical care. After accounting for external financial incentive and symptom overreporting, results showed that the inadequate effort group (n = 23, mean age = 42.48) scored higher than the adequate effort group (n = 29, mean age = 44.31) on neurologic complaints (NUC) and lower on behavioral/externalizing dysfunction (BXD), antisocial behaviors (RC4), and disconstraint (DISC-r). Lower scores on BXD, RC4, and DISC-r could indicate higher behavioral constraint—a psychological characteristic that has been linked to the pursuit of high-value future rewards. Alternatively, lower scores on these scales could have reflected a self-presentation strategy aimed at minimizing externalizing and RC4 in order to appear more psychological healthy. Implications of each of these interpretations are discussed.

Introduction

The validity of neuropsychological test data hinges on the extent to which patients exert adequate effort on cognitive tests. The detection of inadequate effort, therefore, is crucial to the accurate interpretation of neuropsychological profiles. Although research aimed at enhancing inadequate effort detection after it has occurred is vital, little research has been devoted to better understanding of why inadequate effort is put forth by certain individuals and not others. In this context, external incentive is undoubtedly important, particularly when inadequate effort represents intentional exaggeration of cognitive dysfunction (i.e., malingering). However, not all compensation-seeking individuals put forth inadequate effort. Rates of invalid test data across different settings wherein external incentive is a factor tend to fall between 30% and 50% (Larrabee, 2003; Larrabee, Mills, & Meyers, 2009; Mittenberg, Patton, Canyock, & Condit, 2002), indicating that a large portion of neuropsychological evaluations performed in a secondary gain context generate valid test data (∼50%–70%). This suggests that external incentive alone may not be sufficient to explain inadequate effort; other intrinsic factors likely interact with external incentive to increase the probability of inadequate effort. Identifying such factors was the primary objective of the present study.

Performance Validity Tests

Substantial research has been devoted to developing and refining cognitive tests designed to detect inadequate effort. These are referred to as performance validity tests (PVTs). Two commonly used PVTs are the Test of Memory Malingering (TOMM; Tombaugh, 1996) and the Word Memory Test (WMT; Green, 2003). The TOMM is a two-alternative forced-choice test of picture recognition. Test administration consists of up to three trials (two learning trials and an optional delayed retention trial) and standard cutoff scores are used to determine effort level. Numerous studies support the TOMM's utility in detecting inadequate effort across medical and forensic settings in various populations, including patients with traumatic brain injury (TBI; Greve, Ord, Curtis, Bianchini, & Brennan, 2008), learning disability (Lindstrom, Lindstrom, Coleman, Nelson, & Gregg, 2009), mixed psychiatric and neurologic diagnoses (Pivovarova, Rosenfeld, Dole, Green, & Zapf, 2009; Tombaugh, 1997), mental retardation (Shandera et al., 2010), attention-deficit hyperactivity disorder (ADHD; Sollman, Ranseen, & Berry, 2010), toxin exposure (Greve et al., 2006), and simulators (Pivovarova et al., 2009). A recent meta-analysis examined the TOMM's classification accuracy in 21 studies (using both simulation and known-group samples) across various clinical populations, including compensation- and non-compensation-seeking individuals with TBI, PTSD, mental retardation, chronic pain, toxic exposure, learning disability, depression, ADHD, and mixed neurologic conditions (Sollman & Berry, 2011). The authors found an aggregate Hedge's d effect size of 1.59 (95% CI = 1.48–1.71) for separating groups of individuals believed to have put forth adequate versus inadequate effort. Moreover, 15 of 21 studies showed specificity values ≥0.90, with only one study showing specificity <0.81 (specificity of 0.69 for discriminating between healthy simulators and individuals with mild mental retardation using test manual cutoff scores; Graue et al., 2007). Sensitivity values, however, tended to be much more variable depending on the patient population, ranging from 0.34 in chronic pain patients (based on TOMM Retention Trial; Greve et al., 2009) to 1.00 in several different patient groups (using both Trial 2 and Retention; Sollman & Berry, 2011).

The WMT is a computerized test of verbal memory that includes forced-choice, multiple-choice, paired associates, and free-recall components. Classifying performance validity is done on the basis of cutoff scores and comparisons with normative samples (Green, 2003). A recent meta-analysis of six studies examining the WMT's classification accuracy revealed a comparable effect size with that of the TOMM, d = 1.61 (95% CI = 1.47–1.76), though specificity values tended to be lower and more variable, ranging from 0.36 to 0.92 (Sollman & Berry, 2011). Importantly, a large number of WMT studies did not meet inclusion criteria for this particular meta-analysis, which is noteworthy as the WMT has received support in many different patient populations. For instance, studies comparing known clinical groups against highly educated individuals asked to feign cognitive impairment without being detected have shown sensitivity values ranging from 0.96 to 1.00 and specificity of 1.00 (Green, Lees-Haley, & Allen, 2003; Iverson, Green, & Gervais, 1999). Comparable findings have since been obtained using English (Tan, Slick, Strauss, & Hultsch, 2002), German (Brockhaus & Merten, 2004), Turkish (Brockhaus, Peker, & Fritze, 2003), and Russian (Tydecks, Merten, & Gubbay, 2006) language versions of the WMT. Moreover, a review of the WMT notes that “converging lines of evidence using clinical groups suggest that the WMT is sensitive to poor effort, yet refractory to the effects of all but the most severe genuine neurological and psychiatric abnormalities” (Wynkoop & Denney, 2005, p. 103). Taken together, these studies support the utility of both the TOMM and WMT as tests of neuropsychological performance validity.

Self-Report Inventories and PVT Performance

A complementary body of research has examined the association between PVT failure and scores on self-report inventories of personality and psychopathology. Using the Minnesota Multiphasic Personality Inventory, 2nd Edition (MMPI-2; Butcher, 2001), Boone and Lu (1999) observed that two thirds of patients in a mixed outpatient sample who exhibited a somatization/conversion profile on the MMPI-2 (i.e., 1–3/3–1 code type) demonstrated inadequate effort on standalone and/or embedded cognitive effort measures. Similar results have been found among medically and neurologically normal litigants claiming brain damage (Larrabee, 1998). Subsequent studies examining the MMPI-2 Restructured Clinical (RC) scales (Tellegen, 2003) have found that disability claimants who failed the WMT demonstrated the greatest scale elevation on somatic complaints (RC1; Wygant, Gervais, & Ben-Porath, 2005). Similarly, personal injury/disability claimants and criminal defendants who failed PVTs (TOMM and/or WMT) scored higher on RC1 and the Symptom Validity scale (FBS; a validity scale that indexes exaggerated somatic or neurologic complaints, NUC) compared with those who passed PVTs (Wygant et al., 2007). Comparable results on RC1 have been found in TBI litigants (Thomas & Youngjohn, 2009).

More recent investigations have also examined the association between PVT performance and certain scales on the MMPI-2-Restructured Form (MMPI-2-RF; Ben-Porath & Tellegen, 2008). The MMPI-2-RF is a 338-item self-report measure derived from the MMPI-2 item pool that includes both symptom validity and substantive scales. Several of the symptom validity scales provide measures of symptom overreporting and response bias, including Infrequent Responses (F-r), Infrequent Psychopathology Responses (Fp-r), Infrequent Somatic Responses (Fs), Symptom Validity (FBS-r), and Response Bias (RBS). A large number of studies have observed associations between PVT failure and elevated scores on symptom overreporting scales in a variety of patient populations. For example, in a sample of personal injury and disability claimants, Wygant and colleagues (2009) examined the relationship between MMPI-2-RF validity scales and PVT performance using the TOMM, WMT, Computerized Assessment of Response Bias (CARB; Allen, Conder, Green, & Cox, 1997), and Victoria Symptom Validity Test (VSVT; Slick, Hopp, Strauss, & Thompson, 1997). They observed that all overreporting scale scores (F-r, Fp-r, Fs, and FBS-r) increased significantly in relation to the number of PVTs failed, indicating an association between exaggerated cognitive impairment and overreported somatic, cognitive, and psychological complaints. Gervais, Ben-Porath, Wygant, and Sellbom (2010) similarly found that disability claimants who failed at least one PVT scored significantly higher on all MMPI-2-RF overreporting scales relative to claimants who passed all PVTs, which included the WMT, CARB, TOMM, and Medical Symptom Validity Test (MSVT; Green, 2004). This pattern of results was replicated when forensic disability claimants were separated according to sex, with both men and women showing higher scores on overreporting scales as the number of failed PVTs increased (Gervais, Wygant, Sellbom, & Ben-Porath, 2011). Jones and Ingram (2011) evaluated MMPI-2 and MMPI-2-RF validity scales in relation to PVT performance (TOMM or VSVT) among active duty military members referred for various neurologic conditions (primarily TBI). The authors found that patients who failed one or both PVTs scored significantly higher on all validity scales, with Cohen's d effect sizes ranging from 0.25 to 1.16. Jones, Ingram, and Ben-Porath (2012) subsequently examined associations between the full range of MMPI-2-RF scales and increasing levels of PVT failure in a sample of active duty military who completed a neuropsychological evaluation primarily for TBI, with PVTs including the TOMM, VSVT, WMT, and an embedded effort index from the Repeatable Battery of Neuropsychological Status (Randolph, 1998). They found linear relationships between number of failed PVTs and all overreporting scales, with medium-to-large partial eta square (ηp2) effects sizes for each scale. Interestingly, Youngjohn, Wershba, Stevenson, Sturgeon, and Thomas (2011) found that only FBS-r accounted for a significant portion of the variance between TBI litigants who passed and failed PVTs, which included the WMT, Portland Digit Recognition Test (Binder, 1993), and Dot Counting Test (Rey, 1941). Nonsignificant trends were also observed for Fs and F-r. Finally, Tarescavage, Wygant, Gervais, and Ben-Porath (2013) found that higher scores on MMPI-2-RF overreporting scales, namely F-r and RBS, were associated with probable and definite Malingered Neurocognitive Dysfunction (MND; Slick, Sherman, & Iverson, 1999) in a large sample of nonhead injury disability claimants.

Overall, these studies suggest that higher scores on MMPI-2-RF overreporting scales are associated with inadequate effort on cognitive testing. A few studies have also examined the association between PVT failure and MMPI-2-RF substantive scales, which are designed to characterize various dimensions of psychopathology, interpersonal functioning, interests, and personality. Youngjohn and colleagues (2011) examined both Validity and Somatic/Cognitive scales in a sample of TBI litigants, and found the latter to be better predictors of PVT failure, with Malaise (MLS) emerging as the single best predictor. Gervais and colleagues (2010) observed that non-TBI disability claimants who failed PVTs scored higher on most substantive scales, with the exception of behavioral/externalizing dysfunction (BXD), antisocial behavior (RC4), and hypomanic activation (RC9). The largest elevations were observed on the somatic/cognitive scales. Using a sample of forensic disability claimants, Gervais and colleagues (2011) examined the full suite of MMPI-2-RF substantive scales. Although slightly different patterns emerged for men and women, results indicated that patients who failed PVTs had more prominent elevations on measures of somatic complaints, neurocognitive problems, and emotional distress, including Cognitive Complaints (COG), Headache Complaints (HPC), Emotional/Internalizing Disorders (EID), Demoralization (RCd), RC1, Low-Positive Emotion (RC2), and Introversion/Low-Positive Emotion (INTR-r). Jones and colleagues (2012) also examined the full range of MMPI-2-RF scales in a sample of active duty military personnel who completed a neuropsychological evaluation primarily for mild TBI. Consistent with Gervais and colleagues (2011), elevated scores on scales indexing somatic/COG and emotional dysfunction were generally the best discriminators of inadequate versus adequate effort on PVTs. Specifically, all Somatic/Cognitive scales (except Gastrointestinal Complaints, GIC) and RC1 had large between-group effect sizes (inadequate effort > adequate effort; all Cohen's d > 0.90), with COG being the largest (d = 1.19). Concerning emotional dysfunction, the largest effect sizes in descending order were Anxiety (ANX), RC2, INTR-r, EID, RCd, Self-Doubt (SFD), Negative Emotionality-Neuroticism (NEGE-r), Anger Proneness (ANP), and Dysfunctional Negative Emotions (RC7) (all d > 0.80). Similar results were obtained by Tarescavage and colleagues (2013) in their sample of nonhead injury disability claimants. Specifically, patients who exhibited evidence of MND endorsed significantly higher scores relative to incentive only patients (i.e., patients with external incentive who demonstrated adequate effort) across almost substantive scales, particularly those that embody emotional distress, interpersonal avoidance, somatic symptoms, and COG. Collectively, these studies suggest that highly elevated somatic/COG and negative emotionality are associated with increased PVT failure, suggesting overreporting in these domains may facilitate inadequate effort detection.

It is important to note, however, that several studies suggest performance validity and symptom validity may be dissociable constructs. For example, using exploratory factor analysis, Nelson, Sweet, Berry, Bryant, and Granacher (2007) examined whether forced-choice PVT measures and MMPI-2 validity scales load on the same or independent factors. From this analysis, four distinct factors emerged: psychological symptom underreporting, neurotic symptom overreporting, inadequate cognitive effort, and overreporting psychotic/rarely endorsed symptoms. Importantly, the cognitive effort factor showed only a moderate association with the overreporting neurotic symptoms factor, and small to negligible associations with the other factors, suggesting symptom and performance validity measures offered unique information regarding response validity. Ruocco and colleagues (2008) obtained similar results from exploratory factor analysis using validity scales from another self-report measure, the Millon Clinical Multiaxial Inventory, 3rd Edition (MCMI-III; Millon, Davis, & Millon, 1997). Van Dyke and colleagues (2013) subsequently used confirmatory factor analysis to test different models describing the relationship between cognitive performance, standard symptom self-report on clinical scales, performance validity, and symptom validity. The strongest model was a three-factor model that included cognitive performance, performance validity, and self-reported symptoms (including standard and symptom validity measures), further suggesting a dissociation between performance validity and symptom validity constructs. Using regression analysis, Whitney (2013) found “unacceptably low positive and negative predictive values” (p. 222) when examining the ability of MMPI-2 validity scales to predict PVT performance using the TOMM and MSVT. Taken together, these data suggest some degree of heterogeneity in response validity, such that failure in one domain (e.g., performance validity) does not necessarily invalidate other domains (e.g., symptom validity), and both areas should be assessed separately during neuropsychological evaluation (Temple, McBride, Horner, & Taylor, 2003).

On balance, the MMPI-2-RF literature supports the association between symptom overreporting and PVT failure. As noted above, however, only three studies to date have investigated the full range of MMPI-2-RF validity and substantive scales in relation to PVT performance (Gervais et al., 2011; Jones et al., 2012; Tarescavage et al., 2013), with only two of these studies exclusively examining individuals with external incentive (Gervais et al., 2011; Tarescavage et al., 2013). Importantly, the chief concern of these studies was investigating how self-report inventories can be used to enhance detection of inadequate effort. In contrast, the current study's objective was to better understand why some individuals put forth inadequate effort and others do not, with specific focus of intrinsic psychological factors. Accordingly, we investigated psychological functioning in individuals who put forth inadequate versus adequate effort using the full range of MMPI-2-RF scales. The role of external financial incentive was experimentally controlled by including only patients who had external incentive at the time of testing, though it was not known whether this incentive was related to the clinical reason for referral (see Methods). In addition, variance attributable to broad symptom overreporting was statistically accounted for using covariate analysis. This is a noteworthy extension of previous research as no studies to date have accounted for symptom overreporting when exploring group differences on MMPI-2-RF substantive scales. Not surprisingly, the only studies to examine the full range of scales observed significant group differences on almost all overreporting (F) scales and numerous substantive scales (Gervais et al., 2011; Jones et al., 2012; Tarescavage et al, 2013). However, any interpretation of group differences on substantive scales would likely be contaminated by variance in symptom overreporting. Although not a limitation of these studies (as their primary concern was determining the utility of the MMPI-2-RF in facilitating inadequate effort detection), it nevertheless limits the interpretability of those data with respect to identifying psychological constructs that are unique to individuals who put forth inadequate versus adequate effort. Therefore, by statistically accounting for variance attributable to symptom overreporting, the current study represents an important extension of previous work as this approach allowed us to obtain a less biased snapshot of psychological functioning in our patient sample.

Although we expected differences in psychological functioning between patients who put forth adequate versus inadequate effort, no hypotheses were made with respect to specific MMPI-2-RF scale differences for two reasons. First, the current study was the first to statistically account for variance attributable to symptom overreporting, thereby precluding a priori hypotheses on the basis of comparable previous research. Second, to our knowledge, no theoretical papers have been published that conceptualize inadequate effort in terms of specific psychological dimensions. Consequently, the current analysis was exploratory in nature and is most appropriately viewed as a guide for generating more specific hypotheses and research questions.

Methods

Participants

The present study examined an archival sample of U.S. Veterans referred for outpatient neuropsychological evaluation at a Department of Veterans Affairs Medical Center. Data were drawn from a sample of consecutive patients referred from throughout the Medical Center, including Primary Care, Neurology, Mental Health, and other VA clinics. To be included in the analysis, patients had to have some form of external financial incentive at the time of testing, including seeking compensation from the VA system for any condition (e.g., obtaining or increasing service-connected disability benefits), as well as other potential secondary financial gain not directly related to the VA system (e.g., Social Security disability, worker's compensation benefits, or civil litigation). This information was obtained by self-report during the clinical interview as part of routine patient care in the Neuropsychology Clinic. Importantly, however, the current sample included only patients who were referred for purely clinical reasons, such as concerns about cognitive functioning on the part of the patient, the patient's family, or other healthcare providers. These were not forensic evaluations and no patients were assessed for the purpose of adjudicating disability status or service connection. Therefore, although all patients had external incentive at the time of testing, data were not available as to whether this incentive was related to the clinical condition for which the referral was made, or to cognition more generally, as it may have been incidental. For example, a patient might have been pursuing disability benefits for orthopedic issues, but was also undergoing a cognitive evaluation for memory complaints. This is an important point of emphasis, as it precluded us from stating unequivocally that inadequate effort in the current sample reflected malingering.

In addition to the above criteria, patients must have been administered the TOMM or WMT (or both) and the MMPI-2-RF. All patients who met these criteria, and for whom data were available for the full range of MMPI-2-RF scales, were included in the initial sample. A total of 60 patients met these criteria, from which four were excluded for having MMPI-2-RF profiles with T-scores ≥80 on either the TRIN or VRIN scales. This was done to minimize error variance related to response inconsistency and/or poor comprehension of item content. Four more patients were excluded due to ambiguity in PVT performance (e.g., failed TOMM, but passed WMT; see below). The final sample consisted of 52 patients (see Table 1 for sample characteristics).

Table 1.

Sample characteristics

 Inadequate effort (n = 23) Adequate effort (n = 29) t or χ2 p 
Sex: male/female (n17/6 29/3 2.22 .14 
Age 42.48 (11.42) 44.31 (11.32) 0.34 .74 
Education (years) 13.04 (1.67) 13.21 (1.76) 0.58 .57 
Handedness (R/L/Amb) 22/1/0 25/1/2 1.72 .42 
History of TBI with LOC (n0.58 .97 
Primary diagnosis (n
 Mood disorder — — 
  ANX disorder — — 
 Substance use disorder — — 
 Psychotic disorder — — 
 ADHD/cognitive disorder* — — 
 Personality disorder — — 
 Malingering — — 
 None — — 
Medications (n
 Narcotic 0.75 .39 
 Anticonvulsant 0.01 .94 
 Antipsychotic 0.13 .72 
 Antidepressant 20 20 2.34 .13 
 Anxiolytic 10 1.43 .23 
Race: AA/C/H/O (n11/11/0/1 8/20/0/0 3.63 .16 
Disability 22 26 — — 
Litigation — — 
 Inadequate effort (n = 23) Adequate effort (n = 29) t or χ2 p 
Sex: male/female (n17/6 29/3 2.22 .14 
Age 42.48 (11.42) 44.31 (11.32) 0.34 .74 
Education (years) 13.04 (1.67) 13.21 (1.76) 0.58 .57 
Handedness (R/L/Amb) 22/1/0 25/1/2 1.72 .42 
History of TBI with LOC (n0.58 .97 
Primary diagnosis (n
 Mood disorder — — 
  ANX disorder — — 
 Substance use disorder — — 
 Psychotic disorder — — 
 ADHD/cognitive disorder* — — 
 Personality disorder — — 
 Malingering — — 
 None — — 
Medications (n
 Narcotic 0.75 .39 
 Anticonvulsant 0.01 .94 
 Antipsychotic 0.13 .72 
 Antidepressant 20 20 2.34 .13 
 Anxiolytic 10 1.43 .23 
Race: AA/C/H/O (n11/11/0/1 8/20/0/0 3.63 .16 
Disability 22 26 — — 
Litigation — — 

Notes: Age and education data are reported as mean (SD). History of TBI was determined on the basis of patient self-report and/or review of available medical records. Diagnoses were assigned at the time of clinical service delivery and were determined according to criteria outlined in the Diagnostic and Statistical Manual of Mental Disorders, 4th Edition Text Revision (DSM-IV-TR). Values for medications represent the number of patients who were prescribed each class of medication at the time of testing. *A cognitive disorder diagnosis could not be given to patients in the inadequate effort group due to invalid test data. AA = African American; C = Caucasian; H = Hispanic, O = other; R = right handed; L = left handed; Amb = ambidextrous; TBI = traumatic brain injury; LOC = loss of consciousness.

Procedure

Patients were administered a test battery consistent with their referral question, and tests were administered and scored according to standardized procedures. Inadequate effort was determined on the basis of a combination of criteria similar to those proposed by Slick and colleagues (1999), and previously reported by Bortnik, Horner, and Bachman (2013). These criteria included the following: first, unambiguous PVT failure according to cutoffs provided in the test manuals; secondly, deficits on testing that were grossly disproportionate to the patient's observed functional level, or that reported by family members (e.g., a patient who exhibits profound memory disturbance on testing, but provided rich details of his or her activities in the preceding few days); finally, implausible errors, patterns of responses, or test scores (e.g., a patient who was reportedly unable to remember highly overlearned information that even very severely impaired patients can recall without difficulty). Although the latter two criteria were used by Slick and colleagues (1999) in determining malingering status, these behaviors in and of themselves are not specific to malingering and could reflect inadequate effort for any number of reasons, such as fluctuating motivation or distrust of the healthcare system. As noted earlier, since we did not know if patients' external incentive was directly related to the clinical reason for referral, we could not definitively say these behaviors were reflective of malingered cognitive impairment, which is why the current paper focuses on inadequate effort more broadly, as opposed to malingering.

Patients were assigned to the inadequate effort group (n = 23) if they demonstrated unambiguous PVT failure and at least one of the behavioral indicators noted above. Patients were assigned to the adequate effort group (n = 29) if they passed all PVTs and did not exhibit any of the behavioral indicators noted above. This method of group assignment is consistent with the approach recommended by Bush and colleagues (2005) and proposed by Slick and colleagues (1999). All decisions regarding effort were made at the time of clinical service delivery. The groups did not differ on any demographic characteristics (Table 1).

Data Analysis

Prior to running the main analyses, bivariate correlations were performed between F-r T-scores from the MMPI-2-RF and PVT scores (i.e., TOMM Trial 2 and Retention; WMT-IR, DR, and CON). This was done in order to quantify the association between performance validity (i.e., TOMM and WMT scores) and symptom validity (i.e., F-r scale T-scores), thereby allowing us to ascertain the extent to which the present data align with previous research showing an association between these two constructs. Multivariate analysis of variance (MANOVA) was then performed using just MMPI-2-RF validity scale T-scores as dependent variables and group (inadequate versus adequate effort) as the independent variable. Next, group differences across the full range of substantive scales were examined using a separate MANOVA. A follow-up MANCOVA on substantive scales was then performed using F-r T-scores as a covariate. This was done to minimize variance attributable to symptom overreporting and, in turn, reduce the likelihood of spuriously concluding that group differences reflected actual differences in psychological functioning, as opposed to simply overreporting. The F-r scale was specifically chosen because it provides a broader index of symptom overreporting across psychological, cognitive, and somatic domains, whereas the other F-scales have a narrower focus on overreporting somatic/cognitive symptoms (Fs and FBS-r) and psychopathology (Fp-r). Although consideration was given to using Fp-r as a covariate given its focus on psychopathology, it was ultimately determined that accounting for overreporting across a broader range of symptoms was more important given that the current study included the full suite of MMPI-2-RF scales in the analysis (MANCOVA test statistics and effect size using Fp-r as a covariate, F(41,9) = 2.58, p = .07, ηp2 = 0.92, were nearly identical to the results obtained from using F-r as a covariate (see Results), indicating a similar effect using either overreporting scale at an omnibus level.). Post hoc analyses were performed with univariate F-tests using a more conservative alpha value of 0.01 to account for multiple comparisons. Effect sizes are reported as ηp2.

Results

Summary of PVT Performance

In the adequate effort group, 17 patients completed the TOMM and 20 completed the WMT. In the inadequate group, 17 patients completed the TOMM and 32 completed the WMT. Note that several patients in each group completed both PVTs. Table 2 summarizes PVT performance across both groups.

Table 2.

Summary of PVT performance

 Inadequate effort (n = 23) Adequate effort (n = 29) 
TOMM Trial 2 42.42 (7.01) 49.41 (0.87) 
TOMM Retention 42.85 (7.83) 49.71 (0.49) 
WMT-IR 69.45 (17.31) 97.30 (2.64) 
WMT-DR 64.50 (16.08) 96.21 (3.34) 
WMT-CNS 65.14 (11.17) 95.84 (4.10) 
 Inadequate effort (n = 23) Adequate effort (n = 29) 
TOMM Trial 2 42.42 (7.01) 49.41 (0.87) 
TOMM Retention 42.85 (7.83) 49.71 (0.49) 
WMT-IR 69.45 (17.31) 97.30 (2.64) 
WMT-DR 64.50 (16.08) 96.21 (3.34) 
WMT-CNS 65.14 (11.17) 95.84 (4.10) 

Notes: Mean (SD); TOMM scores represent total number of correct responses. WMT scores represent percent accuracy. PVT = Performance Validity Test; TOMM = Test of Memory Malingering; WMT = Word Memory Test; IR = Immediate Recall; DR = Delayed Recall; CNS = Consistency.

Correlations Between F-r and PVT Scores

This analysis showed significant negative correlations between F-r and TOMM Trial 2 (r = −.36, p = .04), WMT-IR (r = −.47, p < .01), WMT-DR (r = −.43, p < .01), and WMT-CNS (r = −.48, p < .01). These correlations confirm that poorer PVT performance was associated with greater symptom overreporting on the MMPI-2-RF.

Validity Scales

Omnibus MANOVA on validity scales showed a significant main effect of group, F(8, 43) = 3.51, p = .003, ηp2 = 0.40, Λ = 0.61. Post hoc comparisons showed that the inadequate effort group scored significantly higher on Infrequent Responses (F-r), Infrequent Psychopathology Responses (Fp-r), Infrequent Somatic Responses (Fs), and Symptom Validity (FBS-r) (Table 3).

Table 3.

Summary of T-scores and between-group differences on MMPI-2-RF validity scales

 Inadequate effort Adequate effort F P ηp2 
VRIN-r 51.83 (8.45) 56.00 (9.21) 2.16 .15 0.05 
TRIN-r 57.65 (6.65) 57.24 (5.72) 0.20 .65 <0.01 
F-r 98.04 (22.95) 76.17 (19.15) 14.04 <.01* 0.22 
Fp-r 75.43 (24.22) 60.24 (13.77) 8.13 <.01* 0.14 
Fs 88.70 (20.25) 66.31 (18.90) 16.89 <.01* 0.25 
FBS-r 84.52 (12.54) 70.07 (10.79) 19.94 <.01* 0.29 
L-r 58.65 (9.84) 56.97 (9.84) 0.38 .54 <0.01 
K-r 39.26 (9.27) 41.76 (9.30) 0.93 .34 0.02 
 Inadequate effort Adequate effort F P ηp2 
VRIN-r 51.83 (8.45) 56.00 (9.21) 2.16 .15 0.05 
TRIN-r 57.65 (6.65) 57.24 (5.72) 0.20 .65 <0.01 
F-r 98.04 (22.95) 76.17 (19.15) 14.04 <.01* 0.22 
Fp-r 75.43 (24.22) 60.24 (13.77) 8.13 <.01* 0.14 
Fs 88.70 (20.25) 66.31 (18.90) 16.89 <.01* 0.25 
FBS-r 84.52 (12.54) 70.07 (10.79) 19.94 <.01* 0.29 
L-r 58.65 (9.84) 56.97 (9.84) 0.38 .54 <0.01 
K-r 39.26 (9.27) 41.76 (9.30) 0.93 .34 0.02 

Notes:T-scores are reported as mean (SD). Test statistics represent post hoc between-group differences using an adjusted alpha value of 0.01 to account for multiple comparisons. Statistically significant group differences are denoted with an asterisk. Effect sizes are reported as partial eta squared (ηp2).

Substantive Scale Differences Without F-r as a Covariate

Omnibus MANOVA on the substantive scales showed a significant main effect of group, F(41, 10) = 3.51, p = .02, ηp2 = 0.93, Λ = 0.07. Post hoc comparisons showed that the inadequate effort group scored significantly higher on the following scales: Thought Dysfunction (THD), RCd, RC1, RC7, Aberrant Experiences (RC8), MLS, HPC, NUC, COG, Inefficacy (NFC), ANX, Behavior Restricting Fears (BRF), Disaffiliativeness (DSF), Psychoticism (PSYC-r), and INTR-r (Table 4).

Table 4.

Mean (SD) T-scores on MMPI-2-RF substantive scales and post hoc between-group differences before and after covarying for F-r

 Inadequate effort Adequate effort No covariate
 
F-r covariate
 
F p ηp2 F p ηp2 
Higher order 
 EID 72.61 (13.70) 63.90 (11.58) 6.17 .02 0.11 <0.01 .98 <0.01 
 THD 75.09 (18.56) 60.31 (13.12) 11.29 <.01 0.18* 0.24 .63 <0.01 
 BXD 50.74 (10.76) 56.28 (10.80) 3.38 .07 0.06 7.22 <.01 0.13* 
Restructured clinical 
 RCd 73.43 (10.05) 63.83 (10.13) 11.63 <.01 0.19* 1.63 .21 0.03 
 RC1 83.91 (9.77) 70.07 (11.90) 20.26 <.01 0.29* 6.08 .02 0.11 
 RC2 74.61 (15.47) 64.62 (14.69) 5.66 .02 0.10 0.23 .63 <0.01 
 RC3 61.22 (12.18) 59.62 (11.35) 0.24 .63 <0.01 1.85 .18 0.04 
 RC4 52.91 (8.61) 55.69 (10.29) 1.08 .31 0.02 6.69 <.01 0.12* 
 RC6 73.83 (19.78) 65.14 (14.09) 3.42 .07 0.06 2.44 .13 0.05 
 RC7 68.43 (12.57) 59.07 (12.46) 7.19 <.01 0.13* 0.04 .84 <0.01 
 RC8 75.87 (14.62) 60.34 (11.90) 17.84 <.01 0.26* 3.40 .07 0.07 
 RC9 50.87 (10.57) 54.28 (10.11) 1.40 .24 0.03 3.42 .07 0.07 
Somatic/cognitive 
 MLS 78.17 (9.60) 69.72 (12.27) 7.34 <.01 0.13* 0.28 .60 <0.01 
 GIC 69.52 (14.68) 60.21 (16.80) 4.40 .04 0.08 0.21 .65 <0.01 
 HPC 78.09 (9.61) 66.97 (10.95) 14.73 <.01 0.23* 4.90 .03 0.09 
 NUC 85.65 (9.40) 69.66 (13.51) 23.26 <.01 0.32* 9.29 <.01 0.16* 
 COG 86.65 (8.47) 73.69 (12.61) 17.86 <.01 0.26* 5.37 .03 0.10 
Internalizing 
 HLP 65.78 (16.15) 58.90 (13.49) 2.81 .10 0.05 0.03 .87 <0.01 
 SFD 64.65 (12.22) 56.52 (10.40) 6.72 .01 0.12 1.13 .29 0.02 
 NFC 65.43 (10.36) 55.34 (11.10) 11.23 <.01 0.18* 3.48 .07 0.07 
 STW 63.22 (12.15) 59.07 (10.05) 1.82 .18 0.04 0.04 .85 <0.01 
 AXY 79.78 (19.70) 64.72 (16.72) 8.89 <.01 0.15* 0.16 .69 <0.01 
 ANP 65.00 (8.30) 63.45 (13.19) 0.24 .63 <0.01 0.10 .75 <0.01 
 BRF 67.48 (17.71) 56.00 (10.47) 8.48 <.01 0.15* 1.64 .21 0.03 
 MSF 53.65 (13.01) 47.00 (7.66) 5.29 .03 0.10 1.56 .22 0.03 
Externalizing 
 JCP 51.30 (9.37) 56.17 (13.88) 2.08 .16 0.04 4.06 .05 0.08 
 SUB 49.48 (9.67) 48.72 (8.60) 0.09 .77 <0.01 2.02 .16 0.04 
 AGG 63.74 (14.07) 58.14 (11.17) 2.57 .12 0.05 0.30 .59 <0.01 
 ACT 49.87 (11.41) 49.28 (8.41) 0.05 .83 <0.01 0.31 .58 <0.01 
Interpersonal 
 FML 59.22 (11.79) 54.69 (13.14) 1.67 .20 0.03 0.31 .58 <0.01 
 IPP 52.52 (11.41) 47.10 (10.51) 3.16 .08 0.06 2.57 .12 0.05 
 SAV 70.87 (10.94) 63.76 (13.13) 4.35 .04 0.08 0.21 .65 <0.01 
 SHY 53.65 (10.33) 51.21 (11.70) 0.62 .44 0.01 0.02 .90 <0.01 
 DSF 76.17 (16.60) 64.69 (11.63) 8.59 <.01 0.15* 1.10 .30 0.02 
Interest 
 AES 37.70 (4.42) 41.34 (7.35) 4.40 .04 0.08 4.22 .05 0.08 
 MEC 52.00 (10.80) 57.24 (9.41) 3.49 .07 0.07 3.09 .09 0.06 
PSY-5 
 AGGR-r 53.39 (12.98) 57.34 (12.29) 1.26 .27 0.03 3.83 .06 0.07 
 PSYC-r 73.43 (18.36) 58.72 (12.44) 11.81 <.01 0.19* 0.49 .49 0.01 
 DISC-r 48.35 (10.50) 55.21 (9.98) 5.79 .02 0.10 6.62 <.01 0.12* 
 NEGE-r 69.65 (11.85) 62.03 (11.48) 5.49 .02 0.10 0.04 .85 <0.01 
 INTR-r 75.30 (12.93) 63.90 (13.89) 9.19 <.01 0.16* 2.39 .13 0.05 
 Inadequate effort Adequate effort No covariate
 
F-r covariate
 
F p ηp2 F p ηp2 
Higher order 
 EID 72.61 (13.70) 63.90 (11.58) 6.17 .02 0.11 <0.01 .98 <0.01 
 THD 75.09 (18.56) 60.31 (13.12) 11.29 <.01 0.18* 0.24 .63 <0.01 
 BXD 50.74 (10.76) 56.28 (10.80) 3.38 .07 0.06 7.22 <.01 0.13* 
Restructured clinical 
 RCd 73.43 (10.05) 63.83 (10.13) 11.63 <.01 0.19* 1.63 .21 0.03 
 RC1 83.91 (9.77) 70.07 (11.90) 20.26 <.01 0.29* 6.08 .02 0.11 
 RC2 74.61 (15.47) 64.62 (14.69) 5.66 .02 0.10 0.23 .63 <0.01 
 RC3 61.22 (12.18) 59.62 (11.35) 0.24 .63 <0.01 1.85 .18 0.04 
 RC4 52.91 (8.61) 55.69 (10.29) 1.08 .31 0.02 6.69 <.01 0.12* 
 RC6 73.83 (19.78) 65.14 (14.09) 3.42 .07 0.06 2.44 .13 0.05 
 RC7 68.43 (12.57) 59.07 (12.46) 7.19 <.01 0.13* 0.04 .84 <0.01 
 RC8 75.87 (14.62) 60.34 (11.90) 17.84 <.01 0.26* 3.40 .07 0.07 
 RC9 50.87 (10.57) 54.28 (10.11) 1.40 .24 0.03 3.42 .07 0.07 
Somatic/cognitive 
 MLS 78.17 (9.60) 69.72 (12.27) 7.34 <.01 0.13* 0.28 .60 <0.01 
 GIC 69.52 (14.68) 60.21 (16.80) 4.40 .04 0.08 0.21 .65 <0.01 
 HPC 78.09 (9.61) 66.97 (10.95) 14.73 <.01 0.23* 4.90 .03 0.09 
 NUC 85.65 (9.40) 69.66 (13.51) 23.26 <.01 0.32* 9.29 <.01 0.16* 
 COG 86.65 (8.47) 73.69 (12.61) 17.86 <.01 0.26* 5.37 .03 0.10 
Internalizing 
 HLP 65.78 (16.15) 58.90 (13.49) 2.81 .10 0.05 0.03 .87 <0.01 
 SFD 64.65 (12.22) 56.52 (10.40) 6.72 .01 0.12 1.13 .29 0.02 
 NFC 65.43 (10.36) 55.34 (11.10) 11.23 <.01 0.18* 3.48 .07 0.07 
 STW 63.22 (12.15) 59.07 (10.05) 1.82 .18 0.04 0.04 .85 <0.01 
 AXY 79.78 (19.70) 64.72 (16.72) 8.89 <.01 0.15* 0.16 .69 <0.01 
 ANP 65.00 (8.30) 63.45 (13.19) 0.24 .63 <0.01 0.10 .75 <0.01 
 BRF 67.48 (17.71) 56.00 (10.47) 8.48 <.01 0.15* 1.64 .21 0.03 
 MSF 53.65 (13.01) 47.00 (7.66) 5.29 .03 0.10 1.56 .22 0.03 
Externalizing 
 JCP 51.30 (9.37) 56.17 (13.88) 2.08 .16 0.04 4.06 .05 0.08 
 SUB 49.48 (9.67) 48.72 (8.60) 0.09 .77 <0.01 2.02 .16 0.04 
 AGG 63.74 (14.07) 58.14 (11.17) 2.57 .12 0.05 0.30 .59 <0.01 
 ACT 49.87 (11.41) 49.28 (8.41) 0.05 .83 <0.01 0.31 .58 <0.01 
Interpersonal 
 FML 59.22 (11.79) 54.69 (13.14) 1.67 .20 0.03 0.31 .58 <0.01 
 IPP 52.52 (11.41) 47.10 (10.51) 3.16 .08 0.06 2.57 .12 0.05 
 SAV 70.87 (10.94) 63.76 (13.13) 4.35 .04 0.08 0.21 .65 <0.01 
 SHY 53.65 (10.33) 51.21 (11.70) 0.62 .44 0.01 0.02 .90 <0.01 
 DSF 76.17 (16.60) 64.69 (11.63) 8.59 <.01 0.15* 1.10 .30 0.02 
Interest 
 AES 37.70 (4.42) 41.34 (7.35) 4.40 .04 0.08 4.22 .05 0.08 
 MEC 52.00 (10.80) 57.24 (9.41) 3.49 .07 0.07 3.09 .09 0.06 
PSY-5 
 AGGR-r 53.39 (12.98) 57.34 (12.29) 1.26 .27 0.03 3.83 .06 0.07 
 PSYC-r 73.43 (18.36) 58.72 (12.44) 11.81 <.01 0.19* 0.49 .49 0.01 
 DISC-r 48.35 (10.50) 55.21 (9.98) 5.79 .02 0.10 6.62 <.01 0.12* 
 NEGE-r 69.65 (11.85) 62.03 (11.48) 5.49 .02 0.10 0.04 .85 <0.01 
 INTR-r 75.30 (12.93) 63.90 (13.89) 9.19 <.01 0.16* 2.39 .13 0.05 

Notes: Scores are grouped according to the organization scheme provided by an MMPI-2-RF interpretive report (Ben-Porath & Tellegen, 2008). Significance testing was performed with an adjusted alpha value of 0.01 to account for multiple comparisons. Statistically significant group differences are denoted with an asterisk. Effect sizes are reported as ηp2.

Substantive Scale Differences with F-r as a Covariate

Prior to running the MANCOVA, we performed multicollinearity diagnostics by examining the correlation matrix between the two predictors (F-r T-scores and group) using Pearson point-biserial correlations. This showed a small-to-moderate correlation between F-r and group (r = .46), which was below a pre-defined and conservative cutoff value of 0.60. Omnibus MANCOVA on substantive scales using F-r as a covariate showed a nonsignificant trend towards a main effect of group, F(41, 9) = 2.32, p = .08, ηp2 = 0.91, Λ = 0.09. Although the omnibus effect did not reach statistical significance at p < .05, the very large effect size warranted post hoc analyses of substantive scale differences (Note: benchmarks of 0.01, 0.06, and 0.14 were used for interpreting small, medium, and large ηp2 effect sizes, respectively; see Cohen, 1969; Richardson, 2011). The inadequate effort group scored significantly higher on NUC, and significantly lower on BXD, RC4, and Disconstraint (DISC-r) (Table 4). Full MMPI-2-RF profiles for both groups are provided in Fig. 1.

Fig. 1.

Profile configurations for adequate and inadequate effort groups across the full range of MMPI-2-RF scales, including (a) Validity, (b) Higher-Order, (c) Restructured Clinical, (d) Somatic/Cognitive, (e) Internalizing, (f) Externalizing, (g) Interpersonal, (h) Interests, and (i) PSY-5. The upper dashed line indicates the 92nd percentile (T = 65) and represents the minimal level of scale elevation required for clinically significant interpretive recommendations on the Substantive Scales. The lower dashed line indicates the 8th percentile (T = 38) and is designed to assist in identifying clinically interpretable low scores (Ben-Porath, 2012). The area between the dashed lines is generally considered to be within normal limits. Statistically significant between-group differences after covarying for F-r are denoted by an asterisk.

Fig. 1.

Profile configurations for adequate and inadequate effort groups across the full range of MMPI-2-RF scales, including (a) Validity, (b) Higher-Order, (c) Restructured Clinical, (d) Somatic/Cognitive, (e) Internalizing, (f) Externalizing, (g) Interpersonal, (h) Interests, and (i) PSY-5. The upper dashed line indicates the 92nd percentile (T = 65) and represents the minimal level of scale elevation required for clinically significant interpretive recommendations on the Substantive Scales. The lower dashed line indicates the 8th percentile (T = 38) and is designed to assist in identifying clinically interpretable low scores (Ben-Porath, 2012). The area between the dashed lines is generally considered to be within normal limits. Statistically significant between-group differences after covarying for F-r are denoted by an asterisk.

Discussion

The present study sought to identify dimensions of psychological functioning associated with patients who put forth inadequate effort. This was done by examining the full range of MMPI-2-RF scores in patients who put forth inadequate versus adequate effort during a neuropsychological evaluation. All patients were evaluated for clinical reasons, but reported some sort of external financial incentive at the time of testing. Group differences were examined before and after accounting for variance attributable to symptom overreporting.

With respect to validity scales, patients in the inadequate effort group had higher scores on all symptom overreporting (F) scales. This is consistent with most previous research (Gervais et al. 2010, 2011; Jones & Ingram 2011; Jones et al. 2012; Wygant et al. 2009, 2010; Youngjohn et al. 2011; Tarescavage et al. 2013). In conjunction with the observed correlation data, these findings reaffirm the association between symptom overreporting on the MMPI-2-RF and inadequate cognitive effort. In turn, this finding bolsters the assertion that broad overreporting of psychological dysfunction should prompt greater scrutiny of possible inadequate effort. More importantly, these data highlight the potential importance of accounting for symptom overreporting when attempting to identify genuine psychological characteristics unique to individuals who put forth inadequate effort.

In this context, an important finding from the present study concerns the altered pattern of group differences observed after covarying for symptom overreporting. Prior to including F-r as a covariate, the inadequate effort group scored significantly higher on 15 different substantive scales encompassing a wide array of symptom dimensions. They reported higher levels of somatic and COG (MLS, NUC, RC1, COG, and HPC), disordered thought processes and perceptual experiences (THD, RC8, and PSYC-r), and internalizing symptomatology and social disconnectedness (RCd, RC7, NFC, AXY, BRF, INTR-r, and DSF). These results are generally consistent with previous research showing elevations on somatic/COG (Gervais et al., 2011; Jones et al., 2012; Thomas & Youngjohn, 2009; Wygant et al., 2007) and internalizing/negative emotionality (Gervais et al., 2011; Jones et al., 2012) in patients who fail PVTs. However, after including F-r as a covariate, the groups differed on just four substantive scales (BXD, RC4, NUC, and DISC-r), with the inadequate effort group showing higher scores on only NUC. While these differences should be viewed with some degree of caution given the nonsignificant omnibus effect, they nevertheless suggest that a considerable portion of between-group variance on substantive scales identified in previous research might be attributable to differences in response style (i.e., symptom overreporting), rather than genuine differences in psychological functioning.

The inadequate effort group scored higher on NUC, indicating these patients endorsed more neurologic symptoms, such as dizziness, numbness, weakness, and sensorimotor dysfunction. While it is possible that higher scores in the inadequate effort group could reflect genuine symptomatology, perhaps stemming from history of TBI, this explanation seems unlikely given the nearly identical proportions of patients with history of TBI in each group. It is also possible that elevated NUC reflects symptom overreporting, even after covarying for F-r, as willful exaggeration of neurologic symptoms, in particular, might be expected in this context. This interpretation is also quite speculative, however, and it remains unclear from the current data why the inadequate effort group scored higher on NUC.

The inadequate effort group scored lower on BXD, RC4, and DISC-r after covarying for F-r. This pattern is broadly consistent with the findings of Gervais and colleagues (2010), who observed that non-TBI disability claimants who failed PVTs scored higher on most substantive scales of the MMPI-2-RF, with the exception of BXD, RC4, and Hypomania (RC9), though that study did not account for symptom validity scales when identifying group differences. BXD, RC4, and DISC-r are highly intercorrelated in healthy volunteer and clinical outpatient samples (Tellegen & Ben-Porath, 2008) and provide a broad index of behavioral constraint. Lower scores on these scales collectively suggest that the inadequate effort group was characterized by more behavioral constraint, both at the time of testing (BXD and DISC-r) and in the past (RC4). At a phenomenological level, higher behavioral constraint might manifest as fewer externalizing behaviors, such as impulsivity, sensation-seeking, risk-taking, and interpersonal aggression. These patients may also be more deliberative, exercise greater caution and restraint before acting, and may be more apt to abide by rules and norms. In addition, these features are likely to be characterological in nature, rather than state-related qualities that emerged at the time of testing. It should be noted, however, that mean T-scores on each of these scales fell within normal limits for both groups. Therefore, although patients who put forth inadequate effort had higher levels of behavioral constraint, neither group reported levels (high or low) indicative of psychopathology on this dimension.

This pattern of group differences begs the question of why higher behavioral constraint was more strongly associated with inadequate effort. Given the potential for secondary gain, it is reasonable to suggest that inadequate effort reflected a calculated attempt at feigning dysfunction in order to increase the likelihood of securing monetary compensation in the future—that is, malingering. On one hand, such behavior seems inconsistent with certain qualities embodied by individuals with higher behavioral constraint, such as the tendency to conform to rules and norms. However, when viewed within the context of behavioral economic theories of reward seeking, the relationship between constraint and malingering is worthy of consideration. In this vein, Carver (2005) relates the dimension of constraint-impulsivity to dual-system models of behavioral economics—namely, Metcalfe and Mischel's (1999) “hot/cool” system of delayed gratification. In the “hot” system, decisions are made rapidly, reactively, and in service of obtaining an immediate reward (i.e., impulsivity). In the “cool” system, decisions are strategic, deliberative, and motivated by the desire to secure a more highly valued future reward (i.e., constraint). From this, Carver (2005) suggests that “the roots of human constraint lie primarily in acquiring mental strategies that permit a more extended pursuit of larger and more enduring incentives” (p. 319). That is, individuals who are characterized by higher behavioral constraint are more likely to adopt decision-making strategies that increase the likelihood of securing a temporally distant, higher-value reward. Extending this line of reasoning to the current study, inadequate effort may have reflected a decision-making strategy employed by patients with more behavioral constraint aimed at increasing the likelihood of obtaining a financial reward in the future. If this finding proves reliable, it suggests that elevated behavioral constraint may be an intrinsic psychological factor that increases the likelihood of malingering in patients who are tested in a secondary gain context.

There are notable caveats to the present study, however, that limit the explanatory power of the behavioral economics argument. First, the current dataset did not specify whether disability or compensation-seeking was specifically related to the patients' reason for clinical referral. That is, for an unspecified proportion of the current sample, the neuropsychological evaluation may have been completely unrelated to the claimed or sought disability. Such patients would have little motivation to feign cognitive dysfunction as it would have little to no influence on the likelihood of securing their sought-after compensation. Removing such financial incentive would make explanations grounded in behavioral economics, and direct links to malingering, far less compelling. This issue could be clarified in future experiments by dichotomizing patients according to whether or not the neuropsychological evaluation was directly tied to the likelihood of obtaining the sought-after external incentive.

That said, Chafetz, Prentkowski, and Rao (2011) reported that among individuals presumably motivated to obtain work who underwent neuropsychological evaluation to assess vocational readiness, PVT failure was observed only in those who were also seeking disability. Thus, even though the purpose of the evaluation was not directly related to the sought-after compensation, individuals with external incentive were still more likely to generate invalid test data. Extending this finding to the current sample, it remains possible that despite these evaluations being done for purely clinical purposes, the presence of potential secondary gain may have still influenced performance, perhaps among those who exhibited greater behavioral constraint.

A second, related caveat is that the current data do not inform whether behavioral constraint is elevated among all patients who put forth inadequate effort, or just those with external incentive. That is, the current findings cannot be generalized to patients who put forth inadequate effort in the absence of any external incentive. Although the behavioral economics argument outlined above emphasizes the importance of future reward, the current data are limited in this respect given that the entire sample had some form of financial incentive. This question could be clarified by running a similar experiment using patients without any external incentive.

An alternative explanation for lower scores on BXD, RC4, and DISC-r in the inadequate effort group is that patients adopted a self-presentation strategy aimed at minimizing the presence of externalizing, antisocial, and impulsive behaviors. Doing so might have reflected a volitional attempt at generating a more “psychologically healthy” profile free of overtly deviant and possibly immoral behaviors. In this context, lower scores on BXD, RC4, and DISC-r might have reflected an intentional and strategic response bias, rather than genuinely elevated behavioral constraint. Such impression management seems plausible if patients thought denying socially unacceptable or deviant behaviors on self-report would make their feigned cognitive dysfunction appear more genuine. If this were true, we might expect that the inadequate effort group would show higher T-scores on the L-r scale, which is elevated when patients attempt to portray themselves in a highly favorable light. However, this was not the case as groups did not differ on L-r scores and both groups were within normal limits (Table 3). This interpretation also hinges on the extent to which the test items comprising BXD, RC4, and DISC-r have sufficient face validity to allow for focal denial of these symptoms and behaviors, while concurrently endorsing a globally high level of distress and other symptomatology, as indicated by the elevated scores across all F-scales. In turn, this suggests a rather high degree of sophistication among patients in the inadequate effort group that is somewhat inconsistent with their very unambiguous failure on PVTs. Thus, additional research is clearly needed to help ascertain whether the association between behavioral constraint and inadequate effort in a secondary gain context reflects strategic impression management, genuine differences in psychological functioning, or some alternative factors not identified in the current study.

There are several other limitations of the current study that warrant further consideration and can be used to guide future research. First, both groups had a large male-to-female ratio. Although this likely mitigated any influence of sex-differences across groups, the current dataset may not be applicable to the general population. Similarly, it is unclear whether these findings are specific to a U.S. Veteran sample or are representative of outpatient samples more broadly. Future research would benefit from having equal sex ratios between groups and extending the analysis to non-Veteran samples.

Second, the current sample was relatively small compared with other studies that have examined the relationship between PVTs and self-report inventories. This may have reduced statistical power and hindered our ability to detect subtle group differences, as suggested by the nonsignificant omnibus effect. Post hoc examination of observed power for univariate F-tests (following omnibus MANCOVA) showed values <0.80 on nearly all substantive scales. Consequently, it is possible that the groups differed on other psychological dimensions not identified by the present analysis. These results, therefore, are most appropriately viewed as preliminary and require replication in larger samples.

Third, it is possible that the current results do not entirely reflect unique psychological characteristics associated with effort level. Rather, the observed profiles might reflect the prominent diagnostic categories within each group. Given the relatively small sample size, it was not possible to stratify groups according to diagnosis for a secondary analysis. Adopting this approach in future research will permit a finer grained assessment of psychological features associated with inadequate effort independent of diagnostic category.

Fourth, there is a reasonable possibility that the adequate effort group contained a small number patients who had in fact exerted inadequate effort, given potential limited sensitivity of the TOMM in particular (Sollman & Berry, 2011). If this were the case, it would suggest that the magnitude of group differences observed in the current study may actually underestimate the true differences in psychological functioning between those who put forth adequate versus inadequate effort. This question could be explored further by replicating the current experiment using additional PVTs with higher sensitivity for detecting inadequate effort.

Finally, it is not clear from the present data whether groups differed on reading comprehension and general intelligence as such tests were not uniformly administered across patients. This is notable as these abilities could have influenced patients' capacity to read and understand items on the MMPI-2-RF. Although the VRIN and TRIN scales were used to exclude patients for potential random responding and/or poor comprehension of item content, future experiments could benefit from specifically testing these constructs and matching groups accordingly.

Summary and Conclusions

The present study sought to characterize dimensions of psychological functioning associated with patients who put forth inadequate versus adequate effort on cognitive testing. The role of external motivation was accounted for by including only patients who were tested in a secondary gain context. Variance attributable to symptom overreporting was also accounted for by covarying for F-r. Results suggest that inadequate effort may have been associated with higher levels of behavioral constraint, as indexed by lower scores on BXD, RC4, and DISC-r. Within behavioral economic models of reward-seeking, constraint has been linked to modes of information processing that are more strategic, deliberative, and guided by and the desire to secure a highly valued future reward (i.e., “cool” cognition). Inadequate effort, therefore, may have reflected a decision-making strategy whereby patients with higher behavioral constraint sought to increase the likelihood of obtaining secondary financial gain. Alternatively, lower scores on BXD, RC4, and DISC-r could have reflected a self-presentation strategy aimed at minimizing the presence of externalizing, antisocial, and impulsive behaviors in order to generate a more “psychologically healthy” profile. However, there are notable caveats associated with each of these interpretations that highlight the need for additional research. Accordingly, the current results are most appropriately viewed as preliminary, and can be used to generate new hypotheses for future research to characterize specific psychological differences between patients who exert adequate and inadequate effort on neuropsychological examination.

Conflict of Interest

None declared.

Acknowledgements

The authors thank Dr. Travis Turner and Dr. Kathryn VanKirk at the Ralph H. Johnson VA Medical Center (Neuropsychology Clinic) for their invaluable contributions to patient testing, data collection, and data management.

References

Allen
L. M.
Conder
R. L.
Green
P.
Cox
D. R.
CARB'97 manual for the computerized assessment of response bias
 
1997
Durham, NC
CogniSyst
Ben-Porath
Y. S.
Interpreting the MMPI-2-RF
 
2012
Minneapolis, MN: University of Minnesota Press
Ben-Porath
Y. S.
Tellegen
A.
MMPI-2-RF: Manual for administration, scoring and interpretation
 
2008
Minneapolis, MN: University of Minnesota Press
Binder
L. M.
An abbreviated form of the Portland Digit Recognition Test
The Clinical Neuropsychologist
 
1993
7
104
107
Boone
K.
Lu
P. H.
Impact of somatoform symptomatology on credibility of cognitive performance
The Clinical Neuropsychologist
 
1999
13
414
419
Bortnik
K. E.
Horner
M. D.
Bachman
D. L.
Performance on standard indexes of effort among patients with dementia
Applied Neuropsychology: Adult
 
2013
20
233
242
Brockhaus
D. P. R.
Merten
T.
Neuropsychologische diagnostik suboptimalen leistungsverhaltens mit dem Word Memory Test
Der Nervenarzt
 
2004
75
882
887
Brockhaus
R.
Peker
Ö.
Fritze
E.
2003
August. Testing effort in Turkish-speaking subjects: Validation of a translation of the Word Memory Test (WMT). In Poster at annual convention of Gesellschaft der Neuropsychologie (Berlin, Germany)
Bush
S. S.
Ruff
R. M.
Tröster
A. I.
Barth
J. T.
Koffler
S. P.
Pliskin
N. H.
et al.  
Symptom validity assessment: Practice issues and medical necessity: NAN Policy & Planning Committee
Archives of Clinical Neuropsychology
 
2005
20
419
426
Butcher
J. N.
MMPI-2: Minnesota multiphasic personality inventory-2: Manual for administration, scoring, and interpretation
 
2001
Minneapolis, MN: University of Minnesota Press
2001
Carver
C. S.
Impulse and constraint: Perspectives from personality psychology, convergence with theory in other areas, and potential for integration
Personality and Social Psychology Review
 
2005
9
312
333
Chafetz
M. D.
Prentkowski
E.
Rao
A.
To work or not to work: Motivation (not low IQ) determines symptom validity test findings
Archives of Clinical Neuropsychology
 
2011
26
306
313
Cohen
J.
Statistical power analysis for the behavioral sciences
 
1969
San Deigo, CA: Academic Press
Gervais
R. O.
Ben-Porath
Y. S.
Wygant
D. B.
Sellbom
M.
Incremental validity of the MMPI-2-RF over-reporting scales and RBS in assessing the veracity of memory complaints
Archives of Clinical Neuropsychology
 
2010
25
274
284
Gervais
R. O.
Wygant
D. B.
Sellbom
M.
Ben-Porath
Y. S.
Associations between symptom validity test failure and scores on the MMPI-2-RF validity and substantive scales
Journal of Personality Assessment
 
2011
93
508
517
Graue
L. O.
Berry
D. T.
Clark
J. A.
Sollman
M. J.
Cardi
M.
Hopkins
J.
et al.  
Identification of feigned mental retardation using the new generation of malingering detection instruments: Preliminary findings
The Clinical Neuropsychologist
 
2007
21
929
942
Green
P.
Green‘s word memory test for windows: User‘s manual
 
2003
Edmonton, AB
Green's Publishing
Green
P.
Medical Symptom Validity Test (MSVT) for microsoft windows: User's manual
 
2004
Edmonton, AB
Paul Green Publishing
Green
P.
Lees-Haley
P. R.
Allen
L. M.
III
The Word Memory Test and the validity of neuropsychological test scores
Journal of Forensic Neuropsychology
 
2003
2
97
124
Greve
K. W.
Bianchini
K. J.
Black
F. W.
Heinly
M. T.
Love
J. M.
Swift
D. A.
et al.  
Classification accuracy of the Test of Memory Malingering in persons reporting exposure to environmental and industrial toxins: Results of a known-groups analysis
Archives of Clinical Neuropsychology
 
2006
21
439
448
Greve
K. W.
Ord
J.
Curtis
K. L.
Bianchini
K. J.
Brennan
A.
Detecting malingering in traumatic brain injury and chronic pain: A comparison of three forced-choice symptom validity tests
The Clinical Neuropsychologist
 
2008
22
896
918
Greve
K. W.
Ord
J. S.
Bianchini
K. J.
Curtis
K. L.
Prevalence of malingering in patients with chronic pain referred for psychologic evaluation in a medico-legal context
Archives of Physical Medicine and Rehabilitation
 
2009
90
1117
1126
Iverson
G.
Green
P.
Gervais
R.
Using the word memory test to detect biased responding in head injury litigation
Journal of Cognitive Rehabilitation
 
1999
17
4
8
Jones
A.
Ingram
M. V.
A comparison of selected MMPI-2 and MMPI-2-RF validity scales in assessing effort on cognitive tests in a military sample
The Clinical Neuropsychologist
 
2011
25
1207
1227
Jones
A.
Ingram
M. V.
Ben-Porath
Y. S.
Scores on the MMPI-2-RF scales as a function of increasing levels of failure on cognitive symptom validity tests in a military sample
The Clinical Neuropsychologist
 
2012
26
790
815
Larrabee
G. J.
Somatic malingering on the MMPI and MMPI-2 in personal injury litigants
The Clinical Neuropsychologist
 
1998
12
179
188
Larrabee
G. J.
Detection of malingering using atypical performance patterns on standard neuropsychological tests
The Clinical Neuropsychologist
 
2003
17
410
425
Larrabee
G. J.
Millis
S. R.
Meyers
J. E.
40 plus or minus 10, a new magical number: Reply to Russell
The Clinical Neuropsychologist
 
2009
23
841
849
Lindstrom
W. A.
Lindstrom
J. H.
Coleman
C.
Nelson
J.
Gregg
N.
The diagnostic accuracy of symptom validity tests when used with postsecondary students with learning disabilities: A preliminary investigation
Archives of Clinical Neuropsychology
 
2009
24
659
669
Metcalfe
J.
Mischel
W.
A hot/cool-system analysis of delay of gratification: Dynamics of willpower
Psychological Review
 
1999
106
3
Millon
T.
Davis
R. D.
Millon
C.
Manual for the millon clinical multiaxial inventory-III (MCMI-III)
 
1997
Minneapolis, MN
National Computer Systems
Mittenberg
W.
Patton
C.
Canyock
E. M.
Condit
D. C.
Base rates of malingering and symptom exaggeration
Journal of Clinical and Experimental Neuropsychology
 
2002
24
1094
1102
Nelson
N. W.
Sweet
J. J.
Berry
D. T.
Bryant
F. B.
Granacher
R. P.
Response validity in forensic neuropsychology: Exploratory factor analytic evidence of distinct cognitive and psychological constructs
Journal of the International Neuropsychological Society
 
2007
13
440
449
Pivovarova
E.
Rosenfeld
B.
Dole
T.
Green
D.
Zapf
P.
Are measures of cognitive effort and motivation useful in differentiating feigned from genuine psychiatric symptoms?
International Journal of Forensic Mental Health
 
2009
8
271
278
Randolph
C.
RBANS manual: Repeatable battery for the assessment of neuropsychological status
 
1998
San Antonio, TX
The Psychological Corporation
Rey
A.
L'examen psychologique dans les cas d'encephalopathie traumatique
Archives of Psychology
 
1941
28
286
340
Richardson
J. T.
Eta squared and partial eta squared as measures of effect size in educational research
Educational Research Review
 
2011
6
135
147
Ruocco
A. C.
Swirsky-Sacchetti
T.
Chute
D. L.
Mandel
S.
Platek
S. M.
Zillmer
E. A.
Distinguishing between neuropsychological malingering and exaggerated psychiatric symptoms in a neuropsychological setting
The Clinical Neuropsychologist
 
2008
22
547
564
Shandera
A. L.
Berry
D. T.
Clark
J. A.
Schipper
L. J.
Graue
L. O.
Harp
J. P.
Detection of malingered mental retardation
Psychological Assessment
 
2010
22
50
Slick
D.
Hopp
G.
Strauss
E.
Thompson
G. B.
VSVT, Victoria symptom validity test: Version 1.0, professional manual
 
1997
Odessa, FL
Psychological Assessment Resources
Slick
D. J.
Sherman
E. M.
Iverson
G. L.
Diagnostic criteria for Malingered Neurocognitive Dysfunction: Proposed standards for clinical practice and research
The Clinical Neuropsychologist
 
1999
13
545
561
Sollman
M. J.
Berry
D. T.
Detection of inadequate effort on neuropsychological testing: A meta-analytic update and extension
Archives of Clinical Neuropsychology
 
2011
26
774
789
Sollman
M. J.
Ranseen
J. D.
Berry
D. T.
Detection of feigned ADHD in college students
Psychological Assessment
 
2010
22
325
Tan
J. E.
Slick
D. J.
Strauss
E.
Hultsch
D. F.
How'd they do it? Malingering strategies on symptom validity tests
The Clinical Neuropsychologist
 
2002
16
495
505
Tarescavage
A. M.
Wygant
D. B.
Gervais
R. O.
Ben-Porath
Y. S.
Association between the MMPI-2 Restructured Form (MMPI-2-RF) and Malingered Neurocognitive Dysfunction among non-head injury disability claimants
The Clinical Neuropsychologist
 
2013
27
313
335
Tellegen
A.
The MMPI-2 Restructured Clinical (RC) scales: Development, validation, and interpretation
 
2003
Minneapolis, MN: University of Minnesota Press
2003
Tellegen
A.
Ben-Porath
Y. S.
MMPI-2-RF, Minnesota multiphasic personality inventory-2 restructured form: Technical manual
 
2008
Minneapolis, MN: University of Minnesota Press
Temple
R. O.
McBride
A. M.
Horner
M. D.
Taylor
R. M.
Personality characteristics of patients showing suboptimal cognitive effort
The Clinical Neuropsychologist
 
2003
17
402
409
Thomas
M. L.
Youngjohn
J. R.
Let's not get hysterical: Comparing the MMPI-2 validity, clinical, and RC scales in TBI litigants tested for effort
The Clinical Neuropsychologist
 
2009
23
1067
1084
Tombaugh
T. N.
Test of Memory Malingering: TOMM
 
1996
North Tonawanda, NY
Multi-Health Systems
Tombaugh
T. N.
The Test of Memory Malingering (TOMM): Normative data from cognitively intact and cognitively impaired individuals
Psychological Assessment
 
1997
9
260
Tydecks
S.
Merten
T.
Gubbay
J.
The Word Memory Test and the One-in-Five-Test in an analogue study with Russian-speaking participants
International Journal of Forensic Psychology
 
2006
1
29
37
Van Dyke
S. A.
Millis
S. R.
Axelrod
B. N.
Hanks
R. A.
Assessing effort: Differentiating performance and symptom validity
The Clinical Neuropsychologist
 
2013
27
1234
1246
Whitney
K. A.
Predicting Test of Memory Malingering and Medical Symptom Validity Test failure within a Veterans Affairs Medical Center: Use of the Response Bias scale and the Henry–Heilbronner Index
Archives of Clinical Neuropsychology
 
2013
28
222
235
Wygant
D. B.
Ben-Porath
Y. S.
Arbisi
P. A.
Berry
D. T.
Freeman
D. B.
Heilbronner
R. L.
Examination of the MMPI-2 restructured form (MMPI-2-RF) validity scales in civil forensic settings: Findings from simulation and known group samples
Archives of Clinical Neuropsychology
 
2009
24
671
680
Wygant
D. B.
Gervais
R. O.
Ben-Porath
Y. S.
Correlation between Word Memory Test (WMT) and the MMPI-2 Restructured Clinical (RC) scales
Archives of Clinical Neuropsychology
 
2005
20
892
Wygant
D. B.
Sellbom
M.
Ben-Porath
Y. S.
Stafford
K. P.
Freeman
D. B.
Heilbronner
R. L.
The relation between symptom validity testing and MMPI-2 scores as a function of forensic evaluation context
Archives of Clinical Neuropsychology
 
2007
22
489
499
Wygant
D. B.
Sellbom
M.
Gervais
R. O.
Ben-Porath
Y. S.
Stafford
K. P.
Freeman
D. B.
et al.  
Further validation of the MMPI-2 and MMPI-2-RF Response Bias Scale: Findings from disability and criminal forensic settings
Psychological Assessment
 
2010
22
745
Wynkoop
T. F.
Denney
R. L.
Test review: Green's Word Memory Test (WMT) for windows
Journal of Forensic Neuropsychology
 
2005
4
3
101
105
Youngjohn
J. R.
Wershba
R.
Stevenson
M.
Sturgeon
J.
Thomas
M. L.
Independent validation of the MMPI-2-RF somatic/cognitive and validity scales in TBI litigants tested for effort
The Clinical Neuropsychologist
 
2011
25
463
476