Abstract

Although recent findings have indicated that a portion of college students presenting for psychoeducational evaluations fail validity measures, methods for determining the validity of cognitive test results in psychoeducational evaluations remain under-studied. In light of this, data are needed to evaluate utility of validity indices in this population and to provide base rates for students meeting research criteria for malingering and to report the relationship between testing performance and the level of external incentive. The authors utilized archival data from: (i) a university psychological clinic (n = 986) and (ii) a university control sample (n = 182). Empirically supported embedded validity indices were utilized to identify retrospectively suspected malingering patients. Group performance, according to invalidity and the level of incentive seeking, was evaluated through a series of multivariate mean comparisons. The current study supports classifying patients according to the level of incentive seeking when evaluating neurocognitive performance and feigned/exaggerated deficits.

Introduction

Clinical neuropsychologists routinely perform assessments when patients have external incentives that may affect neuropsychological test performance (Binder & Rohling, 1996; Green, Rohling, Lees-Haley, & Allen, 2001; Nelson et al., 2010; Slick, Sherman, & Iverson, 1999). As a result, several researchers and policy edicts have supported the use of instruments to identify, quantify, or otherwise account for negative response bias or effort in illegitimate/exaggerated claims (e.g., identify those with invalid or malingered performance; Bush et al., 2005; Heilbronner et al., 2009). Poor effort or response bias is common in some settings (e.g., disability, litigation), although the rate is typically much lower in patients not involved with litigation or seeking financial compensation (Ardolf, Denney, & Houston, 2007; Delis & Wetter, 2007; Gouvier, Cubic, Jones, & Brantley, 1992; Kirmayer & Sartorius, 2007; Lees-Haley, Earnest, & Dolezal-Wood, 1995; Mittenberg, Patton, Canyock, & Condit, 2002). While it is widely accepted that symptom validity should be assessed in a forensic context, the need to evaluate the possible role of external motivation in compromising the validity of test results in nonforensic settings has been emphasized to a lesser degree.

For instance, if students demonstrate functional limitations due to neurocognitive (e.g., Attention-Deficit Hyperactivity Disorder [ADHD], Learning Disorder [LD]) or psychological disorders, educational institutions are obliged to provide them with an accommodating academic environment (per the Rehabilitation Act of 1973) by allowing time on exams and assignments, tests in special settings (e.g., a quiet room), alternate response formats on exams (e.g., marking on test forms rather than a Scantron response sheet), or note-takers (Evans, Serpell, Schultz, & Pastor, 2007; Hadley, 2007). Another possible external incentive for students is to obtain medications (e.g., amphetamine derivatives; Pary et al., 2002; Peterson, McDonagh, & Rongwei, 2008), which may aid academic performance. Just as prescription rates for psychostimulants for attentional disorders has increased (McCabe, Teter, Boyd, & Guthrie, 2004; Woodworth, 2000), nonprescription and nonmedical adoption (e.g., recreational abuse, gaining an academic edge) of such drugs have risen in normative student populations (Advokat, Guidry, & Martino, 2008; Babcock & Byme, 2000; McCabe, Teter, & Boyd, 2006; Moline & Frankenberger, 2001; White, Becker-Blease, & Grace-Bishop, 2006).

Research has indicated that ADHD simulators self-report more or similar levels of attentional symptoms compared with ADHD patients (Fisher & Watkins, 2008; Harrison, Edwards, & Parker, 2007; Jachimowicz & Geiselman, 2004; Quinn, 2003, Sollman, Ranseen, & Berry, 2010; Tucha, Sontag, Walitza, & Lange, 2009). Further, studies have indicated that those who feign ADHD or LD show high failure rates on symptom validity tests (SVTs) (Booksh, Pella, Singh, & Gouvier, 2010; Frazier, Frazier, Busch, Kerwood, & Demaree, 2008; Sollman et al., 2010) and show suppressed neurocognitive performance (Harrison et al., 2007; Marshall et al., 2010; Sollman et al., 2010; Suhr, Hammers, Dobbins-Buckland, Zimak, & Hughes, 2008; Suhr, Sullivan, & Rodriguez, 2011). For instance, Sullivan, May, and Galbally (2007) reported that patients evaluated for LD and ADHD showed a 22.4% overall failure rate on the Word Memory Test (Green, 2003) and similar results have been reported elsewhere (Marshall et al., 2010; Suhr et al., 2008, 2011). Marshall and colleagues (2010) also noted that 17% of patients diagnosed with ADHD by their clinic avoided detection and obtained an ADHD diagnosis at the time of evaluation. Taken together with the other SVT research in this population, the base rate for invalid performance among individuals seeking psychoeducational evaluations seems higher than in typical medical settings without external incentives, but similar to rates found in patient groups seeking compensation or disability payments.

Specific to LD assessment, Osmon, Plambeck, Klein, and Mano (2006) reported that the Word Memory Test performed well in a simulation study of feigned reading disorders. Moreover, the authors indicated that their newly developed Word Reading Test showed promise as a specialized SVT for feigned reading disorders and dyslexic conditions, which has received support elsewhere (Lindstrom, Lindstrom, Coleman, Nelson, & Greg, 2009). Similarly, favorable results have been shown for the Dyslexia Assessment of Simulation (DASH) in differentiating simulated from bonafide reading disorders (Harrison, Edwards, Armstrong, & Parker, 2010; Harrison, Edwards, & Parker, 2008). Though they have potential as unique indicators of validity, neither of those specialized SVTs is readily available nor have they been well validated in clinical samples.

Purpose and Rationale

Reports of high SVT failure rates have only recently been noted in young adults seeking psychoeducational evaluation and large control samples have been absent. Further, research has not adequately accounted for such patients' level of external incentive in the context of criteria for invalid performance (i.e., Slick and colleagues, 1999, criteria [Slick Criteria] for malingered neurocognitive dysfunction [MND]). Without more information regarding evaluation context, it is unclear if failure rates in this population are related to seeking incentives. While it is rational to assume that overt medication-seeking and/or academic accommodation-seeking may affect neuropsychological performance, symptom reporting, or SVT failure, this has not been investigated directly. Such a lack of knowledge regarding the relationship of external incentives, SVT performance, and neuropsychological testing in this population is a concern because there has been a significant rise in the number of adults and post-secondary students complaining of cognitive problems (Nichols, Harrison, McCloskey, & Weintraub, 2002) and the number of individuals younger than 22 years of age in federally funded educational programs for the disabled has doubled within the past 20 years (U.S. Department of Education, 2005). Therefore, there is a growing need to ensure that social programs are provided to deserving individuals.

In general, the above simulation and clinical studies have indicated that self-report and objective cognitive instruments along with stand-alone SVTs may be helpful in detecting feigned performance in the ADHD and LD population. Although embedded validity measures have several advantages (Boone, 2009; Mathias, Greve, Bianchini, Houston, & Crouch, 2002), they have not been well examined in student samples, often correlate with age, ability, intelligence, and years of education, and may have high false-positive rates in certain samples (Harrison, Rosenblum, & Currie, 2010). These are particularly relevant issues as neuropsychologists often assess academic-related disorders with measures containing embedded indices that are sensitive to response bias and effort (Crank & Deshler, 2001). Nevertheless, a literature review failed to uncover published work reporting the wide range of WAIS-III/WMS-III embedded indices in a university sample. This is germane given that the Wechsler measures are among the most utilized tests by psychologists (Rabin, Barr, & Burton, 2005).

To address the current literature gap, the current study explored the association of nonfinancial external gain and popular neuropsychological tests and embedded validity indices with the expectation that performance of young adults overtly seeking nonfinancial incentives is poorer and less valid than that of individuals not seeking incentives. Moreover, it was hypothesized that those seeking external incentives demonstrate particularly lower scores on measures of working memory and processing speed as suggested by previous research in patients showing invalid performance. The authors also considered overt medication-seeking and/or academic accommodation-seeking in context of the Slick Criteria for MND as these criteria have rarely been applied to samples without financial incentives. Finally, since clinicians are not adept at recognizing invalid performance based on behavioral observations and subjective data (Heaton, Smith, Lehman, & Vogt, 1978), it was expected that patients seeking external gain and those who meet the Slick Criteria would obtain a diagnosis and receive sought after recommendations at a high rate.

Methods

Participants

The authors included archival data from patients (n = 986) presenting to a Psychological Services Center at a southern university who were typically self-referred due to academic problems. To reduce the chance of false-positive findings on the following embedded measures, exclusion criteria included visual and/or hearing impairment, current psychiatric or neurological disorders warranting legal disability status, history of moderate–severe traumatic brain injury, stroke, dementia, chronic/severe neurological condition(s), or English as a second language. The mean age in the clinical sample was 22.62 (SD = 6.80) with 513 (52.0%) men and 473 (48.0%) women. Ethnic groups included Caucasian (n = 836, 84.7%), African American (n = 112, 11.3%), Asian (n = 6, 0.6%), Hispanic (n = 18, 1.8%), Middle-Eastern (n = 6, 0.6%), and other (n = 8, 0.8%). Demographic characteristics of the clinical sample per Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR; APA, 2000) diagnostic category are presented in Table 1.

Table 1.

Frequency and percentage of cases by primary diagnostic category by clinical group

 No external incentive
 
External incentive
 
Total
 
 n n n 
Depressive disorders 54 5.5 21 2.1 75 7.6 
Anxiety disorders 71 7.2 43 4.4 114 11.6 
Adjustment disorders 0.7 0.5 12 1.2 
Bipolar disorders 0.9 0.6 15 1.5 
Other mood disorders 0.2 0.2 0.4 
ADHD 85 8.6 151 15.3 236 23.9 
Learning disorders 58 5.9 115 11.7 173 17.5 
Academic V-code 0.6 0.6 12 1.2 
Cognitive disorders 34 3.4 50 5.1 84 8.5 
Eating disorders 0.1 0.1 0.2 
Substance use disorders 0.7 0.2 0.9 
Misc. V-codes 0.3 0.2 0.5 
Information unavailable 0.3 0.2 0.5 
Other 0.7 0.9 16 1.6 
Diagnosis deferred 0.9 0.6 15 1.5 
No diagnosis 122 12.4 87 8.8 209 21.2 
Total 478 48.5 508 51.5 986 100 
 No external incentive
 
External incentive
 
Total
 
 n n n 
Depressive disorders 54 5.5 21 2.1 75 7.6 
Anxiety disorders 71 7.2 43 4.4 114 11.6 
Adjustment disorders 0.7 0.5 12 1.2 
Bipolar disorders 0.9 0.6 15 1.5 
Other mood disorders 0.2 0.2 0.4 
ADHD 85 8.6 151 15.3 236 23.9 
Learning disorders 58 5.9 115 11.7 173 17.5 
Academic V-code 0.6 0.6 12 1.2 
Cognitive disorders 34 3.4 50 5.1 84 8.5 
Eating disorders 0.1 0.1 0.2 
Substance use disorders 0.7 0.2 0.9 
Misc. V-codes 0.3 0.2 0.5 
Information unavailable 0.3 0.2 0.5 
Other 0.7 0.9 16 1.6 
Diagnosis deferred 0.9 0.6 15 1.5 
No diagnosis 122 12.4 87 8.8 209 21.2 
Total 478 48.5 508 51.5 986 100 

For comparison purposes, the study also included data from a separate construct validation research project (Shelton, Elliott, Matthews, Hill, & Gouvier, 2010) utilizing a nonclinical university control sample (n = 182). Exclusion criteria for the control sample included visual and/or hearing impairment, a psychiatric diagnosis resulting in cognitive impairment, or English as a second language. The mean age in the control sample was 20.56 (SD = 3.67) with 47 (25.8%) men and 135 (74.2%) women. Ethnicities in the control sample included Caucasian (n = 149, 81.9%), African American (n = 15, 8.2%), Hispanic (n = 6, 3.3%), and other (n = 12, 6.6%). All clinical and control participants consented to have their data used in research, and this project was reviewed and approved by an Institutional Review Board.

Grouping Variables

To establish the group status in the clinical sample, charts were reviewed to determine each individual's expressed reason for obtaining an evaluation. Of particular interest, when clinical demographic and/or referral information included patients’ expressed intent to obtain recommendations for academic accommodations and/or stimulant medication consultation, they were categorized into the External Incentive group (n = 508). Of those in the External Incentive group, 428 (84.3%) were seeking academic accommodations, 50 (9.8%) were seeking medications, and 30 (5.9%) were seeking medications and accommodations. If no such information was available, patients were assigned to the No External Incentive group (n = 478) to avoid potential false-positive MND cases. Therefore, three groups were examined in this study: (i) Control, (ii) No External Incentive, and (iii) External Incentive. Diagnostic rates in each clinical group (No External Incentive, External Incentive) are found in Table 1.

Measures

As part of a comprehensive evaluation, individuals were administered tests of intelligence, memory, attention/concentration, academic achievement, and personality/emotional functioning. For the purposes of this study, only data regarding intellectual and memory functioning were analyzed for comparisons between all groups. While the final clinical sample was 986, the patients were administered a fixed/flexible battery; therefore, the individual test-by-test analyses may not employ the entire sample, but be limited to participants who completed the particular measure of interest. Further, control participants were not administered personality testing, which precluded group comparisons on that variable. Measures included the Wechsler Adult Intelligence Scale—Third Edition (WAIS-III; Wechsler, 1997a), Wechsler Memory Scale—Third Edition (WMS-III; Wechsler, 1997b), and Personality Assessment Inventory (PAI; Morey, 1991). All measures were administered and scored according to standardization procedures.

Malingering Index Scores and Invalid Classification

Each embedded index score was calculated according to the methodology described by the researchers cited in Table 2. The indices were then used to apply the Slick Criteria to individual cases to identify MND. As described elsewhere (Babikian, Boone, Lu, & Arnold, 2006), evidence from two or more measures fulfilling Criterion B (direct evidence of a response bias from objective neuropsychological testing) level evidence of the Slick Criteria in the context of external incentives (Criterion A) suggests probable MND. Therefore, performance on any two or more embedded indices from the WAIS-III/WMS-III is sufficient to meet Criterion B-level evidence. Another method to classify a patient as a probable MND case is to use evidence from at least one of the WAIS-III/WMS-III indices and self-report evidence (PAI) of a response bias (Criterion C of the Slick Criteria). Similar classification schemes have been reported elsewhere (Greve et al., 2007; Heinly, Greve, Bianchini, Love, & Brennan, 2005).

Table 2.

Cutoff scores corresponding to index

Index Cutoff score 
Reliable Digit Span (Greiffenstein et al., 1994≤6 
Mittenberg Index (Mittenberg et al., 1995>0.21 
Vocabulary minus Digit Span (Mittenberg et al., 1995≥4 
Maximum Digits Forward (Babikian et al., 2006; Heinly et al., 2005≤4 
Age-Corrected Scale Score (Iverson, 1991; Iverson & Franzen, 1994≤4 
Processing Speed Index (Etherton, Bianchini, & Heinly, Greve, 2006≤70 
Rarely Missed Index (Killgore & DellaPietra, 2000b≤40 
WAIS-III Working Memory (Etherton, Bianchini, Ciota, Heinly, & Greve, 2006≤70 
Faces I (Glassmire et al., 2003≤23 
Auditory Delayed Recognition (Langeluddecke & Lucas, 2003≤42 
Ord et al. Index (Ord et al., 2008≥3 
Negative Impression Management t-score (Morey, 1991≥81t 
Rogers' Discriminant Function t-score (Rogers et al., 1996≥60t 
Malingering Index (Morey, 1991≥3 
Index Cutoff score 
Reliable Digit Span (Greiffenstein et al., 1994≤6 
Mittenberg Index (Mittenberg et al., 1995>0.21 
Vocabulary minus Digit Span (Mittenberg et al., 1995≥4 
Maximum Digits Forward (Babikian et al., 2006; Heinly et al., 2005≤4 
Age-Corrected Scale Score (Iverson, 1991; Iverson & Franzen, 1994≤4 
Processing Speed Index (Etherton, Bianchini, & Heinly, Greve, 2006≤70 
Rarely Missed Index (Killgore & DellaPietra, 2000b≤40 
WAIS-III Working Memory (Etherton, Bianchini, Ciota, Heinly, & Greve, 2006≤70 
Faces I (Glassmire et al., 2003≤23 
Auditory Delayed Recognition (Langeluddecke & Lucas, 2003≤42 
Ord et al. Index (Ord et al., 2008≥3 
Negative Impression Management t-score (Morey, 1991≥81t 
Rogers' Discriminant Function t-score (Rogers et al., 1996≥60t 
Malingering Index (Morey, 1991≥3 

In accordance with current recommendations (Bianchini, Mathias, & Greve, 2001; Greve & Bianchini, 2004), the authors used cut scores validated in known malingering groups with high specificity (≥90%, see Table 2) for a number of clinical groups (e.g., TBI, neurologic, pain, psychiatric, etc.) and controls (see associated references in Table 2). It should be noted that such cut scores should be used with caution when applying them to groups with severe impairments, such as dementia or intellectual disabilities in order to guard against false positives. In keeping with this conservative approach, indices that share subtest values in their calculation (e.g., Mittenberg Index and Reliable Digits) were used to fulfill only one of the two Criterion B-level findings per case (Larrabee, 2008; Rosenfeld, Sands, & Van Gorp, 2000). The classification of MND used in the current study was not employed at the time of the clinical evaluations; therefore, no patient was determined to be malingering at the time of their evaluation in the clinic.

Results

Group Comparisons

Two separate one-way MANOVAs were conducted to test the hypothesis that the External Incentive group showed lower performance on the WAIS-III and WMS-III compared with the No External Incentive and Control groups. Full-Scale Intelligence Quotient (FSIQ) Standard Scores were generally in the average range: Control (M = 110.13, SD = 11.32), No External Incentive (M = 105.62, SD = 12.60), and External Incentive (M = 101.63, SD = 12.47). The first MANOVA included Group (Control, External Incentive, and No External Incentive) X WAIS-III subtests, revealing a significance between group main effect, F (13, 1114) = 11.03, p < .001, η2 = .12. As can be seen from the post-hoc comparisons (Table 3), group differences fell in the expected direction with the External Incentive group scoring lower than the other two groups across subtests.

Table 3.

Univariate analysis of variance for WAIS-III variables by group (clinical vs. control)

 Control (n = 182) No external incentive (n = 442) External incentive (n = 490) F p Partial η2 
 Mean (SDMean (SDMean (SD
Vocabularya,b,c 13.01 (2.37) 11.77 (2.67) 11.10 (2.85) 33.51 .001 .07 
Similaritiesb 11.61 (2.71) 11.03 (2.81) 10.64 (2.88) 8.09 .001 .01 
Arithmeticb,c 10.74 (2.36) 10.56 (2.60) 9.65 (2.93) 17.42 .001 .03 
Digit Spana,b,c 11.41 (2.73) 10.08 (2.78) 9.34 (2.70) 38.49 .001 .07 
Informationb,c 11.70 (2.41) 11.39 (2.65) 10.60 (2.75) 15.83 .001 .03 
Comprehension 11.91 (2.47) 11.66 (2.68) 11.37 (2.87) 3.01 .050 .01 
Letter-Number Seq.a,b,c 11.45 (2.72) 10.59 (2.62) 9.86 (2.64) 25.68 .001 .04 
Picture Completionb,c 10.73 (3.09) 10.54 (2.95) 10.06 (3.00) 4.72 .009 .01 
Digit Symbol Cda,b,c 11.54 (2.53) 9.36 (2.51) 8.78 (2.65) 76.82 .001 .12 
Block Designb,c 11.20 (2.81) 10.79 (2.84) 10.24 (3.01) 8.64 .001 .02 
Matrix Reasoningb,c 12.21 (2.07) 11.91 (2.58) 11.34 (2.71) 9.91 .001 .02 
Picture Arrangement 10.37 (2.75) 10.21 (2.64) 10.01 (2.82) 1.34 .263 .002 
Symbol Searcha,b,c 12.04 (2.43) 10.06 (2.71) 9.39 (2.73) 64.88 .001 .11 
 Control (n = 182) No external incentive (n = 442) External incentive (n = 490) F p Partial η2 
 Mean (SDMean (SDMean (SD
Vocabularya,b,c 13.01 (2.37) 11.77 (2.67) 11.10 (2.85) 33.51 .001 .07 
Similaritiesb 11.61 (2.71) 11.03 (2.81) 10.64 (2.88) 8.09 .001 .01 
Arithmeticb,c 10.74 (2.36) 10.56 (2.60) 9.65 (2.93) 17.42 .001 .03 
Digit Spana,b,c 11.41 (2.73) 10.08 (2.78) 9.34 (2.70) 38.49 .001 .07 
Informationb,c 11.70 (2.41) 11.39 (2.65) 10.60 (2.75) 15.83 .001 .03 
Comprehension 11.91 (2.47) 11.66 (2.68) 11.37 (2.87) 3.01 .050 .01 
Letter-Number Seq.a,b,c 11.45 (2.72) 10.59 (2.62) 9.86 (2.64) 25.68 .001 .04 
Picture Completionb,c 10.73 (3.09) 10.54 (2.95) 10.06 (3.00) 4.72 .009 .01 
Digit Symbol Cda,b,c 11.54 (2.53) 9.36 (2.51) 8.78 (2.65) 76.82 .001 .12 
Block Designb,c 11.20 (2.81) 10.79 (2.84) 10.24 (3.01) 8.64 .001 .02 
Matrix Reasoningb,c 12.21 (2.07) 11.91 (2.58) 11.34 (2.71) 9.91 .001 .02 
Picture Arrangement 10.37 (2.75) 10.21 (2.64) 10.01 (2.82) 1.34 .263 .002 
Symbol Searcha,b,c 12.04 (2.43) 10.06 (2.71) 9.39 (2.73) 64.88 .001 .11 

Note: All Flagged Post-Hoc Comparisons, p < .05.

aControl versus no external incentive.

bControl versus external incentive.

cExternal incentive versus no external incentive.

Mean scores for the WMS-III General Memory Index by group were as follows: Control (M = 107.62, SD = 10.69), No External Incentive (M = 100.97, SD = 13.98), and External Incentive (M = 98.51, SD = 13.96). The second MANOVA included Group X WMS-III core subtests, indicating a significant between group main effect, F (8, 1151) = 4.44, p < .001, η2 = .03 (Table 4). While mean group scores on all of the measures were in the expected direction with the External Incentive group scoring lower than the other two groups, the External Incentive group performed only significantly lower on the Logical Memory subtests.

Table 4.

Univariate analysis of variance for WMS-III variables by group (clinical vs. control)

 Control (n = 182) No External incentive (n = 466) External incentive (n = 503) F p Partial η2 
 Mean (SDMean (SDMean (SD
Logical Memory Ia,b,c 11.13 (2.53) 10.09 (2.84) 9.55 (2.81) 21.81 .001 .04 
Faces Ib 10.38 (2.92) 9.93 (2.93) 9.60 (2.98) 4.88 .008 .01 
Verbal Paired Ass. Ib 10.67 (2.51) 10.32 (2.93) 9.98 (2.97) 4.18f .016 .01 
Family Pictures Ia,b 10.90 (2.40) 10.05 (3.11) 9.82 (3.18) 8.43 .001 .01 
Logical Memory IIa,b,c 11.76 (2.66) 10.41 (2.95) 9.84 (2.84) 30.19 .001 .05 
Faces IIa,b 10.41 (2.58) 9.83 (2.67) 9.79 (2.79) 3.73 .024 .01 
Verbal Paired Ass. IIa,b 11.19 (1.75) 10.59 (2.55) 10.42 (2.69) 6.33f .002 .01 
Family Pictures IIa,b 10.81 (2.43) 9.85 (3.23) 9.64 (3.26) 9.54 .001 .02 
 Control (n = 182) No External incentive (n = 466) External incentive (n = 503) F p Partial η2 
 Mean (SDMean (SDMean (SD
Logical Memory Ia,b,c 11.13 (2.53) 10.09 (2.84) 9.55 (2.81) 21.81 .001 .04 
Faces Ib 10.38 (2.92) 9.93 (2.93) 9.60 (2.98) 4.88 .008 .01 
Verbal Paired Ass. Ib 10.67 (2.51) 10.32 (2.93) 9.98 (2.97) 4.18f .016 .01 
Family Pictures Ia,b 10.90 (2.40) 10.05 (3.11) 9.82 (3.18) 8.43 .001 .01 
Logical Memory IIa,b,c 11.76 (2.66) 10.41 (2.95) 9.84 (2.84) 30.19 .001 .05 
Faces IIa,b 10.41 (2.58) 9.83 (2.67) 9.79 (2.79) 3.73 .024 .01 
Verbal Paired Ass. IIa,b 11.19 (1.75) 10.59 (2.55) 10.42 (2.69) 6.33f .002 .01 
Family Pictures IIa,b 10.81 (2.43) 9.85 (3.23) 9.64 (3.26) 9.54 .001 .02 

Note: All flagged post-hoc comparisons, p<.05.

aControl versus no external incentive.

bControl versus external incentive.

cExternal incentive versus no external incentive.

The next series of analyses tested the hypothesis that the External Incentive group performed differently from the other groups on validity indices, indicating a higher level of invalid performance in those patients. Group again served as the independent variable and each of the WAIS-III/WMS-III validity indices were dependent variables in a one-way MANOVA, resulting in a between group main effect, F (11, 1100) = 12.34, p < .001, η2 = .11 (Table 5). Since the Ord and colleagues (2007) and Rarely Missed Indices from the WMS-III violated conservative conventions for skewness and/or kurtosis (≤ −1 or ≥1), nonparametric comparisons were conducted with those variables. Omnibus one-way Kruskal–Wallis ANOVA revealed that group significantly affected the level of performance on the Ord and colleagues’ index, χ2 (2, 1152) = 33.27, p < .001. Follow-up pairwise comparisons were conducted with Mann–Whitney U, controlling α for multiple comparisons that established significance at p < .017. Control participants performed lower than those in the No External Incentive group, U (1, 648) = 36,415.0, p < .001, and those in the External Incentive group, U (1, 685) = 37,069.0, p < .001. However, the No External Incentive and External Incentive groups did not differ, U (1, 969) = 111,992.5, p = .07. An omnibus one-way Kruskal–Wallis ANOVA revealed that the group status was not significantly associated with the Rarely Missed Index, χ2 (2, 1146) = 0.58, p = .75.

Table 5.

Univariate analysis of variance for WAIS-III/WMS-III validity indices by group (clinical vs. control)

 Control (n = 182) No external incentive (n = 435) External incentive (n = 487) F p Partial η2 
 Mean (SDMean (SDMean (SD
WAIS-III WMIa,b,c 106.79 (12.73) 102.25 (13.43) 97.38 (13.55) 36.48 .001 .06 
Processing Speed Indexa,b,c 109.81 (12.30) 98.40 (12.82) 94.82 (12.99) 91.02 .001 .14 
Auditory Recognition-Delayed Rawa,b,c 50.81 (2.31) 49.80 (2.75) 49.30 (2.99) 19.67 .001 .04 
WMS-III Faces I Rawb 39.23 (4.22) 38.23 (4.60) 37.57 (4.86) 8.63 .001 .02 
Max. Digits Fwd.a,b,c 7.20 (1.08) 6.72 (1.28) 6.50 (1.24) 21.16 .001 .04 
Mittenberg Index −0.41 (.96) −0.40 (1.05) −0.25 (1.01) 3.03 .049 .01 
Reliable Digitsa,b,c 10.95 (2.04) 10.04 (2.14) 9.40 (2.12) 37.39 .001 .06 
Vocabulary-Digit Span 1.60 (2.76) 1.72 (3.33) 1.78 (3.36) .20 .816 .001 
 Control (n = 182) No external incentive (n = 435) External incentive (n = 487) F p Partial η2 
 Mean (SDMean (SDMean (SD
WAIS-III WMIa,b,c 106.79 (12.73) 102.25 (13.43) 97.38 (13.55) 36.48 .001 .06 
Processing Speed Indexa,b,c 109.81 (12.30) 98.40 (12.82) 94.82 (12.99) 91.02 .001 .14 
Auditory Recognition-Delayed Rawa,b,c 50.81 (2.31) 49.80 (2.75) 49.30 (2.99) 19.67 .001 .04 
WMS-III Faces I Rawb 39.23 (4.22) 38.23 (4.60) 37.57 (4.86) 8.63 .001 .02 
Max. Digits Fwd.a,b,c 7.20 (1.08) 6.72 (1.28) 6.50 (1.24) 21.16 .001 .04 
Mittenberg Index −0.41 (.96) −0.40 (1.05) −0.25 (1.01) 3.03 .049 .01 
Reliable Digitsa,b,c 10.95 (2.04) 10.04 (2.14) 9.40 (2.12) 37.39 .001 .06 
Vocabulary-Digit Span 1.60 (2.76) 1.72 (3.33) 1.78 (3.36) .20 .816 .001 

Note: All flagged post-hoc comparisons p < .05.

aControl versus no external incentive.

bControl versus external incentive.

cExternal incentive versus no external incentive.

A separate MANOVA was conducted comparing the External Incentive and No External Incentive groups on the PAI indices, indicating a significance between group main effect, F (3, 657) = 7.35, p < .001, η2 = .03. While the External Incentive and No External Incentive group did not differ on the Malingering Index from the PAI, the External Incentive group scored significantly higher on Rogers’ Discriminant Function. Unexpectedly, the No External Incentive group demonstrated significantly higher mean scores on the Negative Impression Management scale, counter to the hypothesis (Table 6).

Table 6.

Univariate analysis of variance for PAI validity indices by clinical group (no external incentive versus external incentive)

 No external incentive (n = 305) External incentive (n = 356) 
 Mean (SDMean (SD
Rogers' function** 50.15 (10.15) 53.20 (10.37) 
Malingering index 0.79 (.87) 0.78 (.89) 
Neg. Impression Mgt.* 55.15 (12.12) 53.32 (11.03) 
 No external incentive (n = 305) External incentive (n = 356) 
 Mean (SDMean (SD
Rogers' function** 50.15 (10.15) 53.20 (10.37) 
Malingering index 0.79 (.87) 0.78 (.89) 
Neg. Impression Mgt.* 55.15 (12.12) 53.32 (11.03) 

*p < .05, **p < .001.

Classification of Invalid Performance

As can be seen in Table 7, when comparing the External Incentive and No External Incentive groups, the proportion of patients failing indices differed on only three measures (Auditory Recognition-Delayed Raw, Mittenberg Index, and Reliable Digits). Table 8 contains the cumulative percentage of WAIS-III/WMS-III validity indices failed per group. While 46.3% of patients in the External Incentive group failed at least one index from the WAIS-III/WMS-III, this occurred in 36.8% of the No External Incentive group, χ2 (1) = 9.0, p = .003. The high failure rate in the No External Incentive group suggests that some indices are overly sensitive in this population. When using the WAIS-III/WMS-III indices alone, just 17 (3.5%) of patients in the External Incentive group and 9 (.9%) patients in the No External Incentive group satisfied Criterion B of the Slick Criteria, χ2 (1) = 2.1, p = .15. Therefore, just 26 (3%) of the entire clinical group met Criterion B evidence when using only the WAIS-III/WMS-III indices, which increased to 10.2% when including the PAI indices. The rate of probable MND, when accounting for all indices (including the PAI), was also lower than that expected (n = 59, 12.1% of the External Incentive group or 5.98% of the entire clinical sample). Although patients in the No External Incentive group did not satisfy Criterion A (presence of a substantial external incentive) of the Slick Criteria, 42 (9.7%) individuals in that group did satisfy Slick Criterion B, which was unexpectedly not significantly different from the External Incentive group (12.1%), χ2 (1) = 2.1, p = .14. None of the control participants’ combination of index scores met Criterion B of the Slick Criteria. For descriptive purposes, means for the WAIS-III and WMS-III are reported for individuals who did not fail any indices, individuals failing only one index, and participants failing two or more indices (Tables 9 and 10).

Table 7.

Participants scoring beyond cutoff scores for validity indices by group*

 Control No external incentive External incentive 
 n (%) n (%) n (%) 
WAIS-III WMI 0 (0) 2 (0.4) 6 (1.2) 
Processing Speed Index 0 (0) 4 (0.9) 6 (1.2) 
Digit Span Scale Score 0 (0) 4 (0.9) 9 (1.8) 
Auditory Recognition-Delayed Rawb 0 (0) 3 (0.6) 15 (3.0) 
WMS-III Faces I Raw 0 (0) 1 (0.2) 5 (1.0) 
Max. Digits Fwd. 3 (1.6) 10 (2.2) 16 (3.2) 
Mittenberg Indexb 42 (23.1) 125 (28) 170 (34.1) 
Reliable Digitsb 2 (1.1) 12 (2.6) 32 (6.4) 
Vocabulary-Digit Span 40 (22) 131 (27.3) 148 (29.5) 
Ord et al. Index 0 (0) 12 (2.6) 20 (4.0) 
Rarely Missed Index 0 (0) 2 (0.4) 1 (0.2) 
Rogers’ Functiona – 59 (19.3) 81 (22.8) 
Malingering Indexa – 12 (3.9) 16 (4.5) 
Neg. Impression Mgta – 20 (6.6) 16 (4.5) 
 Control No external incentive External incentive 
 n (%) n (%) n (%) 
WAIS-III WMI 0 (0) 2 (0.4) 6 (1.2) 
Processing Speed Index 0 (0) 4 (0.9) 6 (1.2) 
Digit Span Scale Score 0 (0) 4 (0.9) 9 (1.8) 
Auditory Recognition-Delayed Rawb 0 (0) 3 (0.6) 15 (3.0) 
WMS-III Faces I Raw 0 (0) 1 (0.2) 5 (1.0) 
Max. Digits Fwd. 3 (1.6) 10 (2.2) 16 (3.2) 
Mittenberg Indexb 42 (23.1) 125 (28) 170 (34.1) 
Reliable Digitsb 2 (1.1) 12 (2.6) 32 (6.4) 
Vocabulary-Digit Span 40 (22) 131 (27.3) 148 (29.5) 
Ord et al. Index 0 (0) 12 (2.6) 20 (4.0) 
Rarely Missed Index 0 (0) 2 (0.4) 1 (0.2) 
Rogers’ Functiona – 59 (19.3) 81 (22.8) 
Malingering Indexa – 12 (3.9) 16 (4.5) 
Neg. Impression Mgta – 20 (6.6) 16 (4.5) 

*Reflects percentage of participants in analyses.

aIndex not administered to control participants.

bAll flagged chi-square comparisons: no external incentive versus external incentive, p < .05.

Table 8.

Number of WAIS-III/WMS-III indices failed according to percentage of participants beyond cutoff scores

 Control No external incentive External incentive Total clinical 
Number of indices failed n (%) n (%) n (%) n (%) 
123 (67.6) 302 (63.2) 273 (53.7) 575 (58.3) 
35 (19.2) 75 (15.7) 102 (20.1) 177 (18.0) 
21 (11.5) 79 (16.5) 96 (18.9) 175 (17.7) 
2 (1.1) 13 (2.7) 17 (3.3) 30 (3.0) 
1 (0.5) 7 (1.5) 15 (3.0) 22 (2.2) 
– 2 (0.4) 2 (0.4) 4 (0.4) 
 – 2 (0.4) 2 (0.2) 
  1 (0.2) 1 (0.1) 
  – – 
 Control No external incentive External incentive Total clinical 
Number of indices failed n (%) n (%) n (%) n (%) 
123 (67.6) 302 (63.2) 273 (53.7) 575 (58.3) 
35 (19.2) 75 (15.7) 102 (20.1) 177 (18.0) 
21 (11.5) 79 (16.5) 96 (18.9) 175 (17.7) 
2 (1.1) 13 (2.7) 17 (3.3) 30 (3.0) 
1 (0.5) 7 (1.5) 15 (3.0) 22 (2.2) 
– 2 (0.4) 2 (0.4) 4 (0.4) 
 – 2 (0.4) 2 (0.2) 
  1 (0.2) 1 (0.1) 
  – – 
Table 9.

Means for WAIS-III variables by failing a given number of indices

 Failing none (n = 695) Failing one (n = 210) Failing ≥2 (n = 26) 
 Mean (SDMean (SDMean (SD
Vocabulary 11.14 (2.48) 12.13 (2.97 10.33 (2.91) 
Similarities 10.49 (2.68) 11.28 (2.77) 10.29 (3.20) 
Arithmetic 10.53 (2.61) 10.11 (2.89) 7.08 (3.01) 
Digit Span 11.23 (2.57) 9.09 (2.01) 6.79 (1.56) 
Information 10.86 (2.61) 11.22 (2.66) 9.71 (3.46) 
Comprehension 11.29 (2.52) 11.74 (3.09) 10.29 (2.79) 
Letter-Number Seq. 11.09 (2.62) 9.99 (2.63) 7.58 (2.45) 
Picture Completion 10.45 (2.96) 10.40 (3.09) 8.71 (2.90) 
Digit Symbol Cd 9.68 (2.71) 9.26 (2.72) 6.63 (2.78) 
Block Design 10.88 (2.90) 10.52 (2.99) 8.63 (3.01) 
Matrix Reasoning 12.04 (2.50) 11.43 (2.63) 9.13 (2.54) 
Picture Arrangement 10.25 (2.69) 10.07 (2.87) 7.38 (2.10) 
Symbol Search 10.39 (6.46) 9.95 (2.61) 6.46 (2.93) 
 Failing none (n = 695) Failing one (n = 210) Failing ≥2 (n = 26) 
 Mean (SDMean (SDMean (SD
Vocabulary 11.14 (2.48) 12.13 (2.97 10.33 (2.91) 
Similarities 10.49 (2.68) 11.28 (2.77) 10.29 (3.20) 
Arithmetic 10.53 (2.61) 10.11 (2.89) 7.08 (3.01) 
Digit Span 11.23 (2.57) 9.09 (2.01) 6.79 (1.56) 
Information 10.86 (2.61) 11.22 (2.66) 9.71 (3.46) 
Comprehension 11.29 (2.52) 11.74 (3.09) 10.29 (2.79) 
Letter-Number Seq. 11.09 (2.62) 9.99 (2.63) 7.58 (2.45) 
Picture Completion 10.45 (2.96) 10.40 (3.09) 8.71 (2.90) 
Digit Symbol Cd 9.68 (2.71) 9.26 (2.72) 6.63 (2.78) 
Block Design 10.88 (2.90) 10.52 (2.99) 8.63 (3.01) 
Matrix Reasoning 12.04 (2.50) 11.43 (2.63) 9.13 (2.54) 
Picture Arrangement 10.25 (2.69) 10.07 (2.87) 7.38 (2.10) 
Symbol Search 10.39 (6.46) 9.95 (2.61) 6.46 (2.93) 
Table 10.

Means for WMS-III variables by failing a given number of indices

 Failing none (n = 210) Failing none (n = 695) Failing ≥2 (n = 26) 
 Mean (SDMean (SDMean (SD
Logical Memory I 10.24 (2.75) 9.89 (3.03) 6.69 (2.56) 
Faces I 10.02 (2.92) 9.56 (3.08) 7.88 (3.30) 
Verbal Paired Ass. I 10.26 (2.77) 10.16 (2.95) 7.54 (2.64) 
Family Pictures I 10.27 (2.98) 9.83 (3.24) 6.65 (2.87) 
Logical Memory II 10.65 (2.86) 10.19 (3.20) 6.81 (2.42) 
Faces II 9.99 (2.53) 9.88 (3.20) 7.54 (2.98) 
Verbal Paired Ass. II 10.78 (2.32) 10.40 (2.88) 8.04 (3.09) 
Family Pictures II 10.15 (3.11) 9.68 (3.27) 6.27 (3.13) 
 Failing none (n = 210) Failing none (n = 695) Failing ≥2 (n = 26) 
 Mean (SDMean (SDMean (SD
Logical Memory I 10.24 (2.75) 9.89 (3.03) 6.69 (2.56) 
Faces I 10.02 (2.92) 9.56 (3.08) 7.88 (3.30) 
Verbal Paired Ass. I 10.26 (2.77) 10.16 (2.95) 7.54 (2.64) 
Family Pictures I 10.27 (2.98) 9.83 (3.24) 6.65 (2.87) 
Logical Memory II 10.65 (2.86) 10.19 (3.20) 6.81 (2.42) 
Faces II 9.99 (2.53) 9.88 (3.20) 7.54 (2.98) 
Verbal Paired Ass. II 10.78 (2.32) 10.40 (2.88) 8.04 (3.09) 
Family Pictures II 10.15 (3.11) 9.68 (3.27) 6.27 (3.13) 

Diagnoses and Accommodations

It was expected that those meeting criteria for MND were frequently diagnosed with neurocognitive disorders (e.g., Cognitive Disorder NOS, ADHD, and LD) as an outcome of their evaluation. This hypothesis was investigated by subdividing the MND group and remaining the clinical group according to DSM-IV-TR diagnostic category. Forty-seven (79.7%) MND patients were diagnosed with a mental disorder according to their evaluation, which was not significantly different from the remainder of the total clinical sample (n = 693, 74.8%), χ2 (1) = 0.71, p = .40. In contrast, the rate of being diagnosed with a neurocognitive disorder in the MND group (n = 38, 64.4%) was higher than the remainder of the entire clinical group (n = 455, 49.1%), χ2 (1) = 5.2, p = .02. The proportion of those diagnosed with a neurocognitive disorder in the External Incentive group (n = 316, 62.2%) was also higher than the No External Incentive group (n = 183, 38.3%), χ2 (1) = 56.4, p < .001.

The proportion of recommendations for accommodations and/or medication consultation was higher in the External Incentive (n = 343, 71.0%) than the No External Incentive group (n = 211, 47.4%), χ2 (1) = 53.61, p = .001. Moreover, 43 (75.4%) patients in the MND group and 511 (58.7%) of the remaining patients received at least one of those recommendations, χ2 (1) = 6.25, p = .012. However, those meeting Slick Criteria for MND did not receive those recommendations at a higher rate than individuals in the remaining External Incentive group (n = 300, 70%).

Discussion

Evaluating performance validity is crucial when conducting neuropsychological evaluations, especially in the context of potential external gain (Bianchini et al., 2001; Bush et al., 2005; Mittenberg et al., 2002; Slick et al., 1999). It has recently been noted that there are external incentives in academic settings, that disorders such as ADHD and LDs can be simulated, and that a high proportion of evaluated college students fail SVTs. Although evaluations often occur in the context of students seeking external incentives (e.g., academic accommodations and/or stimulant medication), the influence of such incentives has not been previously studied.

Results from the current study supported the notion that patients who overtly seek evaluations to obtain academic accommodations and/or a stimulant medications show suppressed test scores compared with patients who did not make this intention known. Although performance was generally lower on IQ and memory tests, the largest group differences were among subtests related to working memory and processing speed, which is consistent with what other researchers have suggested (Harrison et al., 2007; Marshall et al., 2010; Sollman et al., 2010; Suhr et al., 2008, 2011). Further, when compared with the No External Incentive group, the External Incentive group performed significantly toward the invalid range on half of the WAIS-III/WMS-III indices, although indices from the PAI were less consistently related to the incentive level.

While not all group comparisons revealed significant parametric differences, all comparisons fell in the expected direction, indicating that the External Incentive group had poorer neuropsychological performance and a higher rate of failing embedded measures. This likely suggests that, although the External Incentive group performed to a greater degree in the direction of invalidity on each index, the difference may not be as clinically meaningful for some indices, which raises the possibility that some of the indices do not function well in this population. The parametric nonsignificance of some analyses may also be due to an experimental artifact in the grouping variables. For instance, the level of external incentive seeking was determined via retrospective chart reviews wherein information regarding patients' true intentions for obtaining an evaluation may not have been, at times, explicitly stated. Therefore, such patients were included in the No External Incentive group to avoid false-positive MND cases in keeping with the conservative approach in this study. Because of this, the No External Incentive group probably contained individuals who actually had an external incentive, but did not directly indicate this at the time of evaluation, resulting in a contaminated group that masked or diluted group differences. However, that experimental artifact likely bolsters the robustness of the significant group differences that were noted. This classification method, along with its challenges, is similar to typical nonforensic clinical practice where the clinician is often uncertain as to the patients’ explicit motivations for evaluation or possible incentives that may exist.

It was expected that the proportion of patients showing invalid performance would be similar to previous reports. Accordingly, a high proportion (41.6%) of patients failed at least one embedded index from the WAIS-III/WMS-III, which is consistent with single SVT failure rates in other clinical samples (Marshall et al., 2010; Victor, Boone, Serpa, Buehler, & Zeigler, 2009). Notably, this rate was above the single index failure rate from the Word Memory Test reported in similar samples (Suhr et al., 2008 = 31%; Sullivan et al., 2007 = 22%). Unexpectedly, control participants also had a high failure rate (32.4%) on single measures, suggesting that some indices were too sensitive in this sample. In contrast, the Word Memory Test has shown very few false positives in similar samples (Tan, Slick, Strauss, & Hultsch, 2002).

The Mittenberg and Vocabulary minus Digit Span Indices were particularly sensitive to failure, indicating poor specificity. Since both the Mittenberg and Vocabulary minus Digit Span indices are essentially based on score patterns, the high failure rates on those measures, particularly in controls, may indicate that discrepancy approaches or regression formulas are suboptimal in high functioning samples due to increased subtest scatter (Harrison et al., 2010; Hawkins & Tulsky, 2001; Saklofske, Tulsky, Wilkins, & Weiss, 2003). All other indices demonstrated an excellent specificity in controls (>0.98). This highlights that reliance on any single validity indicator with poor sensitivity or poor specificity may result in false findings of MND and that not all SVTs are equivalent (Boone, 2007; Bush et al., 2005; Larrabee, 2008; Meyers & Volbrecht, 2003; Rosenfeld et al., 2000; Victor et al., 2009).

As a result, failure on one of the embedded measures used in this study may increase the post-test odds of “noncredibility” according to the Slick Criteria, but does not necessarily confirm it. Rather, abnormal findings on singular embedded measures with low specificity used in this study may have utility as screening measures of invalid test performance as there is considerable literature supporting their usage in other samples. In any case, caution should be taken to guard against undue stigmatization when generalizing the literature related to the embedded measures to a university population.

Despite the seemingly high failure rate on individual indices, the proportion of patients satisfying criteria for probable MND was only 6% of the entire patient sample, although 10.2% of patients met Criterion B of the Slick Criteria (direct evidence of a response bias from objective neuropsychological testing on two or more measures). However, there was no difference between the No External Incentive and External Incentive groups meeting Criteria B evidence, suggesting a portion of this population displays suboptimal effort. Thus, the rate of invalid performance in this sample was far below the rate in forensic settings (Mittenberg et al., 2002) and lower than the 22% rate in other student samples reported by Suhr and colleagues (2011) and Marshall and colleagues (2010). It should also be noted that no control participant satisfied Criterion B of the Slick Criteria, resulting in 100% specificity for the current methodology in defining MND.

Aspects of this study support the suggestion by Marshall and colleagues (2010) that students who feign symptoms may not be labeled as doing so and succeed in obtaining a desired diagnosis. As such, 64% of patients who retrospectively met probable MND criteria for the current study were diagnosed with a neurocognitive disorder as a result of their clinical evaluation compared with 49% of those not meeting MND criteria. Moreover, patients with external incentives and patients meeting criteria for probable MND received recommendations for academic accommodations and/or a stimulant medication consultation at a high rate. Thus, there is the possibility that university students are able to successfully exaggerate symptoms to the level that they obtain desired outcomes and avoid raising suspicion when SVTs are not used.

This study has several limitations. Perhaps, the greatest limitation is that there was no independently validated patient group meeting Slick Criteria according to measures specifically validated for use with ADHD and LD samples. Because of this, it is not possible to definitively state that those meeting Slick Criteria according to the current methodology were, in fact, feigning deficits in order to receive an external gain. Nonetheless, the number of individuals meeting Slick Criteria in this study did not appear excessive and was actually lower than expected given estimates of invalid performance in other clinical settings with established external incentives. In fact, the current scheme likely underestimated the rates of MND performance because not all levels of both B and C criteria from the Slick Criteria were used. Therefore, the identified MND cases in this sample likely represent extreme cases.

While the current study did not include an independently validated group of patients meeting research criteria for MND, the extant literature does not provided much technical guidance in this area for the current population of interest. As noted above, nearly all of the research on invalid neurocognitive performance is concerned with forensic cases having identifiable high stakes outcomes that affect public policy, service provision, and public as well as private financial considerations. In fact, mention of nonforensic malingering has been sparse and vague (Rogers, Salekin, Sewell, Goldstein, & Leonard, 1998), making it unclear if current models of malingering and invalid performance in forensic and other settings are applicable to nonforensic clinical evaluations with less-sensational external reasons for seeking assessment. As such, although there have been specialized tests (i. e., DASH, Word Reading Test) and applications of stand-alone measures in this population, much of the work has been preliminary, reflecting the emerging nature of such research with those seeking ADHD or LD evaluations. Thus, the application of existing methods for detecting invalid neuropsychological performance needs to be extended to and validated in additional populations without potential for financial gain to explore the effects of incentive level. Since neuropsychologists spend the majority of their time outside the realm of forensic practice (Kanauss, Schatz, & Puente, 2005), there is a need to research the impact of various sources of secondary gain independent of the legal arena.

It has been shown that there may be a dose–response relationship between the magnitude of external financial incentive, neuropsychological test performance, and SVT failure rate (Bianchini, Curtis, & Greve, 2006). As such, incentives can be conceptualized on a continuum that includes such factors as the perception, desirability, and type of external incentive—all of which are potential influences on how a patient presents in clinic. Therefore, failure rates on neuropsychological validity indices and performance on neuropsychological tests can be examined in the context of how powerful an incentive may be (e.g., nonfinancial awards, financial claims, disability, etc.). As it relates to the current study, future investigations would be well served to operationalize and examine various types of external academic incentives and neuropsychological performance. It will also be necessary to further refine and more clearly quantify how substantial particular external incentives may be viewed by individuals being evaluated. In this vein, a theoretically driven approach to studying invalid performance in the context of external incentives may be offered in order to elucidate otherwise difficult to explain poor neuropsychological performance. The results of the current study support extending research and clinical investigation of invalid performance into the academic setting and highlight the possibility that external incentives in this population impact clinical presentation.

Conflict of Interest

None declared.

References

Advokat
C. D.
Guidry
D.
Martino
L.
Licit and illicit use of medications for Attention-Deficit Hyperactivity Disorder in undergraduate college students
Journal of American College Health
 , 
2008
, vol. 
56
 
6
(pg. 
601
-
606
)
APA
Diagnostic and statistical manual of mental disorders (4th ed.—Text Revision)
 , 
2000
Washington, D.C.
Author
Ardolf
B. R.
Denney
R. L.
Houston
C. M.
Base rates of negative response bias and malingered neurocognitive dysfunction among criminal defendants referred for neuropsychological evaluation
The Clinical Neuropsychologist
 , 
2007
, vol. 
21
 
6
(pg. 
899
-
916
)
Babcock
Q.
Byme
T.
Student perceptions of methylphenidate abuse at a public liberal arts college
Journal of American College Health
 , 
2000
, vol. 
49
 (pg. 
143
-
145
)
Babikian
T.
Boone
K. B.
Lu
P.
Arnold
G.
Sensitivity and specificity of various digit span scores in the detection of suspect effort
Clinical Neuropsychologist
 , 
2006
, vol. 
20
 
1
(pg. 
145
-
159
)
Bianchini
K. J.
Curtis
K. L.
Greve
K. W.
Compensation and malingering in traumatic brain injury: A dose–response relationship
The Clinical Neuropsychologist
 , 
2006
, vol. 
20
 (pg. 
831
-
847
)
Bianchini
K. J.
Mathias
C. W.
Greve
K. W.
Symptom validity testing: A critical review
Clinical Neuropsychologist
 , 
2001
, vol. 
15
 
1
(pg. 
19
-
45
)
Binder
L. M.
Rohling
M. L.
Money matters: a meta-analytic review of the effects of financial incentives on recovery after closed-head injury
The American Journal of Psychiatry
 , 
1996
, vol. 
153
 
1
(pg. 
7
-
10
)
Booksh
R. L.
Pella
R. D.
Singh
A. N.
Gouvier
W. D.
Ability of college students to simulate ADHD on objective measures of attention
Journal Attention Disorders
 , 
2010
, vol. 
13
 
4
(pg. 
325
-
338
)
Boone
K.
Boone
K. B.
A reconsideration of the Slick et al. (1999) criteria for malingered neurocognitive dysfunction
Assessment of feigned cognitive impairment
 , 
2007
New York, NY
The Guilford Press
(pg. 
29
-
49
)
Boone
K.
The need for continuous and comprehensive sampling of effort/response bias during neuropsychological examinations
The Clinical Neuropsychologist
 , 
2009
, vol. 
23
 
4
(pg. 
729
-
741
)
Bush
S. S.
Ruff
R. M.
Tröster
A. I.
Barth
J. T.
Koffler
S. P.
Pliskin
N. H.
, et al.  . 
Symptom validity assessment: Practice issues and medical necessity: NAN Policy & Planning Committee
Archives of Clinical Neuropsychology
 , 
2005
, vol. 
20
 
4
(pg. 
419
-
426
)
Crank
J. N.
Deshler
D. D.
Disability eligibility issues and university student assessment outcomes
Journal of Vocational Rehabilitation
 , 
2001
, vol. 
16
 (pg. 
217
-
226
)
Delis
D. C.
Wetter
S. R.
Cogniform Disorder and Cogniform Condition: Proposed diagnoses for excessive cognitive symptoms
Archives of Clinical Neuropsychology
 , 
2007
, vol. 
22
 
5
(pg. 
589
-
604
)
Etherton
J. L.
Bianchini
K. J.
Ciota
M. A.
Heinly
M. T.
Greve
K. W.
Pain, malingering and the WAIS-III Working Memory Index
The Spine Journal
 , 
2006
, vol. 
6
 
1
(pg. 
61
-
71
)
Etherton
J. L.
Bianchini
K. J.
Heinly
M. T.
Greve
K. W.
Pain, malingering, and performance on the WAIS-III Processing Speed Index
Journal of Clinical & Experimental Neuropsychology
 , 
2006
, vol. 
28
 
7
(pg. 
1218
-
1237
)
Evans
S. W.
Serpell
Z. N.
Schultz
B. K.
Pastor
D. A.
Cumulative benefits of secondary school-based treatment of students with Attention Deficit Hyperactivity Disorder
School Psychology Review
 , 
2007
, vol. 
36
 
2
(pg. 
256
-
273
)
Fisher
A. B.
Watkins
M. W.
ADHD rating scales’ susceptibility to faking in a college student sample
Journal of Postsecondary Education and Disability
 , 
2008
, vol. 
20
 
2
(pg. 
81
-
92
)
Frazier
T. W.
Frazier
A. R.
Busch
R. M.
Kerwood
M. A.
Demaree
H. A.
Detection of simulated ADHD and reading disorder using symptom validity measures
Archives of Clinical Neuropsychology
 , 
2008
, vol. 
23
 (pg. 
501
-
509
)
Glassmire
D. M.
Bierley
R. A.
Wisniewski
A. M.
Greene
R. L.
Kennedy
J. E.
Date
E.
Using the WMS-III faces subtest to detect malingered memory impairment
Journal of Clinical & Experimental Neuropsychology
 , 
2003
, vol. 
25
 
4
pg. 
465
 
Gouvier
W. D.
Cubic
B.
Jones
G.
Brantley
P.
Postconcussion symptoms and daily stress in normal and head-injured college populations
Archives of Clinical Neuropsychology
 , 
1992
, vol. 
7
 
3
(pg. 
193
-
211
)
Green
P.
Green's Word Memory Test for Microsoft Windows
 , 
2003
Edmonton, AL
Green's Publishing Inc
Green
P.
Rohling
M. L.
Lees-Haley
P. R.
Allen
L. M.
III
Effort has a greater effect on test scores than severe brain injury in compensation claimants
Brain Injury
 , 
2001
, vol. 
15
 
12
(pg. 
1045
-
1060
)
Greiffenstein
M. F.
Baker
W. J.
Gola
T.
Validation of malingered amnesia measures with a large clinical sample
Psychological Assessment
 , 
1994
, vol. 
6
 
3
(pg. 
218
-
224
)
Greve
K. W.
Bianchini
K. J.
Setting empirical cut-offs on psychometric indicators of negative response bias: a methodological commentary with recommendations
Archives of Clinical Neuropsychology
 , 
2004
, vol. 
19
 
4
(pg. 
533
-
541
)
Greve
K. W.
Springer
S.
Bianchini
K. J.
Black
F. W.
Heinly
M. T.
Love
J. M.
, et al.  . 
Malingering in toxic exposure: Classification accuracy of reliable digit span and WAIS-III digit span scaled scores
Assessment
 , 
2007
, vol. 
14
 
1
(pg. 
12
-
21
)
Hadley
W. M.
The necessity of academic accommodations for first-year college students with learning disabilities
Journal of College Admission
 , 
2007
, vol. 
195
 (pg. 
9
-
13
)
Harrison
A. G.
Edwards
M. J.
Armstrong
I.
Parker
K. C. H.
An investigation of methods to detect feigned reading disabilities
Archives of Clinical Neuropsychology
 , 
2010
, vol. 
25
 
2
(pg. 
89
-
98
)
Harrison
A. G.
Edwards
M. J.
Parker
K. C.
Identifying students feigning dyslexia: preliminary findings and strategies for detection
Dyslexia
 , 
2008
, vol. 
14
 
3
(pg. 
228
-
246
)
Harrison
A. G.
Edwards
M. J.
Parker
K. C. H.
Identifying students faking ADHD: Preliminary findings and strategies for detection
Archives of Clinical Neuropsychology
 , 
2007
, vol. 
22
 
5
(pg. 
577
-
588
)
Harrison
A. G.
Rosenblum
Y.
Currie
S.
Examining unusual digit span performance in a population of postsecondary students assessed for academic difficulties
Assessment
 , 
2010
, vol. 
17
 
3
(pg. 
283
-
293
)
Hawkins
K. A.
Tulsky
D. S.
The influence of IQ stratification on WAIS-III/WMS-III FSIQ-General Memory Index discrepancy base-rates in the standardization sample
Journal of the International Neuropsychological Society
 , 
2001
, vol. 
7
 
7
(pg. 
875
-
880
)
Heaton
R. K.
Smith
H. H.
Lehman
R. A.
Vogt
A. T.
Prospects for faking believable deficits on neuropsychological testing
Journal of Consulting and Clinical Psychology
 , 
1978
, vol. 
46
 
5
(pg. 
892
-
900
)
Heilbronner
R. L.
Sweet
J. J.
Morgan
J. E.
Larrabee
G. J.
Millis
S. R.
Conference Participants
American Academy of Clinical Neuropsychology Consensus Conference statement on the neuropsychological assessment of effort, response bias, and malingering
The Clinical Neuropsychologist
 , 
2009
, vol. 
23
 
7
(pg. 
1093
-
1129
)
Heinly
M. T.
Greve
K. W.
Bianchini
K. J.
Love
J. M.
Brennan
A.
WAIS digit span-based indicators of malingered neurocognitive dysfunction: Classification accuracy in traumatic brain injury
Assessment
 , 
2005
, vol. 
12
 
4
(pg. 
429
-
444
)
Iverson
G. L.
Detecting malingered memory deficits through the use of multiple objective measures: A preliminary investigation: Unpublished master's thesis
1991
Morgantown
West Virginia University
Iverson
G. L.
Franzen
M. D.
The Recognition Memory Test, digit span, and Knox Cube Test as markers of malingered memory impairment
Assessment
 , 
1994
, vol. 
1
 
4
(pg. 
323
-
334
)
Jachimowicz
G.
Geiselman
R. E.
Comparison of ease of falsification of Attention Deficit Hyperactivity Disorder diagnosis using standard behavioral rating scales
Cognitive Science Online
 , 
2004
, vol. 
2
 (pg. 
6
-
20
)
Kanauss
K.
Schatz
P.
Puente
A. E.
Current trends in the reimbursement of professional neuropsychological services
Archives of Clinical Neuropsychology
 , 
2005
, vol. 
20
 
3
(pg. 
341
-
353
)
Killgore
W. D. S.
DellaPietra
L.
Using the WMS-III to detect malingering: Empirical validation of the Rarely Missed Index (RMI)
Journal of Clinical & Experimental Neuropsychology
 , 
2000
, vol. 
22
 
6
(pg. 
761
-
771
)
Kirmayer
L. J.
Sartorius
N.
Cultural models and somatic syndromes
Psychosomatic Medicine
 , 
2007
, vol. 
69
 
9
(pg. 
832
-
840
)
Langeluddecke
P. M.
Lucas
S. K.
Quantitative measures of memory malingering on the Wechsler Memory Scale – Third edition in mild head injury litigants
Archives of Clinical Neuropsychology
 , 
2003
, vol. 
18
 
2
(pg. 
181
-
197
)
Larrabee
G. J.
Aggregation across multiple indicators improves the detection of malingering: Relationship to likelihood ratios
The Clinical Neuropsychologist
 , 
2008
, vol. 
22
 
4
(pg. 
666
-
679
)
Lees-Haley
P. R.
Earnest
K.
Dolezal-Wood
S.
Post-concussive symptoms: Base rates and etiology in psychiatric patients
The Clinical Neuropsychologist
 , 
1995
, vol. 
9
 
1
(pg. 
89
-
92
)
Lindstrom
W. A.
Lindstrom
J. J. H.
Coleman
C.
Nelson
J.
Greg
N.
The diagnostic accuracy of symptom validity tests when used with postsecondary students with Learning Disabilities: A preliminary investigation
Archives of Clinical Neuropsychology
 , 
2009
, vol. 
24
 
7
(pg. 
659
-
669
)
Marshall
P.
Schroeder
R.
O'Brien
J.
Fischer
R.
Ries
A.
Blesi
B.
, et al.  . 
Effectiveness of symptom validity measures in identifying cognitive and behavioral symptom exaggeration in adult attention deficit hyperactivity disorder
The Clinical Neuropsychologist
 , 
2010
, vol. 
24
 
7
(pg. 
1204
-
1237
)
Mathias
C. W.
Greve
K. W.
Bianchini
K. J.
Houston
R. J.
Crouch
J. A.
Detecting malingered neurocognitive dysfunction using the Reliable Digit Span in traumatic brain injury
Assessment
 , 
2002
, vol. 
9
 
3
(pg. 
301
-
308
)
McCabe
S. E.
Teter
C. J.
Boyd
C. J.
Medical use, illicit use, and diversion of prescription stimulant medication
Journal of Psychoactive Drugs
 , 
2006
, vol. 
38
 
1
(pg. 
43
-
56
)
McCabe
S. E.
Teter
C. J.
Boyd
C. J.
Guthrie
S. K.
Prevalence and correlates of illicit methylphenidate use among 8th, 10th, and 12th grade students in the United States, 2001
Journal of Adolescent Health
 , 
2004
, vol. 
35
 
6
(pg. 
501
-
504
)
Meyers
J. E.
Volbrecht
M.
A validation of multiple malingering detection methods in a large clinical sample
Archives of Clinical Neuropsychology
 , 
2003
, vol. 
18
 (pg. 
261
-
276
)
Mittenberg
W.
Patton
C.
Canyock
E. M.
Condit
D. C.
Base rates of malingering and symptom exaggeration
Journal of Clinical & Experimental Neuropsychology
 , 
2002
, vol. 
24
 
8
(pg. 
1094
-
1102
)
Mittenberg
W.
Theroux-Fichera
S.
Zielinski
R.
Heilbronner
R. L.
Identification of malingered head injury on the Wechsler Adult Intelligence Scale–Revised
Professional Psychology: Research and Practice
 , 
1995
, vol. 
26
 
5
(pg. 
491
-
498
)
Moline
S.
Frankenberger
W.
Use of stimulant medication for treatment of Attention-Deficit/Hyperactivity Disorder: A survey of middle and high school students’ attitudes
Psychology in the Schools
 , 
2001
, vol. 
38
 (pg. 
569
-
584
)
Morey
L. C.
Personality assessment inventory
 , 
1991
Odessa, FL
Psychological Assessment Resources, Inc
Nelson
N. W.
Hoelzle
J. B.
Mcguire
K. A.
Ferrier-Auerbach
A. G.
Charlesworth
M. J.
Sponheim
S. R.
Evaluation context impacts neuropsychological performance of OEF/OIF veterans with reported combat-related concussion
Archives of Clinical Neuropsychology
 , 
2010
, vol. 
25
 
8
(pg. 
713
-
723
)
Nichols
E.
Harrison
A. G.
McCloskey
L.
Weintraub
L.
Learning opportunities task force 1997–2002: Final report
 , 
2002
Richmond Hill
Learning Opportunities Task Force
Ord
J. S.
Greve
K. W.
Bianchini
K. J.
Using the Wechsler Memory Scale-III to detect malingering in mild traumatic brain injury
The Clinical Neuropsychologist
 , 
2008
, vol. 
22
 (pg. 
689
-
704
)
Osmon
D.
Plambeck
E.
Klein
L.
Mano
Q.
The Word Reading Test of Effort in adult learning disability: A simulation study
The Clinical Neuropsychologist
 , 
2006
, vol. 
20
 
2
(pg. 
315
-
324
)
Pary
R.
Lewis
S.
Matuschka
P.
Rudzinskiy
P.
Safi
M.
Lippman
S.
Attention deficit disorder in adults
Annals of Clinical Psychiatry
 , 
2002
, vol. 
14
 (pg. 
105
-
111
)
Peterson
K.
McDonagh
M. S.
Rongwei
F.
Comparative benefits and harms of competing medications for adults with attention-deficit hyperactivity disorder: a systematic review and indirect comparison meta-analysis
Psychopharmacology
 , 
2008
, vol. 
197
 
1
(pg. 
1
-
11
)
Quinn
C. A.
Detection of malingering in assessment of adult ADHD
Archives of Clinical Neuropsychology
 , 
2003
, vol. 
18
 
4
(pg. 
379
-
395
)
Rabin
L. A.
Barr
W. B.
Burton
L. A.
Assessment practices of clinical neuropsychologists in the United States and Canada: A survey of INS, NAN, and APA Division 40 members
Archives of Clinical Neuropsychology
 , 
2005
, vol. 
20
 
1
(pg. 
33
-
65
)
 
Rehabilitation Act of 1973, 29 U.S.C. § 701 et seq
Rogers
R.
Salekin
R. T.
Sewell
K. W.
Goldstein
A.
Leonard
K.
A comparison of forensic and non-forensic malingerers: A prototypical analysis of explanatory models
Law and Human Behavior
 , 
1998
, vol. 
22
 
4
(pg. 
353
-
367
)
Rogers
R.
Sewell
K. W.
Morey
L. C.
Ustad
K. L.
Detection of feigned mental disorders on the Personality Assessment Inventory: A discriminant analysis
Journal of Personality Assessment
 , 
1996
, vol. 
67
 
3
pg. 
629
 
Rosenfeld
B.
Sands
S. A.
Van Gorp
W. G.
Have we forgotten the base rate problem? Methodological issues in the detection of distortion
Archives of Clinical Neuropsychology
 , 
2000
, vol. 
15
 
4
(pg. 
349
-
359
)
Saklofske
D. H.
Tulsky
D. S.
Wilkins
C.
Weiss
L. G.
Canadian WISC-III directional base rates of score discrepancies by ability level
Canadian Journal of Behavioural Science/Revue Canadienne des Sciences du Comportement
 , 
2003
, vol. 
35
 
3
(pg. 
210
-
218
)
Shelton
J. T.
Elliott
E.
Matthews
R.
Hill
B. D.
Gouvier
W. D.
The relationships of working memory, secondary memory, and general fluid intelligence: Working memory is special
Journal of Experimental Psychology: Learning, Memory, and Cognition
 , 
2010
, vol. 
36
 
3
(pg. 
813
-
820
)
Slick
D. J.
Sherman
E. M. S.
Iverson
G. L.
Diagnostic criteria for malingered neurocognitive dysfunction: Proposed standards for clinical practice and research
The Clinical Neuropsychologist
 , 
1999
, vol. 
13
 
4
(pg. 
545
-
561
)
Sollman
M. J.
Ranseen
J. D.
Berry
D. T. R.
Detection of feigned ADHD in college students
Psychological Assessment
 , 
2010
, vol. 
22
 
2
(pg. 
325
-
335
)
Suhr
J.
Hammers
D.
Dobbins-Buckland
K.
Zimak
E.
Hughes
C.
The relationship of malingering test failure to self-reported symptoms and neuropsychological findings in adults referred for ADHD evaluation
Archives of Clinical Neuropsychology
 , 
2008
, vol. 
23
 
5
(pg. 
521
-
530
)
Suhr
J.
Sullivan
B. K.
Rodriguez
J. L.
The relationship of noncredible performance to Continuous Performance Test scores in adults referred for Attention-Deficit/Hyperactivity Disorder Evaluation
Archives of Clinical Neuropsychology
 , 
2011
, vol. 
26
 
1
(pg. 
1
-
7
)
Sullivan
B. K.
May
K.
Galbally
L.
Symptom exaggeration by college adults in Attention-Deficit Hyperactivity Disorder and Learning Disorder assessments
Applied Neuropsychology
 , 
2007
, vol. 
14
 
3
(pg. 
189
-
207
)
Tan
J. E.
Slick
D. J.
Strauss
E.
Hultsch
D. F.
How'd they do it? Malingering strategies on symptom validity tests
The Clinical Neuropsychologist
 , 
2002
, vol. 
16
 
4
(pg. 
495
-
505
)
Tucha
L.
Sontag
T. A.
Walitza
S.
Lange
K. W.
Detection of malingered attention deficit hyperactivity disorder
ADHD: Attention Deficit-Hyperactivity Disorder
 , 
2009
, vol. 
1
 (pg. 
47
-
53
)
United States Department of Education
Office of Special Education and Rehabilitative Services, Annual Report to Congress on the Implementation of The Individuals with Disabilities Education Act
2005
 
selected years, 1977 through 2005; and Individuals with Disabilities Education Act (IDEA) database. April 22, 2008, from http://www.ideadata.org/PartBdata.asp
Victor
T. L.
Boone
K. B.
Serpa
G. J.
Buehler
J.
Zeigler
E. A.
Interpreting the meaning of multiple symptom validity test failure
Clinical Neuropsychologist
 , 
2009
, vol. 
23
 (pg. 
297
-
313
)
Wechsler
D.
Wechsler Adult Intelligence Scale (3rd ed.)
 , 
1997
San Antonio, TX
The Psychological Corporation
Wechsler
D.
Wechsler Memory Scale (3rd ed.)
 , 
1997
San Antonio, TX
The Psychological Corporation
White
B. P.
Becker-Blease
K. A.
Grace-Bishop
K.
Stimulant medication use, misuse, and abuse in an undergraduate and graduate student sample
Journal of American College Health
 , 
2006
, vol. 
54
 
5
(pg. 
261
-
268
)
Woodworth
T.
DEA congressional testimony
2000