Abstract

Current research suggests that effort indices designed for the detection of malingered neurocognitive functioning do not have adequate sensitivity or specificity for use in cases where malingered mental retardation (MR) is the issue. Therefore, development and validation of reliable, objective measures for the detection of malingered MR have become imperative for both forensic and disability cases in recent years. The purpose of this study was to develop and validate an embedded malingering index for the Stanford Binet Intelligence Scales, Fifth Edition. Data from individuals in the SB5 standardization sample, who had intellectual deficits in the MR range, were used. Items that were rarely missed by the MR sample were pooled and validated using a sample of 54 college students asked to feign MR. Nonverbal items that were missed significantly more frequently by the malingering “analog MR sample” were retained and composed the Stanford Binet Rarely Missed Items-Nonverbal (SBRMI-NV) index. When only individuals who successfully malingered MR (FSIQ < 71) were included, sensitivity of 0.88 and specificity of 1.00 were obtained. Results indicate that although the SBRMI-NV needs further validation, it shows great promise in the detection of malingered MR.

Introduction

Research focused on the detection of malingered psychiatric and neurocognitive impairments has grown exponentially toward the end of the 20th century and into the new millennium. Numerous stand-alone and embedded effort indexes have been developed to provide information about performance during neuropsychological evaluations. Not only has the development of valid effort indices expanded research on malingering, but symptom validity tests (SVTs), such as the Test of Memory Malingering (TOMM; Tombaugh, 1996) and Word Memory Test (WMT; Green, Allen, & Astner, 1996), have also been incorporated into clinical practice for assessment of individuals at risk for showing poor effort in neuropsychological testing.

Despite such advances in detecting malingered neurocognitive impairments, research into the detection of feigned mental retardation (MR) has lagged behind. External incentives to malinger MR became particularly salient with the 2002 Supreme Court ruling in the case of Atkins v. Virginia, 536, U.S. 304. In this case, it was established that mentally retarded individuals are no longer eligible for the death penalty. The Supreme Court reasoned that the death penalty qualifies as cruel and unusual punishment, violating the Eighth Amendment, in individuals with MR because MR is associated with a diminished capacity for reasoning and appreciation. The need for standardized, objective psychological assessment in forensic cases was illustrated during the Atkins trial. The prosecution's expert witness testified about the defendant's IQ using information based on interviews with the defendant and with correctional officers that were supplemented by a review of school records. Justice Hassel of Virginia's Supreme Court stated that prosecution's expert witness's opinion was “incredulous as a matter of law” (Atkins, 2002, Dissent Justice Hassel, III, cited in Everington & Olley, 2008). In addition to this, the Supreme Court ruled in Daubert v. Merrell Dow Pharmaceuticals, Inc. (1993) that expert testimony may only be admitted if it is deemed reliable and relevant (Pope, Butcher, & Seelen, 2006). These decisions by the Supreme Court highlight the need for reliable measures of IQ in forensic settings. Even so, it is imperative to be able to determine whether or not the IQ score obtained is an accurate measure of that individual's abilities or reflects the intentional distortion of responses. Justice Scalia, in his minority dissent during the Atkins case, verbalized his concerns about the ease with which an individual could feign MR merely by researching diagnostic criteria (Atkins v. Virginia, 2002, p. 17). This concern underscores the need for reliable, embedded validity indicators that provide information about effort exhibited on intelligence tests.

To date, no studies have reported base rates for malingered MR in Atkins cases. However, studies have examined base rates of malingering in other criminal forensic settings. A survey of American Board of Clinical Neuropsychology members found that, after adjusting for referral source, neuropsychologists estimated 22.78% of cases meet criteria for probable malingering in criminal cases (Mittenberg, Patton, Canyock, & Condit, 2002). Mittenberg and colleagues also reported that estimated rates of malingering in criminal cases varied significantly by setting. A more recent study examined the rates of malingering using Slick criteria (Slick, Sherman, & Iverson, 1999) and found that 54.29% of the sample met criteria for definite or probable malingering with an additional 26.7% classified as possible malingerers in 105 pretrial competency to stand trial cases (Ardolf, Denney, & Houston, 2007). These studies highlight the necessity of investigating malingering in criminal forensic settings.

In addition to its relevance in criminal forensic cases, more recent literature suggests that malingered MR is highly prevalent in state disability cases. Chafetz, Abrahams, and Kohlmaier (2007) noted that there is a need to discriminate individuals' putting forth valid effort from individuals attempting to defraud the system. Mittenberg and colleagues (2002) reported that, after adjusting for referral source, the mean base rate of probable malingering in disability cases was 32.73%. In their survey, base rates of malingering in disability cases did not vary significantly by geographic location or practice settings. Chafetz (2008) estimated that the base rate of malingering on disability evaluations in Louisiana is near 50%. Chafetz and Lambert (2005) found that 53% of adults and 25% of children failed the TOMM during disability evaluations. Even more alarming, Chafetz (2008) found that 79% of his sample failed at least one effort indicator in disability evaluations, suggesting that a majority of individuals exhibit questionable effort. While it is widely held that the use of only one validity measure provides insufficient information to determine whether or not a person is malingering (Larrabee, 2008; Meyers & Volbrecht, 2003; Victor, Boone, Serpa, Buehler, & Ziegler, 2009), these findings suggest that multiple, validated effort indicators are necessary to distinguish actual from exaggerated deficits in this population.

Little empirical research has examined the usefulness of neuropsychologists' current armamentarium of embedded or stand-alone effort indices in malingered MR cases. Findings of a recent study suggest that failure on effort tests is inversely correlated with fullscale IQ in individuals with no external incentive to malinger (Dean, Victor, Boone, & Arnold, 2008). However, these results are tentative, given that only one individual was diagnosed with MR in the Dean and colleagues (2008) sample. While these findings raise questions about whether MR individuals will incorrectly be identified as malingering, generalization to MR populations can only be inferred.

The detection of malingered MR has received some empirical attention. One approach taken by investigators is to examine the ability of current effort indices (designed to assess malingering of neurocognitive impairment) to discriminate feigned MR from actual MR. Some studies have focused solely on the false-positive rate (percentage of MR individuals who fail) of current effort indices in individuals that meet diagnostic criteria for MR and have no external incentive to malinger. This line of research is especially important because, in capital cases, neuropsychologists want to identify tests that have low false-positive rates (i.e., do not classify any individuals who meet diagnostic criteria for MR as malingering). In other words, only tests with specificity (percentage of nonmalingering individuals identified as such) at or close to 1.00 are acceptable.

Several studies have examined the false-positive rate of current effort indices in MR populations. Brockhaus and Merten (2004) administered the WMT to 32 German adults that had been diagnosed with MR (cited in Dean et al., 2008). Brockhaus and Merten (2004) reported a high specificity (0.97); however, Dean and colleagues (2008) noted that the WMT was not administered to some of the individuals because they were considered too impaired for testing. Hurley and Deal (2006) investigated the usefulness of the Memory for Fifteen Items Test (MFIT; Lezak, Howieson, & Loring, 2004), Dot Counting Test (DCT; Boone et al., 2002; Lezak et al., 2004; Rey, 1941), TOMM, and the Structured Interview of Reported Symptoms (SIRS; Rogers, Bagby, & Dickens, 1992) in 39 individuals who lived in residential facilities for individuals with MR. Individuals had no criminal history, and intellectual functioning ranged from 50 to 78. The authors reported that tests in their battery were failed frequently by individuals with MR (i.e., SIRS = 53.8%; TOMM = 41%; MFIT = 79.5%; DCT = 2.6%).

In addition, Marshall and Happe (2007) determined that individuals with mild MR obtained scores above the empirically derived cutoffs on the Vocabulary-Digit Span (98% passed; Mittenberg, Theroux-Fichera, Zielinski, & Heilbronner, 1995), the Rarely Missed items (91% passed; Killgore & DellaPietra, 2000), and the California Verbal Learning Test-II (89% passed; Delis, Kramer, Kaplan, & Ober, 2000). Nonetheless, they found that the MFIT (55% failed; 83% failed MFIT with recognition), the Rey DCT (79% failed), and the Reliable Digit Span (69% failed; Greiffenstein, Baker, & Gola, 1994) did not show adequate specificity in this population. The authors suggested that Voc-Digits, Rarely Missed Items Index, and CVLT-II may be useful in detecting malingered MR; however, this study did not have a comparison group of individuals suspected of malingering or analog malingerers so sensitivity of these tests could not be assessed. Although some established measures do hold promise, studies also showed that many established measures of neurocognitive malingering lack the specificity to allow differentiation between actual and simulated MR.

Two analog studies have investigated the usefulness of current effort indices in detection of malingering. Graue and colleagues (2007) investigated malingering in 26 individuals with confirmed diagnoses of mild MR, 25 community volunteers given instructions to feign MR, and 10 community volunteers asked to respond honestly. Community volunteers completed a maximum of 11 years of education. The authors concluded that these individuals would be comparable with those who may choose to feign MR because they may plausibly claim to have intellectual deficits. Results indicated that the volunteers asked to malinger scored significantly lower on the TOMM. They proposed a revised cutoff score of <60% for the TOMM Trial 2 which showed a sensitivity of 0.56 and specificity of 0.96. In the same study, psychiatric feigning tests and the Wechsler Adult Intelligence Scale, 3rd Edition (WAIS-III; Tulsky & Zhu, 1997; Wechsler, 1997), embedded validity indices were not able to adequately discriminate true MR from feigned MR (Graue et al., 2007). In a cross-validation study, Shandera and colleagues (2010) used 25 individuals that met diagnostic criteria for mild MR, 25 community volunteers asked to feign MR, and 10 community volunteers asked to respond honestly. As with Graue and colleagues (2007), community volunteers had no more than 11 years of education. Shandera and colleagues (2010) also examined the utility of embedded indices from the WAIS-III. The Reliable Digit Span, Age-Corrected Digit Span (Babikian, Boone, Lu, & Arnold, 2006), and Mittenberg Discriminant Function (Mittenberg et al., 2001) indices did not have adequate specificity (0.33, 0.38, and 0.58, respectively). The Vocabulary-Digit Span demonstrated excellent specificity (1.00) but very poor sensitivity (0.04), indicating that none of the WAIS-III-embedded indices showed utility in detecting feigned MR. In the same study, the WMT did not demonstrated very high specificity for the Immediate Recall (IR), Delayed Recall (DR), or Consistency subtests (0.42, 0.42, or 0.25, respectively). In this study, the TOMM Trial 2 produced a sensitivity of 0.40 and specificity of 0.88. When lower cutoff scores were used (<60% correct), the specificity improved (0.96) but the sensitivity declined significantly (0.24). The authors concluded that overall, neurocognitive feigning measures do not work well in the detection of malingered MR.

Finally, several authors have attempted to examine utility of neurocognitive effort measures in forensic settings with MR individuals. For example, Simon (2007) examined the failure of the TOMM in 21 adjudicated men with a confirmed diagnosis of mild MR and found that only one participant scored below the cutoff (45) on Trial 2. In this study, the TOMM demonstrated adequate specificity. Two studies to date have examined neurocognitive effort measures in forensic settings where individuals had external incentives to malinger. Hayes, Hale,  and Gouvier (1997) found that MR individuals residing in a state facility for the criminally insane, who had previously been identified as malingerers, demonstrated better performances on the MFIT, M-Test, and DCTs than those with MR who had no external incentive. The authors concluded that caution is warranted when employing traditional effort indices in MR individuals as they may not be appropriate for use in this population. In a subsequent study, Hayes, Hale, and Gouvier (1998) added the SIRS (Rogers et al., 1992) to their battery and obtained a correct classification rate of malingering of 95% in this forensic setting. However, subsequent studies that have investigated the utility of the SIRS and other measures of feigned psychiatric symptoms found that the measures did not demonstrate adequate sensitivity or specificity (Graue et al., 2007; Hurley & Deal, 2006; Shandera et al., 2010). One reason for this discrepancy may be the populations used, as it is likely that malingered psychiatric symptoms would be more prominent in a facility for the criminally insane compared with other populations.

It appears that individuals who meet diagnostic criteria for MR have significant difficulty on many widely used effort measures. Based on the current literature, no stand-alone or embedded effort indices have been validated for use in MR populations where malingering is suspected. Only one author to date has attempted to design a measure specifically for the purpose of detecting malingered MR. Chafetz and colleagues (2007) developed the Psychological Consultative Evaluation (PCE) Malingering Rating Scale for use in disability evaluations. The scale is comprised of 11 items that are embedded in the standard disability evaluation. Chafetz (2008) examined the validity of the PCE Malingering Rating Scale in a sample of individuals applying for disability. He found that a score of 12 on the PCE Malingering Rating Scale demonstrated excellent sensitivity and adequate specificity. Overall, all malingerers were classified as such and 79% of individuals with MR were correctly classified. Although this scale does show promise in detecting malingered MR, it is our stance that in this type of evaluation, specificity of instruments should be higher, as 21% of persons with actual MR were misclassified and potentially stigmatized by being labeled as malingerers.

Overall, the limited empirical research available indicates that current tests of neurocognitive and psychiatric feigning designed to detect malingering do not adequately assess feigned MR. In forensic cases, acceptance of expert testimony is dependent upon Daubert criteria. Based on the current literature, current neurocognitive and psychiatric effort measures have not been validated for the detection malingered MR and thus do not meet Daubert criteria for forensic testimony. This is an issue that has not been sufficiently recognized by the courts but could be raised at any time by defense attorneys. Clearly, there is a need to develop new means of assessing effort designed and validated specifically for detecting malingered MR. A newer measure designed to detect poor effort in individuals with MR has shown some promise in disability settings where malingered MR is prominent, but it needs further validation (Chafetz, 2008; Chafetz et al., 2007).

Rationale for Present Study

The current study proposes an embedded effort measure for the Stanford Binet-5 that aims to detect malingered MR. The Stanford Binet Intelligence Scales, Fifth Edition (SB5) author and publisher, Pro-Ed, Inc., graciously provided normative data on the SB5′s validation of a sample of mentally retarded individuals. The goal was to derive an embedded index of rarely missed items from items of the SB5, thus providing clinicians with an alternative intelligence test with embedded effort indices to circumvent practice effects when the WAIS-III has already been administered. The rationale for deriving this index is that individuals “guessing” how to suppress their full-scale IQ scores are likely to miss some items that a majority of MR individuals do not miss. The ultimate goal is to develop an index that is able to discriminate in a way that no individual with MR is wrongfully classified as malingering (100% specificity), while sensitivity remains sufficient to ensure that more probably than not, malingerers will be detected.

Method

Participants

Standardization MR sample

Stanford Binet-5 standardization data were used for the standardization MR sample. These data included 17 individuals who met diagnostic criteria for MR. All individuals included had obtained SB5 fullscale IQ scores >50. Data from these 17 individuals were combined with data from 14 individuals from the standardization sample who obtained FSIQ scores <71 on the SB5 but were not diagnosed with MR. Therefore, 31 individuals with intellectual deficits falling in the MR range were retained for this study. Demographic information for the standardization MR sample is presented in Table 1.

Table 1.

Demographic information for the AMR, SMR, and college control sample

 College control (n= 54) Analog MR (n= 54) Standardization MR (n= 31) Analog MR versus college control Analog MR versus standardization MR 
Age 
 Mean 20 20.96 18.35 t = 1.22 t = 2.0* 
SD 2.41 5.28 6.67   
 Range 18–29 18–48 10–34   
Shipley FSIQ 
 Mean 106.17 104.74  t = 0.94  
SD 7.52 8.27    
 Range 87–118 85–118    
SB5 FSIQ 
 Mean 104.48 66.13 63.68 z = 7.90** z = 0.18 
SD 10.91 20.56 8.49   
 Range 78–126 40–110 52–75   
Men (%) 66.7 61.1 48.4 χ2= 0.36 χ2= 0.34 
Caucasian (%) 72.2 79.6 25.8 χ2= 1.11 χ2= 22.74** 
 College control (n= 54) Analog MR (n= 54) Standardization MR (n= 31) Analog MR versus college control Analog MR versus standardization MR 
Age 
 Mean 20 20.96 18.35 t = 1.22 t = 2.0* 
SD 2.41 5.28 6.67   
 Range 18–29 18–48 10–34   
Shipley FSIQ 
 Mean 106.17 104.74  t = 0.94  
SD 7.52 8.27    
 Range 87–118 85–118    
SB5 FSIQ 
 Mean 104.48 66.13 63.68 z = 7.90** z = 0.18 
SD 10.91 20.56 8.49   
 Range 78–126 40–110 52–75   
Men (%) 66.7 61.1 48.4 χ2= 0.36 χ2= 0.34 
Caucasian (%) 72.2 79.6 25.8 χ2= 1.11 χ2= 22.74** 

Notes: MR = mental retardation; FSIQ = full-scale IQ; SD= standard deviation; Shipley = estimated WAIS-R FSIQ; t= t-score; z= Mann–Whitney U-test z-score; χ2= chi-square statistic.

*p< .05.

**p< .001.

College sample

A sample of 108 undergraduate student volunteers was recruited from a large university in the southeast USA. Students were asked to participate in this study in exchange for extra credit in introductory psychology classes. Participants were randomly selected to be placed into one of two groups: Analog malingerers (n= 54) or normal controls (n= 54). Demographic information can be found in Table 1.

Materials

Measures of intellectual functioning

The Shipley Institute of Living Scale (Shipley, 1940) has two subtests: Verbal and abstraction. In the verbal subtest, individuals are presented with a list of 40 words and asked to choose a synonym for each word out of four choices, a target and three foils. The abstraction subtest is composed of 20 fill-in-the-blank problems that increase in difficulty as the test progresses. Both subtests were administered and scores were converted into estimated WAIS-R FSIQ scores. The Shipley was used for the college control sample in order to estimate true FSIQ scores of the analog malingerer group, but was also administered to the healthy control sample for comparison of the two groups.

The SB5 (Roid, 2003a, 2003b, 2003c) is a broad measure of intellectual and cognitive ability normed for ages 2–85+. The SB5 was standardized based on a regionally stratified national sample. It measures verbal and nonverbal performance of five domains: Fluid Reasoning, Knowledge, Quantitative Reasoning, Visual-Spatial Processing, and Working Memory. First, individuals are tested with a nonverbal routing subtest then a verbal routing subtest. The routing subtests determine starting points on subsequent verbal and nonverbal tasks, respectively. There are five verbal and six nonverbal testlets for each of the five domains, and individuals are awarded a maximum of six points per testlet. Testlets increase in difficulty as the test progresses. Testlet administration depends upon basal and ceiling rules that are operationalized in the manual. Administration time typically ranges between 45 and 75 min. Test–retest reliabilities of the SB5 reported in the manual ranged from 0.66 to 0.93 for the verbal and nonverbal subtests and 0.89 to 0.95 for the full-test IQ scales. Correlations with the WAIS-III factor indexes range from 0.69 to 0.80.

Effort measures

The TOMM (Tombaugh, 1996) is a nonverbal, forced-choice test widely used in clinical and forensic practices. Individuals are presented with 50 simple line drawings and told to remember them. Afterwards, they are asked to indicate which picture they were previously shown by discriminating between two pictures, the target and a foil. Individuals are informed of the accuracy of each response. In Trial 2, the task is repeated, and individuals are again given feedback. There is a 15-min retention trial, but only Trials 1 and 2 were administered in the present study. A cutoff score of 45 (90% accuracy) on Trial 2 is recommended for the detection of poor effort (Rees, Tombaugh, Gansler, & Moczynski, 1998).

The WMT (Green et al., 1996) is computer administered and computer scored. Individuals are presented with 20 pairs of words and told to remember them. In the IR task, individuals are presented with a word from the list and asked to choose the word that it was paired with from two choices, the target and a foil. The DR task is exactly the same but is administered after 30 min. A consistency score is calculated based on the accuracy of responses for each item. A cutoff score of ≤82.5% is indicative of insufficient effort for the IR and DR subtests.

In addition to the two stand-alone effort tests administered, the participants were asked to complete the Digit Span subtest of WAIS-III so that reliable digit span scores (Greiffenstein et al., 1994) could be computed. During the Digit Span Forward subtest, individuals are asked to correctly repeat strings of numbers, increasing in length as the test progresses. In Digit Span Backwards, individuals are required to repeat each string in reverse order. There are two trials per item. Reliable Digit Span is based on the sum of the longest string repeated accurately for both trials of the Forward and Backward conditions. Scores of <7 are considered to be indicative of poor effort.

Procedure

The standardization procedure for the SB5 sample is reported in the manual (Roid, 2003c). Individuals in the standardization did not complete any stand-alone effort indices. Analyses were approved by Pro-Ed, Inc. and the University's Institutional Review Board.

Students first read and signed a consent form. They were then asked to complete a brief demographic questionnaire and the Shipley while giving their best effort. Afterwards, students were asked to read one of two scenarios based on which condition they had been randomly assigned to.

Normal Control condition: As a child, you were able to complete schoolwork, read, write, spell, and do math as well as other children. In addition, you were able to adequately perform daily functions related to interpersonal relationships, monetary transactions, communication, and self-care.

As you grew older, you perceived yourself as “normal” and were never diagnosed with an intellectual disability. You graduated from high school and were accepted to the University. You began hanging around people who convinced you that obtaining a college education was the best way to support yourself. One night you decided to sign up for a psychology course at LSU. The psychology course offered you an opportunity for research credit or extra credit by participating in psychology experiments. You chose to sign up for this psychology experiment.

Now you are sitting in the Psychological Services Center participating in this study for research credit or extra credit. You are asked to take some tests as part of a neuropsychological evaluation. The examiner explains that, if you perform with your best effort, you will contribute to psychological research. The examiner further explains that psychological experiments are crucial to testing hypotheses and developing theories.

The validity of this research study depends on the effort you put forth on these tests. You are instructed to respond to the tests with your very best effort.

Analog MR malingerers: As a child, you struggled continuously with schoolwork and were never able to read, write, spell, or do math as well as other children. In addition, you struggled with daily functions related to interpersonal relationships, monetary transactions, communication, and self-care.

As you grew older, you knew “something was wrong.” You either “squeaked by” though school, got socially promoted, or dropped out. You continued to struggle with basic skills. You began hanging around people who convinced you that stealing was the best way to support yourself. One night you broke into a house to steal some money, and the homeowner confronted you with a gun. In a split second reaction, you shot him fatally.

Now you are in prison, facing the death penalty. You are asked to take some tests as part of a neuropsychological evaluation. Your lawyer explains that, if the tests show your IQ is <70, you can be found mentally retarded and escape the death penalty, as mentally retarded persons cannot be executed. Your lawyer further explains, however, that, if the examiner discovers you are faking, you will likely receive the death penalty.

Your life is now depending on the scores of these tests. You are instructed to respond to the tests in a manner that ensures you will be found mentally retarded without being detected as a faker.

After reading the appropriate scenario, students were asked to complete a battery including the SB5 and SVTs (i.e., WMT, TOMM Trial 1 and Trial 2, Digit Span subtest of the WAIS-III). Administration of the SB5 was counterbalanced with the current neuropsychological effort indices.

Results

Demographic Information

Demographic information was compared using t-tests for continuous variables and χ2 tests for categorical variables (such as race and gender). Comparisons of the analog MR malingerers, standardization MR, and normal control samples can be found in Table 1. The college control sample and analog MR sample were well matched for age, gender, race, and estimated WAIS-R IQ scores. It was noted that there were significantly fewer Caucasians and significantly more African Americans and Hispanics in the standardization MR sample. Also, t-tests revealed that the standardization MR sample was significantly younger than the analog MR sample.

Manipulation Check

The SB5 FSIQ scores of analog MR malingerers were compared with college controls and to the standardization MR sample to determine whether the analog MR malingerers attempted to feign MR. Given that the MR samples were relatively small and have scores in the lower, restricted, and skewed range of the normal curve, nonparametric statistics (Siegel, 1956) were used to compare samples. The Mann–Whitney U-tests (Mann & Whitney, 1949), shown to have excellent power in comparison to the parametric t-test (Siegel, 1956), were employed. The results revealed that the analog MR malingerers' SB5 FSIQ scores were significantly lower than the college control sample (p< .001). The Mann–Whitney U-tests also indicated that the standardization MR and analog MR malingerer groups did not differ in SB5 FSIQ scores. Of the analog MR malingerer sample, 20 individuals failed to suppress their FSIQ scores to ≤70. Of these, 10 failed to obtain FSIQ scores <85 (1SD below the mean), indicating that they did not follow instructions. Subsequent analyses will examine performance when all individuals are included, and when individuals who failed to distort their responses are excluded. We chose 85 as a cutoff because it indicates that individuals made an attempt to suppress FSIQ scores 1 SD below the mean, even though they were not successful in obtaining scores that fell in the MR range. The analyses were run a final time including only individuals that suppressed their FSIQ scores to the MR range (<70).

As an additional manipulation check, this study also sought to examine analog MR malingerers' responses on current stand-alone effort indices (i.e., TOMM and WMT) and one embedded effort index from the WAIS-III (RDS). Performance of the analog MR malingerer sample was compared with college controls using the Mann–Whitney U-tests. The analog MR malingerer group scored significantly lower on the TOMM Trial 1 (z= 7.28; p< .001), TOMM Trial 2 (z= 6.96; p< .001), Reliable Digits Index (z= 5.41; p< .001), WMT IR (z= 7.32; p< .001), and WMT DR (z= 7.83; p< .001) tasks compared with college controls. Means and standard deviations are presented in Table 2.

Table 2.

Manipulation check: mean ± SD scores for analog MR malingerers and college controls

 Analog MR (mean ± SDCollege controls (mean ± SDp-value 
Shipley IQ 104.74 ± 8.27 106.17 ± 7.52 .351 
SB5 FSIQ 66.13 ± 20.56 104.48 ± 10.91 .001 
TOMM T1 34.19 ± 10.73 48.48 ± 2.34 .001 
TOMM T2 37.26 ± 10.74 49.92 ± 0.35 .001 
RDS 6.91 ± 3.28 10.31 ± 2.11 .001 
WMT IR 70.63 ± 19.46 96.53 ± 12.03 .001 
WMT DR 70.48 ± 19.27 98.58 ± 3.11 .001 
 Analog MR (mean ± SDCollege controls (mean ± SDp-value 
Shipley IQ 104.74 ± 8.27 106.17 ± 7.52 .351 
SB5 FSIQ 66.13 ± 20.56 104.48 ± 10.91 .001 
TOMM T1 34.19 ± 10.73 48.48 ± 2.34 .001 
TOMM T2 37.26 ± 10.74 49.92 ± 0.35 .001 
RDS 6.91 ± 3.28 10.31 ± 2.11 .001 
WMT IR 70.63 ± 19.46 96.53 ± 12.03 .001 
WMT DR 70.48 ± 19.27 98.58 ± 3.11 .001 

Notes: MR = mental retardation; FSIQ = full-scale IQ; SD= standard deviation; SB5 = Stanford Binet Intelligence Scales, 5th Edition; TOMM = Test of Memory Malingering (T1 = Trial 1; T2 = Trial 2); RDS = Reliable Digit Span; WMT = Word Memory Test (IR = Immediate Recall; DR = Delayed Recall).

Stanford Binet-5 Rarely Missed Items Index-NV

All verbal and nonverbal items in subtests levels 1–4 were examined. Routing items below item 20 were also examined. Analysis of items compared standardization MR sample (n= 31) with the analog MR malingerer sample (n= 54). χ2 tests were used to determine which items were missed by significantly more analog MR malingerer participants compared with the standardization MR group. A conservative approach (p< .001 threshold) was used for item retention because of multitude of items tested. Three verbal items met the threshold (p< .001); however, they were not included in the SBRMI-NV because of the overwhelming majority of nonverbal items that met criteria. College students appeared to use different processes for feigning MR on verbal (Verbal IQ; mean ± SD: 68.68 ± 18.81) compared with nonverbal (Nonverbal IQ; mean ± SD: 53.16 ± 12.72) subtests. Specifically, they did not suppress their scores as low on the verbal subtests and there was far more variability in scores the verbal subtest. Due to such discrepancies, the authors chose to use only items from the nonverbal subtests in order to increase homogeneity of content and explore the verbal items separately in later work.

Overall, a total of 21 non-verbal items meeting the p< .001 threshold were retained for the SBRMI-NV (Table 3). Items were scored 1 point if answered correctly and 0 points if answered incorrectly. Total points of all items were summed to obtain the total SBRMI-NV. Essentially, a total score of 21 indicates that an individual responded correctly to all SBRMI-NV items while a score of 0 indicates that an individual missed every item.

Table 3.

Performance on the SBRMI-NV: Standardization MR versus analog MR samples

SB5 items Standardization MR (n= 31)
 
Analog MR (n= 54)
 
Pearson's (χ2Fisher's exact (two-sided) 
 0 pt 1 pt 0 pt 1 pt   
NV Routing-7 29 21 33 10.50 .001 
NV Routing-8 31 16 38 11.32 .001 
NV Routing-12 31 16 38 11.32 .001 
NV Routing-13 31 22 32 17.04 .001 
NV Routing-14 31 20 34 15.01 .001 
NV Routing-15 29 26 28 15.50 .001 
NV Routing-16 31 20 34 15.01 .001 
NV Routing-17 30 27 27 19.51 .001 
NV-QR Level 2-4 31 17 37 12.20 .001 
NV-VS Level 2-3 31 14 40 9.62 .001 
NV-VS Level 2-5 31 14 40 9.62 .001 
NV-VS Level 2-6 31 17 37 12.20 .001 
NV-WM Level 2-4 31 14 40 9.62 .001 
NV-WM Level 2-5 31 18 36 13.11 .001 
NV-WM Level 2-6 31 21 33 16.01 .001 
NV-QR Level 3-3 31 26 28 21.50 .001 
NV-QR Level 3-4 31 24 30 19.20 .001 
NV-WM Level 3-1 30 27 27 19.51 .001 
NV-WM Level 3-2 31 27 27 22.72 .001 
NV-WM Level 3-3 28 30 24 17.45 .001 
NV-KN Level 4-1 23 35 19 11.99 .001 
SB5 items Standardization MR (n= 31)
 
Analog MR (n= 54)
 
Pearson's (χ2Fisher's exact (two-sided) 
 0 pt 1 pt 0 pt 1 pt   
NV Routing-7 29 21 33 10.50 .001 
NV Routing-8 31 16 38 11.32 .001 
NV Routing-12 31 16 38 11.32 .001 
NV Routing-13 31 22 32 17.04 .001 
NV Routing-14 31 20 34 15.01 .001 
NV Routing-15 29 26 28 15.50 .001 
NV Routing-16 31 20 34 15.01 .001 
NV Routing-17 30 27 27 19.51 .001 
NV-QR Level 2-4 31 17 37 12.20 .001 
NV-VS Level 2-3 31 14 40 9.62 .001 
NV-VS Level 2-5 31 14 40 9.62 .001 
NV-VS Level 2-6 31 17 37 12.20 .001 
NV-WM Level 2-4 31 14 40 9.62 .001 
NV-WM Level 2-5 31 18 36 13.11 .001 
NV-WM Level 2-6 31 21 33 16.01 .001 
NV-QR Level 3-3 31 26 28 21.50 .001 
NV-QR Level 3-4 31 24 30 19.20 .001 
NV-WM Level 3-1 30 27 27 19.51 .001 
NV-WM Level 3-2 31 27 27 22.72 .001 
NV-WM Level 3-3 28 30 24 17.45 .001 
NV-KN Level 4-1 23 35 19 11.99 .001 

Notes: SBRMI = Stanford Binet Rarely Missed Items; MR = mental retardation; NV = nonverbal; QR = Quantitative Reasoning; VS = Visual Spatial Processing; WM = Working Memory; KN = Knowledge.

Logistic regression analyses were used to investigate the usefulness of the SBRMI-NV. The first logistic regression analysis examined the utility of the SBRMI-NV in distinguishing all individuals in the analog MR malingerer (n= 54) sample from those in the standardization MR sample. In the logistic regression, the malingerer group was coded 1, and the standardization MR group was coded 2. The SBRMI-NV was a significant predictor of malingered MR χ2(1, n= 85) = 41.18, p< .001, odds ratio = 1.92, and CIs [1.27, 2.98]. The next logistic regression analysis included individuals from the analog MR malingerer sample that suppressed their FSIQ scores <85 (n= 44). The SBRMI-NV remained a significant predictor of malingering χ2(1, n= 75) = 56.87, p< .001, the odds ratio increased = 2.69, and CIs [1.56, 4.63]. A final logistic regression analysis included only individuals from the analog MR sample that successfully suppressed their FSIQ scores to ≤70 (n= 34). The SBRMI-NV was again a significant predictor of malingering χ2(1, n= 65) = 73.84, p< .001, the odds ratio = 5.95, and CIs [1.72, 20.54].

Receiver operating characteristic (ROC) curves allow researchers to determine appropriate cutoff scores by plotting sensitivity and respective false-positive rates (1-specificity). In this study, ROC curves were used to examine the tradeoff between sensitivity and specificity of various cutoff scores for the SBRMI-NV (Hsaio, Bartko, & Potter, 1989; Larrabee & Berry, 2007). As stated previously, the authors were seeking a cutoff score that was highly specific in excluding individuals that meet diagnostic criteria for MR. Results of an ROC indicated that a cutoff score of 17 yielded sensitivity of 0.59 and a perfect specificity (1.00). Overall classification was adequate with 77.6% accuracy, 74.1% of analog malingers in the analog MR malingerer group were correctly identified, and 83.9% of actual MR individuals in the standardization MR were identified correctly when the SBRMI-NV total score was entered into a logistic regression as the predictor. When individuals who failed to suppress their SB5 FSIQ scores <85 were eliminated (n= 44), a cutoff score of 17 yielded 0.71 sensitivity and 1.00 specificity. Overall classification improved to 86.7% with 88.6% of analog MR malingerer group and 83.9% of the standardization MR group correctly classified. When only analog MR malingerers who suppressed their FSIQ to 70 or below were included (n= 34), a cutoff score of 17 on the ROC curve demonstrated 0.88 sensitivity and 1.00 specificity. Overall classification improved to 92.3%. Of the analog MR malingerer group, 90.3% were successfully classified while 94.1% of the standardization MR group was correctly classified. See Table 4 for additional operating characteristics.

Table 4.

Receiver operating characteristics for the SBRMI-NV

SBRMI-NV score Analog MR (n= 54)a
 
Analog MR > 85 (n= 44)b
 
Analog MR > 70 (n= 34)c
 
 Sensitivity Specificity Sensitivity Specificity Sensitivity Specificity 
1.00 1.00 1.00 
0.019 1.00 0.023 1.00 0.029 1.00 
0.056 1.00 0.068 1.00 0.088 1.00 
0.148 1.00 0.182 1.00 0.235 1.00 
0.204 1.00 0.25 1.00 0.321 1.00 
0.241 1.00 0.295 1.00 0.382 1.00 
0.259 1.00 0.318 1.00 0.412 1.00 
0.278 1.00 0.341 1.00 0.441 1.00 
0.333 1.00 0.409 1.00 0.529 1.00 
0.389 1.00 0.477 1.00 0.588 1.00 
11 0.426 1.00 0.523 1.00 0.647 1.00 
12 0.444 1.00 0.545 1.00 0.676 1.00 
13 0.481 1.00 0.591 1.00 0.735 1.00 
14 0.5 1.00 0.641 1.00 0.765 1.00 
15 0.5 1.00 0.641 1.00 0.765 1.00 
16 0.519 1.00 0.636 1.00 0.794 1.00 
17 0.593 1.00 0.705 1.00 0.882 1.00 
18 0.685 0.90 0.818 0.90 0.941 0.90 
19 0.741 0.84 0.886 0.84 1.00 0.84 
20 0.815 0.71 0.932 0.71 1.00 0.71 
21 1.00 
SBRMI-NV score Analog MR (n= 54)a
 
Analog MR > 85 (n= 44)b
 
Analog MR > 70 (n= 34)c
 
 Sensitivity Specificity Sensitivity Specificity Sensitivity Specificity 
1.00 1.00 1.00 
0.019 1.00 0.023 1.00 0.029 1.00 
0.056 1.00 0.068 1.00 0.088 1.00 
0.148 1.00 0.182 1.00 0.235 1.00 
0.204 1.00 0.25 1.00 0.321 1.00 
0.241 1.00 0.295 1.00 0.382 1.00 
0.259 1.00 0.318 1.00 0.412 1.00 
0.278 1.00 0.341 1.00 0.441 1.00 
0.333 1.00 0.409 1.00 0.529 1.00 
0.389 1.00 0.477 1.00 0.588 1.00 
11 0.426 1.00 0.523 1.00 0.647 1.00 
12 0.444 1.00 0.545 1.00 0.676 1.00 
13 0.481 1.00 0.591 1.00 0.735 1.00 
14 0.5 1.00 0.641 1.00 0.765 1.00 
15 0.5 1.00 0.641 1.00 0.765 1.00 
16 0.519 1.00 0.636 1.00 0.794 1.00 
17 0.593 1.00 0.705 1.00 0.882 1.00 
18 0.685 0.90 0.818 0.90 0.941 0.90 
19 0.741 0.84 0.886 0.84 1.00 0.84 
20 0.815 0.71 0.932 0.71 1.00 0.71 
21 1.00 

Notes: SBRMI-NV = Stanford Binet Rarely Missed Items-Nonverbal; MR = mental retardation; FSIQ = full-scale IQ; SB5 = Stanford Binet Intelligence Scales, 5th Edition.

aAll analog MR malingerers included.

bAnalog MR malingerers with SB5 FSIQ > 85.

cAnalog MR malingerers with SB5 FSIQ > 70.

Concurrent Validity Check

The analog MR malingerer data were used to compare the SBRMI-NV with current effort indices in order to examine concurrent validity. The SBRMI-NV was significantly correlated with the TOMM Trial 2 (n= 54; r= .40; p< .001), RDS (n= 54; r= .57; p< .001), WMT IR (n= 54; r= .42; p< .01), and WMT DR (n= 54; r= .37; p< .01) for the analog MR malingerer group. When all 54 analog MR malingerers were included, 63% failed Trial 2 of the TOMM, 50% failed the RDS, 68.5% failed the WMT IR, 64.8% failed the WMT DR, and 59.3% failed the SBRMI-NV. After excluding individuals who failed to suppress their SB5 FSIQ scores to 85 or below, 70.5% failed Trial 2 of the TOMM, 61.4% failed the RDS, 72.7% failed the WMT IR, 72.7% failed the WMT DR, and 70.5% failed the SBRMI-NV. When only individuals that suppressed their SB5 FSIQ scores to 70 or below were included, 82.4% failed Trial 2 of the TOMM, 73.5% failed the RDS, 85.3% failed the WMT IR, 82.4% failed the WMT DR, and 88.2% failed the SBRMI-NV-NV.

Discussion

The current study sought to derive an embedded malingering index for the SB5, the SBRMI-NV. The index was based on nonverbal items that were rarely missed by individuals that met diagnostic criteria for MR but frequently missed by individuals attempting to feign MR in a criminal forensic setting. College students were asked either to give their best effort or to malinger mild MR (analog MR malingerer group). The analog MR samples' scores were then compared with those obtained by individuals with MR from the standardization sample. Nonverbal items that were missed significantly more frequently (p< .001) by the analog MR group were combined into the SBRMI-NV. Overall, the SBRMI-NV demonstrated good sensitivity and perfect specificity in discriminating between successful malingerers and individuals with MR.

Current stand-alone and embedded effort measures were administered as a manipulation check. Analog malingerers scored significantly lower on the TOMM, WMT, and RDS compared with college controls, suggesting that the analog MR malingerer group intentionally attempted to distort their responses. Limited research has reported conflicting results about the utility of the TOMM, RDS, and WMT in samples of individuals diagnosed with MR (Graue et al, 2007; Hurley & Deal, 2006; Marshall & Happe, 2007; Shandera et al., 2010; Simon, 2007). These measures have not demonstrated adequate specificity in MR populations. Overall, a review of the literature indicates that neurocognitive effort measures are not valid or reliable in an intellectually disabled population, although some authors postulate that new cutoff scores for current measures could be effective.

Unfortunately, no individual in the standardization MR sample completed effort measures during the standardization process. Therefore, the relationship between the sensitivity and specificity of the TOMM, WMT, and RDS could not be compared with the standardization MR sample. While the present study could not determine the specificity of current neurocognitive measures, it did produce excellent specificity for the SBRMI-NV. Because recent research indicates the current effort indices have relatively poor specificity in MR populations, it is hypothesized that future studies of the SBRMI-NV will find it superior in detecting feigned MR compared with the current neurocognitive effort measures.

While the cutoff score for the SBRMI-NV was originally chosen for its perfect specificity in this sample, it is important to note that the sensitivity of the cutoff score of <17 increased drastically as unsuccessful malingerers were excluded. It was encouraging that a cutoff score yielding perfect specificity existed. The sensitivity of the SBRMI-NV was 0.59 with all analog MR subjects, 0.71 when those with FSIQ scores >85 were excluded, and 0.88 when only those who suppressed FSIQ scores to 70 or below were included. In forensic cases and clinical practice, it is highly unlikely that individuals with average intellectual functioning would ever be considered in Atkins cases or MR disability claims. Most individuals who would be tested under these circumstances likely have intellectual deficits that fall in the low average to borderline ranges. In fact, most states have strict cutoff scores of 70 and below for MR claims, making the need to detect unsuccessful malingering (FSIQ > 70) a moot point. However, as demonstrated, the SBRMI-NV does show promise in detecting purposeful feigning in individuals who do not successfully malinger MR indicating that it may still be a useful effort indicator in individuals with borderline intellectual functioning.

Most importantly, there was a cutoff that yielded no false positives; that is, no standardization MR individuals were incorrectly classified as malingering when a cutoff score of <17 was applied. Persons in the standardization MR data ranged in age from 10 to 34. It is intriguing that even younger children with mild MR passed the cutoff on the SBRMI-NV. Further evidence for the specificity of the SBRMI-NV is that none of the college controls were incorrectly identified.

Eight nonverbal routing items were included in the SBRMI-NV. It is likely that these items would be useful in detecting malingered MR in the abbreviated SB5 battery which is composed of the two routing subtests. Future research may wish to investigate the utility of an abbreviated SBRMI-NV for detecting malingered MR when only the abbreviated battery is administered. This may prove especially useful when screening for cognitive impairments and intellectual deficits are an issue.

As expected, individuals asked to simulate mild MR in a criminal forensic setting adopted a variety of response styles on the battery of tests administered. The mean SB5 FSIQ scores of the analog MR malingerer group fell in the MR range and their SB5 FSIQ scores did not differ significantly from the standardization MR group. Overall, 34 of 54 college students asked to feign mild MR suppressed their FSIQ scores to the MR range (i.e., <70). These findings suggest that the scenario used in the current study was successful in instructing students about Atkins-like scenarios and the complex problems associated with such cases. While most scenarios used in studying analog malingers feigning neurocognitive deficits tend to be much shorter (e.g., Mittenberg et al., 1995; Swihart, Harris, & Hatcher, 2008), the authors thought that it was important to portray the life-long complications experienced by many such individuals and the potential motivations to respond dishonestly.

This study has several limitations. Most importantly, as with all analog research, the generalizability of findings is an issue. In order to meet Daubert criteria, SBRMI-NV needs to demonstrate adequate reliability in both MR individuals (FSIQ < 70) with no external incentive to feign and probable malingerers. In addition, the SBRMI-NV must demonstrate the ability to discriminate malingerers from individuals who meet criteria for MR. The SBRMI-NV should also be validated in populations with psychopathology such as psychosis and major depression. The standardization MR sample was younger than most individuals who would file Atkins claims (aged 10–34). A significant percentage of individuals in the standardization MR sample were under the age of 18. However, as stated previously, this may be advantageous as even children with mild MR did not fall below the cutoff of 17 on the SBRMI-NV. This may also prove advantageous for disability cases where children may be involved as, with further validation, it may be useful in detecting poor effort in children as well. At this point, researchers should examine the utility of the SBRMI-NV and further validation is required before it should be used in clinical or forensic practice.

The method by which individuals choose to feign MR has not been widely investigated in the literature. Based on the large discrepancy of rarely missed items on verbal compared with nonverbal subtests, it appears that college students asked to feign MR in a forensic setting demonstrated different strategies to feigning verbal versus nonverbal items. Three verbal items met criteria (p< .001) for the rarely missed items index but were not included in the analyses due to the overwhelming majority of nonverbal items that discriminated the standardization MR sample from analog MR malingerers. Future research should pursue examining and identifying a larger set of verbal items sufficient to constitute the formulation of a verbal rarely missed items scale.

In conclusion, the present study provides initial development and validation of a rarely missed item index for the SB5, the SBRMI-NV. The SBRMI-NV shows promise in its ability to differentiate individuals who meet diagnostic criteria for MR from persons attempting to malingered MR in a criminal forensic setting. The 21 items of the SBRMI-NV are awarded one points for each correct responses, for a perfect total score of 21 points. Overall, a cutoff score of <17 (i.e., ≥4 items missed) demonstrated the best sensitivity while maintaining perfect specificity and excellent overall classification rates. These rates were better than those reported in the literature for current effort indices' ability to identify malingered MR (Graue et al, 2007; Hurley & Deal, 2006; Marshall & Happe, 2007; Shandera et al., 2010; Simon, 2007). Currently, the generalizability of these findings for the SBRMI-NV is limited as college students were asked to simulate malingered MR in a criminal forensic setting. Future research is needed to validate the SBRMI-NV in forensic and disability cases. The SBRMI-NV is the first embedded effort index designed specifically for the detection of feigned MR, and thus far, it has demonstrated excellent specificity in correctly classifying MR individuals. This is an important step in creating a measure that will expand the tool kit of clinicians and enable them to confidently and reliably assess malingering in a population of individuals feigning MR amongst mentally retarded individuals.

Conflict of Interest

Dr. Gale Roid is the author of the Stanford Binet-5.

Acknowledgements

We would like to thank Stephanie Bastidas, Rachel Chatham, Anthony Correro, Seth Courrege, Hong Hoang, Devin Levio, Jeffrey Manning, Chad Roberts, and Marcus Venable for their assistance with data collection. We would also like to thank Pro-Ed, Inc., for providing standardization data from the Stanford Binet-5.

References

Ardolf
B. R.
Denney
R. L.
Houston
C. M.
Base rates of negative response bias and malingered neurocognitive dysfunction among criminal defendants referred for neuropsychological evaluation
The Clinical Neuropsychologist
 , 
2007
, vol. 
21
 (pg. 
899
-
916
)
Atkins v. Virginia
2002
 
260 Va. 375, 534S. E. 2d 312.
Babikian
T.
Boone
K.
Lu
P.
Arnold
G.
Sensitivity and specificity of various Digit Span scores in the detection of suspect effort
The Clinical Neuropsychologist
 , 
2006
, vol. 
20
 (pg. 
145
-
159
)
Boone
K.
Lu
P.
Back
C.
King
C.
Lee
A.
Philpott
L.
, et al.  . 
Sensitivity and specificity of the Rey Dot Counting Test in patients with suspect effort and various clinical samples
Archives of Clinical Neuropsychology
 , 
2002
, vol. 
17
 (pg. 
625
-
642
)
Brockhaus
R.
Merten
T.
Neuropsychological assessment of suboptimal performance: The Word Memory Test
Nervenarzt
 , 
2004
, vol. 
75
 (pg. 
882
-
887
)
Chafetz
M. D.
Malingering on the social security disability consultative exam: Predictors and base rates
The Clinical Neuropsychologist
 , 
2008
, vol. 
22
 (pg. 
529
-
546
)
Chafetz
M. D.
Abrahams
J. P.
Kohlmaier
J.
Malingering on the Social Security disability consultative exam: A new rating scale
Archives of Clinical Neuropsychology
 , 
2007
, vol. 
22
 (pg. 
1
-
14
)
Chafetz
M. D.
Lambert
J.
Malingering on the social security disability consultative exam
The Clinical Neuropsychologist
 , 
2005
, vol. 
19
 
2
(pg. 
146
-
147
)
Daubert v. Merrell Dow Pharmaceuticals, Inc
1993
 
509 U.S. 579
Dean
A. C.
Victor
T. L.
Boone
K. B.
Arnold
G.
The relationship of IQ to effort test performance
The Clinical Neuropsychologist
 , 
2008
, vol. 
22
 (pg. 
702
-
722
)
Delis
D. C.
Kramer
J. H.
Kaplan
E.
Ober
B. A.
California Verbal Learning Test
 , 
2000
2nd ed.
San Antonio, TX
The Psychological Corporation, Harcourt Assessment Company
Everington
C.
Olley
J. G.
Implication of Atkins v. Virginia: Issues in defining and diagnosing mental retardation
Journal of Forensic Psychology Practice
 , 
2008
, vol. 
8
 
1
(pg. 
1
-
23
)
Graue
L. O.
Berry
D. T. R.
Clark
J. A.
Sollman
M. J.
Cardi
M.
Hopkins
J.
, et al.  . 
Identification of feigned mental retardation using the new generation of malingering detection instruments: Preliminary findings
The Clinical Neuropsychologist
 , 
2007
, vol. 
21
 
6
(pg. 
929
-
942
)
Green
P.
Allen
L. M.
Astner
K.
The Word Memory Test: A user's guide to the oral and computer-administered forms, US version 1.1
 , 
1996
Durham, NC
CogniSyst
Greiffenstein
M. F.
Baker
W. J.
Gola
T.
Validation of malingered amnesia measures with a large clinical sample
Psychological Assessment
 , 
1994
, vol. 
6
 
3
(pg. 
218
-
224
)
Hayes
J. S.
Hale
D. B.
Gouvier
W. D.
Do tests predict malingering in defendants with mental retardation?
Journal of Psychology
 , 
1997
, vol. 
131
 
5
(pg. 
575
-
576
)
Hayes
J. S.
Hale
D. B.
Gouvier
W. D.
Malingering detection in a mentally retarded forensic population
Applied Neuropsychology
 , 
1998
, vol. 
5
 (pg. 
33
-
36
)
Hsaio
J. K.
Bartko
J. J.
Potter
W. Z.
Diagnosing diagnoses: Receiver operating characteristic methods and psychiatry
Archives of General Psychiatry
 , 
1989
, vol. 
46
 (pg. 
664
-
667
)
Hurley
K. E.
Deal
W. P.
Assessment instruments measuring malingering used with individuals who have mental retardation: Potential problems and issues
Mental Retardation
 , 
2006
, vol. 
44
 (pg. 
112
-
119
)
Killgore
W. D. S.
DellaPietra
L.
Using the WMS-III to detect malingering: Empirical validation of the rarely missed index (RMI)
Journal of Clinical and Experimental Neuropsychology
 , 
2000
, vol. 
22
 
6
(pg. 
761
-
771
)
Larrabee
G. J.
Aggregation across multiple indicators improves the detection of malingering: Relationship to likelihood ratios [Electronic Version].
The Clinical Neuropsychologist
 , 
2008
, vol. 
22
 
4
(pg. 
666
-
679
)
Larrabee
G. J.
Berry
D. T. R.
Larrabee
G. J.
Diagnostic classification statistics and diagnostic validity of malingering assessment
Assessment of malingered neuropsychological deficits
 , 
2007
New York
Oxford University Press
(pg. 
14
-
26
)
Lezak
M. D.
Howieson
D. B.
Loring
D. W.
Neuropsychological assessment
 , 
2004
4th ed.
New York
Oxford University Press
Mann
H. B.
Whitney
D. R.
On a test of whether one of two random variables is stochastically larger than the other
Annals of Mathematical Statistics
 , 
1949
, vol. 
18
 (pg. 
50
-
60
)
Marshall
P.
Happe
M.
The performance of individuals with mental retardation on cognitive tests assessing effort and motivation
The Clinical Neurologist
 , 
2007
, vol. 
21
 (pg. 
826
-
840
)
Meyers
J. E.
Volbrecht
M. E.
A validation of multiple malingering detection methods in a large clinical sample
Archives of Clinical Neuropsychology
 , 
2003
, vol. 
18
 
3
(pg. 
261
-
276
)
Mittenberg
W.
Patton
C.
Canyock
E. M.
Condit
D. C.
Base rates of malingering and symptoms exaggeration
Journal of Clinical and Experimental Neuropsychology
 , 
2002
, vol. 
24
 
8
(pg. 
1094
-
1102
)
Mittenberg
W.
Theroux
S.
Aguila-Puentas
G.
Bianchini
K. J.
Greve
K. W.
Rayls
K.
Identification of malingered head injury on the Wechsler Adult Intelligence Scale (3rd ed.)
Clinical Neuropsychologist
 , 
2001
, vol. 
15
 (pg. 
440
-
445
)
Mittenberg
W.
Theroux-Fichera
S.
Zielinski
R. E.
Heilbronner
R. L.
Identification of malingered head-injury on the Wechsler Adult Intelligence Scale-Revised
Professional Psychology: Research and Practice
 , 
1995
, vol. 
26
 
5
(pg. 
491
-
498
)
Pope
K. S.
Butcher
J. N.
Seelen
J.
The MMPI, MMPI-2, and MMPI-A in court: A practical guide for expert witnesses and attorneys (3rd Edition)
 , 
2006
Washington, DC
American Psychological Association
Rees
L. M.
Tombaugh
T. N.
Gansler
D. A.
Moczynski
N. P.
Five validation experiments of the Test of Memory Malingering (TOMM)
Psychological Assessment
 , 
1998
, vol. 
10
 
1
(pg. 
10
-
20
)
Rey
A.
The psychological examination in cases of traumatic encephalopathy
Archives of Psychology
 , 
1941
, vol. 
28
 (pg. 
286
-
340
)
Rogers
R.
Bagby
R. M.
Dickens
S. E.
Structured Interview of Reported Symptoms: Professional manual
 , 
1992
Odessa, FL
Psychological Assessment Resources
Roid
G. H.
Stanford-Binet Intelligence Scales: Fifth Edition
 , 
2003
Itasca, IL
Riverside
Roid
G. H.
Stanford-Binet Intelligence Scales: Fifth Edition, examiner's manual
 , 
2003
Itasca, IL
Riverside
Roid
G. H.
Stanford-Binet Intelligence Scales, Fifth Edition, technical manual
 , 
2003
Itasca, IL
Riverside
Shandera
A. L.
Berry
D. T. R.
Clark
J. A.
Schipper
L. J.
Graue
L. O.
Harp
J. P.
Detection of malingered mental retardation
Psychological Assessment
 , 
2010
, vol. 
22
 
1
(pg. 
50
-
56
)
Shipley
W. C.
A self-administering scale for measuring intellectual impairment and deterioration
Journal of Psychology
 , 
1940
, vol. 
9
 (pg. 
371
-
377
)
Siegel
S.
Nonparametric statistics for the behavioral sciences
 , 
1956
New York
McGraw-Hill
Simon
M.
Performance of mentally retarded forensic patients on the Test of Memory Malingering
Journal of Clinical Psychology
 , 
2007
, vol. 
63
 
4
(pg. 
339
-
344
)
Slick
D. J.
Sherman
E. M. S.
Iverson
G. L.
Diagnostic criteria for malingered neurocognitive dysfunction: Proposed standards for clinical practice and research
The Clinical Neuropsychologist
 , 
1999
, vol. 
13
 
4
(pg. 
545
-
561
)
Swihart
A. A.
Harris
K. M.
Hatcher
L. L.
Inability of the Rarely Missed Index to identify simulated malingering under more realistic assessment conditions
Journal of Clinical and Experimental Neuropsychology
 , 
2008
, vol. 
30
 
1
(pg. 
120
-
126
)
Tombaugh
T.
Test of Memory Malingering (TOMM)
 , 
1996
New York
Multi Health Systems
Tulsky
D.
Zhu
J.
WAIS-III/WMS-III technical manual
 , 
1997
San Antonio
Psychological Corporation
Victor
T. L.
Boone
K. B.
Serpa
J. G.
Buehler
J.
Ziegler
E. A.
Interpreting the meaning of multiple symptom validity test failure
The Clinical Neuropsychologist
 , 
2009
, vol. 
23
 
2
(pg. 
297
-
313
)
Wechsler
D.
Wechsler Adult Intelligence Scale (3rd ed.)
 , 
1997
San Antonio, TX
The Psychological Corporation