Abstract

The assessment of effort is an important aspect of a comprehensive neuropsychological evaluation, as this can significantly impact data interpretation. While recent work has validated the appropriateness of adult-derived cutoffs for standalone effort measures in younger populations, little research has focused on embedded effort measures in children. The present study includes 54 clinically referred children and adolescents (32 males/22 females; aged 6–17) with a confirmed diagnosis of epilepsy. Reliable Digit Spans (RDSs) were calculated and the Test of Memory Malingering (TOMM) was administered in the context of a comprehensive neuropsychological evaluation. Using a previously published RDS cutoff of ≤6, a pass rate of only 65% was obtained, well below the recommended 90% pass rate for an effective effort index. In contrast, when adult criteria were used on TOMM Trial 2, a 90% pass rate was observed. RDS scores were significantly correlated with IQ estimates (r = .59, p < .001) and age (r = .61, p < .001). The difference between RDS and the TOMM on the participant outcome was statistically significant (χ2 = 9.05, p = .003). These results suggest that RDS appears to yield a large number of false positives and, therefore, may be of limited utility in detecting poor effort in a pediatric epilepsy population. These findings likely extend to other pediatric populations that are known to have significant cognitive loss.

Introduction

Given potential effects on the interpretation of neuropsychological data, the evaluation of effort is increasingly becoming standard practice (Bush et al., 2005). This is particularly true in forensic evaluations, where secondary gain may be involved, and thus, it is incumbent upon the neuropsychologist to formally measure and document effort and response bias (Lee, Loring, & Martin, 1992). This said, it is also strongly recommended that objective measures of effort assessment become part of routine clinical practice to maximize the confidence in test findings (Bush et al., 2005; Richman et al., 2006).

Although the formal assessment of effort has become largely customary in adult neuropsychological evaluations, those working with younger populations have been slower to adopt such practices. Despite prior research demonstrating that children can be coached to feign cognitive deficits on neuropsychological evaluations (Heubrock, 2001), many pediatric-oriented clinicians continue to believe that children do not “malinger” or that poor effort is always easy to detect through qualitative observation. Unfortunately, the literature indicates that children are able to feign ignorance when it benefited them. This occurs in 50%–79% of preschoolers as a result of material gain, avoidance of punishment, confusion, or playfulness (Newton, Reddy, & Bull, 2000; Polak & Harris, 1999; Stouthamber-Loeber & Loeber, 1986). Moreover, children older than 9 may even exhibit the capability of feigning neuropsychological impairment after being coached, as well as “faking bad” and “faking good” to convey certain personality characteristics (Faust, Hart, & Guilmette, 1988; Hall & Pritchard, 1996; Oldershaw & Bagby, 1997; Power & Stoppard, 1973). Further, deception in children is often difficult to detect and is not always a deliberate effort by the child to produce false results. Indeed, boredom, playfulness, or fatigue may cause a child to put forth suboptimal or inconsistent effort (Faust et al., 1988). In sum, due to the influence that motivation may have on the outcome of a neuropsychological evaluation, the importance of measuring effort in pediatric populations is gaining recognition. A recent consensus conference statement, on effort assessment in the context of neuropsychological evaluation, put forth by the American Academy of Clinical Neuropsychology (AACN), recommended that “effort measures and embedded validity indicators should be applied to pediatric samples (Heilbronner, Sweet, Morgan, Larrabee, & Millis, 2009).”

While failure of Symptom Validity Tests (SVTs) may be a result of severe impairment, the literature suggests that in many cases, it is related to effort or motivation. Empirical research has demonstrated that many “standalone” effort tests initially developed for adults can be used with confidence in children and adolescents. For example, 86% of 7–18-year olds are able to pass the Word Memory Test (WMT; Green, Allen, & Astner, 1996) at the cut-off points established for adults. Importantly, of those who failed the task, all conceded that they had not tried their hardest during testing and all but one passed upon retesting (Green & Flaro, 2003). The Medical SVT (MSVT; Green, 2004) has demonstrated potential for detecting poor effort in school-age children. For example, Kirkwood and Kirk (2010) found a base rate of 17% for suboptimal effort in a mild traumatic brain injury (mTBI) population. Previous studies conducted across multiple diagnostic, age and intellectual ability groups have also demonstrated the utility of the MSVT. A study by Chafetz, Prentkowki, and Rao (2011) suggests that the MSVT accurately explains intrinsic motivation among individuals seeking compensation for an inability to work and those hoping to return to work or to be reunited with their children, as those seeking to work or reunify with their children passed at high rates. The MSVT has also shown utility across the lifespan and a range of intellectual abilities. Children aged 6–10 and those with low IQs (Full-Scale IQ of 65) are able to pass the MSVT (Carone, 2009; Gill, Green, Flaro, & Pucci, 2007; Green, 2004). Similarly, children with attention deficit hyperactivity disorder and fetal alcohol syndrome have also been shown to pass the MSVT (Carone, 2009). At the other end of the age spectrum, the MSVT demonstrated 100% specificity in individuals with dementia and 80% specificity in a dementia simulator group (Singhal, Green, Ashaye, Shankar, & Gill, 2009).

The Test of Memory Malingering (TOMM; Tombaugh, 1996) has also demonstrated utility in children, and research indicates that the adult-derived cut-off scores on the TOMM are suitable for use in children (Blaskewitz, Merten, & Kathmann, 2008; Brooks, Sherman, & Krol, 2012; Constantinou & McCaffrey, 2003; Donders, 2005; MacAllister, Nakhutina, Bender, Karantzoulis, & Carlson, 2009; Rienstra, Spaan, & Schmand, 2010). Prior research has demonstrated varying utility of the TOMM in pediatric samples. For example, MacAllister and colleagues (2009) used the TOMM to establish a 90% pass rate in a sample of children and adolescents with epilepsy, indicating that it may be a valid measure of effort in this population; however, Blaskewitz and colleagues (2008) indicated that the MSVT demonstrated higher sensitivity than the TOMM in child simulators. Some studies caution against the use of adult-derived cutoffs. For example, such cutoffs for the WMT and the Computerized Assessment of Response Bias (Allen, Conder, Green, & Cox, 1997) may not be appropriate for younger children, due to inherent differences in reading skills and developmental level. Notably, Green and Flaro (2003) present strong evidence indicating that a reading level below 3rd grade is a contraindication for these cutoffs, rather than age alone.

While standalone effort measures have been shown to be useful in pediatric populations, they can be time-consuming. Given the fact that, in clinical practice, there are often time constraints placed on clinicians, many are disinclined to add tests to their batteries. In response to these concerns, the field is moving toward the use of less onerous, more efficient measures to provide the ongoing assessment of effort across the testing battery, as opposed to assessment via isolated measures at specific time points (Boone, 2009). The ideal practice would involve the use of “embedded measures” in addition to “standalone measures,” as recommended by the AACN (Heilbronner et al., 2009); “embedded measures” refer to indices of effort already included in a standard battery.

One embedded measure used regularly in neuropsychological evaluations is Reliable Digit Span (RDS). Derived from the Digit Span subtest of the Wechsler intelligence scales (e.g., Wechsler Intelligence Scale for Children-4th Edition, WISC-IV; Wechsler, 2003), RDS is calculated by summing the longest string of digits forward and backward that were provided without error across the two trials. Adult studies suggest that a cutoff of RDS ≤ 7 indicates poor effort (e.g., Greiffenstein, Baker, & Gola, 1994; Meyers & Volbrecht, 1998). However, a study of children age 6–11 found that 59% of participants “failed” using this criterion (Blaskewitz et al., 2008). In a pediatric mTBI sample, there was a significant difference in RDS scores between those who did and did not fail the MSVT and TOMM and, as a result, an RDS cut-off score of ≤6 was recommended as a threshold to detect effort in children (Kirkwood, Hargrave, & Kirk, 2011). Although the authors suggested that RDS cut-off scores are likely associated with an elevated rate of false positives in children, they note that Digit Span scores may have some utility as an embedded measure of effort in a relatively high-functioning pediatric sample.

Importantly, however, the utility of RDS is largely unknown in young neurologic populations that show greater degrees of cognitive impairment than seen in mTBI. The purpose of the current study is to evaluate the utility of RDS in a sample of children and adolescents with epilepsy. In evaluating the utility of this embedded effort index, the overall pass rate in this sample will be determined and sensitivity and specificity data will be provided for various cutoffs. In addition, the relations between RDS scores and overall cognitive functioning (i.e., intellectual functioning) will be evaluated. Further, RDS scores will be contrasted with scores on the TOMM for this sample.

Methods

Participants

The participants were 54 clinically referred children and adolescents (32 males, 22 females; two non-native English speakers) consecutively evaluated at the New York University Comprehensive Epilepsy Center. This sample included 54 of the 60 discussed in a prior publication regarding the assessment of effort in pediatric epilepsy (MacAllister et al., 2009). Participants were administered both the TOMM and the Digit Span subtest of the WISC-IV or Wechsler Adult Intelligence Scale-3rd Edition (WAIS-III) were included. Participants were between the ages of 6 and 17 (mean = 13.0, SD = 3.80). All participants had a diagnosis of epilepsy as confirmed by neurologic work-up that included electroencephalogram (EEG). Thirty-three participants had diagnoses of partial epilepsy syndromes; 10 had primary generalized epilepsies. The remaining participants were unclassified or had mixed epilepsy syndromes. Four participants had received prior surgical intervention; one had a right frontal glioma resection, one received right frontal subpial transections, one received a left temporal resection, and one received a left frontal resection. Effort measures were administered as part of a larger neuropsychological evaluation. Participants were referred for the evaluation of cognitive functioning in order to characterize the effects of epilepsy and to guide pharmacological and educational treatment, to lateralize and localize dysfunction as part of a comprehensive pre-surgical evaluation, and/or to monitor cognitive changes subsequent to surgical intervention and in the presence of continued epilepsy despite surgery.

Measures

Each patient received the Digit Span subtest of the Wechsler scales and the TOMM (Trials 1 and 2) as part of a larger battery of neuropsychological measures, which included a measure of general intellectual functioning. RDSs were calculated for each Digit Span administration. Adult literature suggests that if an RDS ≤7 is obtained, low effort should be suspected (Greiffenstein et al., 1994; Meyers & Volbrecht, 1998). As noted above, the pediatric literature has been variable, suggesting cutoffs of ≤6 and ≤7 (Blaskewitz et al., 2008; Kirkwood et al., 2011).

The TOMM is a visually based, 50-item, forced-choice measure of effort. The items are presented sequentially with a 3-s exposure per item. The participant is then shown sets of two items: one target item and one foil. The task is to identify the target. Two learning trials are given. A score below the recommended cutoff on Trial 2 is suspicious for poor effort (Tombaugh, 1996). Although the administration of a third Retention Trial is offered as an option, Constantinou and McCaffrey (2003) demonstrated that it is not necessary as almost all children are able to pass the TOMM on Trial 2.

To assess general intellectual functioning, patients were administered the WISC-IV (Wechsler, 2003), the WAIS-III (Wechsler, 1997), or the Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, 1999). For the WISC-IV and WAIS-III, the General Abilities Index (GAI) was used as a global summary score of intellectual functioning to make these scores more directly comparable with the FSIQ of the WASI; as this latter index score includes verbal and performance items without the influence of working memory or processing speed and is more comparable with the WISC-IV and WAIS-III GAI scores and also serves to remove the Digit Span subtest from the IQ estimate. Raven's Progressive Matrices (Raven, 1960) was used to estimate intelligence in the two non-native English speakers. All index scores were transformed to standard scores (mean = 100, SD = 15).

Procedure

All patients were evaluated in the outpatient offices of the New York University Comprehensive Epilepsy Center or at bedside during inpatient video-EEG monitoring. Descriptive statistics examined scores on the TOMM, RDS, Longest Digit Span Forward (LDSF), and Longest Digit Span Backward (LDSB), as well as intelligence estimates to characterize the sample. The Pearson correlation coefficients assessed the relations between RDS scores, age, and intelligence estimates. One-way ANOVAs examined the relationships between the RDS, TOMM, and IQ scores of individuals with partial epilepsy versus those with generalized epilepsy. In addition, RDS, TOMM, and IQ scores were compared among three age groups (6–8, 9–12, and 13–17). Pass rates for RDS (using Kirkwood et al.'s (2011) recommendation of ≤6) and TOMM Trial 2 cutoffs were determined for the overall sample and for participants' whose IQ estimates fell in the range associated with intellectual disability (i.e., <70). χ2 tests with Yates' correction assessed for differences in pass rates between RDS and the TOMM Trial 2. Finally, sensitivity and specificity as well as positive predictive power (PPP) and negative predictive power (NPP) were calculated for various RDS cut-off scores.

Results

Scores on Trial 1 of the TOMM ranged from 22 to 50 (mean = 43.83, SD = 6.31). Trial 2 scores ranged from 31 to 50 (mean = 47.89, SD = 4.09). RDS scores ranged from 0 to 13 (mean = 6.98, SD = 2.80). LDSF scores ranged from 3 to 9 and LDSB scores ranged from 0 to 7. Not surprisingly given the nature of the sample, participants' estimated intelligence varied greatly; estimated IQ ranged from 41 to 122 (mean = 87.0, SD = 19.96). Nine of the participants had IQ estimates that fell in the range typically associated with intellectual disability (i.e., <70). As depicted in Table 1, RDS scores were significantly correlated with both IQ estimates (r = .59, p < .001) and age (r = .61, p < .001). TOMM scores, however, were not correlated with age (Trial 1 r = .19, p = .169; Trial 2 r = .17, p = .210) but were correlated with IQ (Trial 1 r = .61, p < .001; Trial 2 r = .45, p < .01). Using an RDS cutoff of ≤6 and the recommended cutoff on TOMM, the difference between RDS and the TOMM failure rate was significant, with more children obtaining RDS scores below the cutoff (χ2 = 9.05, p = .003).

Table 1.

RDS and TOMM Trial 2 correlations with age and IQ

  RDS
 
TOMM Trial 2
 
r-value p-value r-value p-value 
Age .606 <.001 .173 .210 
IQ .591 <.001 .450 .001 
  RDS
 
TOMM Trial 2
 
r-value p-value r-value p-value 
Age .606 <.001 .173 .210 
IQ .591 <.001 .450 .001 

Notes: RDS = Reliable Digit Span; TOMM = Test of Memory Malingering.

When comparing participants with partial epilepsy to those with generalized epilepsy, no significant differences existed with respect to RDS scores, F(1, 41) = 0.016 and p = .899, TOMM Trial 2 scores, F(1, 41) = 0.008 and p = .930, and IQ, F(1, 41) = 0.784 and p = .381. When considering age groups, a significant relationship existed among the three age groups examined on RDS scores—F(2, 51) = 9.623 and p < .001. Post hoc analyses (with Bonferroni correction) suggested that the 6–8-year-old participants performed significantly worse on RDS than the 13–17-year-old participants.

The recommended cut-off scores for both RDS and TOMM were used to assess “pass rates” for this sample. Using an RDS ≤6, a pass rate of 65% (35 of 54) was obtained, which was well below the recommended 90% pass rate for an effective effort index (Donders, 2005). In contrast, using the recommended adult cut score on the TOMM, a 90% (49 of 54) pass rate was achieved.

Of the participants who failed RDS, 74% passed the TOMM. Only five participants (ages 6–15) of the overall sample failed both the TOMM and RDS, and of those, three had IQ estimates <70, while the other two were in the youngest age group (ages 6 and 7). Of the nine participants whose IQ estimates were associated with intellectual disability, six (67%) achieved passing scores on the TOMM; only one (11%) of those achieved a passing score on RDS and the other five (56%) failed RDS. Three (33%) participants failed both RDS and the TOMM. Of the two participants with IQ estimates in the range associated with mild intellectual disability (IQ = 60–69), both (100%) passed the TOMM, but neither (0%) passed RDS. When considering lower intellectual functioning (i.e., IQ <59), four of seven (57%) passed the TOMM and one of seven (14%) passed RDS. Of the participants with IQ estimates >70, 43 of 45 passed the TOMM and 34 of 45 passed RDS. It should be noted that the two non-native English speakers in this sample achieved passing scores on both RDS and the TOMM. Using TOMM pass/failure as the criterion variable, sensitivity and specificity as well as positive predictive power (PPP) and negative predictive power (NPP) were calculated for each RDS cutoff. Results indicate that the cut-off score of 6, as recommended by Kirkwood and colleagues (2011) yields a sensitivity of 100%, but poorer specificity (71%) and PPP (26%). Table 2 provides sensitivity and specificity for each RDS score and Table 3 provides PPP and NPP ratings.

Table 2.

Sensitivity and specificity values for various RDS cutoffs in comparison with TOMM

RDS Sensitivity (%) Specificity (%) 
96 
≤1 96 
≤2 20 92 
≤3 20 90 
≤4 60 89 
≤5 60 82 
≤6 100 71 
≤7 100 48 
≤8 100 24 
≤9 100 18 
≤10 100 
≤11 100 
≤12 100 
RDS Sensitivity (%) Specificity (%) 
96 
≤1 96 
≤2 20 92 
≤3 20 90 
≤4 60 89 
≤5 60 82 
≤6 100 71 
≤7 100 48 
≤8 100 24 
≤9 100 18 
≤10 100 
≤11 100 
≤12 100 

Note: RDS = Reliable Digit Span.

Table 3.

PPP and NPP values for poor effort, using the TOMM as a means for comparison

RDS PPP (%) NPP (%) 
≤1 90 
≤2 20 92 
≤3 17 92 
≤4 30 95 
≤5 25 95 
≤6 26 100 
≤7 18 100 
≤8 12 100 
RDS PPP (%) NPP (%) 
≤1 90 
≤2 20 92 
≤3 17 92 
≤4 30 95 
≤5 25 95 
≤6 26 100 
≤7 18 100 
≤8 12 100 

Notes: RDS = Reliable Digit Span; PPP = positive predictive power; NPP = negative predictive power.

Discussion

This study is the first to assess the utility of RDS in a sample of children and adolescents with epilepsy. The major goal of the present study was to ascertain whether previously established cutoffs for RDS scores were appropriate for children with epilepsy. To do so, the overall pass/failure rate of RDS scores was calculated using previously suggested cutoffs, and the relationship between RDS scores and age, as well as RDS and intellectual functioning, was determined. Using the TOMM as the criterion for adequate effort, the sensitivity, specificity, PPP, and NPP of RDS scores at various cut points were determined.

Recommended guidelines for an appropriate effort measure in children and adults suggest that such a measure should not show strong associations with cognitive scores and should not be affected by other patient characteristics. Moreover, 90% of the sample should be able to obtain passing scores on the effort measure (Donders, 2005). The results of our investigation demonstrate that RDS scores do show strong correlations with clinical and cognitive variables, including age of participant and intellectual functioning. Moreover, the overall pass rate of RDS scores in this sample was fairly low, with only 65% of the sample attaining passing scores according to previously recommended pediatric cutoffs (Kirkwood et al., 2011). This low overall pass rate in our sample calls the utility of this measure in children and adolescents with epilepsy, or other neurologic conditions with known cognitive sequela, into question. Its utility as a measure of effort may be especially limited in those with intellectual disability or in younger children (i.e., age below 8).

In evaluating the utility of RDS as an effort measure, several reasons for poor performance need to be considered. First, a directionality problem exists. That is, it is unknown whether the observed correlations between effort indices and IQ estimates reflect the fact that low effort would also depress IQ scores or that those with low IQ would score more poorly on effort indices. Accordingly, one must never rely on a single source of data to conclude that effort was insufficient. In cases where a child's IQ falls in the intellectually disabled range, one must corroborate findings with other sources of data (i.e., adaptive behavior ratings from parents and/or teachers). Contributing factors to poor performance, specifically on Digit Span, may include dislike of (or difficulty with) numerical tasks, distractibility, playfulness, or an overall negative attitude toward testing. It should also be acknowledged that some such factors may not only produce poor effort on Digit Span, but may also influence effort on tests of intellectual functioning leading to spuriously high associations between RDS and intellectual functioning.

In addition to assessing effectiveness of current cutoffs, the sensitivity and specificity and PPP and NPP of various RDS cutoffs were examined in this study. It was determined that the recommended cutoff of ≤6 (Kirkwood et al., 2011) yielded strong sensitivity, but poor specificity and PPP in our pediatric epilepsy population. Babikian, Boone, Lu, and Arnold (2006) recommend specificity values of ≥90% for acceptable cutoffs on effort indices. As such, a cutoff of ≤3 (sensitivity = 20; specificity = 90) or ≤4 (sensitivity = 60; specificity = 89) would be more appropriate using these guidelines, but, as one can see, at such thresholds sensitivity and PPP are severely compromised.

There were five participants in this sample who failed both the TOMM and RDS; three of these children were older (i.e., ages 13–15) but had intellectual disability and two had average intellectual ability but were very young (ages 6 and 7). While it is possible that these younger children demonstrated poor effort, it is more likely that, given the relations to age, RDS may be especially inappropriate as an effort index in young children due to developmental factors.

Intellectual disability, associated with poor outcomes on the effort measure (RDS), is often connected with epilepsy. Unfortunately, although several studies have considered intellectually disabled populations, most have failed to determine effective effort indices; these studies have shown poor specificity and a high correlation between proposed effort measures and IQ (Dean, Victor, Boone, & Arnold, 2008; Hurley & Deal, 2006; Marshall & Happe, 2007; Reznek, 2005; Shandera et al., 2010). Brockhaus and Merten (2004), however, did successfully use the WMT with mental retardation individuals and was able to identify high performance motivation in all but one case. It should be noted that, as a result of excluding subjects with extremely poor functioning, it has been suggested that the results be interpreted with caution (Dean et al., 2008).

Though the TOMM has shown some utility in an intellectually disabled population (Kennedy et al., 2005), findings have been variable. In fact, our own prior work demonstrated that the TOMM is also correlated with intellectual ability in children and adolescents with epilepsy (MacAllister et al., 2009). Accordingly, it is likely that present results, where the TOMM is used as the criterion measure for poor effort, are questionable in low IQ participants. On the other hand, Chafetz and colleagues (2011) found that low IQ (mild intellectual disability) alone does not predict SVT failure, but rather the motivation of the examinee determines SVT failure. All this considered, the consensus is that considerable caution must be exercised when interpreting effort indices in individuals with documented intellectual disability/mental retardation though this may not be the sole reason for SVT failure.

As one can see, it will be important to validate measures of effort that are not unduly affected by low intellectual function. Further, separating low effort versus ability failure will be important for future studies. A larger study might be helpful in making this distinction. Digit Span likely requires significant cognitive effort to complete and, as such, the effectiveness of RDS as a measure of effort in other neurological populations with mild to moderate deficits should be examined.

Several methodological limitations should be highlighted. First, the study used a sample comprised of clinically referred children from a tertiary care epilepsy center. In such centers, epilepsy surgery candidates are typically assessed, and thus, low intellectual functioning is not uncommon. Thus, the present results may not be entirely generalizable to the populations evaluated in outpatient offices where epilepsy patients with less cognitive impairment are assessed. Additionally, as memory dysfunction is common in this population and often varies depending on the type and severity of the epilepsy, it is possible that RDS scores within the current sample may be differentially affected by the type and severity of seizures (e.g., partial epilepsy vs. generalized) and whether or not a surgical intervention was performed. Thus, differentiating between poor effort and genuine (severe) cognitive impairment poses a challenge in the present study, as in others where low-functioning individuals are assessed. Studies that examine the effects of cognitive factors, such as verbal memory, are needed to determine whether they substantially affect the RDS outcomes.

Further, given that this sample was limited to children and adolescents with epilepsy, the applicability of these findings to populations with other medical diagnoses is unclear. It is possible that RDS may show greater clinical utility as an effort measure in populations that typically show fewer cognitive sequelae, such as learning disabilities or attention deficit/hyperactivity disorders, but this remains to be seen. The motivation of the participants must be considered as well, as noted by Chafetz and colleagues (2011). Though none of the parents of the participants in the present study disclosed actively seeking disability on the participant's behalf, many were seeking support services in school, potentially affecting data outcomes. Finally, the current study used only one measure of effort (TOMM) against which to compare RDS. Future work should seek to measure the effectiveness of the RDS by comparing it with additional validated measures of effort, both free standing and embedded.

References

Allen
L. M.
III
Conder
R. L.
Green
P.
Cox
D. R.
CARB ‘97: Manual for the Computerized Assessment of Response Bias.
 , 
1997
Durham, NC
CogniSyst
Babikian
T.
Boone
K. B.
Lu
P.
Arnold
G.
Sensitivity and specificity of various digit span scores in the detection of suspect effort
The Clinical Neuropsychologist
 , 
2006
, vol. 
20
 (pg. 
145
-
159
)
Blaskewitz
N.
Merten
T.
Kathmann
N.
Performance of children on symptom validity tests: TOMM, MSVT, and FIT
Archives of Clinical Neuropsychology
 , 
2008
, vol. 
23
 (pg. 
379
-
391
)
Boone
K. B.
The need for continuous and comprehensive sampling of effort/response bias during neuropsychological examinations
The Clinical Neuropsychologist
 , 
2009
, vol. 
23
 (pg. 
729
-
741
)
Brockhaus
R.
Merten
T.
Neuropsychologische diagnostik suboptimalen: Leistungverhaltens mit dem Word Memory Test [in German]
Nervenarzt
 , 
2004
, vol. 
75
 (pg. 
882
-
887
)
Brooks
B. L.
Sherman
E. M. S.
Krol
A. L.
Utility of TOMM Trial1 as an indicator of effort in children and adolescents
Archives of Clinical Neuropsychology
 , 
2012
, vol. 
27
 (pg. 
23
-
29
)
Bush
S. S.
Ruff
R. M.
Troster
A. I.
Barth
J. T.
Koffler
S. P.
Pliskin
N. H.
, et al.  . 
Symptom validity assessment: Practice issues and medical necessity NAN policy & planning committee
Archives of Clinical Neuropsychology
 , 
2005
, vol. 
20
 (pg. 
419
-
426
)
Carone
D. A.
Test review of the medical symptom validity test
Applied Neuropsychology
 , 
2009
, vol. 
16
 (pg. 
309
-
311
)
Chafetz
M. D.
Prentkowki
E. P.
Rao
A.
To work or not to work: Motivation (not low IQ) determines symptom validity test findings
Archives of Clinical Neuropsychology
 , 
2011
, vol. 
26
 (pg. 
306
-
313
)
Constantinou
M.
McCaffrey
R. J.
Using the TOMM for evaluating children's effort to perform optimally on neuropsychological measures
Child Neuropsychology
 , 
2003
, vol. 
9
 (pg. 
81
-
90
)
Dean
A. C.
Victor
T. L.
Boone
K. B.
Arnold
G.
The relationship of IQ to effort test performance
The Clinical Neuropsychologist
 , 
2008
, vol. 
22
 (pg. 
705
-
722
)
Donders
J.
Performance on the Test of Memory Malingering in a mixed pediatric sample
Child Neuropsychology
 , 
2005
, vol. 
11
 (pg. 
221
-
227
)
Faust
D.
Hart
K. J.
Guilmette
T. J.
Pediatric malingering: The capacity of children to fake believable deficits on neuropsychological testing
Journal of Consulting and Clinical Psychology
 , 
1988
, vol. 
56
 (pg. 
578
-
582
)
Gill
D.
Green
P.
Flaro
L.
Pucci
T.
The role of effort testing in independent medical examinations
Medico-Legal Journal
 , 
2007
, vol. 
75
 
Pt 2
(pg. 
64
-
71
)
Green
P.
Green's Medical Symptom Validity Test (MSVT) for Microsoft Windows (user manual)
 , 
2004
Edmonton, Canada
Green's Publishing
Green
P.
Allen
L. M.
Astner
K.
The Word Memory Test: A user's guide to the oral and computer administered forms, US Version 1.1
 , 
1996
Durham, NC
CogniSyst
Green
P.
Flaro
L.
Word memory test performance in children
Child Neuropsychology
 , 
2003
, vol. 
9
 (pg. 
189
-
207
)
Greiffenstein
M. F.
Baker
W. J.
Gola
T.
Validation of malingered amnesia measures with a large clinical sample
Psychological Assessment
 , 
1994
, vol. 
6
 (pg. 
218
-
224
)
Hall
H. V.
Pritchard
D. A.
Detecting malingering and deception
 , 
1996
Delrey Beach, FL
St. Lucie Press
(pg. 
3
-
24
)
Heilbronner
R. L.
Sweet
J. J.
Morgan
J. E.
Larrabee
G. J.
Millis
S. R.
American Academy of Clinical Neuropsychology consensus conference statement on the neuropsychological assessment of effort, response, bias, and malingering
The Clinical Neuropsychologist
 , 
2009
, vol. 
23
 (pg. 
1093
-
1129
)
Heubrock
D.
Münchhausen by proxy syndrome in clinical child neuropsychology: A case presenting with neuropsychological symptoms
Child Neuropsychology
 , 
2001
, vol. 
7
 
4
pg. 
273
 
Hurley
K. E.
Deal
P.
Assessment instruments measuring malingering used with individuals who may have mental retardation: Potential problems and issues
Mental Retardation
 , 
2006
, vol. 
44
 (pg. 
112
-
119
)
Kennedy
C.
Shaver
S.
Weinborn
M.
Manley
J.
Broshek
D.
Marcopulos
B.
Use of the Test of Memory Malingering (TOMM) in Individuals with FSIQ below 70
2005
Poster session presented at the 25th annual meeting of the National Academy of Neuropsychology
Tampa, FL
Kirkwood
M. W.
Hargrave
D. D.
Kirk
J. W.
The value of the WISC-IV Digit Span subtest in detecting noncredible performance during pediatric neuropsychological examinations
Archives of Clinical Neuropsychology
 , 
2011
, vol. 
25
 (pg. 
377
-
384
)
Kirkwood
M. W.
Kirk
J. W.
The base rate of suboptimal effort in a pediatric mild TBI sample: Performance on the Medical Symptom Validity Test
The Clinical Neuropsychologist
 , 
2010
, vol. 
24
 (pg. 
860
-
872
)
Lee
G. P.
Loring
D. W.
Martin
R. C.
Rey's 15-iten visual memory test for the detection of malingering: Normative observations on patients with neurological disorders
Psychological Assessment
 , 
1992
, vol. 
4
 (pg. 
43
-
46
)
MacAllister
W. S.
Nakhutina
L.
Bender
H. A.
Karantzoulis
S.
Carlson
C.
Assessing effort during neuropsychological evaluation with the TOMM in children and adolescents with epilepsy
Child Neuropsychology
 , 
2009
, vol. 
15
 (pg. 
521
-
531
)
Marshall
P.
Happe
M.
The performance of individuals with mental retardation on cognitive tests assessing effort and motivation
Clinical Neuropsychologist
 , 
2007
, vol. 
21
 (pg. 
826
-
840
)
Meyers
J. E.
Volbrecht
M.
Validation of Reliable Digits for detection of malingering
Assessment
 , 
1998
, vol. 
5
 (pg. 
303
-
307
)
Newton
P.
Reddy
V.
Bull
R.
Children's everyday deception and performance on false-belief tasks
British Journal of Developmental Psychology
 , 
2000
, vol. 
18
 (pg. 
297
-
319
)
Oldershaw
L.
Bagby
R. M.
Rogers
R.
Children and deception
Clinical assessment of malingering and deception
 , 
1997
2nd ed.
New York, NY
The Guildford Press
(pg. 
153
-
166
)
Polak
A.
Harris
P. L.
Deception by young children following noncompliance
Developmental Psychology
 , 
1999
, vol. 
35
 (pg. 
561
-
568
)
Power
R. P.
Stoppard
J. M.
Simulation of stable and neurotic personalities on the J.E.P.I by secondary school boys
The Irish Journal of Psychology
 , 
1973
, vol. 
2
 (pg. 
9
-
17
)
Raven
J. C.
Guide to the standard progressive matrices
 , 
1960
London
HK Lewis
Reznek
L.
The Rey 15-item memory test for malingering: A meta-analysis
Brain Injury
 , 
2005
, vol. 
19
 (pg. 
539
-
543
)
Richman
J.
Green
P.
Gervais
R.
Flaro
L.
Merten
T.
Brockhaus
R.
, et al.  . 
Objective tests of symptom exaggeration in independent medical examinations
Journal of Occupational and Environmental Medicine
 , 
2006
, vol. 
48
 (pg. 
303
-
311
)
Rienstra
A.
Spaan
P. E. J.
Schmand
B.
Validation of symptom validity tests using a “Child-model” of adult cognitive impairments
Archives of Clinical Neuropsychology
 , 
2010
, vol. 
25
 (pg. 
371
-
382
)
Shandera
A. L.
Berry
D. T. R.
Clark
J. A.
Schipper
L. J.
Graue
L. O.
Harp
J. P.
Detection of malingered mental retardation
Psychological Assessment
 , 
2010
, vol. 
22
 (pg. 
50
-
56
)
Singhal
A.
Green
P.
Ashaye
K.
Shankar
K.
Gill
D.
High specificity of the Medical Symptom Validity Test in patients with very severe memory impairment
Archives of Clinical Neuropsychology
 , 
2009
, vol. 
24
 (pg. 
721
-
728
)
Stouthamber-Loeber
M.
Loeber
R.
Boys who lie
Journal of Abnormal Child Psychology
 , 
1986
, vol. 
14
 (pg. 
551
-
564
)
Tombaugh
T. N.
Test of Memory Malingering
 , 
1996
Toronto
Multi-Health Systems
Wechsler
D.
The Wechsler Adult Intelligence Scale
 , 
1997
3rd ed.
San Antonio, TX
The Psychological Corporation
Wechsler
D.
The Wechsler Abbreviated Scale of Intelligence
 , 
1999
San Antonio, TX
The Psychological Corporation
Wechsler
D.
The Wechsler Intelligence Scale for Children
 , 
2003
4th ed.
San Antonio, TX
The Psychological Corporation