Abstract

No clinically proven method currently exists to determine if a test taker is feigning or exaggerating symptoms of a specific reading disability (RD) for potential secondary gain (i.e., extra time on examinations, access to bursary funds, or tax benefits). Our objective was to examine the utility of previously proposed symptom validity measures (i.e., the Dyslexia Assessment of Simulation or Honesty [DASH] and the resulting Feigning Index [FI]) in discriminating students with genuine RDs from sophisticated simulators given ample time to prepare, who were warned that noncredible performance could be detected. The DASH correctly classified almost 83% of coached simulators with no false positives. The FI accurately classified 86% of post-secondary students feigning RD without misidentifying any students with a genuine RD, resulting in 91.8% overall classification accuracy. These two methods show promise as a means of detecting noncredible performance in the assessment of RD.

Introduction

Students with disabilities are entering the post-secondary educational sector in increasing numbers (Blackorby & Wagner, 1996; Canadian Association of Disability Service Providers in Postsecondary Education, 1999; Harrison & Wolforth, 2006; McGuire, 1998). Within this population, learning disorders (LDs) are the most frequently accommodated disabilities, affecting almost half of all students receiving services (Harrison & Wolforth, 2006; Ministry of Training, Colleges and Universities, 2008). Normal learning is affected in these individuals due to impairments of neuropsychological processes (Ghelani, Sidhu, Jain, & Tannock, 2004; Kibby, Marks, Morgan, & Long, 2004; Learning Disabilities Association of Ontario, 2003). When those impairments are well accommodated, learning improves substantially. Because of the potential economic and academic benefits of obtaining an LD diagnosis, concerns have been raised regarding the possibility that unimpaired students might feign LDs in order to gain access to these accommodations and supports (Harrison, Edwards, & Parker, 2007; Mullis, 2003; Sullivan, May, & Galbally, 2007).

Students may exaggerate symptoms for a number of reasons and may not always qualify for the label of “malingering” (Kane, 2008). For instance, students seeking updated psychoeducational assessments may be motivated to exaggerate or magnify actual symptoms in order to ensure retention of accommodations provided to them throughout their high-school career (Harrison, Larochette, & Nichols, 2007). These students often come to rely on accommodations such as extra time and a computer and may fear losing them if they now perform adequately on tests of reading speed or decoding. This is of considerable concern because low motivation to perform optimally may invalidate test results and reduce the diagnostic accuracy. Another factor that may interfere with accurate interpretation of test scores and, consequently, incorrect diagnosis is unwillingness to put forth good effort on activities that are frustrating or unappealing (Adelman, Lauber, Nelson, & Smith, 1989).

Until recently, most clinicians assumed that students could not feign a specific LD, such as a reading disability (RD; Alfano & Boone, 2007), and that the base rate for such malingering in psychoeducational assessments was very low; however, this has proven not to be the case. Indeed, Sullivan and colleagues (2007) demonstrated that approximately 16% of post-secondary students undergoing evaluations for LD at their clinic failed a relatively easy symptom validity test (SVT), calling the reliability of their remaining scores into question. Similarly, Harrison, Edwards, and Parker (2008) demonstrated that psychological tests commonly employed in the assessment of RD can be easily manipulated by those instructed to feign a disability. For example, on tests of reading speed, phonological decoding, and visual processing, such students returned scores that were equally or more impaired than those produced by students with genuine RD. This clearly demonstrates the need for an SVT specific to those feigning or exaggerating symptoms of an RD and supports the position taken by Alfano and Boone (2007, p. 373) that “Effort in specific populations is best detected by tests designed to address layperson notions of the deficits associated with that population's disorder”. Not only will development of such an RD-specific SVT improve diagnostic accuracy, but it will also ensure that those who are not truly disabled cannot easily obtain the academic accommodations, financial supports, and other secondary gains derived from being given this diagnosis. A meta-analysis conducted by Alfano and Boone (2007) found that base rates of malingered LD and/or ADHD in post-secondary settings may equal those found in the medical-legal arena, indicating that the incidence of feigned RD is much greater than initially suspected.

In the field of neuropsychology, it is now generally accepted that SVTs must be included in any assessment occurring in a medico-legal or disability-application context (Bush et al., 2005; Green, 2007; Green et al., 2001; Iverson, 2007; Larrabee, 2003). Unfortunately, most of the currently published tests of effort or symptom validity were developed in the context of evaluating individuals feigning memory impairments and may therefore not be sensitive to individuals feigning a specific RD (Alfano & Boone, 2007). Given that students are successfully able to feign many symptoms of RD (Harrison et al., 2008), the question that remains is how best to identify such behavior when it occurs, so as to accurately interpret test scores.

Osmon, Plambeck, Klein, and Mano (2006) were the first to investigate ways to identify those feigning a reading disorder. In their study, two simulation groups were instructed to feign either reading speed or word decoding impairments. When compared with students told to exert good effort, students simulating an RD showed poor performance on a widely used neuropsychological SVT (Green's Word Memory Test [WMT; Green, 2003]). However, although scores on the WMT were highly specific (96%), they were relatively less sensitive to malingering of RD (65%). In contrast, Osmon and colleagues demonstrated that an experimental SVT, the Word Reading Test, was 90% sensitive in identifying reading simulators and 74% sensitive to speed simulators as well, both with 100% specificity. These researchers concluded that the Word Reading Test was more effective than the WMT in detecting feigned RD and advised that any SVT developed to identify individuals feigning RD must address the layperson's notion of how this disorder presents itself.

A similar study was conducted by Frazier and colleagues (2008), in which they demonstrated that two commonly used SVTs (the Victoria SVT [Slick, Hopp, Strauss, & Thompson, 1997] and the Validity Indicator Profile [Frederick, 2003]) could both differentiate simulated RD students from honest respondents at fairly high rates of specificity and sensitivity. The difficulty with both of these studies, however, is that neither included individuals with RD to ensure that these SVTs are insensitive to the actual cognitive impairments demonstrated by those with this condition. In other words, any SVT to be employed in identification of those feigning or exaggerating symptoms of RD must also ensure a low or nonexistent false-positive rate.

The Dyslexia Assessment of Simulation or Honesty (DASH) and the Feigning Index (FI) derived from it have already shown promise as measures of malingered dyslexia, at least in a preliminary investigation (Harrison et al., 2008). The premise of the DASH is based on the work of Rawlinson (1976), who found that scrambling letters in the middle of words (leaving the first and last letters in place) had little or no effect on the ability of skilled readers to understand the text. Davis (2003) expanded on this early research and noted that adults can read mixed-up words if they are small, if key letters are kept together (like “sh” or “th”), or if the word cannot be confused with many other words. The DASH presents Grade 3 reading level passages adapted from the Gray Oral Reading Test (Wiederholt & Bryant, 2001) that have been randomized in either an easy (letters scrambled according to the principles identified by Davis) or difficult (letters within words scrambled randomly) manner, and also includes two timed practice trials (one with nonscrambled words and one with scrambled words). These passages are presented on paper to subjects, who are instructed to unscramble the words and read the passages out loud as quickly as possible. Although this task appears difficult, preliminary investigations showed that post-secondary level students with RD did not find the practice and easy passages challenging to read and that their rate and accuracy were not substantially different from that of their nondisabled peers (Harrison et al., 2008). In contrast, students feigning RD read these passages more slowly and made significantly more errors on other commonly used achievement tests than did either controls or true RD students.

Harrison and colleagues (2008) developed an FI derived from the DASH and other achievement test scores to compare the performance of both honest responders and students diagnosed with RD to those obtained from analog malingerers asked to feign symptoms of RD. Initial piloting of the DASH showed that although it was not able to accurately identify all naïve simulators feigning RD, the FI demonstrated a 75% hit rate and a 0% false-positive rate. Although a strength in the design of this initial study was the inclusion of actual students diagnosed with RD, it remains unclear whether the FI is able to identify the efforts of more sophisticated malingerers who have researched ways of successfully feigning symptoms. Moreover, recent research suggests that coached simulators in other contexts may be better able to feign a disability and avoid detection (e.g., Frederick, Sarfaty, Johnston, & Powel, 1994; Inman & Berry, 2002; Martin, Bolter, Todd, Gouvier, & Niccolls, 1993; Orey, Cragar, & Berry, 2000; Shum, Gorman, & Alpar, 2004). Further replication of these findings using a different sample of students with RD, as well as a more sophisticated group of malingerers, was therefore necessary.

The present study sought to validate the DASH and the FI to determine how well they could identify individuals who had taken time to research ways to feign RD. We expected that students who invested more time and effort exploring means of feigning RD would be able to avoid detection, especially if they were informed ahead of time that noncredible performance could be detected. It was thought that the DASH and the FI would be less sensitive to these sophisticated simulators compared with the scores returned by naïve simulators as reported by Harrison and colleagues (2008) and that the scores obtained by the sophisticated malingerers on achievement and processing tests would more closely mirror those returned by students with true RD.

Materials and Methods

Participants

We recruited two groups of students for the present study, none of whom had been included in the earlier study investigating the DASH. Students in both groups consented to participate in the present ethics-approved study: Group 1 were reading disabled students (n = 20) diagnosed at a university-based regional assessment center between 2008 and 2009; Group 2 were sophisticated malingerers (n = 29), comprised of upper year education or psychology students who had received a 1-hr guest lecture on learning disabilities and then were invited to take part in a study evaluating their ability to successfully feign an RD. Although each subject in the sophisticated malingering group was informed they would be paid $10 for their participation, they were additionally motivated to perform well by hearing that they could double this amount if they performed in a manner that was credible and avoided detection of malingering (c.f. Frederick et al., 1994; Inman & Berry, 2002; Martin et al., 1993; Orey et al., 2000; Shum et al., 2004). However, all participants received $20 after they completed the task regardless of performance.

None of the sophisticated malingerer group had a current diagnosis of a learning disability nor was any receiving scholastic accommodation, but one participant had been diagnosed with a learning disability at an earlier time. (This participant self-described as a good reader, spoke English as a first language and did not have family members with a learning disability.) All of these students enjoyed reading and only one participant self-described as a poor reader. Two participants' first language was not English and one of these participants also read first in a language other than English. Three participants had family members with a learning disability. These participants were not excluded from the analyses because persons who already have some understanding of reading problems might in fact be better able to dissimulate.

Twenty students were diagnosed with a reading disorder at our clinic during the span of the study, and all agreed to participate. RD students in Group 1 had met the DSM-IV diagnostic criteria for a reading disorder, demonstrating a significant discrepancy between measured intellectual functioning and actual academic achievement. In addition, they also met the more stringent Learning Disabilities Association of Ontario (2003) definition of a specific RD, which requires there also be evidence that the academic discrepancy is logically related to measurable deficits in the processes underlying reading (e.g., phonological awareness, naming, or processing speed [PS]). All tests except the DASH had been administered prior to diagnosis. In an effort to ensure that the pattern of test scores obtained from those with RD was an accurate reflection of actual ability, the validity of the obtained test scores for this group was initially evaluated using the Slick, Sherman, and Iverson (1999) criteria. A diagnosis of RD was rendered only if the subject failed to show evidence of malingered neurocognitive dysfunction, assessed in part by the Green WMT (Green, 2003), an instrument that has been used in many other studies as a proxy measure of test taking effort (e.g., Bauer, O'Bryant, Lynch, McCaffrey, & Fisher, 2007; Flaro, Green, & Robertson, 2007; Iverson, Green, & Gervais, 1999; Suhr et al., 2008; Sullivan et al., 2007). In other words, failure on this or any other of the criteria outlined by Slick and colleagues meant that a diagnosis was not provided as their was reasons to suspect that low effort or motivation may have negatively influenced test performance (c.f. Green, Rohling, Lees-Haley, & Allen, 2001). Furthermore, as an additional safeguard to ensure honest responding, the DASH was administered to RD students only after they had received a diagnosis and had been given their assessment report.

Materials

In order to estimate overall intelligence, the sophisticated malingering group completed the North American Adult Reading Test (NAART; Uttl, 2002) before receiving specific study instructions. Research has generally reported moderately high correlations between the NAART and the measures of general intellectual ability (Crawford, Stewart, Cochrane, Parker, & Besson, 1989; Strauss, Sherman, & Spreen, 2006; Wiens, Bryan, & Crossen, 1993). RD subjects had already been assessed with the Wechsler Adult Intelligence Scale-III (Wechsler, 1997) as part of their evaluation, and so did not complete the NAART.

Subjects in both groups (RD and sophisticated) completed the newly developed DASH (Harrison et al., 2008), which involves subjects being asked to read a series of Grade 3 reading level passages aloud as quickly and accurately as possible. First, participants read a very short (40 words) nonscrambled practice passage (DASH A), followed by a short (52 words) practice passage (DASH B), where the letters in larger words are scrambled in an easy manner (e.g., baking spelled “bkaing”), but with two- and three-letter words remaining unchanged, allowing for contextual clues to assist in reading the scrambled words (e.g., “The boy was bkaing a cake for moethr”). Next, participants read a set of two scrambled reading passages. Passage 1 was comprised of 104 words, and Passage 2 had 106 words. (To protect the integrity of the DASH, more detail about the content of the passages has not been included in this paper. Interested readers may contact the corresponding author to obtain a copy of the DASH.) Completion times for all of the DASH passages were recorded, as were the number of errors made, number of extra attempts taken, and number of words skipped or passed. (Errors = words pronounced incorrectly; extra attempts = the extra times words were read or partly read either correctly or incorrectly; skips = words inadvertently missed; passes = words the subject openly said would be passed.)

All participants also completed five subtests from the Woodcock Johnson Psychoeducational Battery-III (WJPB-III; Woodcock, McGrew, & Mather, 2001). The reading fluency (RF) test is a time-limited, norm-based measure of reading speed and accuracy. Letter–word identification and word attack measure phonological processing and decoding skills when reading regular and nonsense words, respectively. Weak ability in these areas is often used as evidence of RD. The PS index is comprised of two timed measures, visual matching (VM), and decision speed (DS). The WJPB-III tests were chosen because academic accommodations such as extra time are frequently recommended for students with RD on the basis of poor performance on timed tests or reading and/or PS. It is also true that speed of information processing deficits have been implicated in LD (Weiler et al., 2000), and it would be important to know how vulnerable such tests are to deliberate manipulation.

Procedure

Students comprising the sophisticated malingering group initially met with a research assistant who first administered the NAART and then provided them with test instructions. These included information regarding the secondary gains that may be available to students who feign in a believable manner and instructions to use all resources available to them in an effort to research how best to feign an RD in a nondetectable manner (i.e., without being caught; copies of the specific instructions given to these participants may be requested from the corresponding author). We asked that they keep track of the websites from which they derived their information. Similar to the Harrison and colleagues (2008) study, these students were also provided with the DSM-IV criteria for diagnosis of a reading disorder, and a list of common symptoms found in those with RD. All other experimental tasks were administered to the sophisticated malingerers seven days later, after they had had time to research reading disorders.

Malingering participants completed the remaining tasks in the following order: The DASH, letter–word identification, word attack, RF, VM, and DS. Participants were debriefed following completion of these measures. During the debriefing, participants were asked what strategies they used to fake RD, and these responses were recorded. They also answered four post-test questions pertaining to strategies employed when feigning, perceived motivation associated with cash prize, perceived success in feigning, and overall effort invested in the task.

As noted above, all RD subjects had completed these same tests as part of their assessment and were given the DASH only after being provided with a diagnosis of RD.

Results

Table 1 shows the demographic information regarding these subjects. The mean level of intelligence in the two groups was not significantly different, t25 = 1.58, p > .05. When scores were collapsed across all scales, the sophisticated malingerers scored lower than the RD group on all WJPB-III scales, F(1, 45) = 63.6, p < .001, r = .77, Cohen's d = 2.4. Pairwise comparisons by subtest confirmed the difference (all ps ≤ .001, see Table 2). Using the WJPB-III subtests, we found that the sophisticated malingerers were very good at appearing to have a reading impairment. Given, however, that there was almost no overlap in test scores between groups, they also appeared to overdo the degree of impairment demonstrated. Cut-off values employed by Harrison and colleagues (2008) for the WJPB-III scaled scores (80 and 85) were again used as proxy measures of impaired reading or processing, corresponding to scores below the 10th percentile and scores greater than 1 SD below the mean, respectively. Table 2 provides the percentage of each group flagged as having impaired functioning using both cut-off scores. Of note, the sophisticated malingerers were successful at achieving low scaled scores on each of the four measures using either cut point criterion. Indeed, 66% of the feigners scored below threshold on all four of the WJPB-III scales, whereas only 65% of the RD group scored below threshold on even one of the WJPB-III scales.

Table 1.

Demographic details by instructional group

 Group
 
 Reading disabled Sophisticated malingerers 
n 20 29 
Age 
M 20.8 20.7 
SD 3.7 1.9 
 Min 17 19 
 Max 32 28 
Sex 
 Percent men 50 34.5 
 Percent women 50 65.6 
FSIQ/NAART 
M 104 108.5 
SD 12.0 5.5 
 Group
 
 Reading disabled Sophisticated malingerers 
n 20 29 
Age 
M 20.8 20.7 
SD 3.7 1.9 
 Min 17 19 
 Max 32 28 
Sex 
 Percent men 50 34.5 
 Percent women 50 65.6 
FSIQ/NAART 
M 104 108.5 
SD 12.0 5.5 
Table 2.

Woodcock Johnson Test Battery Mean Scaled Scores and percentage of participants with scores less than 80 and 85, by group

Group Letter–word identification Word attack Reading fluency Processing speeda 
WJPB-III subtest 
 Reading disabled 85.1 (11.2) 80.9 (11.8) 82.9 (8.5) 93.3 (8.5) 
 Sophisticated malingerers 65.6 (18.6)* 64.7 (19.5)* 64.4 (8.6)* 51.8 (21.1)* 
Scaled scores cut at 80 
 Reading disabled 5 (25%) 9 (45%) 9 (45%) 
 Sophisticated malingerers 19 (66%)* 23 (79%)* 25 (86%)* 26 (90%)* 
Scaled scores cut at 85 
 Reading disabled 9 (45%) 12 (60%) 13 (65%) 2 (10%) 
 Sophisticated malingerers 26 (90%)* 25 (86%)* 25 (86%)* 28 (97%)* 
Group Letter–word identification Word attack Reading fluency Processing speeda 
WJPB-III subtest 
 Reading disabled 85.1 (11.2) 80.9 (11.8) 82.9 (8.5) 93.3 (8.5) 
 Sophisticated malingerers 65.6 (18.6)* 64.7 (19.5)* 64.4 (8.6)* 51.8 (21.1)* 
Scaled scores cut at 80 
 Reading disabled 5 (25%) 9 (45%) 9 (45%) 
 Sophisticated malingerers 19 (66%)* 23 (79%)* 25 (86%)* 26 (90%)* 
Scaled scores cut at 85 
 Reading disabled 9 (45%) 12 (60%) 13 (65%) 2 (10%) 
 Sophisticated malingerers 26 (90%)* 25 (86%)* 25 (86%)* 28 (97%)* 

Note: WJPB-III = Woodcock Johnson Psychoeducational Battery-III. n = 20 for reading disabled and n = 29 for sophisticated malingerers. Values are given in Mean scores (SDs).

aCombined score based on performance of WJPB-III visual matching and decision speed.

*RD − malingering difference significant at p ≤ .01.

Performance on the DASH items varied by group. As seen in Fig. 1, the sophisticated malingering group took more time than the RD group to read both the practice and test passages. As shown in Fig. 2, the sophisticated malingerers made more errors (p < .001), skips (p < .05), and extra attempts (p < .01) than did the RD group. In contrast, malingerers did not make more passes than the RD group (p > .75).

Fig. 1.

Reading time on the DASH items as a function of passage difficulty and group membership.

Fig. 1.

Reading time on the DASH items as a function of passage difficulty and group membership.

Fig. 2.

Error measures on the DASH items as a function of passage difficulty and group membership.

Fig. 2.

Error measures on the DASH items as a function of passage difficulty and group membership.

We used the raw DASH scores (i.e., time taken for each passage in seconds, number of errors, skips, extra attempts, and passes made) in a discriminant function analysis to investigate classification accuracy of participants. Table 3 shows the result of the “leave out one” classification. No RD participant was incorrectly defined as malingering, and a total of five malingerers were incorrectly defined as Honest, resulting in a total classification accuracy of 89.8%. Variables that contributed most strongly to the discrimination included all of the variables that contribute to the FI were weighted highly in the DFA, as was time to complete the difficult DASH passage. Variables related to the number of passes contributed minimally to the final discrimination.

Table 3.

Discriminant function analysis using raw DASH values

Actual group Predicted group membership
 
 Reading disabled (n = 20) Sophisticated malingerers (n = 29) 
Reading disabled 20 (100%) 
Sophisticated malingerers 5 (17.2%) 24 (82.8%) 
Sensitivity .828 PPV 1.00 Canonical correlation .806 
Specificity 1.00 NPV .8 Wilk's Lambda .351 
df = 12 p < .001  
89.8% correctly classified 77.6% cross-validated  
Actual group Predicted group membership
 
 Reading disabled (n = 20) Sophisticated malingerers (n = 29) 
Reading disabled 20 (100%) 
Sophisticated malingerers 5 (17.2%) 24 (82.8%) 
Sensitivity .828 PPV 1.00 Canonical correlation .806 
Specificity 1.00 NPV .8 Wilk's Lambda .351 
df = 12 p < .001  
89.8% correctly classified 77.6% cross-validated  

Notes: DASH = Dyslexia Assessment of Simulation or Honesty; PPV = the probability that an individual who receives a low score is actually malingering; NPV = the probability that an individual with a normal score is responding honestly.

Feigning Index

On the basis of a set of variables that best discriminated between the various groups, Harrison and colleagues (2008) derived an FI from certain DASH variables as well number of errors and abnormally low scores from the WJPB. This index allowed for classification of an individual test taker as belonging to either the feigning or the honestly performing (including RD students) group. The positive predictive value for this FI was 100%, with a negative predictive value of 65%.

In the present study, the FI correctly classified 91.8% of the total sample with a positive predictive value of 100% and a negative predictive value of 90% (Table 4). These figures were consistent with those reported by Harrison and colleagues. Only four malingerers were incorrectly classified; of these, all but one produced a scaled score below 80 on WJPB subtests for PS and RF, and only one had a score below 80 on WJPB subtests for letter–word recognition and word attack skills; this increased to two participants when the cut score was set at 85. A chi-squared analysis (χ2 = 35.2 p < 0.001, Fishers exact test p < 0.001) suggests that the rate of identification of feigners by the FI is better than one would expect by chance alone even in the current sample where feigners outnumbered those with RD.

Table 4.

Discriminant function analysis using the Feigning Index

Actual group Predicted group membership
 
 Reading disorder (n = 20) Sophisticated malingerers (n = 29) 
Reading disorder 20 
Sophisticated malingerers 4 (13.8%) 25 (86.2%) 
Sensitivity = .862 PPV = 1.00 Canonical Correlation = .889 df = 1 
Specificity = 1.00 NPV = .901 Wilk's lambda = .210 p < .001 
Actual group Predicted group membership
 
 Reading disorder (n = 20) Sophisticated malingerers (n = 29) 
Reading disorder 20 
Sophisticated malingerers 4 (13.8%) 25 (86.2%) 
Sensitivity = .862 PPV = 1.00 Canonical Correlation = .889 df = 1 
Specificity = 1.00 NPV = .901 Wilk's lambda = .210 p < .001 

Notes: PPV = the probability that an individual who receives a low score is actually malingering; NPV = the probability that an individual with a normal score is responding honestly. Overall cases correctly classified: 91.8%.

Strategies Used by the Sophisticated Malingerers

Participants in the malingering group were asked to describe the amount of time spent learning about RD, the amount of effort they put forth, estimate their success at faking an RD, and whether or not the cash incentive motivated their efforts. Table 5 lists the strategies that were used by at least 10% of the group.

Table 5.

Listing of the strategies (>10%) used by malingering participants to appear dyslexic

Strategy Percentage of reporting 
Reversed, mixed letters, and numbers 64.3 
Use synonyms or made up words 57.1 
Read slowly 53.6 
Read phonetically 42.9 
Skip words, letters, lines 39.3 
Look/act stressed/distracted 25 
Read haltingly, monotone 17.9 
Be accurate with simple words 17.9 
Perform better on nonword tests 10.7 
Strategy Percentage of reporting 
Reversed, mixed letters, and numbers 64.3 
Use synonyms or made up words 57.1 
Read slowly 53.6 
Read phonetically 42.9 
Skip words, letters, lines 39.3 
Look/act stressed/distracted 25 
Read haltingly, monotone 17.9 
Be accurate with simple words 17.9 
Perform better on nonword tests 10.7 

Scores on the FI were not associated with any of the following measures: Participants' estimate of their success at faking dyslexia, the amount of time participants spent researching dyslexia, nor the motivational factor attached to the cash incentive. However, the FI was associated with participants' perception of their effort. Those who perceived their effort to be highest also scored higher on the FI (r = .39, p < .05). In other words, the harder participants felt they had tried to fake bad, the more easily they were detected.

Discussion

We sought to validate the DASH and the FI to determine how well these measures could identify individuals who had taken time to research ways to feign RD in a credible manner in comparison with students with carefully diagnosed reading disabilities.

In contrast to expectations, sophisticated malingerers were not good at avoiding detection, either by the DASH alone or by using the FI. This was despite the fact that the sophisticated feigning group were better able to produce achievement scores lower than those returned by students with true RD, meaning that on the basis of test scores alone these subjects would have succeeded in feigning RD in almost all cases. The results of the present study therefore provide strong support for the inclusion of an SVT when evaluating post-secondary students for the presence a reading disorder, and point to the vulnerability of such tests to manipulation without the addition of valid measures of effort and motivation.

The DASH appears to work well in large part because those feigning RD tend to overdo the symptoms they are producing, and this tendency has been documented in feigning of other disorders (c.f. Liljequist, Kinder, & Schinka, 1998; Wiggins & Brandt, 1988). In general, those feigning RD read even the easy, nonscrambled practice passages more slowly than did almost all of the true RD subjects, and exaggerated their deficits in an even less credible fashion when asked to decode the scrambled text. Hence, as is true of other popular SVTs, having normative data regarding the performance of those with a true disability allows for even greater classification accuracy when identifying those who are feigning or exaggerating symptoms.

Although the DASH alone achieved an acceptable rate of classifying participants, the present study confirmed that the FI derived from DASH scores plus scores from commonly used WJPB-III subtests produced an even more accurate method of identifying students motivated to feign RD. Although the FI achieved a zero percent false-positive rate, it was also able to accurately classify over 86% of the feigning subjects, a hit rate that is unusual for most SVTs. Further, the 100% positive predictive value coupled with a high negative predictive value is consistent with the previous findings of Harrison and colleagues (2008), indicating the excellent ability of the FI to provide an accurate means of identifying those feigning RD for some reason. Although Osmon and colleagues (2006) were able to correctly classify between 74% and 90% of simulators in comparison with honest students, their experimental Word Reading Test has not yet demonstrated good specificity and sensitivity when discriminating malingerers from genuinely disabled students.

Of those students who successfully avoided detection, one might suppose that they were not very motivated to feign in general, and perhaps simply failed to follow the instructions and just answered all items honestly. This did not prove to be the case, as almost all of those who avoided detection by the FI produced scores on the WJPB subtests that fell within the impaired range as defined in this study. Clearly, there are some students who are able to produce scores that would be interpreted as indicative of an RD and who would also evade detection using the FI. No observable pattern was found in the present study to better identify these individuals; more research is therefore needed to help determine how best to detect such students.

Clinicians may worry that the use of SVTs in psychoeducational assessments could wrongly accuse students of “faking;” however, the results of this investigation show that while simply using achievement scores can indeed result in misclassification of true RD, inclusion of scores from an SVT such as the FI were extremely specific, correctly classifying 86% of the feigning group and 100% of the RD subjects. In this sample, no one would have been falsely accused.

The strategies employed by sophisticated malingerers in an effort to convincingly feign RD differed in interesting ways from those used by the naïve feigners in the Harrison and colleagues (2008) study. Indeed, sophisticated feigners reported that their main strategy was to either reverse or mix up letters or numbers, followed by using synonyms or made up words when reading. Reading slowly was a strategy employed by just over half of the subjects. In contrast, the main feigning strategy reported in the original study (employed by 68% of the students) was to just read more slowly, followed by completing timed tasks more slowly, mispronouncing words, or skipping words, sentences, or lines. This finding echoes that reported by Osmon and colleagues (2006) suggesting that there is no consistency with respect to strategies chosen by those attempting to feign RD, although all strategies employed seem to mirror the lay person's notions of RD. Given that not all who feign RD use the same strategy, it was encouraging to find such a high rate of identification when using the DASH and the FI and suggests that these may be a useful adjunct to any assessment of such disorders.

Unlike studies of sophisticated malingering in the area of brain injury research (e.g., Inman & Berry, 2002; Martin et al., 1993; Orey et al., 2000; Shum et al., 2004), the present study found that none of the information readily accessed by motivated students helped them avoid detection of their deceit. The reason for this may be that although information exists to assist students in producing credible symptom profiles on standard achievement tests, no information is easily available regarding the use of SVTs in psychoeducational assessments. Thus, it is possible that even sophisticated malingerers are currently unaware of the methods used to identify noncredible performance in assessment of RD. Given the research by Ruiz, Drake, Glass, Marcotte, and Van Gorp (2002) showing how easily students could independently discover information on the internet to allow them to escape detection when feigning psychiatric symptoms, it was heartening to discover that similar information about symptom validity testing is not yet widely available in the context of RD assessment. It will therefore be important to protect the integrity of any SVTs developed in this context to minimize the risk that their use becomes widely known and publicized.

Limitations

Although we tried to ensure that all those diagnosed with RD were performing honestly, it is possible that some may not have performed in a credible manner on some aspects of the tests administered. Future studies should therefore administer all WJPB subtests after the diagnosis is rendered to avoid any possible effects of secondary gain on test performance.

It is also possible that the malingering group did not have the same motivation or access to information as would students undergoing a re-assessment to determine whether accommodations will be provided at the post-secondary level. Given that students at the secondary-school level are currently provided with academic accommodations and supports without requiring a formal diagnosis of a disability (Harrison, Larochette, et al., 2007), it is possible that such students fear the academic consequences of losing their long-standing accommodations and may be more savvy regarding the types of deficits expected from persons with RD, especially if they have been working around and writing tests with those who really do have disabilities. Future studies should therefore attempt to include a group of students who have specific knowledge of RD based on first-hand experience. Future studies should also include groups of poor readers to investigate the specificity of the DASH and the FI in such populations.

Given the relatively small sample size and the overrepresentation of those feigning RD, the classification rates may not be stable. Replication with a larger sample is therefore warranted.

In conclusion, our results provide a strong support for the use of the DASH and the FI in the detection of students presenting with noncredible symptoms of a reading disorder. With perfect specificity and a high level of sensitivity, both measures promise to aid clinicians in more accurately diagnosing specific reading disorders when they are present. The utility of the DASH and FI should now be investigated more broadly in a variety of settings with different clinical groups.

Funding

Partial funding for this research was generously provided by the National Academy of Neuropsychology Clinical Research Grant. The publication's contents are solely the responsibility of the authors and do not necessarily represent the views of the funding agency.

Conflict of Interest

None declared.

References

Adelman
H. S.
Lauber
B.
Nelson
P.
Smith
D.
Minimizing and detecting false positive diagnoses of learning disabilities
Journal of Learning Disabilities
 , 
1989
, vol. 
22
 (pg. 
234
-
244
)
Alfano
K.
Boone
K. B.
Boone
K. B.
The use of effort tests in the context of actual versus feigned attention-deficit/hyperactivity disorder and learning disability
Assessment of feigned cognitive impairment: A neuropsychological perspective
 , 
2007
New York
Guilford Press
(pg. 
366
-
383
)
Bauer
L.
O'Bryant
S. E.
Lynch
J. K.
McCaffrey
R. J.
Fisher
J. M.
Examining the Test of Memory Malingering Trial 1 and Word Memory Test Immediate Recognition as screening tools for insufficient effort
Assessment
 , 
2007
, vol. 
14
 (pg. 
215
-
222
)
Blackorby
J.
Wagner
M.
Longitudinal postschool outcomes of youth with disabilities: Findings from the National Longitudinal Transition Study
Exceptional Children
 , 
1996
, vol. 
62
 (pg. 
399
-
413
)
Bush
S.
Ruff
R.
Troster
A.
Barth
J.
Koffler
S.
Pliskin
N.
, et al.  . 
Symptom validity assessment: Practice issues and medical necessity. Official position of the National Academy of Neuropsychology
Archives of Clinical Neuropsychology
 , 
2005
, vol. 
20
 (pg. 
419
-
426
)
Canadian Association of Disability Service Providers in Postsecondary Education.
Towards developing professional standards for students with disabilities in postsecondary education in Canada
 , 
1999
Ottawa, ON
Author
Crawford
J. R.
Stewart
L. E.
Cochrane
R. H.
Parker
D. M.
Besson
J. A.
Construct validity of the National Adult Reading Test: A factor analytic study
Personality and Individual Differences
 , 
1989
, vol. 
10
 (pg. 
585
-
587
)
Davis
M.
Aoccdrnig to a rscheearch at Cmabrigde Uinervtisy
2003
 
Flaro
L.
Green
P.
Robertson
E.
Word Memory Test failure 23 times higher in mild brain injury than in parents seeking custody: The power of external incentives
Brain Injury
 , 
2007
, vol. 
21
 (pg. 
373
-
383
)
Frazier
T. W.
Frazier
A. R.
Busch
R. M.
Kerwood
M. A.
Demaree
H. A.
Detection of simulated ADHD and reading disorder using symptom validity measures
Archives of Clinical Neuropsychology
 , 
2008
, vol. 
23
 (pg. 
501
-
509
)
Frederick
R. I.
VIP (Validity Indictor Profile) manual
 , 
2003
Minneapolis, MN
NCS Pearson
Frederick
R. I.
Sarfaty
S. D.
Johnston
D. J.
Powel
J.
Validation of a detector of response bias on a forced-choice test of nonverbal ability
Neuropsychology
 , 
1994
, vol. 
8
 (pg. 
118
-
125
)
Ghelani
K.
Sidhu
R.
Jain
U.
Tannock
R.
Reading comprehension and reading related abilities in adolescents with reading disabilities and attention-deficit/hyperactivity disorder
Dyslexia
 , 
2004
, vol. 
10
 (pg. 
364
-
384
)
Green
P.
Word Memory Test for Windows: User's manual and program
 , 
2003
Edmonton, AB
Green's Publishing
Green
P.
Boone
K.
Spoiled for choice: Making comparisons between forced-choice effort tests
Malingering of neurocognitive disorders
 , 
2007
New York
Guilford Press
(pg. 
50
-
77
)
Green
P.
Rohling
M. L.
Lees-Haley
P. R.
Allen
L. M.
Effort has a greater effect on test scores than severe brain injury in compensation claimants
Brain Injury
 , 
2001
, vol. 
15
 (pg. 
1045
-
1060
)
Harrison
A. G.
Edwards
M. J.
Parker
K. C. H.
Identifying students faking ADHD: Preliminary findings and strategies for detection
Archives of Clinical Neuropsychology
 , 
2007
, vol. 
22
 (pg. 
577
-
588
)
Harrison
A. G.
Edwards
M. J.
Parker
K. C. H.
Identifying students feigning dyslexia: Preliminary findings and strategies for detection
Dyslexia
 , 
2008
, vol. 
14
 (pg. 
228
-
246
)
Harrison
A. G.
Larochette
A.
Nichols
E.
Students with learning disabilities in postsecondary education: Selected initial characteristics
Exceptionality Education Canada
 , 
2007
, vol. 
17
 (pg. 
135
-
154
)
Harrison
A. G.
Wolforth
J.
Results from a national survey of disability service providers in the Canadian post-secondary system
 , 
2006
 
Unpublished manuscript
Inman
T. H.
Berry
D. T. R.
Cross-validation of indicators of malingering: A comparison of nine neuropsychological tests, four tests of malingering and behavioral observations
Archives of Neuropsychology
 , 
2002
, vol. 
17
 (pg. 
1
-
23
)
Iverson
G.
Identifying exaggeration and malingering
Pain Practice
 , 
2007
, vol. 
7
 (pg. 
94
-
102
)
Iverson
G.
Green
P.
Gervais
R.
Using the Word Memory Test to detect biased responding in head injury litigation
Journal of Cognitive Rehabilitation
 , 
1999
, vol. 
17
 
2
(pg. 
4
-
8
)
Kane
S.
Minimizing malingering and poor effort in the LD/ADHD evaluation process
ADHD Report
 , 
2008
, vol. 
16
 
5
(pg. 
5
-
9
)
Kibby
M. Y.
Marks
W.
Morgan
S.
Long
C. J.
Specific impairment in developmental reading disabilities: A working memory approach
Journal of Learning Disabilities
 , 
2004
, vol. 
37
 (pg. 
349
-
363
)
Larrabee
G. J.
Detection of malingering using atypical performance patterns on standard neuropsychological tests
Clinical Neuropsychologist
 , 
2003
, vol. 
17
 (pg. 
410
-
425
)
Learning Disabilities Association of Ontario.
Recommended best practices for assessment, diagnosis and documentation of learning disabilities.
 , 
2003
 
Liljequist
L.
Kinder
B. K.
Schinka
J.
An investigation of malingering post-traumatic stress disorder on the Personality Assessment Inventory
Journal of Personality Assessment
 , 
1998
, vol. 
71
 (pg. 
322
-
336
)
Martin
R. C.
Bolter
J. F.
Todd
M. E.
Gouvier
W. D.
Niccolls
R.
Effects of sophistication and motivation on the detection of malingered memory performance using a computerized forced-choice task
Journal of Clinical and Experimental Neuropsychology
 , 
1993
, vol. 
15
 (pg. 
867
-
880
)
McGuire
J. M.
Gordon
M.
Keiser
S.
Educational accommodations: A university administrator's view
Accommodations in higher education under the Americans with Disabilities Act (ADA): A no-nonsense guide for clinicians, educators, lawyers and administrators
 , 
1998
New York
Guilford
(pg. 
20
-
45
)
Ministry of Training, Colleges and Universities: Postsecondary Education Division.
Disabilities statistics at Ontario universities for discussion at IDIA
 , 
2008
 
(PowerPoint slides)
Mullis
C.
Trouble on the SAT
PsychologyToday
 , 
2003
 
Orey
S. A.
Cragar
D. E.
Berry
D. T. R.
The effects of two motivational manipulations on the neuropsychological performance of mildly head injured college students
Archives of Clinical Neuropsychology
 , 
2000
, vol. 
15
 (pg. 
335
-
348
)
Osmon
D. C.
Plambeck
E. A.
Klein
L.
Mano
Q.
The Word Reading Test of effort in adult learning disability: A simulation study
Clinical Neuropsychologist
 , 
2006
, vol. 
20
 (pg. 
315
-
324
)
Rawlinson
G. E.
The significance of letter position in word recognition
 , 
1976
Nottingham, UK
Unpublished doctoral dissertation, University of Nottingham
Ruiz
M.
Drake
E.
Glass
A.
Marcotte
D.
Van Gorp
W.
Trying to beat the system: Misuse of the internet to assist in avoiding the detection of psychological symptom dissimulation
Professional Psychology: Research & Practice
 , 
2002
, vol. 
33
 (pg. 
294
-
299
)
Shum
D. H. K.
Gorman
J. G. O.
Alpar
A.
Effects of incentive and preparation time on performance and classification accuracy of standard and malingering-specific memory tests
Archives of Clinical Neuropsychology
 , 
2004
, vol. 
19
 (pg. 
817
-
823
)
Slick
D.
Hopp
G.
Strauss
E.
Thompson
G. B.
VSVT: Victoria Symptom Validity Test, version 1.0, professional manual
 , 
1997
Odessa, FL
Psychological Assessment Resources
Slick
D.
Sherman
E.
Iverson
G.
Diagnostic criteria for malingered neurocognitive dysfunction: Proposed standards for clinical practice and research
Clinical Neuropsychologist
 , 
1999
, vol. 
13
 (pg. 
545
-
561
)
Strauss
E.
Sherman
E. M.
Spreen
O.
A compendium of neuropsychological tests: Administration, norms and commentary
 , 
2006
3rd ed.
New York
Oxford University Press
Suhr
J.
Hammers
D.
Dobbins-Buckland
K.
Zimak
E.
Hughes
C.
The relationship of malingering test failure to self-reported symptoms and neuropsychological findings in adults referred for ADHD evaluation
Archives of Clinical Neuropsychology
 , 
2008
, vol. 
23
 (pg. 
521
-
530
)
Sullivan
B.
May
K.
Galbally
L.
Symptom exaggeration by college adults in attention-deficit hyperactivity disorder and learning disorder assessments
Applied Neuropsychology
 , 
2007
, vol. 
14
 (pg. 
189
-
207
)
Uttl
B.
North American adult reading test: Age norms, reliability, and validity
Journal of Clinical and Experimental Neuropsychology
 , 
2002
, vol. 
24
 (pg. 
1123
-
1137
)
Wechsler
D.
WAIS-III administration and scoring manual
 , 
1997
San Antonio, TX
Psychological Corporation
Weiler
M.
Harris
N.
Marcus
D.
Bellinger
D.
Kosslyn
S.
Waber
D.
Speed of Information Processing in Children Referred for Learning Problems
Journal of Learning Disabilities
 , 
2000
, vol. 
33
 
6
pg. 
538
 
Wiederholt
J. L.
Bryant
B. R.
GORT 4: Gray Oral Reading Tests Examiner's manual.
 , 
2001
Austin, TX
Pro-Ed
Wiens
A. N.
Bryan
J. E.
Crossen
J. R.
Estimating WAIS-R FSIQ from the National Adult Reading Test-Revised in normal subjects
Clinical Neuropsychologist
 , 
1993
, vol. 
7
 (pg. 
70
-
84
)
Wiggins
E. C.
Brandt
J.
The detection of simulated amnesias
Law and Human Behavior
 , 
1988
, vol. 
12
 (pg. 
57
-
77
)
Woodcock
R. W.
McGrew
K. S.
Mather
N.
The Woodcock-Johnson III
 , 
2001
Itasca, IL
Riverside