Abstract

Failure on effort tests usually implies insufficient effort to produce valid cognitive test scores. However, many people with very severe cognitive impairment, such as dementia patients, will produce failing scores on nearly all effort tests. In such patients, effort tests have low specificity. The Medical Symptom Validity Test (MSVT) and the nonverbal MSVT (NV-MSVT) were designed to address this problem. They produce profiles of scores across multiple subtests to facilitate discrimination between low scores from people trying to feign impairment and low scores attributable to severe impairment. To study the specificity of the MSVT and NV-MSVT in people with very severe memory impairment, we tested (a) 10 institutionalized patients with dementia and (b) 10 volunteers who were asked to simulate memory impairment. It was hypothesized that the “possible dementia profile” would be found significantly more often in the dementia patients than in the simulators. The MSVT and the NV-MSVT both displayed 100% specificity in the dementia group, while retaining a combined sensitivity of 80% to suboptimal effort in the simulator group.

Introduction

It is now recommended that objective tests should be used to measure the effort applied during testing, especially if there is any external incentive that might induce the person to exaggerate their deficits by failing to make a full effort to do well on testing (AACN Board of Directors, 2007; Bush et al., 2005). However, Merten, Bossink, and Schmand (2007) have argued that most effort tests have low specificity in people with very severe cognitive impairment. Many people with dementia have severe memory impairment and, in such cases, there is a high risk of a “false positive” on almost any memory-based effort test.

A good example of false-positive classifications in those with very severe impairment comes from the data from 40 people with dementia, whose scores are listed individually in the manual for the Test of Memory Malingering (TOMM, Tombaugh, 1996). Out of 37 testable cases, 10 failed the TOMM, using the cut-off specified in the test manual. This represents a 27% false-positive rate in patients presumed to suffer from genuine severe memory impairment, not counting those who were too impaired to test. A similar problem emerges with young children, whose abilities are much lower than those of healthy adults. Blaskewitz, Merten, and Kathmann (2008) reported that 70% of Grade II children making a good effort failed the Reliable Digit Span (RDS) effort indicator, using the usual adult cut-off (Greiffenstein, Baker, & Gola, 1994) and 62% of Grade III children failed the RDS. Yet almost all of these children passed both the TOMM and the MSVT (Blaskewitz et al., 2008, Table 4). These data illustrate how genuinely low ability can lead to false positives for suboptimal effort in those of low ability, especially in the very young and the elderly. If an effort test like the TOMM or RDS is failed in such cases, it is impossible to tell from the test scores alone whether failure should be attributed to very severe impairment or suboptimal effort. This opens up the possibility that genuine severe impairment will be mistakenly classified as feigned impairment.

One solution to the problem of low specificity for effort tests in cases of severe impairment is to reduce the cut-off score for failure. Unfortunately, that invariably leads to a reduced level of sensitivity to suboptimal effort. The current study attempts to address this problem differently by using effort tests that are based not only on very easy tasks but also on the profile of scores across subtests, with special attention to the magnitude of the difference between scores on easier and harder subtests. The MSVT (Medical Symptom Validity Test; Green, 2004), the NV-MSVT (nonverbal MSVT; Green, 2008), and the WMT (Word Memory Test; Green, 2003; Green & Astner, 1995; Green, Lees-Haley, & Allen, 2003) contain subtests that are very easy for children with severe brain injury (Carone, 2008) for healthy children (Blaskewitz et al., 2008) and for children with developmental disabilities (Green, 2008). Each of these tests consists of multiple subtests varying widely in the level of difficulty. Some people with very severe impairment will fail the easy subtests. However, they produce profiles of test scores that are different from those of simulators. In principle, analyzing these profiles can minimize false-positive classifications in cases with severe cognitive impairment.

The first evidence of the possible value of profile analysis came from German data showing that although dementia patients sometimes failed the easy WMT subtests, they produced a different profile of scores than people asked to simulate impairment (Green, 2005). Simulators scored lower than the mean from dementia patients on very easy subtests but paradoxically higher than the dementia mean on harder tasks, producing scores inconsistent with known patterns of impairment. Using the MSVT, Howe, Anderson, Kaufman, Sachs, and Loring (2007) studied patients with dementia in a memory clinic. The use of profile analysis led to less than 5% of cases being classified as invalid responders (specificity of 96.2%) because most dementia patients failing the easy subtests produced profiles empirically associated with dementia or equivalent severe cognitive impairment. Howe and Loring (2009) explained that, without profile analysis, many dementia cases in their study who failed the easy subtests could have been wrongly labeled as suboptimal effort but the vast majority produced dementia profiles. In contrast, they pointed out that credible dementia profiles on the MSVT were not produced by volunteers who were asked to simulate dementia in a Brazilian study (Green, 2004, Appendix B).

The use of profile analysis with the nonverbal equivalent test, the NV-MSVT, yielded 95% specificity in dementia patients and 100% specificity in good effort volunteers, while retaining 72.5% sensitivity to suboptimal effort in the simulators (Green, 2008). In an independent replication study in Germany (Henry, Merten, Wolf, & Harth, 2009), the specificity of the NV-MSVT as a predictor of suboptimal effort was 98.5% in 65 neurological patients. The specificity of the NV-MSVT was 100% in the 21 dementia patients, using the criteria for profile analysis defined in the test manual. The contribution of the above studies toward understanding symptom validity is that they offer methods that might help in reducing the number of false-positive identifications of suboptimal effort in people with genuine severe memory impairment, arising not only from dementia but from any brain disease.

It may be argued that the “gold standard for specificity” on a memory-based effort test is a test which does not misclassify people who are unable to score any higher than chance levels on very easy forced-choice recognition memory tasks. Such low scores imply that the person has virtually no useful memory and patients of this type are the most likely of all to produce false-positive results. They offer us an opportunity to evaluate effort test specificity in people with genuine severe impairment. The main goal of the current study was to test such patients using both the MSVT and the NV-MSVT. We recruited 10 elderly dementia patients, who were permanent residents of a hospital unit in England specializing in long-term care for people with dementia. In such cases, we would expect to find false positives on almost any effort test. We compared their MSVT and NV-MSVT results with those from 10 adult volunteers who were asked to try to simulate dementia on these tests. It was hypothesized that there would be very few poor effort profiles in the dementia group and more “possible dementia profiles” in the dementia group than in the group simulating impairment on the MSVT and the NV-MSVT.

Materials and Methods

Participants

Before the experiment started, the written protocol was reviewed and approved by both the Cambridgeshire and the Hertfordshire research ethics committees and by the local Research & Development Department. Ten people with dementia were tested while permanent residents of Lister Hospital in Stevenage, England. They are cases labeled D1–D10 in Tables 1–3. Diagnosis was made by the treating psychiatrist, based on history and clinical assessment. Six were diagnosed with Alzheimer's disease and the remaining four had diagnoses of dementia of an undetermined type. Patients with comorbid mental illnesses such as major depression were excluded. The 10 patients in this study represented all the advanced dementia patients considered by the treating staff to be able to participate in the test protocol. The mean age of the dementia group was 81.7 years (SD 4.6) and it consisted of five women and five men. The mean number of years of education in the sample was 10 (SD 2.9, range 5–13). Each patient was given a face-to-face explanation of what would happen in the process of testing and an information sheet about the study and was invited to participate. The question of capacity to give consent was not considered to be a problem by any of the three British ethics committees because, unlike drug trials which pose significant risks to the person taking part in the study, the current test procedures were harmless and posed no risk.

Table 1.

Results of the MSVT in 10 volunteer simulators and 10 patients with advanced dementia

 MSVT scores in % correct
 
Failure criteria
 
 IR DR CNS PA FR A: Fail easy subtests B1: Easy–hard (<20) B2: Order violation in scores 
Simulator Group 
 S1 85 100 85 80 80 Yes Yes Yes 
 [S2] 50 45 35 20 15 Yes No No 
 S3 50 50 60 50 50 Yes Yes Yes 
 [S4] 75 65 90 50 35 Yes No No 
 S5 45 45 60 20 25 Yes No Yes 
 S6 55 40 65 50 15 Yes No Yes 
 S7 45 45 30 50 55 Yes Yes Yes 
 S8 40 60 50 40 40 Yes Yes Yes 
 [S9] 55 65 70 30 25 Yes No No 
 [S10] 70 55 55 30 20 Yes No No 
 Mean 57 57 60 42 36 — — — 
SD 15 18 19 18 21 — — — 
Dementia Group 
 D1 70 75 95 10 Yes No No 
 D2 75 60 75 Yes No No 
 D3 65 65 40 50 Yes No No 
 D4 60 65 55 10 Yes No No 
 D5 85 95 80 70 20 Yes No No 
 D6 70 70 50 50 20 Yes No No 
 D7 70 75 65 30 25 Yes No No 
 D8 80 55 55 50 10 Yes No No 
 D9 80 85 65 60 20 Yes No No 
 D10 45 50 45 40 10 Yes No No 
 Mean 70 70 63 37 11 — — — 
SD 12 14 17 24 11 — — — 
 MSVT scores in % correct
 
Failure criteria
 
 IR DR CNS PA FR A: Fail easy subtests B1: Easy–hard (<20) B2: Order violation in scores 
Simulator Group 
 S1 85 100 85 80 80 Yes Yes Yes 
 [S2] 50 45 35 20 15 Yes No No 
 S3 50 50 60 50 50 Yes Yes Yes 
 [S4] 75 65 90 50 35 Yes No No 
 S5 45 45 60 20 25 Yes No Yes 
 S6 55 40 65 50 15 Yes No Yes 
 S7 45 45 30 50 55 Yes Yes Yes 
 S8 40 60 50 40 40 Yes Yes Yes 
 [S9] 55 65 70 30 25 Yes No No 
 [S10] 70 55 55 30 20 Yes No No 
 Mean 57 57 60 42 36 — — — 
SD 15 18 19 18 21 — — — 
Dementia Group 
 D1 70 75 95 10 Yes No No 
 D2 75 60 75 Yes No No 
 D3 65 65 40 50 Yes No No 
 D4 60 65 55 10 Yes No No 
 D5 85 95 80 70 20 Yes No No 
 D6 70 70 50 50 20 Yes No No 
 D7 70 75 65 30 25 Yes No No 
 D8 80 55 55 50 10 Yes No No 
 D9 80 85 65 60 20 Yes No No 
 D10 45 50 45 40 10 Yes No No 
 Mean 70 70 63 37 11 — — — 
SD 12 14 17 24 11 — — — 

Notes: Square Brackets show simulators who produced a possible dementia profile on MSVT using criteria A, B1, and B2. MSVT = Medical Symptom Validity Test; IR = Immediate Recognition; DR = Delayed Recognition; CNS = Consistency; PA = Paired Associate subtest; FR = Free Recall subtest.

Table 2.

NV-MSVT results in the same 10 volunteer simulators and 10 patients with advanced dementia shown in Table 1

 NV-MSVT scores in % correct
 
Failure criteria
 
 IR DR CNS DRA DRV PA FR A: Fail easy subtests B1: PA is too high B2: Easy–hard (<20) 
Simulator Group 
 S1 100 95 95 100 90 90 75 No No No 
 [S2] 45 35 50 40 30 20 30 Yes No Yes 
 S3 50 50 60 45 20 40 40 Yes Yes Yes 
 [S4] 75 70 75 45 30 50 10 Yes Yes No 
 S5 55 35 50 65 30 20 25 Yes No No 
 S6 55 60 45 75 20 50 55 Yes Yes Yes 
 S7 45 55 60 70 30 60 65 Yes Yes Yes 
 S8 45 45 70 60 60 50 25 Yes Yes Yes 
 [S9] 55 60 55 50 50 50 30 Yes Yes Yes 
 [S10] 50 50 60 50 40 40 40 Yes Yes Yes 
 Mean 58 56 62 60 40 47 40 Yes Yes Yes 
SD 17 18 15 18 22 20 20 — — — 
Dementia group 
 D1 35 30 55 65 50 Yes No No 
 D2 65 70 55 55 60 15 Yes No No 
 D3 60 55 45 65 50 30 Yes No No 
 D4 55 80 55 70 40 20 Yes No No 
 D5 90 95 85 90 70 80 20 Yes Yes No 
 D6 70 40 40 70 60 30 20 Yes No No 
 D7 60 60 70 70 40 40 15 Yes No No 
 D8 30 55 65 75 40 Yes No No 
 D9 95 85 90 100 80 60 Yes No No 
 D10 60 70 50 70 60 30 10 Yes No No 
 Mean 62 64 61 73 51 33 Yes No No 
SD 20 20 17 13 22 25 — — — 
 NV-MSVT scores in % correct
 
Failure criteria
 
 IR DR CNS DRA DRV PA FR A: Fail easy subtests B1: PA is too high B2: Easy–hard (<20) 
Simulator Group 
 S1 100 95 95 100 90 90 75 No No No 
 [S2] 45 35 50 40 30 20 30 Yes No Yes 
 S3 50 50 60 45 20 40 40 Yes Yes Yes 
 [S4] 75 70 75 45 30 50 10 Yes Yes No 
 S5 55 35 50 65 30 20 25 Yes No No 
 S6 55 60 45 75 20 50 55 Yes Yes Yes 
 S7 45 55 60 70 30 60 65 Yes Yes Yes 
 S8 45 45 70 60 60 50 25 Yes Yes Yes 
 [S9] 55 60 55 50 50 50 30 Yes Yes Yes 
 [S10] 50 50 60 50 40 40 40 Yes Yes Yes 
 Mean 58 56 62 60 40 47 40 Yes Yes Yes 
SD 17 18 15 18 22 20 20 — — — 
Dementia group 
 D1 35 30 55 65 50 Yes No No 
 D2 65 70 55 55 60 15 Yes No No 
 D3 60 55 45 65 50 30 Yes No No 
 D4 55 80 55 70 40 20 Yes No No 
 D5 90 95 85 90 70 80 20 Yes Yes No 
 D6 70 40 40 70 60 30 20 Yes No No 
 D7 60 60 70 70 40 40 15 Yes No No 
 D8 30 55 65 75 40 Yes No No 
 D9 95 85 90 100 80 60 Yes No No 
 D10 60 70 50 70 60 30 10 Yes No No 
 Mean 62 64 61 73 51 33 Yes No No 
SD 20 20 17 13 22 25 — — — 

Notes: Square brackets identify simulators who produced a “possible dementia” profile on the MSVT using criteria A, B1, and B2. MSVT = Medical Symptom Validity Test; NV-MSVT = nonverbal MSVT; IR = Immediate Recognition; DR = Delayed Recognition; CNS = Consistency; PA = Paired Associate subtest; FR = Free Recall subtest; DRA = Delayed Recognition-Archetypes; DRV = Delayed Recognition-Variations.

Table 3.

Details of criteria A, B1, and B2 in all cases and the presence of possible dementia profiles in each group (far right)

Group MSVT criteria
 
Summary 1 (MSVT) NV-MSVT criteria
 
Summary 2 (NV-MSVT) Summary 3 (both tests) 
 B1 B2 Meets (A and B1) or (A and B2) on MSVT B1 B2 Meets all A, B1, and B2 on NV-MSVT “Possible dementia profile” on both MSVT and NV-MSVT 
Simulator 
 S1 Yes No No 
 [S2] No No Yes 
 S3 Yes Yes No 
 [S4] No No Yes 
 S5 Yes No No 
 S6 Yes Yes No 
 S7 Yes Yes No 
 S8 Yes Yes No 
 [S9] No Yes No 
 [S10] No Yes No 
Dementia 
 D1 No No Yes 
 D2 No No Yes 
 D3 No No Yes 
 D4 No No Yes 
 D5 No No Yes 
 D6 No No Yes 
 D7 No No Yes 
 D8 No No Yes 
 D9 No No Yes 
 D10 No No Yes 
Group MSVT criteria
 
Summary 1 (MSVT) NV-MSVT criteria
 
Summary 2 (NV-MSVT) Summary 3 (both tests) 
 B1 B2 Meets (A and B1) or (A and B2) on MSVT B1 B2 Meets all A, B1, and B2 on NV-MSVT “Possible dementia profile” on both MSVT and NV-MSVT 
Simulator 
 S1 Yes No No 
 [S2] No No Yes 
 S3 Yes Yes No 
 [S4] No No Yes 
 S5 Yes No No 
 S6 Yes Yes No 
 S7 Yes Yes No 
 S8 Yes Yes No 
 [S9] No Yes No 
 [S10] No Yes No 
Dementia 
 D1 No No Yes 
 D2 No No Yes 
 D3 No No Yes 
 D4 No No Yes 
 D5 No No Yes 
 D6 No No Yes 
 D7 No No Yes 
 D8 No No Yes 
 D9 No No Yes 
 D10 No No Yes 

Notes: Square brackets show simulators who produced a “possible dementia” profile on the MSVT using criteria A, B1, and B2. MSVT = Medical Symptom Validity Test; NV-MSVT = nonverbal MSVT.

Ten healthy adult volunteers were recruited from the staff in the hospital and colleagues of the researchers (AS and DG) and they are listed as simulator cases S1–S10 in Tables 1–3. The volunteer simulators were told that the MSVT and the NV-MSVT measure effort and memory and that people with dementia produce certain patterns of scores but they were not told what patterns. They were asked to take the MSVT and the NV-MSVT and to try to simulate memory impairment of the type they would expect to see in early dementia. The reason they were asked to simulate “early” dementia was that we wanted them to adopt a subtle approach, such that they would try to minimize the chance of being detected as simulating by the automated rules of interpretation. Their mean age was 36 years (SD 10) and they had an above average level of education (mean 17 years, SD 2).

Test Procedures

All participants were tested with the MSVT and the NV-MSVT according to the instructions in the test manual. Testing the patients was very challenging because of their advanced dementia and it took 45–90 min per patient for both tests combined, including the 10 min delay between Immediate and Delayed Recognition (IR and DR) trials on both tests. None of the patients were capable of operating the computer and so they were allowed to indicate their choices by pointing or saying the item name aloud while their responses were entered into the computer by the researcher (AS). This is a standard practice for those who are very impaired. In some cases, instructions needed to be repeated with every single change of the computer screen within the same subtest and it is questionable whether some of the patients were able to retain an instructional set for more than a few seconds. They were so impaired that they would be very likely to produce false positives, even on easy effort tests. For the volunteer simulators, testing took about 5–7 min on each task plus the 10 min delay for each test between IR and DR.

Medical Symptom Validity Test

The computerized MSVT (Green, 2004) involves presenting a list of 10 word pairs on screen and then testing recognition memory using a target-foil combination (IR). A similar recognition memory test is administered after a 10 min delay (DR). Then the slightly harder Paired Associate subtest (PA) is given, followed by the Free Recall subtest (FR). Criterion A is met if the score on the IR or DR subtest or on consistency (CNS) between the two is at or below the cut-off specified in the test manual. When criterion A is met, we have to conclude that either (a) effort is poor or (b) there is very severe impairment, equivalent to that seen in dementia and the decision is made using criteria B1 and B2. Criterion B1 is met when the mean score on the easy subtests minus the mean score on the harder subtests is less than 20 points, in which case the results would be described as implausible, probably unreliable and suggestive of suboptimal effort (Green, 2004; Howe and Loring, 2009). Conversely, if the mean of the easier subtests is at least 20 points higher than the mean of the harder subtests (criterion B1 not met), then a “possible dementia profile” is present. If so, suboptimal effort would be concluded only if very severe impairment can be ruled out based on other information. Criterion B2 is met when there is an “order violation” on the MSVT, meaning that a score on a harder subtest matches or exceeds a score on an easier subtest, suggesting inconsistent effort and unreliable data. The use of A and B1 alone produces higher specificity in dementia than the use of all the three criteria but to remain consistent with the Howe and coworkers (2007, 2009) papers, B2 will also be reported in this study.

Nonverbal Medical Symptom Validity Test

The computerized NV-MSVT (Green, 2008) is similar to the MSVT but there are no words on screen and there are six subtests. Ten paired images drawn by an artist (e.g., horse and cart) are presented twice on a computer screen, followed by four recognition memory subtests, called IR, DR, DR-Archetypes (DRA), and DR-Variations (DRV). There is then a PA recall task on which the person sees one part of the image (e.g., horse) and is asked to say what went with it in the original list (cart). Finally, on FR, the person has to recall as many images as possible from the original list. The NV-MSVT program calculates consistency of responses between the IR and DR subtests (CNS).

Henry and colleagues (2009) created an excellent flow chart to illustrate in a simple way a series of rules for classifying results on the NV-MSVT, based on both the mean scores on easy subtests and on differences between scores on easier versus harder subtests. The person meets criterion A if the mean of the easy subtests (IR, DR, CNS, DRV, DRA, and PA or DR, CNS, DRA, and DRV) is below a certain level, as described in the test manual. Using criterion A, no good effort volunteer was wrongly classified as suboptimal effort but 97.5% of simulators failed the NV-MSVT (Green, 2008). When criterion A is met, we have to conclude either (a) that effort is suboptimal or (b) that there is very severe impairment, equivalent to that seen in dementia. To be classified as a “suboptimal effort profile” on the NV-MSVT, the person's scores must meet not only criterion A but also criteria B1 and B2. If the person fails the easy subtests (criterion A) but does not meet both B1 and B2, the profile is called a “possible dementia profile” (i.e., rule out very severe memory impairment from dementia or another condition). Criterion B1 is met when the PA score is not at least 11 points below the mean of the previous four scores in the chart (DR, CNS, DRA, and DRV). Criterion B2 is met when the mean of the easier subtests (IR + DR + CNS) is not at least 20 points higher than the mean of the two most difficult subtests (PA and FR); 72.5% of simulators in the test manual met all three criteria (A, B1, and B2) but 95% of dementia cases did not meet all three criteria (Green, 2008).

Results

Severity

On the Mini-Mental State Examination (MMSE; Folstein, Folstein, & McHugh, 1975), the mean score recorded in patient files was 15.5 (SD 5.3). Using the categories from the test manual, no case fell into the “normal” or “mild cognitive impairment” range on the MMSE (21+), nine cases fell into the “moderate impairment” range (10–20) and one case was in the “severe impairment” range (<10). The memory impairment in this sample was severe, as shown by mean scores in the chance range on very easy forced-choice subtests of the MSVT (IR and DR) and NV-MSVT (IR, DR, and DRV in Tables 1 and 2). Scores in the chance range on these extremely easy recognition memory tests suggest no useful memory for very recent events.

Group Differences on MSVT and NV-MSVT Subtests

Using nonparametric statistics because the data on MSVT IR, DR, CNS, and PA are not normally distributed, there were no significant differences between the simulators and the dementia patients on IR, CNS, or PA but the simulators scored significantly lower than the dementia patients on the DR subtest (Mann–Whitney U-test, p < .05, Table 1). The FR scores approximate a normal distribution in large clinical samples and the simulator's mean FR score was significantly and very obviously higher than that of the dementia patients (36, SD 21 vs. 10.5, SD 11; F (1,18) = 12.1, p < .01). In Table 2, the mean scores of the simulators on the easiest NV-MSVT subtests (IR, DR, DRA, and DRV) were all the same as or lower than the mean scores in the group of patients with advanced dementia. Using nonparametric statistics because the data on IR, DR, CNS, DRA, DRV, and PA are not normally distributed, there were no significant differences between the simulators and the dementia patients on these variables (Mann–Whitney). However, the simulators' mean score (40% correct, SD 20) was more than four times higher than that of the dementia patients (9% correct, SD 9) on the most difficult subtest (FR), which is normally distributed (F = 20.2, df 1, 18, p < .001).

Criteria A, B1, and B2

In each column of Table 3, “1” indicates that the criterion is met and “0” means that it is not met. In the summary columns 1 and 2, it is stated whether or not each case would be classified as “possible dementia” on the test, using criteria A, B1, and B2. The far right column (“Summary 3”) indicates cases in which a possible dementia profile was found on both tests. All simulators and all dementia patients in this study failed the easy MSVT subtests, meeting criterion A. However, the easy–hard difference was at least 20 points on the MSVT in all dementia cases and so no patient met both criteria A and B1 (Table 1). They all displayed the “possible dementia profile.” With results of this type, we would need to rule out dementia before concluding suboptimal effort. Dementia could not be ruled out in any of these dementia patients. Four simulators (S2, S4, S9, and S10 in Table 1) managed to achieve a possible dementia profile on the MSVT because they failed on criterion A but they passed B1 and B2.

All dementia patients met criterion A on the NV-MSVT, producing average scores of only 62% and 64% correct on the IR and DR subtests. Despite their severe memory impairment, shown by very low scores on the easy subtests, no dementia patient met all three of criteria A, B1, and B2 on the NV-MSVT, which were determined to provide optimal discrimination between simulators and dementia cases in the NV-MSVT test manual (Green, 2008). On the other hand, 6 out of 10 of the simulators met all three criteria. All 10 dementia cases produced a possible dementia profile on both the MSVT and the NV-MSVT (100% specificity) but only two simulators did so (80% sensitivity). In clinical cases producing dementia profiles, other data are used to determine if dementia is credible and, if not, poor effort is concluded.

Discussion

By definition, false-positive results imply that people are trying their best to do well but that they fail easy tests because of severe impairment. Therefore, the highest risk of false-positive classifications of suboptimal effort on memory tasks occurs in people with very severe memory impairment. A person who has literally no memory for previously presented stimuli will score in the chance range on the very easy recognition memory subtests of the MSVT and NV-MSVT and on most symptom validity tests. The long-term patients with dementia in the current study did score in the chance range on these subtests. Nevertheless, the results in Tables 1–3 reveal that none of the dementia patients were wrongly labeled as making a suboptimal effort. They failed the easy subtests of the MSVT and the NV-MSVT but their profiles were of the type seen in dementia.

The MSVT and NV-MSVT are often used clinically in combination. In the current study, combined profile analysis yielded 80% sensitivity to suboptimal effort in simulators and 100% specificity for genuine impairment in dementia, exceeding the performance of either test alone (Table 3). Caution is indicated in generalizing from these results because the sample size is small but they suggest that, in clinical testing, the analysis of MSVT and NV-MSVT profiles will probably yield few false positives for suboptimal effort. Another reason why the current findings need to be independently replicated is that they are original and novel. It has been argued that people with chance level recognition memory, like the dementia patients in this study, represent a gold standard for judging the specificity of effort tests. With the exception of the WMT (Montijo, 2008), no other effort test has been shown to be capable of approaching 100% specificity in any group with dementia. The high specificity for the MSVT in this study supports the conclusions of Howe and coworkers (2007, 2009) that profile analysis reduces false positives on the MSVT. The high specificity of the NV-MSVT in this study replicates previous findings in dementia patients reported in the test manual (Green, 2008) and it is further supported by the findings of the independent German study showing 100% specificity for the NV-MSVT in dementia (Henry et al., 2009). In the latter study, the dementia group's mean scores on the forced-choice subtests were well above the upper limit of the chance range, meaning that the memory impairment in that study was not as severe as in the dementia patients in our sample. The current study is original because it demonstrates high specificity for both the MSVT and NV-MSVT in people with chance level recognition memory test scores.

Assuming that further replication studies support current findings with the MSVT and NV-MSVT, we may question the validity of test results if a person fails the easy subtests of the MSVT or NV-MSVT but the person is clearly not suffering from dementia or any equivalent severe cognitive impairment. If there is a possible dementia profile present on both tests, it is necessary to rule out very severe impairment, based on other historical and clinical evidence, before concluding that failure is a result of suboptimal effort. Studies of profiles of scores on the MSVT and NV-MSVT and on other tests in people with severe memory impairment are to be encouraged because, in principle, they offer a way to minimize false-positive classifications in single cases.

One limitation to the study, in addition to a small sample size, is that it did not assess sensitivity to suboptimal effort in clinical cases with real incentives to exaggerate memory impairment. This will be necessary for the combination of tests to be used clinically because sensitivity to suboptimal effort is as important for an effort test as specificity. It is notable that the simulators in the current study produced the same profile on the MSVT found independently by various investigators in English, German, French, or Brazilian Portuguese (Green, 2004). On both the MSVT and NV-MSVT, the simulators scored as low as or lower than people with dementia on very easy subtests but higher than the dementia mean on the harder subtests. Such a pattern is internally inconsistent and it is the same pattern found in cases making a suboptimal effort, when tested with the MSVT as part of independent medical examinations for disability purposes (Gill, Green, Flaro, & Pucci, 2007; Richman et al., 2006) or when tested with the NV-MSVT (Green, 2008). What we do know with near certainty is that none of the simulators were performing as well as they were capable of performing on the tests employed and that this was detected in 80% of cases. Yet no dementia case was misclassified as making a suboptimal effort. These promising findings need to be replicated with larger sample sizes employing both the MSVT and the NV-MSVT. The simulators in this study were more sophisticated than the average person with regard to memory and effort testing because they were professionals working in a psychiatric hospital where effort tests are employed or they were relatives of such people. It is possible that higher sensitivity to poor effort would have been observed if people with no professional links to a hospital had been tested.

If profile analysis yields very few false positives in people with extremely severe memory impairment, we would expect false positives to be even lower in people with diagnoses implying minimal or no memory impairment. A good example would be the adults with mild head injuries who failed the MSVT in the study of Carone (2008). Their mean MSVT profile was not consistent with the “possible dementia profile” but it was typical of people making a suboptimal effort. We may predict that false positives for poor effort on the MSVT and NV-MSVT will be low in people with traumatic brain injury because dementia patients with very severe impairment and chance level scores are rarely misclassified but this will need to be determined separately. Of particular interest will be people with mild traumatic brain injury with and without external incentives to appear impaired. Further research with developmentally disabled children will be important to determine how many fail the easy subtests of the MSVT and the NV-MSVT and with what profiles. There is also the potential for studying these tests as diagnostic tests for identifying people with dementia but this was not the purpose of the current study.

Funding

Dr. Singhal (A.S.) used research time as part of his contract with Hertfordshire Partnership NHS Foundation Trust. Otherwise there was no research grant or funding from any source.

Conflict of Interest

With the exception of PG, the current authors have no financial or business interest in the tests used in this study. PG is the author and owner of the computerized tests used in the study. He was not involved in the testing of any participants but was involved in planning the study design and writing this report.

References

AACN Board of Directors.
American Academy of Clinical Neuropsychology (AACN) practice guidelines for neuropsychological assessment and consultation
The Clinical Neuropsychologist
 , 
2007
, vol. 
21
 (pg. 
209
-
231
)
Blaskewitz
N.
Merten
T.
Kathmann
N.
Performance of children on symptom validity tests: TOMM, MSVT, and FIT
Archives of Clinical Neuropsychology
 , 
2008
, vol. 
23
 (pg. 
379
-
391
)
Bush
S.
Ruff
R.
Troster
A.
Barth
J.
Koffler
S.
Pliskin
N.
, et al.  . 
Symptom Validity Assessment: Practice issues and medical necessity (NAN Policy and Planning Committee)
Archives of Clinical Neuropsychology
 , 
2005
, vol. 
20
 (pg. 
419
-
426
)
Carone
D.
Children with moderate/severe brain damage/dysfunction outperform adults with mild to no brain damage on the Medical Symptom Validity Test
Brain Injury
 , 
2008
, vol. 
22
 
12
(pg. 
960
-
971
)
Folstein
M. F.
Folstein
S. E.
McHugh
P. R.
Mini-Mental State: A practical method for grading the cognitive state of patients for the clinician
Journal of Psychiatric Research
 , 
1975
, vol. 
12
 (pg. 
189
-
198
)
Gill
D.
Green
P.
Flaro
L.
Pucci
T.
The role of effort testing in independent medical examinations
The Medico-Legal Journal
 , 
2007
, vol. 
75
 (pg. 
64
-
72
)
Green
P.
Manual for the Word Memory Test for Microsoft Windows
 , 
2003
Edmonton
Green's Publishing
Green
P.
Test manual for the Medical Symptom Validity Test
 , 
2004
Edmonton
Green's Publishing
Green
P.
Appendix F; how simulators differ from dementia patients
Manual for the Word Memory Test for Microsoft Windows
 , 
2005
revised
Edmonton
Green's Publishing
Green
P.
Test manual for the Nonverbal Medical Symptom Validity Test
 , 
2008
Edmonton
Green's Publishing
Green
P.
Astner
K.
Manual for the Oral Word Memory Test
 , 
1995
NC
Cognisyst. Now out-of-print but available from Green's Publishing, Edmonton
Green
P.
Lees-Haley
P.
Allen
L.
Hom
J.
Denney
R. L.
The Word Memory Test and the validity of neuropsychological test scores
Detection of response bias in forensic neuropsychology
 , 
2003
New York
Haworth Medical Press
Greiffenstein
M. F.
Baker
W. J.
Gola
T.
Validation of malingered amnesia measures with a large clinical sample
Psychological Assessment
 , 
1994
, vol. 
6
 (pg. 
218
-
224
)
Henry
M.
Merten
T.
Wolf
S.
Harth
S.
Nonverbal Medical Symptom Validity Test performance of elderly healthy adults and clinical neurology patients
Journal of Clinical and Experimental Neuropsychology
 , 
2009
, vol. 
iFirst
 (pg. 
1
-
10
Apr 8 [Epub ahead of print]
Howe
L.
Anderson
A.
Kaufman
D.
Sachs
B.
Loring
D.
Characterization of the Medical Symptom Validity Test in evaluation of clinically referred memory disorders clinic patients
Archives of Clinical Neuropsychology
 , 
2007
, vol. 
22
 (pg. 
753
-
761
)
Howe
L. S.
Loring
D. W.
Classification accuracy of the Medical Symptom Validity Test's dementia profile and general memory impairment profile
The Clinical Neuropsychologist
 , 
2009
, vol. 
23
 (pg. 
329
-
342
)
Merten
T.
Bossink
L.
Schmand
B.
On the limits of effort testing: Symptom Validity Tests and severity of neurocognitive symptoms in nonlitigating patients
Journal of Clinical and Experimental Neuropsychology
 , 
2007
, vol. 
29
 (pg. 
308
-
318
)
Montijo
J. A.
Calaf
M.
Mercado
R.
Carrión-Baralt
R. J.
The Diagnostic Utility of Green s Word Memory Test in the Assessment of Dementia in a Puerto Rican Sample
Archives of Clinical Neuropsychology
 , 
2008
, vol. 
23
 
6
pg. 
633
 
Richman
J.
Green
P.
Gervais
R.
Flaro
L.
Merten
T.
Brockhaus
R.
, et al.  . 
Objective Tests of Symptom Exaggeration in independent medical examinations
Journal of Occupational and Environmental Medicine
 , 
2006
, vol. 
48
 
3
(pg. 
303
-
311
)
Tombaugh
T.
Test of Memory Malingering
 , 
1996
Toronto
Multi-Health Systems