Abstract

Ethnically diverse examinees tend to perform lower on neuropsychological tests. The practice of adjusting normative comparisons for the education level and/or race to prevent overpathologizing low scores is problematic. Education quality, as measured by reading recognition, appears to be a more accurate benchmark for premorbid functioning in certain populations. The present study aimed to extend this line of research to traumatic brain injury (TBI). We hypothesized that a measure of reading recognition, the Wechsler Test of Adult Reading (WTAR), would account for racial differences in neuropsychological performance after TBI. Fifty participants (72% African American, 28% Caucasian) with moderate to severe TBI underwent neuropsychological testing at 1-year post-injury. Reading recognition accounted for all the same variance in neuropsychological performance as race and education (together), as well as considerable additional variance. Estimation of premorbid functioning in African Americans with TBI could be refined by considering reading recognition.

Introduction

Compared wuth Caucasians, African Americans have been found to score lower on a range of neuropsychological tests (e.g., Manly, Jacobs, et al., 1998; Manly, Jacobs, Touradji, Small, & Stern, 2002; Norman, Evans, Miller, & Heaton, 2000; Schwartz et al., 2004; Welsh et al., 1995), raising the risk for false-positive diagnostic errors or overpathologizing in African Americans (Manly et al., 1998; Welsh et al., 1995). Using race-specific normative data in neuropsychological test interpretation has been proposed as an interim solution to this problem (seeManly, 2005; Nabors, Evans, & Strickland, 2000). However, race is a complex phenomenon, not a unitary or biological construct with clear boundaries into which people can be easily grouped (Kaplan & Bennett, 2003; Manly, 2005, 2006; Nabors et al., 2000). In addition, racial differences may have nothing to do with race, as conventionally defined by self-identification. Other confounding variables such as education history may better explain disparities in neuropsychological performance (Manly, 2005, 2006; Nabors et al., 2000; Ryan et al., 2005).

Adjusting normative comparisons for education level (i.e., years of formal education completed) is insufficient to eliminate racial differences, probably because of the historically lower quality of education for many African Americans (Manly, 2005; Nabors et al., 2000; Ryan et al., 2005). In other words, because of disparities in education quality or achievement, years of education may not translate into the same expected ability level for Caucasians and African Americans. Adjusting obtained scores for word recognition performance, considered a proxy measure of education “quality” and its corollaries (e.g., literacy; Manly et al., 2002; Schneider & Lichtenberg, 2011), has emerged as a promising alternative. Manly and colleagues (2002) found that differences between education-matched healthy elderly African Americans and Caucasians on tests of word list learning and memory, figure memory, abstract reasoning, word generation, and visuospatial skills were attenuated by controlling for word recognition scores on the Wide Range Achievement Test, Version 3 (WRAT-3). In another study with older non-demented African Americans, WRAT-3 scores strongly predicted neuropsychological performance above and beyond age, sex, years of education, and acculturation (Manly, Byrd, Touradji, & Stern, 2004). Likewise, when elderly Caribbean- and U.S.-born African Americans were compared, WRAT-3 scores were more accurate predictors of neuropsychological test performance than age and years of education (Byrd, Sanchez, & Manly, 2005). In addition, group differences in performance between elderly African Americans and Caucasians on two cancellation tasks were eliminated after equating participants on WRAT-3 scores; the same could not be achieved by matching participants on years of education (Byrd, Touradji, Tang, & Manly, 2004).

These findings in healthy older adults are encouraging but validation in patients with acquired brain dysfunction is necessary before education quality-based normative adjustments can be adopted into clinical practice with ethnically diverse examinees. To date, HIV is the only clinical condition that has been studied. HIV-positive African Americans and Hispanics were more likely than Caucasians to have discrepant education level and quality (indexed by WRAT-3 scores). This discrepancy, but not racial status, helped explain lower neuropsychological performance in ethnically diverse participants (Ryan et al., 2005). Education quality (i.e., WRAT-3 scores) also outperformed education level with regard to the diagnostic accuracy of HIV-associated cognitive impairment in African Americans (Rohit et al., 2007). The explanatory power of education quality in other neurological conditions is not yet known, particularly those with more prominent cognitive sequelae.

The purpose of this study was to examine whether education quality, as indexed by word recognition performance, can account for differences in neuropsychological outcomes from traumatic brain injury (TBI) in adult African Americans and Caucasians. TBI is a leading cause of disability in young- to middle-adulthood (Thornhill et al., 2000; Thurman, Alverson, Dunn, Guerrero, & Sniezek, 1999) and neuropsychological assessment is a key component of TBI clinical management. Minority racial status and lower education level have both been associated with worse outcomes from TBI (Arango-Lasprilla et al., 2008; Gary, Arango-Lasprilla, & Stevens, 2009; Kesler, Adams, Blasey, & Bigler, 2003). The risk of misattributing low neuropsychological test scores to TBI instead of premorbid factors is higher in African Americans due to (on average) poorer education quality. Such overpathologizing can lead to inappropriate clinical decisions (e.g., delayed return to work). Gaining a better understanding of cultural differences and their mediators will help guide clinicians to interpret neuropsychological tests more accurately in ethnically diverse examinees with TBI. Given that word recognition tests are resilient to the effects of TBI and other acquired neurological conditions (Bright, Jaldow, & Kopelman, 2002; Franzen, Burgess, & Smith-Seemiller, 1997; Green et al., 2008; Orme, Johnstone, Hanks, & Novack, 2004; Vanderploeg & Schinka, 2004), they should validly measure the influence of education history even in the presence of severe cognitive impairment. The present study could provide evidence for another application of word recognition tests in TBI, where they already appear useful for estimating premorbid intellectual functioning (Franzen et al., 1997; Green et al., 2008) and predicting post-acute functional abilities, community integration, and level of disability (Hanks, Rapport, Millis, & Deshpande, 1999; Hanks et al., 2008) irrespective of race.

Method

Fifty participants in the Southeastern Michigan Traumatic Brain Injury System program were included in this study. All incurred external-force head trauma resulting in (a) post-traumatic confusion duration >24 h, (b) loss of consciousness duration >30 min, (c) Glasgow Coma Scale score <13 in emergency department, and/or (d) trauma-related intracranial neuroimaging abnormalities. They were, on average, 38.1-year old (SD = 14.0) at the time of their injury. More participants were African American (72%) versus Caucasian (28%); 78% were men. Patients in the Traumatic Brain Injury Model Systems database are described in more detail elsewhere (Corrigan et al., 2007).

As part of a research program, participants underwent neuropsychological testing at 1-year post-injury. Raw scores on six measures, Trail Making Test (Reitan & Wolfson, 1985) Part A and Part B, California Verbal Learning Test, Second Edition (Delis, Kramer, Kaplan, & Ober, 2000) Total Learning and Long Delay Free Recall, phonemic fluency (FAS version) total score, and Grooved Pegboard Test (Reitan & Wolfson, 1985) dominant hand time, were converted to demographically unadjusted T-scores by (a) looking up the scaled score equivalents in Appendix C of Heaton and colleagues (2004) normative system, normalizing the score distributions and ensuring that higher scores always reflect better performance and then (b) converting the scaled score to a T-score based on the normal distribution to provide a more familiar and precise metric. (Note that the test battery in the Heaton, Miller, Taylor, & Grant, 2004 normative system includes the California Verbal Learning Test (CVLT), not the CVLT-2. However, according to the CVLT-2 technical manual, the two variables analyzed in the present study, Total Learning and Long Delay Free Recall, are equivalent across the CVLT and CVLT-2 versions.) A summary score, the overall test battery mean (OTBM), was then created by averaging these T-scores (sample mean = 38.3 ± 7.1). This approach of creating an OTBM is described elsewhere (Miller & Rohling, 2001). It has been validated in TBI (Rohling, Meyers, & Millis, 2003) and shown to perform comparably with traditional aggregate indices of neuropsychological impairment (Larrabee, Millis, & Meyers, 2008; Rohling, Williamson, Miller, & Adams, 2003). Participants also completed the Wechsler Test of Adult Reading (WTAR; Wechsler, 2001). Like the WRAT, the WTAR is a word recognition test that requires participants to read a list of words aloud. Although most of the above-reviewed studies used the WRAT, the newer WTAR has the advantages of (a) including only irregularly spelled words, for which correct pronunciation depends on past exposure and learning, possibly making it more resilient to neurological insult (Franzen et al., 1997), and (b) providing predicted (Wechsler Adult Intelligence and Memory Scales (third edition)) index scores with confidence intervals, improving linkage to commonly used neuropsychological tests (Wechsler, 2001). The WTAR has been validated in TBI (Green et al., 2008). Scaled scores from the WTAR were analyzed in the present study.

The assessment battery also included the Digit Vigilance Test (Lewis & Rennick, 1979), a measure of attention and processing speed. As well, participants completed self-report measures of psychological distress (Brief Symptom Inventory-18; Derogatis, 2001), problematic alcohol use (Short Michigan Alcoholism Screening Test; Selzer, Vinokur, & van Rooijen, 1975), and substance abuse (Drug Abuse Screening Test; Skinner, 1982).

Results

Demographic and clinical characteristics of the sample are presented in Table 1, as are racial group comparisons on these variables. Caucasian and African American participants had similar demographic profiles with respect to age, education, and gender. The two groups also did not differ on measures of TBI severity, including the Glasgow Coma Scale score at emergency department admission, loss of consciousness duration, and post-traumatic amnesia duration. Finally, Caucasians and African Americans did not differ on measures of psychological distress or problematic alcohol and drug use (Table 1). To confirm racial differences in the neuropsychological outcome after TBI in the present sample, we conducted an analysis of variance with race as the sole fixed factor and the OTBM as the dependent variable. Caucasian participants had higher OTBMs than African Americans, F(1, 48) = 6.27, p = .016, Cohen's d = 0.84. This main effect remained significant after controlling for post-traumatic amnesia duration (an indicator of TBI severity), F(1, 45) = 7.03, p = .011, Cohen's d = 0.85. As expected, the effect of post-traumatic amnesia severity was also significant in this model, F(1, 45) = 4.27, p = .045.

Table 1.

Demographic and clinical characteristics of Caucasian and African American participants

 Caucasian (N = 14) African American (N = 36) Statistical test value p-value 
Age M = 36.48
SD = 14.75 
M = 38.73
SD = 13.87 
t = 0.51 .620 
Education M = 12.21
SD = 2.23 
M = 11.22
SD = 1.97 
t = 1.54 .130 
Gender 71% male 81% male χ2 = 0.49 .484 
Time since injury (days) M = 363.64
SD = 32.78 
M = 365.31
SD = 33.77 
t = 0.16 .875 
Glasgow Coma Scale 3–8 = 43%
9–12 = 36%
13–15 = 21% 
3–8 = 57%
9–12 = 23%
13–15 = 20% 
χ2 = 1.02 .601 
Loss of consciousness (days) M = 6.58
SD = 10.19 
M = 6.31
SD = 12.17 
t = 0.07 .945 
Post-traumatic amnesia (days) M = 60.38
SD = 67.07 
M = 43.71
SD = 51.87 
t = 0.93 .359 
Brief Symptom Inventory (Global Severity Index T score) M = 59.43
SD = 12.74 
M = 54.97
SD = 10.80 
t = 1.25 .219 
Short Michigan Alcoholism Screening Test M = 3.08
SD = 4.30 
M = 2.09
SD = 2.39 
Mann–Whitney U .759 
Drug Abuse Screening Test M = 2.15
SD = 3.11 
M = 1.08
SD = 1.59 
Mann–Whitney U .452 
 Caucasian (N = 14) African American (N = 36) Statistical test value p-value 
Age M = 36.48
SD = 14.75 
M = 38.73
SD = 13.87 
t = 0.51 .620 
Education M = 12.21
SD = 2.23 
M = 11.22
SD = 1.97 
t = 1.54 .130 
Gender 71% male 81% male χ2 = 0.49 .484 
Time since injury (days) M = 363.64
SD = 32.78 
M = 365.31
SD = 33.77 
t = 0.16 .875 
Glasgow Coma Scale 3–8 = 43%
9–12 = 36%
13–15 = 21% 
3–8 = 57%
9–12 = 23%
13–15 = 20% 
χ2 = 1.02 .601 
Loss of consciousness (days) M = 6.58
SD = 10.19 
M = 6.31
SD = 12.17 
t = 0.07 .945 
Post-traumatic amnesia (days) M = 60.38
SD = 67.07 
M = 43.71
SD = 51.87 
t = 0.93 .359 
Brief Symptom Inventory (Global Severity Index T score) M = 59.43
SD = 12.74 
M = 54.97
SD = 10.80 
t = 1.25 .219 
Short Michigan Alcoholism Screening Test M = 3.08
SD = 4.30 
M = 2.09
SD = 2.39 
Mann–Whitney U .759 
Drug Abuse Screening Test M = 2.15
SD = 3.11 
M = 1.08
SD = 1.59 
Mann–Whitney U .452 

Race (1 = Caucasian, 2 = African American), years of education attainment, and WTAR (scaled scores) were then entered into a linear regression model, with OTBM as the dependent variable. The overall model was significant, F(3,43) = 10.26, p < .001, R2 = .41. The Bayesian Information Criterion (BIC), an index of model fit that facilitates between-model comparison, was 319.58. Wald's tests of the beta coefficients demonstrated that only the WTAR contributed unique variance (Table 2). Tolerance values were all >.50 (i.e., acceptable), but an inspection of conditional indices revealed two elevations (>15, the third and fourth dimensions), suggesting possible multicolinearity. The pattern of variance proportions was then inspected; WTAR and education loaded strongly on the third dimension, whereas WTAR and race loaded on the fourth dimension. Zero-order (non-parametric) correlations confirmed that WTAR was significantly correlated with race, rs(49) = −.44, p = .002, and education, rs(49) = .47, p = .001, whereas race and education were less correlated with each other, rs(50) = −.26, p = .070. This evidence of (mild) multicolinearity suggested that WTAR may be “going after” much of the same variance in the OTBM as race and education. As well, variance in the OTBM in this full (three-predictor) model might have been arbitrarily assigned to WTAR, producing a significant beta weight and misleadingly indicating that it is the most important predictor. WTAR and race/education models were therefore examined separately.

Table 2.

Regression models predicting OTBM

Model Predictors Unstandardized coefficients Standard error Beta coefficients t p-value 
Race −1.11 1.97 −0.07 −0.56 .577 
  Education 0.38 0.45 0.11 0.85 .399 
 WTAR 0.24 0.07 0.54 3.75 .001 
WTAR 0.28 0.05 0.63 5.51 <.001 
Race −3.89 2.07 −0.25 −1.88 .066 
 Education 1.17 0.45 0.34 2.57 .013 
Model Predictors Unstandardized coefficients Standard error Beta coefficients t p-value 
Race −1.11 1.97 −0.07 −0.56 .577 
  Education 0.38 0.45 0.11 0.85 .399 
 WTAR 0.24 0.07 0.54 3.75 .001 
WTAR 0.28 0.05 0.63 5.51 <.001 
Race −3.89 2.07 −0.25 −1.88 .066 
 Education 1.17 0.45 0.34 2.57 .013 

Note: WTAR = Wechsler Test of Adult Reading.

Removing race and education from the full model resulted in non-significant degradation with respect to variance explained, R2 change = −.01, F(2, 45) = 0.53, p = .594. Put differently, race and education added essentially no value over and above WTAR in predicting the neuropsychological outcome. The WTAR-only was significant, F(1, 47) = 30.35, p < .001, R2 = .39, BIC = 312.93. The magnitude of difference in BIC values between the full and WTAR-only models provided “positive” evidence (by conventional interpretation; Hardin & Hilbe, 2007) that the latter was a better fit to the data. That is, removing race and education resulted in a highly favorable parsimony-for-predictive power tradeoff.

Alternatively, removing WTAR from the full model resulted in a major loss of OTBM variance explained, R2 change = −.18, F(1, 45) = 14.03, p = .001. This race and education model was still significant, F(2, 46) = 6.53, p = .003, R2 = .22. Its BIC value (336.91) was significantly higher than both the WTAR-only and full models, providing strong evidence that it was the most ill-fitted of the three candidate models.

It is possible that the WTAR explained the same variance in the neuropsychological outcome as race and education plus additional variance, not because it better measures the same underlying construct (education quality), but merely because of shared methods (i.e., the WTAR and OTBM are both performance-based ability tests). To evaluate this alternative explanation, we examined another neuropsychological variable, time to complete the Digit Vigilance Test, which is not part of the OTBM. The Digit Vigilance Test was more modestly correlated with race, rs(49) = −.21, p = .153, and education, rs(49) = .30, p = .045. It also had minimal multicolinearity with these two variables in a regression model—only race (0.41) and education (0.51) loaded on the sole elevated conditional index (18.96)—while uniquely predicting OTBM (Wald's test = 3.42, p = .001). Moreover, whereas race and education explained no unique variance in the OTBM over and above WTAR scores (as described above), these two variables did explain 11.5% variance over and above Digit Vigilance Test scores, F(2, 42) = 4.11, p = .024.

Discussion

Teasing apart premorbid factors from acquired brain dysfunction is a critical step in neuropsychological test interpretation for patients with TBI. Adjusting normative comparisons for race and/or education level is currently standard practice to avoid misattributing low scores to acquired brain dysfunction in ethnically diverse examinees. However, these demographic variables lack sensitivity to within-race differences and disparities in educational experience. Manly (2005, 2006) and Manly and Echemendia (2007) argue that race should be deconstructed into underlying variables that explain racial differences (e.g., Caucasian > African American) on neuropsychological testing, such as education quality. We found that racial differences in the post-acute neuropsychological outcome from moderate-to-severe TBI were attributable to word recognition performance, a proxy indicator of education quality. Race and education level were essentially unrelated to the TBI outcome, after the influence of word recognition was accounted for. We conclude that education quality may largely explain so-called “racial” differences in the TBI outcome. The findings from this study are consistent with the previous literature demonstrating that correcting for word recognition performance strongly attenuates Caucasian–African American differences in neuropsychological test scores in non-TBI samples, including healthy older adults and patients with HIV (Byrd et al., 2004; Manly et al., 2002; Ryan et al., 2005).

As racial differences in neuropsychological performance became apparent, race-stratified normative data emerged as a practical interim solution to counter overpathologizing in African Americans. Our study adds to the mounting literature that education quality is the more relevant construct. Specifically, we demonstrated that this is also true for patients with TBI, using the WTAR scaled score as a measure of word recognition.

Although education quality is difficult to measure directly, especially retrospectively, word recognition tests may be the best available proxy. Considering acculturation (Kennepohl, Shore, Nabors, & Hanks, 2004; Manly et al., 2004; Manly, Miller, et al., 1998) and cultural matches in examinee–examiner dyads (Kennepohl et al., 2004) may further refine neuropsychological assessment of ethnically diverse examinees. Ideally, these variables would have been measured in the present study. Further limiting our findings, we did not screen for symptom invalidity. Although participants were seen in a research setting, it is possible that a non-trivial proportion performed below their capacity, which could have confounded our findings. The effect of education quality may differ across neuropsychological domains (Schneider & Lichtenberg, 2011), but our modest sample size restricted us to an omnibus test of global neuropsychological functioning. Another limitation of our study, and of prior studies using similar methodology, is that word recognition tests may correlate with neuropsychological performance more than education at least in part because of shared method variance (i.e., both involve standardized test-taking). Our supplemental analyses with a neuropsychological predictor that was not part of the OTBM reduced the credibility of this alternative explanation. Race and education lost their predictor power when considered alongside the WTAR, but not when considered alongside an attention task, demonstrating specificity of our main finding.

Given that the results from this study demonstrated that the WTAR accounted for all the same variance in the OTBM as race and education (together), as well as considerable additional variance not explained by these demographic variables, clinicians assembling a neuropsychological test battery for an examinee with TBI may want to include a word recognition test such as the WTAR. This also holds for cognitive research batteries studying outcome after TBI, as word recognition may prove to be a unique factor with regard to functional abilities and community integration after TBI, and may be a more important predictor or covariate than the level of education. Normative data stratified by word recognition performance (transformed to a premorbid intelligence quotient) are becoming increasingly available (e.g., Brooks, Holdnack, & Iverson, 2011), as they have been found to improve diagnostic accuracy even in examinees from the cultural majority (Testa, Winicki, Pearlson, Gordon, & Schretlen, 2009).

The construct(s) measured by word recognition tests such as the WTAR should be clarified in future research. Since decades before their application to understanding racial diversity, such tests have been used to estimate premorbid cognitive ability in neuropsychological assessment (Franzen et al., 1997; Schretlen, Buffington, Meyer, & Pearlson, 2005). Word recognition performance may reflect a combination of innate cognitive ability, cumulative life experience (perhaps not restricted to that gained from formal education), and cultural differences beyond education quality. Moreover, literacy is often used synonymously with education quality in the racial differences literature but may actually be one of several education corollaries.

Funding

This work was supported by the National Institute on Disability and Rehabilitation Research (H133A080044) and Vancouver Coastal Health Research Institute.

Conflict of Interest

None declared.

References

Arango-Lasprilla
J. C.
Ketchum
J. M.
Williams
K.
Kreutzer
J. S.
Marquez de la Plata
C. D.
O'Neil-Pirozzi
T. M.
, et al.  . 
Racial differences in employment outcomes after traumatic brain injury
Archives of Physical Medicine and Rehabilitation
 , 
2008
, vol. 
89
 (pg. 
988
-
995
doi:10.1016/j.apmr.2008.02.012.
Bright
P.
Jaldow
E.
Kopelman
M. D.
The National Adult Reading Test as a measure of premorbid intelligence: A comparison with estimates derived from demographic variables
Journal of the International Neuropsychological Society
 , 
2002
, vol. 
8
 (pg. 
847
-
854
doi:10.1017.S1355617702860131.
Brooks
B. L.
Holdnack
J. A.
Iverson
G. L.
Advanced clinical interpretation of the WAIS-IV and WMS-IV: Prevalence of low scores varies by level of intelligence and years of education
Assessment
 , 
2011
, vol. 
18
 
2
(pg. 
156
-
167
doi:10.1177/1073191110385316.
Byrd
D. A.
Sanchez
D.
Manly
J. J.
Neuropsychological test performance among Caribbean-born and U.S.-born African American elderly: The role of age, education, and reading level
Journal of Clinical and Experimental Neuropsychology
 , 
2005
, vol. 
27
 (pg. 
1056
-
1069
doi:10.1080/13803390490919353.
Byrd
D. A.
Touradji
P.
Tang
M. X.
Manly
J. J.
Cancellation test performance in African American, Hispanic, and White elderly
Journal of the International Neuropsychological Society
 , 
2004
, vol. 
10
 
3
(pg. 
401
-
11
)
Corrigan
J. D.
Selassie
A. W.
Lineberry
L. A.
Millis
S. R.
Wood
K. D.
Pickelsimer
E. E.
, et al.  . 
Comparison of the Traumatic Brain Injury (TBI) Model Systems national dataset to a population-based cohort of TBI hospitalizations
Archives of Physical Medicine and Rehabilitation
 , 
2007
, vol. 
88
 
4
(pg. 
418
-
26
)
Delis
D.
Kramer
J.
Kaplan
E.
Ober
B.
California Verbal Learning Test – Second Edition Manual
 , 
2000
San Antonio, TX
The Psychological Corporation
Derogatis
L.
Brief Symptom Inventory 18: Administration, scoring, and procedures manual
 , 
2001
Minneapolis, MN
NCS Assessments
Franzen
M. D.
Burgess
E. J.
Smith-Seemiller
L.
Methods of estimating premorbid functioning
Archives of Clinical Neuropsychology
 , 
1997
, vol. 
12
 (pg. 
711
-
738
)
Gary
K. W.
Arango-Lasprilla
J. C.
Stevens
L. F.
Do racial/ethnic differences exist in post-injury outcomes after TBI? A comprehensive review of the literature
Brain Injury
 , 
2009
, vol. 
23
 
10
(pg. 
775
-
789
doi:10.1080/02699050903200563.
Green
R. E. A.
Melo
B.
Christensen
B.
Ngo
L. A.
Monette
G.
Bradbury
C.
Measuring premorbid IQ in traumatic brain injury: An examination of the validity of the Wechsler Test of Adult Reading (WTAR)
Journal of Clinical and Experimental Neuropsychology
 , 
2008
, vol. 
30
 
2
(pg. 
163
-
172
doi:10.1080/13803390701300524
Hanks
R. A.
Millis
S. R.
Ricker
J. H.
Giacino
J. T.
Nakese-Richardson
R.
Frol
A. B.
, et al.  . 
The predictive validity of a brief inpatient neuropsychologic battery for persons with traumatic brain injury
Archives of Physical Medicine and Rehabilitation
 , 
2008
, vol. 
89
 (pg. 
950
-
957
doi:10.1016/j.apmr.2008.01.011
Hanks
R. A.
Rapport
L. J.
Millis
S. R.
Deshpande
S. A.
Measures of executive functioning as predictors of functional ability and social integration in a rehabilitation sample
Archives of Physical Medicine and Rehabilitation
 , 
1999
, vol. 
80
 (pg. 
1030
-
1037
)
Hardin
J. W.
Hilbe
J.
Generalized Linear Model and extensions
 , 
2007
2nd ed.
College Station, TX
Strata Press
Heaton
R. K.
Miller
S.
Taylor
M. J.
Grant
I.
Revised Comprehensive Norms for an Expanded Halstead-Reitan Battery: Demographically adjusted neuropsychological norms for African American and Caucasian adults
 , 
2004
Lutz, FL
Psychological Assessment Resources
Kaplan
J. B.
Bennett
T.
Use of race and ethnicity in biomedical publication
Journal of the American Medical Association
 , 
2003
, vol. 
289
 (pg. 
2709
-
2716
)
Kennepohl
S.
Shore
D.
Nabors
N.
Hanks
R.
African American acculturation and neuropsychological test performance following traumatic brain injury
Journal of the International Neuropsychological Society
 , 
2004
, vol. 
10
 (pg. 
566
-
577
doi:10.10170S1355617704104128.
Kesler
S. R.
Adams
H. F.
Blasey
C. M.
Bigler
E. D.
Premorbid intellectual functioning, education, and brain size in traumatic brain injury: An investigation of the cognitive reserve hypothesis
Applied Neuropsychology: Adult
 , 
2003
, vol. 
10
 (pg. 
153
-
162
doi:10.1207/S15324826AN1003_04.
Larrabee
G. J.
Millis
S. R.
Meyers
J. E.
Sensitivity to brain dysfunction of the Halstead-Reitan vs an ability-focused neuropsychological battery
The Clinical Neuropsychologist
 , 
2008
, vol. 
22
 (pg. 
813
-
825
doi:10.1080/13854040701625846
Lewis
R.F.
Rennick
P.M.
Manual for the Repeatable Cognitive Perceptual Motor Battery
1979
 
Axon Publishing Company. Grosse Pointe Parke, MI
Manly
J. J.
Advantages and disadvantages of separate norms for African Americans
The Clinical Neuropsychologist
 , 
2005
, vol. 
19
 (pg. 
270
-
275
doi:10.1080/13854040590945346
Manly
J. J.
Deconstructing race and ethnicity: Implications for measurements of health outcomes
Medical Care
 , 
2006
, vol. 
44
 
11
(pg. 
S10
-
S16
)
Manly
J. J.
Echemendia
R. J.
Race-specific norms: Using the model of hypertension to understand issues of race, culture, and education in neuropsychology
Archives of Clinical Neuropsychology
 , 
2007
, vol. 
22
 (pg. 
319
-
325
doi:10.1016/j.acn.2007.01.006.
Manly
J. J.
Byrd
D. A.
Touradji
P.
Stern
Y.
Acculturation, reading level, and neuropsychological test performance among African American elders
Applied Neuropsychology
 , 
2004
, vol. 
11
 (pg. 
37
-
46
)
Manly
J. J.
Jacobs
D. M.
Sano
M.
Bell
K.
Merchant
C. A.
Small
S. A.
, et al.  . 
Cognitive test performance among nondemented elderly African Americans and whites
Neurology
 , 
1998
, vol. 
50
 (pg. 
1238
-
1245
)
Manly
J. J.
Jacobs
D. M.
Touradji
P.
Small
S. A.
Stern
Y.
Reading level attenuates differences in neuropsychological test performance between African American and White elders
Journal of the International Neuropsychological Society
 , 
2002
, vol. 
8
 (pg. 
341
-
348
)
Manly
J. J.
Miller
S. W.
Heaton
R. K.
Byrd
D.
Reilly
J.
Velasquez
R. J.
, et al.  . 
The effect of African-American acculturation on neuropsychological test performance in normal and HIV positive individuals
Journal of the International Neuropsychological Society
 , 
1998
, vol. 
4
 (pg. 
291
-
302
)
Miller
L. S.
Rohling
M. L.
A statistical interpretive method for neuropsychological test data
Neuropsychology Review
 , 
2001
, vol. 
11
 (pg. 
143
-
69
)
Nabors
N. A.
Evans
J. D.
Strickland
T. L.
Fletcher-Janzen
E.
Strickland
T. L.
Reynolds
C. R.
Neuropsychological assessment and intervention with African Americans
Handbook of Cross-Cultural Neuropsychology
 , 
2000
New York
Kluwer Academic/Plenum
(pg. 
31
-
42
)
Norman
M. A.
Evans
J. D.
Miller
W. S.
Heaton
R. K.
Demographically corrected norms for the California Verbal Learning Test
Journal of Clinical and Experimental Neuropsychology
 , 
2000
, vol. 
22
 (pg. 
80
-
94
doi:10.1076/1380-3395(200002)22:1;1–8;FT080.
Orme
D. R.
Johnstone
B.
Hanks
R.
Novack
T.
The WRAT-3 reading subtest as a measure of premorbid intelligence among persons with brain injury
Rehabilitation Psychology
 , 
2004
, vol. 
49
 
3
(pg. 
250
-
253
doi:10.1037/0090-5550.49.3.250.
Reitan
R. M.
Wolfson
D.
The Halstead-Reitan Neuropsychological Test Battery: Theory and clinical interpretation
 , 
1985
Tucson, AZ
Neuropsychology Press
Rohit
M.
Levine
A.
Hinkin
C.
Abramyan
S.
Saxton
E.
Valdes-Sueiras
M.
, et al.  . 
Education correction using years in school or reading grade-level equivalent? Comparing the accuracy of two methods in diagnosing HIV-associated neurocognitive impairment
Journal of the International Neuropsychological Society
 , 
2007
, vol. 
13
 (pg. 
462
-
470
doi:10.10170S1355617707070506.
Rohling
M. L.
Meyers
J. E.
Millis
S. R.
Neuropsychological impairment following traumatic brain injury: A dose-response analysis
The Clinical Neuropsychologist
 , 
2003
, vol. 
17
 (pg. 
289
-
302
)
Rohling
M. L.
Williamson
D. J.
Miller
L. S.
Adams
R. L.
Using the Halstead-Reitan Battery to diagnose brain damage: A comparison of the predictive power of traditional techniques to Rohling's Interpretive Method
The Clinical Neuropsychologist
 , 
2003
, vol. 
17
 (pg. 
531
-
543
)
Ryan
E. L.
Baird
R.
Rivera Mindt
M.
Byrd
D.
Monzones
J.
Morgello
S.
, et al.  . 
Neuropsychological impairment in racial/ethnic minorities with HIV infection and low literacy levels: Effects of education and reading level in participant characterization
Journal of the International Neuropsychological Society
 , 
2005
, vol. 
11
 
7
(pg. 
889
-
898
)
Schneider
B. C.
Lichtenberg
P. A.
Influence of reading ability on neuropsychological performance in African American elders
Archives of Clinical Neuropsychology
 , 
2011
, vol. 
26
 (pg. 
624
-
631
doi:10.1093/arclin/acr062.
Schretlen
D. J.
Buffington
A. L. H.
Meyer
S. M.
Pearlson
G. D.
The use of word-reading to estimate ‘premorbid’ ability in cognitive domains other than intelligence
Journal of the International Neuropsychological Society
 , 
2005
, vol. 
11
 
6
(pg. 
784
-
787
doi:10.1017/S1355617705050939.
Schwartz
B. S.
Glass
T. A.
Bolla
K. I.
Stewart
W. F.
Glass
G.
Rasmussen
M.
, et al.  . 
Disparities in cognitive functioning by race/ethnicity in the Baltimore Memory Study
Environmental Health Perspectives
 , 
2004
, vol. 
112
 (pg. 
314
-
320
doi:10.1289/ehp.6727.
Selzer
M. L.
Vinokur
A.
van Rooijen
M. A.
A self-administered Short Michigan Alcoholism Screening Test (SMAST)
Journal of Studies on Alcohol
 , 
1975
, vol. 
36
 (pg. 
117
-
126
)
Skinner
H. A.
The drug abuse screening test
Addictive Behaviors
 , 
1982
, vol. 
7
 (pg. 
363
-
371
)
Testa
S. M.
Winicki
J. M.
Pearlson
G. D.
Gordon
B.
Schretlen
D. J.
Accounting for estimated IQ in neuropsychological test performance with regression-based techniques
Journal of the International Neuropsychological Society
 , 
2009
, vol. 
15
 (pg. 
1012
-
1022
doi:10.1017/S1355617709990713.
Thornhill
S.
Teasdale
G. M.
Murray
G. D.
McEwen
J.
Roy
C. W.
Penny
K. I.
Disability in young people and adults one year after head injury: Prospective cohort study
British Medical Journal
 , 
2000
, vol. 
320
 (pg. 
1631
-
1635
)
Thurman
D. J.
Alverson
C.
Dunn
K. A.
Guerrero
J.
Sniezek
J. E.
Traumatic brain injury in the United States: A public health perspective
Journal of Head Trauma Rehabilitation
 , 
1999
, vol. 
14
 
6
(pg. 
602
-
615
)
Vanderploeg
R. D.
Schinka
J. A.
Ricker
J. H.
Estimation of premorbid cognitive abilities: Issues and approaches
Differential diagnosis in adult neuropsychological assessment
 , 
2004
New York
Springer
(pg. 
27
-
65
)
Wechsler
D.
Wechsler Test of Adult Reading
 , 
2001
San Antonio, TX
The Psychological Corporation
Welsh
K. A.
Fillenbaum
G.
Wilkinson
W.
Heyman
A.
Mohs
R. C.
Stern
Y.
, et al.  . 
Neuropsychological test performance in African-American and white patients with Alzheimer's disease
Neurology
 , 
1995
, vol. 
45
 (pg. 
2207
-
2211
)