Abstract

This study documented baseline neurocognitive performance of 23,815 athletes on the Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) test. Specifically, 9,733 Hispanic, Spanish-speaking athletes who completed the ImPACT test in English and 2,087 Hispanic, Spanish-speaking athletes who completed the test in Spanish were compared with 11,955 English-speaking athletes who completed the test in English. Athletes were assigned to age groups (13–15, 16–18). Results revealed a significant effect of language group (p < .001; partial η2 = 0.06) and age (p < .001; partial η2 = 0.01) on test performance. Younger athletes performed more poorly than older athletes, and Spanish-speaking athletes completing the test in Spanish scored more poorly than Spanish-speaking and English-speaking athletes completing the test in English, on all Composite scores and Total Symptom scores. Spanish-speaking athletes completing the test in English also performed more poorly than English-speaking athletes completing the test in English on three Composite scores. These differences in performance and reported symptoms highlight the need for caution in interpreting ImPACT test data for Hispanic Americans.

Introduction

The value of neuropsychological assessment in athletics was first recognized by Barth and colleagues (1989) who examined the performance of preseason and post-concussion neurocognitive functioning of college football players utilizing standardized, neuropsychological paper–pencil tests. Over the past decade, the practice of baseline and post-injury neuropsychological assessment has been adopted by many as a standard of care in concussion management for professional, collegiate, and high-school sports (Aubry et al., 2002; Echemendia & Cantu, 2003; Guskiewicz et al., 2004; McCrory et al., 2005, 2009, 2013; Moser et al., 2007; Van Kampen et al., 2006). Moreover, neurocognitive testing has been described as a “cornerstone” of sports concussion assessment (Aubry et al., 2002); athletic programs at the high school, club league, collegiate, and professional levels have implemented baseline and post-concussion testing as a component of comprehensive concussion management programs.

To date, all but one state in the USA have passed legislation regarding the immediate removal from play of athletes suspected of sustaining a concussion along with mandated management guidelines. In this regard, many high schools and youth sport groups across the country have also incorporated neurocognitive assessment protocols for use with concussed athletes. The utility of neuropsychological assessment in assisting the clinician in making more accurate and safe return to play decisions without relying solely on athlete self-report has been established throughout the sport concussion literature (Broglio, Macciocchi, & Ferrara, 2007; Grindel, Lovell, & Collins, 2001; Fazio, Lovell, Pardini, & Collins, 2007; Lau, Collins, & Lovell, 2011; Schatz, Pardini, Lovell, Collis, & Podell, 2005; Van Kampen et al., 2006). As an alternative to paper–pencil neuropsychological tests, several computer-based neurocognitive assessment batteries have been developed. The transition to computerized neurocognitive test platforms was necessary due to the time- and labor-intensive work demand of paper–pencil tests, the standardization of test instructions and scoring of results by computerized administration, the precision of response times afforded by computers, and the absence of sufficient numbers of clinical neuropsychologists to administer and score the paper–pencil tests. Many computerized test programs also encompass alternate test versions (with computerized randomization of test stimuli) in order to minimize practice effects across frequent administrations (Johnson, Kegel, & Collins, 2011; Putukian, 2011; Schatz & Zilmer, 2003). There are several computerized neurocognitive tests that have been utilized in the assessment of athletes over the past decade, including the Automated Neuropsychological Assessment Matrices (ANAM; Vista Life Sciences, Washington DC), CogSport (CogState Ltd, Australia), and HeadMinder (HeadMinder, Inc., New York).

The Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT; ImPACT Applications Inc., Pittsburgh, PA) has received the majority of attention in North America as a clinical and research tool in the field of sports concussion. The use of neurocognitive testing in general (Randolph & Kirkwood, 2009), as well as the diagnostic utility and sensitivity of ImPACT in the evaluation of sports-related concussion, has been challenged and debated (Mayers & Redick, 2012; Randolph, Lovell, & Laker, 2011; Randolph, McCrea, & Barr, 2005; Resch et al., 2013; Schatz, Kontos, & Elbin, 2012). However, several researchers have demonstrated the ImPACT test to be a reliable instrument for assessing neurocognitive abilities following sports-related concussion (Elbin, Schatz, & Covassin, 2011; Iverson, Lovell, & Collins, 2003; Schatz, 2009; Schatz & Ferris, 2013). Further, researchers have documented the validity of ImPACT (Allen & Gfeller, 2011; Iverson, Gaetz, Lovell, & Collins, 2005; Iverson, Lovell, & Collins, 2005; Maerlender et al., 2010, 2013), and neurocognitive test data have been shown to have “added value” beyond symptom reporting (Van Kampen, Lovell, Pardini, Collins, & Fu, 2006). The sensitivity of the ImPACT test to acute concussion (e.g., 2–3 days post-injury) has been documented in the 80%–90% range (Broglio et al., 2007; Schatz, Pardini, Lovell, Collins, & Podell, 2006; Schatz & Sandel, 2012) when compared with 23% (McCrea et al., 2005) and 44% (Broglio et al., 2007) for pencil-and-paper test measures. Although there remains no “gold standard” for the diagnosis of concussion, neurocognitive tests such as ImPACT were not intended to represent or serve as a substitute for a comprehensive neuropsychological evaluation (Lovell, 2006) and are recommended to be used as one measure in a battery of tests, which include symptom reporting, clinical interview, and cognitive screening (McCrory et al., 2013).

According to the ImPACT developers (Lovell, 2012), a 6th grade reading level is required for examinees to effectively respond in a valid fashion to the test items. Baseline ImPACT normative data have been stratified on the basis of age and gender (Lovell, 2012) and the test is available in 21 different languages. Although the athletic populations utilizing ImPACT are culturally and linguistically diverse, to date, the psychometric research on ImPACT has not included normative data for various ethno-cultural groups or stratification by sport. Within the field of neuropsychological assessment, age- and gender-related differences in many abilities have been documented (Covassin et al., 2006; Covassin, Elbin, Kontos, & Larson, 2010; Keith, Rynolds, Patel, & Ridley, 2008; Maitland, Intrieri, Schaie, & Willis, 2000), as have cultural/ethnic differences on traditional paper–pencil neuropsychological tests (Bure-Reyes et al., 2013; Norman et al., 2011). The importance of linguistic and cultural influences on cognitive test performance has been recognized for decades by test developers. For example, the Wechsler scales of intelligence and memory have been translated into 18 languages, and these scales have been culturally adapted and normed for use in numerous European, Asian, and Middle-Eastern nations (Harris, Tulsky, & Schultheis, 2003). As well, separate norms have been developed for African Americans on the Extended Halstead-Reitan Neuropsychological Test Battery (Heaton, Miller, Taylor, & Grant, 2004).

Racial and cultural differences in athlete performance on computerized baseline and post-injury tests have only recently been investigated. Shuttleworth-Edwards, Whitefield-Alexander, Radloff, Taylor, and Lovell (2009) examined cross-cultural neurocognitive differences on baseline computerized assessment among South African rugby players versus US football players utilizing the ImPACT test in three age groups (11–13, 14–16, and 17–21 years), with administrations conducted in English for both groups. They found comparable results between the two groups on composite scores of Verbal Memory, Visual Memory, Visual Motor Speed, and Reaction Time indices, concluding that US ImPACT normative data are appropriate for English-speaking South African athletes. However, despite similarities in cognitive performance the South African rugby players endorsed significantly higher symptom scores than US football players. Shuttleworth-Edwards and colleagues (2009) acknowledged that this difference could reflect cultural differences in symptom report and impressionability, factors that should be considered in post-concussion assessment and management. Importantly, the authors cautioned that while US norms may be suitable for use with English-first-language South African rugby players from “relatively advantaged” backgrounds, they may not be suitable for non-English-first-language individuals (e.g., non-white individuals) with “less advantaged” backgrounds. In this regard, Kontos, Elbin, Covassin, and Larson (2010) compared pre- and post-concussion performance on ImPACT among African American and White high school and collegiate athletes. Each group was composed of 39 men and 9 women. Participants were matched on variables including concussion history, age, and gender and sport affiliations such as football (n = 55), men's soccer (n = 8), women's soccer (n = 7), men's wrestling (n = 11), and women's gymnastics (n = 6). Age was comparable for the two groups ranging from 14 to 23 years. Although there were no significant differences between African American and White athletes on baseline symptoms, Verbal Memory, Visual Memory, and Reaction Time composite scores, a difference did emerge at 7 days post-injury in which the African American group demonstrated a greater likelihood (2.4 times) toward at least one significantly deficient composite score. The African American athletes were also found to exhibit a significant decline in Visual Motor Speed performance when compared with baseline. Given these ethnic differences in the post-concussion performance, it would be premature to view the ImPACT test as being entirely “culturally fair” for use with African American and White high school and collegiate athletes.

A series of investigations utilizing traditional “pencil and paper” neuropsychological tests (e.g., Boston Naming Test, Trail Making Test, and Wechsler Scale subtests of Information and Vocabulary) addressed the effects of cultural and ethnic variables on the cognitive performance of African American adults (Kennepohl, Shore, Nabors, & Hanks, 2004; Manly et al., 1998; Manly, Byrd, Touradji, & Stern, 2004; Manly & Echemendia, 2007). These studies demonstrate the potential importance of variables such as degree of acculturation (i.e., degree to which one participates in the language, values, and practices of the dominant culture), language usage (Black English vs. Standard English), and quality of education on level of test performance. Until the effects of these variables are examined using ImPACT, the conclusions of Kontos and colleagues (2010) concerning the degree to which this computerized measure is a culturally fair assessment tool should be considered preliminary.

According to US Census Data (2010), the Hispanic population is the largest ethnic minority group, making up 16.3% of the nation's total population. By 2050, the Census Bureau estimates that 30% of the country's population will be of Hispanic origin. With this growth, increases in the number of Hispanics that participate in sports would also be anticipated. The NCAA's published data on ethnic minority athletic participation for Divisions I, II, and III (combined) for the 2009–2010 academic year indicate that approximately 4% of participants were of Hispanic origin (NCAA, 2010).

With consensus expert agreement that neurocognitive assessment is an integral part of proper concussion management (McCrory et al. 2009, 2013), the use of computerized neurocognitive assessment tools such as ImPACT will likely continue to be a major component of the standard of care in sport-related concussion. Thus, impact of culture and linguistic background on neuropsychological test results cannot be overlooked and if appropriate, separate normative data may be necessary for various ethnic and racial groups in order to offer valid interpretations. Unfortunately, the majority of the studies that have examined performance on neuropsychological tests of Hispanics have been conducted with elderly subjects, utilizing paper–pencil neurocognitive tests (e.g., Spanish NEUROMA Project; Alegret et al., 2012; Pena-Casanova et al., 2009; Quintana et al., 2011), decreasing their utility for comparison with a high school or college-aged sample. Investigations utilizing younger Hispanic Americans have indicated that factors such as level of acculturation, bilingualism (i.e., Spanish Dominant, Balanced, or English Dominant), quality of education, and language of test administration can significantly affect scores on a variety of neuropsychological and intellectual measures (Arnold, Montgomery, Castaneda, & Longoria, 1993; Cosentino, Manly, & Mungas, 2007; Gasquoine & Gonzales, 2012; Gasquoine, Croyle, Cavazos-Gonzales, & Sandoval, 2007; Manly & Echemendia, 2007; Razani, Murcia, Tabares, & Wong, 2007). To date, possible cross-cultural differences in baseline computerized neurocognitive performance between Hispanics and other racial groups have not been explored. Additionally, there is also a lack of information regarding the normal range of performance for Hispanics who are bilingual when computerized neurocognitive tests are administered in their native language versus a secondary language.

The purpose of the current study was to document patterns in baseline neurocognitive performance in a Spanish-speaking sample. In doing so, Spanish-speaking Hispanic athletes who elected to complete ImPACT in Spanish were compared with bilingual Spanish-speaking Hispanic athletes who elected to complete the test in English, along with comparisons to English-speaking athletes completing the test in English. It was hypothesized that Spanish-speaking examinees would perform better when assessed in their preferred, native language (e.g., Spanish) than those examinees that chose to take ImPACT in their second language (e.g. English). Based on the intelligence testing literature (Prifitera, Saklofske, & Weiss, 2008; Weiss, Saklofske, Coalson, & Raiford, 2010), it was also hypothesized that English-speaking examinees taking the test in English would outperform Hispanic examinees taking the test in either English or Spanish.

Method

Participants

A sample of 12,653 Hispanic athletes, aged 13–18, were initially selected for inclusion in this study from a larger database provided by the Lead Programmer at ImPACT, Inc., who was blind to the purpose of the study. All athletes were bilingual and reported Spanish to be their first language, with English designated as their second language, with 10,286 preferring to complete the test in English and 2,367 preferring to complete the test in Spanish.

An additional sample of 12,644 English-speaking athletes, aged 13–18, who completed the ImPACT test in English was extracted from the ImPACT normative database, provided by the Lead Programmer at ImPACT, Inc., who was blind to the purpose of the study. Invalid assessments were previously excluded from the normative database of English-speaking athletes who completed the test in English; therefore, no invalid baselines were included in this sample. Baseline assessment test reports were automatically flagged as “invalid” with the denotation “Baseline ++” using predetermined cut-offs for lack of validity that are built into the ImPACT test. Detailed descriptions of the formulae for ImPACT composite scores and cut-offs for invalid baselines are provided in Table 1. Using this mechanism for identifying lack of validity, baseline assessments from 6.6% of athletes (n = 833) were deemed invalid, with a significantly greater percentage of invalid baselines (11.8%) obtained from bilingual Spanish-speaking Hispanic athletes completing ImPACT in Spanish, when compared with Hispanic athletes completing ImPACT in English (5.4%), χ2(1) = 130.3 (p < .001).

Table 1.

Subtests and composite scores and validity indicators for ImPACT (online version)

Test name Neurocognitive domain measured 
Word Memory Word recognition memory (learning and retention) 
Design Memory Design recognition memory (learning and retention) 
X's and O's Visual working memory and cognitive speed 
Symbol Match Memory and visual-motor speed 
Color Match Impulse inhibition and visual-motor speed 
Three letter memory Verbal working memory and cognitive speed 
Symptom Scale Rating of individual self-reported symptoms 
Composite Score Contributing Scores/Formula (average of scores presented) 
Verbal Memory Word Memory (hits, correct distractors, immediate and delayed)/48 
Symbol Match (total correct hidden)/9 
Three Letters (total letters correct)/15 
Design Memory Design Memory (hits, correct distractors, immediate and delayed)/48 
X's and O's (total correct memory)/12 
Reaction Time X's and O's (average counted correct reaction time), Symbol Match (average weighted reaction time for correct responses), Color Match (average reaction time for correct response) 
Visual Motor X's and O's (total correct)/4 
Speed Three Letters (average correctly counted) × 3 
Impulse Control X's and O's (number of incorrect distracters) 
Color Match (number of commission errors) 
Test name Neurocognitive domain measured 
Word Memory Word recognition memory (learning and retention) 
Design Memory Design recognition memory (learning and retention) 
X's and O's Visual working memory and cognitive speed 
Symbol Match Memory and visual-motor speed 
Color Match Impulse inhibition and visual-motor speed 
Three letter memory Verbal working memory and cognitive speed 
Symptom Scale Rating of individual self-reported symptoms 
Composite Score Contributing Scores/Formula (average of scores presented) 
Verbal Memory Word Memory (hits, correct distractors, immediate and delayed)/48 
Symbol Match (total correct hidden)/9 
Three Letters (total letters correct)/15 
Design Memory Design Memory (hits, correct distractors, immediate and delayed)/48 
X's and O's (total correct memory)/12 
Reaction Time X's and O's (average counted correct reaction time), Symbol Match (average weighted reaction time for correct responses), Color Match (average reaction time for correct response) 
Visual Motor X's and O's (total correct)/4 
Speed Three Letters (average correctly counted) × 3 
Impulse Control X's and O's (number of incorrect distracters) 
Color Match (number of commission errors) 

Validity Indicators for ImPACT Baseline Testing (online version): Impulse Control Composite Score is >30;

Word Memory Learning Percentage is <69%; Design Memory Learning Percentage is <50%; X's and O's Total Correct Interference is >30; Three Letters Total Letters Correct is <8.

Following the removal of invalid test results, English-speaking athletes from the normative sample were identified as significantly more likely to have been diagnosed with ADD or LD (5.1%), when compared with Spanish-speaking Hispanic athletes completing the test in Spanish or English (2.5% vs. 1.3%), χ2(2) = 249 (p = .002).

After excluding those athletes with invalid baselines and a self-reported history of ADD/LD, the resulting sample was 11,820 bilingual Hispanic, Spanish-speaking athletes, 9,733 of whom elected to complete the test in English, and 2,087 who preferred to complete the test in Spanish, and a sample of 11,955 English-speaking athletes who completed the test in English.

Given that the ImPACT normative database included athletes with a history of concussion, athletes were included from both Spanish-speaking samples with a self-reported history of concussion. The breakdown of gender, concussion history, and primary sport is presented in Table 2.

Table 2.

Group demographics

 Normative (N = 11,955) Hispanic-English (N = 9,733) Hispanic-Spanish (N = 2,087) 
Gender (%)* 
 Men 71.2 66.8 77.0 
 Women 28.8 33.2 23.0% 
Sport 
 Baseball 392 (3.3%) 322 (3.3%) 167 (8.0%) 
 Basketball 901 (7.5%) 454 (4.7%) 112 (5.4%) 
 Football 3,925 (32.7%) 1,365 (14.0%) 109 (5.2%) 
 Soccer 2,045 (17.0%) 4,978 (51.1%) 1,280 (61.3%) 
 Volleyball 446 (3.7%) 446 (4.6%) 80 (3.8%) 
 Wrestling 205 (1.7%) 572 (5.9%) 65 (3.1%) 
 Track and Field 203 (1.7%) 388 (4.0%) 33 (1.6%) 
 Other 3,878 (32.3%) 1,208 (12.4%) 241 (11.5%) 
Previous Concussion** 
 None 7,985 (78.3%) 9,235 (94.9%) 2,035 (97.5%) 
 One 1,541 (15.1%) 412 (4.2%) 43 (2.1%) 
 Two or more 676 (6.6%) 86 (0.9%) 9 (0.4%) 
 Normative (N = 11,955) Hispanic-English (N = 9,733) Hispanic-Spanish (N = 2,087) 
Gender (%)* 
 Men 71.2 66.8 77.0 
 Women 28.8 33.2 23.0% 
Sport 
 Baseball 392 (3.3%) 322 (3.3%) 167 (8.0%) 
 Basketball 901 (7.5%) 454 (4.7%) 112 (5.4%) 
 Football 3,925 (32.7%) 1,365 (14.0%) 109 (5.2%) 
 Soccer 2,045 (17.0%) 4,978 (51.1%) 1,280 (61.3%) 
 Volleyball 446 (3.7%) 446 (4.6%) 80 (3.8%) 
 Wrestling 205 (1.7%) 572 (5.9%) 65 (3.1%) 
 Track and Field 203 (1.7%) 388 (4.0%) 33 (1.6%) 
 Other 3,878 (32.3%) 1,208 (12.4%) 241 (11.5%) 
Previous Concussion** 
 None 7,985 (78.3%) 9,235 (94.9%) 2,035 (97.5%) 
 One 1,541 (15.1%) 412 (4.2%) 43 (2.1%) 
 Two or more 676 (6.6%) 86 (0.9%) 9 (0.4%) 

*χ2(2) = 103.2; p = .001.

**χ2(4) = 1,489.6; p = .001.

Materials and Procedures

Immediate Post-Concussion Assessment and Cognitive Testing

First developed in the late 1990s and commercially available in 2002 as a desktop program, ImPACT has undergone multiple revisions over the past several years including the inclusion of additional test stimuli, revised test instructions, and transition to a web-based version in 2008. The ImPACT test takes approximately 25 min to complete and includes a demographic section, symptom inventory, and six subtests measuring attention, memory, processing speed, and reaction time. The demographic section requests information concerning the examinee's age, gender, medical (e.g., migraines, seizure disorder, psychiatric illness, substance abuse) and educational history (e.g., years of education, presence of learning disability, attention deficit disorder; ADD/LD) and information pertaining to sport (i.e., years played, position). The symptom inventory includes examinee self-report of 22 symptoms using a 7-point Likert-type rating scale (0–6) that when summed, provides an overall symptom score. The six subtests yield individual composite scores including Verbal Memory, Visual Memory, Visual Motor (processing) Speed, Reaction time, and Impulse Control.

Procedures

All athletes were required by their educational institutions or club sport organizations to complete an ImPACT baseline evaluation as part of their pre-participation in sport, and all participants completed the online version of the ImPACT test. Bilingual Spanish speaking athletes were allowed to take ImPACT in either Spanish or English. Institutional Review Board approval was obtained for retrospective analysis of de-identified data. The ImPACT Technical Manual (Lovell, 2012) provides guidelines for test administration, indicating that test administration should occur in a supervised fashion, either individually or in a group setting that is free from distraction. When ImPACT is administered in a group format, preferential seating to include at least one vacant seat between examinees to help reduce distraction is recommended. Test administration is described by the developers of ImPACT as being “relatively self-explanatory” and completion of the test is based on the assumption of an approximate sixth grade reading level (Lovell, 2012). Given that this study was retrospective in nature, the circumstances by which participants were administered the ImPACT baseline assessment was not systematically documented or controlled. More specifically, the quality of the testing environment and the number of examinees that were administered ImPACT in a group versus individual format was not known, nor was the level of training and credentials of test administrators, or the reading level of examinees.

Statistical Analyses

Chi-squared analyses were performed to identify likelihood of previous concussions by language group, and one-way ANOVAs were performed to identify effects of history of concussion on ImPACT test performance.

Multivariate analysis of variance (MANOVA) was performed to identify between-groups differences. The independent variables were language group (English-speakers taking ImPACT in English, Hispanic bilingual Spanish-speakers taking ImPACT in English, Hispanic bilingual Spanish-speakers taking ImPACT in Spanish) and age group (based on age groupings in the ImPACT Manual [Lovell, 2012] athletes ages 13–15 and athletes ages 16–18). The dependent variables were the ImPACT composite scores (Verbal Memory, Visual Memory, Visual Motor Speed, Reaction Time) and Total Symptom scores.

Subsequent MANOVA was performed to identify between-groups differences on the ImPACT subscales that contribute to the composite scores. The independent variables were language group and age group (as listed above) and the dependent variables were the subscales contributing to the ImPACT Composite Scores (Table 1).

Partial-eta squared (η2) were calculated as a measure of effect size, with 0.01 constituting a small effect, 0.06 a medium effect, and 0.14 a large effect (Cohen, 1988).

Results

A Chi-square test revealed that English-speaking athletes completing the test in English were significantly more likely to report a history of previous concussion (21.7%) than Hispanic athletes completing the test in English or Spanish (5.1% and 2.5%, respectively), χ2(4) = 1,489.6 (p < .001). However, ANOVA revealed no significant effect of history of concussion on ImPACT test performance, as measured by the four composite scores, although there was a significant effect of concussion history on Total Symptom score—F(2, 11,809) = 52.8, p < .001, partial η2 = 0.01 (small effect). MANOVA revealed a significant multivariate effect of language group, F(10, 47,602) = 319.1, p < .001, partial η2 = 0.06 (medium effect), and age group, F(5, 23,801) = 48.7, p < .001, partial η2 = 0.01 (small effect), on neurocognitive test performance, with a significant age by language group interaction, F(10, 47,602) = 6.0, p < .001, partial η2 = 0.001 (small effect). Subsequent univariate analyses revealed language group effects on all four ImPACT Composite scores as well as Total Symptom scores, with effect sizes ranging from small to large, and age group effects on all four ImPACT Composite scores, with small effect sizes. Younger athletes performed more poorly than older athletes on all ImPACT composite scores, but reported fewer symptoms. With respect to the language group, Spanish-speaking athletes completing the test in Spanish scored more poorly on all Composite scores and Total Symptom scores than Spanish-speaking and English-speaking athletes completing the test in English. In addition, Spanish-speaking athletes completing the test in English also performed more poorly than English-speaking athletes completing the test in English on Verbal Memory, Visual Motor Speed, and Reaction Time. Interaction effects of age and language groups were noted only with Visual Motor Speed, Reaction Time, and Total Symptom scores, with small effect sizes. Post hoc results, group means and standard deviations, p-values, and effect sizes are presented in Table 3.

Table 3.

ImPACT composite scores between groups

Variable Normative Spanish-English Spanish-Spanish p/η2p/η2** 
Verbal Memory 
 13–15 83.6 (9.5) 83.1 (9.8) 81.0 (9.7) .001/0.006 .001/0.001 
 16–18 84.7 (9.7) 83.6 (9.9) 81.5 (10.0)   
Visual Memory 
 13–15 72.7 (13.2) 72.5 (13.0) 69.8 (14.2) .001/0.006 .001/0.001 
 16–18 74.0 (13.2) 73.2 (12.9) 69.8 (14.4)   
Visual Motor Speed 
 13–15 36.6 (6.7) 32.9 (6.8) 29.4 (8.1) .001/0.12 .001/0.01 
 16–18 39.2 (6.9) 34.7(7.2) 30.8 (8.4)   
Reaction Time 
 13–15 0.61 (0.08) 0.63 (0.10) 0.65 (0.10) .001/0.026 .001/0.003 
 16–18 0.59 (0.08) 0.62 (0.12) 0.64 (0.11)   
Symptom Scale 
 13–15 5.3 (8.9) 5.9 (9.4) 7.6 (11.5) .001/0.001 .89/0.00 
 16–18 6.3 (9.9) 6.0 (9.7) 6.4 (9.3)   
Variable Normative Spanish-English Spanish-Spanish p/η2p/η2** 
Verbal Memory 
 13–15 83.6 (9.5) 83.1 (9.8) 81.0 (9.7) .001/0.006 .001/0.001 
 16–18 84.7 (9.7) 83.6 (9.9) 81.5 (10.0)   
Visual Memory 
 13–15 72.7 (13.2) 72.5 (13.0) 69.8 (14.2) .001/0.006 .001/0.001 
 16–18 74.0 (13.2) 73.2 (12.9) 69.8 (14.4)   
Visual Motor Speed 
 13–15 36.6 (6.7) 32.9 (6.8) 29.4 (8.1) .001/0.12 .001/0.01 
 16–18 39.2 (6.9) 34.7(7.2) 30.8 (8.4)   
Reaction Time 
 13–15 0.61 (0.08) 0.63 (0.10) 0.65 (0.10) .001/0.026 .001/0.003 
 16–18 0.59 (0.08) 0.62 (0.12) 0.64 (0.11)   
Symptom Scale 
 13–15 5.3 (8.9) 5.9 (9.4) 7.6 (11.5) .001/0.001 .89/0.00 
 16–18 6.3 (9.9) 6.0 (9.7) 6.4 (9.3)   

Notes: Post hoc results: Verbal Memory, Visual Motor Speed, Reaction Time: Normative > Spanish-English > Spanish-Spanish. Visual Memory, Total Symptom Score: Normative, Spanish-English < Spanish-Spanish.

*p-values and η2 (partial eta-squared) for language group comparisons; df = (2, 23,811).

**p-values and η2 (partial eta-squared) for age group comparisons; df = (1, 23,811).

Given that between-groups differences were identified on ImPACT Composite scores, subsequent analyses were conducted to identify which ImPACT subscales were contributing to these differences. MANOVA revealed a significant multivariate effect of language group, F(20, 47,592) = 183.7, p < .001, partial η2 = 0.07 (medium effect), and age group, F(10, 23,796) = 40.0, p < .001, partial η2 = 0.02 (small effect), on neurocognitive test performance. With a significant age by language group interaction—F(20, 47,592) = 2.4, p < .001, partial η2 = 0.001 (small effect). Subsequent univariate analyses revealed language group effects on all contributing ImPACT subscale scores, with effect sizes ranging from small to large and age group effects on all but three contributing ImPACT subscale scores (Word Memory, Symbol Match Total Correct Hidden, Symbol Match Average Correct Reaction Time), all with small effect sizes. Interaction effects of age and language groups were noted only on Three Letters Average Counted Correctly and Color Match Average Correct Reaction Time, with small effect sizes. Post hoc results, group means and standard deviations, p-values, and effect sizes are presented in Table 4.

Table 4.

ImPACT subscale scores between groups

Variable Normative Spanish -Eng Spanish -Spanish p/η2p/η2** 
Word Memorya 
 13–15 93.6 (6.1) 93.5 (6.1) 91.9 (6.1) .001/0.006 .94/0.00 
 16–18 93.6 (6.1) 93.6 (6.1) 91.7 (6.1)   
Symbol Match Total Correct Hidden 
 13–15 6.1 (1.9) 6.2 (2.0) 6.0 (2.0) .001/0.001 .20/0.00 
 16–18 6.2 (2.0) 6.2 (2.1) 6.0 (2.0)   
Three Letters Total Letters Correct 
 13–15 13.5 (1.8) 13.1 (1.9) 12.8 (1.9) .001/0.02 .001/0.002 
 16–18 13.8 (1.6) 13.3 (1.9) 12.9 (2.0)   
Design Memoryb 
 13–15 79.8 (12.5) 78.4 (12.8) 76.2 (12.8) .001/0.008 .09/0.00 
 16–18 80.4 (12.4) 78.8 (12.8) 76.4 (13.4)   
X's and O's Total Correct Memory 
 13–15 7.9 (2.4) 8.0 (2.3) 7.4 (2.6) .001/0.004 .001/0.001 
 16–18 8.1 (2.4) 8.1 (2.3) 7.6 (2.6)   
X's and O's Total Correct Interference 
 13–15 111.3 (8.5) 109.7 (9.9) 107.1 (11.2) .001/0.02 .001/0.01 
 16–18 114.5 (8.0) 112.3 (10.5) 109.4 (12.6)   
Three Letters Average Counted Correctly 
 13–15 15.2 (4.1) 12.8 (4.1) 10.7 (4.9) .001/0.12 .001/0.008 
 16–18 16.6 (4.3) 13.8 (4.4) 11.4 (5.1)   
X's and O's Average Correct RT 
 13–15 0.51 (0.07) 0.53 (0.08) 0.55 (0.12) .001/0.02 .001/0.005 
 16–18 0.49 (0.07) 0.52 (0.14) 0.53 (0.10)   
Symbol Match Average Correct RT 
 13–15 1.57 (0.41) 1.66 (0.52) 1.67 (0.49) .001/0.01 .36/0.00 
 16–18 1.54 (0.38) 1.64 (0.53) 1.69 (0.61)   
Color Match Average Correct RT 
 13–15 0.79 (0.12) 0.79 (0.18) 0.80 (0.22) .001/0.004 .001/0.003 
 16–18 0.75 (0.13) 0.78 (0.17) 0.79 (0.20)   
Variable Normative Spanish -Eng Spanish -Spanish p/η2p/η2** 
Word Memorya 
 13–15 93.6 (6.1) 93.5 (6.1) 91.9 (6.1) .001/0.006 .94/0.00 
 16–18 93.6 (6.1) 93.6 (6.1) 91.7 (6.1)   
Symbol Match Total Correct Hidden 
 13–15 6.1 (1.9) 6.2 (2.0) 6.0 (2.0) .001/0.001 .20/0.00 
 16–18 6.2 (2.0) 6.2 (2.1) 6.0 (2.0)   
Three Letters Total Letters Correct 
 13–15 13.5 (1.8) 13.1 (1.9) 12.8 (1.9) .001/0.02 .001/0.002 
 16–18 13.8 (1.6) 13.3 (1.9) 12.9 (2.0)   
Design Memoryb 
 13–15 79.8 (12.5) 78.4 (12.8) 76.2 (12.8) .001/0.008 .09/0.00 
 16–18 80.4 (12.4) 78.8 (12.8) 76.4 (13.4)   
X's and O's Total Correct Memory 
 13–15 7.9 (2.4) 8.0 (2.3) 7.4 (2.6) .001/0.004 .001/0.001 
 16–18 8.1 (2.4) 8.1 (2.3) 7.6 (2.6)   
X's and O's Total Correct Interference 
 13–15 111.3 (8.5) 109.7 (9.9) 107.1 (11.2) .001/0.02 .001/0.01 
 16–18 114.5 (8.0) 112.3 (10.5) 109.4 (12.6)   
Three Letters Average Counted Correctly 
 13–15 15.2 (4.1) 12.8 (4.1) 10.7 (4.9) .001/0.12 .001/0.008 
 16–18 16.6 (4.3) 13.8 (4.4) 11.4 (5.1)   
X's and O's Average Correct RT 
 13–15 0.51 (0.07) 0.53 (0.08) 0.55 (0.12) .001/0.02 .001/0.005 
 16–18 0.49 (0.07) 0.52 (0.14) 0.53 (0.10)   
Symbol Match Average Correct RT 
 13–15 1.57 (0.41) 1.66 (0.52) 1.67 (0.49) .001/0.01 .36/0.00 
 16–18 1.54 (0.38) 1.64 (0.53) 1.69 (0.61)   
Color Match Average Correct RT 
 13–15 0.79 (0.12) 0.79 (0.18) 0.80 (0.22) .001/0.004 .001/0.003 
 16–18 0.75 (0.13) 0.78 (0.17) 0.79 (0.20)   

Notes: Post-Hoc Results: Normative > Spanish-English > Spanish-Spanish: Design Memory, Three Letters Total Letters Correct, X's and O's Average Correct RT, Symbol Match Average Correct RT, X's and O's Total Correct Interference, Three Letters Average Counted Correctly. Normative, Spanish-English < Spanish-Spanish: Symbol Match Total Correct Hidden, X's and O's Total Correct Memory. Normative < Spanish-English, Spanish-Spanish: Color Match Average Correct RT.

aWord Memory = (Word Memory Hits + Distractors + Delayed Hits + Delayed Distractors)/48.

bDesign Memory = (Design Memory Hits + Distractors + Delayed Hits + Delayed Distractors)/48.

*p-values and η2 (partial eta-squared) for language group comparisons; df = (2, 23,811).

**p-values and η2 (partial eta-squared) for age group comparisons; df = (1, 23,811).

Discussion

This is the first study to examine baseline neurocognitive performance of bilingual Hispanic athletes on ImPACT by examining group differences according to the language in which the test was administered. Small but significant differences were found in the performance of bilingual Hispanic athletes completing the ImPACT test in English versus Spanish, compared with English-speaking athletes. Regardless of the language of test administration, current results revealed that English-speaking athletes outperformed their bilingual Hispanic peers on Verbal Memory, Visual Memory, Visual Motor Speed, and Reaction time. English-speaking athletes also demonstrated a significantly lower Total Symptom scores compared with bilingual Hispanics who completed the ImPACT test in Spanish. Given the large sample size, small between-groups differences can result in significant p-values but reflect small effect sizes (Cohen, 1988). In this study, the differences between the English-speaking athletes who completed ImPACT in English and the Hispanic athletes who completed the test in Spanish reflect 50% or more of the “reliable change” documented by Iverson and colleagues (2003).

More specifically, differences between English-speaking athletes taking ImPACT in English versus Spanish-speaking athletes taking the test in Spanish (Table 3) easily surpass the 4.98 points required for Reliable Change (80% confidence interval) on Visual Motor Speed and approach the 0.06 points necessary for Reliable Change on Reaction Time. In comparison with normative data, the mean for Visual Motor Speed for Spanish-speaking athletes taking the test in Spanish, aged 13–15 (29.4), is consistent with performance at the 14th percentile, whereas the mean for Visual Motor Speed for English-speaking athletes taking the test in English, aged 13–15 (36.6), is consistent with performance at the 53rd percentile, reflecting a 1-SD difference. The reason for such a between-groups difference is not clear, especially on what would appear to be a “culture-free” subscale such as Visual Motor Speed. Similarly large-scale cross-cultural investigations (e.g., American, European performance on the Wechsler Adult Intelligence Scale-Third Edition [WAIS-III]) documented decreased processing speed scores among Europeans, perhaps reflecting differences in test-taking approaches (Roivainen, 2010). For instance, whereas speed of responding (or “fastness”) is culturally valued in the USA, it may be less valued in other cultures (Rosselli & Ardila, 2003). Thus, what appear to be small between differences between culture-based groups may have significant implications should post-concussion test scores be compared with normative data and not baseline test data (Echemendia et al., 2012).

The current results appear to be consistent with previous studies examining neurocognitive baseline performance between African American and White high school and college athletes (Kontos et al., 2010) and US football players and South African rugby players (Shuttleworth-Edwards et al., 2009) which questioned the cultural fairness of ImPACT. In this regard, the current findings raise the possibility that ImPACT is not culturally equivalent for Spanish-speaking Hispanics, English-speaking Hispanics, and English-speaking athletes. If the current results are correct, there is need for caution when interpreting neurocognitive test data, using ImPACT, for monolingual White and bilingual Hispanic Americans. Interestingly, the largest differences between language groups (e.g., small-to-medium effect sizes) were found on Visual Motor Speed and Reaction Time, and these findings are supported by ancillary analyses which reflect the largest effect sizes on those subscales contributing to the Visual Motor Speed and Reaction Time composite scores. On these composites, English-speaking athletes taking the test in English outperformed Spanish-speaking athletes taking the test in English, who, in turn, outperformed Spanish-speaking athletes taking the test in Spanish; these trends were noted on both the composite scores as well as the subscales comprising these composite scores. One might expect these two indices to be more “culture-free” than Verbal Memory, given that the language group was the primary grouping variable.

In the absence of baseline data for Hispanic athletes who complete ImPACT in Spanish, comparisons with normative data may result in misclassification of post-concussion scores. As such, neuropsychologists should be aware of normative indications for these athletes and consider these differences when interpreting test data, especially useful with concussed athletes that do not possess a pre-injury neurocognitive baseline. It has been the clinical experience of the authors that many Hispanic athletes often come from disadvantaged financial and educational backgrounds which may not afford them the opportunity to undergo a neurocognitive baseline assessment. Cultural differences aside, normative data can also provide useful comparisons for detecting cognitive impairment post-injury in any athlete whose baseline was invalid for a variety of reasons including suboptimal effort, poor testing conditions, mechanical test administration issues, and so on.

The present study raises the importance of caution when interpreting data from Hispanic Americans, although this could be difficult task, given that this is not a homogenous group, and such individuals may be Black, White, Black White hybrids, Native American, or mixed White and Native American (Lynn, 2006). In the USA, Hispanic individuals, or their ancestors, may have their origins in Spain, Mexico, Latin America, Cuba, Puerto Rico, South America, or one of many Caribbean islands. Cultural differences among these groups are likely to be significant, as well as the numerous distinct linguistic subgroups often represented under the broad label of “Spanish-speaking.” Test translation might also be an influencing factor in this study as there are likely regional dialects that should be considered. Unfortunately, information pertaining to these aforementioned factors were not obtained or controlled for in this study.

One explanation for the ethnic differences in baseline neurocognitive performance may be explained by potential variations in environmental influences and quality of education as the result of discrepancies in socioeconomic status (Shuttleworth-Edwards et al., 2004). More specifically, individuals with better economic means might have greater access to educational resources (e.g., computers) or enriched school environments where taking tests and instilling competitive motivation for obtaining optimal performance is a common practice. Perhaps the present sample of bilingual Hispanic athletes was not exposed to the same educational opportunities within a competitive environment, as were the English-speaking athletes. While English-speaking athletes self-reported significantly more previous concussions, it may also be possible that the bilingual Hispanic sample possessed a larger number of examinees with a history of undiagnosed or unreported concussion, or misidentified learning disabilities thereby producing poorer results and greater endorsement of symptoms. Although athletes who self-reported learning disabilities were excluded from analyses, it is possible that some athletes with the aforementioned history were not recognized and, therefore, their test data were included in the analysis. Differences in motivation to excel academically or to participate in sports might also be a factor that influences differences within ethnic groups. The difference in symptom endorsement that emerged between Hispanic-speaking athletes who completed the ImPACT test in Spanish, and English-speaking examinees may also reflect cultural differences or specific sensitivity and variability in linguistic background (Shuttleworth-Edwards, 2009). It is possible that confusion and lack of familiarity with the meaning of symptoms could influence responses.

It is important to note that there were significant differences in the sports in which athletes participated, as a function of the language groups. Although (American) Football was the most common sport reported by English-speaking test-takers (32.7%), (American) Football was only reported by 14.0% and 5.2% of Spanish-speaking athletes taking the test in English and Spanish (respectively). In contrast, Soccer was reported by only 17% of English-speaking test-takers, when compared with 51.1% and 61.3% of Spanish-speaking athletes taking the test in English and Spanish (respectively). Repetitive heading in soccer has been linked to white matter changes in the brain (Lipton et al., 2013), as well as neurocognitive deficits (Kaminski, Wikstrom, Gutierrez, & Glutting, 2007; Stephens, Rutherford, Potter, & Fernie, 2005; Witol & Webbe, 2003), although several studies show no neurocognitive effects of heading (Kontos, Dolese, Elbin, Covassin, & Warren, 2011; Rieder & Jansen, 2011; Rutherford, Stephens, Fernie, & Potter, 2009). However, given the possible effects of heading in soccer, as well as the greater number of Hispanic athletes participating in soccer, the current findings may be influenced by these group/sport differences.

The results also show that that bilingual Hispanic athletes completing ImPACT in English outperformed bilingual Hispanic athletes completing the test in their native language. It was initially hypothesized that Hispanic athletes might exhibit better performance taking the test in their native language compared with Hispanic athletes who took the test in English, but this was not supported by the results. The bilingual Hispanic athletes were allowed to assert a language preference when taking the test. This leads to the hypothesis that Hispanic participants who elected to take the test in English may have achieved a more advanced degree of acculturation than those who preferred to take the test in Spanish (Harris, Tulsky, & Schultheis, 2003). Yet, even with a greater degree of acculturation, the bilingual Hispanic athletes who completed the test in English earned poorer scores than the English-speaking athletes on several of the composite scores. This finding is in agreement with research that consistently reports lower scores on measures of cognitive ability by Hispanics than by Whites living in the USA (Lynn, 2006). In a recent investigation, the IQs of White and Hispanic individuals from the WAIS-IV (Wechsler, 2008) standardization over sample were compared. Their Hispanic participants were fluent in English and the WAIS-IV was administered in English. When the variance attributable to ethnicity, education, occupation, income, region of the country, and gender was accounted for using a regression-based methodology, the White participants still earned an average Full-Scale IQ that was approximately 7 points higher than the average Full-Scale IQ of the Hispanic participants. Obviously, unmeasured factors contributing to the Hispanic/White IQ gap remain to be identified.

Study Limitations

Information regarding socioeconomic status, quality of education, level of acculturation, reading equivalency, and number of years speaking respective languages was not obtained during the present study. A better-controlled study in which examinees are matched on these characteristics is needed and such an investigation is currently underway by the authors. It is possible that bilingual individuals would respond differently on a test as a result of the language in which the test is administered. For example, a bilingual examinee may perform more optimally when a test is administered in one language compared with test administration in another language (e.g., English vs. Spanish). This study did not examine how bilingual Spanish-speaking athletes would perform on both Spanish and English versions of ImPACT. Additionally, participants were not screened independently for neurological, attention deficit, or learning disorders, nor were demographic information provided by examinees verified. These factors may also be considered potential limitations of the present study.

We also recognize that the language groups were not homogenous with respect to gender distribution. However, considering that the difference in gender distribution only covered a 10% range (e.g., 67% men in the Hispanic-English group and 77% men in the Hispanic-Spanish group), this variable does not appear to be a major contributor to the present findings. To date, research pertaining to gender differences in baseline neurocognitive abilities in athletes has produced equivocal results (Barr, 2003; Brown, Guskiewicz, & Bleiberg, 2007; Covassin et al., 2006; Zuckerman et al., 2012). In addition, although ImPACT test developers provide test administration guidelines for optimal testing conditions, the current study was retrospective and conditions for test administration were not controlled. Similarly, information regarding the credentials and training of those overseeing the administration of the ImPACT assessment is unknown.

Conclusions

Overall, the present study revealed that regardless of the language of test administration (e.g., Spanish or English), significant differences emerged in neurocognitive performance and reported symptoms between Hispanic and English-speaking athletes, suggesting the need for caution in interpreting neurocognitive test data for Hispanic Americans on ImPACT. The present study suggests the importance of considering cultural and linguistic differences when administering and interpreting baseline neurocognitive assessment results. With culturally diverse groups, examiners should acquire Information pertaining to level of acculturation, language proficiency, and so on, prior to assessment so that valid results can be obtained and potential errors regarding misattribution or misdiagnosis of results can be minimized.

Conflict of Interest

S.O. has received honoraria from ImPACT Applications to conduct educational workshops. P.S. and S.O. have served as consultants to ImPACT Applications, Inc., however, ImPACT Applications, Inc., had no role in the conceptualization of the study, the collection or analysis of the data, the writing of the article, or the decision to submit it for publication. J.J.R. reports no conflict of interest relevant to this publication.

References

Alegret
M.
Espinosa
A.
Vinyes-Junque
G.
Valero
S.
Hernandez
I.
Tarraga
L.
, et al.  . 
Normative data of a brief neuropsychological battery for Spanish individuals older than 49
Journal of Clinical and Experimental Neuropsychology
 , 
2012
, vol. 
34
 (pg. 
209
-
219
)
Allen
B. J.
Gfeller
J. D.
The immediate post concussion assessment and cognitive testing battery and traditional neuropsychological measures: A construct and concurrent validity study
Brain Injury
 , 
2011
, vol. 
25
 (pg. 
179
-
191
)
Arnold
B. R.
Montgomery
G. T.
Castaneda
I.
Longoria
R.
Acculturation and performance of Hispanics on selected Halstead-Reitan neuropsychological tests
Assessment
 , 
1993
, vol. 
1
 (pg. 
239
-
248
)
Aubry
M.
Cantu
R.
Dvorak
J.
Johnston
K.
Kelly
J.
Lovell
M.
, et al.  . 
Summary and agreement statement of the first International Conference on Concussion in Sport, Vienna, 2001
British Journal of Sport Medicine
 , 
2002
, vol. 
36
 (pg. 
3
-
7
)
Barr
W.
Neuropsychological testing of high school athletes preliminary norms and test-retest indices
Archives of Clinical Neuropsychology
 , 
2003
, vol. 
18
 (pg. 
91
-
101
)
Barth
J. T.
Alves
W. M.
Ryan
T. V.
Macciocchi
S. N.
Rimel
R. W.
Jane
J. J.
, et al.  . 
Levin
H. S.
Eisenberg
H. M.
Benton
A .L.
Mild head injury in sports: Neuropsychological sequelae and recovery of function
Mild head injury
 , 
1989
New York
Oxford University Press
(pg. 
257
-
275
)
Broglio
S. P.
Macciocchi
S. N.
Ferrara
M. S.
Neurocognitive performance of concussed athletes when symptom free
Journal of Athletic Training
 , 
2007
, vol. 
42
 (pg. 
504
-
508
)
Brown
C.
Guskiewicz
K.
Bleiberg
J.
Athlete characteristics and outcome scores for computerized neuropsychological assessment: A preliminary analysis
Journal of Athletic Training
 , 
2007
, vol. 
42
 (pg. 
515
-
523
)
Bure-Reyes
A.
Hidalgo-Ruzzante
N.
Vilar-Lopez
R.
Gontier
J.
Sanchez
L.
Perez-Garcia
M.
, et al.  . 
Neuropsychological test performance of Spanish speakers: Is performance different across different Spanish-speaking groups?
Journal of Clinical and Experimental Neuropsychology
 , 
2013
, vol. 
35
 
4
(pg. 
404
-
412
)
Cohen
J.
Statistical power analysis for the behavioral sciences
 , 
1988
Hillside, NJ
Lawrence Erlbaum Associates
Cosentino
S.
Manly
J.
Mungas
D.
Do reading tests measure the same construct in multiethnic and multilingual older persons?
Journal of the International Neuropsychological Society
 , 
2007
, vol. 
13
 (pg. 
228
-
236
)
Covassin
T.
Elbin
R.
Kontos
A.
Larson
E.
Investigating baseline and neurocognitive performance between male and female athletes with a history of multiple concussions
Journal of Neurology, Neurosurgery, and Psychiatry
 , 
2010
, vol. 
8
 
25
(pg. 
734
-
744
)
Covassin
T.
Swanik
C.
Sachs
M.
, et al.  . 
Sex differences in baseline neuropsychological function and concussion symptoms of collegiate athletes
British Journal of Sports Medicine
 , 
2006
, vol. 
40
 (pg. 
923
-
927
)
Echemendia
R. J.
Bruce
J. M.
Bailey
C. M.
Sanders
J. F.
Arnett
P.
Vargas
G.
The utility of post-concussion neuropsychological data in identifying cognitive change following sports-related MTBI in the absence of baseline data [Comparative Study]
The Clinical Neuropsychologist
 , 
2012
, vol. 
26
 
7
(pg. 
1077
-
1091
)
Echemendia
R. J.
Cantu
R. C.
Return to play following sports-related mild traumatic brain injury: The role for neuropsychology
Applied Neuropsychology
 , 
2003
, vol. 
10
 (pg. 
48
-
55
)
Elbin
R. J.
Schatz
P.
Covassin
T.
One-year test-restest reliability of the online version of the ImPACT in high school athletes
American Journal of Sports Medicine
 , 
2011
, vol. 
39
 (pg. 
2319
-
2324
)
Fazio
V. C.
Lovell
M. R.
Pardini
J. E.
Collins
M. W.
The relation between post concussion symptoms and neurocognitive performance in concussed athletes
NeuroRehabilitation
 , 
2007
, vol. 
22
 (pg. 
207
-
216
)
Gasquoine
P. G.
Croyle
K. L.
Cavazos-Gonzales
C.
Sandoval
O.
Language of administration and neuropsychological test performance in neurologically intact Hispanic American bilingual adults
Archives of Clinical Neuropsychology
 , 
2007
, vol. 
22
 (pg. 
991
-
1001
)
Gasquoine
P. G.
Gonzalez
C. D.
Using monolingual neuropsychological test norms with bilingual Hispanic Americans: Application of an individual comparison standard
Archives of Clinical Neuropsychology
 , 
2012
, vol. 
27
 (pg. 
268
-
276
)
Grindel
S. H.
Lovell
M. R.
Collins
M. W.
The assessment of sports-related concussion: The evidence behind neuropsychological testing and management
Clinical Journal of Sport Medicine
 , 
2001
, vol. 
11
 (pg. 
134
-
143
)
Guskiewicz
K. M.
Bruce
S. L.
Cantu
R. C.
Ferrara
M. S.
McCrea
M.
Putukian
M.
, et al.  . 
National athletic trainers’ association position statement: Management of sports-related concussion
Journal of Athletic Training
 , 
2004
, vol. 
39
 (pg. 
280
-
297
)
Harris
J. G.
Tulsky
D. S.
Schultheis
M. T.
Tulsky
D. S.
Saklofske
D. H.
Chelune
G. J.
Heaton
R. K.
Ivnik
R. J.
Bornstein
R.
, et al.  . 
Assessment of the non-native English speaker: Assimilating history and research findings to guide clinical practice
Clinical interpretation of the WAIS-III and WMS-III
 , 
2003
Boston
Academic Press
Heaton
R.
Miller
S.
Taylor
M.
Grant
I.
Revised comprehensive norms for an expanded Halstead-Reitan battery: Demographically adjusted neuropsychological norms for African Americans and Caucasian adults
 , 
2004
Lutz, FL
Psychological Assessment Resources
Iverson
G. L.
Gaetz
M.
Lovell
M.
Collins
M.
Validity of ImPACT for measuring processing speed following sports-related concussion
Journal of Clinical and Experimental Neuropsychology
 , 
2005
, vol. 
27
 (pg. 
683
-
689
)
Iverson
G. L.
Lovell
M. R.
Collins
M. W.
Interpreting change on ImPACT following sports concussion
The Clinical Neuropsychologist
 , 
2003
, vol. 
7
 (pg. 
460
-
467
)
Iverson
G. L.
Lovell
M. R.
Collins
M. W.
Validity of ImPACT for measuring attention, processing speed following sports-related concussion
Journal of Clinical and Experimental Neuropsychology
 , 
2005
, vol. 
27
 (pg. 
683
-
689
)
Johnson
E. W.
Kegel
N. E.
Collins
M. W.
Neuropsychological assessment of sport-related concussion
Clinics in Sports Medicine
 , 
2011
, vol. 
30
 (pg. 
73
-
88
)
Kaminski
T. W.
Wikstrom
A. M.
Gutierrez
G. M.
Glutting
J. J.
Purposeful heading during a season does not influence cognitive function or balance in female soccer players
Journal of Clinical and Experimental Neuropsychology
 , 
2007
, vol. 
29
 (pg. 
742
-
751
)
Keith
T.
Rynolds
M.
Patel
P.
Ridley
K.
Sex differences in latent cognitive abilities ages 6 to 59: Evidence from the Woodcock –Johnson III tests of cognitive abilities
Intelligence
 , 
2008
, vol. 
36
 (pg. 
502
-
525
)
Kennepohl
S.
Shore
D.
Nabors
N.
Hanks
R.
African American acculturation and neuropsychological test performance following traumatic brain injury
Journal of the International Neuropsychological Society
 , 
2004
, vol. 
10
 (pg. 
566
-
577
)
Kontos
A.
Elbin
R.
Covassin
T.
Larson
E.
Exploring differences in computerized neurocognitive concussion testing between African American and White athletes
Archives of Clinical Neuropsychology
 , 
2010
, vol. 
25
 
8
(pg. 
734
-
744
)
Kontos
A. P.
Dolese
A.
Elbin
R. J.
Covassin
T.
Warren
B. L.
Relationship of soccer heading to computerized neurocognitive performance and symptoms among female and male youth soccer players [Comparative Study]
Brain Injury
 , 
2011
, vol. 
25
 
12
(pg. 
1234
-
1241
)
Lau
B. C.
Collins
M. W.
Lovell
M. R.
Sensitivity and specificity of subacute computerized neurocognitive testing and symptom evaluation in predicting outcomes after sports related concussion
American Journal of Sports Medicine
 , 
2011
, vol. 
39
 (pg. 
1209
-
1216
)
Lipton
M. L.
Kim
N.
Zimmerman
M. E.
Kim
M.
Stewart
W. F.
Branch
C. A.
, et al.  . 
Soccer heading is associated with white matter microstructural and cognitive abnormalities
Radiology
 , 
2013
, vol. 
268
 
3
(pg. 
850
-
857
)
Lovell
M.
Echemendia
R. J.
The ImPACT neuropsychological test battery
Sports neuropsychology: Assessment and management of traumatic brain injury
 , 
2006
New York
The Guilford Press
(pg. 
193
-
215
)
Lovell
M. R.
Clinical interpretation manual, online ImPACT 2007–2012
 , 
2012
Pittsburgh, PA
ImPACT Applications
Lynn
R.
Race differences in intelligence: An evolutionary analysis
 , 
2006
Augusta, GA
Washington Summit Publishers
Maerlender
A.
Flashman
L.
Kessler
A.
Kumbhani
S.
Greenwald
R.
Tosteson
T.
, et al.  . 
Examination of the construct validity of ImPACT computerized test, traditional, and experimental neuropsychological measures [Research Support, N.I.H., Extramural]
The Clinical Neuropsychologist
 , 
2010
, vol. 
24
 
8
(pg. 
1309
-
1325
)
Maerlender
A.
Flashman
L.
Kessler
A.
Kumbhani
S.
Greenwald
R.
Tosteson
T.
, et al.  . 
Discriminant construct validity of ImPACT: A companion study [Research Support, N.I.H., Extramural]
The Clinical Neuropsychologist
 , 
2013
, vol. 
27
 
2
(pg. 
290
-
299
)
Maitland
S.
Intrieri
R.
Schaie
W.
Willis
S.
Gender differences and changes in cognitive abilities across the adult life span
Aging, Neuropsychology, and Cognition
 , 
2000
, vol. 
7
 (pg. 
32
-
53
)
Manly
J. J.
Byrd
D. A.
Touradji
P.
Stern
Y.
Acculturation, reading level, and neuropsychological test performance among African American elders
Applied Neuropsychology
 , 
2004
, vol. 
11
 (pg. 
37
-
46
)
Manly
J. J.
Echemendia
R. J.
Race specific norms: Using the model of hypertension to understand issues of race, culture, and education in neuropsychology
Archives of Clinical Neuropsychology
 , 
2007
, vol. 
22
 (pg. 
319
-
325
)
Manly
J. J.
Miller
S. W.
Heaton
R. K.
Byrd
D.
Reilly
J.
Velasquez
R. J.
, et al.  . 
The HIV Neurobehavioral Research Center (HNCR) Group
The effect of African-American acculturation on neuropsychological test performance in normal and HIV-positive individuals
Journal of the International Neuropsychological Society
 , 
1998
, vol. 
4
 (pg. 
291
-
302
)
Mayers
L. B.
Redick
T. S.
Clinical utility of ImPACT assessment for post-concussion return-to-play counseling: Psychometric issues
Journal of Clinical and Experimental Neuropsychology
 , 
2012
, vol. 
34
 (pg. 
235
-
242
)
McCrea
M.
Barr
W. B.
Guskiewicz
K.
Randolph
C.
Marshall
S. W.
Cantu
R.
, et al.  . 
Standard regression-based methods for measuring recovery after sport-related concussion
Journal of the International Neuropsychological Society
 , 
2005
, vol. 
11
 
1
(pg. 
58
-
69
)
McCrory
P.
Johnston
K.
Meeuwisse
W.
Aubry
M.
Cantu
R.
Dvorak
J.
, et al.  . 
Summary and agreement statement of the 2nd international conference on concussion in sport, Prague, 2004
British Journal of Sports Medicine
 , 
2005
, vol. 
39
 (pg. 
196
-
204
)
McCrory
P.
Meeuwisee
W.
Aubry
M.
Cantu
B.
Dvorak
J.
Echemendia
R.
, et al.  . 
Consensus statement on concussion in sport: The 4th international conference on concussion in sport held in Zurch, November 2012
British Journal of Sports Medicine
 , 
2013
, vol. 
47
 (pg. 
250
-
258
)
McCrory
P.
Meeuwisse
W.
Johnston
K.
Dvorak
M.
Aubry
M.
Molloy
R.
, et al.  . 
Consensus statement on concussion in sport-the 3rd international conference in sport, held in Zurich, November 2008
Journal of Clinical Neuroscience
 , 
2009
, vol. 
16
 (pg. 
755
-
763
)
Moser
R. S.
Iverson
G. L.
Echemendia
R. J.
Lovell
M. R.
Schatz
P.
Webbe
F. M.
, et al.  . 
Neuropsychological evaluation in the diagnosis and management of sports-related concussion: NAN position paper
Archives of Clinical Neuropsychology
 , 
2007
, vol. 
22
 (pg. 
909
-
916
)
NCAA
Race and Gender: Member Institutions’ Personnel Report. The NCAA Minority Opportunities and Interests Committee's Annual Study
2010
Indianapolis, Indiana
National Collegiate Athletic Association
Norman
M.
Moore
D.
Taylor
M.
Franklin
D.
Cysique
L.
Ake
C.
, et al.  . 
Demographically corrected norms for African Americans and Caucasians on the Hopkins verbal learning test-revised, brief visuospatial memory test-revised, Stroop color and word test, and Wisconsin card sorting test 64-card version
Journal of Clinical and Experimental Neuropsychology
 , 
2011
, vol. 
33
 
7
(pg. 
793
-
804
)
Pena-Casanova
J.
Blesa
R.
Aguilar
M.
Gramunt-Fombuena
N.
Gomez-Anson
B.
Oliva
R.
, et al.  . 
Spanish multicenter normative studies (NEURONORMA Project): Methods and sample characteristics
Archives of Clinical Neuropsychology
 , 
2009
, vol. 
24
 (pg. 
307
-
319
)
Prifitera
A.
Saklofske
D. H.
Weiss
L. G.
WISC-IV: Clinical assessment and intervention
 , 
2008
2nd ed.
Boston
Academic Press
 
(Eds.)
Putukian
M.
Neuropsychological testing as it relates to recovery from sports-related concussion
Physical Medicine and Rehabilitation
 , 
2011
, vol. 
3
 (pg. 
S425
-
S432
)
Quintana
M.
Pena-Casanova
J.
Sanchez-Benavides
G.
Langohr
K.
Manero
R.
Aguilar
M.
, et al.  . 
Spanish multicenter normative studies (Neuronorma Project): Norms for the abbreviated Barcelona test
Archives of Clinical Neuropsychology
 , 
2011
, vol. 
26
 (pg. 
144
-
157
)
Randolph
C.
Kirkwood
M.
What are the real risks of sport-related concussion, and are they modifiable?
Journal of the International Neuropsychological Society
 , 
2009
, vol. 
15
 (pg. 
512
-
520
)
Randolph
C.
Lovell
M. R.
Laker
S. R.
Neuropsychological Testing Point/Counterpoint
Physical Medicine and Rehabilitation
 , 
2011
, vol. 
3
 (pg. 
S433
-
S439
)
Randolph
C.
McCrea
M.
Barr
W. B.
Is neuropsychological testing useful in the management of sport-related concussion?
Journal of Athletic Training
 , 
2005
, vol. 
40
 
3
(pg. 
139
-
152
)
Razani
J.
Murcia
G.
Tabares
J.
Wong
J.
The effects of culture on WASI test performance in ethnically diverse individuals
The Clinical Neuropsychologist
 , 
2007
, vol. 
21
 (pg. 
776
-
788
)
Resch
J.
Driscoll
A.
McCaffrey
N.
Brown
C.
Ferrara
M. S.
Macciocchi
S.
, et al.  . 
ImPACT test-retest reliability: Reliably unreliable?
Journal of Athletic Training
 , 
2013
, vol. 
48
 
4
(pg. 
506
-
511
)
Rieder
C.
Jansen
P.
No neuropsychological consequence in male and female soccer players after a short heading training
Archives of Clinical Neuropsychology
 , 
2011
, vol. 
26
 
7
(pg. 
583
-
591
)
Roivainen
E.
European and American WAIS-III norms: Cross-national differences in performance subtest scores
Intelligence
 , 
2010
, vol. 
38
 
1
(pg. 
187
-
192
)
Rosselli
M.
Ardila
A.
The impact of culture and education on non-verbal neuropsychological measurements: A critical review
Brain and Cognition
 , 
2003
, vol. 
52
 
3
(pg. 
326
-
333
)
Rutherford
A.
Stephens
R.
Fernie
G.
Potter
D.
Do UK university football club players suffer neuropsychological impairment as a consequence of their football (soccer) play? [Research Support, Non-U.S. Gov't]
Journal of Clinical and Experimental Neuropsychology
 , 
2009
, vol. 
31
 
6
(pg. 
664
-
681
)
Schatz
P.
Long-term test-retest reliability of baseline cognitive assessments using ImPACT
American Journal of Sports Medicine
 , 
2009
, vol. 
38
 (pg. 
47
-
53
)
Schatz
P.
Ferris
C. S.
One-Month test-retest reliability of the ImPACT Test Battery
Archives of Clinical Neuropsychology
 , 
2013
, vol. 
28
 (pg. 
499
-
504
)
Schatz
P.
Kontos
A.
Elbin
R. J.
Response to Mayers and Redick: “Clinical utility of ImPACT assessment for postconcussion return-to-play counseling: Psychometric issues.”
Journal of Clinical and Experimental Neuropsychology
 , 
2012
, vol. 
34
 (pg. 
428
-
434
)
Schatz
P.
Pardini
J.
Lovell
M. R.
Collins
M. W.
Podell
K.
Sensitivity and specificity of the ImPACT test battery for concussion in athletes
Archives of Clinical Neuropsychology
 , 
2006
, vol. 
21
 
1
(pg. 
91
-
99
)
Schatz
P.
Sandel
N.
Sensitivity and specificity of the online version of ImPACT in high school and collegiate athletes
American Journal of Sports Medicine
 , 
2012
, vol. 
41
 (pg. 
321
-
326
)
Schatz
P.
Zilmer
E. A.
Computer-based assessment of sports-related concussion
Applied Neuropsychology
 , 
2003
, vol. 
10
 (pg. 
42
-
47
)
Shuttleworth-Edwards
A.
Whitefield-Alexander
V.
Radloff
S.
Taylor
A.
Lovell
M.
Computerized neuropsyhological profiles of South African versus United States athletes: A basis for commentary on cross-cultural norming issues in the sports concussion arena
The Physician and Sports Medicine
 , 
2009
, vol. 
37
 (pg. 
45
-
51
)
Shuttleworth-Edwards
A. B.
Kemp
R.
Rust
A.
Muirhead
J.
Hartman
N.
Radloff
S.
Cross cultural effects on IQ performance: A review and preliminary normative indications on the WAIS-III test performance
Journal of Clinical and Experimental Neuropsychology
 , 
2004
, vol. 
26
 (pg. 
903
-
920
)
Stephens
R.
Rutherford
A.
Potter
D.
Fernie
G.
Neuropsychological impairment as a consequence of football (soccer) play and football heading: a preliminary analysis and report on school students (13–16 years)
2005
, vol. 
11
 (pg. 
513
-
526
)
US Census
2010
Washington, DC
U.S. Census Bureau
 
Census Summary, U.S. Department of Commerce, Economics and Statistics Administration
Van Kampen
D. A.
Lovell
M. R.
Pardini
J. E.
Collins
M. W.
Fu
F. H.
The “value added” of neurocognitive testing after sports-related concussion
The American Journal of Sports Medicine
 , 
2006
, vol. 
34
 (pg. 
1630
-
1635
)
Wechsler
D.
WAIS-IV. Wechsler adult intelligence scale-fourth edition. Technical manual
 , 
2008
San Antonio, TX
The Psychological Corporation
Weiss
L. G.
Saklofske
D. H.
Coalson
D.
Raiford
S. E.
WAIS-IV: Clinical use and interpretation
 , 
2010
Boston
Academic Press
 
(Eds.)
Witol
A. D.
Webbe
F. M.
Soccer heading frequency predicts neuropsychological deficits
Archives of Clinical Neuropsychology
 , 
2003
, vol. 
18
 
4
(pg. 
397
-
417
)
Zuckerman
S.
Solomon
G.
Forbes
M.
Haase
R.
Sills
A.
Lovell
M.
Response to acute concussive injury in soccer players: Is gender a modifying factor?
Journal of Neurosurgery: Pediatrics
 , 
2012
, vol. 
10
 (pg. 
1
-
7
)