Abstract

The use of clinical neuropsychological tests in the evaluation of National Football League (NFL) players has been ongoing for more than two decades. Prior research has demonstrated that the NFL population may perform differently than the general population on standard paper and pencil neuropsychological tests. Given the increased interest in the longitudinal and long-term assessment of neurocognitive functioning in this group of athletes, we reviewed the published neuropsychological literature in an attempt to compile an initial compendium of available normative data on paper and pencil as well as computerized neuropsychological tests for this group of football players. Thirteen published studies met the inclusion criteria, and the results are presented by athlete status (active vs. retired) and classified by neuropsychological domain. Suggestions for potential core batteries with this population are discussed, as are directions for future research.

Introduction

The use of baseline neuropsychological testing for concussion assessment and evaluation had its genesis in the National Football League (NFL) in 1993. The Pittsburgh Steelers, with a grant from NFL Charities, initiated a pilot program with a group of players who underwent baseline and post-concussion neurocognitive testing voluntarily. Due to highly publicized incidents with several concussed players in the early 1990s, baseline neuropsychological testing for concussion gained momentum in the NFL and, by the mid-1990s, a number of teams voluntarily adopted the baseline testing approach. The rationale of the baseline approach was to obtain an individualized measure of an athlete's brain, and in the event of a concussive injury, to repeat the neurocognitive test in an attempt to determine when neurocognitive functioning had returned to baseline.

In 1994, the NFL created the mild traumatic brain injury (mTBI) Committee and began a clinical-research program collecting baseline and post-concussion test data. These initial baseline testing efforts employed traditional paper and pencil neuropsychological tests adapted for sports concussion purposes. The NFL Neuropsychological Test Battery (Pellman, Lovell, Viano, Casson, & Tucker, 2004) included the Hopkins Verbal Learning Test-Revised (HVLT-R; Brandt, 1991), Brief Visuospatial Learning Test-Revised (BVMT-R; Benedict, Schretlen, Groninger, Dobraski, & Shpritz, 1996), Trail Making Test Parts A & B (TMT-A, TMT-B; Reitan, 1958), Digit Span and Symbol Search from the WAIS (Wechsler, 1997), Symbol Digit Modalities Test (SDMT; Smith, 1982), and Controlled Oral Word Association Test (COWAT; Benton & Hamsher, 1978). With the transition to computer-based neurocognitive testing in the late 1990s, studies began to appear utilizing the newer technology.

Since 2000, several studies have been published in the literature providing neurocognitive data on NFL draft picks, athletes, and retirees, reporting the results of both paper and pencil and computerized neurocognitive tests. The first published paper appeared in 2000, when Kutner and colleagues studied 53 active NFL players who were administered MicroCog (Powell et al., 1993) and underwent apolipoprotein ε4 (ApoE ε4) genetic testing. Athletes were grouped by age and ApoE ε4 status, with age serving as a surrogate index of concussion exposure. This study found that older athletes who were positive for ApoE ε4 had lower cognitive test scores than any of the other groups. Summary data on the MicroCog General Cognitive Functioning Summary Index were reported by group, but basic psychometric data were not reported for the entire sample, as this was not a primary purpose of the study.

The second paper to appear in the literature presented the results of the NFL mTBI's Committee's study of athletes' performances on the NFL Neuropsychological Test Battery (Pellman et al., 2004). A major purpose of this study was to assess the time to return to neurocognitive baseline in concussed NFL athletes. Means and SDs for the paper and pencil measures were reported for 165–655 athletes (the number of athletes per test administered varied). Statistics related to concussion history of the entire sample were not provided, although analyses of players sustaining ≤3 concussions and ≥3 concussions were reported, with n's ranging from 12 to 95 on the different neuropsychological tests, and no significant between-group effect noted.

The results of a second sample of the NFL athletes participating voluntarily in a baseline testing study (number of athletes ranged from 95 to 366 per test) were reported by Lovell and Solomon (2011). This second sample of NFL athletes with paper and pencil data (not included in the initial cohort, reported in the Pellman et al., 2004 study) revealed some minor between-group differences, with mean test score differences found between active NFL athletes and general population norms on immediate and delayed recall of the HVLT-R, Trails A&B, and COWAT (Lovell & Solomon, 2011). Concussion history of this sample was not reported. Lovell and Solomon also performed a factor analysis on 503 active players who had completed all aspects of the paper and pencil testing. The pattern of the variable loadings yielded identifiable factors of verbal learning (HVLT-R), visual learning (BVMT-R), and executive functioning (an amalgam of Trails A&B and COWAT). Digit Span and Symbol Digit Modalities data were excluded from the factor analysis since some NFL neuropsychological consultants used the WAIS-R version of Digit Span, while others utilized the WAIS-III version (the NFL study began in 1996 and the WAIS-III was published in 1997). Similarly, some consultants utilized Digit Symbol from the WAIS-R or WAIS-III while others utilized Symbol Digit Modalities.

The second study of NFL players' performance on computerized neurocognitive tests was published by Pellman, Lovell, Viano, and Casson (2006). Pellman and colleagues reported Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT; Lovell, Collins, Podell, Powell, & Maroon, 2000) and demographic data on 68 NFL athletes who completed baseline assessments (mean number of prior concussions was 0.65, with a range of 0–4), and 48 other NFL athletes who completed at least 2 post-concussion assessments (mean number of prior concussions was 1.2, with a range of 0–4). A separate group of high school students was also evaluated for comparative purposes. A major finding of this study was that the NFL athletes returned to neurocognitive baseline on ImPACT more rapidly than high school athletes.

A study of ImPACT normative data in NFL players was published by Solomon and Haase (2008). This study detailed performances on ImPACT among a sample of “consecutively admitted” athletes to an NFL team's roster, and examined the effects of age, years of education, prior concussions (over 61% of the athletes reported no concussion history, 22.6% reported one concussion, and 15.7% reported ≥2 concussions), history of ADHD or LD, and headache status on ImPACT performance. The results of that study revealed an absence of an effect for age, education, or number of prior concussions on ImPACT scores, and partial effects for ADHD/LD and headache on ImPACT scores.

Hart et al. (2013) assessed 34 retired NFL players (mean age, 61.8 years) with neurological examinations and neuropsychological testing; a subset of 26 players also underwent neuroimaging. The neuropsychological tests included a two-subtest form of the Wechsler Abbreviated Scale of Intelligence (Vocabulary and Matrix Reasoning), TMT-A, TMT-B, the Digit Span and Coding subtests from the Wechsler Adult Intelligence Scale–IV (Wechsler, 2008), Category Fluency, the COWAT, Boston Naming Test (BNT; Kaplan, Goodglass, and Weintraub, 1983), Rey–Osterrieth Complex Figure Test, including copy and memory components (ROCFT; Rey, 1941), California Verbal Learning Test-II (CVLT-II; Delis, Kramer, Kaplan, & Ober, 2000), the Semantic Object Retrieval Test (SORT; Kraut et al., 2006), with one score for semantic memory retrieval and another for name production (word finding), and the Beck Depression Inventory-II (BDI; Beck, Steer, & Brown, 1996). All but 2 of the 34 athletes had sustained at least one concussion; mean lifetime concussions for the group was 4.0, with a concussion range of 1–13. Using established clinical criteria, 20 of the 34 retired NFL players were cognitively normal, 4 were diagnosed as having a fixed cognitive deficit, 8 were diagnosed with mild cognitive impairment, and 2 were diagnosed with dementia. Eight of the retirees were diagnosed with depression. Hart et al. reported neuropsychological group data on 12 cognitively unimpaired NFL retirees and 10 cognitively impaired retirees. Between-group differences (with Bonferroni corrections set at p < .004) were noted on CVLT episodic verbal, ROCFT delayed visual memory, BNT naming, and SORT word finding.

In a study that utilized neuroimaging as well as neuropsychological assessment, Amen and colleagues (2011) evaluated 100 active and retired NFL players (average age = 57.27, SD = 12.37) with SPECT, qEEG, MicroCog, and Conners' Continuous Performance Test-II (CPT; Conners & Staff, 2000). Although general concussion history was collected, only concussions with loss of consciousness (LOC) were reported in the article, with a range of 0–20 noted. Thirty-seven percent reported no episodes with LOC, 15% each reported 1 or 2 episodes, 18% reported 3 episodes, and 14% reported ≥5 episodes. When compared with a normal control group of 20 males with benign concussion and neuropsychiatric histories (mean age = 50.0 years, SD = 16.1), the authors reported that “Significant decreases from normal values were found in most neuropsychological tests.”

In a later study, Ford, Giovanello, and Guskiewicz (2013) studied 27 retired NFL players (mean age = 63.4 years SD = 5.9) with functional neuroimaging and neuropsychological tests. They divided the retirees into a low concussion group, defined as 2-or-fewer concussions, and a high concussion group, defined as 3-or-more concussions. The neuropsychological tests included the Telephone Interview for Cognitive Status (Brandt, Spencer, & Folstein, 1988), Mini-Mental State Exam (MMSE; Folstein, Folstein, & McHugh, 1975), COWAT, Trail Making Test Part B, BNT, Wechsler Test of Adult Reading (WTAR; Wechsler, 2001), the Geriatric Depression Scale (GDS; Yesavage et al., 1983), and a four subtests form (Arithmetic, Similarities, Picture Completion, and Digit Symbol-Coding) of the WAIS-III (Wechsler, 1997). Although extensive analyses of the fMRI data are reported, statistical tests were not conducted on the neuropsychological tests to assess for between-group differences.

In a more recent study of retired athletes, Casson, Viano, Haacke, Kou, & LeStrange (2014) studied 45 NFL players (mean age 45.6 ± 8.9 years) via a multimodal assessment technique which included neurological and neuropsychological examinations, neuroimaging, blood tests, and genetic typing. The players in this study reported a mean (SD) of 6.9 (6.2) NFL concussions, with a maximum number of 25. Over three-fourths of the athletes reported ≥3 NFL concussions. The lifetime football mean (SD) number of concussions for this sample was 9.0 (6.9). Neuropsychological tests included the BDI, MMSE, WTAR, Test of Memory Malingering, CVLT, BVMT-R, Verbal Fluency, Category Fluency, ImPACT, WAIS-III Digit Symbol and Letter Number Sequencing, and Trail Making Test-Part B. Casson and colleagues then created three groups based on t- and standard scores (mean = 10, SD = 3) on the paper and pencil neuropsychological data. Group 0 was comprised of unimpaired athletes with t-scores of 38 or greater and standard scores of 7 or greater. Group 1 included athletes with borderline performance, defined as a t-score of 33–37 or a standard score of <5. Group 2 included retirees with impaired performance, defined as t-scores of 32 or below and standard scores of 4 or below. Athletes were then assigned to groups 1–6 based largely on the number of impaired performances. The groups were delineated as follows: Group 0 = normal, Group 1 = borderline (borderline scores on 1 or more subtests (domains) but no impaired scores, Group 2 = impaired on only 1 subtest (domain), Group 3 = impaired on exactly 2 subtests (domains), Group 4 = impaired on 3 or more subtests (domains), Group 5 = borderline or impaired performance on 1 or more subtests (domains) with abnormal effort test (Test of Memory Malingering) score or Verbal IQ < 80. All athletes in Groups 0–4 had Verbal IQ > 80 and normal TOMM results. Casson and colleagues (2014) reported that the neuropsychological testing revealed isolated impairments in 11 players (24%), but none had dementia.

Other neuropsychometric studies of NFL athletes have been published in the recent past (e.g., Randolph, Karantzoulis, & Guskiewicz, 2013, with concussion history not reported; Seichepine et al., 2013, with a mean concussion history of 5.7; and Solomon & Kuhn, 2014, with 71% reporting a benign concussion history). The published neurocognitive data base on active and former NFL players is expanding, and the roles played by clinical neuropsychologists in the evaluation of professional football players are increasing (Solomon, Haase, & Kuhn, 2013). Given the importance of neuropsychological testing in the objective assessment of neurocognitive status, the role played by neuropsychological testing in disability determination, and the growing number of retired NFL claimants, it is likely that many clinical neuropsychologists are receiving (or will be receiving) an increasing number of referrals for neuropsychological testing for this population. As mentioned, a prior study (Lovell & Solomon, 2011) had indicated that published normative neuropsychological test data for the general population may not reflect adequately the normative values for professional American football players. For this reason, we reviewed the current literature to ascertain the available published neuropsychological data for professional football players, and to summarize the results in an attempt to provide an initial compendium of empirical data for this population.

Method

The PUBMED electronic database was searched systematically in the English language between the years 1960 and September, 2014. Search terms included professional football, NFL, concussion, neurocognitive testing, and neuropsychological testing. Inclusion criteria were the availability of test scores (with means and SDs and/or standard scores) from standardized neuropsychological tests on active or retired NFL players.

Results

Thirteen studies met the inclusion criteria and are summarized in Table 1. Quantitative results are presented by neuropsychological domain/content area and the tests utilized. Paper and pencil test findings are reviewed first, followed by computerized test results.

Table 1.

Studies reviewed

Study n Agea Player status 
Amen and colleagues (2011) 100 57.27 (12.37) Active and Retired 
Casson and colleagues (2014) 45 45.6 (8.9) Retired 
Didehbani and colleagues (2013) 30 58.60 (10.33) Retired 
Ford and colleagues (2013), low concussionb 15 64.1 (6.8) Retired 
Ford and colleagues (2013), high concussionc 12 62.6 (5.0) Retired 
Hart and colleagues (2013), cognitively unimpaired 12 55.4 (95% CI = 47.2–63.4) Retired 
Hart and colleagues (2013), cognitively impaired 10 66.6 (95% CI = 61.6–71.6) Retired 
Kutner and colleagues (2000) 53 27.1 (2.65) Active 
Lovell and Solomon (2011)d 95–366 NRe Active 
Lovell and Solomon, (2011)f 503 NR Active 
Pellman and colleagues (2004) 155–655 25.4 (NR) Active 
Pellman and colleagues (2006) 68 24.3 (NR) Active 
Randolph and colleagues (2013) 41 64.2 (5.5) Retired 
Seichepine and colleagues (2013) 29 48.3 (13.0) Retired 
Solomon and Haase (2008159 27.0 (2.89) Active 
Solomon and colleagues (2013) 89 23.0 (1.23) Draft picks 
Study n Agea Player status 
Amen and colleagues (2011) 100 57.27 (12.37) Active and Retired 
Casson and colleagues (2014) 45 45.6 (8.9) Retired 
Didehbani and colleagues (2013) 30 58.60 (10.33) Retired 
Ford and colleagues (2013), low concussionb 15 64.1 (6.8) Retired 
Ford and colleagues (2013), high concussionc 12 62.6 (5.0) Retired 
Hart and colleagues (2013), cognitively unimpaired 12 55.4 (95% CI = 47.2–63.4) Retired 
Hart and colleagues (2013), cognitively impaired 10 66.6 (95% CI = 61.6–71.6) Retired 
Kutner and colleagues (2000) 53 27.1 (2.65) Active 
Lovell and Solomon (2011)d 95–366 NRe Active 
Lovell and Solomon, (2011)f 503 NR Active 
Pellman and colleagues (2004) 155–655 25.4 (NR) Active 
Pellman and colleagues (2006) 68 24.3 (NR) Active 
Randolph and colleagues (2013) 41 64.2 (5.5) Retired 
Seichepine and colleagues (2013) 29 48.3 (13.0) Retired 
Solomon and Haase (2008159 27.0 (2.89) Active 
Solomon and colleagues (2013) 89 23.0 (1.23) Draft picks 

Notes:aYears: mean (SD).

bLow concussion group = 2 or fewer concussions.

cHigh concussion group = 3 or more concussions.

dCross-validation NFL group for paper and pencil tests.

eNR = not reported.

fFactor analytic sample of NFL players completing all paper and pencil tests.

Paper and Pencil Tests

Depression

Three studies were found, including two with the BDI-II and one with the GDS (Yesavage et al., 1983).

Beck Depression Inventory-II: Table 2 presents the studies with BDI data. Didehbani, Cullum, Mansinghani, Conover, and Hart (2013), using a three-factor model analysis (Buckley, Parker, & Heggie, 2001) of BDI-II results, found mean (SD) subscale scores of 2.62 (SD = 3.68) for Cognitive, 1.59 (SD = 2.21) for Affective, and 4.38 (SD = 3.68) for Somatic.

Table 2.

Beck depression inventory-II scores for retired NFL players

Study n and player status Age: mean years (SDBDI: mean (SD
Didehbani and colleagues (2013) 30 retirees 58.6 (10.33) 8.80 (8.83) 
Casson and colleagues (2014) 45 retirees 45.6 (8.9) 9.7 (9.6) 
Study n and player status Age: mean years (SDBDI: mean (SD
Didehbani and colleagues (2013) 30 retirees 58.6 (10.33) 8.80 (8.83) 
Casson and colleagues (2014) 45 retirees 45.6 (8.9) 9.7 (9.6) 

Geriatric Depression Scale: On the 30-item form of the GDS, Ford and colleagues (2013) found a mean score of 5.5 (SD = 4.6) in the 15 member group with two-or-less concussions (mean age 64.1; SD = 6.8 years), and a mean score of 8.8 (SD = 7.2) in the 12 member three-or-more concussion group (mean age = 62.5; SD = 5.0 years).

Brief cognitive screening examinations

Two studies were found.

Telephone Interview for Cognitive Status: Ford and colleagues (2013) found a mean score of 34.7 (SD = 4.8) in the 15 member two-or-less concussion group (mean age 64.1; SD = 6.8 years), and a mean score of 35.2 (SD = 3.9) in the 12 member three-or-more concussion group (mean age = 62.5; SD = 5.0 years).

Mini-Mental State Examination: Casson and colleagues (2014), in their study of 45 retired NFL players (mean age 45.6 years, SD = 8.9), reported a mean MMSE score of 28.5 (SD = 1.3).

Wechsler scales, subtests, and premorbid predictors (WAIS/WTAR)

Two studies reported estimated IQs based on abbreviated batteries, and three studies reporting data on selected subtests, including Digit Span, Digit Symbol, and Letter Number Sequencing. Digit Span (WAIS-R and WAIS-III) data were reported by Pellman and colleagues (2004), while Hart et al. (2013) administered the WAIS-IV version of this subtest. One study utilized the WAIS-III (Wechsler, 1997) Digit Symbol and Letter Number Sequencing subtests (Casson et al., 2014). Another study reported data on the SDMT (Smith, 1982), and is included here due to its conceptual similarity to the Wechsler Digit Symbol/Coding subtests. Two studies using the WTAR (Wechsler, 2001) were found.

IQ Scores:Randolph and colleagues (2013) studied 41 NFL retired players (mean age = 64.2, SD = 5.5 years) with an unspecified short form of the WAIS-III (Donnell, Pliskin, Holdnack, Axelrod, & Randolph, 2007) and found a mean IQ of 110.8 (SD = 15.6). Ford and colleagues (2013) utilized a four subtest short form of the WAIS-III (Arithmetic, Similarities, Digit Symbol, and Picture Completion). They reported a mean IQ of 114.0 (SD = 12.9) for the 15 member two-or-fewer concussion group (mean age = 64.1; SD = 6.8 years), and a mean IQ of 115.6 (SD = 19.5) for the 12 members of the three-or-more concussion group (mean age 62.6, SD = 5.0 years). The results are displayed in Table 3.

Table 3.

Estimated Wechsler IQ scores

Study n Player status Subtests Mean (SD
Randolph and colleagues (2013) 41 Retirees Not specified 110.8 (15.6) 
Ford and colleagues (2013) 15 with 2 or fewer concussions
12 with 3 or more concussions 
Retirees Arithmetic, similarities, digit symbol, and picture completion 114.0 (12.9)
115.6 (19.5) 
Study n Player status Subtests Mean (SD
Randolph and colleagues (2013) 41 Retirees Not specified 110.8 (15.6) 
Ford and colleagues (2013) 15 with 2 or fewer concussions
12 with 3 or more concussions 
Retirees Arithmetic, similarities, digit symbol, and picture completion 114.0 (12.9)
115.6 (19.5) 

Digit Span subtest:Pellman and colleagues (2004) reported on a sample of 646 active NFL players who underwent neuropsychological testing between the 1996 and 2001 seasons. Mean age of the cohort was 25.4 years, with a range of 20–44. The mean total raw score on the WAIS-R or WAIS-III Digit Span subtests of 14.9 (SD = 4.0). Hart et al. (2013) administered the WAIS-IV (Wechsler, 2008) Digit Span subtest to 27 NFL retirees. It was determined by these investigators that 12 players were cognitively unimpaired; their mean age was 55.4 (range = 47.2–63.4 years) and their average standard score (mean = 10, SD = 3) on Digit Span was 9.3 (95% CI = 7.2–11.5). Among the 15 NFL retirees found to be cognitively impaired (mean age = 66.6; range 61.6–71.6 years), the mean standard score was 10.3 (95% CI = 8.4–12.2).

Digit Symbol subtest:Casson and colleagues (2014) reported a mean raw score of 66.48 (SD = 15.10) on the WAIS-III version of this subtest among 45 retirees (mean age = 45.60, SD = 8.9).

Symbol Digit Modalities Test: Pellman and colleagues (2004) reported a mean SDMT raw score of 56.6 (SD = 8.4) among 155 active NFL players (mean age of 25.4 years).

Letter Number Sequencing subtest:Casson and colleagues (2014) reported a mean raw score of 10.02 (SD = 2.20) on the WAIS-III version among 45 retirees (mean age = 45.60, SD = 8.9).

Wechsler Test of Adult Reading: Ford and colleagues (2013) reported a mean WTAR estimated IQ score of 102.8 (SD = 10.3) in the 15 member retiree group with two-or-fewer concussions (mean age = 64.1 years) and a mean estimated IQ score of 94.8 (SD = 13.3) in the 12 member retiree group with three-or-more concussions (mean age = 62.6 years). Casson and colleagues (2014) administered the WTAR to 45 retirees (mean age = 45.60 years, SD = 8.9) and reported a mean raw score of 32.2 (10.1). For a 45-year-old individual, this raw score translates to a WTAR standard score equivalent of ∼99 (Wechsler, 2001). The results are summarized in Table 4.

Table 4.

WTAR scores

Study n Player status Mean (SD
Ford and colleagues (2013) 15 with ≤2 concussions
12 with ≥3 concussions 
Retirees 102.8 (10.3)a
94.8 (13.3)a 
Casson and colleagues (2014) 45 Retirees 32.2 (10.1)b 
Study n Player status Mean (SD
Ford and colleagues (2013) 15 with ≤2 concussions
12 with ≥3 concussions 
Retirees 102.8 (10.3)a
94.8 (13.3)a 
Casson and colleagues (2014) 45 Retirees 32.2 (10.1)b 

Notes:aStandard scores.

bRaw scores.

Neurocognitive batteries

One study was found.

Repeatable Battery for the Assessment of Neuropsychological Status (RBANS; Randolph, Tierney, Mohr, & Chase, 1998): Randolph and colleagues (2013) reported on 41 NFL retirees with a mean age of 64.2 (SD = 5.5) who completed the RBANS. Means (SD) on the indexes included Immediate Memory = 89.4 (14.5), Visuospatial/Constructional = 94.9 (12.5), Language = 96.8 (9.4), Attention = 103.0 (14.9), Delayed Memory = 87.6 (17.5), and Total Scale Score = 92.0 (12.0).

Self-report executive functions

One study was found.

Behavior Rating Inventory of Executive Function-Adult Version (BRIEF-A; Roth, Isquith, & Gioia, 2005): Seichepine and colleagues (2013) administered the BRIEF-A to 29 professional and 35 collegiate football players. The authors provided age-adjusted t-scores for the professional football players (mean age = 48.3, SD = 13.0 years) and the 35 collegiate football players (mean age = 45.9, SD = 13.1 years) as a group. The overall mean age of the sample was 47.0 (SD = 13.6) years. Unfortunately, the study did not report BRIEF-A results specific to the professional players, but did report findings on the total football sample compared with control subjects, for which multiple significant differences were found (e.g., on all 3 BRIEF-A index scores as well as on 5 of the 9 clinical scales). The study also reported significant group differences between younger (less than age 40 years) and older football players. Although a useful study, the data reported have limited utility for the purpose of this review since data specific to NFL players was not reported.

Verbal learning and memory

There were two studies found using the HVLT-R, two studies using the CVLT-II (Delis et al., 2000), and one study using the SORT (Kraut et al., 2006).

Hopkins Verbal Learning Test-Revised: Means and SDs for IR (immediate recall, representing the total number of words learned across trials 1–3) and DR (delayed recall, representing the total number of words recalled after a 20 min delay) for active NFL players were reported by Pellman and colleagues (2004) and Lovell and Solomon (2011), and are summarized in Table 5. The scores from a subset of active NFL players from both cohorts (Pellman et al., 2004; Lovell and Solomon, 2011) who completed all of the paper and pencil tests used in the NFL test battery (n = 503) were selected for a factor analysis. The raw score mean performance across trials was 7.02 (SD = 1.51) for learning trial 1, 9.57 (SD = 1.46) for trial 2, and 10.21 (SD = 1.28) for trial 3. The raw score mean for delayed recall was 9.34 (SD = 2.14). Raw scores, t-scores, and percentiles for the combined paper and pencil samples on the HVLT-R were reported by Lovell and Solomon (2011), and are summarized in Tables 6 and 7.

Table 5.

HVLT-R raw score means (SDs) for active NFL playersa

Study N IRb DRb 
Pellman and colleagues (2004) 653b/630c 26.0 (3.9) 9.0 (2.1) 
Lovell and Solomon (2011)d 366b/363c 26.4 (4.07) 9.1 (2.3) 
Lovell and Solomon (2011)e 503 Not reportedf 9.34 (2.13) 
Study N IRb DRb 
Pellman and colleagues (2004) 653b/630c 26.0 (3.9) 9.0 (2.1) 
Lovell and Solomon (2011)d 366b/363c 26.4 (4.07) 9.1 (2.3) 
Lovell and Solomon (2011)e 503 Not reportedf 9.34 (2.13) 

Notes:aApproximate mean age 25.4 years, range 20–44.

bIR = immediate recall: total words learned across learning trials 1–3.

cDR = delayed recall: total words learned after a 20-min delay.

dSecond NFL paper and pencil sample.

eNFL paper and pencil sample factor analytic study.

fScores by trials were reported but a total score was not included in the factor analysis.

Table 6.

Raw scores, percentiles, and t-scores for HVLT-R immediate recalla

Raw score Percentile t-score 
15 0.1 22 
16 0.4 25 
17 0.7 27 
18 1.2 30 
19 2.7 32 
20 5.1 35 
21 8.3 37 
22 13.2 40 
23 19.6 42 
24 27.0 45 
25 34.4 47 
26 42.6 50 
27 49.5 52 
28 60.9 55 
29 70.0 57 
30 77.7 60 
31 84.9 62 
32 90.4 65 
33 95.1 67 
34 98.3 70 
35 99.9 72 
36 100.0 75 
Raw score Percentile t-score 
15 0.1 22 
16 0.4 25 
17 0.7 27 
18 1.2 30 
19 2.7 32 
20 5.1 35 
21 8.3 37 
22 13.2 40 
23 19.6 42 
24 27.0 45 
25 34.4 47 
26 42.6 50 
27 49.5 52 
28 60.9 55 
29 70.0 57 
30 77.7 60 
31 84.9 62 
32 90.4 65 
33 95.1 67 
34 98.3 70 
35 99.9 72 
36 100.0 75 

Note:an = 964 active NFL players.

Table 7.

Raw scores, percentiles, and t-scores for HVLT-R delayed recalla

Raw score Percentile t-score 
0.3 18 
0.8 23 
1.4 27 
2.9 32 
6.1 36 
14.3 31 
24.0 45 
35.7 50 
10 52.5 54 
11 70.4 59 
12 86.0 63 
Raw score Percentile t-score 
0.3 18 
0.8 23 
1.4 27 
2.9 32 
6.1 36 
14.3 31 
24.0 45 
35.7 50 
10 52.5 54 
11 70.4 59 
12 86.0 63 

Note:an = 942 active NFL players.

California Verbal Learning Test-II: Hart et al. (2013) provided CVLT-II results for 12 cognitively unimpaired and 10 cognitively impaired NFL retirees. CVLT-II mean age and education-corrected t-scores were 52.1 (95% CI = 46.9–57.3) for the unimpaired group and 39.6 (95% CI = 32.5–46.7) for the impaired group. Casson and colleagues (2014) administered the CVLT-II to their sample of 45 retirees (mean age = 42.8, SD = 9.5 years). Mean t-scores for sum of trials was 38.9 (SD = 8.97). Mean delayed short-term free recall was t = 39.47 (10.42), and long-term free recall was t = 37.98 (SD = 8.36).

Semantic Object Retrieval Test: Hart et al. (2013) reported a mean total raw score of 29.6 (95% CI = 28.9–30.0) for the unimpaired retirees and a mean raw score of 27.9 (95% CI = 26.6–29.2) for impaired retirees.

Visual learning and memory

Four studies were found, three using the BVMT-R and one using the ROCFT (Rey, 1941).

Brief Visuospatial Memory Test-Revised: Three studies were found. Means and SDs for IR (immediate recall, representing the total number of points earned across trials 1–3) and DR (delayed recall, representing the total number of points recalled after a 25 min delay) for active NFL players were reported by Pellman and colleagues (2004) and Lovell and Solomon (2011), and are summarized in Table 8. For the 503 active players with complete paper and pencil data used in the factor analytic study (Lovell & Solomon, 2011), the mean delayed recall score was 10.44 (SD = 1.96). Across learning trials one through three, the means (SDs) for the factor analytic sample were 6.95 (2.29), 9.62 (2.29), and 10.74 (1.66). Raw scores, t-scores, and percentiles for the combined paper and pencil samples were reported by Lovell and Solomon (2011), and are summarized in Tables 9 and 10. Casson and colleagues (2014) administered the BVMT-R to 45 retirees, and the results are presented in Table 8.

Table 8.

BVMT-R raw score means (SDs) for NFL players

Study n and player status IRa DRb 
Pellman and colleagues (2004)c 479a/478b active 27.2 (5.7) 10.4 (2.04) 
Lovell and Solomon (2011)d 96a/95b active 27.9 (4.37) 10.9 (1.6) 
Lovell and Solomon (2011)c,e 503 active Not reportedf 10.44 (1.96) 
Casson and colleagues (2014) 45 retirees 21.3 (6.4) 9.0 (2.52) 
Study n and player status IRa DRb 
Pellman and colleagues (2004)c 479a/478b active 27.2 (5.7) 10.4 (2.04) 
Lovell and Solomon (2011)d 96a/95b active 27.9 (4.37) 10.9 (1.6) 
Lovell and Solomon (2011)c,e 503 active Not reportedf 10.44 (1.96) 
Casson and colleagues (2014) 45 retirees 21.3 (6.4) 9.0 (2.52) 

Notes:aIR = immediate recall: total points across learning trials 1–3.

bDR = delayed recall: total points after a 25-min delay.

cApproximate mean age 25.4 years, range 20–44.

dSecond NFL paper and pencil sample.

eNFL paper and pencil sample factor analytic study.

fScores by trials were reported but a total score was not included in the factor analysis.

Table 9.

Raw scores, percentiles, and t-scores for BVMT-R immediate recalla

Raw score Percentile t-score 
0.2 17 
10 0.6 18 
11 0.8 20 
13 2.3 24 
15 2.5 28 
16 2.9 29 
17 3.3 31 
18 4.6 33 
19 7.3 35 
20 8.1 37 
21 11.2 38 
22 16.7 40 
23 20.0 42 
24 23.5 44 
25 26.3 46 
26 31.0 48 
27 36.9 49 
28 44.6 51 
29 50.0 53 
30 56.3 55 
31 63.5 57 
32 78.8 58 
33 85.8 60 
34 90.0 62 
35 94.8 64 
36 98.7 66 
Raw score Percentile t-score 
0.2 17 
10 0.6 18 
11 0.8 20 
13 2.3 24 
15 2.5 28 
16 2.9 29 
17 3.3 31 
18 4.6 33 
19 7.3 35 
20 8.1 37 
21 11.2 38 
22 16.7 40 
23 20.0 42 
24 23.5 44 
25 26.3 46 
26 31.0 48 
27 36.9 49 
28 44.6 51 
29 50.0 53 
30 56.3 55 
31 63.5 57 
32 78.8 58 
33 85.8 60 
34 90.0 62 
35 94.8 64 
36 98.7 66 

Note:an = 520 active NFL players.

Table 10.

Raw scores, percentiles, and t-scores for BVMT-R delayed recalla

Raw score Percentile t-score 
<1 17 
2.1 22 
3.3 27 
5.0 32 
10.0 37 
15.8 43 
10 24.3 48 
11 34.7 53 
12 58.9 58 
Raw score Percentile t-score 
<1 17 
2.1 22 
3.3 27 
5.0 32 
10.0 37 
15.8 43 
10 24.3 48 
11 34.7 53 
12 58.9 58 

Note:an = 518 active NFL players.

Rey–Osterrieth Complex Figure Test: Hart et al. (2013) published results on the ROCFT for 12 cognitively unimpaired and 10 cognitively impaired NFL retirees. For the copy score, mean age and education-corrected t-scores were 46.3 (95% CI = 38.9–53.8) for the unimpaired group and 48.7 (95% CI = 42.2–55.1) for the impaired group. Delayed recall age- and education-corrected t-scores were 56.6 (95% CI = 48.1–65.0) for the unimpaired group and 35.5 (95% CI 28.5–42.5) for the impaired group.

Attention/Concentration/Executive Functioning

Trail Making Test-Part A (Reitan, 1958)

Five studies were found; the results are summarized in Table 11.

Table 11.

Trail Making Test Part A scores

Study n and player status Age: mean (SDTMT-A: mean (SD
Pellman and colleagues (2004) 595 active 25.4 (range 20–44) 21.4 (7.41)a 
Lovell and Solomon (2011)b 96 active ∼25.4 (range 20–44) 20.5 (7.08)a 
Lovell and Solomon (2011)c 503 active ∼25.4 (range 20–44) 21.28 (7.72)a 
Hart and colleagues (2013) 12 cognitively unimpaired retirees 55.4 (47.2–63.4) 50.2 (44.2–56.2)d 
Hart and colleagues (2013) 10 cognitively impaired retirees 66.6 (61.6–71.6) 52.0 (47.8–56.2)d 
Casson and colleagues (2014) 45 retirees 45.60 (8.9) 27.9 (11.4)a 
Study n and player status Age: mean (SDTMT-A: mean (SD
Pellman and colleagues (2004) 595 active 25.4 (range 20–44) 21.4 (7.41)a 
Lovell and Solomon (2011)b 96 active ∼25.4 (range 20–44) 20.5 (7.08)a 
Lovell and Solomon (2011)c 503 active ∼25.4 (range 20–44) 21.28 (7.72)a 
Hart and colleagues (2013) 12 cognitively unimpaired retirees 55.4 (47.2–63.4) 50.2 (44.2–56.2)d 
Hart and colleagues (2013) 10 cognitively impaired retirees 66.6 (61.6–71.6) 52.0 (47.8–56.2)d 
Casson and colleagues (2014) 45 retirees 45.60 (8.9) 27.9 (11.4)a 

Notes:aSeconds to completion.

bResults from the second NFL paper and pencil battery sample.

cResults from the sample used for NFL factor analysis.

dAge- and education-corrected t-scores.

Trail Making Test-Part B (Reitan, 1958)

Six studies were found and are summarized in Table 12.

Table 12.

Trail Making Test Part B scores

Study n and player status Age: mean (SDTMT-B: mean (SD
Pellman and colleagues (2004) 654 active 25.4 (range 20–44) 55.5 (16.8)a 
Lovell and Solomon (2011)b 365 active ∼25.4 (range 20–44) 60.2 (22.1)a 
Lovell and Solomon (2011)c 503 active ∼25.4 (range 20–44) 54.73 (16.79)a 
Hart and colleagues (2013) 12 cognitively unimpaired retirees 55.4 (47.2–63.4) 51.9 (95% CI = 44.9–58.9)d 
Hart and colleagues (2013) 10 cognitively impaired retirees 46.8 (40.5–53.1) 46.8 (95% CI = 40.5–53.1)d 
Ford and colleagues (2013)d 15 retirees with 2 or less concussions 64.1 (6.8) 74.0 (15.9)a 
Ford and colleagues (2013)d 12 retirees with 3 or more concussions 62.6 (5.0) 87.0 (34.1)a 
Casson and colleagues (2014) 44 retireese 45.6 (8.9) 80.7 (35.3) 
Study n and player status Age: mean (SDTMT-B: mean (SD
Pellman and colleagues (2004) 654 active 25.4 (range 20–44) 55.5 (16.8)a 
Lovell and Solomon (2011)b 365 active ∼25.4 (range 20–44) 60.2 (22.1)a 
Lovell and Solomon (2011)c 503 active ∼25.4 (range 20–44) 54.73 (16.79)a 
Hart and colleagues (2013) 12 cognitively unimpaired retirees 55.4 (47.2–63.4) 51.9 (95% CI = 44.9–58.9)d 
Hart and colleagues (2013) 10 cognitively impaired retirees 46.8 (40.5–53.1) 46.8 (95% CI = 40.5–53.1)d 
Ford and colleagues (2013)d 15 retirees with 2 or less concussions 64.1 (6.8) 74.0 (15.9)a 
Ford and colleagues (2013)d 12 retirees with 3 or more concussions 62.6 (5.0) 87.0 (34.1)a 
Casson and colleagues (2014) 44 retireese 45.6 (8.9) 80.7 (35.3) 

Notes:aSeconds to completion.

bResults from the second NFL paper and pencil battery sample; scores are in seconds to completion.

cResults from the NFL sample used for factor analysis; scores are in seconds to completion.

dAge- and education-corrected t-scores.

eOne subject in this study had missing data.

Language

Category Fluency

Two studies were found using a category fluency measure. Casson and colleagues (2014) studied animal naming among 45 retirees, and reported a mean score of 21.48 (SD = 4.70). Hart et al. (2013) reported mean age- and education-corrected t-scores on the Category Fluency subtest of the Boston Diagnostic Aphasia Examination (Goodglass, Kaplan, & Baressi, 2001). Mean scores for the 12 cognitively unimpaired retires was 49.5 (95% CI = 42.5–56.5), and 42.3 (95% CI = 37.0–47.6) for the 10 cognitively impaired retirees Table 13.

Table 13.

Boston Naming Test scores for NFL retirees

Study N Years: mean (SDBNT: mean (SD
Ford and colleagues (2013) 15 retirees with 2 or less concussions 64.1 (6.8) 51.7 (4.6)a 
Ford and colleagues (2013) 12 retirees with 3 or more concussions 62.6 (5.0) 52.5 (6.1)a 
Hart and colleagues (2013) 12 cognitively unimpaired retirees 55.4 (47.2–63.4) 48.6 (95% CI = 42.3–54.9)b 
Hart and colleagues (2013) 10 cognitively impaired retirees 66.6 (61.6–71.6) 36.2 (95% CI = 29.8–42.6)b 
Study N Years: mean (SDBNT: mean (SD
Ford and colleagues (2013) 15 retirees with 2 or less concussions 64.1 (6.8) 51.7 (4.6)a 
Ford and colleagues (2013) 12 retirees with 3 or more concussions 62.6 (5.0) 52.5 (6.1)a 
Hart and colleagues (2013) 12 cognitively unimpaired retirees 55.4 (47.2–63.4) 48.6 (95% CI = 42.3–54.9)b 
Hart and colleagues (2013) 10 cognitively impaired retirees 66.6 (61.6–71.6) 36.2 (95% CI = 29.8–42.6)b 

Notes:aRaw number correct.

bAge- and education-corrected t-scores.

Controlled Oral Word Association Test: Five studies were found and are summarized in Table 14.

Table 14.

Controlled oral word association test scores

Study n and player status Age: mean (SDCOWAT: mean (SD)a 
Pellman and colleagues (2006) 655 active 25.4 (range 20–44) 41.9 (11.4) 
Lovell and Solomon (2011)b 362 active ∼25.4 (range 20–44) 39.9 (10.7) 
Lovell and Solomon (2011)c 503 active ∼25.4 (range 20–44) 41.61 (11.66) 
Ford and colleagues (2013) 15 retirees with 2-or-fewer concussions 64.1 (6.8) 43.9 (8.7) 
Ford and colleagues (2013) 12 retirees with 3 or more concussions 62.6 (5.0) 40.1 (10.4) 
Casson and colleagues (2014) 45 retirees 45.6 (8.9) 41.6 (10.9) 
Study n and player status Age: mean (SDCOWAT: mean (SD)a 
Pellman and colleagues (2006) 655 active 25.4 (range 20–44) 41.9 (11.4) 
Lovell and Solomon (2011)b 362 active ∼25.4 (range 20–44) 39.9 (10.7) 
Lovell and Solomon (2011)c 503 active ∼25.4 (range 20–44) 41.61 (11.66) 
Ford and colleagues (2013) 15 retirees with 2-or-fewer concussions 64.1 (6.8) 43.9 (8.7) 
Ford and colleagues (2013) 12 retirees with 3 or more concussions 62.6 (5.0) 40.1 (10.4) 
Casson and colleagues (2014) 45 retirees 45.6 (8.9) 41.6 (10.9) 

Notes:aTotal number of words generated across three trials.

bResults from the second NFL paper and pencil battery sample.

cResults from the NFL sample used for factor analysis.

Computerized neurocognitive tests and batteries

One study was found using Conners' CPT-II (Conners & Staff, 2000), two studies used MicroCog (Powell et al., 1993), and four studies reported ImPACT (Lovell et al., 2000) data.

Connors Continuous Performance Test: Amen and colleagues (2011) reported that 81 of 97 NFL players/retirees, average age 57.27 (SD = 12.37) years, had “50% greater chance of having ADHD based on CCPT II.” Further data were not reported on this measure.

MicroCog: Kutner, Erlanger, Tsai, Jordan, & Relkin (2000), in their study assessing ApoE ε4 in a cohort of NFL players, utilized MicroCog, and reported a total mean score of −0.02 (SD = 1.0) for the 53 roster players on the General Cognitive Functioning Summary Index. Other means and SDs for the group were not provided, although regression plots of the cognitive scores, by genotype status, was reported. Amen and colleagues (2011) administered MicroCog to 97 active or retired NFL players (age range 25–82). The results are presented in Table 15.

Table 15.

Microcog percentile scores means and SDsa

Domain Mean SD 
General cognitive functioning 33.3 25.5 
General cognitive proficiency 27.6 23.2 
Processing speed 33.3 26.0 
Processing accuracy 41.2 26.8 
Attention 40.8 26.3 
Reasoning 34.4 27.5 
Memory 36.5 28.7 
Spatial processing 68.0 21.9 
Reaction time 70.2 24.5 
Domain Mean SD 
General cognitive functioning 33.3 25.5 
General cognitive proficiency 27.6 23.2 
Processing speed 33.3 26.0 
Processing accuracy 41.2 26.8 
Attention 40.8 26.3 
Reasoning 34.4 27.5 
Memory 36.5 28.7 
Spatial processing 68.0 21.9 
Reaction time 70.2 24.5 

Note:an = 97 NFL retirees, age range 25–82 (mean = 57.27).

Immediate Post-Concussion Assessment and Cognitive Testing: Four studies were found with normative ImPACT data; one sample of draft picks, two samples of active players, and one with retirees. Group means are summarized in Table 16. When comparing the 45 retirees from the Casson and colleagues (2014) study with the equivalent age group from the ImPACT normative data base, the corresponding mean (SD) percentile scores were 36.62 (28.62) for Verbal Memory, 27.76 (23.98) for Visual Memory, 26.64 (25.40) for Visual Motor Speed, and 29.24 (24.93) for Reaction Time.

Table 16.

ImPACT raw score means (SDs) for NFL draft picks, active players, and retirees

Study n and player status Age: mean (SDVerbal memory Visual memory Visual motor speed Reaction time Total symptom score 
Pellman and colleagues (2006) 68 active 24.30 (range 20–34) 85.7 (9.1) 77.3 (13.2) 35.5 (5.6) 0.58 (0.08) 1.34 (2.73) 
Solomon and Haase 2008 159 active 23.0 (1.23) 85.33 (10.02) 78.40 (12.89) 34.08 (6.06) 0.59 (0.09) 3.19 (4.96) 
Solomon and colleagues (2013) 89 draft picks 27.0 (2.89) 80.97 (0.14) 78.49 (10.65) 37.51 (7.09) 0.58 (0.08) 2.65 (3.82) 
Casson and colleagues (2014) 45 retirees 45.60 (8.90) 75.90 (12.0) 59.0 (13.6) 28.60 (8.3) 0.77 (0.19) 20.0 (21.3) 
Study n and player status Age: mean (SDVerbal memory Visual memory Visual motor speed Reaction time Total symptom score 
Pellman and colleagues (2006) 68 active 24.30 (range 20–34) 85.7 (9.1) 77.3 (13.2) 35.5 (5.6) 0.58 (0.08) 1.34 (2.73) 
Solomon and Haase 2008 159 active 23.0 (1.23) 85.33 (10.02) 78.40 (12.89) 34.08 (6.06) 0.59 (0.09) 3.19 (4.96) 
Solomon and colleagues (2013) 89 draft picks 27.0 (2.89) 80.97 (0.14) 78.49 (10.65) 37.51 (7.09) 0.58 (0.08) 2.65 (3.82) 
Casson and colleagues (2014) 45 retirees 45.60 (8.90) 75.90 (12.0) 59.0 (13.6) 28.60 (8.3) 0.77 (0.19) 20.0 (21.3) 

Discussion

Thirteen studies were found with quantitative data regarding NFL draft picks, active players, and retirees. By far, the largest cohorts of active players came from the NFL paper and pencil battery study performed from 1996 to 2001. Studies of retired NFL players are now emerging. Only one study has assessed draft picks.

The question of appropriate norms for interpretation of neuropsychological test scores has been discussed by Hannay and Lezak (2004). In short, these authors state that “finding appropriate norms for each patient is still a challenge for clinicians” (p. 147). Further, Hannay and Lezak discuss the importance of demographic variables such as age and education in the interpretation of neuropsychological test scores. It is a goal of this paper to provide an initial compendium of normative neurocognitive data for this group of professional athletes (who are similar in age and education) for future comparative purposes.

We summarize the findings of our review by athlete group.

Retirees

The review of published studies employing the BDI does not suggest that these samples were reporting clinically relevant symptoms of depression. In fact, these data are somewhat at odds with the reported high rates of depression in NFL retirees found in survey studies (e.g., Kerr, Marshall, Harding, & Guskiewicz, 2012). This difference may be due to the inherent limitations of self-report survey information in the absence of a more comprehensive psychological assessment.

With regard to performance of retired players on a standardized test of visual memory, the retirees score ∼0.5 SDs lower than general population normative values on BVMT-R immediate (but not delayed) recall. Although slightly lower than the general population, this may be due to multiple factors and not necessarily secondary to concussion.

With regards to pooled data on a test of visuomotor speed (Trails A; 67 retirees) and executive functioning (Trails B; 97 retirees), the mean level of performance across the age ranges tends to mirror general population norms. The results of the COWAT scores among 72 retirees yielded scores very similar to the general population norms.

On a computer-based neurocognitive test battery (ImPACT), retirees performed at a lower level than the active players, consistent with an age effect. When the retirees were compared with an appropriate age comparison group from the ImPACT normative sample for age, their scores were in the average to low average range.

Active Players

Normative data exists on ∼1000 players on the BVMT-R. Generally speaking, active players (average age 25.4 years) score ∼27.5 on immediate recall (approximately SD of 5) and about 10.5 on delayed recall (approximately SD of 1.75). For the 24–28 year age group (n = 47), the BVMT-R Manual (Benedict, 1997) reports a mean immediate recall score of 27.83 (SD = 4.48) and a mean delayed recall score of 10.46 (SD = 1.48). Therefore, NFL players' performances on the BVMT-R are similar to those reported in the test manual.

Normative data also exist on ∼1000 players on the HVLT-R. Generally speaking, active NFL players' mean scores are slightly lower on the HVLT-R immediate and delayed recall indexes versus general population norms (Brandt & Benedict, 2001).

Performance on the COWAT also yielded consistent scores across studies, with an approximate mean score of 41 (SD =∼11) noted in over 1000 players. Once again, these results are not significantly discrepant from the general population.

Pooled normative ImPACT data are available for about 227 active NFL players (Pellman et al., 2004; Solomon & Haase, 2008). Normative data for 89 draft picks also is available (Solomon & Kuhn, 2014). Mean scores across the active player and draft pick groups are quite consistent, suggesting similar test results for active players and for athletes on the verge of entering the NFL.

Recommendations and future considerations

In reviewing the published studies on NFL athletes, it is important to acknowledge some of the challenges in studying this select group of athletes. First of all, when the initial paper and pencil data were collected in the 1990s and the first part of the 21st century, no systematic neuropsychological test protocol existed throughout the NFL, participation in the assessments was not mandatory, and data were contributed by teams and team neuropsychologists on a voluntary basis. This resulted in some differences in tests administered, subtests utilized, and in the documentation of relevant on-field concussion parameters (e.g., LOC and amnesia). It is our understanding that the current mandatory, league-wide protocol has alleviated these issues.

This initial compendium of normative data for NFL players suggests that for active players, a large data set exists for the following paper and pencil tests: HVLT-R, BVMT-R, Trails A&B, and COWAT (Pellman et al., 2004; Lovell & Solomon, 2011). Consideration should be given to the inclusion of these tests in any paper and pencil battery utilized with this population. There is also a large data base of ImPACT scores for active players (Solomon & Haase, 2008) and for draft picks (Solomon & Kuhn, 2014), which could provide good normative values for active players, and its inclusion also may be warranted. Therefore, a “hybrid” model may be a valuable approach to assessment.

There is less clarity for a choice of neurocognitive tests for retired NFL players. The largest study to date was performed by Amen and colleagues (2011), but that study included a mixture of active and retired NFL players, and MicroCog was the only neurocognitive test administered. Casson and colleagues (2014) and Randolph and colleagues (2013) provided ImPACT and RBANS data on 45 and 41 retirees, respectively, which are the largest reported cohorts to date. The paper and pencil battery used in the Casson and colleagues study was more comprehensive, was vetted by a five person panel of members from the National Academy of Neuropsychology, and should be viewed as a model battery worth emulating in the future. To date, the literature indicates that there are <100 clearly identified retirees with reported normative data on paper and pencil or computerized neuropsychological tests.

It would be ideal going forward to identify a core battery of paper and pencil and/or computerized neuropsychological tests for the assessment of active and retired NFL players. We hope that the results of this survey study will offer some useful recommendations for the establishment of a battery of this type. We also specifically recommend the reporting of raw score means and SDs on all neuropsychological tests administered in future studies with NFL athletes. Ideally, it would also be quite useful to include relevant demographic and biopsychosocial data including (at least): (a) concussion history, (b) presence of diagnosed attention deficit spectrum disorder and/or learning disability, (c) medical/psychiatric/substance abuse history, and (d) any available neuroimaging results or genotyping data.

Finally, it would also be very useful to have more published studies that document the inter-relationships between all of the types of tests administered if they are to be utilized in the evaluation of NFL athletes. Unfortunately, there is a current dearth of published information that investigates the relationship between computer based and paper and pencil assessment in this group. Therefore, there is a need for more empirical studies assessing the relationships of paper and pencil test results with computerized neurocognitive test data in NFL athletes.

Conflict of Interest

Dr. Lovell is Chairman and Chief Scientific Officer of ImPACT, a test mentioned in this manuscript. He also served as a member of the NFL's MTBI Committee from 1994–2009. Dr. Solomon receives consulting fees from the Tennessee Titans, the Nashville Predators, and the athletic departments of several universities. He is a member of the ImPACT Advisory Board, and receives reimbursement for expenses to board meetings. He receives royalties from book sales. Drs. Casson and Viano were members of the NFL MTBI Committee from 1994–2009 and served as co-chairs from 2007–2009. Dr. Casson has done consulting for law firms and insurance companies.

References

Amen
D. G.
Newberg
A.
Thatcher
R.
Jin
Y.
Wu
J.
Keator
D.
, et al.  . 
Impact of playing American professional football on long-term brain function
Journal of Neuropsychiatry and Clinical Neurosciences
 , 
2011
, vol. 
23
 
1
(pg. 
98
-
106
)
Beck
A. T.
Steer
R. A.
Brown
G. K.
Beck depression inventory-II
 , 
1996
San Antonio, TX
The Psychological Corporation
Benedict
R.
Brief visuospatial memory test-revised: Professional manual
 , 
1997
Odessa, FL
Psychological Assessment Resources
Benedict
R. H.
Schretlen
D.
Groninger
L.
Dobraski
M.
Shpritz
B.
Revision of the Brief Visuospatial Memory Test: Studies of normal performance, reliability, and validity
Psychological Assessment
 , 
1996
, vol. 
8
 
2
pg. 
145
 
Benton
A. L.
Hamsher
K.
Multilingual aphasia examination
 , 
1978
2nd
Iowa City, IA
AJA Associates
Brandt
J.
The Hopkins Verbal Learning Test: Development of a new memory test with six equivalent forms
The Clinical Neuropsychologist
 , 
1991
, vol. 
5
 
2
(pg. 
125
-
142
)
Brandt
J.
Benedict
R.
Hopkins verbal learning test-revised: Professional manual
 , 
2001
Odessa, FL
Psychological Assessment Resources
Brandt
J.
Spencer
M.
Folstein
M.
The telephone interview for cognitive status
Cognitive and Behavioral Neurology
 , 
1988
, vol. 
1
 
2
(pg. 
111
-
118
)
Buckley
T. C.
Parker
J. D.
Heggie
J.
A psychometric evaluation of the BDI-II in treatment-seeking substance abusers
Journal of Substance Abuse Treatment
 , 
2001
, vol. 
20
 
3
(pg. 
197
-
204
)
Casson
I. R.
Viano
D. C.
Haacke
E. M.
Kou
Z.
LeStrange
D. G.
Is there chronic brain damage in retired NFL players? Neuroradiology, neuropsychology, and neurology examinations of 45 retired players
Sports Health: A Multidisciplinary Approach
 , 
2014
, vol. 
6
 
5
(pg. 
384
-
395
)
Conners
C. K.
Staff
M. H. S.
Conners’ continuous performance test II (CPT II V. 5)
 , 
2000
North Tonawanda, NY
Multi-Health Systems Inc
Delis
D. C.
Kramer
J. H.
Kaplan
E.
Ober
B. A.
CVLT-II
 , 
2000
San Antonio, TX
The Psychological Corporation
Didehbani
N.
Cullum
C. M.
Mansinghani
S.
Conover
H.
Hart
J.
Depressive symptoms and concussions in aging retired NFL players
Archives of Clinical Neuropsychology
 , 
2013
, vol. 
28
 
5
(pg. 
418
-
424
)
Donnell
A. J.
Pliskin
N.
Holdnack
J.
Axelrod
B.
Randolph
C.
Rapidly-administered short forms of the Wechsler Adult Intelligence Scale—3rd edition
Archives of Clinical Neuropsychology
 , 
2007
, vol. 
22
 
8
(pg. 
917
-
924
)
Folstein
M. F.
Folstein
S. E.
McHugh
P. R.
“Mini-mental state:” A practical method for grading the cognitive state of patients for the clinician
Journal of Psychiatric Research
 , 
1975
, vol. 
12
 
3
(pg. 
189
-
198
)
Ford
J. H.
Giovanello
K. S.
Guskiewicz
K. M.
Episodic memory in former professional football players with a history of concussion: An event-related functional neuroimaging study
Journal of Neurotrauma
 , 
2013
, vol. 
30
 
20
(pg. 
1683
-
1701
)
Goodglass
H.
Kaplan
E.
Barresi
B.
Boston diagnostic aphasia examination
 , 
2001
3rd ed
Philadelphia
Lippincott Williams & Wilkins
Hannay
H. J.
Lezak
M. D.
Lezak
M. D.
Howieson
D. B.
Loring
D. W.
The neuropsychological examination: Interpretation
Neuropsychological assessment
 , 
2004
4th ed.
New York
Oxford University Press
Hart
J. H.
Kraut
M. A.
Womack
K. B.
Strain
J.
Didehbani
N.
Bartz
E.
, et al.  . 
Neuroimaging of cognitive dysfunction and depression in aging retired National Football League players: A cross sectional study
JAMA Neurology
 , 
2013
, vol. 
70
 
3
(pg. 
326
-
335
)
Kaplan
E.
Goodglass
H.
Weintraub
S.
The Boston naming test
 , 
1983
Philadelphia
Lea & Febiger
Kerr
Z. Y.
Marshall
S. W.
Harding
H. P.
Guskiewicz
K. M.
Nine year risk of depression diagnosis increases with increasing self-reported concussions in retired professional football players
American Journal of Sports Medicine
 , 
2012
, vol. 
40
 
10
(pg. 
2206
-
2212
)
Kraut
M. A.
Cherry
B.
Pitcock
J. A.
Vestal
L.
Henderson
V. W.
Hart
J.
Jr
The semantic object retrieval test (SORT) in normal aging and Alzheimer disease
Cognitive and Behavioral Neurology
 , 
2006
, vol. 
19
 
4
(pg. 
177
-
184
)
Kutner
K. C.
Erlanger
D. M.
Tsai
J.
Jordan
B.
Relkin
N. R.
Lower cognitive performance of older football players possessing apolipoprotein E ε4
Neurosurgery
 , 
2000
, vol. 
47
 
3
(pg. 
651
-
658
)
Lovell
M. R.
Collins
M. W.
Podell
K.
Powell
J.
Maroon
J.
ImPACT: Immediate post-concussion assessment and cognitive testing
 , 
2000
Pittsburgh, PA
NeuroHealth Systems, LLC
Lovell
M. R.
Solomon
G. S.
Psychometric data for the NFL neuropsychological test battery
Applied Neuropsychology
 , 
2011
, vol. 
18
 
3
(pg. 
197
-
209
)
Pellman
E. J.
Lovell
M. R.
Viano
D. C.
Casson
I. R.
Concussion in professional football: Recovery of NFL and high school athletes assessed by computerized neuropsychological testing—Part 12
Neurosurgery
 , 
2006
, vol. 
58
 
2
(pg. 
263
-
274
)
Pellman
E. J.
Lovell
M. R.
Viano
D. C.
Casson
I. R.
Tucker
A. M.
Concussion in professional football: Neuropsychological testing—Part 6
Neurosurgery
 , 
2004
, vol. 
55
 
6
(pg. 
1290
-
1305
)
Powell
D. H.
Kaplan
E. F.
Whitla
D.
Weinstraub
S.
Catlin
R.
Funkenstein
H. H.
Microcog assessment of cognitive functioning
 , 
1993
San Antonio, TX
The Psychological Corporation
Randolph
C.
Karantzoulis
S.
Guskiewicz
K.
Prevalence and characterization of mild cognitive impairment in retired National Football League players
Journal of the International Neuropsychological Society
 , 
2013
, vol. 
19
 
08
(pg. 
873
-
880
)
Randolph
C.
Tierney
M. C.
Mohr
E.
Chase
T. N.
The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS): Preliminary clinical validity
Journal of Clinical and Experimental Neuropsychology
 , 
1998
, vol. 
20
 
3
(pg. 
310
-
319
)
Reitan
R. M.
Validity of the Trail Making Test as an indicator of organic brain damage
Perceptual and Motor Skills
 , 
1958
, vol. 
8
 
3
(pg. 
271
-
276
)
Rey
A.
L'examen psychologique dans les cas d'encephalopathie traumatique
Archives de psychologie
 , 
1941
, vol. 
28
 (pg. 
286
-
340
)
Roth
R. M.
Isquith
P. K.
Gioia
G. A.
Behavior rating inventory of executive function–Adult version
 , 
2005
Lutz, FL: Psychological Assessment Resources
Seichepine
D. R.
Stamm
J. M.
Daneshvar
D. H.
Riley
D. O.
Baugh
C. M.
Gavett
B. E.
, et al.  . 
Profile of self-reported problems with executive functioning in college and professional football players
Journal of Neurotrauma
 , 
2013
, vol. 
30
 
14
(pg. 
1299
-
1304
)
Smith
A.
Symbol digit modality test manual
 , 
1982
Los Angeles
Western Psychological Services
Solomon
G. S.
Haase
R. F.
Biopsychosocial characteristics and neurocognitive test performance in National Football League players: An initial assessment
Archives of Clinical Neuropsychology
 , 
2008
, vol. 
23
 
5
(pg. 
563
-
577
)
Solomon
G. S.
Haase
R. F.
Kuhn
A.
The relationship among neurocognitive performances and biopsychosocial characteristics of National Football League draft picks: An exploratory investigation
Archives of Clinical Neuropsychology
 , 
2013
, vol. 
28
 (pg. 
9
-
20
)
Solomon
G. S.
Kuhn
A.
Relationship between concussion history and neurocognitive test performance in National Football League draft picks
American Journal of Sports Medicine
 , 
2014
, vol. 
42
 (pg. 
934
-
939
)
Wechsler
D.
WAIS-III
 , 
1997
San Antonio
Psychological Corporation
Wechsler
D.
Wechsler test of adult reading: WTAR
 , 
2001
San Antonio, TX
Psychological Corporation
Wechsler
D.
Wechsler adult intelligence scale-Fourth edition (WAIS-IV)
 , 
2008
San Antonio, TX
NCS Pearson
Yesavage
J. A.
Brink
T. L.
Rose
T. L.
Lum
O.
Huang
V.
Adey
M.
, et al.  . 
Development and validation of a geriatric depression screening scale: A preliminary report
Journal of Psychiatric Research
 , 
1983
, vol. 
17
 
1
(pg. 
37
-
49
)