Abstract

Subjective feedback about distractions or problems encountered during computerized assessment was provided by 538 out of a pool of 1659 high-school athletes who completed baseline testing using ImPACT (Immediate Post-Concussion Assessment and Cognitive Testing). Three types of feedback were included: (a) environmental, (b) computer-based (mechanical), and (c) instruction-based (associated with difficulty understanding test instructions). One-way analyses of variance were conducted and revealed relationships between greater symptom reporting and any type of feedback, environmental feedback, and instruction-based feedback. Increased symptom reporting was noted for female students. Additional relationships were noted between providing computer-based feedback and faster reaction time; and between history of concussion and providing instruction-based feedback. Athletes endorsing more symptoms at baseline scored significantly worse on ImPACT, as reflected in decreased visual memory performance. Results suggest that feedback provided during computerized assessment may yield information about symptom reporting and test-taking style, which may also be of particular interpretive utility when athletes minimize their symptoms.

Introduction

The use of computerized testing has become popularized among clinicians who measure intellectual performance and cognitive functioning (Cernich, Brennana, Barker, & Bleiberg, 2007). Early proponents of computerized testing advocated the greater sense of control and enhanced motivation on the part of the test-taker (Bartram & Bayliss, 1984). Salient, emergent properties of computerized assessment have included portability to multiple geographic sites and simultaneous testing of multiple individuals (Collie & Maruff, 2003; Schatz & Browndyke, 2002), thus making computerized neurocognitive assessment attractive for clinicians who test teams of athletes.

Segalowitz and colleagues (2007) noted the advantages of reduced cost and the standardization of procedures. However, others have described pitfalls associated with computerized assessment, cautioning that an emotional detachment might occur between the computer and the test-taker (Wilson & McMillan, 1991), possibly creating a depersonalizing experience (Space, 1981). Another drawback of computerized assessment is the potential to create crowded testing conditions, such as when baseline testing an entire team of athletes.

The literature documenting effects of distraction on test performance spans decades, with a consensus that tests should be administered in conditions free of distracting noises or other disruptions (Pierce, 1963). Authors of intelligence tests (Wechsler, 1955) have typically advocated that their tests be conducted in settings that are quiet and disciplined. Practically speaking, it is difficult to keep test settings completely free of distraction. As a result, some research has focused on measuring the influence of visual and noise distracters on test performance and attention. Auditory distraction has been shown to negatively affect the detection of grammatical errors (Weinstein, 1974). Visual distraction has been revealed to negatively affect response to a visual target (Chen & Cave, 2006). Perceptual distraction (e.g., background noise) has been linked to reduced cortical information processing and increased cerebral energy consumption during test-taking (Trimmel & Poelzl, 2006). Task-irrelevant auditory distraction appears to initiate pre-attentive “obligatory auditory processing” which, in turn, interferes with working memory (Macken, Phelps, & Jones, 2009). Similarly, task-irrelevant visual distraction has been linked to incorrect eye-gaze shifts and longer response latencies (Corneil & Munoz, 1996).

Despite the growth and widespread use of computer-based testing for the assessment and management of sports-related concussion, the effects of environmental distraction and user/computer interaction issues in this realm of assessment have not been systematically studied. The purpose of the present study is to evaluate the relationship between subjective feedback provided by test-takers in the areas of (a) environmental distraction, (b) mechanical problems with computers during testing, and (c) difficulty understanding instructions and cognitive test performance.

Materials and Methods

Participants

Participants were 1,818 high-school students attending a highly selective, academically competitive secondary boarding school in suburban New Jersey. All students were required to participate in athletics and completed preseason baseline cognitive assessment as a mandatory requirement between September 2001 and October 2007.

Students averaged 15.5 years of age (SD = 1.3), with a gender distribution of 59.2% men and 40.8% women. Approximately 18% of the sample reported experiencing at least one previous concussion, with 13% experiencing one previous concussion and 5% experiencing two or more previous concussions. Demographic data are presented in Table 1. All data were de-identified for retrospective analysis, and appropriate Institutional Review Board approval was received.

Table 1.

Demographics of the sample

 “Valid” Sample (N = 1659) 
Men 982 (59.2%) 
Women 677 (40.8%) 
Age 15.5 (1.3) 
Previous concussions 
 None 1,376 (82.4%) 
 Yes 292 (17.6%) 
Self-reported problems and services received 
 Reading 18 (1.1%) 
 Spelling 22 (1.3%) 
 Math 12 (0.7%) 
 Learning 16 (1.0%) 
 Hyperactive 88 (5.3%) 
 Special Ed. 12 (0.7%) 
 Speech Ther. 116 (7.0%) 
 Any of the above 236 (14.2%) 
 “Valid” Sample (N = 1659) 
Men 982 (59.2%) 
Women 677 (40.8%) 
Age 15.5 (1.3) 
Previous concussions 
 None 1,376 (82.4%) 
 Yes 292 (17.6%) 
Self-reported problems and services received 
 Reading 18 (1.1%) 
 Spelling 22 (1.3%) 
 Math 12 (0.7%) 
 Learning 16 (1.0%) 
 Hyperactive 88 (5.3%) 
 Special Ed. 12 (0.7%) 
 Speech Ther. 116 (7.0%) 
 Any of the above 236 (14.2%) 

Materials

Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT), a computerized neuropsychological test battery, was administered to all participants. ImPACT consists of six neuropsychological tests, each designed to target different aspects of cognitive functioning including attention, memory, processing speed, and reaction time. From these six tests, five separate composite scores are generated: Verbal memory, visual memory, visual-motor (processing) speed, reaction time, and impulse control (seeIverson, Gaetz, Lovell, & Collins, 2004; Lovell et al., 2003; Podell, 2004). The Post-Concussion Symptom Scale (Lovell & Collins, 1998) is included in the ImPACT test battery and is used to document and track symptoms associated with concussion (Lovell, 1999; Lovell & Burke, 2002). The symptom checklist asks the athlete to rate each symptom on a 7-point scale, with zero indicating no experience of a symptom and 6 indicating a severe symptom. This particular scale is useful because it presents “common” terms to describe symptoms and avoids jargon and less familiar medical terminology (e.g., sensitivity to light is used instead of photophobia). ImPACT software was upgraded over the 6 years of data collection, from versions 1.2 to 2.3, and then to 3.4. The visual memory composite score was not available in version 1.2, so the analyses of baseline assessment data obtained using this version did not include this composite score.

The ImPACT desktop software permits test-takers to provide feedback at the completion of testing. Specific response fields are presented for the purpose of providing feedback related to environmental distracters, problems with the computer, and/or difficulties with the test instructions.

Procedures

When completing baseline assessments, students interacted with the ImPACT battery in a set format and order: (a) provide demographic information, (b) provide information on “current status,” including current symptoms, (c) interact with the actual test stimuli, and (d) provide optional feedback at the end of the test session.

Students completed baseline assessments in groups of 12, seated next to one another in a single computer-equipped laboratory. A physician, responsible for medical oversight of the students, administered baseline testing. Data analysis and interpretation was completed through a collaborative relationship with the authors. Test-takers were first assigned to two independent groups on the basis of whether or not they provided feedback at the completion of cognitive assessment. Next, test-takers who provided feedback were further assigned to three independent groups on the basis of the specific type of feedback provided: Environmental distracters, problems with the computer, or difficulties with test instructions. These test-takers could provide feedback in more than one category, allowing them to be assigned to more than one group. As such, all analyses were conducted for “any feedback” as well as for specific forms of feedback.

Analyses

Participants were assigned to independent groups on the basis of: (a) having provided any type of feedback (yes/no); (b) having provided feedback in the areas of environmental distraction (yes/no), problems with the computer (yes/no), and problems with test instructions (yes/no); (c) history of concussion (none, previous). Each of the above-listed groupings served as independent variables. Composite scores on ImPACT (visual memory, verbal memory, processing speed, and reaction time), as well as symptom scale scores, served as the dependent variables. Chi-square analyses were conducted to evaluate increased likelihood of providing subjective feedback on the basis of concussion history. Analyses of variance were conducted to test for the relationship between feedback provision and performance on ImPACT. In order to control for family-wise error (e.g., increased chance of Type I error using four univariate analyses), the Bonferroni correction set the alpha level to p = .0125 for analyses within each feedback type.

Results

Demographics

In order to insure that motivational factors were not contributing to students' tendency to provide feedback, students scoring above 30 on the Impulse Control composite score were identified, as such a score suggests invalidity of the test results (Lovell, 2007). Test scores from a total of 159 students fell beyond this cut-off (8.7%), while individuals with Impulse Control composite scores above 30 were no more likely to provide feedback during testing (32.4%) than those with test scores 30 or lower (33.3%), χ2(1) = 0.1, p = .82, all subsequent analyses were conducted with these cases removed.

The subjective feedback of students completing baseline assessments was reviewed. Environmental feedback consisted of noise (talking/laughing, cell phone ringing), distractions from mouse or keyboard clicking and other issues (discomfort, boredom, room temperature). Computer-based feedback consisted primarily of mechanical problems with the mouse, as well as other issues (problems with the monitor, pop-up messages). Problems with test instructions included difficulty understanding specific modules of the test, as well as general issues (confusion, failure to read instructions, not devoting enough time).

A total of 538 students reported subjective feedback at the completion of testing (32.4%). With respect to the type of feedback, 161 students reported environmental distracters (9.7%), 199 reported problems with the computer (12%), and 297 reported problems with the test instructions (17.9%). Refer to Table 2 for a breakdown of specific feedback provided within these categories.

Table 2.

Subjective feedback provided by participants

Feedback area n (%) 
Environment 161 (9.7%) 
Computer 199 (12.0%) 
Instruction 297 (17.9%) 
Totala 538 (32.4%) 
Feedback area n (%) 
Environment 161 (9.7%) 
Computer 199 (12.0%) 
Instruction 297 (17.9%) 
Totala 538 (32.4%) 

aParticipants provided feedback in more than one category, so totals reflect unique numbers of individuals providing feedback.

History of ADD/LD and History of Concussion

ImPACT contains self-report indicators of learning difficulties (ADD, ADHD, receiving special education), which have been linked to decreased performance on cognitive testing (Collins et al., 1999). Students self-reporting learning problems or having received special educational services received did not differ on the basis of providing feedback, χ2(1) = 0.45, p = .50.

History of concussion was reported in 17.6% of the total sample. Chi-square analyses revealed that the history of concussion was not related to providing subjective feedback (p = .05). This was true for environmental feedback (p = .43) as well as computer-based feedback (p = .99); however, 23.6% of those students with a history of concussion provided subjective feedback in regard to understanding test instructions, when compared with 16.7% of those with no history of concussion, χ2(1) = 7.9, p = .005. Analyses of variance revealed no relationship between the history of concussion and performance on ImPACT composite scores or symptom scale.

Providing Any Form of Feedback

One-way analyses of variance revealed a significant relationship between providing feedback (any type) and total symptom scores, F(1,1657) = 18.6, p < .001, d = .21. Individuals providing any form of feedback reported significantly higher symptom scores; group means and effect sizes are provided in Table 3. Follow-up analysis of specific symptoms reported revealed that students providing feedback endorsed more physical symptoms (headache, nausea, fatigue, drowsiness, sleeping less, numbness/tingling), cognitive symptoms (feeling slowed down, mentally foggy, difficulty concentrating, difficulty remembering), and emotional symptoms (feeling irritable, feeling more emotional). Analysis by gender revealed that female students were more likely to endorse physical symptoms (balance problems, fatigue, dizziness, sleeping less, difficulty falling asleep) and emotional symptoms (feeling irritable, sadness, feeling nervous, feeling more emotional). Mean symptoms for men and women are within normal ranges with respect to baseline demographic symptom data reported in the literature (seeLovell et al., 2006). All other comparisons failed to yield significant findings.

Table 3.

ImPACT means by “any” feedback group

 No feedback Feedback F-value p-value d-value 
Verbal memory 88.3 (8.2) 87.7 (8.2) 2.2 .14 .07 
Visual memory* 78.3 (12.3) 77.9 (11.0) 0.4 .54 .04 
Process. speed 40.2 (6.6) 39.6 (6.9) 3.2 .08 .09 
Reaction time 54.5 (6.6) 54.1 (6.9) 1.4 .23 .06 
Total symptoms 5.6 (7.8) 7.5 (9.0) 18.6 .001 .21 
 Men 5.0 (7.1) 6.6 (7.9)    
 Women 6.5 (7.9) 8.7 (10.3)    
 No feedback Feedback F-value p-value d-value 
Verbal memory 88.3 (8.2) 87.7 (8.2) 2.2 .14 .07 
Visual memory* 78.3 (12.3) 77.9 (11.0) 0.4 .54 .04 
Process. speed 40.2 (6.6) 39.6 (6.9) 3.2 .08 .09 
Reaction time 54.5 (6.6) 54.1 (6.9) 1.4 .23 .06 
Total symptoms 5.6 (7.8) 7.5 (9.0) 18.6 .001 .21 
 Men 5.0 (7.1) 6.6 (7.9)    
 Women 6.5 (7.9) 8.7 (10.3)    

*Visual memory analysis based on df of 1,116, as ImPACT v1.0 does not include this composite score. All other analyses are based on df of 1,657.

Environmental Feedback

One-way analyses of variance revealed that reporting environmental distraction was significantly related to total symptom scores, F(1,1657) = 17.8, p < .001, d = .21. Individuals reporting environmental distraction endorsed significantly more symptoms; group means and effect sizes are provided in Table 4. Follow-up analysis of specific symptoms reported revealed that students providing environmental feedback endorsed more physical symptoms (headache, fatigue, difficulty falling asleep, drowsiness, sleeping less), cognitive symptoms (feeling slowed down, mentally foggy, difficulty concentrating, difficulty remembering), and emotional symptoms (feeling irritable, sadness, nervousness, feeling more emotional). Analysis by gender revealed that female students were more likely to endorse physical symptoms (headache, balance problems, nausea, dizziness, fatigue, sleeping less, difficulty falling asleep) and emotional symptoms (feeling irritable, sadness, feeling nervous, feeling more emotional). All other comparisons failed to yield significant findings.

Table 4.

ImPACT group means by environmental feedback group

 No feedback Feedback F-value p-value d-value 
Verbal memory 88.1 (8.2) 87.5 (8.3) 0.8 .38 .04 
Visual memory* 78.3 (12.1) 77.1 (10.5) 1.0 .31 .06 
Process. speed 40.0 (6.7) 39.98 (6.6) 0.04 .84 .01 
Reaction time 54.3 (6.7) 54.3 (6.6) 0.01 .98 .00 
Total symptoms 5.9 (8.0) 8.8 (8.9) 17.8 .001 .21 
 Men 5.4 (7.4) 6.5 (7.1)    
 Women 6.6 (7.1) 11.5 (11.3)    
 No feedback Feedback F-value p-value d-value 
Verbal memory 88.1 (8.2) 87.5 (8.3) 0.8 .38 .04 
Visual memory* 78.3 (12.1) 77.1 (10.5) 1.0 .31 .06 
Process. speed 40.0 (6.7) 39.98 (6.6) 0.04 .84 .01 
Reaction time 54.3 (6.7) 54.3 (6.6) 0.01 .98 .00 
Total symptoms 5.9 (8.0) 8.8 (8.9) 17.8 .001 .21 
 Men 5.4 (7.4) 6.5 (7.1)    
 Women 6.6 (7.1) 11.5 (11.3)    

*Visual memory analysis based on df of 1,116, as ImPACT v1.0 does not include this composite score. All other analyses are based on df of 1,657.

Computer-Based Feedback

One-way analyses of variance revealed a significant relationship between reporting computer problems and reaction time composite scores, F(1,1657) = 8.2, p = .004, d = .14. Of note, students reporting computer-based feedback scored significantly “faster” on reaction time composite scores. Group means and effect sizes are provided in Table 5. All other comparisons failed to yield significant findings.

Table 5.

ImPACT group means by computer-based feedback

 No feedback Feedback F-value p-value d-value 
Verbal memory 88.2 (8.2) 87.2 (8.2) 2.7 .10 .08 
Visual memory* 78.4 (12.1) 78.5 (10.6) 0.1 .90 .00 
Process. speed 40.0 (6.6) 40.2 (7.3) 0.1 .89 .01 
Reaction time 54.5 (6.7) 53.1 (6.0) 8.2 .004 .14 
Total symptoms 6.0 (8.1) 6.8 (8.2) 1.0 .31 .05 
 No feedback Feedback F-value p-value d-value 
Verbal memory 88.2 (8.2) 87.2 (8.2) 2.7 .10 .08 
Visual memory* 78.4 (12.1) 78.5 (10.6) 0.1 .90 .00 
Process. speed 40.0 (6.6) 40.2 (7.3) 0.1 .89 .01 
Reaction time 54.5 (6.7) 53.1 (6.0) 8.2 .004 .14 
Total symptoms 6.0 (8.1) 6.8 (8.2) 1.0 .31 .05 

*Visual memory analysis based on df of 1,116, as ImPACT v1.0 does not include this composite score. All other analyses are based on df of 1,657.

Instruction-Based Feedback

One-way analyses of variance revealed a significant relationship between reporting problems with instructions and total symptom scores, F(1,1657) = 14.1, p < .001, d = .19. Individuals reporting problems with test instructions endorsed significantly more symptoms; group means and effect sizes are provided in Table 6. Follow-up analysis of specific symptoms reported revealed that students providing instruction-based feedback endorsed more physical symptoms (balance problems, fatigue, drowsiness, sleeping less), cognitive symptoms (feeling slowed down, difficulty concentrating), and emotional symptoms (feeling irritable, feeling more emotional). Analysis by gender revealed that female students were more likely to endorse physical symptoms (balance problems, fatigue, dizziness, sleeping less, difficulty falling asleep) and emotional symptoms (feeling irritable, sadness, feeling nervous, feeling more emotional). All other comparisons failed to yield significant findings.

Table 6.

ImPACT group means by instruction-based feedback

 No feedback Feedback F-value p-value d-value 
Verbal memory 88.2 (8.1) 87.6 (8.4) 1.0 .32 .05 
Visual memory* 78.3 (11.9) 77.2 (12.2) 1.0 .32 .06 
Process. speed 40.2 (6.7) 39.2 (6.8) 4.8 .03 .11 
Reaction time 54.3 (6.6) 54.3 (7.4) 0.01 .93 .00 
Total symptoms 5.9 (7.9) 7.8 (9.43) 14.1 .001 .19 
 Men 5.4 (7.5) 6.5 (7.1)    
 Women 6.6 (8.8) 11.5 (11.3)    
 No feedback Feedback F-value p-value d-value 
Verbal memory 88.2 (8.1) 87.6 (8.4) 1.0 .32 .05 
Visual memory* 78.3 (11.9) 77.2 (12.2) 1.0 .32 .06 
Process. speed 40.2 (6.7) 39.2 (6.8) 4.8 .03 .11 
Reaction time 54.3 (6.6) 54.3 (7.4) 0.01 .93 .00 
Total symptoms 5.9 (7.9) 7.8 (9.43) 14.1 .001 .19 
 Men 5.4 (7.5) 6.5 (7.1)    
 Women 6.6 (8.8) 11.5 (11.3)    

*Visual memory analysis based on df of 1,116, as ImPACT v1.0 does not include this composite score. All other analyses are based on df of 1,657.

Relationship Between Symptom Reporting, Providing Feedback, and ImPACT Performance

Given the observed relationship between provision of feedback and symptom reporting, ImPACT performance was analyzed in the context of these two factors. Participants were assigned to independent groups on the basis of total symptom scores using a median split (low = 0–3 on total symptom score; high = 4+ on total symptom score). A 2 × 2 MANOVA was conducted with reporting any feedback (yes/no) and symptom level (high/low) as the independent variables and with the ImPACT composite scores as the dependent measures (visual memory, verbal memory, processing speed, reaction time). Results revealed a significant multivariate effect of symptom level on ImPACT performance, F(4,1111) = 2.40, p = .048, explained by univariate effects of symptom level on visual memory scores, F(1,1114) = 7.17, p = .008 (none of the other composite scores yielded significant univariate results). No multivariate effect of providing feedback was noted on ImPACT performance, F(4,1111) = 1.74, p = .14, and no symptom by feedback interaction was noted, F(4,1111) = 0.75, p = .58. All group means were within the limits of normative data provided by the test developers (Iverson, Lovell, & Collins, 2003).

Discussion

Significant differences were observed in endorsed symptoms between groups of high-school students at baseline as a function of reported environmental distraction and/or difficulty understanding test instructions. The relationship between self-report symptoms and self-report problems during testing (environmental or instruction-based) reveals that students endorsing more symptoms at baseline also endorse more problems with the environment and test. As noted, symptom reporting on ImPACT takes place “prior to” the provision of feedback, so any relationships must be viewed as correlational and not causal. Although increased symptoms were present in those providing feedback, these increases may reflect personality styles, reactions to the test-taking environment, or other intrapersonal issues which co-present or manifest as symptoms and complaints about the testing environment, materials, or instructions.

This study is the first to analyze subjective feedback provided by test-takers and its relationship with performance and self-reported symptomology. The finding that students endorsing more symptoms at baseline also endorse more problems with the environment and test may, in part, be explained by Trimmel and Poelzl (2006) who found that attentional functions are sensitive to environmental conditions, thus demanding a higher concentration on the working task, resulting in increased mental load and fatigue. It is unclear if the presence of distracters affects attention in such a stressful way that the test-taker endorses more symptoms than usual, or whether those who are already symptomatic prior to testing are more prone to voice complaints about the testing.

Significant differences were also observed in reaction time scores between groups of high-school students at baseline as a function of reported problems with the computer during testing; however, these differences were not in the expected direction. Although group differences were small, and perhaps clinically insignificant, these results cannot be explained by group differences in variance, co-variance, or presence of outliers. The relationship between reported computer problems and reaction time performance is supported in the literature. In the present study, many test-takers reported problems with right/left confusion (on mouse clicking) during test modules measuring choice reaction time. Researchers (e.g., Wilson & McMillan, 1991) predicted such problems when specific response keys are in close proximity to one another. Test-takers also reported problems with the cursor “jumping” during testing. Traditional computer mice containing a roller ball are prone to collecting dirt and debris, thus impeding sensitivity to movement. Similarly, more modern optical computer mice are susceptible to movement error due to the reflectance of the surface (e.g., too glossy, variations in mouse pad color or design), as well as dirt build-up on the L.E.D. lens or pads of the mouse.

As stated, those students reporting computer-based feedback scored significantly faster on reaction time composite scores. Baddeley, Papagno, and Andrade (1993) postulated a form of attentional filtering that allows subjects to concentrate on relevant items and exclude irrelevant stimuli. In this regard, relatively infrequent, moderately arousing, and temporally isolated problems during testing may have served to increase students' arousal levels, thus optimizing performance (e.g., Yerkes-Dodson, 1908).

Gender differences in symptom reporting among high-school students, during baseline assessment, are not widely documented in the literature. Barr (2003) documented gender differences on baseline neuropsychological test performance, with women out-performing men on three indices; however, data on symptom endorsement were not collected. Our current results are consistent with previously published findings by Lovell and colleagues (2006) for high-school varsity athletes (men: Mean = 4.8, SD = 7.9; women: Mean = 7.7, SD = 13.7). Differences in specific symptoms reported are consistent with published data from a collegiate sample (Covassin et al., 2006) in which female athletes endorsed significantly more symptoms at baseline when compared with male athletes: Headache, nausea, fatigue, sleeping more than usual, drowsiness, sensitivity to light and noise, sadness, nervousness, feeling more emotional, difficulty concentrating, and visual problems.

The utility of subjective feedback may be enhanced in the context of symptom minimization. Researchers have pointed to athletes' motivation to under-report symptoms following a concussion (Hall, Hall, & Chapman, 2005), in order to return to athletic competition (Echemendia & Cantu, 2003), or for fear of removal from a game or losing their position (Lovell et al., 2002). Van Kampen, Lovell, Pardini, Collins, and Fu (2006) noted that even athletes who report being symptom free may continue to exhibit neurocognitive deficits, which they are either unaware of or are failing to report. Although the current results demonstrate a relationship between symptom reporting and subjective feedback at baseline, such a relationship might also occur post-concussion. In the event that an athlete may under-report symptoms, knowing that subjective feedback may be another indicator of difficulty with test demands could aid or assist the clinician in interpreting test results. Since the present study included high-school students with a variety of athletic exposure and level of play, it might be useful to compare students involved in intramural vs. varsity sports with regard to symptom endorsement and provision of feedback. Such comparisons may help generalize these results to varsity athletes, and how such feedback should be interpreted.

This study is not without its limitations. First, the participants were high-school students participating in mandatory preseason testing. Motivation to complete the testing varied, resulting in a percentage of students being excluded due to excessive scores on the Impulse Control composite. This percentage (8.7%) was considerably lower than the rate reported for college undergraduate volunteers completing three successive computer-based test batteries in one test session (35%; Broglio, Ferrara, Macciocchi, Baumgartner, & Elliott, 2007), higher than the rate (5%) reported for NFL players (Solomon & Haase, 2008) completing computer-based testing, and lower than the rate (11%) reported for a high-school athletic sample completing paper-based assessments (Hunt, Ferrara, Miller, & Macciocchi, 2007). Second, the computer facilities used for testing were also used for a variety of other purposes. Although this is common practice in most educational settings, this may have resulted in variability in computers (screen resolutions, use/quality of mouse pads, pop-up warnings, etc.). Third, it is not clear if any participants had suffered unreported, unidentified concussions that were temporally close to the baseline testing sessions. Fourth, the distractions experienced by student athletes at baseline may be different from those encountered during sideline or laboratory post-concussion evaluations. It has not been systematically determined if minor, brief noise/distraction enhances or hinders performance, if more constant noise/distraction is detrimental, or if such hindrance applies to laboratory and in vivo settings. Finally, as the sample was comprised of high-school students, these results may not generalize to older (collegiate or professional) athletes.

Nevertheless, these results suggest that computer-based test developers should include post-assessment opportunities for test-takers to provide feedback, and test administrators should inquire about the test-taking environment and experience. Certainly, the relationship between symptom reporting and the tendency to provide subjective feedback related to the environment, test instructions, or computer interface may warrant further consideration. Research investigating intra-individual or personality factors related to sensitivity to environmental conditions, understanding test instructions, or interacting with computer-based assessments may further elucidate or explain these findings.

Conflict of Interest

None declared.

References

Baddeley
A.
Papagno
C.
Andrade
J.
The sandwich effect: The role of attentional factors in serial recall
Journal of Experimental Psychology: Learning, Memory, and Cognition
 , 
1993
, vol. 
19
 
4
(pg. 
862
-
870
)
Barr
W. B.
Neuropsychological testing of high school athletes: Preliminary norms and test–retest indices
Archives of Clinical Neuropsychology
 , 
2003
, vol. 
18
 (pg. 
91
-
101
)
Bartram
D.
Bayliss
R.
Automated testing: Past, present and future
Journal of Occupational Psychology
 , 
1984
, vol. 
57
 (pg. 
221
-
237
)
[PubMed]
Broglio
S. P.
Ferrara
M. S.
Macciocchi
S. N.
Baumgartner
T. A.
Elliott
R.
Test–retest reliability of computerized concussion assessment programs
Journal of Athletic Training
 , 
2007
, vol. 
42
 
4
(pg. 
509
-
514
)
[PubMed]
Cernich
A. N.
Brennana
D. M.
Barker
L. M.
Bleiberg
J.
Sources of error in computerized neuropsychological assessment
Archives of Clinical Neuropsychology
 , 
2007
, vol. 
22
 
Suppl. 1
(pg. 
S39
-
S48
)
Chen
Z.
Cave
K. R.
When does visual attention select all features of a distractor?
Journal of Experimental Psychology: Human Perception and Performance
 , 
2006
, vol. 
32
 
6
(pg. 
1452
-
1464
)
Collie
A.
Maruff
P.
Computerised neuropsychological testing
British Journal of Sports Medicine
 , 
2003
, vol. 
37
 
1
(pg. 
2
-
3
)
Collins
M. W.
Grindel
S. H.
Lovell
M. R.
Dede
D. E.
Moser
D. J.
Phalin
B. R.
, et al.  . 
Relationship between concussion and neuropsychological performance in college football players
JAMA: The Journal of the American Medical Association
 , 
1999
, vol. 
282
 
10
(pg. 
964
-
970
)
Corneil
B. D.
Munoz
D. P.
The influence of auditory and visual distractors on human orienting gaze shifts
Journal of Neuroscience
 , 
1996
, vol. 
16
 
24
(pg. 
8193
-
8207
)
[PubMed]
Covassin
T.
Swanik
C. B.
Sachs
M. L.
Kendrick
Z.
Schatz
P.
Zillmer
E.
Kaminaris
C.
Sex differences on baseline neuropsychological function and concussion symptoms of collegiate athletes
British Journal of Sports Medicine
 , 
2006
, vol. 
40
 (pg. 
923
-
927
)
Echemendia
R. J.
Cantu
R. C.
Return to play following sports-related mild traumatic brain injury: The role for neuropsychology
Applied Neuropsychology
 , 
2003
, vol. 
10
 
1
(pg. 
48
-
55
)
Hall
R. C.
Hall
R. C.
Chapman
M. J.
Definition, diagnosis, and forensic implications of postconcussional syndrome
Psychosomatics
 , 
2005
, vol. 
46
 
3
(pg. 
195
-
202
)
Hunt
T. N.
Ferrara
M. S.
Miller
L. S.
Macciocchi
S.
The effect of effort on baseline neuropsychological test scores in high school football athletes
Archives in Clinical Neuropsychology
 , 
2007
, vol. 
22
 
5
(pg. 
615
-
621
)
Iverson
G.
Gaetz
M.
Lovell
M.
Collins
M.
Relation between subjective fogginess and neuropsychological testing following concussion
Journal of the International Neuropsychological Society
 , 
2004
, vol. 
10
 (pg. 
1
-
3
)
Iverson
G. L.
Lovell
M. R.
Collins
M. W.
Normative data: For Foxpro (Desktop) version of ImPACT only
 , 
2003
 
Retrieved December 3, 2009, from http://www.impacttest.com/normative.php
Lovell
M.
Bailes
J. E.
Lovell
M.
Maroon
J.
Evaluation of the professional athlete
Sports-related concussion
 , 
1999
St Louis
Quality Medical Publishing
(pg. 
200
-
214
)
Lovell
M.
ImPACT version 6.0 clinical user's manual
 , 
2007
 
Retrieved December 3, 2009, from http://www.impacttest.com/
Lovell
M.
Burke
C. J.
Cantu
R. C.
The NHL concussion program
Neurologic athletic head and spine injury
 , 
2002
Philadelphia
WB Saunders
(pg. 
32
-
45
)
Lovell
M.
Collins
M. W.
Neuropsychological assessment of the college football player
Journal of Head Trauma and Rehabilitation
 , 
1998
, vol. 
13
 
2
(pg. 
9
-
26
)
Lovell
M. R.
Collins
M. W.
Iverson
G. L.
Field
M.
Maroon
J. C.
Cantu
R.
, et al.  . 
Recovery from mild concussion in high school athletes
Journal of Neurosurgery
 , 
2003
, vol. 
98
 
2
(pg. 
296
-
301
)
Lovell
M. R.
Collins
M. W.
Maroon
J. C.
Cantu
R.
Hawn
M. A.
Burke
C. J.
, et al.  . 
Inaccuracy of symptom reporting following concussion in athletes
Medicine and Science in Sports and Exercise
 , 
2002
, vol. 
34
 
5
pg. 
S298
 
Lovell
M. R.
Iverson
G. L.
Collins
M. W.
Podell
K.
Johnston
K. M.
Pardini
D.
, et al.  . 
Measurement of symptoms following sports-related concussion: Reliability and normative data for the post-concussion scale
Applied Neuropsychology
 , 
2006
, vol. 
13
 
3
(pg. 
166
-
174
)
Macken
W. J.
Phelps
F. G.
Jones
D. M.
What causes auditory distraction?
Psychonomic Bulletin and Review
 , 
2009
, vol. 
16
 
1
(pg. 
139
-
144
)
Pierce
R. C.
Note on testing conditions
Journal of Consulting Psychology
 , 
1963
, vol. 
27
 
6
(pg. 
536
-
537
)
Podell
K.
Lovell
M. R.
Echemendia
R. J.
Barth
J. T.
Collins
M. W.
Computerized assessment of sports-related concussions
Traumatic brain injury in sports
 , 
2004
Lisse, The Netherlands
Swets & Zeitlinger
(pg. 
375
-
396
)
Schatz
P.
Browndyke
J. N.
Applications of computer-based neuropsychological assessment
Journal of Head Trauma Rehabilitation
 , 
2002
, vol. 
17
 
5
(pg. 
395
-
410
)
Segalowitz
S. J.
Mahaney
P.
Santesso
D. L.
MacGregor
L.
Dywan
J.
Willer
B.
Retest reliability in adolescents of a computerized neuropsychological battery used to assess recovery from concussion
NeuroRehabilitation
 , 
2007
, vol. 
22
 
3
(pg. 
243
-
251
)
[PubMed]
Solomon
G. S.
Haase
R. F.
Biopsychosocial characteristics and neurocognitive test performance in National Football League players: An initial assessment
Archives in Clinical Neuropsychology
 , 
2008
, vol. 
23
 
5
(pg. 
563
-
577
)
Space
L. G.
The computer as psychometrician
Behavior Research Methods and Instrumentation
 , 
1981
, vol. 
13
 
4
(pg. 
595
-
606
)
Trimmel
M.
Poelzl
G.
Impact of background noise on reaction time and brain DC potential changes of VDT-based spatial attention
Ergonomics
 , 
2006
, vol. 
49
 
2
(pg. 
202
-
208
)
Van Kampen
D. A.
Lovell
M. R.
Pardini
J. E.
Collins
M. W.
Fu
F. H.
The “value added” of neurocognitive testing after sports-related concussion
American Journal of Sports Medicine
 , 
2006
, vol. 
34
 
10
(pg. 
1630
-
1635
)
Wechsler
D.
Manual for the Wechsler Adult Intelligence Scale
 , 
1955
Oxford, England
Psychological Corp
Weinstein
N. D.
Effect of noise on intellectual performance
Journal of Applied Psychology
 , 
1974
, vol. 
59
 
5
(pg. 
548
-
554
)
Wilson
S. L.
McMillan
T. M.
Ager
A.
Microcomputers in psychometric and neuropsychological assessment
Microcomputers and clinical psychology: Issues, applications and future developments
 , 
1991
Chichester, England
John Wiley and Sons, Ltd
(pg. 
79
-
94
)
Yerkes
R. M.
Dodson
J. D.
The relation of strength of stimulus to rapidity of habit-formation
Journal of Comparative Neurology and Psychology
 , 
1908
, vol. 
18
 (pg. 
459
-
482
)