Abstract

This article reports on three pre- versus post-season prospective studies in which male university and high school contact sport players predominantly of Rugby Union (hereafter rugby) were compared with age, education, and IQ equivalent non-contact sport controls on the ImPACT (Immediate Postconcussion Assessment and Cognitive Testing) test. All analyses revealed a relative absence of practice effects on the Visual Motor Speed (VMS) composite for contact sport groups compared with controls. The VMS data for rugby players from each study were pooled and subjected to additional analysis (Rugby, n = 145; Controls, n = 106). Controls revealed significant improvement over the season (p < .001), whereas no learning effect was in evidence for rugby players whose performance remained the same (interaction effect, p = .028). It is apparent that practice effects have diagnostic potential in this context, implicating vulnerability on speeded visuomotor processing in association with participation in rugby. Pointers for further research and concussion management in the individual case are explored.

Introduction

Research on Contact Sport in General

In recent years, there has been a gathering body of cross-sectional research that provides provocative indications of persistent deleterious neurocognitive effects of repeated mild traumatic brain injury (MTBI) in participants of field contact sports when compared with equivalent non-contact sport controls, for example, in respect of soccer (Matser, Kessels, Jordan, Lezak, & Troost, 1998; Matser, Kessels, Lezak, Jordan, & Troost, 1999; Matser, Kessels, Lezak, & Troost, 2001; Webbe & Ochs, 2003; Witol & Webbe, 2003), American football (Grindel, Lovell, & Collins, 2001; Guskiewicz et al., 2005; Iverson, Gaetz, Lovell, & Collins, 2004; Kutner, Erlanger, Tsai, Jordan, & Relkin, 2000; Moser, Schatz, & Jordan, 2005), and Rugby Union (Farace, Ferree, Hollier, Barth, & Shaffrey, 2003; Gardner, Shores, & Batchelor, 2010; Pettersen & Skelton, 2000; Shuttleworth-Edwards, Border, Reid, & Radloff, 2004; Shuttleworth-Edwards & Radloff, 2008; Shuttleworth-Jordan, Puchert, & Balarin, 1993; Shuttleworth-Edwards, Smith, & Radloff, 2008).1 In addition, a series of studies that have investigated neuropsychological effects within a contact sport cohort in terms of the number of reported concussions (rather than contact versus non-contact groups) have demonstrated long-term cognitive or symptomatic deficit in association with increasing numbers of concussions (Gardner et al., 2010; Killam, Cautin, & Santucci, 2005; Moser et al., 2005). All these studies implicate residual (i.e., permanent) deleterious effects of extended exposure to concussive and subconcussive events in association with the various field contact sports. In turn, this raises concerns about the development of dementia among players of these sports at older age stages such as has been linked to amateur and professional soccer (Autti, Sipala, Autti, & Salonen, 1997; Spear, 1995) and American football (Stern et al., 2011). To the authors' knowledge, no such long-term research has been conducted as yet in respect of players of Rugby Union, although there is no reason to believe that the sport will be exempt from equivalent concerns.

Specifically in respect of school and university level participation in the field contact sports, there have been a number of pertinent findings given that such athletes are engaged in scholarly activities. It has been demonstrated that university football players with a history of two or more concussions and a learning disability reveal significantly worse performance on tests of executive function and speed of mental processing than those with a similar history of concussions and the absence of a learning disability (Collins et al., 1999). A further youth-related study provides evidence for persistent neurocognitive difficulties in a sample of symptom-free high school athletes with two or more concussions and lower cumulative grade averages than those with no history of concussion (Moser et al., 2005). It was uncertain whether the link between concussions and low school grades was as a result of the concussion history, or a characteristic of those predisposed to concussion, or a combination of both of these factors.

Commensurate with the implications of these reports, a review covering a spectrum of rugby/football sports, that takes methodological considerations critically into account concludes that the evidence in support of persistent negative neurocognitive outcome for contact sports players is based on a series of relatively robust studies compared with a minority of weaker studies that produce null results (Shuttleworth-Edwards & Whitefield, 2007). These authors go on to express considerable ethical disquiet about youth participation in these sports, reiterating the concern of others (Gaetz, Goodman, & Weinberg, 2000), that a significant proportion of such youth will have sustained what could be a permanent reduction in brain function prior to reaching maturity. In contrast to this view, a recent study targeted specifically at high school level football found no evidence for increased risk of later developing dementia, compared with non football playing high school boys (Savica, Parisi, Wold, Josephs, & Ahlskog, 2012). Further, in a review restricted to American football, it was suggested that evidence for prolonged deleterious neurocognitive effects of concussion in collegiate and high school athletes is mixed and not convincing (Solomon, Ott, & Lovell, 2011). However, these authors admit to their study representing a limited review in this regard and highlight a number of moderator variables that typically are not controlled for in sports concussion studies, such as sex, age, pre-existing medical and psychiatric conditions, and sports affiliation, thereby affecting the state of knowledge.

A further methodological concern to take into consideration, which may influence the state of knowledge in sports concussion studies, is the contribution of Type II error, that is, the failure to identify effects of the MTBI when it is present. There has been comprehensive discussion in the sports MTBI literature (Rutherford, Stephens, Potter, & Fernie, 2005) that the use of multiple measures in the research context calls for substantial adjustments toward statistical stringency in order to guard against Type I error. However, it has also been vehemently pointed out how this procedure in turn increases the risk of Type II error to the extent of being provocatively labeled a “crime” or “misdemeanour” when this translates into the failure to identify clinically relevant effects in group research that have relevance in the individual case (Brandt, 2007). In order to circumvent the problem, it has been suggested that data derived from collections of smaller studies should be combined to gain the increased statistical power that becomes available in a larger sample (Demakis, 2006). Also, the employment of a restricted neurocognitive variable with a view to focused hypothesis testing (rather than the application of a wide-ranging exploratory test battery) is considered the more appropriate methodological route to follow (Frencham, Fox, & Maybery, 2005). Accordingly, a prior analysis was conducted by the present researchers in which the outcome for a series of cross-sectional rugby studies, conducted from high school through to the professional level, was combined purely in respect of visuomotor function (Trail Making Test [TMT] and Digit Symbol Substitution; Shuttleworth-Edwards & Radloff, 2008). The composite analysis yielded outcome strongly in support of clinically relevant deleterious speeded visuomotor function for participants of rugby, which was attributed to cumulative concussive and subconcussive effects over years of participation in the sport.

In sum from this review, it is clear that the jury is still out on the issue of whether or not there might be long-term deleterious consequences arising out of high school and college contact sport and that there is a need for more methodologically sophisticated research in the area. As pointed out by Yeates (2010) in a recent review of mild traumatic brain outside of the sports arena, even if only a small proportion of young individuals with MTBI suffer negative sequelae then this is a serious health concern. Consequently, there is a growing consensus of opinion that upholds the need for careful identification of sports concussion in the first instance, followed by the medical management of the injury on an individualized basis at all levels of play from school to professional levels, in a process that ideally includes computerized neurocognitive evaluation to facilitate accurate diagnosis and safe return to play (Moser et al., 2007; Schatz & Moser, 2011).

Research on Rugby Union

Rugby Union (designated by the term “rugby” for the purposes of this article) has repeatedly emerged in reviews as the rugby/football sport with the highest incidence of concussion (Shuttleworth-Edwards, Noakes, et al., 2008). Of the seven neuropsychological studies identified specifically in respect of rugby (cited in the introductory paragraph above), all have supported the presence of compromised cognitive performance for athletes at the school, university, and professional levels, predominantly in the modalities of memory and speeded visuomotor attentional skills. Specifically, Digit Span Backwards showed sensitivity to lowered performance relative to controls in four of these studies and the TMT in five studies.

In two of these studies on university level rugby playing populations (Shuttleworth-Edwards, Smith, et al., 2008; Shuttleworth-Jordan et al., 1993), an important differentiating factor was that the rugby playing participants, none of whom had been diagnosed with a concussion during the season, failed to benefit from a practice effect on speeded and unspeeded attentional tasks between the pre- versus post-season test intervals to the same extent as was in evidence for age and educationally equivalent controls. Specifically, the phenomenon occurred in respect of TMT A and B and Digit Supraspan (Shuttleworth-Jordan et al., 1993) and TMT A and B and Digits Backwards (Shuttleworth-Edwards, Smith, et al., 2008). These findings served to replicate those of an earlier study in which a similar absence of a pre- versus post-season practice effect was in evidence for college level players of American football on the Symbol Digit Substitution task (Barth et al., 1989).

In addition to paper and pencil tasks, computerized testing using the ImPACT (Immediate Postconcussion Assessment and Cognitive Testing) test (Lovell, 2004) was employed in the more recent university level rugby study of Shuttleworth-Edwards, Smith, and colleagues (2008), on the basis of which practice effects did not serve markedly to differentiate the rugby players from controls on any of the composite scores including Visual and Verbal Memory, Reaction Time, or Visual Motor Speed (VMS). Descriptively, however, for the ImPACT VMS composite alone the tendency was in evidence, in that there was an “improvement” in the mean score for controls, whereas in contrast the mean score for rugby players actually got worse. Albeit subtle, the finding of differential performance between rugby and controls, specifically in respect of ImPACT VMS score and not the two memory scores, is conceptually consistent with the fact that the ImPACT VMS score has revealed divergent construct validity as a memory test, but convergent construct validity as a test of processing speed, correlating highly with the Symbol Digit Modality test (Iverson, Lovell, & Collins, 2005), a test that is typically characterized by a learning effect (Lezak, Howieson, & Loring, 2004). Moreover, an ImPACT reliability study on adolescents and university adults indicated that although scores on the ImPACT verbal and visual memory composites did not increase with repeat testing over an average retest interval of less than a week, a small practice effect was in evidence for the ImPACT VMS composite (Iverson, Lovell, & Collins, 2003).

Therefore, considering the ImPACT VMS composite outcome together with the outcome of the paper and pencil testing on the same university-based study (Shuttleworth-Edwards, Smith, et al., 2008), the expected improvement between pre- and post-season neurocognitive test outcome that occurred for controls, and the absence of practice effects that was in evidence for rugby players can be considered clinically meaningful. The reason being that when a test phenomenon of this nature occurs, a decrement in learning is implicated with diagnostic implications (Duff et al., 2007; Duff, 2012). For instance, the investigation of practice effects has been successfully applied as a predictive tool for cognitive outcome in respect of a series of studies in Mild Cognitive Impairment (MCI), Human Immunodeficiency Virus, and Huntington's disease (Duff et al., 2007). Moreover, as suggested by Lezak and colleagues (2004) citing Knight (1992), where a dementing condition is suspected even mildly lowered scores on tests vulnerable to a practice effect may suggest a deteriorating process. On this basis, even the mildly diminished scores that occurred in this Shuttleworth-Edwards, Smith, and colleagues (2008) study on the ImPACT VMS composite, compared with mildly improved scores for controls, can be seen to have provided a potentially critical additional diagnostic marker of vulnerability in the rugby sample in that study.

Importantly, the tendency for practice to be a differentiating factor between rugby players and controls on this university study exclusively for the ImPACT VMS composite, but not to any of the other ImPACT composites, gains potency when taken in conjunction with the pre- versus post-test outcome of two similar as yet unpublished studies using the ImPACT test on high school contact sport players made up predominantly of rugby players without a diagnosable concussion during the season (Horsman, 2010; Whitefield, 2007). As with the university study, the outcome for these two studies (Horsman, 2010; Whitefield, 2007) is provided on each of the core ImPACT composite scores including Verbal and Visual Memory, Reaction Time, as well as VMS. And again, as with the university study, there is a consistent tendency for a practice effect to occur for non-contact sport controls in the absence of this occurrence being in evidence for contact sport players on the ImPACT VMS composite only and not on any of the other three composites. The further investigation of practice effects between contact and non-contact sports groups on the basis of these three ImPACT test studies is thereby warranted in respect of the ImPACT VMS data alone. For more detailed discussion purposes, the pre- versus post-season dependent t-test analyses of these three studies in respect of the ImPACT VMS composite only have been isolated and appear in Table 1.

Table 1.

Studies 1–3: Dependent t-test comparisons of ImPACT VMS scores at the pre- versus post-season intervals for the Rugby and Rugby/Soccer versus Non-Contact sports groups (Horsman, 2012; Whitefield, 2007; Shuttleworth-Edwards, Smith, et al., 2008, respectively)

Source Sample size Pre-season Mean (SDPost-season Mean (SDt-value Effect size
 
p-value 
δ-value 95% CI 
Study 1 
 Rugby VMS n = 27 36.18 (6.43) 35.60 (7.84) 0.387 0.08 −0.30, 0.45 .351 
 Non-Contact VMS n = 18 39.37 (6.26) 40.26 (6.20) −0.925 −0.22 −0.68, 0.25 .184 
Study 2 
 Rugby/Soccera VMS n = 88 38.02 (6.74) 37.77 (6.23) 0.435 0.05 −0.16, 0.26 .664 
 Non-Contact VMS n = 62 37.27 (7.62) 39.03 (6.92) −2.625 −0.33 −0.59, −0.08 .011* 
Study 3 
 Rugby VMS n = 39 34.18 (6.99) 36.05 (6.66) −1.840 −0.30 −0.61, 0.03 .074 
 Non-Contact VMS n = 29 33.44 (5.87) 35.55 (5.54) −4.249 −0.79 −1.20, −0.37 <.001** 
Source Sample size Pre-season Mean (SDPost-season Mean (SDt-value Effect size
 
p-value 
δ-value 95% CI 
Study 1 
 Rugby VMS n = 27 36.18 (6.43) 35.60 (7.84) 0.387 0.08 −0.30, 0.45 .351 
 Non-Contact VMS n = 18 39.37 (6.26) 40.26 (6.20) −0.925 −0.22 −0.68, 0.25 .184 
Study 2 
 Rugby/Soccera VMS n = 88 38.02 (6.74) 37.77 (6.23) 0.435 0.05 −0.16, 0.26 .664 
 Non-Contact VMS n = 62 37.27 (7.62) 39.03 (6.92) −2.625 −0.33 −0.59, −0.08 .011* 
Study 3 
 Rugby VMS n = 39 34.18 (6.99) 36.05 (6.66) −1.840 −0.30 −0.61, 0.03 .074 
 Non-Contact VMS n = 29 33.44 (5.87) 35.55 (5.54) −4.249 −0.79 −1.20, −0.37 <.001** 

Note: VMS = Visual Motor Speed.

aRugby/Soccer = 90% rugby, 10% soccer.

*p < .05.

**p < .001.

Perusal of Table 1 reveals that for the study of Shuttleworth-Edwards, Smith, and colleagues (2008) (hereafter Study 1), there were no significant effects between the pre- and post-season intervals for either Rugby or the non-contact groups. Descriptively, however, as discussed earlier, the mean score for the Rugby group at post-season is less than at pre-season, in contrast to the mean score for Controls being higher at post-season, such as occurred also in the study of Whitefield (2007) (hereafter Study 2) for the combined Rugby/Soccer and Non-Contact groups, respectively. However, for Study 2, the improvement for controls was significant (p < .011) compared with no improvement for rugby/soccer players (p = .664). For the study of Horsman (2010) (hereafter Study 3), both groups improve between pre- and post-season, although the improvement for controls is highly significant (p < .001), whereas that for rugby fails to reach significance (p = .074).

Clearly, a common factor in the pre- versus post-season ImPACT VMS scores for these three studies was the tendency for the control sport groups to improve over the time period, whereas the contact sport groups of predominantly rugby players did not. As discussed above, both conceptually and empirically, it was considered that this feature had heuristic relevance as a possible diagnostic indicator of cerebral impairment (Duff et al., 2007; Duff, 2012; Lezak et al., 2004). However, to the author's knowledge, there has been little in the way of focused attention being given to the phenomenon of practice effects in the area of sports MTBI, as a clinical feature worthy of analysis and publication in its own right. Accordingly, it was decided to combine the prospective pre- versus post-season outcome data available from the three South African rugby studies delineated above on the single functional modality of visuoperceptual speed as tapped by the ImPACT VMS composite, in order to identify the presence of any vulnerability in the contact sport groups compared with equivalent non-contact sport controls due to failure to benefit from practice on this task.

In South Africa rugby participation begins at the age of 8, for around 8 months of the year (March–October), and therefore, exposure to rugby ranges from 10 to 15 years for top high school attendees and university scholars, respectively. Research indicates that around 13% of South African high school rugby players will be followed up for a perceived concussion, and while adequate data for South African university level rugby are not available, the seasonal incidence per player for South African adult level rugby at Club and professional levels (usually including university-based rugby players) is 23% (Shuttleworth-Edwards, Noakes, et al., 2008). For all three studies targeted for the purposes of the new analysis, namely the university level study of Shuttleworth-Edwards, Smith, and colleagues (2008) and the two high school level studies of Whitefield (2007) and Horsman (2010) (Studies 1, 2, and 3, respectively), the pre- versus post-season analyses were conducted on athletes who were not diagnosed with a concussion during the season under investigation (an exclusion criterion for the studies), in order to identify any effects of unreported concussive or subconcussive during the season in interaction with a prior history of exposure to such events.

Therefore, a common feature for the participants of the three studies in question is that there is a high probability that they would have been subjected to concussive or subconcussive events in previous seasons, yet at the time of the studies they would formally be considered to be free of subacute concussive symptoms having not been diagnosed with a concussion during the current season. This would be important in that another key feature of these participants is that they were all scholars who would be subjected to the speed-focused South African examination process within weeks of the post-season ImPACT neurocognitive evaluations. For best possible success in many such examinations, a critical element for a scholar is likely to be optimal capacity in the processing speed neurocognitive modality such as is called upon for performance on the ImPACT VMS test.

Hypothesis for the Combined Study

On the basis of the preceding rationale, the following specific hypothesis was posed for the purposes of the new combined analysis. It was expected that the pre- versus post-season prospective evaluation of the ImPACT VMS scores of rugby players, without a diagnosis of concussion over the current season, would serve to identify the presence of any clinically relevant cognitive vulnerability in the speeded visuomotor processing area for rugby players by virtue of their inability to benefit from a learning effect to the extent that would be possible for non-contact sport controls. It was proposed that this would be due to a long-term history of many years of participation in a sport with a high risk of concussive insults, in interaction with the effects of any unrecognized concussive and subconcussive events sustained during the most recent season of participation in the sport (i.e., hidden within season effects), none of which would be characteristic of the controls. It was acknowledged that it would not be possible to distinguish the extent to which each of these long-term or more immediate causative factors might be contributing to any failure to benefit from practice on the ImPACT VMS test in the Rugby group on the basis of the current group-based research. Nevertheless, it was considered that the overall post-season outcome would have relevance in its own right, in that it would reflect the cognitive status of high school and university rugby players, none of whom would have been formally identified as recently concussed, shortly before the onset of their end of year examinations.

Method

Details of the Data Pool

The data pool for the present analysis was taken from the three separate investigations of South African rugby playing scholars cited earlier of Shuttleworth-Edwards, Smith, and colleagues (2008), Whitefield (2007), and Horsman (2010) (designated Studies 1, 2, and 3, respectively), which were carried out under the coordination of the first author for masters and doctoral level dissertation purposes. In each study, a contact sport group without a diagnosable concussion over the targeted sports season was compared with demographically equivalent non-contact sport controls. The studies in question involved cross-sectional as well as prospective analyses, and the incorporation usually of a wider battery than only the ImPACT test, the details of which can be accessed elsewhere (Horsman, 2010; Smith, 2006; Whitefield, 2007). The samples involved University level rugby players (Shuttleworth-Edwards, Smith, et al., 2008; Study 1), high school contact sport players made up of 90% rugby and 10% soccer players (Whitefield, 2007; Study 2), and high school rugby players (Horsman, 2010; Study 3).

All three studies encompassed broadly equivalent methodological parameters. The objective of the present report was to focus purely on the prospective aspect of these studies including the investigation of pre-versus post-season seasonal differences for rugby players and likewise for non-contact sport controls on the ImPACT VMS component of the neurocognitive screening test alone, as a means of possibly differentiating between the groups. The reason for the inclusion of the ImPACT VMS composite in the absence of the other ImPACT composites was two-fold: (i) as indicated earlier, this was the only ImPACT composite in the overall outcome of these three studies consistently to reveal any indication of a greater practice effect for controls than rugby players thereby warranting its isolation for further directed hypothesis testing rather than exploratory research and (ii) as also discussed earlier, the ability to design a hypothesis driven focused study in respect of a single measure is considered methodologically advantageous in terms of circumventing Type II error in MTBI research (Demakis, 2006; Frencham et al., 2005).

Measure: Neurocognitive Test

The ImPACT VMS test (Lovell, 2004). The ImPACT test is a program that has been developed specifically for use in the sporting arena to evaluate concussion and was used in all three studies from which the pooled data for the present study were derived, in order to investigate the neurocognitive profiles of the study participants. Questions have been raised about the validity and reliability of the ImPACT programme, and therefore, the clinical utility of the programme and psychometric testing per se in the concussion context (Mayers & Redick, 2012; Randolph, McCrea, & Barr, 2005). However, these are contentious viewpoints that have been strongly challenged by those who consider the criticisms to be biased, empirically unrepresentative and/or based on assumptions that are insufficiently grounded in the tenets of modern clinical neuropsychology (Schatz, Kontos, & Elbin, 2012; Shuttleworth-Edwards, 2011). Notwithstanding these debates, the program continues to be used massively in concussion management programs world-wide, and many research studies can be cited to show that the test is a relatively reliable and valid instrument for use in the area of concussion per se, as well as when compared with other programs of this type (Iverson et al., 2003; Lau, Collins, & Lovell, 2011; Schatz, Pardini, Lovell, & Collins, 2006; Schatz & Sandel, 2013). There is a translation of the English version of the test into Afrikaans (a South African language derived from Dutch, the language spoken by early Dutch immigrants).

ImPACT 2.0 was used for Study 2, English version, and ImPACT 3.0 was used for Studies 1 and 3 in the English and Afrikaans versions, respectively. These two versions are equivalent in terms of the basic test constructs including the VMS subcomponent. In both versions, the VMS test composite score is made up of the average score from two interference tasks, including (i) the total number correct for matched responses to alternating colored geometric shapes (a timed interference task that recurs for each trial of the Xs and Os pattern recall task) and (ii) the average number counted correctly from 25 down to 1 (a timed interference task that recurs for each trial of the Three-Letter series recall task). Differences between the two versions of the test are mainly in respect of technical changes such as upgrading to a web-based system in the third edition. A subtle test-taking change on the X's and O's subtest that contributes to the VMS composite score includes the switch from unilateral two button clicking with a mouse (on 2.0) to bilateral key pressing for the same task on that subtest (on 3.0). It was not anticipated that there would be any differences in the overall test findings due to the use of these two versions of the test, in that all participants used similar desktop PCs, and highly equivalent proportions of contact sport participants and controls were exposed to the same technical and/or language versions of the tests. Controls made up 43%, 44%, and 43% of the samples of Studies 1, 2, and 3, respectively.

As the battery was designed to be used repeatedly over short intervals, the test consists of near infinite number of random forms by alternating the stimulus array with each module, with a view to minimizing practice effects (Maroon et al., 2000), although as indicated earlier a small practice effect has nevertheless been demonstrated specifically on the ImPACT VMS test (Iverson et al., 2003). There is a baseline test option and four follow-up options that can be applied for repeat testing occasions following the concussive injury.

For Study 1, the baseline version was used at pre-season and the first follow-up test was used at post-season; for Studies 2 and 3, the baseline test option was repeated at the post-season interval. Given that the test interval was of at least 7-month duration, it was not expected that it would make a substantial difference on any learning effect at the post-season evaluation as to whether the baseline test was re-administered or whether the first post-concussion test was administered. Due to the long test–retest interval, it was anticipated that it would be prior procedural learning on the VMS test, rather than the recall of specific content that would cognitively available to the participants. However, regardless of whether the baseline or alternative version was repeated, the objective of the present study was to provide a mechanism for the differentiation between the cerebrally at risk rugby players from controls by means of a learning effect, and with that in mind, the use of the same rather than an alternate version of the test (such as occurred for the bulk of the sample in the study) might be considered likely to have the superior heuristic potential. As to which one of these two mechanisms would yield the maximum amount of practice effect over a long test–retest period is a different research question with academic and clinical implications in its own right.

Test-Taking Procedure

In all three studies from which the data for the new analysis were extracted, pre-season testing took place between February and late March before any significant participation in rugby training and competitive play had began; post-season testing took place in late September and early October. The research teams for each study consisted of the chief researcher and a suitably trained research assistant who had a post-graduate degree in psychology. Before testing took place, the researchers and research assistants were trained in the administration of the ImPACT test. All testing took place under suitable test conditions without any noise or distractions. For Study 2, the ImPACT 2.0 test was put onto the school's network and participants were tested on PCs in groups of approximately 25 boys in the school computer laboratory. For Studies 1 and 3, all the ImPACT 3.0 web-based testing was conducted on PCs at the researchers' affiliated University computer laboratory, in groups not exceeding 25 participants.

Test results for the ImPACT test were automatically generated on the computer to produce five neurocognitive composite scores in the modalities of Verbal Memory, Visual Memory, VMS, and Reaction Time. Although all composites were included for analysis in the original three studies (as cited above), for the purposes of the present combined analysis, the VMS composite scores were isolated.

Sampling Procedure

Participants elicited for the study were South African male players of rugby (Studies 1 and 3) and 90% rugby and 10% soccer (Study 2), versus demographically equivalent South African non-contact sport controls, all of whom were engaged in full-time scholastic enterprises at either university or school level. In the case of each study, participants (contact sport and controls) were drawn from the same traditionally white English or Afrikaans medium institution, and the targeted cohorts in each instance were predominantly (at least 85% white), with no substantial differences in racial composition between contact sport players and controls. Further, all testees were attendees of relatively advantaged South African educational settings, and South African research has repeatedly demonstrated broad equivalence in neurocognitive functioning in both the verbal and non-verbal modalities regardless of race when the quality of education is equivalent (Shuttleworth-Edwards, Gaylard, & Radloff, 2013; Shuttleworth-Jordan, 1996).

In Study 1, all rugby players in the top two university teams in terms of seniority and sporting ability within the institution were approached for participation in the research; in Study 2, all Grade 12 players of contact and non-contact sports were targeted for inclusion in the research; in Study 3, all rugby players in the top two high school teams in terms of seniority and sporting ability within the institution were approached for participation in the research. Controls, drawn from the same institutions and top team sporting levels as the contact sports players in each case, were players exclusively of one or more non-contact sports including a mixture mainly of field hockey, cricket, and swimming athletes, for which there is a documented very low incidence of concussion relative to the rugby/football sports. It is important to differentiate ice hockey from field hockey for which there is a very high versus very low incidence of concussion, respectively. For example, in a high school study, the MTBI injury rates per 100 player-seasons were among the lowest for girls field hockey and boys baseball (a game that is broadly comparable with cricket) at 0.46 and 0.23, respectively, compared with 3.66 for American football (Powell & Barber-Foss, 1999). Research on South African top team boys from three high schools (Shuttleworth-Edwards, Ackermann, Beilinsohn, Border & Radloff, 2001, cited in Shuttleworth-Edwards et al., 2004), established an average incidence of only 0.4 (range 0–1) prior reported concussions for field hockey players compared with a history of 2.3 (range 0–7) prior concussions for rugby players. In a recent study, the majority of concussions among American high school athletes occurred in football (47.1%), whereas swimming did not rank among sports with a high incidence of concussion (Marar, McIlvain, Fields, & Comstock, 2012).

Approval for each of the three research investigations was obtained from the applicable departmental and university ethics committees under whose auspices the studies were conducted, as well as the Eastern and Western Cape Education Departments and heads of the relevant schools and university sections in order to gain access to sports players. All parents of the targeted school boys were sent letters informing them of the nature and purpose of the research and all were given the option of withdrawing their child from the study by signing an attached waiver form. No parents withdrew their children from the studies. School and university participants were briefed about the study and provided written consent for the use of their de-identified data in group research. Participation in the research was voluntary, and participants were free to withdraw from the project at any stage of the procedure.

A biographical questionnaire elicited medical information for sampling purposes, as well as a record of all past concussive injuries warranting even temporary cessation of the related activity, or follow-up medical management. For the purposes of this research, concussion was defined broadly according to the practice parameter of the American Academy of Neurology (AAN, 1997), as any alteration in mental status experienced as a result of trauma that may or may not include loss of consciousness. Participants with a history of moderate to severe head injury, delineated by loss of consciousness greater than 30 min (Kibby & Long, 1996), were excluded from the study. It can, therefore, be assumed that the documented past history of concussions reported by the participants fell within the milder spectrum of TBI severity, such as is typical of the sports-related concussion (Lovell, Iverson, Collins, McKeag, & Maroon, 1999). Further, to ensure that the end of season testing was conducted on an apparently clinically normal group in terms of the subacute effects of concussion, all individuals who received a diagnosis of a concussion of any severity during the preceding season were excluded from each of the three studies. A total of 18 concussed individuals were removed from the dependent pre- versus post-season analyses on that account. The overall concussion incidence was 11% of the total sample numbers available for the three studies at the pre-season interval prior to test-taking attrition at post-season. Additional exclusion criteria were a history of alcohol or substance abuse, neurological disease, learning disability, and attention deficit hyperactivity disorder.

Sampling Characteristics: Studies 1, 2, and 3

A number of sampling characteristics pertaining to Studies 1, 2, and 3 have relevance for the combined data pool, as follows. In all studies, participants' ages were documented at the time of pre-season baseline testing, and the calculation of years of education was made according to the number of years it normally takes to achieve a level. For Studies 1 and 3, there was some within study variation in levels of education, whereas in study 2 participants were restricted to Grade 12 and this variable was captured in the database as 12.00 years of education for all participants. For the two school studies (Studies 2 and 3), independent t-tests revealed no significant differences for age or education between the Rugby/Soccer and Rugby groups versus their equivalent Non-Contact sport groups, respectively (p > .05 in all instances). However, for the university study (Study 1; Shuttleworth-Edwards, Smith, et al., 2008), an independent t-test indicated that the average age and the educational level for the Rugby group was lower than those of the Non-Contact group (p < .01 in each instance) with a 3.54 mean age difference and a 1.56 mean educational level difference between groups. Support for unlikely confounding effects in this regard, however, was provided by a correlational analysis between age, education, and test scores at the pre- and post-season intervals that revealed no significant relationships for 27 of the 28 analyses, with low correlations ranging from r = .045 to r = −.280 (p > .05 in all cases).

Finally, for each of the three studies, an estimate of IQ was incorporated to serve as a control variable, and in all instances, independent t-tests revealed no difference for estimated IQ between the Rugby and Rugby/Soccer contact sport groups and their equivalent Non-Contact control groups (p = .417, p = .251, and p = .246 for studies 1, 2, and 3, respectively). Different measures were used in each study for the IQ estimate, all of which are considered legitimate for this purpose (Lezak et al., 2004), including the combined score of the Wechsler Adult Intelligence Scale-III (WAIS-III) Vocabulary and Picture Completion tests (Wechsler, 1997; Study 1), the Ravens' Standard Progressive Matrices (Abdel-Khalek & Raven, 2006; Study 2), and the WAIS-III Vocabulary test (Wechsler, 1997; Study 3).

Sampling Characteristics: Combined Analysis

The combined analysis involved data derived from the pre- versus post-season ImPACT test data for all participants of Studies 1, 2, and 3, except for a number of additional exclusions. As indicated above, for high school Studies 2 and 3, there was no significant difference on t-test comparisons of age and education between the Rugby/Soccer and Rugby groups versus their equivalent Non-Contact control groups, but significant differences in this regard were identified for the university Study 1. Therefore, it was decided to remove the three oldest participants from Study 1, who were also among those with the most years of post-graduate education, in order to ensure that the comparative groups for the combined analysis were clearly equivalent for the age and education variables. Furthermore, in the interests of investigating effects purely in relation to rugby, the nine soccer players in Study 2 were excluded from the new analysis. On pooling the remaining data, the combined sample yielded the following sampling characteristics: contact sport n = 145 (mean age = 17.58, SD = 1.35; mean education = 11.72, SD = 1.26) and Controls n = 106 (mean age = 17.87, SD = 1.98; mean education = 11.91, SD = 1.37; Table 2). Independent t-tests revealed no significant difference for age or education between the contact sport and Control groups (p = .173 and p = .279, respectively; Table 2).

Table 2.

Combined study: Comparative independent t-test comparisons of pooled data for university and school level Rugby versus Non-Contact sports group including age, education, and number of prior concussions

 Rugby (n = 145) Non-Contact (n = 106) t-value Effect size
 
p-value 
Mean (SDMean (SDδ-value 95% CI 
Age 17.58 (1.35) 17.87 (1.98) −1.365 −0.18 −0.43, 0.08 .173 
Education 11.72 (1.26) 11.91 (1.37) −1.085 −0.15 −0.40, 0.11 .279 
Number of concussions 1.20 (1.36) 0.27 (0.72) 6.392 0.82 0.56, 1.08 <.001** 
 Rugby (n = 145) Non-Contact (n = 106) t-value Effect size
 
p-value 
Mean (SDMean (SDδ-value 95% CI 
Age 17.58 (1.35) 17.87 (1.98) −1.365 −0.18 −0.43, 0.08 .173 
Education 11.72 (1.26) 11.91 (1.37) −1.085 −0.15 −0.40, 0.11 .279 
Number of concussions 1.20 (1.36) 0.27 (0.72) 6.392 0.82 0.56, 1.08 <.001** 

Notes: Final contact sport group was 100% rugby players (following removal of nine soccer players from Study 2), equivalent for age and education with the final Non-Contact sports group (following the removal of the three oldest non-contact sports participants from Study 1).

**p < .001.

The step of removing 12 participants from the original Studies 1 and 2 was considered appropriate for the following reasons. First, it was considered preferable to remove outliers on the age and education variables in preference to making statistical corrections to the data to control for bias due to possible age and education effects post hoc and thereby attract criticism for poorly matched comparative groups. Second, in view of differences in the nature of the games of soccer and rugby, it was considered preferable to procure the ideal of a pure rugby group that was not confounded by the presence of nine soccer players. It is well documented that soccer has a much lower incidence of reported concussion than American football and Rugby Union at both the youth and adult levels of play (Cassidy, Carroll, Peloso Borg, & von Holst, 2004; Powell & Barber-Foss, 1999). Generally, studies investigating concussion effects on mixed sports groups is considered less than optimal in this research area (Shuttleworth-Edwards & Whitefield-Alexander, 2007). Overall, therefore, it is contended that the removal of these 12 participants for the purposes of the combined study was appropriate data cleaning for the purposes of the newly devised study, providing a more refined research investigation from the methodological point of view, in terms of equivalence for age and education, and a homogenous contact sport group, being sampling weaknesses applicable to the original Studies 1 and 2, respectively.

It was not possible directly to compare IQ estimates for the combined sample of rugby and non-contact sports players as these estimates were based on different indexes within each of the separate studies. However, it can be presumed that the pooled comparative groups were equivalent on this parameter as well in that, as indicated above, on the basis of independent t-test analyses, the comparative groups within each study in isolation were well-matched for estimated IQ (p = .417, p = .251, and p = .246, for studies 1, 2, and 3, respectively). In light of the elimination of some participants from Studies 1 and 2 for the pooled analysis, additional independent t-test analyses served to confirm that the reduced comparative groups within these studies were still equivalent for estimated IQ as follows: Study 1, Vocabulary/Picture Completion Rugby mean = 10.65 (SD = 1.74) versus Control mean = 10.03 (SD = 2.03; p = .308); Study 2, Vocabulary Rugby mean = 12.30 (SD = 1.96) versus Control mean = 12.66 (SD = 2.25; p = .316).

Independent t-tests revealed a highly significant difference in the entire past history of number of concussions reported between the pooled Rugby and Non-Contact sports groups with means of 1.20 (SD = 1.36) and 0.27 (SD = 0.72) concussions, respectively, p < .001 (Table 2), accompanied by a high effect size of clinical relevance (d = 0.82, CI: 0.56, 1.08). A Pearson's chi-square analysis indicated that 33% of the combined contact group sample reported having sustained 2+ prior concussions versus only 3% of controls. Thirteen rugby players reported three prior concussions, seven reported four prior concussions, three players reported five prior concussions, and 1 player reported six prior concussions. Importantly, these concussion figures are based on self-reported historical information, rather than observed concussion incidence in a focused prospective study. They are likely, therefore, to be an “underestimate” of concussions incurred, due to possible poor recall of these events, and the well-documented tendency for underreporting of concussions among athletes involved in contact sports due to ignorance about the injury (Delaney, Abuzeyad, Correa, & Foxford, 2005; Delaney, Al-Kashmiri, Drummond, & Correa, 2008).

Finally, it was considered important to consider the validity of brain damage inferences when taking into consideration the reliability of the test outcome. The Impulse Control composite score can be used as a test-taking validity indicator on the ImPACT test, with a cut-off point of >20 raising the possibility of an unreliable test result (Lovell, 2004). However, this is not necessarily the case. An inflated Impulse Control score may also be indicative of an erratic performance style that is genuinely characteristic of the testee when undertaking the test, with associated neuropathological implications. Therefore, unless poor effort is particularly suspected, it is arguably preferable not to use a high Impulse Control score as an exclusion criterion for a brain injury study, in the same way as it would be of dubious scientific value to remove very poor scores on any of the test parameters. In the present series of studies, none of the samples were made up of professional sportsmen who would stand to benefit from “sandbagging,” and nor would the control participants, whose participation was voluntary, have had any reason purposefully to perform suboptimally. Accordingly, no participants were excluded on the basis of an Impulse Control cut off score, on the assumption that for the most part the test protocols were likely to represent genuine performances.

Commensurate with this supposition, independent t-test analyses of Impulse Control scores revealed no significant differences between the combined Rugby and Non-Contact control group at either of the pre- or of the post-season test intervals (p = .336 and p = .360, respectively). Dependent t-test analyses similarly revealed no significant differences for either the Rugby or the Control group at pre-season versus post-season (p = .458 and p = .541, respectively). Mean scores for rugby and controls at the pre- versus post-season intervals, respectively, were: 7.66 (SD = 6.50), 8.21 (SD = 7.73); 6.89 (SD = 6.00), 7.37 (SD = 6.30). The overall low mean scores of around 6 and 7 (i.e., substantially lower scores than the cut-off point of >20 for an invalid protocol) were highly equivalent for rugby and controls. Similarly, the standard deviations in all instances were highly equivalent across the comparative sports groups at around 6 and 7 integers, implicating the acceptable variability of scores that were on average well below the >20 cutoff point. The implication is that poor effort or “sandbagging” (i.e., the purposeful attempt to achieve a depressed baseline score) was not a problem per se in this study, and nor did this variable serve as a differentiating feature between the two comparative groups that might serve to explain the presence or the absence of practice effects.

In contrast, the test–retest reliability of the ImPACT VMS composite served to differentiate between the rugby and controls. The correlation coefficients, when considered in light of other test–retest indices across 1 year that have been reported on the ImPACT VMS test (Schatz et al., 2012), were somewhat poor for the Rugby group (r = .5934) compared with reasonably good reliability for the Control group (r = .7917). Overall, the indication from the test–retest correlational coefficients, therefore, is of substantially less stability in the cerebrally at risk rugby group. This was not considered to be a reason to invalidate the test data in the Rugby group, in that greater inter- and intra-individual variability on cognitive testing is a well-documented indicator of cerebral dysfunction in general (Stuss, Murphy, Binn, & Alexander, 2003; Tales et al., 2012), as well as a feature of the cerebral dysfunction that occurs in association with normal ageing (Dykiert, Der, Starr, & Deary, 2012; Jordan, 1997).

Results

Dependent t-test comparisons of the ImPACT VMS scores at pre- versus post-season for the newly combined sample of Rugby versus Non-Contact sports groups (Table 3) revealed a highly significant improvement for the non-contact controls (p < .001), with a small to medium clinically relevant effect size that did not contain zero in the confidence interval (d = 0.39, CI: −0.58, −0.19), in contrast to the absence of significant improvement for rugby players (p = .695). A repeated-measures ANOVA showed a significant season effect for ImPACT VMS scores (F1,249 = 7.74, p = .006), and a significant interaction effect for sport (F1,249 = 4.91, p = .028). The ImPACT VMS results for season and sport are illustrated in Fig. 1.

Table 3.

Combined sample: Dependent t-test comparisons of ImPACT VMS scores at the pre- versus post-season intervals for the Rugby and Non-Contact sports groups

Source Sample size Pre-season Mean (SDPost-season Mean (SDt-value Effect size
 
p-value 
δ-value 95% CI 
Rugby VMS n = 145 36.57 (6.80) 36.77 (6.62) −0.393 −0.03 −0.20, 0.13 .695 
Non-Contact VMS n = 106 36.65 (7.24) 38.39 (6.70) −3.967 −0.39 −0.58, −0.19 <.001*** 
Source Sample size Pre-season Mean (SDPost-season Mean (SDt-value Effect size
 
p-value 
δ-value 95% CI 
Rugby VMS n = 145 36.57 (6.80) 36.77 (6.62) −0.393 −0.03 −0.20, 0.13 .695 
Non-Contact VMS n = 106 36.65 (7.24) 38.39 (6.70) −3.967 −0.39 −0.58, −0.19 <.001*** 

Notes: VMS = Visual Motor Speed. A repeated-measures ANOVA showed a significant season effect for ImPACT VMS scores (F1,249 = 7.74, p = .006**) and a significant interaction effect for sport (F1,249 = 4.91, p = .028*).

*p < .05.

**p < .01.

***p < .001.

Fig. 1.

Combined study: Pre- versus post-season ImPACT VMS scores (±SE) for university and high school Rugby players versus Non-Contact sport controls.

Fig. 1.

Combined study: Pre- versus post-season ImPACT VMS scores (±SE) for university and high school Rugby players versus Non-Contact sport controls.

Discussion

The current study sought to determine whether there was evidence for deleterious effects of repetitive concussive and subconcussive brain injury on speeded visuomotor function as revealed by outcome on the computerized ImPACT VMS composite score, among high school and university players of Rugby Union. The investigation took the form of isolating the data for the ImPACT VMS composite score from three more wide-ranging neurocognitive prospective analyses, in order to conduct a focused combined analysis of the ImPACT VMS data alone by means of a repeat measures group-by-season analysis at the pre- and post-season intervals for rugby players versus non-contact sports controls. The phenomenon of a practice effect can be seen to have important diagnostic utility in the identification of cerebral pathology (Duff, 2012; Lezak et al., 2004). The investigative challenge of the present study, therefore, was to see whether by combining the results of several studies to produce a reasonably large sample number and by targeting a single functional modality in the interests of protecting against Type II error (Demakis, 2006; Frencham et al., 2005), the phenomenon of practice might serve significantly to differentiate rugby from non-contact sports players on this particular test.

Repeat-measures analyses of season effects indicated that the performance of the rugby playing school and university participants on the ImPACT VMS composite remained relatively stable between the pre- versus post-testing intervals with no significant effect, whereas in contrast controls showed a highly significant improvement compared with the pre-season test result, accompanied by a clinically relevant medium effect size (Table 3, Fig. 1). The finding of an improvement for controls is commensurate with the fact that practice effects have been identified for this particular composite score on the ImPACT test (Iverson et al., 2003), as well as with the robust demonstration of sensitivity of commonly employed processing speed tasks to a learning effect (Lezak et al., 2004; Strauss et al., 2006). In contrast, the total absence of a practice effect for rugby players when it is expected, supported by a significant group by season interaction effect (Table 3, Fig. 1), may be considered clinically meaningful implicating the presence of persistent deleterious effects on the ImPACT VMS composite that might be attributable to brain injury sustained over years of playing rugby in combination with an overlay of any such effects sustained by players over the most recent rugby season.

Although the ImPACT VMS practice effect was not large in the study conducted by Iverson and colleagues (2003) over a very short test–retest interval on different versions of the test, the practice effect demonstrated in the present study is clearly substantial. At a cursory glance, this contrasting result might arguably be due to the fact that a large proportion of the combined sample in the present research completed the same version of the test twice rather than an alternate version. However, on fine analysis, the content recall factor within the interference tasks that make up the VMS composite score is minimal (being the total number correct for matched responses to alternating colored geometric shapes over several trials, and the average number counted correctly from 25 down to 1 over several trials). Therefore, it is extremely unlikely that there would have been any retrieval of specific test material after the long test–retest interval of around 7–8 months. It appears more likely that the very short test interval of on average less than 1 week in the Iverson and colleagues study might have been confounded by the presence of a proactive interference effect of competing new stimuli in one or both aspects of memory tasks that immediately precede the VMS interference tasks, thereby presenting a more taxing test situation overall that might impact negatively on the speed at which the two interference tasks are completed. In turn, this would have obscured the benefit of procedural learning that might otherwise have occurred on these tasks in the short test–retest interval cohort, compared with the long test–retest interval of the present cohort. Importantly, the extent to which each of the two interference components that go to make up the VMS test is responsible for the practice effect has not been explored in the present research and would be worthy of future investigation.

Alternatively, or in addition to the above postulate in terms of differential learning effects that might occur between the short- and long-term test intervals, the larger practice/learning effect in the present study might be due to differential sampling effects. It is possible, for instance, that the South African participants overall were of a higher level of intelligence than the Iverson and colleagues sample, being made up exclusively of scholars from particularly advantaged South African educational institutions. Research has indicated a tendency for the cognitive test performance of such South African cohorts to be elevated in relation to U.S. norms (Shuttleworth-Edwards et al., 2013).

Following participant exclusions from the original three studies (Horsman, 2010; Smith, 2006; Whitefield, 2007), the comparative groups for the combined sample can be considered well-controlled for the crucial variables of age, education, IQ, and sport affiliation. It is unlikely, therefore, that this marked discrepancy in performance between the two groups can be attributed to pre-existing characteristics in that regard. Moreover, the absence of a practice effect for the at risk rugby group on the ImPACT VMS test is commensurate with a gathering set of paper-and-pencil test outcome from pre- versus post-season analyses that demonstrate reduced learning capacity on the processing speed function amongst young adult and/or university level rugby/football athletes (Barth et al., 1989; Shuttleworth-Edwards, Smith, et al., 2008; Shuttleworth-Jordan et al., 1993). In turn, these studies serve to increase the likely generalizability of the outcome on the present analysis to university athletes as well as high school athletes. This is important given the limiting factor of the very small subset of university level participants that was included in the overall sample of predominantly high school participants.

The obvious differentiating feature between the two groups of the newly combined sample is the clinically relevant, significantly larger number of reported concussions for rugby players relative to controls (Table 2), despite the likely massive underreporting of concussions among athletes involved in contact sports due to ignorance about the injury (Delaney et al., 2005, 2008), and notwithstanding the consequences of multiple unreported subconcussive episodes that occur in association in a rugby/football sport (Killam et al., 2005; Rutherford, Stephens, & Potter, 2003). Clearly, from the chi-square analysis, there is a substantial proportion of the Rugby group who had sustained in excess of two and three concussions, a phenomenon that was not true for the Control group most of whom had never sustained a single concussion.

An additional differentiating feature of note between the comparative groups was the relatively poor test–retest reliability on ImPACT VMS performance for the Rugby group with a correlation coefficient of around r = .6 compared with reasonably good reliability for the Control group of around r = .8. Neither of these correlation coefficients, taken together with suitably low mean scores on the ImPACT Impulse Control validity indicator for both the Rugby and Control groups, implicate a completely erratic pre- versus post-test analysis that would invalidate the study as a whole. However, the indication of less stability in cognitive test performance over time for the Rugby group compared with the controls (i.e., enhanced intra-individual variability in the former group) is commensurate with the expected performance of a cerebrally compromised group, being typically described in brain damaged populations, including those with Mild Cognitive Impairment (MCI) (Stuss et al., 2003; Tales et al., 2012), and in association with normal aging (Dykiert et al., 2012; Jordan, 1997). Importantly, the sign of a relatively unstable performance over a pre- versus post-season test period for the Rugby group within the present study is conceptually coherent with the absence of a learning effect among the Rugby group in this study over the same test interval relative to Controls. Its occurrence adds weight to the proposed likely presence of neurocognitive vulnerability in the cerebrally at risk Rugby group in association with years of participation in the sport. Clearly, the relevance of using test–retest data in rugby/football studies as a possible diagnostic indicator of the presence of brain impairment, such as was investigated for the present study on differential practice effects, is an issue worthy of a full review and further investigation in its own right.

Accordingly, from a neuropsychological perspective, there is a compelling implication arising out of the present research of neurocognitive vulnerability among a significant proportion of the Rugby Group due to the effects of brain-related insults that typically occur over years of participation in the sport, and/or the effects of more recent insults sustained over the season and/or a combination of both of these indistinguishable factors on the basis of the current research. The vulnerability may implicate executive dysfunction and/or slowed processing speed per se in association with the fronto-temporal diffuse brain damage that typically accompanies the acceleration–deceleration closed head injury. In turn, the combined effect of subtle executive dysfunction and/or slow processing speed among the concussion prone rugby players versus controls can account for a smaller than normal practice effect in the Rugby group. It is of some considerable concern, therefore, regardless of whether the deleterious effects are of chronic or subacute origin or a combination of both of these factors, that there are likely to be a significant number of rugby players embarking on end-of-year examinations at school and university with less than optimal neurocognitive capacity in respect of one or both of these functional modalities, which are called upon for prime success in the examination system.

Clinical Implications and Indications for Further Research

Importantly, the outcome of the present research has implications for the interpretation of recovery of the ImPACT VMS composite in the management of the concussive injury in the individual case, and the same issue is likely to apply to a greater or lesser extent for processing speed tasks in general that tend to characterize the available computerized neurocognitive test programs developed for use within the sports concussion arena (Schnirring, 2001). The critical point being that where a significant practice effect is anticipated, a return to the baseline level only on a post-concussion follow-up test may provide a valuable diagnostic indicator that full recovery has not occurred. However, the failure to improve with practice may be due not only to the subacute effects of the most recent concussion in question that may continue to resolve, but could be the permanent result of residual dysfunction in association with years of participation in rugby, or the combined effect of both of these factors.

Basically in this situation, for the purposes of making a valid recommendation about recovery from the current injury and safe return to play, the presence of subacute versus chronic effects on the absence of any practice effect need to be unraveled. The task demands a sophisticated level of interpretation by a suitably trained neuropsychologist who can pay due attention to the scatter available on a multifunction test such as ImPACT, the outcome on all prior baseline evaluations if these exist, concussion history, current symptom reports, scholastic background, and current scholastic performance since the incident, and so on. Clearly, the complicating factor of procedural learning to which processing speed tasks are prone is a useful diagnostic aid within the context of a multifunction test profile when well-managed. However, it affirms the frequently expressed concern that a simple ‘yes/ no’ actuarial solution to the presence or absence of brain vulnerability, that is provided simplistically especially on the basis of a single computerized index score by the non-psychologist, incurs grave risk of misdiagnosis in the sports concussion arena (Echemendia, Herring, & Bailes, 2009; Moser et al., 2007; Shuttleworth-Edwards & Border, 2002).

The results of the present study give rise to a number of unanswered questions in need of further research, which will impact on the interpretation of test outcome in the clinical case. For instance, it is not clear whether a significant practice effect might be less in evidence in the instance of a retest occurring very shortly (i.e., within days) after baseline testing due to a proactive interference effect, compared with a retest that is called for many months later toward the end of season where it might be expected. The extent to which there will be differences in the amount of practice incurred depending on whether or not the same version of the test is repeated or an alternate version of the test is used is also an unknown factor. Finally, the degree to which the absence of ability to improve with practice on a processing speed task such as ImPACT VMS implicates a level of neurocognitive dysfunction that will impact on actual scholastic performance to a significant degree is unknown. Answers to these queries can be achieved via case-based investigations on a longitudinal basis across the high school and university years, made possible by the growing modern trend to implement routine neurocognitive screening with computerized testing to facilitate the medical management of the concussive injury.

Funding

This research was supported by a Rhodes University Joint Research Council annual research grant and National Research Foundation (NRF) incentive funding (38206 551 105 and 6956 551 306, respectively).

Conflict of Interest

The first and third authors are involved in the use of the ImPACT test in South Africa and the UK for clinical and research purposes.

References

Abdel-Khalek
A.
Raven
J.
Normative data from the standardization of Raven's Standard Progressive matrices in Kuwait in an international context
Social Behaviour and Personality
 , 
2006
, vol. 
34
 (pg. 
169
-
180
)
American Academy of Neurology.
Practice parameter: The management of concussion in sports (summary statement). Report of the Quality Standards Subcommittee
Neurology
 , 
1997
, vol. 
48
 (pg. 
581
-
585
)
Autti
T.
Sipila
L. A.
Autti
H.
Salonen
O.
Brain lesions in players of contact sports
The Lancet
 , 
1997
, vol. 
349
 pg. 
1144
 
Barth
J. T.
Alves
W. M.
Ryan
T. V.
Macciocchi
S. N.
Rimel
R. W.
Jane
J. A.
, et al.  . 
Mild head injury in sports: Neuropsychological sequelae and recovery of function
Mild head injury
 , 
1989
Oxford
Oxford University Press
(pg. 
257
-
275
H. S. Levin, H. Eisenberg, & A. Benton
Brandt
J.
2005 INS presidential address: Neuropsychological crimes and misdemeanors
The Clinical Neuropsychologist
 , 
2007
, vol. 
21
 (pg. 
553
-
568
)
Cassidy
J. D.
Carroll
L. J.
Peloso
P. M.
Borg
J.
von Holst
H.
Incidence, risk factors and prevention of mild traumatic brain injury: Results of the WHO collaborating centre task force on mild traumatic brain injury
Journal of Rehabilitation Medicine
 , 
2004
, vol. 
Suppl. 43
 (pg. 
28
-
60
)
Collins
M. W.
Grindell
S. H.
Lovell
M. R.
Dede
D. E.
Moser
D. J.
Phalin
B. R.
, et al.  . 
Relationship between concussion and neuropsychological performance in college football players
Journal of the American Medical Association
 , 
1999
, vol. 
282
 (pg. 
964
-
970
)
Delaney
J. S.
Abuzeyad
F.
Correa
J. A.
Foxford
R.
Recognition and characteristics of concussions in the emergency department population
Journal of Emergency Medicine
 , 
2005
, vol. 
29
 (pg. 
189
-
197
)
Delaney
J. S.
Al-Kashmiri
A.
Drummond
R.
Correa
J. A.
The effect of protective headgear on head injuries and concussion in adolescent football (soccer) players
British Journal of Sports Medicine
 , 
2008
, vol. 
42
 (pg. 
110
-
115
)
Demakis
G. J.
Meta-analysis in neuropsychology: Basic approaches, findings, and applications
The Clinical Neuropsychologist
 , 
2006
, vol. 
20
 (pg. 
10
-
26
)
Duff
K.
Evidence-based indicators of neuropsychological change in the individual patient: Relevant concepts and methods
Archives of Clinical Neuropsychology
 , 
2012
, vol. 
27
 (pg. 
248
-
261
)
Duff
K.
Beglinger
L.
Schultz
S.
Moser
D. J.
McCaffrey
R. J.
Haase
R. F.
, et al.  . 
Practice effects in the prediction of long-term cognitive outcome in three patient samples: A novel prognostic index
Archives of Clinical Neuropsychology
 , 
2007
, vol. 
22
 (pg. 
15
-
24
)
Dykiert
D.
Der
G.
Starr
J.
Deary
I.
Age differences in intra-individual variability in simple and choice reaction time: Systematic review and meta-analysis
PLoS One
 , 
2012
, vol. 
7
 pg. 
e45759
 
Echemendia
R.
Herring
S.
Bailes
J.
Who should conduct and interpret the neuropsychological assessment in sports-related concussion
British Journal of Sports Medicine
 , 
2009
, vol. 
43
 (pg. 
i32
-
i35
)
Farace
E.
Ferree
R. M.
Hollier
J. A.
Barth
J. T.
Shaffrey
M. E.
Trails A: Neurocognitive effect of previous concussions in woman's rugby sample
Journal of the International Neuropsychological Society
 , 
2003
, vol. 
9
 pg. 
207
 
Frencham
K. A. R.
Fox
A. M.
Maybery
M. T.
Neuropsychological studies of mild traumatic brain injury: A meta-analytic review of research since 1995
Journal of Clinical and Experimental Neuropsychology
 , 
2005
, vol. 
27
 (pg. 
334
-
351
)
Gaetz
M.
Goodman
D.
Weinberg
H.
Electrophysiological evidence for the cumulative effects of concussion
Brain Injury
 , 
2000
, vol. 
14
 (pg. 
1077
-
1088
)
Gardner
A.
Shores
E.
Batchelor
J.
Reduced processing speed in rugby union players reporting three or more previous concussion
Archives of Clinical Neuropsychology
 , 
2010
, vol. 
25
 (pg. 
174
-
181
)
Grindel
S. H.
Lovell
M. R.
Collins
M. W.
The assessment of sport-related concussion: The evidence behind neuropsychological testing and management
Clinical Journal of Sport Medicine
 , 
2001
, vol. 
11
 (pg. 
134
-
143
)
Guskiewicz
K. M.
Marshall
W. W.
Bailes
J.
McCrea
M.
Cantu
R. C.
Randolph
C.
, et al.  . 
Association between recurrent concussion and late cognitive impairment in retired professional football players
Neurosurgery
 , 
2005
, vol. 
57
 (pg. 
719
-
726
)
Horsman
M.
Concussion in contact sport: Investigating the neurocognitive profile of Afrikaans adolescent rugby players
2010
Grahamstown, South Africa
Rhodes University
 
Masters thesis
Iverson
G. L.
Gaetz
M.
Lovell
M. R.
Collins
M. W.
Cumulative effects of concussion in amateur athletes
Brain Injury
 , 
2004
, vol. 
18
 (pg. 
433
-
443
)
Iverson
G. L.
Lovell
M. R.
Collins
M. W.
Interpreting change on ImPACT following sport concussion
The Clinical Neuropsychologist
 , 
2003
, vol. 
17
 pg. 
467
 
Iverson
G. L.
Lovell
M. R.
Collins
M. W.
Validity of ImPACT for measuring processing speed following sports-related concussion
Journal of Clinical and Experimental Neuropsychology
 , 
2005
, vol. 
27
 (pg. 
683
-
689
)
Jordan
A.
The Shuttle Effect. The development of a model for the prediction of variability in cognitive test performance across the adult life span
1997
Grahamstown, South Africa
Department of Psychology, Rhodes University
 
Doctoral dissertation
Kibby
M. Y.
Long
C. J.
Minor head injury: Attempts at clarifying the confusion
Brain Injury
 , 
1996
, vol. 
10
 (pg. 
159
-
186
)
Killam
C.
Cautin
R. L.
Santucci
A. C.
Assessing the enduring residual neuropsychological effects of head trauma in college athletes who participate in contact sports
Archives of Clinical Neuropsychology
 , 
2005
, vol. 
20
 (pg. 
599
-
611
)
Knight
R. G.
The neuropsychology of degenerative brain diseases
 , 
1992
Hillsdale, NJ
Erlbaum
Kutner
K. C.
Erlanger
D. M.
Tsai
J.
Jordan
B. D.
Relkin
N. R.
Lower cognitive performance of older football players possessing apolipoprotein E epsilon 4
Neurosurgery
 , 
2000
, vol. 
47
 (pg. 
651
-
657
)
Lau
B.
Collins
M.
Lovell
M.
Sensitivity and specificity of subacute computerized neurocognitive testing and symptom evaluation in predicting outcomes after sports-related concussion
American Journal of Sports Medicine
 , 
2011
, vol. 
39
 (pg. 
1209
-
1216
)
Lezak
M. D.
Howieson
D. B.
Loring
D. W.
Neuropsychological assessment
 , 
2004
4th ed.
Oxford
Oxford University Press
Lovell
M. R.
ImPACT version 3.0 clinical user's manual
 , 
2004
Pittsburgh, USA
ImPACT
Lovell
M. R.
Iverson
G. L.
Collins
M. W.
McKeag
D.
Maroon
J. C.
Does brief loss of consciousness predict neuropsychological decrements after concussion?
Clinical Journal of Sport Medicine
 , 
1999
, vol. 
9
 (pg. 
193
-
198
)
Marar
M.
McIlvain
N.
Fields
S.
Comstock
R.
Epidemiology of concussions among United States high school athletes in 20 sports
American Journal of Sports Medicine
 , 
2012
, vol. 
40
 (pg. 
747
-
755
)
Maroon
J. C.
Lovell
M. R.
Norwig
J.
Podell
K.
Powell
J. W.
Hartl
R.
Cerebral concussion in athletes: Evaluation and neuropsychological testing
Neurosurgery
 , 
2000
, vol. 
47
 (pg. 
659
-
669
)
Matser
J. T.
Kessels
A. G.
Jordan
B. D.
Lezak
M. D.
Troost
J.
Chronic traumatic brain injury in professional soccer players
Neurology
 , 
1998
, vol. 
51
 (pg. 
791
-
796
)
Matser
J. T.
Kessels
A. G.
Lezak
M. D.
Jordan
B. D.
Troost
J.
Neuropsychological impairment in amateur soccer players
Journal of the American Medical Association
 , 
1999
, vol. 
282
 (pg. 
971
-
973
)
Matser
J. T.
Kessels
A. G.
Lezak
M. D.
Troost
J.
A dose-response relation of headers and concussions with cognitive impairment in professional soccer players
Journal of Clinical and Experimental Neuropsychology
 , 
2001
, vol. 
23
 (pg. 
770
-
774
)
Mayers
L.
Redick
T.
Clinical utility of ImPACT assessment for postconcussion return-to-play counselling: Psychometric issues
Journal of Clinical and Experimental Psychology
 , 
2012
, vol. 
34
 (pg. 
235
-
242
)
Micheli
L. J.
Riseborough
E. M.
The incidence of injuries in rugby football
Journal of Sports Medicine
 , 
1974
, vol. 
2
 (pg. 
93
-
97
)
Moser
R. S.
Iverson
G. L.
Echemendia
R. J.
Lovell
M. R.
Schatz
P.
Webbe
F. M.
, et al.  . 
NAN position paper. Neuropsychological evaluation in the diagnosis and management of sports concussion
Archives of Clinical Neuropsychology
 , 
2007
, vol. 
22
 (pg. 
909
-
916
)
Moser
R. S.
Schatz
P.
Jordan
B. D.
Prolonged effects of concussion in high school athletes
Neurosurgery
 , 
2005
, vol. 
17
 (pg. 
91
-
100
)
Pettersen
J. A.
Skelton
R. W.
Glucose enhances long-term declarative memory in mildly head-injured varsity rugby players
Psychobiology
 , 
2000
, vol. 
28
 (pg. 
81
-
89
)
Powell
J. W.
Barber-Foss
K. D.
Traumatic brain injury in high school athletes
Journal of the American Medical Association
 , 
1999
, vol. 
282
 (pg. 
958
-
963
)
Randolph
C.
McCrea
M.
Barr
W. B.
Is neuropsychological testing useful in the management of sport-related concussion?
Journal of Athletic Training
 , 
2005
, vol. 
40
 (pg. 
139
-
154
)
Rutherford
A.
Stephens
R.
Potter
D.
The neuropsychology of heading and head trauma in association football (soccer): A review
Neuropsychology Review
 , 
2003
, vol. 
13
 (pg. 
153
-
179
)
Rutherford
A.
Stephens
R.
Potter
D.
Fernie
G.
Neuropsychological impairment as a consequence of football (soccer) play and football heading: Preliminary analyses and report on university footballers
Journal of Clinical and Experimental Neuropsychology
 , 
2005
, vol. 
27
 (pg. 
299
-
319
)
Savica
R.
Parisi
J.
Wold
L.
Josephs
K.
Ahlskog
J.
High school football and risk of neurodegeneration: A community-based study
Mayo Clinic Proceedings
 , 
2012
, vol. 
87
 (pg. 
335
-
340
)
Schatz
P.
Kontos
A.
Elbin
R.
Response to Mayers and Redick: “Clinical utility of ImPACT assessment for postconcussion return-to-play counseling: Psychometric issues”
Journal of Clinical and Experimental Neuropsychology
 , 
2012
, vol. 
38
 (pg. 
428
-
434
)
Schatz
P.
Moser
R. S.
Current issues in pediatric sports concussion
The Clinical Neuropsychologist
 , 
2011
, vol. 
25
 (pg. 
1042
-
1057
)
Schatz
P.
Pardini
J.
Lovell
M.
Collins
M.
Sensitivity and specificity of the ImPACT test battery for concussion in athletes
Archives of Clinical Neuropsychology
 , 
2006
, vol. 
21
 (pg. 
91
-
99
)
Schatz
P.
Sandel
N.
Sensitivity and specificity of the online version of ImPACT in high school and collegiate athletes
American Journal of Sports Medicine
 , 
2013
, vol. 
41
 (pg. 
321
-
326
)
Schnirring
L.
How effective is computerized concussion management?
The Physician and Sports Medicine
 , 
2001
, vol. 
29
 (pg. 
11
-
16
)
Shuttleworth-Edwards
A.
Response to the article on baseline testing: Throwing away clinical gold with the statistical bathwater
Current Sports Medicine Reports
 , 
2011
, vol. 
10
 (pg. 
391
-
392
)
Shuttleworth-Edwards
A. B.
Border
M.
Reid
I.
Radloff
S.
Lovell
M. R.
Echemendia
R. E.
Barth
J. T.
Collins
M. W.
South African Rugby Union
Traumatic brain injury in sports: An international neuropsychological perspective
 , 
2004
The Netherlands
Swets & Zeitlinger
(pg. 
149
-
168
)
Shuttleworth-Edwards
A. B.
Border
M. A.
Computer based screening in concussion management: Use versus abuse
British Journal of Sports Medicine
 , 
2002
, vol. 
36
 pg. 
473
 
Shuttleworth-Edwards
A. B.
Gaylard
E. K.
Radloff
S. E.
Cockcroft
K.
Laher
S.
WAIS-III test performance in the South African context: Refinement of an existing cross-cultural normative database
Psychological assessment in South Africa: Research and applications
 , 
2013
Johannesburg
Wits Press
(pg. 
17
-
32
)
Shuttleworth-Edwards
A. B.
Noakes
T. D.
Radloff
S. E.
Whitefield
V. J.
Roberts
C. O.
Clark
S. J.
, et al.  . 
The comparative incidence of reported concussions for follow-up management in South African Rugby Union
Clinical Journal of Sport Medicine
 , 
2008
, vol. 
18
 (pg. 
403
-
409
)
Shuttleworth-Edwards
A. B.
Radloff
S. E.
Compromised visuomotor processing speed in players of Rugby Union from school through to the adult national level
Archives of Clinical Neuropsychology
 , 
2008
, vol. 
23
 (pg. 
511
-
520
)
Shuttleworth-Edwards
A. B.
Smith
I.
Radloff
S. E.
Neurocognitive vulnerability amongst university rugby players versus noncontact sport controls
Journal of Clinical and Experimental Neuropsychology
 , 
2008
, vol. 
30
 (pg. 
870
-
884
)
Shuttleworth-Edwards
A. B.
Whitefield
V. J.
Ethically we can no longer sit on the fence: A neuropsychological perspective on the cerebrally hazardous contact sports
South African Journal of Sports Medicine
 , 
2007
, vol. 
19
 (pg. 
32
-
38
)
Shuttleworth-Jordan
A. B.
On not reinventing the wheel: A clinical perspective on cultural aspects of test usage in South Africa
South African Journal of Psychology
 , 
1996
, vol. 
26
 (pg. 
96
-
102
)
Shuttleworth-Jordan
A. B.
Puchert
J.
Balarin
E.
Plunket
R.
Anderson
S. J.
Negative consequences of mild head injury in rugby: A matter worthy of concern
Proceedings of the Fifth National Neuropsychology Conference
 , 
1993
Durban
South African Clinical Neuropsychological Association (SACNA), University of Natal
(pg. 
38
-
68
)
Smith
I.
Is rugby bad for your intellect? The effect of repetitive mild head injuries on the cognitive functioning of university level rugby players
2006
Grahamstown, South Africa
Rhodes University
 
Masters thesis
Solomon
G.
Ott
D.
Lovell
M.
Long-term neurocognitive dysfunction in sports: What is the evidence?
Clinics in Sports Medicine
 , 
2011
, vol. 
30
 (pg. 
165
-
177
)
Spear
J.
Are professional footballers at risk of developing dementia?
International Journal of Geriatric Psychiatry
 , 
1995
, vol. 
10
 (pg. 
1011
-
1014
)
Stern
R.
Riley
D.
Daneshvar
D.
Nowinski
C.
Cantu
R.
Mckee
A.
Long-term consequences of repetitive brain trauma: Chronic traumatic encephalopathy
American Academy of Physical Medicine and Rehabilitation
 , 
2011
, vol. 
3
 (pg. 
S460
-
S467
)
Strauss
E.
Sherman
E. M. S.
Spreen
O.
A compendium of neuropsychological tests
 , 
2006
Oxford
Oxford University Press
Stuss
D.
Murphy
K.
Binn
M.
Alexander
M.
Staying on the job: The frontal lobes control individual performance variability
Brain
 , 
2003
, vol. 
126
 (pg. 
2363
-
2380
)
Tales
A.
Leonards
U.
Bompas
A.
Snowen
R.
Philips
M.
Porter
G.
, et al.  . 
Intra-individual reaction time variability in amnestic mild cognitive impairment: A precursor to dementia?
Journal of Alzehimers Disease
 , 
2012
, vol. 
32
 (pg. 
457
-
466
)
Webbe
F.
Ochs
S. R.
Recency and frequency of soccer heading interact to decrease neurocognitive performance
Applied Neuropsychology
 , 
2003
, vol. 
10
 (pg. 
31
-
41
)
Wechsler
D.
Wechsler Adult Intelligence Scale-III
 , 
1997
San Antonio
The Psychological Corporation
Whitefield
V.
“Glory is temporary, brain injury may be forever”. A neuropsychological study on the cumulative effects of sports-related concussive brain injury amongst Grade 12 school boy athletes
2007
Grahamstown, South Africa
Rhodes University
 
Doctoral thesis
Witol
A. D.
Webbe
F. M.
Soccer heading frequency predicts neuropsychological deficits
Archives in Clinical Neuropsychology
 , 
2003
, vol. 
18
 pg. 
417
 
Yeates
K. O.
Mild traumatic brain injury and postconcussive symptoms in children and adolescents
Journal of the International Neuropsychological Society
 , 
2010
, vol. 
16
 (pg. 
953
-
960
)
1
Rugby Union is one of a cluster of rugby football games which originated from soccer, the oldest and most widely played of all the football games (Micheli & Riseborough, 1974). In 1823, a schoolboy at Rugby school in England, while playing soccer, picked up the ball and ran with it to put it across the goal line. Over a period of 30 years, this approach took on and developed into the completely separate game of rugby football, which later splintered and developed into several modes: Rugby Union, Rugby League, American football, and Australian Rules football. Rugby Union is now an extremely popular and fast-growing spectator sport worldwide, which is played at school, university, and professional levels in countries such as England, Wales, Scotland, the United States, Canada, Argentina, France, Japan, Australia, New Zealand, Zimbabwe, and South Africa.

Author notes

A version of this paper was presented at an invited symposium entitled “Current research outcomes on computerized neurocognitive screening within the South African sports concussion arena”, for the 30th International Congress of Psychology, Cape Town International Convention Centre (CTICC), Cape Town, South Africa, 22–27 July 2012.