Introduction

Concussion is an emerging and common health concern that affects individuals of all ages. According to the Center for Disease Control and Prevention, an estimated 1.7 million traumatic brain injury (TBI) cases arise every year, with 75% receiving a diagnosis of concussion, or another form of mild TBI (mTBI) [1].

The most updated set of clinical guidelines pertaining to concussion defines concussion as a traumatically, or biomechanically, induced alteration of brain function with emphasis placed on a functional disruption rather than structural disruption of the brain [2]. Despite emphasizing the importance of brain function, these guidelines do not recommend rigorous neuropsychological testing following the original insult [3]. They state that brief neuropsychological testing can be useful to assess the immediate and acute effects following concussion, but in the long term, concussion can be managed without long-term cognitive follow-up.

Several meta-analyses of both sport and non-sport-related concussions have also concluded that the benefits of neuropsychological assessment after injury are limited [4,5,6], suggesting that long-term cognitive deficits following mild head injury are minor if they occur at all [7]. The largest effects are seen within the first few days after the injury and tend to subside within 1–10 days in cases of a sport-related concussion, and within 90 days in cases of a non-sport concussion [8]. Beyond these initial time points, few studies report reliable cognitive deficits following concussion [9], although public opinion and media attention would suggest otherwise [10].

The long-term consequences of concussion, if they exist, are likely to be subtle. The detection of cognitive deficits may be hampered by small sample sizes and the insensitive cognitive measures that have been used in many previous studies [11,12,13]. To date, there has been no large-scale study of the long-term cognitive effects of concussion among the general population, using tests that are sensitive to small changes in performance.

The Cambridge Brain Sciences (CBS) battery assesses aspects of memory, attention, reasoning, and inhibitory control using a web-based interface that can be self-administered at home through any internet browser [14]. These tests have been validated in patients with anatomically specific brain lesions [15], in neurodegenerative populations [16], in pharmacological intervention studies [17], and their neural correlates have been well studied using functional neuroimaging in healthy adults (e.g., [18]), and in neuropathological populations [16]. The tests have been used previously to run several online large-scale studies of cognition in the general population [14, 19].

In a first study, we used this assessment battery to examine cognitive performance in a population-based sample of 19,261 individuals via an online assessment battery and to relate the findings to whether participants reported ever sustaining a concussion or not. Of these participants, 3750 reported having sustained at least one concussion, while 15,511 reported no history of concussion. Based on the literature to date, we hypothesized that self-reported concussion would not be associated with any detectable long-term deficits in cognitive function across this battery of cognitive tests. In fact, we detected a modest, but statistically robust deficit in inhibitory control in the group who reported a previous concussion. This result was then used to generate a second confirmatory hypothesis; that is, that an entirely independent sample of varsity college American football players who have regularly exposed themselves to head impacts over several years would show the same specific deficit of inhibitory control in the absence of more general impairments in cognition.

Study 1 methods

Participants and procedure

Participants for the first population-based study were recruited through the Cambridge Brain Sciences online platform (https://www.cambridgebrainsciences.com/). After visiting the website, they were instructed to register with an email address and provide informed consent, which was approved by the Health Sciences Research Ethics Board of the University of Western Ontario. Next, we asked participants to complete a short questionnaire that included questions regarding demographic and lifestyle factors (e.g., age, gender, education, socio-economic status), as well as medical information, such as level of social contact, whether they experienced any trouble concentrating, anxiety and depression in the last month, and how many concussions they had sustained over the course of their lifetime. Participants then completed a battery of 12 cognitive tests that assessed a broad range of cognitive abilities including reasoning, problem-solving, attention, memory and inhibitory control (see Hampshire et al. [14] for detailed descriptions of each task). The tests are designed to be self-administered and delivered along with instructional videos outlining how to complete each one. All participants completed the demographic questionnaire and the 12 cognitive tests at a time and place of their choosing, which took approximately 60 min.

From an original sample of 33,397 respondents, we included only those who responded to the question, “Have you ever had a concussion?” and completed at least one of the 12 cognitive tests. Following univariate extreme value analysis, we removed participants if their scores were more than four standard deviations from the mean, as values beyond this threshold have a cumulative probability of occurrence of less than 0.01% [20]. Also, we removed individuals who reported improbable or impossible responses to the questionnaire (e.g., reporting 100 concussions). To examine potential biases in the results attributed to the large number of participants who were excluded (N = 14,136) we compared various demographic measures from the excluded sample participants to those included in the study. We found no differences between excluded and included participants in terms of their mean age, reported level of education or reported socio-economic status growing up. In total, we included 19,261 participants in the final analyses.

Statistical analysis

We classified participants into one of two categories: “non-concussed” (no reported history of concussion) or “post-concussion” (reported history of one or more concussions). Data were analyzed using a combination of R version 3.4.2 (2017) for frequentist statistics and JASP version 0.8.4 (2017) for Bayesian statistics. We used JASP’s default settings, which generate a Bayes Factor that can be interpreted as the relative likelihood of one model versus another, given the data and a certain prior expectation [21].

The data were analysed using both frequentist and Bayesian statistics to determine whether there was any relationship between sustaining a concussion and performance on the CBS cognitive tasks. We included both frequentist and Bayesian statistics (positive: BF10 3–20; strong: BF10 20–150; or very strong: BF10 > 150 support of the alternative hypothesis) because they provide complementary perspectives—controlling for Type 1 and Type 2 errors, and determining the likelihood the result falls under the null hypothesis, respectively [22].

To determine whether sustaining one or more (self-reported) concussions was associated with changes in different aspects of cognition, we ran a two-way ANOVA with concussion status (2) and cognition (12 tests) as factors. If we identified a significant interaction effect, we evaluated post hoc comparisons using a series of t tests and reported the effect sizes using Cohen’s d [23]. We were also interested in whether having sustained one or more concussions affected non-cognitive life factors. To test this, we ran four Welch two-sample t tests, one for each of the non-cognitive life factors in our dataset: self-reported anxiety, depression, social contact, and trouble concentrating over the past month.

Multiple linear regressions were used to construct models to predict performance on each of the 12 cognitive tests from the number of reported concussions. To investigate whether long-term cognitive effects of concussion(s) emerge only in specific sub-populations, or whether sustaining a concussion accelerates the natural aging of the brain (e.g., [24]), we included age as a variable of interest, along with its interaction with the number of reported concussions. Gender (e.g., [25]), level of education (e.g., [26]), and SES (e.g., [27]) were included as categorical control variables. All results were corrected using a Bonferroni correction for multiple comparisons.

Study 1 results

Descriptive statistics

All participants reported how many concussions they had experienced in their lifetime. Scores ranged from 0 to 28 concussions: 15,511 participants reported never having had a concussion and 3750 reported having at least one concussion.

Age and gender

Of the final sample of 19,261 participants, 11,478 were female (59.5%), 7412 (38.5%) were male, 157 (0.7%) identified as other and 244 (1.3%) did not answer this question. The mean reported age of the non-concussed participants was 41.39 ± 14.31 and the mean reported age of the post-concussion participants was 41.94 ± 13.65.

Level of education

Most participants indicated that the highest level of education that they had completed was high school (25.02%), a Bachelor’s degree (39.84%), a Master’s degree (19.93%), or a Doctoral or Professional degree (9.74%). 3.51% of participants indicated that they had not completed any level of education. The remaining 1.96% did not answer this question.

Socio-economic status

Participants were asked to indicate their socio-economic status, growing up as at, or above, the poverty level or below the poverty level. Most participants indicated that they grew up at, or above, the poverty level (91.7%) and 7.55% indicated that they grew up below the poverty level. The remaining participants (0.75%) did not answer the question.

Differences in cognitive performance

A two-way omnibus analysis of variance (ANOVA) was conducted to examine the relationship between reported concussion and cognition, with concussion (no concussion, post-concussion) and cognition (12 tests) as factors. We found no statistically significant overall difference between post-concussion and non-concussed individuals in terms of their cognitive performance across the 12 tests (F(1,178466) = 3.654, p = 0.056). There was, however, a statistically significant interaction between the effects of concussion and standardized cognitive test score (F(11,178766) = 2.962, p < 0.001), indicating that there were differences in performance across different tests (Fig. 1). Post hoc Welch two-sample t tests, with Bonferroni correction for multiple comparisons (α = 0.004), showed that post-concussion and non-concussed individuals were indistinguishable on 11 of the 12 cognitive tests (t < 1.792, p > 0.196, Cohen’s d < 0.044, power < 0.225, BF10 < 0.035, Rhat < 1.002), but differed significantly on one test of inhibitory control (“Double Trouble”—for details, see [14]) (t(14681) = 4.353, p < 0.001, Cohen’s d = 0.091, power = 0.934, BF10 = 292.390, Rhat = 1).

Fig. 1
figure 1

Standardized cognitive test scores for the post-concussion and non-concussed groups. DT double trouble, OOO odd one out, ST spatial tree, GM grammatical reasoning, DS digit span, TS token search, PA paired associates, SS spatial span, FM feature match, ROT rotations; POL polygons, ML monkey ladder (for details, see Hampshire et al. [14]). The only significant difference after correcting for multiple comparisons was the double trouble task. *Significant at p < 0.004

Full size image

The test of inhibitory control used here is a variant of the Stroop task [28] and requires participants to respond to the word at the bottom of the computer screen (choices) that correctly describes the colour of the word presented at the top of the screen (probe). Both the probes and the choices can be congruent (e.g., “Red” written in a red-coloured font) or incongruent (e.g., “Red” written in a blue-coloured font) between the written word and the font colour. As such, performance on the task can be broken down into four different conditions: congruent probe/congruent choices (CC), congruent probe/incongruent choices (CI), incongruent probe/congruent choices (IC), incongruent probe/incongruent choices (II) (Fig. 2).

Fig. 2
figure 2

Depiction of the four possible double trouble tasks. CC congruent probe/congruent choices, CI congruent probe/incongruent choices, IC incongruent probe/congruent choices, II incongruent probe/incongruent choices

Full size image

Embedded in a participant’s final score is information about speed and accuracy across all four conditions. To further explore why post-concussion individuals performed the inhibitory control task more poorly than those who did not report having had a concussion, we examined their performance broken down by these two factors. Two two-way ANOVAs were run to examine differences in (1) average correct responses, and (2) average response times for each congruency condition between post-concussion participants and non-concussed participants.

The two-way ANOVA examining the effects of concussion and condition on the participants’ correct responses (i.e., accuracy) showed a significant main effect of concussion (F(1, 62108) = 35.10, p < 0.001) and a significant interaction between the effects of concussion and condition (F(3, 62108) = 2.66, p = 0.0464). Post hoc Welch two sample t tests with Bonferroni correction for multiple comparisons (α = 0.012) showed significant differences between post-concussion and non-concussed individual’s average correct responses in all three incongruent conditions: CI (t(15528) = 3.081, p = 0.002, Cohen’s d = 0.062, power = 0.713, BF10 = 2.607, Rhat = 1), IC (t(15528) = 3.756, p < 0.001, Cohen’s d = 0.076, power = 0.895, BF10 = 26.073, Rhat = 1) and II (t(15528) = 3.165, p = 0.002, Cohen’s d = 0.064, power = 0.746, BF10 = 3.383, Rhat = 1). In contrast, no significant difference was observed between post-concussion and non-concussed individuals in the congruent (CC) condition (t(15528) = 1.690, p = 0.077, power = 0.236, BF10 = 0.109, Rhat = 1) (Fig. 3). The two-way ANOVA examining the effects of concussion and condition on the participants’ response times in each condition did not show any significant main effect of concussion (F(1, 62108) = 1.949, p = 0.163) nor any significant interaction between concussion and condition (F(3, 62108) = 0.464, p = 0.707).

Fig. 3
figure 3

Average percent correct scores and average response time for the post-concussion and non-concussed groups on the test of inhibitory control. CC congruent probe/congruent targets, CI congruent probe/incongruent targets, IC incongruent probe/congruent targets, II incongruent probe/incongruent targets. After correction for multiple comparisons, there were significant differences in average percent correct for all three incongruent conditions (CI, IC, II) but not for the congruent condition (CC). No differences in response time were observed. *Significant at p < 0.125

Full size image

Multiple linear regression

To provide a complete examination of the relationship between a reported previous concussion and performance on the cognitive tasks, we conducted a series of hierarchical linear regressions on all 12 tests in the post-concussion group. We examined standardized cognitive test scores in a single four-step hierarchical multiple regression model using age (Step 2), number of concussions (Step 3) and the interaction between age and concussions (Step 4) as the independent variables. Standardized cognitive test scores served as the dependent variable, with gender, socioeconomic status, and level of education entered at Step 1 as control variables. For this study, we were most interested in examining the change statistic (∆R2) for Step 3, when the number of reported concussions was added into the regression equation, to assess if sustaining at least one concussion accounted for any variability in cognitive test scores. After Bonferroni correction for multiple comparisons across all 12 cognitive tests (α = 0.004), number of concussions (Step 3) led to a significant change in the variance accounted for (0.4 per cent), solely on the basis of scores on the test of inhibitory control, and increased the total variance accounted for on this task to 9.4 per cent. The number of reported concussions did not significantly increase the variance accounted for on any of the 11 other tests. Table 1 summarizes the results of this regression analysis for standardized inhibitory control scores. Furthermore, the interaction between age and number of concussions (Step 4) did not lead to a significant change in variance accounted for on any of the cognitive tasks. In other words, the number of reported concussions significantly improved the prediction of the inhibitory control scores in the post-concussion group while the interaction between age and number of concussions did not improve to this prediction model.

Table 1 Summary of hierarchical multiple regression of inhibitory control standardized scores on control variables, age, number of concussions, and the interaction between age and concussions
Full size table

Differences in non-cognitive variables

As previously described, participants were asked to self-report how often they experienced social contact, trouble concentrating, anxiety and depression monthly. Welch two-sample t tests comparing both concussion groups were performed on each non-cognitive variable of interest. Concussed and non-concussed individuals showed significant differences in their reported monthly levels of trouble concentrating (t(19149) = − 11.218, p < 0\(.\)001, Cohen’s d = − 0\(.\)205, power = 1, BF10 = 3.302e + 25, Rhat = 1), anxiety (t(19154) = − 8.642, p < 0\(.\)001, Cohen’s d = − 0.158, power = 1, BF10 = 3.004e + 14, Rhat = 1) and depression (t(19226) = − 8.989, p < 0.001, Cohen’s d = − 0.164, power = 1, BF10 = 6.286e + 15, Rhat = 1). The two concussion groups did not show any significant differences in their reported levels of monthly social contact (t(19085) = 1.012, p = 0.312, Cohen’s d = 0.019, power = 0.180, BF10 = 0.034, Rhat = 1).

Study 2 methods

Participants and procedure

To examine whether the pattern of cognitive deficits observed in Study 1 could predict cognitive performance in an independent group of participants who were generally considered to be healthy, yet routinely received multiple impacts to the head, we conducted a second hypothesis-driven study in a group of varsity college athletes. Accordingly, we administered the same 12 cognitive tests to 74 varsity college football players, currently playing for a university team, and results compared to a combined group of varsity college rowers and online volunteers with no history of head injury. The varsity athletes (football players and rowers) were recruited at the University of Western Ontario while the online workers were recruited through Amazon’s Mechanical Turk (MTurk) online platform. Participants from all three groups completed the 12 cognitive tests online just before the start of the football season. This was the first time any of the participants had completed these tasks. Both the football players and rowers were current athletes. The football players and all controls were of a similar, university, age (18–26).

In the football group, the average number of seasons played was 7.5 (years). Most of the players included in this sample indicated that they started playing football at a competitive level in high school and continued to play at the varsity level in university. Rowers and MTurk workers were excluded from the study if they had any history of playing football at any varsity level (youth, high school or college).

Statistical analysis

We separated the varsity athlete dataset into two groups: footballers (history of varsity football) and controls (rowers + MTurk workers; no history of varsity football). To ensure that a meaningful comparison could be made between the results of Study 1 and Study 2, we followed a similar analysis protocol in the second study to that used in the first. As in Study 1, we analyzed the data using a combination of R version 3.4.2 (2017) for frequentist statistics and JASP version 0.8.4 (2017) for Bayesian statistics, using JASP’s default settings. Using a priori hypotheses generated from the results from Study 1, we performed 12 Welch one-sided t tests comparing the standardized CBS scores of footballers to controls.

Study 2 results

We compared 74 varsity football players from the University of Western Ontario to a group of 36 controls comprised of varsity rowers and MTurk workers. We conducted one-sided Welch t tests with Bonferroni correction for multiple comparisons (α = 0.004), comparing the varsity football players to controls on each of the cognitive tasks. As predicted from the population-level results, varsity football players scored significantly lower on the test of inhibitory control (t(83.71) = − 3.681, p < 0.001, Cohen’s d = − 0.721, power = 0.692, BF10 = 66.920, Rhat = 1) and did not show any significant differences on any of the 11 other tasks (t < − 2.015, p > 0.024, Cohen’s d < 0.411, power < 0.192, BF10 < 2.538, Rhat < 1.03). See Fig. 4.

Fig. 4
figure 4

Standardized cognitive scores for the varsity football players and control group. DT double trouble, OOO odd one out, ST spatial tree, GM grammatical reasoning, DS digit span, TS token search, PA paired associates, SS spatial span, FM feature match, ROT rotations; POL polygons, ML monkey ladder. The only significant difference after correcting for multiple comparisons was the double trouble task. *Significant at p < 0.004. For test details, see Hampshire et al. [14]

Full size image

For whole data posterior distributions (with 89% HDI) and posterior distributions of Monte Carlo Markov Chain (MCMC) draws (with 89% HDI) for Study 1 and Study 2, see Supplementary Materials.

Discussion

In the first study, we first took a population-level approach to examine the cognitive and social characteristics of individuals who reported having sustained at least one concussion during their lifetime. We found that post-concussion individuals were significantly impaired relative to a group of non-concussed participants on only one of 12 cognitive tasks; a test of inhibitory control, which is a modification on the Stroop task [28]. Indeed, a close examination of their responses revealed a cognitively specific accuracy-related deficit. That is to say, these participants were less accurate relative to non-concussed individuals on every type of incongruent trial (i.e., when the probe font colour did not match the word, such as “Red” written in a blue font or vice versa).

In contrast, we found no difference between the two groups when comparing performance on congruent trials, confirming that both groups understood the basic requirements of the task. Moreover, we found no significant differences between the two groups in terms of their reaction times, independent of whether the trial was congruent or incongruent. Thus, post-concussion individuals appeared to be performing just as quickly, but not as accurately, as non-concussed individuals when the trial included an incongruency between target and probe, suggesting that these participants were specifically impaired at resisting interference from prepotent word processing rules. Furthermore, multiple regression analyses revealed that the number of reported concussions significantly predicted performance on this test of inhibitory control, but no other cognitive task.

What is perhaps most notable about this set of results is that individuals who reported sustaining a previous concussion were entirely unimpaired on most of a broad range of cognitive tests assessing aspects of working memory, reasoning, problem-solving and inhibitory control. Many of these tests have been shown previously to be sensitive to damage to frontal-lobe regions [15], neurodegenerative conditions such as Parkinson’s disease [16], and to a variety of pharmacological manipulations in both patients and healthy individuals [16, 17]. It is fair to say then that the results of this study suggest that concussion (at least self-reported concussion) is not associated with long-term global effects on cognition. That said, the significant impairment on incongruent trials of the inhibitory control task survived scrutiny under a variety of statistical techniques and corrections and returned a Bayes factor of 292.390. Moreover, this impairment appears to be related to the number of concussions reported.

We also examined four non-cognitive characteristics that have been associated previously with concussion. In line with previous results [29, 30], post-concussion participants reported higher levels of anxiety, depression, and concentration difficulties than the non-concussed group. These non-cognitive factors are comorbidities that are known to complicate recovery after mTBI [31]. Athletes often develop anxiety and depression following a sports-related injury, and some studies argue that these symptoms are a function of the frustration and dissatisfaction with the length of the recovery process, rather than the injury itself [32]. In the current study, we explored each of these non-cognitive factors with a single question, e.g., “How often have you experienced anxiety/depression/trouble concentrating in the last month?” Therefore, it was not possible to explore any possible causal relationship between these non-cognitive life factors and cognition, nor to rule out any pre-existing emotional issues. Nevertheless, the positive results for three out of four of these factors suggest that future studies should explore this relationship using a more specific and tailored questionnaire.

Our population-level approach in the first study generated a very specific hypothesis, which we were able to test in Study 2 by examining the cognitive profile of an independent group of varsity footballers who regularly experience impacts to the head. As in Study 1, it was impossible to confirm whether any of these individuals had actually sustained a clinically-verifiable concussion. However, this only serves to strengthen our hypothesis, because unlike Study 1, participants were not selected according to whether they had (or reported to have had) a concussion, but rather, because they had regularly participated in an activity that was likely to involve regular blows to the head. Thus, in Study 2, we hypothesized that varsity college football players with an average of 7.5 years in the sport would show significantly lower scores on the inhibitory control task of the cognitive battery when compared to a group of controls with no experience in varsity football. In addition, we predicted that these athletes would show no differences on the 11 other cognitive tasks. As hypothesized, we were able to replicate the results of the population-level analyses in this independent sample of varsity college football players. The football players scored significantly lower on the inhibitory control task, while also being indistinguishable from controls on all other cognitive tests. These athletes had, for the most part, been playing football since high school, a division of the sport known to have the highest incidence of head injuries [33]. To put this in perspective, the absolute difference between footballers and controls on the task of inhibitory control (Double Trouble) was 11 errors. Therefore, on a task that runs for 90 s, footballers are making an error of inhibition approximately every 8 s. While this effect was statistically robust and meaningful, it is not clear how it might affect everyday activities.

Based on these two studies, we conclude that those who report sustaining a concussion at some time in their lives, and those who regularly expose themselves to blows to the head through sporting activities, do have a modest, but statistically robust deficit of inhibitory control, but are broadly unimpaired on many other aspects of cognitive performance.

In Study 1, time constraints did not allow us to ask participants how much time had elapsed between sustaining their concussion(s) and completing the cognitive tasks, or how much time had passed between multiple concussions. Moreover, we did not ask participants how they sustained their concussion(s) and how they confirmed that they were, indeed, concussed. As previously mentioned, there are differences in outcomes associated with sport and non-sport concussions [8, 34], and future work will seek to establish how these differences manifest in follow-up population-based studies. Whether or not a concussion was medically diagnosed may also have affected the validity of our Study 1 sample by including participants who self-reported a concussion, but did not have one. Nevertheless, the results of Study 2 suggest that this may be immaterial; we observed the same specific pattern of cognitive deficits whether self-reported concussion, or involvement in a contact sport with a high probability of head impacts, was used as the criterion for selection. The goal of Study 1 was to use the power of numbers to examine whether individuals who feel that they have sustained a concussion (whether medically-diagnosed or not) differ in terms of their long-term cognitive and emotional well-being, relative to individuals who do not report having had a concussion. Accordingly, robust effects were observed on one test of inhibitory control and reported levels of anxiety, depression, and concentration. In the former case, we replicated this effect in a group of varsity football players with an average of 7.5 years in the game.