FormalPara Key Points

The symptom-free waiting period should be tailored to each individual athlete’s risk for injury after return-to-play.

Athletes with a higher risk of injury after return-to-play achieve greater benefit from longer symptom-free waiting periods compared with athletes with a lower risk of injury.

Our study findings can be used to help clinicians select a symptom-free waiting period based on their risk tolerance and athlete-specific injury risks.

1 Introduction

For athletes diagnosed with sport-related concussion, the timing of return-to-play (RTP), that is, return to full participation in sport, is critical to mitigating immediate risk of injury and potential long-term health-related consequences. Premature RTP, for example, has been linked with increased risk of repeat concussion [1]. Likewise, having a recent concussion has been linked with increased risk for musculoskeletal injuries, especially lower extremity injuries [2, 3]. To this end, current international guidelines recommend a graduated RTP protocol with 24 h between steps in the protocol, no same-day RTP, and no RTP while symptomatic [4]. Yet, little guidance exists beyond these broad recommendations; the exact criteria and length of time for each component in the RTP process is unspecified. As a result, the RTP decision is often left to the clinical judgment of the medical provider, thereby increasing the variability in health-related outcomes for athletes with concussion. This variability in the timing of RTP is evidenced by the most recent analyses on the natural history of concussion [5].

One key component in the RTP decision is the symptom-free waiting period (SFWP), which is the time period from when the athlete is first considered asymptomatic until the time they are permitted to begin unrestricted RTP. McCrea et al. initially found that the presence of a SFWP reduces the risk of repeat concussion [6]. While the length of the SFWP has increased in recent years with increased concussion awareness and changes to clinical protocols [1], the optimal length of time for the SFWP remains undetermined. Furthermore, previous studies of the SFWP have focused on collegiate football players, so tailoring the SFWP for athletes outside of this demographic group remains an open question.

For many ethical reasons, it is not possible to conduct a clinical trial to investigate the relationship between the SFWP and pertinent health outcomes (e.g., risk and timing of reinjury). As such, establishing the right evidence to ‘optimize’ the SFWP is challenging. To this end, computational simulation techniques may provide a reasonable approach to estimate these relationships using data without the need to test varying SFWPs in vivo. Simulation has long been used in various areas of healthcare (e.g., cancer [7, 8], genetic testing [9], opioids [10], obesity [11, 12], and liver disease [13, 14]) to assess health outcomes for different treatment policies, particularly when it may be infeasible to conduct a study due to cost or ethical concerns, among other reasons.

In this research, we apply a simulation analysis to investigate the relationship between the SFWP and risk for (1) repeat concussion and (2) non-concussion time-loss injury for male and female collegiate athletes across a wide variety of sports. Our findings demonstrate the importance of tailoring the SFWP to each athlete based on their likely recovery trajectories and risk of injury after RTP.

2 Methods

2.1 Study Data

In this study, we leveraged data from the National Collegiate Athletic Association (NCAA)-Department of Defense (DoD) Concussion Assessment, Research, and Education (CARE) Consortium. The CARE Consortium is composed of 30 NCAA universities and military service academies and was established to study the natural history and neurobiological recovery of concussion. CARE Consortium data was collected from 2014 to 2019 and contain 34,709 baseline testing assessments and 3603 post-injury assessment trajectories from 3224 varsity athletes across the NCAA and military service academies. Post-injury trajectories include assessments performed at pre-specified timepoints: < 48 h post-injury, at the start of the RTP protocol (asymptomatic), and the time at which the athlete was allowed unrestricted participation in sport. All study sites had approval from their local Institutional Review Board and the Human Research Protection Office, and all study participants provided written informed consent.

2.2 Sample Selection

From the CARE Consortium data, we included in our analysis all post-injury assessment data for male and female athletes with one or no previous concussions. We excluded athletes with two or more previous concussions from our analysis (N = 342) since athletes with one or no previous concussions have similar recovery trajectories [5] and those with two or more previous concussions comprise a much smaller proportion of the CARE data. Male and female athletes were analyzed separately due to potential differences in recovery trajectory [15] and sport-specific injury rates.

For each post-injury trajectory, we extracted the assessment scores taken in each timepoint for the Standard Assessment of Concussion (SAC) total score, Balance Error Scoring System (BESS) total score, and the total symptom severity score from the Sport Concussion Assessment Tool (SCAT) graded symptom checklist. This study utilizes symptom scores from the SCAT3, which was the only version of the SCAT administered throughout the entire CARE study. We did not include the total number of symptoms reported on the SCAT graded symptom checklist to avoid numerical issues in our methodologies which may arise from the statistical dependence between the total number of symptoms reported and the symptom severity score. Moreover, these two measures perform similarly in detecting concussion based on past univariate analysis [16].

Briefly, the SAC is a neurocognitive assessment which measures potential impairments in orientation, immediate memory, concentration, and delayed recall. The SAC total score sums scoring in each dimension and can range from 0 to 30. The BESS is a postural control assessment which measures the number of errors committed by an athlete across three stances. Finally, the SCAT graded symptom checklist is composed of 22 symptoms, each of which is scored from 0 (symptom is not being experienced) to 6 (symptom is being experienced at maximum severity). By employing these three assessments in our simulation analysis, we can replicate a realistic multi-dimensional approach to concussion assessment as recommended by current international guidelines [4].

2.3 Modeling Concussion Recovery Trajectories

We modeled concussion recovery trajectories during the RTP process using a Hidden Markov Model (HMM) [17]. HMMs are a well-established class of latent variable models that have been used to model many types of diseases with complex dynamics including cancer [8, 18,19,20,21], organ transplantation [22], infectious diseases [23], and chronic diseases [24, 25], as well as recent applications in sports medicine [26, 27]. Our pre-RTP HMM model (see Fig. 1A) represents the severity of concussion as an unobservable ‘state’. In alignment with common terminology used in the literature, we name these concussion severity states symptomatic, asymptomatic, and recovered. At the start of each day, the athlete’s unobservable concussion state transitions probabilistically based on its state in the previous day according to a set of state-transition probabilities. After the new state is realized, it produces observable measures (i.e., composite scores for the SAC, BESS, and SCAT symptom severity) which are used to estimate which unobservable state the concussion is in. The likelihood of observing different SAC, BESS, and SCAT symptom severity scores based on the unobserved state are specified in a matrix of observation probabilities. Note that since HMMs are probabilistic models, it may be possible for an athlete in the asymptomatic or recovered state to report worse SAC, BESS, or SCAT symptom severity scores compared with an athlete in the symptomatic state—although it is highly unlikely. As described in the remainder of this section, these probabilities will be determined by the CARE Consortium data.

Fig. 1
figure 1

Illustration of A pre-RTP (return-to-play) Hidden Markov Model and B Markov Model representing state transitions after RTP. In A, arrows between unobservable latent states indicate possible transitions between concussion recovery status. Observable measures are generated probabilistically based on the latent state. In B, solid lines indicate transitions between concussion recovery states and dashed lines indicate transitions to injured states. BESS Balance Error Scoring System, SAC Standard Assessment of Concussion, SCAT Sport Concussion Assessment Tool

Full size image

To parameterize and calibrate our HMM model, we first separated post-injury data by trajectory. For each post-injury trajectory, we applied linear interpolation to estimate post-injury assessment scores on days that were not recorded in the data. Since the HMM model requires observations to be contained in a discrete and finite set, we separately categorized SAC, BESS, and SCAT symptom scores for males and females based on percentiles identified in the data. Specifically, we categorized SAC and BESS total scores by quintiles. For SCAT symptom severity scores, we created a category for a total severity score of 0 or 1 since most symptom scores were 0 or 1. Then, we created separate categories for each quintile severity score > 1. These categories are summarized in Table 1.

Table 1 Categories used for standard assessment of concussion (SAC) total score, balance error scoring system (BESS) total score, and Sport Concussion Assessment Tool (SCAT) total symptom severity score in the Hidden Markov model
Full size table

After categorizing assessment scores, we subdivided the male and female datasets into a training set (80%) for parameterizing the HMM model and validation set (20%) for calibrating the HMM model. We evaluated differences between the training and validation sets by sex, timepoint, and assessment using the two-sample bootstrap t test [28] and Cohen’s d for effect sizes.

We applied the Baum–Welch algorithm [17] on the training data to estimate the state-transition and observation probabilities for each group. Since the Baum–Welch algorithm is dependent on its initialization, we randomly initialized the algorithm 100 times, with each producing its own HMM model. We then used a stacking algorithm [29] with fivefold cross-validation to simultaneously aggregate these 100 models and calibrate them to the validation data. We performed our final calibration step by adjusting the transition probabilities so that the expected time to clinical resolution of concussion (based on the SAC, BESS, and SCAT symptom severity) closely matched values reported in the literature [5, 30].

2.4 Modeling Post-RTP (Return-to-Play) Trajectories

To estimate the relationship between the timing of RTP and risk of subsequent injury, we developed a Markov model to simulate an athlete’s 30-day risk of injury after RTP. We selected a 30-day post-RTP period because a period that is too short (e.g., 3 days) or too long (e.g., 90 days) may not adequately capture the effect of the SFWP on post-RTP injury risks. This Markov model (see Fig. 1B) appends two absorbing ‘time-loss injury’ states to the HMM’s states. In alignment with existing literature on the topic, a time-loss injury is defined as any injury resulting in at least one missed day of athletic participation. Specifically, we consider both a ‘repeat concussion’ and ‘non-concussion time-loss injury’ state. The probability of transitioning to the injured states were derived using sex- and sport-specific injury rates from the literature. Specifically, concussion rates by sport were drawn from a meta-analysis by Van Pelt et al. [31] while time-loss injury rates were drawn from sports epidemiology literature at the collegiate level (see Table 2). As such, we limited our analysis to include only the sports for which both concussion rates and time-loss injury rates at the collegiate level were available. In order to convert these injury rates into transition probabilities, we assumed that there was only one athletic exposure per day for all sports. The sports included in our analysis are listed in Table 2.

Table 2 Time-loss injury rates and sources for parameterizing post-return-to-play Markov models
Full size table

2.5 Simulation Analysis of Symptom-Free Waiting Period (SFWP)

By combining the pre-RTP HMM and post-RTP Markov model, we created a two-stage simulation model to replicate the RTP process for female and male athletes with no or one previous concussion across various sports. Our pre-RTP simulation framework is illustrated in Fig. 2. The purpose of this simulation is to estimate the proportion of athletes who are symptomatic, asymptomatic, and recovered for a given SFWP. The inputs to our pre-RTP simulation model were an athlete’s sex (to determine which HMM to use) and a chosen SFWP (to determine the RTP decision process). We considered SFWPs ranging from 3 to 21 days. After selecting our inputs, we initialized our simulation with an athlete having symptomatic concussion. Then, to mimic a clinician’s judgment for determining when an athlete is ready to begin the graduated RTP protocol, we tracked the athlete’s concussion recovery trajectory using the HMM until their SAC, BESS, and SCAT symptom scores had all normalized, that is, returned to normative levels. Based on mean performance at baseline from previous analysis of CARE Consortium data [30], we say that the SAC, BESS, and SCAT symptom severity scores have normalized once they have reached scores of 27, 13, and 6, respectively, for females and scores of 27, 14, and 3, respectively, for males. After the SAC, BESS, and SCAT measures have normalized, we simulated the HMM for the number of days specified by the SFWP, at which point the pre-RTP simulation concludes. We repeated this simulation 1000 times for each sex and SFWP combination, using the common random number method to minimize variance in simulation outputs and to ensure that different SFWPs were comparable for athletes of the same sex [45]. Pre-RTP simulation outputs were used to parameterize the initial state distribution over concussion status at the time of RTP as a function of SFWP for each sex.

Fig. 2
figure 2

Illustration of pre-RTP (return-to-play) simulation logic for a fixed athlete sex and SFWP (symptom-free waiting period). Blue boxes represent components that are generated from the Hidden Markov Model. t represents the day and d represents the number of days within the SFWP that have elapsed. BESS Balance Error Scoring System, SAC Standard Assessment of Concussion, SCAT Sport Concussion Assessment Tool

Full size image

After completing all pre-RTP SFWP simulations, we used the post-RTP Markov model to estimate two outputs of interest: (1) the probability of repeat concussion in the 30 days after RTP and (2) the probability of a non-concussion time-loss injury in the 30 days after RTP by sex, SFWP, and sport. Specifically, for each sex, SFWP, and sport, we used the output concussion latent state from each pre-RTP simulation as an input into the post-RTP Markov model, for which we could compute our outputs of interest using standard equations from Markov chain theory (see Section S1 of the electronic supplementary material [ESM]) [46].

2.6 Analysis of Differences by Sex and Concussion Risk Grouping

We next compared the effects of SFWP and sex on post-RTP injury probabilities by concussion risk group. To perform this analysis, we grouped sports by concussion risk according to the classifications by Van Pelt et al. [31]. Since we did not include rugby in our analysis, we could only construct a medium-risk group composed of football, ice hockey, and wrestling and a low-risk group containing all other sports. After performing this grouping, we created Beta regression models to estimate the mean effects of SFWP, sex, and the interaction between SFWP and sex on post-RTP injury rates for low- and medium-risk groups. We performed all Beta regression analysis using the betareg package in the R statistical software, version 4.0.3.

3 Results

3.1 Data Description

The study data are summarized in Table 3. Overall, there were 735 unique post-injury trajectories from female athletes and 1094 trajectories from male athletes. There were no significant differences between training and validation sets for the female athletes or male athletes at any timepoint. Moreover, all effect sizes had a magnitude < 0.15, indicating that the training and validation data were relatively similar. The number of post-injury trajectories by sport are summarized in Table S1 (see Section S2 of the ESM). Among post-injury trajectories in the training set for female athletes, 111 (18.9%) came from soccer, 68 (11.6%) from volleyball, and 64 (10.9%) from basketball. In the validation set for female athletes, 33 (22.4%) came from soccer, 14 (9.5%) came from basketball, and 13 (8.8%) came from volleyball. Among post-injury trajectories for male athletes, the training set contained 403 (46.1%) trajectories from American football, 91 (10.4%) from soccer, and 63 (7.2%) from basketball. The validation set contained 112 (51.1%) trajectories from American football, 21 (9.6%) from soccer, and 19 (8.7%) from basketball.

Table 3 Summary of study data with respect to modeling variables
Full size table

3.2 Post-RTP Injury Rates for Female Athletes

We present the estimated rates of repeat concussion and non-concussion time-loss injuries within 30 days post-RTP as a function of SFWP in Fig. 3 for female athletes. A comparison between the 6-day and 13-day SFWPs are presented in tabular form in Table S2 (see Section S3 of the ESM). At a SFWP of 6 days (i.e., median SFWP reported in CARE [1]), the rate of repeat concussion within 30 days post-RTP per 1000 simulations was greatest in ice hockey (20.31, 95% CI 20.16–20.46) and least in swimming and diving (1.11, 95% CI 1.10–1.11). The repeat concussion rate was near the median rate across all sports of 11.57 concussions per 1000 simulations for field hockey (13.98, 95% CI 13.88–14.07) and gymnastics (9.16, 95% CI 9.09–9.22). For these same sports, extending the SFWP to 13 days reduces this rate to a mean (95% CI) of 18.88 (18.79–18.98) in ice hockey, 13.03 (12.97–13.10) in field hockey, 8.52 (8.47–8.56) in gymnastics, and 1.03 (1.02–1.03) in swimming and diving.

Fig. 3
figure 3

Sport-specific estimated probabilities of A repeat concussions and B non-concussion time-loss injuries within 30 days after return-to-play for female athletes as a function of the symptom-free waiting period. ‘Natural’ refers to the simulated injury rate for athletes not recovering from concussion

Full size image

For an SFWP of 6 days, the rate of non-concussion time-loss injuries within 30 days post-RTP per 1000 simulations was greatest for field hockey (153.66, 95% CI 152.59–154.74) and the least for swimming and diving (14.67, 95% CI 14.55–14.79). Softball (98.43, 95% CI 97.71–99.15) and ice hockey (84.19, 95% CI 83.57–84.80) had injury rates near the median across all sports of 91.31 non-concussion time-loss injuries per 1000 simulations. Increasing the SFWP to 13 days for these same sports reduced these injury rates, resulting in a mean (95% CI) rate of non-concussion time-loss injuries per 1000 athletes of 143.24 (142.53–143.94) for field hockey, 91.46 (91.00–91.93) for softball, 78.23 (77.84–78.64) for ice hockey, and 13.57 (13.50–13.64) for swimming and diving.

3.3 Post-RTP Injury Rates for Male Athletes

We present the estimated rates of repeat concussion and non-concussion time-loss injuries within 30 days post-RTP as a function of SFWP in Fig. 4 for male athletes. A comparison between the 6-day and 13-day SFWPs are presented in tabular form in Table S2 (see Section S3 of the ESM). At a SFWP of 6 days, American football had the greatest repeat concussion rate per 1000 simulations (24.16, 95% CI 24.05–24.28) and tennis had the least (1.45, 95% CI 1.45–1.46). Both basketball (11.25, 95% CI 11.20–11.31) and soccer (10.64, 95% CI 10.59–10.70) had rates near the median rate across all sports of 10.95 concussions per 1000 simulations. Increasing the SFWP to 13 days for these same sports decreased the mean (95% CI) repeat concussion rate to 23.16 (23.09–24.22) for American football, 10.76 (10.73–10.79) for basketball, 10.17 (10.14–10.20) for soccer, and 1.39 (1.38–1.39) for tennis.

Fig. 4
figure 4

Sport-specific estimated probabilities of A repeat concussions and B non-concussion time-loss injuries within 30 days after return-to-play for male athletes as a function of the symptom-free waiting period. ‘Natural’ refers to the simulated injury rate for athletes not recovering from concussion

Full size image

At a SFWP of 6 days, the rate for non-concussion time-loss injuries within 30 days post-RTP per 1000 simulations was greatest in wrestling (247.34, 95% CI 246.20–248.48) and least in swimming and diving (9.54, 95% CI 9.49–9.59). The injury rates in soccer (113.32, 95% CI 112.57–113.73) and baseball (94.09, 95% CI 93.60–94.58) were close to the median rate across all sports of 103.62 non-concussion time-loss injuries per 1000 simulations. By increasing the SFWP to 13 days, these rates decrease to an estimated injury rate (95% CI) of 237.73 (236.67–237.90) for wrestling, 108.08 (107.77–108.39) for soccer, 89.80 (89.54–90.06) for baseball, and 9.08 (9.05–9.10) for swimming and diving.

3.4 Post-RTP Injury Rates by Sex and Sport Concussion Risk Level

In Fig. 5, we illustrate the repeat concussion rate and non-concussion time-loss injury rate within 30 days post-RTP by SFWP, sex, and concussion risk level. The results of our beta regression analysis on these injury rates are shown in Table 4. The repeat concussion probabilities are significantly different between female and male athletes for low-risk sports (OR 0.700; p < 0.001) and medium-risk sports (OR 0.935; p < 0.001), with males having a lower mean probability of repeat concussion at the same SFWP. We also find significant differences in non-concussion time-loss injuries for both low-risk and medium-risk sports; notably, the estimated injury probabilities are greater for males in medium-risk sports (OR 2.437; p < 0.001) and less for males in low-risk sports (OR 0.808; p < 0.001). The interaction between sex and SFWP is significantly greater than 1 for repeat concussions (OR 1.003; p < 0.001 for low-risk and medium-risk sports) and non-concussion time-loss injuries (OR 1.003, p < 0.001 for low-risk sports; OR 1.003, p = 0.002 for medium-risk sports). This result implies that males have a smaller reduction in injury risk for each increase in SFWP compared with their female counterparts, although the magnitude of these differences may not be clinically meaningful.

Fig. 5
figure 5

Rates of A repeat concussion and B non-concussion time-loss injuries within 30 days after return-to-play (RTP) by sex and sport contact type. ‘Natural’ refers to the simulated injury rate for athletes not recovering from concussion

Full size image
Table 4 Odds ratio (OR), 95% confidence interval (CI) around OR, and p values for the symptom-free waiting period (SFWP), sex (i.e., male athlete), and the interaction between SFWP and sex for Beta regression models under varying injury types and concussion risk groups
Full size table

4 Discussion

In this research, we combined Markov modeling with simulation to estimate the relationship between the length of the SFWP and post-RTP injury rates across a wide variety of varsity collegiate sports. Given the ethical concerns of systematically testing varying SFWPs in vivo, simulation viably provides a glimpse into the complex relationships between concussion recovery, RTP decisions, and post-RTP health outcomes. Moreover, simulation enables us to study the SFWP for athletes across a wide range of sports, extending the current literature beyond American football. Overall, our results provide some insights and guidance for medical providers who must make critical post-injury management decisions for a heterogeneous population of student athletes.

The importance of a SFWP is recognized in international guidelines [47]. Our results add to this previous knowledge by demonstrating the importance of tailoring the SFWP by sport and sex. For athletes of the same sex, those who engage in sports with greater risks of injury (e.g., ice hockey, American football) derive a greater post-RTP injury risk reduction by lengthening the SFWP compared with athletes in lower risk sports (e.g., swimming and diving). Specifically, we found that for certain sports (e.g., ice hockey, American football, wrestling, soccer), post-RTP injury rates may be decreased by 4.0–7.1% by increasing the SFWP over the current 6-day median SFWP reported in the CARE Consortium [1] by 1 week, that is, a 13-day SFWP or 19–20 days total time to unrestricted RTP assuming roughly 6–7 days to reach asymptomatic status [5]. To this end, current evidence demonstrates that a mean or median SFWP between 5–7 days is commonly employed at the high school and collegiate levels [1, 48]. However, these data are not disaggregated by sport or sex, and thus, it is challenging to ascertain how our findings might be reflected in current practice. Notably, to investigate the incremental value of increasing the SFWP, we performed an auxiliary analysis of the Population Attributable Fraction, that is, the proportion of the estimated post-RTP injury probabilities attributable to RTP from concussion and the length of the SFWP versus the natural risk inherent in the sport. This analysis (see Section S4 of the ESM) shows that the SFWP may have a non-linear relationship with diminishing returns in post-RTP injury risk reduction relative to each sport’s natural injury probabilities. Specifically, for female athletes across all sports, the incremental reduction in risk at small SFWPs is greater than the incremental reduction at large SFWPs. For male athletes, this relationship is less clear, and the incremental reduction at small SFWPs may be similar to the incremental reduction at large SFWPs.

This study adds to the growing literature on recovery from sports-related concussion. While earlier guidance initially suggested that concussion resolves in 7–14 days post-injury [49], more recent research suggests that the recovery process could be longer; return to baseline levels alone could take 7–14 days [50] while the entire recovery process might take upwards of 21–28 days post-injury for most athletes [5, 51,52,53]. While our results do not explicitly study the natural history of concussion recovery, our results support the growing evidence suggesting that a longer recovery process (i.e., a longer SFWP) may be warranted in clinical practice and clinical guidelines, especially for athletes in sports such as football or wrestling. Of note, our results are also consistent with two other simulation-based and computational studies on optimizing RTP decisions [27, 54], similarly suggesting the need for longer recovery processes in an effort to mitigate injury rates after RTP.

This study underscores the importance of symptom evaluations in concussion management. While our SFWP criteria involved a return to baseline levels for the SAC, BESS, and SCAT symptom scores, we remark that a symptom-driven SFWP criteria introduces the potential for symptom-reporting behavior to influence RTP decisions. In particular, symptom under-reporting is commonplace in the management of sports-related concussion, with some studies estimating that 50–60% of athletes under-report their concussion-related symptoms [55, 56]. Athletes who report no symptom burden may meet their SFWP criteria more quickly than those who report symptoms honestly, which could result in premature RTP. While we did not account for symptom-reporting behavior in this study, literature on the effects of symptom under-reporting in RTP is scarce. To date, we are only aware of one study which quantifies the potential effects of symptom under-reporting on RTP outcomes. In the simulation study by Garcia et al. [54], they estimate that athletes in male collegiate football who under-report symptoms have a 6–10% increased risk of repeat concussion after RTP using a 7-day SFWP compared with athletes who report symptoms honestly. While additional studies are needed to validate and generalize these results, our modeling framework can extend this literature by demonstrating the effect of symptom under-reporting on a wider range of sports, different SFWP durations, and other types of time-loss injuries after RTP.

A primary focus of previous research on SFWP has been on its effect on repeat concussion. These prior studies have had somewhat conflicting results; a 2009 study by McCrea et al. [6] identified a slightly higher rate of repeat concussions in athletes who had undergone a SFWP (vs no SFWP). In contrast, a 2016 study by Pfaller et al. [48] found that the repeat concussion rate in athletes who underwent a SFWP approached the overall (i.e., natural) concussion rate. In support of this more recent finding, a 2020 study by McCrea et al. [1] later showed that current SFWP practices have resulted in a 41% decrease in repeat concussion rate and fewer repeat concussions within 10 days post-RTP compared with similar data from 1999 to 2001, when the SFWP was only at a median of 2 days. These conflicting results may arise due to slight differences in how the SFWP is defined. Another potential reason is the small sample sizes relative to the rate of repeat concussions; repeat concussions are rare events and thus accurate estimates of repeat concussion rates are challenging to derive—especially in sports where concussions occur infrequently. Our results indeed support the findings that increased SFWPs reduce repeat concussions. Moreover, our simulation-based estimates help to establish estimates of repeat concussion rates with tighter confidence intervals. While further validation and calibration studies are needed to ascertain the accuracy of our estimates, we illustrate that simulation-based methodologies have the potential to address critical knowledge gaps in the sports-related concussion literature.

Aside from repeat concussion rates, our analysis also considers the effects of the SFWP on non-concussion time-loss injuries. Indeed, prior work has posited a connection between premature RTP and increased rates for subsequent non-concussion injuries [57,58,59]. In theory, this increased injury rate may be due to RTP prior to complete neuropsychological recovery. Specifically, concussion is associated with poorer dual task and motor control performance, which has been directly linked with elevated rates of musculoskeletal injuries [58, 60, 61]. While more studies are needed to determine the exact mechanisms for these relationships, our findings help to emphasize the complexity of factors that should be considered in RTP decisions; critically, athletes who play sports with low to medium concussion risk but medium to high time-loss injury rates (e.g., gymnastics, basketball, soccer) may still benefit from a lengthier SFWP.

Our findings also add to the ongoing discussion about sex differences in concussion recovery and post-injury management. Based on previously derived concussion-based risk groupings [31], we found that at the same SFWP, repeat concussion rates are greater for female versus male athletes in low-risk sports (i.e., all sports aside from football, ice hockey, and wrestling). Further, at the same SFWP, non-concussion time-loss injury rates are greater for males versus females in medium-risk sports (i.e., football, ice hockey, and wrestling) and greater for females versus males in low-risk sports. We hypothesize that these differences may be driven by potential differences across recovery trajectories (e.g., time to resolution of symptoms) and/or injury rates. Such differences have been identified in previous research on concussion recovery and epidemiology, although there are some mixed results [5, 51, 53, 62, 63]. For example, a recent analysis of CARE Consortium data by Master et al. [53] has found that female athletes in contact sports and male athletes in limited contact sports faced longer recovery times than their opposite sex counterparts. However, there were no significant differences in concussion recovery between female and male athletes in similar sports overall. Further study on the relationship between concussion recovery and sex are needed to optimize and personalize care for athletes with concussion.

Our study findings can be used to design a criteria-driven SFWP selection method. Consider a female soccer player and recall that the natural injury probabilities for this sport are 1.46% (i.e., 14.6 in 1000 simulations) for repeat concussion and 10.69% (i.e., 106.9 per 1000 simulations) for non-concussion time-loss injuries. Suppose that acceptable levels of risk for repeat concussion and non-concussion time-loss injuries within 30 days post-RTP are 1.55% and 11.0%, respectively. The shortest SFWP that meets these criteria for repeat concussions is 10 days and for non-concussion time-loss injuries is 16 days. Thus, to satisfy both criteria, a 16-day SFWP is recommended. Note that this SFWP choice is specific to the athlete’s sport, sex, and ‘safe RTP’ criteria. For example, if the athlete was a female tennis player, then the natural injury probabilities are 0.29% for repeat concussion and 5.62% for non-concussion time-loss injuries. For this type of athlete, the risk of repeat concussion may be considered negligible, so the criteria may be based solely on the risk of non-concussion time-loss injuries. If an acceptable level of risk is 6.3%, then the shortest SFWP satisfying this criterion is 7 days. Thus, a 7-day SFWP would be recommended.

This simple procedure facilitates a shared decision-making approach, allowing clinical staff to work together with athletes and coaching staff to determine a suitable timeline for RTP based on an acceptable risk tolerance. This proposed approach also highlights the importance of tailoring SFWP by sport; low-risk sports, for example, may reach acceptable levels of post-RTP injury risk much more quickly compared with high- or medium-risk sports because their baseline injury rates are already low. Moreover, the marginal reduction in injury risk by increasing SFWP may be less for low-risk sports as time goes on. To this end, our proposed approach only displays one potential decision criterion. Several other criteria can also be considered, such as the Population Attributable Fraction, that is, the proportion of the estimated post-RTP injury probabilities attributable to RTP from concussion and the length of the SFWP versus the natural risk inherent in the sport (see Section S4 of the ESM). Simulation–optimization techniques can also be considered to facilitate the optimization of SFWP for more complicated criteria [27]. While our study suggests that tailoring SFWPs may help clinicians balance the length of the total RTP duration with sport-specific post-RTP injury risks, the implementation of lengthier SFWPs may be met with practical challenges. In particular, symptom under-reporting is well documented in the sport-related concussion literature [55, 56]. While longer SFWPs could reduce injury risks after RTP, athletes may under-report symptoms to expedite RTP and avoid missing future games [64]. As such, multi-faceted approaches to concussion care which include concussion education for athletes [65] in tandem with the optimization of concussion management decisions are critical.

Our study is not without limitations. First, external validation of our simulation results is not currently possible due to the dearth of existing clinical studies on the relationship between SFWP and post-RTP injury rates. To this end, we calibrated our models to recovery trajectory statistics identified from the CARE Consortium data. However, future work can expand our simulation models to increase the number of points at which calibration is possible—with the hope of improving the overall accuracy of simulation-based estimates. Second, our analysis was restricted to collegiate varsity athletes with one or no previous concussions and limited to the SAC, BESS, and SCAT. Further, our concussion recovery models are skewed towards recovery trajectories of athletes in American football, soccer, basketball, and volleyball, which comprise the majority of concussions in our data. To broaden the generalizability of our model to other populations, additional data which includes athletes outside these demographics (e.g., high school athletes) and sports should be used to parameterize our models. Additional assessments such as detailed neuropsychological testing, the Vestibular-Ocular Motor Screening, and blood biomarkers could also be used to extend our modeling framework and evaluate the value of these tools in RTP decisions. Third, we only considered the duration of the SFWP and did not analyze the criteria for beginning the SFWP. In practice, clinicians and athletic trainers must determine when an athlete is asymptomatic before beginning the graded RTP protocol, and these criteria are often left to clinical judgment. Understanding how these criteria impact post-RTP injury rates would be a critical avenue for future research. Finally, our Markov models were designed to replicate ‘average’ concussion recovery trajectories by sex. This approach was useful for generating population- or group-based estimates but may not adequately capture individual differences in concussion recovery (e.g., based on prior injury history, diagnosed medical conditions, etc.). Future research can address this limitation by incorporating varying concussion subtypes [66, 67] or athlete subgroups into our current Markov models.

5 Conclusion

In this research, we estimated the relationship between the SFWP and post-RTP injury rates by sex and sport using simulation. We found that longer SFWPs result in greater injury rate reduction for athletes in higher risk sports compared with athletes in lower risk sports, and that the SFWP should be tailored based on an individual athlete’s risk for future injury. While future research is needed to validate and generalize our findings, this analysis facilitates the development of data-driven concussion management guidelines.