Introduction

Parkinson’s disease (PD) is the second-most common neurodegenerative disorder that affects 2–3% of the population ≥ 65 years of age. Although clinical diagnosis relies on the presence of bradykinesia and other motor features, PD is associated with many non-motor features that add to overall disability [1, 2]. Among the non-motor symptoms, cognitive deficits are probably the most relevant, as they affect patients’ autonomy, increase caregiver burden and wield a considerable socio-economic impact [3]. As reported by the Movement Disorder Society, nearly one-third of non-demented patients with PD are affected by mild cognitive impairment (MCI) [4, 5]. The cognitive profile of these patients is characterized by executive deficits, with specific impairments in attention, processing speed, working memory, set-shifting, and planning [6]. Additionally, these patients might exhibit also deficits in other cognitive domains such as visuospatial abilities, memory, or language [7].

Although dopamine replacement medications and deep brain stimulation ameliorate motor symptoms, they are less effective in the treatment of cognitive deficits [8]. Non-pharmacological interventions focused on the neuropsychological aspects could play a pivotal role for patients’ well-being. Among non-pharmacological interventions, cognitive rehabilitation and cognitive trainings include behavioral interventions aimed at reducing cognitive impairment and straighten cognition [9]. In particular, cognitive trainings refer to the teaching of strategies and to the execution of specific tasks targeting cognitive functions [10]. Although the mechanism of action has not been fully clarified yet, it has been suggested that neuroplasticity processes might account for the beneficial effect observed after cognitive trainings [11]. Probably by inducing changes in the pattern of brain activation and in the gray matter volume [12], these treatments have the potential to slow down symptom progressions and help patients to maintain a high level of autonomy and quality of life [13]. Accordingly, a previous meta-analysis reported that cognitive trainings are effective in improving cognitive functioning in PD patients. Namely, in this study, the authors found that cognitive training improves working memory, processing speed, and executive functions [14]. However, the literature has grown since the publication of this study. Conversely, a more recent meta-analysis [15] reviewed the effects of cognitive trainings in patients with PD-MCI and dementia (PD-D). The authors did not find evidence of any important cognitive improvement after cognitive training. However, the inclusion of studies conducted in patients with PD-D makes these two previous meta-analyses not directly comparable. Indeed, there is a crucial difference in the employment of cognitive trainings in MCI and in demented patients. In patients with MCI, cognitive trainings have the aim of slowing the progression into dementia, whereas in patients with dementia, trainings are employed with the aim of stimulating or compensating more severe cognitive decline.

To date, questions remain as to whether cognitive rehabilitation is effective in improving cognitive functions in PD-MCI when compared to other interventions. As 20% of PD patients present cognitive impairments that might turn into dementia [16], clarifying whether cognitive trainings could be used to manage cognitive symptoms in order to slow down the progression into dementia and to maintain cognitive function would be pivotal for both research and clinical purposes. Additionally, the large percentage of people with PD-MCI at diagnosis, and the societal cost, as well as timely access to high-quality neuropsychological care is important to control, maintain, or increase the quality of life. However, to date, conflicting evidence for efficacy is present in literature. Therefore, the aim of the present study was to investigate whether cognitive rehabilitation improves cognition in PD-MCI patients and, if so, which cognitive domains mostly benefit from the rehabilitation. To this end, we have carried out a systematic review and meta-analysis of previous studies conducted on PD-MCI patients and analyzed behavioral outcomes for each cognitive domain separately.

Methods

Search strategy and study selection

This meta-analysis was conducted following the Preferred Reporting of Systematic Reviews and Meta-Analyses (PRISMA) [17] statement and was registered with PROSPERO, number CRD42020210652.

We searched in the following online databases for published articles and registered trials: Cochrane, EMBASE, SCOPUS, and PubMed from 01/2000 to 09/2020. Namely, we used the following keywords: “Parkinson’s disease,” “cognitive rehabilitation,” “cognitive training,” “cognitive treatment,” and “neuropsychological measures.” We also screened the bibliography of previous meta-analyses as well as of all the included studies (Fig. 1).

Fig. 1
figure 1

Flow diagram based on PRISMA Statement [32]

Full size image

Studies to be included had to investigate changes in cognitive tests in PD-MCI patients before and after a cognitive training compared to a control treatment. In particular, we selected studies that reported individual cognitive trainings counting at least 10 total sessions.

To be included, candidate studies had to meet the following criteria:

  • Being at least single-blind;

  • Sample size ≥ 20 participants;

  • Participants recruited had idiopathic PD diagnosed by using the UK’s Parkinson’s disease society Brain Bank clinical criteria;

  • Participants exhibiting deficits in at least one cognitive domain;

  • Not including patients with dementia;

  • Using cognitive rehabilitation;

  • The effect of the intervention was measured before and after the treatment by using standardized neuropsychological tests;

  • The article was written in English.

Data extraction and risk of bias assessment

Candidate studies were excluded whether they enrolled patients with other neurological or systemic pathologies, whether patients were unblinded or whether means and standard deviations were not reported. If potentially eligible studies did not report means and standard deviations, the corresponding authors were contacted by email and in case of no response in the following two weeks, the study was excluded from the analysis.

Two authors independently screened the article titles and abstracts collected from the database search. Only articles meeting the inclusion criteria were selected. Any doubt or disagreement was discussed and solved among all the authors.

Selected articles were read by one author that extracted relevant information such as participants, interventions, comparisons, outcomes, and study design (PICOS). Additionally, data relative to sample characteristic, methods, and results were extracted. Cognitive outcomes were categorized for the following domains: global cognition, executive function, attention, short term and long-term memory, language, visuospatial processes, and clinical scales. Means and standard deviations were extracted for each outcome before and after the intervention. The Cochrane tool was used to assess the risk of bias [18] (Figure 1 of the supplementary materials). This tool categorizes studies as having low, high, or unclear risk of bias. The specific domains investigated to assess risk of bias are the following: the appropriateness of sequences generation, the allocation concealment, the blindness of participants and experimenters, and the presence of missing or incomplete data.

Data synthesis and analyses

Analyses were performed using R Studio 4.0.2 [19]. The statistical significance was set at p <0.05; for the test relating to publication bias, a p <0.10 was considered significant.

Effects of cognitive training were analyzed separately for each domain considered. Studies using more than one test per cognitive domain were treated as separate studies [20]. Standardized mean differences (SMD) were calculated with Hedges’ g method. Data relative to cognitive tests were adjusted taking into account length and direction of the relative scale in order to allow a comparison among different tests [21].

Heterogeneity among studies was assessed using the total Cochrane Q test (Q) that evaluates if the variability among effect sizes is greater than expected. A significant Q value indicated a lack of homogeneity among studies. Additionally, the I-square inconsistency index (I2) was used to quantify the percentage of true variability in the observed effects (true heterogeneity). I2 values were classified as low (25–50%), moderate (50–75%), or high (75%) [22]. In the case of high heterogeneity, data were pooled using a meta-analytic method based on a random effect model with the Knapp-Hartung adjustment method (HKSJ) [21]. An influence analysis was conducted by using the Baujat Plot and the Leave-one-out method to detect influential cases. A Gosh analysis was performed to individuate and remove outliers among influential cases. Forest plots were generated for each meta-analysis [23]. Publication bias was assessed by performing and inspecting a contour-enhanced funnel plot and by running the Egger’s t-test [24].

Results

Included studies: main characteristics

A total of 14 studies including 767 patients met the inclusion criteria. All studies but one [25] were randomized clinical trials. In the included studies, patients were enrolled if they: i) had an impairment in an objective cognitive test of executive functions but were not demented as assessed with the scales for outcomes in PD cognition scale [26]; ii) had a mini mental state examination (MMSE) score > 24 and absence of diagnosis of dementia [27, 28]; or iii) had a mild to moderate cognitive impairment but not dementia [29, 30]; iv) had a MMSE < 25 but they had not dementia [31,32,33]; v) had a diagnosis of MCI in accordance with Petersen’s criteria [34]. Four studies mentioned as inclusion criteria an overall absence of dementia [25, 35,36,37]. One study included only participants with MCI using established diagnostic criteria [28] and 1 study enrolled participants with single or multiple domain MCI including executive dysfunction [38]. With respect to the treatment, 7 studies compared cognitive trainings with other control conditions [27, 29, 31, 34, 36]. One study used the cognitive training as control condition whereas the experimental group underwent to a virtual reality based training [30]. Two studies used cognitive rehabilitation as control to test whether combining cognitive training with motor exercises induced an improvement in cognition [28, 33]. Another group of studies compared the effect of domain specific cognitive rehabilitation with not specific cognitive trainings [26] or active control conditions [37] (see Table 1). With respect to studies included in the analyses, 5 studies where included for global cognitive functioning [28, 30,31,32, 34, 38], 8 studies contributed to the analysis of attention [25, 26, 28, 30, 31, 34, 35, 38]; 9 studies were included in the meta-analysis for executive functions [25, 26, 28, 30, 34, 35, 37, 38]; 5 studies contributed to the analysis of language [30, 31, 33, 34, 38]; 8 studies were used for the long term memory [25, 30, 31, 33,34,35, 37, 38]; 7 studies contributed to the analysis relative to short term memory [30, 31, 33,34,35, 37, 38]; 5 studies were used for visuospatial abilities [27, 30, 31, 34, 38] and 9 studies contributed to the analysis of the clinical scales [25,26,27,28, 31, 33,34,35, 37] (Table 1 of the supplementary materials).

Global cognition

We pooled data from 5 studies [28, 30, 31, 34, 38] with 314 participants. As studies reporting several cognitive tests for each domain were considered separately, the total number of cases included in the analysis was 12 and the final sample size of the meta-analysis was of 632 participants. Heterogeneity was high (pQ< 0.01; tau2 = 48.02; and I2 = 97.8%). The influence analysis revealed that 9 studies were influential cases. After performing the Gosh analysis, 6 studies were identified as outliers and were removed from the analysis (see Fig. 2a: París a-b, 2011; Maggio a-b, 2018; Reuter a, 2012; Alloni b, 2018). After having removed the outliers, we did not find a significant decrease in heterogeneity (pQ< 0.01; tau2 = 6.33; I2 = 97.5%) but we observed a significant difference between the experimental group and the control group in favor of the experimental group. Although the funnel plot was asymmetrical (Fig. 2b) [35, 40], the Egger’s regression intercept revealed the absence of publication bias (β = −18.09; 95% CI = [−35.81; −0.37]; p = 0.12). This inconsistency is probably due to the numerical scarcity and high heterogeneity of the included studies.

Fig. 2
figure 2

Forest plot (A) and funnel plot (B) of global cognition. The funnel plot revealed a publication bias [35, 40] not confirmed by the Egger’s regression.

Full size image

Attention

Data were pooled from 8 studies [25, 26, 28, 30, 31, 34, 35, 38] with 409 participants. As studies reporting several cognitive tests for each domain were considered separately, the total number of cases included in the analysis was 17 and the final sample size of the metanalysis was of 747 participants. Due to the high heterogeneity (pQ < 0.01; tau2 = 12.54; and I2 = 93.6%), we applied a random effect model. The influence analysis revealed that 3 studies had an influential role biasing the results of the analysis. After performing the Gosh analysis, 4 outliers were identified and were removed (see Fig. 3a: Vlagsma a-b, 2020; Petrelli, 2014; Reuter, 2012), leading to a significant reduction in heterogeneity (pQ =1; tau2 = 0.0006; I2 = 0.0%). Although the general effect shows that there is a difference between cognitive training and control treatment in favor of the cognitive training, none of the studies individually reaches significance. The funnel plot was symmetrical (Fig. 3b). The Egger’s regression intercept revealed no publication bias (β = −0.16; 95% CI = [−1.62; 1.3]; p = 0.83).

Fig. 3
figure 3

Forest plot (A) and funnel plot (B) of attention. The funnel plot shows no publication bias

Full size image

Executive functions

Data were pooled by 9 studies [25, 26, 28, 30, 31, 34, 35, 37, 38] with 461 participants. As studies reporting several cognitive tests were considered separately, the total number of cases included in the analysis was 27 and the final sample size of the metanalysis was of 1341 participants. Due to the presence of high heterogeneity (pQ< 0.0001; tau2 = 275.58; I2 = 97.9%), a random effect model was used. The influential analysis revealed that 9 studies were influential cases, of which 6 outliers were removed by performing the Gosh analysis (see Fig. 4a: Petrelli, 2014; Reuter a-b-c, 2012; Fellman a-b, 2018). We found that the reduction in heterogeneity was not significant (pQ< 0.01; tau2 = 1.80; I2 = 85.7%) but we did find a significant difference between the cognitive training and the control intervention in favor of the cognitive training. The funnel plot suggests the presence of publication bias (Figure 4B) [2, 14, 25, 33]. The bias was confirmed by the Egger’s regression intercept (β = −8.15; 95% CI= −10.91; −5.39]; p < 0.001).

Fig. 4
figure 4

Forest plot (A) and funnel plot (B) of executive functions. The funnel plot shows the presence of a publication bias [2, 14, 25, 33]

Full size image

Language

Data were pooled by 5 studies [30, 31, 33, 34, 38] with 177 participants. As studies reporting several cognitive tests were considered single studies, the total number of cases included in the analysis was 7 and the final sample size of the meta-analysis was of 336 participants. Due to the high heterogeneity (pQ< 0.01; tau2= 5.33; and I2= 91.7%), a random effect model was applied. The influential analysis revealed that 3 studies had an extreme effect size. After having performed the Gosh analysis, 1 study was identified as outlier and was removed from the analysis (see Figure 5A: Petrelli et al., 2014), leading to a significant decrease in heterogeneity (pQ= 0.02; tau2= 0.338; and I2= 63.8%). We did not find evidence in favor of the experimental group. The funnel plot showed some publication biases (Figure 5B) [33]. However, the Egger’s regression intercept was not significant (β = −2.38; 95% CI= [−10.65; 5.89]; p = 0.60). This inconsistency might be due to the numerical scarcity of the studies and their high heterogeneity.

Fig. 5
figure 5

Forest plot (A) and funnel plot (B) of language. The funnel plot shows the presence of a publication bias [33] not confirmed by the Egger's regression.

Full size image

Long-term memory

Data were pooled by 8 studies [25, 30, 31, 33,34,35, 37, 38] with a total of 225 participants. As studies reporting several cognitive tests were considered separately, analyses were performed on 19 cases and the final sample size of the metanalysis was of 706 participants. Due to the high heterogeneity (pQ< 0.01; tau2= 56.32; and I2= 93.6%), a random effect model was applied. The influential analysis revealed that 5 studies had an extreme effect size. After having applied the Gosh analysis, the study of Petrelli and colleagues was identified as outlier (see Figure 6A Petrelli et al., 2014). After having removed the outlier, we found a significant reduction in heterogeneity (pQ< 0.01; tau2 = 0.584; I2 = 78.8%) as well as a significant difference between the experimental and control group. The funnel plot was asymmetrical and the Egger’s regression intercept was significant (β = −6.49; 95% CI = [−12.85; −0.14; p = 0.07]). These results indicated the presence of publication bias (Figure 6B) [2, 14, 34].

Fig. 6
figure 6

Forest plot (A) and funnel plot (B) of long-term memory. The funnel plot shows the presence of a publication bias [2, 14, 34]

Full size image

Short-term memory

Data were pooled from 7 studies [30, 31, 33,34,35, 37, 38] with a total of 219 participants. As studies reporting several cognitive tests were considered separately, analyses were performed on 17 cases and the final sample size of the metanalysis was of 606 participants. Due to the high heterogeneity (pQ< 0.01; tau2 = 2595.13; and I2 = 95.9%), a random effect model was applied. The influential analysis revealed that 6 studies had an extreme effect size. After having applied the Gosh analysis, the study of Petrelli was identified as outlier and was removed from the analysis (see Figure 7A: Petrelli, 2014), leading to a significant reduction in heterogeneity (pQ = <0.01; tau2 =3.25; I2= 91.5%). Overall, the results for the random effects model without outliers showed a significant difference between the experimental and the control group in favor of the experimental group. The funnel plot was asymmetrical and the Egger’s regression intercept was significant (β = −9.02; 95% CI = [−13.44; −4.6]; p =0.0017). These results confirm the presence of publication bias (Figure 7B) [34, 37, 38].

Fig. 7
figure 7

Forest plot (A) and funnel plot (B) of short-term memory. The funnel plot shows the presence of a publication bias [2, 14, 33]

Full size image

Visuo-spatial abilities

Data were pooled by 5 studies [27, 30, 31, 34, 38] with 197 participants. As studies reporting several cognitive tests were considered separately, analyses were performed on 6 cases and the final sample size of the metanalysis was of 225 participants. Due to the high heterogeneity (pQ< 0.01; tau2 = 2.89; and I2 = 93.1%), we applied the random effect model. The influential analysis revealed that 5 studies were influential cases, and 2 studies were outliers (see Figure 8A: Edwards, 2013; Petrelli, 2014). After having removed the outliers by using the Gosh analysis, we found a significant reduction in heterogeneity (pQ< 0.01; tau2 = 1.42; I2 = 84.8%). The result of the meta-analysis revealed no significant differences between the control and the experimental group. The funnel plot was asymmetrical and suggested the presence of publication biases (Figure 8B) [25, 33]. This bias was confirmed by the significance of the Egger’s regression intercept (β = −9.1; 95% CI = [−13.48; −4.72]; p = 0.06).

Fig. 8
figure 8

Forest plot (A) and funnel plot (B) of visuo-spatial abilities. The funnel plot shows the presence of a publication bias [25, 33]

Full size image

Clinical scales

Data were pooled by 9 studies [25, 27, 28, 31, 33,34,35, 37] with 549 participants. As studies reporting several cognitive test were considered separately, analyses were performed on 19 cases and the final sample size of the metanalysis was of 867 participants. Due to the high heterogeneity (pQ< 0.01; tau2= 446.82; and I2= 96.5%), a random effect model was applied. The influential analysis revealed that 7 studies had an extreme effect size. The Gosh analysis identified 9 outliers, which were removed from the analysis (see Figure 9A: Barboza; Naismit a; Parìs a-b; Petrelli b; Peña a-c; Reuter; Vlagsma). After having removed the outliers, we found a heterogeneity increase (pQ< 0.01; tau2= 232.44; I2= 98.1%) but no significant differences between the experimental and control group. The asymmetry of the funnel plot shows that there were publication biases (Figure 9B). This result was confirmed by the significance of the Egger’s regression intercept (β= −8.417; 95% CI= [−13.9; −2.93]; p= 0.024) [25, 35, 31].

Fig. 9
figure 9

Forest plot (A) and funnel plot (B) of clinical scales. The funnel plot shows the presence of a publication bias [29, 34, 35]

Full size image

Discussion

The aim of the present study was to assess the efficacy of cognitive rehabilitation in PD patients. Our results show a selective effect of cognitive trainings in cognition. In particular, we found a beneficial effect of these trainings in global functioning, in executive functions, and in short- and long-term memory. On the other hand, we did not find any beneficial effect in attention, in visuospatial abilities, nor in clinical scales. Overall, our results are in line with previous studies reporting that cognitive rehabilitation could be effective in ameliorating cognition in PD [14]. However, not all the cognitive functions might benefit from the training and the specific cognitive status should be taken into account in order to adapt trainings to the patients’ needs. Indeed, the profile of PD patients with cognitive impairment is variable, both in the affected functions and in the domains impaired first [39].

Previous meta-analyses reported conflicting results [14, 15]. Namely, Leung [14] found that cognitive rehabilitation might have a beneficial impact on cognitive functions, whereas Orgeta [15] did not confirm these findings. Our results insert themselves in this strand of research. Indeed, with respect to the meta-analysis of Leung, our work includes more recent researches whereas, with respect to the metanalysis of Orgeta, we did not include patients with dementia as the severity of the disease might influence the effects of the trainings.

In the present study, we found a significant effect of cognitive trainings in global cognition as measured with several neuropsychological tests. Despite this result should be taken cautiously due to the high heterogeneity of the studies in this domain, it is encouraging. Indeed, improving global cognition is one of the main goals of each cognitive rehabilitation. However, this finding is in contrast with the results reported by two previous meta-analyses conducted by Leung [14] and Orgeta [15], respectively. This discrepancy might be due to differences in studies’ selection. Indeed, we included a larger pool of studies compared to these two previous works. Additionally, studies included in the meta-analysis of Leung mostly assessed global functioning through the MMSE. Similarly, Orgeta and colleagues assessed the effect on global cognition selecting only MMSE scores. On the contrary, in our meta-analysis, global cognition was assessed through several cognitive tests and not limited to the MMSE.

MMSE is not the elective tool for the assessment of cognitive status in PD patients; indeed, it has not enough discriminant validity for the detection of cognitive disorders [40]. The lack of sensitivity of the MMSE might have affected the results of previous studies, reducing the detection of cognitive changes. Indeed, other tests have been recommended for the screening of PD, such as the Montreal Cognitive Assessment and the Parkinson’s disease cognitive rating scale [41]. The inclusion of other scales measuring the global cognitive status in the present meta-analysis probably increased the chance to detect the presence of a cognitive deficit and its relative change after the training.

Global cognition constitutes an indicator of the overall patients’ cognitive status and its assessment often guides clinical practice. Therefore, improving global cognition could greatly affect medical choices with respect to the best practice for the treatment of PD as well as patients’ quality of life.

Besides global cognition, we found a beneficial effect of cognitive training in executive functions. As deficits in this domain are common in patients with PD [42] and play an important role in everyday activities, the evidence that they improved in response to cognitive training is encouraging and suggests the need to further explore this strand of research. The improvement in executive functions is in line with previous studies [14]. Nevertheless, Orgeta and colleagues did not find any evidence of an improvement in executive functions after cognitive rehabilitation. In this last study, executive functions were analyzed by taking into account the performance in two specific executive tests that are the Trail Making Test and the Stocking of Cambridge test. In the present study, other tests have been included such as the Frontal Assessment Battery and the Stroop test. The inclusion of a wider variety of tests might account for our effect in executive functions. Overall, these results highlight that cognitive intervention might be a fundamental treatment to improve patients’ functioning. Due to the relevance that executive functions play in the most common daily life activities, and their role in implementing strategies needed to overcome motor deficits [43], such improvement suggests the need for further studies investigating effects of cognitive training in this pathological population.

We found a beneficial effect of cognitive training in short- and long-term memory. This result is not in line with previous meta-analyses, where no changes in memory were found [14, 15]. In these previous studies, memory domain counted both short- and long-term memory tests. Additionally, in the study of Orgeta, only the verbal component of memory was analyzed. These differences might account for the mismatch in the results. On the other hand, also the different pool and number of studies included in the meta-analyses (9 in our study vs. 5 in Leung’s and Orgeta) might account for the discrepancy in the results. Finally, another possible explanation is that in the present meta-analysis, we included studies focusing on specific memory trainings [25, 28]. Such inclusion might have strengthened the effects in memory domain. In line with these results, cognitive trainings have been found to be overall effective in the treatment of memory impairment in many pathological populations [44].

In line with previous works, we did not find evidence in favor of cognitive treatments in attention. Indeed, although the general effect was in favor of cognitive trainings, none of the studies individually reached significance, probably due to their limited sample sizes. This trend is encouraging and should be further explored in future studies. Additionally, we did not find evidence in favor of cognitive treatments in visuospatial abilities nor in clinical scales. This might be probably due to the lack of training focusing on these aspects.

Due to the lack of efficacy of pharmacological therapies in PD cognitive symptoms, it is important to develop new treatments targeting these common deficits. Cognitive trainings are non-invasive, safe, and low cost; therefore, more studies aimed at improving the quality and the efficacy of cognitive trainings would be needed. For instance, cognitive trainings might be combined with motor and physical activity to reach more stable results and improve quality of life [45]. Additionally, the combination of cognitive intervention in PD patients and psycho-education with caregivers has shown promising results [46].

An important issue that we have to acknowledge is that we observed a high variability among included studies with respect to both the neuropsychological tests used for the diagnosis of MCI and to the criteria used to rule out the presence of dementia. To reduce differences among published studies, more standardized and shared procedures to assess cognitive impairment in PD patients would be needed.

A limitation of the present meta-analysis was that, despite the included studies were conducted on patients with MCI, two of them did not explicitly state the exclusion of patients with dementia. Thus, we cannot exclude that in these studies some of the enrolled participants had dementia.

The number of studies is still low and in turn the small sample size and the high heterogeneity limit the power of the analysis. More studies should be conducted to straighten the power of meta-analysis focusing on this topic.

Conclusion

This meta-analysis suggests that cognitive training overall improves cognitive performance in patients with PD. In particular, global cognition, memory, and executive functions that are generally impaired in these patients benefit from the treatment. These results are encouraging and highlight the pivotal role that cognitive rehabilitation might play in PD. However, to enhance the overall effect and to better understand mechanisms underlying the improvement, future studies employing larger samples, combining rehabilitation with imaging techniques as well as testing novel trainings, are required.