Introduction

Pediatric asthma exacerbations (PAEs) are one of the leading causes of emergency department visits in developed countries, representing up to 20% of emergency departments visits during autumn and winter in France [1], and 640,000 visits annually in the USA [2]. Therefore, all physicians should have a basic knowledge of the management of PAEs, which should be taught during medical studies.

Simulation-based trainings (SBTs) represent an appropriate approach to teach medical students the management of PAEs. During SBTs, students can work on both the theoretical aspects (the sequence of actions to be performed) and the practical aspects (the use of inhaling devices) of PAE management without any risk to the patient. However, access to SBTs is limited because these trainings require important human and financial resources [3, 4]. It is thus essential to optimize the learning efficiency of SBTs.

A key instructional feature that leads to effective learning in technology-enhanced simulation is repetitive practice, which involves intense and repetitive learner engagement in a focused, controlled domain [5, 6]. Whether this repetitive practice should correspond to the repetition of the exact same task until mastery, or should involve different tasks which share similar characteristics is a question that is still unanswered. For skill-based tasks with a unique procedure such as thoracentesis [7], lumbar puncture [8], or central venous catheterization [9, 10], practicing on the exact same task until mastery was associated with improved outcomes. To a lesser extent, rule-based tasks such as the management of a cardiac arrest, which has a unique presentation and limited variation in its management, can also be taught through the repetition of the same scenario until mastery is achieved [11].

The question becomes more complex for medical diseases which vary in their presentation and their management. For example, the presentation of a pediatric asthma exacerbation (PAE) can range from a simple repetitive cough to a severe respiratory distress, with different implications in terms of management. Simulation training on such topics should allow the learners to transfer the skills they learnt to new, unexperienced presentations of the same disease.

A first solution to reach this objective may be to repeat the management of one type of PAE several times, and to explain orally how to manage the other types. The mastery of one type of PAE may decrease the cognitive resources required to manage several aspects of a new PAE, and enable learners to concentrate on new aspects of the situation [12].

A second strategy would be to vary the scenario of PAE during the training. This would not allow mastery learning of each type of PAE. However the “variability of practice” hypothesis holds that practicing with task variation results in improved learning retention and transfer compared to the repetition of the exact same task [13]. A considerable number of studies have provided empirical evidence for this hypothesis (for reviews, see [14,15,16]).

To our knowledge, only one study compared varied and repeated scenarios in SBTs. This study which involved pediatric residents found that practice on varied scenarios of emergency situations was not superior to practice on a unique scenario [17]. However, the different scenarios used in this study were very different from one another (pulseless electrical activity, hypoglycemia, ventricular tachycardia, and seizure), whereas it was shown that some similarities need to exist to expect transfer of learning [18, 19].

This study focused on the management of PAE in outpatient settings and compared the use of varied scenario versus repeated scenarios during a SBT training dedicated to medical students on this topic. The main outcome was the performance score of students during a scenario of PAE which was new to both groups, 1 week after the initial training.

Methods

A monocentric randomized controlled trial was conducted in the Department of Simulation in Healthcare “Ilumens” in Paris Descartes University (France). It was qualified as exempt research by the Institutional Review Board of Saint Antoine Hospital (CPP Ile de France 5, no. 16141), and the study was registered in ClinicalTrials.gov (Identifier: NCT02754310).

Participants and randomization

Participants were third-year medical students from two French medical faculties who voluntarily participated from May to October 2016. All students aged more than 18 years old were eligible. These students corresponded to novice learners regarding the management of PAEs since they did not receive any training on this specific topic during the first years of their medical studies.

Recruitment took place through e-mails and directly during conferences. Written consent was obtained from all participants.

Participants were randomized in blocks of four and assigned into two groups: the “repetition” and the “variation” groups. The simulation clinical educator (D.D.) used an online software (http://www.randomization.com/) to create the random list of numbers used for the allocation of participants.

Study design and interventions

The study design is presented in Fig. 1. On arrival at the training session (first session), each student completed a multiple-choice questionnaire (MCQ) of 20 questions on pediatric emergency situations which included 5 questions on PAE management. They also responded to 5 questions about their self-efficacy in the management of various pediatric emergencies, including one on the management of PAE. Then, the students watched a 5-min video presenting the features of the pediatric high-fidelity manikin used and its environment.

Fig. 1
figure 1

Study design. After randomization, students participated in a first evaluation (pre-test) before being trained either using three different scenarios of asthma exacerbation (variation group) or the same scenario three times (repetition group). All were then evaluated 1 week and 4 months later on four scenarios each time. PAE asthma exacerbation

Full size image

A first evaluation on the simulator was conducted before the training to ensure that students had little or no skills in asthma management. The scenario was a simulated moderate PAE occurring in a private physician’s office, described in Table S1. Following this first evaluation of 5 min, an individual training with a simulation clinical educator specialized in pediatric pulmonology (D.D.) took place for 25 min, using three scenarios of PAE.

These three scenarios were the same scenario of a moderate PAE in the “repetition group,” while the “variation group” faced three different scenarios (a moderate, a mild, and a severe PAE, Fig. 1 and Table S1) for the same amount of time.

During the evaluation session 1 week later, participants completed the same questionnaire on their self-efficacy regarding the management of a PAE and were evaluated individually on four different scenarios. The first scenario was a “transfer scenario” new for both groups corresponding to a PAE rapidly worsening. The second scenario was an “unrelated scenario” corresponding to severe pneumonia which also presented as a respiratory distress but with a completely different management. The third scenario was the moderate PAE already experienced three times by the repetition group and once by the variation group. The fourth scenario was the mild PAE already experienced once by the variation group. At the end of the assessments, a 5 min long feedback was given to each student on his/her performance, and students completed the same MCQ on knowledge.

Four months later, participants attended a last session which was like the second session. The only difference was the content of two of the scenarios. The “transfer scenario” was a PAE improving before relapsing, and the “unrelated scenario” was a pediatric laryngitis.

Simulation and training characteristics

The characteristics of the training are detailed in the supplemental material. Briefly, the high-fidelity manikin used was a SimBaby (Laerdal, Stavanger, Norway). The environment included a short-acting beta-agonist (SABA), a controller medication (fluticasone), an asthma spacer with a facial mask, oral steroids in tablets (prednisone) and in syrup (betamethasone), among other tools.

The learning objectives of the training session were the following: (i) to recognize the symptoms of a PAE on the simulator, (ii) to master the inhalation technique, (iii) to respect the sequence and the dose of the medications needed, (iv) to determine when emergency medical services needed to be called. During the training, the instructor (D.D.) was present with the student in the simulation room. Following the “Four-Component Instructional Design Model,” the instructor provided both procedural and supportive information [20]. Procedural information, which corresponds to the information needed to complete recurrent or routine aspects of learning tasks, was provided through corrective tips and concurrent feedback. It focused mainly on the inhalation technique. Supportive information, which corresponds to information about the subject domain essential for the reasoning and problem-solving aspects of learning tasks, was provided when participants were waiting for the SABA to have an effect. It took the form of general questions on PAE listed in the supplemental material.

Outcomes and measurements

For each scenario, a specific performance score was developed. The main outcome was the performance score of participants on the “transfer” scenario at 1 week. Secondary outcomes were the score of participants on the “unrelated” scenario, the moderate and the mild PAE scenarios at 1 week, along with all these outcomes 4 months later.

All the checklists developed to assess the scenarios of PAEs were based on both international recommendations and discussions with a group of experts in pediatric pulmonology [1, 21]. They are presented in Table S2. Different items related to the sequence, the dose, and the administration technique of the treatments were developed.

During each evaluation, the investigator (D.D.) rated the actions of the participant on these different checklists. A second rater (J.T.) assessed independently 86% of the participants on video records to estimate inter-rater reliability. J.T was blinded to group allocation, while D.D. was not. There was a high level of rater agreement, with a mean intraclass correlation coefficient of 0.86 for the key events checklists (Table S2).

Questionnaires on self-efficacy and knowledge are presented in Table S3. They were filled on Google Forms (Google, California) and exported in Microsoft Excel (Microsoft, Washington).

Primary data analysis

Sample size calculation is detailed in the supplemental material.

Data were analyzed with GraphPad Prism v5.03 (La Jolla, California). The distribution of continuous data was assessed with the Shapiro–Wilk test. Median values and interquartile ranges are reported, and cohorts were compared in Mann–Whitney U tests. A Wilcoxon signed-rank test was used to compare the scores obtained by students on the knowledge and self-efficacy tests on the first, second, and third sessions.

Results

Population

A total of 85 participants were included in this study. Forty-three were randomly assigned to the repetition group and 42 to the variation group. Their characteristics, which were similar, are presented in Table 1. No participant was lost to follow-up.

Table 1 Baseline characteristics of participants
Full size table

Skills

Evaluation pre-training

Before the training, most students were not able to diagnose a PAE (Table 2). Once told the diagnosis, their management was often limited to a first administration of SABA before being lost (Table 2). There was no difference between groups.

Table 2 Pre-test evaluation
Full size table

Performance score on transfer scenarios

When evaluated 1 week after the training, the performance of students on the transfer scenario was similar between the two groups with a median score out of 10 (IQR) of 8.3 (7.4–10.0) in the variation group and 8.0 (6.0–10.0) in the repetition group (p = 0.16) (Fig. 2a). Four months later, on another new scenario of PAE, median scores were also similar between the two groups (13.0 out of 16 (10.4–15.3) versus 12.3 (9.8–16.0) (p = 0.88), respectively, Fig. 2b).

Fig. 2
figure 2

Performance scores of students when evaluated on different scenarios. a Performance scores of students evaluated on a transfer scenario corresponding to a worsening asthma exacerbation 1 week after the training. There was no statistically significant difference between the variation (VAR) and repetition (REP) groups. b Performance scores of students evaluated on a transfer scenario of an asthma exacerbation improving and relapsing 4 months after the training did not differ significantly between groups. c Performance scores of students evaluated on an unrelated scenario of pneumonia 1 week after the training, and on a scenario of laryngitis 4 months after the training. No statistically significant difference was observed between the variation and repetition groups. d Performance scores of students evaluated on a scenario of moderate asthma exacerbation used during the training at 1 week (eval1) and 4 months (eval2). *p < 0.05. e Performance scores of students evaluated on a scenario of mild asthma exacerbation used during the training of the variation group only at 1 week (eval1) and 4 months (eval2). *p < 0.05, **p < 0.01

Full size image

Regarding the “unrelated” scenarios which corresponded to respiratory distresses unrelated to asthma, the median scores were similar between groups for the scenario of pneumonia (3.0 out of 5 (3.0–4.0) versus 4.0 (3.0–4.0), p = 0.17), and the scenario of laryngitis used 4 months later (2.0 out of 5 (2.0–3.0) versus 3.0 (2.0–3.0) p = 0.47, Fig. 2c).

Performance score on other scenarios

Students in the repetition and variation groups were exposed three times and one time, respectively, to the scenario of moderate PAE during their training. When evaluated on this same scenario 1 week later, median scores were similar in the two groups (14.1 out of 16 (13.0–16.0) in the variation group versus 15.0 (13.0–16.0) in the repetition group (p = 0.97)) (Fig. 2d). Four months later, the performance of students decreased significantly in both groups, with a median score of 13.6 (11.9–15.3) in the variation group (p = 0.01) and 13.8 (11.0–15.3) in the repetition group (p = 0.03). These results did not differ significantly between the two groups (p = 0.69) (Fig. 2d).

Finally, students in the variation group were trained on a scenario of mild PAE, whereas students in the repetition group were not. When evaluated on this scenario 1 week later, students in the variation group obtained higher scores than those in the repetition group (median score of 9.0 out of 9 (7.5–9.0) versus 7.0 (2.0–9.0), respectively (p = 0.001), Fig. 2e). Four months later, the median performance score decreased significantly in the variation group to 7.5 (IQR 6.8–9.0, p = 0.047) while it remained at the same level, 7.5 (5.0–9.0) in the repetition group (p = 0.29). The difference between the two groups was no longer statistically significant (p = 0.16) (Fig. 2e).

Self-efficacy

Self-efficacy scores are presented in Table 3. At baseline, the median self-efficacy scores were similar between the two groups. One week after the training, scores increased in both groups but the repetition group reached a significantly higher score than the variation group (Table 2). Four months later, the median self-efficacy score became similar between the two groups, but remained significantly higher than those observed before the training (Table 2).

Table 3 Knowledge and self-efficacy scores
Full size table

Knowledge

The median knowledge scores were similar between the two groups at baseline, improved significantly after the training in both groups in the same proportions, and remained at the same level 4 months later (Table 3).

Discussion

Two main conclusions can be drawn from this study. First, our results revealed that the individual 30-min simulation training proposed to our medical students was very effective in teaching them the management of PAEs. The second conclusion was that for our students who had very little previous knowledge on the management of PAEs, varying or repeating the scenarios of PAEs led to similar performances on the transfer tests 1 week and 4 months after the training.

Before the training, our medical students were not able to manage a PAE: after delivering salbutamol for the first time, most of them were totally lost in the management of the PAE. One week after the individual 30-min training, their performance on three different scenarios of PAEs became excellent, regardless of their group. More importantly, there was evidence of long-term retention with an assessment 4 months later which demonstrated that the performance of students remained very good. SBTs involving asthma exacerbation scenarios were proposed by other authors [22,23,24,25], but to our knowledge, this is the only study to have conducted a rigorous evaluation of the training including transfer and retention tests. These results are important for medical educators who would like to implement trainings on PAEs management in their medical faculties. Another specificity of this study was to place medical students in an outpatient setting rather than in a hospital. This was a deliberated choice because the management of PAEs by physicians in outpatient settings is similar to the management of PAEs by parents at home. The SBT we proposed allowed students to acquire a thorough understanding of the way parents should manage a PAE at home and use inhaling devices, to help them to provide high-quality asthma education in the future. This seems important to us because several studies revealed that most physicians have difficulties in explaining asthma management to families, in particular regarding inhalation techniques [26,27,28,29].

The other finding of this study was that the “variability of practice” hypothesis was not supported by our results: students who practiced with task variation (on three different scenarios of PAEs) did not outperform students who practiced repeatedly on the same task (a unique scenario of moderate PAE) on a transfer test involving a scenario of PAE new to both groups.

Several hypotheses can explain these results. A first reason may be that students in the variation group were exposed to only three different scenarios of PAE during the training. Because it was demonstrated that increasing the number of situations enhances transfer [30], participants in the variation group may have reached better results if they had been exposed to a larger number of scenarios. However, the increased duration of the training would not have been compatible with the time constraints associated with SBTs. Another explanation for the absence of difference observed may be that the scenarios used in the variation group shared many similarities. It can be argued that the low degree of variability between the different scenarios in the variation group was not sufficient to make a real difference with the repetition group. However, the variability of the scenarios we could propose was limited by the variability of PAEs observed in real life, with patients presenting mild, moderate, and severe exacerbations. To increase the degree of variability of our scenarios, the solution would have been to use different causes of respiratory distress (cardiac failure, pneumonia, PAE) rather than three different types of PAEs in the variation group. However, it was shown that transfer occurs when similarities can be found between the different tasks during the training, and between the tasks used during the training and the transfer test [18, 19]. Tofil et al. who trained pediatric residents using varied scenarios (pulseless electrical activity, ventricular tachycardia, and seizure) or similar scenarios (three pulseless electrical activity) did not find any transfer of skills to a new scenario of critical coarctation [17]. Therefore, we preferred a low degree of variability using different scenarios of PAEs rather than a high degree of variability which was previously shown to be ineffective for transfer of skills.

The analysis of the results regarding students’ performances on the two scenarios which were used during the training and the evaluations (moderate and mild PAE) is also interesting. Participants in the repetition group exposed three times to the moderate PAE scenario did not perform better than students in the variation group who were exposed just once. These results contrast with findings from other studies which found that repetition was associated with improved performance [17, 31]. An explanation may be that the scenarios used in our study were easy to learn for the two groups, leading to a “ceiling effect” when students were evaluated 1 week and 4 months later. The conclusion is that for the SBT we proposed, varying the scenario offered the advantage to expose our students to different situations without impairing their performance, particularly on the case that the repetition group encountered three times. Indeed, despite lower self-efficacy scores, students in the variation group performed equally as well as those in the repetition group on all the scenarios used for the evaluation.

This study presents several limitations regarding the generalizability of our results. First, our study included volunteer third-year medical students from two French universities, and the extent to which our results are generalizable to other settings and to students with other levels of experience is unknown. Second, it was a simulation study, and future research should assess whether such findings may be translated in real practice.

In conclusion, the management of different PAEs can be rapidly learnt by medical students using a short SBT. Varying practice with different scenarios was equivalent to repetitive practice on the same scenario in terms of transfer of skills to a new scenario of PAE. Future studies should evaluate whether the acquisition of conceptual understanding of a medical situation can occult the effects of variability in practice.