Introduction

Although improving in recent years, survival following pediatric in-hospital cardiac arrest remains poor, at around 40 % [8]. Recent studies suggest that the quality of cardiopulmonary resuscitation (CPR) delivered by pediatric residents is low and complies poorly to the American Heart Association (AHA) guidelines [12]: deviance from the guidelines is associated with poorer outcomes [23]. Better teaching concerning cardiac arrest to medical students should improve future resident performance.

Simulation technology is essential to teaching resuscitation [21]: compared to no instruction [20], it significantly improves knowledge and skills and adherence to AHA guidelines during actual resuscitation [28]. However, its effectiveness relative to that of another form of instruction is less clear in terms of knowledge [20] and retention of both knowledge and skills may not exceed several months [7, 18, 19]. Very few studies report the real cost [30].

In 2012, the Pierre and Marie Curie medical faculty in Paris introduced a new way of teaching the management of pediatric cardiopulmonary arrest (CPA) to its 400 graduating medical students. This course is based on a 23-min video and 3-h simulation. Pediatric cardiopulmonary resuscitation had previously been taught by a 1-h lecture. The aim of this study was to assess the effectiveness of this new approach in terms of student satisfaction and knowledge and skills and to compare the costs of the innovative and traditional methods.

Methods

Intervention

The pediatric resuscitation curriculum delivered by our faculty consisted of two parts: first, all second-year medical students completed a 1-h basic life support course (BLS) including adult and pediatric BLS. Then, they attended a 1-h lecture dedicated to pediatric cardiac arrest during the fifth year. The BLS delivered during the second year has been maintained, but the lecture delivered during the fifth year was replaced, in 2012 and since, with an innovative pedagogic approach combining video and simulation in three stages:

  • First, students were invited to attend a 23-min video addressing pediatric CPA in a classroom, after completion of a multiple choice questionnaire (MCQ) on the management of pediatric cardiac arrest. The video was divided into two parts: an uncommented fiction of an intra-hospital cardiac arrest in an infant managed by a hospital team and then a step-by-step presentation of how to diagnose and manage pediatric CPA. At the end of the video, a pediatrician was present to answer questions about pediatric CPA. Students were asked to fill out the MCQ again.

  • Then, 1 week before simulation exercises, students were invited by email to review the entire video at home.

  • Finally, students were divided into groups of eight or nine and attended three hands-on sessions, each lasting 1 h. The aim of the first session was to teach recognition and management of a child in respiratory distress on a low-fidelity mannequin. For the second session, students learned basic life support and compression technique using another low-fidelity mannequin. During the last training session, students were divided into groups of two or three and confronted with a high-fidelity mannequin (SimBaby, Laerdal) simulating three different scenarios: uncomplicated bronchiolitis, bronchiolitis with respiratory distress, and severe bronchiolitis worsening to cardiac arrest. Two teachers were present for each session. At the end of the course, students completed the MCQ a third time and were given the answers.

Post-intervention assessments

Following Kirkpatrick guidelines [15] adapted by Mosley and colleagues [19], we conducted two studies to assess different outcomes:

  • In the first study, we evaluated student satisfaction (level 1), student acquisition of knowledge (level 2b), and student retention of knowledge over a period of time (level 2c).

  • In the second study, we assessed student acquisition of skills (level 2b).

Study 1—assessment of satisfaction and knowledge

Student satisfaction was investigated by questionnaire sent by e-mail on the last day of the course. The questionnaire used a Likert scale and focused on usefulness of the video, usefulness of the training sessions, and opinion of the high-fidelity simulation scenario.

Student knowledge was assessed by a MCQ on pediatric cardiac arrest management composed of 10 items. All the students who attended the innovative course completed the test three times: within 10 min before watching the video, immediately after the video, and at the end of the hands-on sessions.

To assess retention of knowledge, students who attended the innovative course were tested again with the same test in December 2014, 6 months to 1 year after completion of the course (median elapsed time, 9 months).

Results were compared to those of a previous class that had attended the traditional course 1 year before. These students were not tested before or immediately after the traditional course, but took the same test in December 2013, 6 months to 1 year after the lecture (median elapsed time, 9 months). Thus, retention of knowledge between the two classes could be compared.

Study 2—skill assessment

All the students who were about to attend the innovative course in June 2014 were invited to participate in a skill evaluation. Those who accepted were assigned to four groups:

  • A control group who did not receive any teaching

  • A video group who viewed only the pedagogic video

  • A simulation group who attended only the simulation exercises

  • A complete group who attended both the pedagogic video and the simulation exercises

Students involved completed a nine-item survey on gender, simulation experience, and resuscitation education.

They were evaluated individually with a simulated CPA of an 8-month infant (Laerdal SimBaby®) previously admitted for severe bronchiolitis. During the evaluation, students remained alone. They were expected to initiate the first steps of pediatric life support in sequential order. Two video cameras, one on each side of the bed, recorded the student’s actions.

The main outcome was the clinical performance score (out of 13) adapted from Donoghue and colleagues [5] as defined in Table 1.

Table 1 Clinical performance score
Full size table

Secondary outcomes were the time to ventilation, defined as the time elapsed between time 0 and the first rescue breath, the time to compression, defined as the time elapsed between time 0 and the first compression, the no-flow fraction, defined as the proportion of time a pulseless patient did not receive compression, and the no-blow fraction, defined as the proportion of time a pulseless patient did not receive respiratory support, i.e., no ventilation for >10 s.

Video records were anonymized and randomly sorted. Then, two pediatricians involved in resuscitation education analyzed all students’ actions. A stopwatch was used to determine the time at which the first ventilation and the first compression occurred and no-flow and no-blow fractions. Ventilation was interpreted as efficient if the simulator’s chest rose.

This study was part of faculty’s educational activity. Primary data analysis was at the group level and deemed to be exempt from ethical review by the Pierre and Marie Curie medical faculty institutional review board.

Costs

Costs related to equipment purchase and maintenance, personnel costs, and facility costs were considered. Costs related to the making of the video and to the purchase of the equipment (including the high-fidelity simulator) were allocated across 5 years.

Nearly all the physicians involved in teaching the course were chef de clinique assistant, the equivalent of fellow in pediatrics. They all work in public hospitals and earn a salary of 17.06 euros per hour.

Statistical analysis

Data were collected in Microsoft Excel (Redmond, Washington) and analyzed with GraphPad prism software v5.03 (La Jolia, California). Categorical data were compared using the chi-square statistic. For continuous data, median values and interquartile ranges are reported, and cohorts were compared with the Kruskall-Wallis test. A post hoc Dunn’s test was added if a significant difference was observed.

Time-to-event analysis was performed for the time to ventilation and the time to compression. Kaplan-Meier survival curves were estimated and compared with the log-rank test.

Results

Study 1—student satisfaction and knowledge

Student satisfaction

The innovative course was attended by 385 medical students during the year 2012–2013; 273 (71 %) answered the satisfaction questionnaire.

Students who attended the innovative course reported that the pedagogic video helped them to understand cardiac arrest management (88 %) and to identify pediatric specificities (67 %).

They found that this new education resource was more instructive than a traditional lecture (79 %).

On a scale from 0 (useless) to 10 (essential for my practice), 83, 93 and 92 % of students rated 7 or more for the hands-on session on respiratory distress, basic life support, and high-fidelity simulation, respectively.

The high-fidelity simulation was appreciated by 89 % of the students who considered it to be useful for future practice and 69 % asked to do it again.

Overall, 91 % students found the innovative course as a whole useful for future practice and 92 % were satisfied with its organization.

Student knowledge

Of the 385 medical students, 374 (97 %) underwent the knowledge test scored out of 10. The median score was 4 (3–5) before intervention and 8 (7–9) after the complete course (p < 0.001) (Fig. 1). The video alone allowed students to improve their score to 7.5 (6–9) (p < 0.001). This score is significantly lower than the score of 8 (7–9) obtained after the complete course (p = 0.02).

Fig. 1
figure 1

Box-plot of student knowledge scores. Innov 6–12 m 6–12 months after innovative teaching, Tradi 6–12 m 6–12 months after traditional course. *p < 0.001

Full size image

Six months to 1 year later, 411 of 437 (94 %) students (including repeaters) underwent the knowledge test: the median score decreased to 6 (4–7), significantly lower than the score immediately after the completion of the course (p < 0.001) (Fig. 1).

These results were compared to the scores for 398 of 423 (94 %) students from a previous class (2011–2012) who received the traditional course. Their median score 6 months to 1 year after the course was 3 (1–4), significantly lower than the scores both immediately after (p < 0.001) and 6 months to 1 year after (p < 0.001) of the students who attended the innovative course (Fig. 1).

Study 2—student skills

A subgroup of 89 students was assigned into three groups of 22 and one group of 23; 18 students did not respond to the emails sent inviting them to participate and 5 did not attend a part of the course. Consequently, 66 (74 %) were included: 20 in the control group, 16 in the video group, 17 in the simulation group, and 13 in the complete group (Fig. 2).

Fig. 2
figure 2

Flow chart of medical students who participated in skill evaluation

Full size image

Resuscitation training before the teaching intervention was similar for all the groups. Half of the students had attended a BLS course before their medical studies, and nearly all had participated in the mandatory BLS course during the second year of medical school using low-fidelity mannequins.

During the simulated CPA, students who attended the complete innovative course scored higher on the clinical performance score than the control group (Table 2). The differences between the various intervention groups (video versus simulation, video versus complete, simulation versus complete) were not significant.

Table 2 Quality of life support
Full size table

The number of medical students who performed ventilation, the time to first ventilation, and the “no-blow” fraction did not differ significantly between groups (Table 2). It was impossible to assess ventilation efficiency for two students: the hair of one student covered the mannequin’s chest and another did not remove the blanket. All of the 10 (100 %) students who had attended the complete course were able to perform ventilation effectively, but this was true for only 6 of the 20 (38 %) in the control group.

The number of students who performed chest compression, quality of chest compression, and “no-flow fraction” were similar between groups (Table 2). Surprisingly, the time to first compression was significantly lower in the control group than after intervention (Table 2).

Costs

Costs were 24 times higher for the innovative teaching than for the traditional lecture (Table 3). The main costs were due to the equipment used (high-fidelity simulator, educational video) and the human resources involved (the traditional lecture needed 1 h of teaching four times a year, the innovative course required 72 h of teaching four times a year).

Table 3 Costs associated with traditional and the innovative teaching
Full size table

Discussion

This study highlights some of the advantages and pitfalls associated with the implementation of simulation in a medical faculty for its medical students.

The video- and simulation-based course was a success in terms of medical student satisfaction and acquisition of knowledge (study 1). Students appreciated both the video and the simulation exercises. The use of video for medical teaching is likely to develop quickly in the coming years. Posted on YouTube (https://www.youtube.com/watch?v=22W5HDqCB6A), the video used in this study, explaining the management of a pediatric cardiac arrest, was viewed more than 100,000 times in the last 2 years, indicating the potential of this media. E-learning for CPA is already used in several universities [22, 26].

Similarly, simulation sessions with high-fidelity mannequins are known to be greatly appreciated by students [18, 20]; however, student satisfaction is not necessarily correlated with learning.

There are numerous reports that knowledge is improved from baseline by simulation [4, 14, 20]. However, meta-analysis did not find a significant improvement when simulation is compared to another form of instruction and in particular the traditional lecture [14, 20]. The large number of students included in our study allowed us to reveal that knowledge up to 12 months after the courses was significantly better among students attending the innovative course than those attending the traditional lecture.

The innovative course yielded mixed results in terms of skills (study 2). Transferring knowledge into practice is a difficulty reported by both junior and senior doctors [25] and simulation may be of great help. Students who attended the innovative course scored higher on the performance score, indicating that they were more compliant to the guidelines for pediatric CPA management. Their ventilation was also more effective.

We consider it important not to focus solely on what students learn but also to study the potential benefits for future patients. From this viewpoint, our results were disappointing. The time elapsed until the first compression was significantly longer after the innovative course than for controls. Our impression was that students in the control group who did not know pediatric specificities of CPA managed the situation as if it were an adult CPA and focused on compressions. Students who had attended the innovative course appeared to think about each step of the resuscitation and concentrated first on ventilation because both the pedagogic video and simulation exercises emphasized airway management and ventilation in pediatric CPA management. Ventilation is essential in pediatric CPA of non-cardiac origin [1], improving survival and survival with favorable neurological outcome [16]. However, a large proportion of cases of pediatric CPA have a cardiac cause (30 % in the recent study by Gupta [9]), and for these patients, ventilation does not help [16].

Hunt and colleagues evaluated their pediatric residents and reported a similar observation: 33 % never started compressions and focused on ventilation [12]. Thus, their team decided to give equal emphasis to “airway and breathing” and “circulation”. These changes to the message delivered associated with “rapid cycle deliberate practice” resulted in a marked improvement in the skills of their residents [11]. A method to improve our students’ performance would have been to simplify the resuscitation algorithm. Following international guidelines, our video described 11 steps for the resuscitation, which turned out to be difficult to remember for our students. Medical students should be offered a simplified pediatric resuscitation course as is the case in other faculties [2]. Indeed, a four-step CPR has been shown to allow better skill retention than an eight-step sequence [10].

As expected, costs were much higher for the innovative course than for the traditional lecture. However, we demonstrate that the innovative course led to improvements in knowledge and skills and may allow earlier recognition of a critically ill child and better compliance to resuscitation guidelines, resulting in improved outcomes [23]. Its annual cost, 29,310€ for 400 medical students, is comparable to the mean cost related to the length of stay following one pediatric cardiac arrest, 26,172€ (22,562£, Euro 2009) in a British report [6]. More broadly, the cost per survivor of an in-hospital acute life-threatening event to hospital discharge was 61,815€ (53,289£, Euro 2009) in this same study. Medical costs following hospital discharge and non-medical costs such as those related to the lost lifetime productivity of a child dying should be added to these calculations [29]. Thus, the high cost of the innovative course should not preclude its implementation in medical faculties because it may save lives and reduce costs related to pediatric life-threatening events. Furthermore, the video was made for medical students but has already been viewed by a much broader public and could be used for other purposes such as continuing professional training.

The course needs to be sustainable in the long-term, and strategies to reduce costs must be considered. A way to reduce human resource costs would be to ask pediatric residents or medical students to become trainers. These trainers should be regularly assessed to maintain the quality of the course. A way to reduce equipment costs would be to use only low-fidelity mannequins. Equipment for high-fidelity simulation is much more expensive but usually leads to better results [4]. Some authors think that the important difference in cost between a high-fidelity and a low or medium-fidelity mannequin is not justified by the little difference in knowledge and skills acquisition observed [13, 17]. In our opinion, we did not maximize the efficiency of simulation teaching. Simulators remain tools, and it is up to the teaching staff to make best use of them. Several approaches are being considered by our faculty to improve the teaching of CPA, including changes in messages delivered, rapid cycle deliberate practice, and individual teaching.

Limitations

Our studies have several limitations. In the first study, we used the same MCQ test to assess knowledge before the teaching intervention, immediately after, and 6 months to 1 year later. This may have, itself, resulted in a learning effect [24]. Therefore, we could not conclude to what extent the progression observed on MCQ scores was due to the video, the hands-on sessions, or the simple fact of testing. However, the completion of this MCQ by students was not limited to this research, but integrated into the innovative course itself. Thus, we could conclude that this innovative course, which included video, hands-on sessions, and tests, led to knowledge improvements. Another limitation is that student knowledge tests were anonymous because our aim was to evaluate the teaching and not individual students. Thus, it was not possible to match the pre- and post-intervention scores and to perform paired tests; such analyses would have been more appropriate because the data were not independent.

In the second study (skill assessment), more medical students from the “complete” group than in the other groups were excluded. This may have induced selection bias, with more high performers in the “complete” group. However, non-participation was primarily the result of difficulties associated with attending the video course or the evaluation and should not have seriously affected our conclusions. Second, we were not able to assess chest compression depth on video recordings, and Vadeboncoeur and colleagues recently showed that this is associated with survival among adults suffering out-of-hospital cardiac arrest [27]. Future studies should enhance skill assessment by using a pediatric mannequin which records CPR quality (depth and rate of compressions and number of ventilations) such as the Resusci Baby QCPR (Laerdal). An alternative method would be to use a CPR-quality device on the chest of other mannequins [3]. Third, all students were assessed during the same week but not on the same day, and some may have communicated the contents of the evaluation to their colleagues despite our instructions.

Finally, this was a single center study and it is unclear to what extent the results can be generalized to other universities.

Conclusion

Our study shows that a course combining video and simulation with high-fidelity mannequins to teach pediatric CPA to medical students is highly appreciated and allows better knowledge retention than a traditional lecture. In terms of skills, students are more compliant to resuscitation guidelines after the complete course than if there is no intervention; however, the benefit to the patient is doubtful because the no-intervention group performs chest compressions earlier and gets similar “no-flow” and “no-blow” fractions. These results are disappointing because the costs (both time and money) of the innovative course are much higher. Medical faculties should be aware of these problems and should (1) adapt pediatric advanced life support courses for medical students, (2) include “no-flow” and “no-blow” fraction when assessing students because simply assessing adherence to guidelines seems insufficient to appreciate the potential benefit for the patient, and (3) find new ways of teaching the management of pediatric CPA to maximize the efficacy and cost-effectiveness of simulators.