FormalPara Take-home message

Computerized simulation accelerates the initial learning curve and reduces the mean required number of transthoracic examinations to reach competence in basic critical care echocardiography. These positive effects involve the practical and technical skills, but not the diagnostic and interpretation skills.

Introduction

Critical care echocardiography (CCE) has recently been recommended as the first-choice modality to assess patients sustaining acute circulatory failure [1]. Basic CCE is based on a qualitative or semi-quantitative hemodynamic assessment using transthoracic two-dimensional echocardiography (TTE), which is aimed at answering a reduced number of binary questions, while advanced CCE refers to a full mastering of echocardiography to perform a comprehensive hemodynamic assessment [2]. Competency in basic CCE must be obtained by all intensivists during their residency [3]. Current recommendations advocate the performance of approximately 30 fully supervised TTE examinations to reach competency in basic CCE after a limited training program [3]. This recommendation based on previously published studies has not yet been confirmed prospectively [4, 5].

Computerized simulation technology seems to have outpaced validation studies to demonstrate the translation of this novel training modality into better technical skills acquirement [6]. High-fidelity simulation has recently been shown to improve the learning curve for mastering advanced hemodynamic assessment using transesophageal echocardiography (TEE) [7]. Nevertheless, this study involved a substantial proportion of residents with previous knowledge in CCE. A recent prospective randomized study confirmed these results [8]. In contrast, the educational benefit of high-fidelity simulation on the learning curve of basic CCE training is yet unknown [9,10,11], and its impact on clinical practice remains to be determined [12]. Accordingly, we prospectively assessed whether adjunctive computerized simulation accelerates the training of basic CCE of noncardiologist residents novice in ultrasound when compared to a previously validated curriculum alone. The secondary objective was to confirm the number of supervised TTE examinations required to reach competency in basic CCE.

Methods

This observational, prospective, bicenter study was conducted in the intensive care units (ICU) of Brest and Limoges teaching hospitals during a 2-year period. Voluntary noncardiologist residents without previous experience in ultrasound participated in the study during their 6-month rotation. Local ethics committee waived the need for informed consent. The absence of opposition to participate in the study was obtained from each patient or his next-of-kin.

Training groups

Two groups were distinguished based on the type of training. The interventional group underwent a validated focused training program associated with adjunctive computerized simulation training using TTE performed on mannequins, whereas the control group solely underwent the same previously described curriculum [5]. All residents of Brest Teaching Hospital were assigned to the interventional group, and all residents of Limoges Teaching Hospital were assigned to the control group. After completion of the same training program as their counterparts during the first week of their rotation, residents belonging to the interventional group attended individually two 3 h-sessions of TTE computerized simulation on mannequins (Vimedix simulator, CAE Healthcare) during the second week of the first and third months of their 6-month rotation. The faculty member was present throughout the simulation training sessions to: (i) initially supervise each trainee individually for the acquisition of adequate images in the required echocardiographic views on a virtual normal heart, and once the practical skill was considered achieved, (ii) subsequently tutor each resident during their examination of abnormal cases (e.g., hypovolemia, ventricular dysfunction, acute cor pulmonale, tamponade). No off-line reviewing of previously acquired sequences was performed, and trainees had otherwise no access to the high-fidelity simulation system. The software displays a three-dimensional anatomical model of the heart together with the relative position of the transducer scan plane and the cross-sectional anatomy to provide the trainee with the virtual location of the TTE probe and its scan plane relative to the heart and the surrounding anatomical structures [10]. The system used in the present study comprised the normal cardiac anatomy module and pathology modules.

After this training period, all patients who required a hemodynamic assessment underwent successively a basic TTE examination by a trainee and by an experienced intensivist with expertise in CCE. TTE examinations were performed and interpreted independently and in random order according to the availability of operators, but within a 15-min time-frame during which therapy was not changed. After full completion of a dedicated clinical research form by each investigator, echocardiographic assessment could be completed by an advanced examination which may include the use of TEE, when necessary. Interpretations of basic TTE examinations were then compared and potential discrepancies discussed at bedside between both the trainee and his tutor. Therapeutic proposals derived from the expert’s echocardiographic examination were solely used to guide therapy.

Skills assessment and scoring system

The skills of each resident were evaluated at the end of the first (M1), third (M3) and sixth (M6) month of trainees’ rotation by the faculty member in charge of the training program, during the same week at both study sites. Individual assessment was based on the TTE examination of two patients with circulatory and/or acute respiratory failure. Several trainees could be assessed while examining the same patient since all skills evaluations were performed independently. A dedicated scoring system adapted from a previous skills assessment form designed for TEE training was completed for each studied patient [13]. A basic CCE scoring system (maximal score: 54 points) was designed to assess (i) the practical skills (ability to obtain echocardiographic views and image quality), (ii) the diagnostic skills (adequate answers to binary clinical questions), (iii) the technical skill (accuracy of two-dimensional measurements), and (iv) the interpretation skills (diagnostic accuracy resulting in adequate therapeutic proposals) (Supplementary Table 1). In each acoustic window (parasternal long- and short-axis, apical four-chamber, subcostal long axis and inferior vena cava long-axis views), image quality was rated according to the proportion of clearly delineated left ventricular (LV) endocardium, as previously defined [4]. Binary questions corresponded to the field of competence of basic CCE [1]. Two-dimensional measurements were limited to the end-diastolic LV diameter in the parasternal long-axis view, the ratio of end-diastolic LV and end-diastolic right ventricular (RV) diameters in the apical four-chamber view, and the maximal inferior vena cava (IVC) diameter obtained during the respiratory cycle in the subcostal view  [4, 5], but were required during each basic CCE examination. Measurements were partially or completely validated if the agreement between the trainee and the expert was < 20 and < 10%, respectively [7, 13]. Color Doppler mapping was systematically used to identify severe left-sided valvular regurgitations [1]. Finally, the adequacy of the identification of the leading mechanism of cardiopulmonary compromise and resulting therapeutic proposals was assessed: hypovolemia, LV systolic dysfunction and/or pulmonary venous congestion, RV dysfunction (acute cor pulmonale being its most severe presentation), severe left-sided valvular insufficiency, tamponade, and vasoplegia (diagnosis of elimination). Mean evaluation score was calculated at M1, M3 and M6, and the number of TTE examinations performed by each trainee during each time frame was recorded. Competency in basic CCE was arbitrarily defined by a score ≥ 49/54 points (i.e. more than 90% of the maximal score), as previously proposed for advanced CCE using TEE (≥ 35/40 points) [13].

Statistical analysis

Quantitative variables are presented as means ± standard deviations and qualitative variables as frequencies and percentages. Normality of distribution of the quantitative variables was assessed using the Shapiro–Wilk test. A univariate analysis was performed to compare the characteristics of patients examined during trainees’ skills assessments in the two study groups. Quantitative variables were compared using Student’s t test and qualitative variables using χ2 test, or Fisher’s exact test when necessary. Analysis of variance was used to compare skills assessment scores over time between study groups. The correlation between the global skills assessment score obtained at M1, M3 and M6 in the two training groups and the number of supervised TTE examinations performed during the same periods of time was evaluated using a Spearman’s coefficient of rank correlation. A P value < 0.05 was considered significant and statistics were performed using the Statview 5.0 and SAS 9.1.3 (SAS Institute, Cary, NC, USA) software.

Results

Twenty-four residents participated in the study (interventional group: n = 12; control group: n = 12). None of them had previous experience in ultrasound and the repartition of medical specialties of residents (anesthesiology vs. other medical specialties) was well balanced between groups (10/12 vs. 9/12: P = 1.0). During the study period, 965 TTE were globally performed by trainees, of which 156 TTE examinations were conducted in 106 patients at the time of skills assessments (mean age: 53 ± 14 years; 66 men; mean Simplified Acute Physiologic Score 2: 55 ± 19; 79% ventilated; 53% under catecholamines). Patients were mostly assessed for acute circulatory failure and their characteristics did not differ between training groups, including the number of diagnoses of RV dilatation, left-sided valvulopathy and pericardial effusion (Table 1).

Table 1 Characteristics of patients assessed using basic critical care echocardiography for cardiopulmonary compromise at the time of trainees’ skills evaluation at 1, 3 and 6 months after the initiation of the training program
Full size table

Overall, mean skills assessment score improved faster in the experimental group (Fig. 1). When compared to the control group, trainees in the experimental group obtained a significantly higher skills assessment score at M1 (41.5 ± 4.9 vs 32.3 ± 3.7: P = 0.0004) and at M3 (45.8 ± 2.8 vs 42.3 ± 3.7: P = 0.0223), but not at M6 (49.7 ± 1.2 vs 50.0 ± 2.7: P = 0.6410). Mean number of TTE studies performed per trainee was similar between groups at M1 and M3, but higher at M6 in the control group (Table 2). Compared to their counterparts, trainees of the experimental group obtained a higher mean score for the practical and technical skills at M1 (10.6 ± 1.9 vs 7.4 ± 1.4 and 2.6 ± 0.8 vs 1.9 ± 0.7: P < 0.05, respectively) and at M3 (13.3 ± 1.2 vs 10.0 ± 2.0 and 3.8 ± 1.1 vs 3.1 ± 1.6: P < 0.05, respectively), but not at M6. Interpretation skills score was higher in the experimental group only at M1 (Table 3). No trainee reached the targeted assessment score of 49/54 at M1, while residents belonging to the interventional group tended to obtain a higher evaluation score at M3 and M6 (Fig. 2). At the end of their rotation, the proportion of trainees who reached the competency score tended to be higher in the interventional group than in the control group (10/12 [83%] vs. 8/12 [67%]: P = 0.6404). Mean number of TTE studies to reach the competency score was significantly higher in the control group when compared to the interventional group (36 ± 7 vs. 30 ± 9: P = 0.0145).

Fig. 1
figure 1

Correlation between the global skills assessment scores obtained in the three successive trainees’ proficiency evaluations at the end of the first (M1), third (M3) and sixth (M6) month of residents’ rotation, with corresponding number of supervised TTE performed individually. Solid lines represent the logarithmic correlation equation for both the interventional (blue) and control (red) groups

Full size image
Table 2 Mean skills assessment score* obtained by trainees at the end of the first (M1), third (M3) and sixth month (M6) and total number of TTE examinations performed by residents during the corresponding periods of time in both the experimental and control groups
Full size table
Table 3 Evolution of the global skills assessment score and of specific scores of basic critical care echocardiography skills at the end of the first (M1), third (M3) and sixth month (M6) in both the experimental and control groups
Full size table
Fig. 2
figure 2

Percentage of trainees reaching the targeted competency assessment score during basic critical care echocardiography skills evaluations performed at the end of the first (M1), third (M3), and sixth (M6) month of the training program for both the interventional (blue) and control (red) groups

Full size image

Discussion

This prospective study shows that adjunctive training based on computerized simulation on a mannequin accelerates the initial learning curve to achieve competence in basic CCE when compared to a previously validated curriculum alone [5]. Differences in skills assessment score was no longer observed at 6 months, but the proportion of trainees who reached the targeted skills assessment score to define competency tended to be higher in the experimental group than in the control group at the end of the rotation. In addition, the mean number of TTE examinations required to reach competency in basic CCE was significantly higher in the absence of adjunctive computerized simulation.

The significantly higher skills assessment scores observed in the interventional group when compared to the control group during the first 3 months of the training period and the higher though not significant proportion of simulator-trained residents who reached the targeted competency score at the end of their rotation can be ascribed to the adjunctive computerized simulation on mannequin for several reasons. First, all voluntary residents were novice in ultrasound, had similar (noncardiologist) medical specialties and were assessed at same time points in the two study groups. Second, the characteristics of patients who were hemodynamically assessed using basic CCE at the time of trainees’ evaluations were not statistically different between study groups. Third, the number of supervised TTE examinations performed during the initial study period (first 3 months) was similar in the two training groups. Fourth, practical (ability to obtain echocardiographic views and image quality) and technical (accuracy of two-dimensional measurements) skills account for the statistically significant increase of global skills assessment score observed in the interventional group. Of note, computerized simulation sessions were organized during the first three months of residents’ rotation. Fifth, no more relevant intergroup difference of skills assessment score was observed at 6 months at the expense of a significantly higher number of TTE examinations performed by trainees in the control group. Finally, with the exception of the first month assessment, adjunctive computerized simulation failed to significantly increase both the diagnostic and interpretation skills scores. These results suggest that the same previously validated curriculum initially followed by all trainees in the two study groups yielded similar diagnostic accuracy and therapeutic proposals adequacy, providing that TTE images were obtained and of adequate quality, and that adjunctive simulation training failed to improve the diagnostic performance of basic CCE when performed by trainees.

Similarly, Prat et al. [7] have recently shown that computerized simulation accelerates the initial learning curve to reach competence in TEE used in advanced CCE [1]. As in the present study, trainees who had access to a TEE simulation system during their training program obtained higher practical and technical skills assessment scores (but neither diagnostic nor interpretation scores) during the first 3 months of the training period (but not at 6 months), using a previously validated scoring system [13]. Nevertheless, this study involved a substantial proportion of residents with previous experience in CCE, used a historical control group, and failed to ascertain that trainees’ medical background and patients’ characteristics examined for TEE skills assessment were similar, irrespective of the training group [7]. A recent prospective randomized study have demonstrated that anesthesiology residents trained with computerized simulation acquired better skills in TTE image acquisition and anatomy identification compared to traditional teaching methods, but failed to assess the potential impact of this educational benefit on the clinical practice of basic CCE in patients with cardiorespiratory compromise [14]. Accordingly, the external validity of our study cannot be properly assessed since no study previously used computerized simulation as an adjunct to conventional curriculum for basic CCE training [9]. Sekiguchi et al. [15] used simulated patients as a sole practical teaching of basic CCE, rather than as an adjunctive training tool to improve technical skills. Computerized simulation is not inferior to hands-on on normal volunteers for initial basic TTE training [16]. It can be efficiently used to assess proficiency [17]. High-resolution simulation systems are widely considered a realistic tool to help trainees in TTE image acquisition and mental reconstruction of three-dimensional anatomical structures [6, 10, 18]. In addition, computerized simulation has several advantages over conventional CCE training. It facilitates tutoring a group of trainees during the same session to develop practical and technical skills, acquisition of adequate spatial probe orientation in a favorable environment without time constraints, discussion of image interpretation when interpreting abnormal cases, and it allows preserving training time for a growing number of eligible trainees. Computerized simulation training in health professions education has marked effects on knowledge, skills, and behaviors, but moderately alters patient-related outcomes [19]. Although current recommendations advocate the incorporation of computerized simulation in the training process only for advanced CCE [20], our results suggest that it may represent a valuable adjunct to basic CCE curriculae. In France, any intensivist who desires reaching advanced CCE competency after having being trained in basic CCE during its residency, as currently recommended [3], can apply to a national CCE certification after 1 year of didactic courses and practical training in ICU patients (100 TTEs and 50 TEEs including 25 personal probe insertions). The diffusion of computerized simulation systems will presumably modify training requirements of certification for advanced CCE in the near future.

The scoring system used in the present study was adapted from that previously proposed to assess competency in advanced CCE using TEE [7], which was based on the mean skills assessment score obtained by experts (> 35/40 points) [21]. Similarly, competency in basic CCE was arbitrarily defined as a score of at least 90% of the maximal score value [7]. Using this cut-off value, competency in basic CCE was obtained after a mean number of 30 TTE in the interventional group and 36 TTE in the control group. These results confirm the mean number of 33 TTE examinations required to obtain a good-to-excellent agreement between the noncardiologist resident novice in ultrasound after a focused training program and an expert in CCE [5], and the approximately 30 fully supervised TTE studies currently recommended to reach competency in basic CCE [3, 9]. See et al. [22] showed that approximately 90% of images are of adequate quality and suitable for correct interpretation after 30 TTE examinations. The number of TTE studies to be performed to maintain competency in basic CCE remains to be determined [9, 23].

Our study has several limitations. First, residents were purposely not randomized to a training group within the two participating centers to avoid a confusion bias among trainees. Accordingly, we chose to assign all residents from a center to the same training group and to implement the same previously validated curriculum in the two study sites [5]. In addition, evaluation bias related to factors that could have influenced the learning curve of basic CCE (e.g., residents’ profiles, characteristics of examined patients at the time of skills assessments, timing of trainees’ proficiency evaluations, scoring system) has been efficiently controlled. Second, the characteristics of the entire study population who underwent a basic CCE during the study period have not been compared between centers to ascertain that recruited patients had a similar profile, since this may have influenced residents’ training at bedside. Nevertheless, the subset of patients examined at random during the predetermined periods of trainees’ proficiency assessment, which represented 16% of all basic CCE examinations performed during the study, exhibited similar characteristics. This suggests that the entire study population was presumably homogeneous. Third, no pre-test was performed in our trainees to ascertain that intergroup baseline level in CCE was similar [9]. Nevertheless, residents of the two training groups had similar medical background and absolutely no previous knowledge in ultrasound. Fourth, the sample of evaluated residents was fairly limited. This is explained by the 6-month duration of the training program and the lowering number of residents who are truly novice in ultrasound at the time of their ICU rotation. Nevertheless, trainees performed more than 950 TTE and were evaluated in examining more than 100 patients during this 2-year prospective study. Unlike other studies comparing a training program including computerized simulation to a historical control group [7], the observed intergroup difference on the learning curve of basic CCE in our trainees can be definitely ascribed to the adjunctive effect of computerized simulation. Narrowing of inter-individual differences in global skills assessment score with time (from M1 to M3) is in keeping with previous studies [7, 21], and underlines the progressive uniformization of proficiency with accumulated tutored TTE examinations. Fifth, two 3-h sessions of computerized simulation training were empirically used in the experimental group. Previous studies on TEE simulation suggested that a median 1-h simulator training was sufficient to result in improved skills acquisition [24,25,26]. Finally, the scoring system used in the present study has not been previously validated. Nevertheless, it was adapted from a scoring system validated for the assessment of critical care TEE training [7, 13].

Conclusion

The present study shows that adjunctive computerized simulation accelerates the learning curve of basic CCE in early improving both the practical and technical skills of noncardiologist residents without previous knowledge in ultrasound. When compared to a previously validated curriculum alone, high-fidelity simulation tends to increase the proportion of residents who validate proficiency after a 6-month training program and significantly reduces the number of supervised TTE studies who are required to attain competency in basic CCE.