Introduction

Echocardiography training is a major part of the training in cardiac imaging stated in the American Board of Pediatrics (ABP) approved entrustable professional activities (EPA). The expectation is that a trainee will be able to “acquire the imaging skills required for all aspects of pediatric and cardiology care” [1]. The graduating trainee is expected to master these skills and the training program is expected to have an education program to achieve this. The primary component of the imaging EPA is mastery of knowledge and skill in transthoracic echocardiography (TTE) with a secondary competence (i.e. exposure) in transesophageal echocardiography (TEE), fetal echocardiography, and advanced non-invasive imaging (CT, MRI, and nuclear imaging) [2]. Although TEE training to the level of independent practice is not a requirement, sufficient knowledge and skill are needed to achieve the necessary exposure and to determine if this is a career interest or need. Therefore, acquisition of TEE knowledge and skill is beneficial as part of the 3-year categorical fellowship. This level of mastery is likely to be institution dependent.

Simulation is a method of teaching and evaluation that allows a trainee to achieve desired skills in a controlled environment which does not expose a patient to harm and allows a comfort level on the part of the trainee [3, 4]. It can also be aligned with a standardized evaluation tool and allows repeated attempts in rapid succession, and thus the application of rapid cycle deliberate practice which may not be applicable in a live patient [5].

The mastery learning approach deviates from the traditional apprenticeship model of medical training where competency is assumed after a trainee “saw one and performed one” during a medical rotation or having performed a predetermined number of procedures or time of exposure [6,7,8,9]. In traditional education, an arbitrary passing score resulted in a certain number of individuals failing, whereas the expectation in mastery learning is of high levels of excellence to be achieved in all learners with little variation in this outcome. In this approach, there are clear, iterative learning objectives, goal directed educational activities, formative testing, and feedback strategies with a minimum (but high) passing standard (MPS) [10, 11] chosen to reflect a desired clinical competency. Reaching the MPS may require repeated attempts. This mastery learning concept can be applied to knowledge skills, conceptual skills (e.g., patient interactions), or technical skills [12, 13].

A bootcamp approach has been used for training in transthoracic echocardiography (TTE) using multi- modality education (didactic lectures, demonstration, and simulation) [14,15,16,17,18]. A similar approach has been used in a TEE bootcamp including simulation [19]. However, there has not been a rigorous simulation-based mastery learning (SBML) approach applied to TEE training described in the literature.

Aims

Our primary aim was for trainees to acquire the TEE skills needed for continued learning and training during pediatric cardiology fellowship. The secondary aim was to demonstrate the use of SBML to help trainees achieve mastery of TEE during a bootcamp experience. Both aims are divided into TEE knowledge and skill achievement. The latter includes skills of basic TEE probe handling and the ability to acquire basic TEE imaging views.

Methods

Learners were pediatric cardiology fellows (categorical fellows) and an anesthesiologist who were invited to participate in a TEE training bootcamp (including fellows who may have participated in a prior TEE bootcamp). The components of this SBML program are shown in Fig. 1.

Fig. 1
figure 1

Flow chart of the education strategy and components of the training curriculum

Full size image

All participants were given an online knowledge pretest as part of the education intervention. The knowledge pretest was based on training guidelines from the imaging EPA as well as published guidelines in the training and performance of a pediatric TEE [1, 2]. A test blueprint was created (supplement 1 [20]), and the questions were formatted using National Board of Medical Examiners (NBME) guidelines for creating multiple-choice questions [21]. The questions evaluated the following: understanding of the indications, strengths, and limitations of TEE in acquired and congenital heart disease; knowledge of TEE application in the assessment of vegetations, intracardiac abscesses and intracardiac thrombi; knowledge about using TEE to guide interventional procedures, i.e., pre- intra and post-operative assessment.

The education intervention involved reading and viewing recorded didactic lectures describing TEE basics, TEE views, a protocol for obtaining TEE, and how to hold and manipulate a TEE probe. The short-term goal was to have trainees demonstrate that they “know” (i.e., have knowledge) and “know how” (i.e., have knowledge of a skill), consistent with Miller’s pyramid [22].

After taking the online pretest and receiving the education materials, trainees repeated the test as a post-test. The same multiple-choice questions were used but randomized in order. Trainees had access to all education materials and were allowed to repeatedly retake the test until a score of > 90% was achieved. This minimum passing standard (MPS) was set by faculty educators as a qualification for the skills session. The high MPS supports a rigorous level of readiness. The education strategies are shown in Fig. 1. The education skills (“shows how” per Miller’s pyramid) [22]) session was divided into two skills: TEE probe handling and TEE image acquisition. It consisted of a short didactic review that included a brief visualization of tomographic views achieved by TEE via a tomographic plane visualization tool. This was followed by a live demonstration of how to hold and handle the probe. After instruction, the trainee was asked to demonstrate skill items on a mastery learning checklist related to probe handling and manipulation (supplement 2). For every item on the checklist, the faculty rater evaluated if the trainee either performed the required task correctly (pass), incorrectly or was unable to perform the required item. Using a method of rapid cycle deliberate practice, the trainee was immediately corrected if an item on the checklist was not completed and continued to move to the next item [5, 23]. The attempt was deemed complete and successful (the minimum passing standard/MPS) once all items on the checklist were correctly completed without the need for any corrective feedback. Otherwise, the trainee repeated the checklist from the beginning. Faculty graders had a prior meeting to decide on a standard to which pass, or fail was given to each skill. The environment for this endeavor was a “fail-forward” approach [24].

SBML for image acquisition was similar to probe handling. Faculty met before the session to set a MPS. For this skill, the trainee was expected to understand and obtain basic 2-D TEE views (image acquisition supplements 3–5). Training and assessment occurred in a psychologically safe environment with a senior trainee or faculty member performing first. This allowed junior trainees to observe what was required. The attempt was complete and successful once all items on the checklist were completed correctly. Image acquisition checklists were divided into 3 levels of increasing complexity. Level 1 was the easiest and level 3 the most difficult. Trainees had the option of not continuing after level 1 was achieved.

The initial time allotment for the skills session was one hour for probe manipulation and 2 h for image acquisition. Trainees of varying skills were present in each of 2 groups and order of participation was based on seniority with the intent that junior trainees could learn through observation of others and hearing the verbal feedback.

Results

There were 8 TEE bootcamp participants: 3 first year pediatric cardiology fellows, 3 second year pediatric cardiology fellows, 1 third year pediatric cardiology fellow and 1 cardiac anesthesia attending physician.

The bootcamp was conducted in two separate sessions with 5 trainees in the first session and 3 in the second session. Of the 5 in the first session, there was one first year cardiology fellow, two 2nd year cardiology fellows, one 3rd year cardiology fellow and an anesthesia attending. Of the 3 participants in the second session there were two 1st year cardiology fellows and one 2nd year cardiology fellow.

The knowledge test scores are provided in Fig. 2. All participants achieved the MPS of 90% or higher on the final attempt. 7 participants needed 2 attempts, and 1, a first-year fellow, required 3 attempts.

Fig. 2
figure 2

Multiple choice examination scores (5) at baseline (pretest) and after repeated attempts in trainees. CF-1 = 1st year fellow, CF-2 = 2nd year fellow, CF-3 = 3rd year fellow, and attending = anesthesiology attending

Full size image

Mastery of the knowledge test resulted in a high level of competence on the initial attempt for the probe handling and manipulation skill with all participants completing > 80% of the checklist correctly regardless of training level (Fig. 3). All participants achieved mastery with 100% of the checklist being performed correctly on 2 or fewer attempts.

Fig. 3
figure 3

Pre- and post-test probe handling and manipulation checklist scores (% correct). The first served as the pre-test score. CF-1 = 1st year fellow, CF-2 = 2nd year fellow, CF-3 = 3rd year fellow, and attending = anesthesiology attending

Full size image

Participants’ scores for initial attempt and final attempt in image acquisition for each of the 3 levels of mastery (Level 1–3) are shown in Fig. 4. All participants achieved mastery for levels 1 and 2 with a maximum of 3 attempts. Four participants attempted skill level 3, with all 4 achieving mastery within 2 attempts. Four participants did not attempt skill level 3 due to a lack of time. In the first group, the most junior fellow was the last to perform the skills and mastered all 3 levels.

Fig. 4
figure 4

Pre- and post-test image acquisition checklist scores (% correct). The first attempt served as the pre-test score. CF-1 = 1st year fellow, CF-2 = 2nd year fellow, CF-3 = 3rd year fellow, and attending = anesthesiology attending. **Note that there were 4 incomplete scores in the 3rd attempt in trainees who neither succeeded nor failed due to the lack of time to engage in the skill

Full size image

Discussion

Our study demonstrates that even novice trainees can be effectively prepared to perform a pediatric TEE correctly including probe manipulation and image acquisition using an SBML curriculum. Improvement in knowledge base in TEE and achievement of a pre-defined mastery level in probe manipulation and image acquisition using SBML provided readiness needed for trainees to perform this invasive diagnostic test for a patient.

Traditional learning in pediatric cardiology involves didactic lectures and skill practice opportunities (time allotted to rotations, reaching target numbers). Competency in knowledge is achieved after passing a standardized test, or evaluation by faculty educators using a formal oral examination or repeated informal “Socratic- like” knowledge evaluation. Technical skills and applied knowledge have typically been assessed using subjective ratings during clinical rotations or specific encounters, e.g., cardiac catheterization, performance of transthoracic or transesophageal echocardiography. In a summative fashion after completion of cardiology fellowship, faculty members subjectively evaluate the trainees as competent: allowing the trainee to be eligible for a standardized (board) examination. The trainee is board certified for practice in pediatric cardiology after passing the written examination. The examination MPS is norm-referenced, typically using a bell-shaped curve to inform this decision.

Pediatric cardiology training has evolved over the past decades. Since 2005, there has been a realization that pediatric cardiology trainees cannot be expected to be competent in all areas of cardiology and thus more specific guidelines were developed [25]. In 2010, there was a shift in pediatric subspecialty training to be competency-based, with a focus on proven mastery of knowledge and skill rather than simple exposure [26, 27]. The latter was concurrent with changes in pediatric cardiology fellowship training programs to incorporate this newer paradigm in 2015 which shifted from equating exposure and target numbers with competency [28]. Although competency remained a goal of training, pediatric cardiology training guidelines were again refined with standardized core competencies and EPAs (entrusted professional activities). Trainees are evaluated on core competencies throughout the training period, ultimately moving from the level of novice or observer to independent performer of a skill or provider of care [29]. EPAs and core competency domains, with their associated milestones, represent broad training goals with more specific anchors for formative feedback. Competency-based medical education, which includes mastery learning, has developed over the last two decades to address the newer emphasis on evaluating and ensuring skills and the application of knowledge [27, 30].

Our study demonstrates that knowledge and skills in TEE can be achieved to a high competency level using a SBML TEE bootcamp. SBML allows for future supervised performance of TEE in live patients as well as incorporation of basic TEE knowledge needed for pediatric cardiology care. This level of competency is also a condition for continued specialty training in cardiac imaging to achieve independent performance and interpretation of TEE. This appropriate level of mastery for pediatric cardiology trainees and cardiac anesthesia providers was achieved among participants with varied levels of training and in a short timeframe.

This study also demonstrates that before the education curriculum and training, trainees at all levels had varied scores relative to the minimum passing standard (MPS). After the education curriculum, the MPS was achieved by all trainees regardless of their training level. The level achieved was higher than the entry level. We speculate that fellows may have a higher entry score if they participated in the bootcamp again.

Limitations of the study include the small numbers of trainees which was further confounded by dividing the trainees into two sessions due to trainee schedule constraints. Additional limitations include the lack of measuring the time duration of attempts at achieving the skill, and the unequal distribution of participants at different levels of training in the two sessions. There appeared to be a subjective benefit in performing later in the queue. Another limitation is the time allotment as it was likely that the 4 trainees who did not attempt level 3 would have done so with adequate time. Furthermore, had there been enough time for deliberate practice so that all trainees achieved the expected competencies of all the levels, the need to assign levels would be obviated. We estimate that an additional hour would be sufficient. Another limitation is not having a formal pretest knowledge and skill assessment before the education endeavor. In this case, the initial attempt at the knowledge test served as the pre-test score. The initial attempts at the skill sessions served as the skills pretest which is a key feature of the mastery education intervention. It is not clear that having a separate dual knowledge and skill pre- assessment would have altered the results of this study.

This study did not formally address fellow skill level at the time of performing a TEE on a live patient. However, faculty observed that the fellows were better prepared than in prior years.

Conclusions

A TEE bootcamp using SBML is a powerful medical education strategy. SBML is a rigorous approach that can be used to achieve high and uniform TEE learning outcomes among pediatric cardiology fellows. This study sets the stage for regional duplication. A larger trial involving other institutions is warranted.