Introduction

Pediatric cardiology had its origins as a distinct field in the 1940s [1]. Despite almost 60 years of experience in training cardiology fellows, there is a paucity of literature with regard to the training of pediatric cardiology fellows [1]. Echocardiography has become the gold standard for non-invasive assessment of congenital and acquired heart disease in the pediatric population [1,2,3]. Within echocardiography, there are two core skill sets which must be mastered: procedural technique and interpretive skill.

Procedural technique is the “act” of performing an echocardiogram and obtaining all of the relevant imaging views. Interpretive skill involves having an understanding of all of the relevant anatomy and physiology obtained from each image and cognitively applying the information to formulate a medically sound diagnosis. Competence in both procedural and interpretive echocardiography is essential to the practice of pediatric cardiology [3]. Echocardiography is an operator-dependent imaging technique and requires a high level of competence in these technical and interpretive skills to maximize diagnostic accuracy [4]. Delayed or inaccurate diagnoses can place children with congenital heart disease at risk for adverse outcomes [5].

On average, cardiology fellows experience 4–6 months of dedicated echocardiography training during a standard 3-year fellowship [3]. Much of this time is spent learning the procedural side of echocardiography. Often, cardiology fellows obtain much of their exposure to interpretive echocardiography during on-call duty when they perform echocardiograms required on nights and weekends. However, changes in training schedules and work-hour restrictions have introduced new challenges into the training of fellows in echocardiography [1].

The current assessment of competency relies heavily on subjective assessments typically performed by faculty overseeing the fellows’ clinical performance. A more objective evaluation would be very beneficial to the fellows and allow faculty to provide a comprehensive, data-derived assessment of performance [6, 7]. Objective testing is advocated by specialty boarding agencies [7,8,9]. There are currently no established objective tools for evaluating trainees’ progress and performance in PE [9,10,11]. More objective measures would enhance: (a) fellows’ self-assessment in mastering essential knowledge and skills in echocardiography; and (b) faculty’s capability of assessing learner performance and the overall impact of the pediatric cardiology curriculum [8]. Among the few pediatric cardiology training programs that have implemented objective online echocardiography testing, improvement in trainee performance on the American Board of Pediatrics examination has been documented [8].

Recently, the TCH Pediatric Cardiology Fellowship Program introduced a pediatric echocardiography “Boot Camp” in an effort to reduce training inconsistencies. This training is designed for incoming first-year pediatric cardiology fellows and was expanded to critical care fellows. It provides expanded exposure to the procedural side of pediatric echocardiography; results to date indicate improved competency in procedural echocardiography [1]. While boot camp places an emphasis on procedural skills, a solid foundation in interpretive echocardiography is of equal importance.

A growing body of literature suggests that the traditional model of exclusively lecture-based education is less effective and less appealing to millennial learners with many learners seeing e-learning and simulation as an important compliment. Many favor a blended learning strategy [10,11,12,13]. E-learning has the advantage of offering learners the opportunity to have control over content, learning sequence, pace, time, and often media, allowing the learner to tailor the educational experience to meet their personal learning objectives [12]. It offers advantages over face to face learning as it can be done in an environment and time interval which suits the learner [14]. It allows for long-distance, multimedia, module-based sessions which can be utilized independently by an unlimited number of users and can even reach previously underserved populations [15]. This educational method is particularly useful in cardiology where images such as echocardiograms and angiograms are widely used [14]. Studies have shown that multimedia learning improves procedural knowledge [16, 17]. Online learning modules have also been shown to be effective in improving knowledge about common cardiac conditions [14]. Most importantly, online education may establish a stimulus for translation of medical knowledge into daily clinical practice [18].

Pediatric cardiologists at TCH have developed an online curriculum designed to improve competency in interpretive echocardiography and reduce errors. It complements the previous introductory training program which was focused more on procedural echocardiography. We sought to evaluate the incremental impact of online learning in combination with lecture-based learning and hands-on teaching on the acquisition of echocardiography knowledge and interpretative skills in incoming pediatric cardiology and critical care fellows. We hypothesize that an interactive, online instructional resource with learning modules will be an effective educational tool in improving short and medium-term retention of knowledge in pediatric echocardiography. We also hypothesize that there will be an additive effect of boot camp in reinforcing these interpretive skills with the additional benefit of reinforcing procedural skills required to perform a pediatric echocardiogram.

Methods

Online Learning Modules

In February 2016, TCH launched an instructional website (www.pedecho.org), an online learning platform developed by pediatric cardiologists for learners in pediatric echocardiography.

The educational website consisted of the following:

  1. (A)

    An initial 80-question multiple choice pre-test to assess users’ baseline level of knowledge.

  2. (B)

    A series of three comprehensive interactive learning modules covering the following topics: normal cardiac anatomy (which included a basic introduction to ultrasound physics and Doppler), atrial septal defects and ventricular septal defects. The learning modules consisted of written learning content, along with numerous echocardiographic videos and still images to illustrate the relevant anatomy and Doppler patterns along with image/video captions.

  3. (C)

    A final post-test to determine what knowledge users had acquired from the learning modules

We prospectively enrolled first-year pediatric cardiology and critical care fellows (fellows having completed standard critical care training now undergoing an additional year of training in cardiac ICU) at TCH from 2016 to 2019. All fellows completed the three online learning modules. Fellows were given one month to complete the learning modules and pre-test prior to the start of fellowship training. In addition, 84 additional online learners from around the world without prior cardiology experience participated in the online learning modules and testing. These learners were categorized using a detailed demographics questionnaire and served as part of the online learning group. Exclusion criteria included any online learner or fellow with prior cardiology experience or any learner who did not complete the modules and testing.

Outcome measures included an 80-question multiple choice exam administered prior to the online modules and following completion of the online modules as well as a self-assessment questionnaire. TCH cardiology and critical care fellows also underwent a cardiology echo boot camp. These individuals took a second post-test within 24 h following the completion of boot camp. In addition, a small subset of these fellows underwent a hands-on testing session in which they were scored on 21 echocardiographic objectives.

Boot Camp

At the beginning of fellowship, TCH Cardiology and first-year cardiac ICU fellows participated in an Echo Boot Camp, a 3-day training program with hands-on workshops and didactic lectures. There were a total of 15 sessions throughout boot camp. Five of these sessions were lecture-based and 10 were hands-on echocardiography scanning sessions. Lecture-based sessions consisted of the following:

  1. (1)

    Basic anatomy: Detailed overview of normal cardiac anatomy, nomenclature with pathology correlates.

  2. (2)

    Echo physics/knobology: An introduction to ultrasound reviewing basic concepts such as frequency, wavelength, sound propagation in various mediums, principles and types of Doppler, features and settings on an echocardiography machine, and how these principles relate to image acquisition and optimization.

  3. (3)

    Multimedia echo planes: 2-h session reviewing the normal echo imaging planes using a model patient (scanned by a pediatric cardiologist). Simultaneous PowerPoint illustrations are used to depict and label relevant normal anatomy from each echo view while live images are obtained.

  4. (4)

    Ventricular function: Introduction to the assessment of ventricular systolic function using echo-based quantification techniques (M-mode with shortening fraction and ejection fraction with bullet and Simpson’s biplane method for the left ventricle and fractional area change and TAPSE for the right ventricle). Echocardiographic correlates are utilized to provide appropriate examples for assessment of normal ventricular function and ventricular dysfunction.

  5. (5)

    Fundamentals of Doppler: Introduction to the basic principles of color and spectral Doppler with an overview of Doppler patterns in the normal heart while comparing and contrasting these normal Doppler patterns with abnormal Doppler patterns that occur in common congenital heart disease lesions.

With the exception of one educational session (2-h session reviewing echo imaging planes using a live model), all educational sessions were one hour in duration. All lectures were given by cardiology faculty members or advanced imaging fellows. All hands-on sessions were proctored by cardiac sonographers, cardiology faculty or senior fellows (3rd year and advanced imaging fellows) from TCH. The hands-on sessions took place in dedicated echocardiography rooms within the cardiology clinic and cardiovascular clinic research core. Echocardiograms were performed with the use of GE Healthcare Vivid 7 and E9, Philips IE33 and Siemens ACUSON Sequoia C512 ultrasound machines. The details of the structure of boot camp is provided in Fig. 1.

Fig. 1
figure 1

Pediatric echocardiography boot camp curriculum. White boxes represent didactic and interactive lectures. Shaded boxes represent hands-on echocardiography scanning sessions

Full size image

Boot Camp Assessment

Prior to the start of boot camp, all fellows completed three online learning modules as well as pre and post-tests. Fellows were provided access to the learning modules 1 month prior to the start of boot camp. Following boot camp, all fellows (cardiology and critical care) took a second identical online post-test within 24 h of completion of boot camp. Consideration was given during the boot camp lectures to providing introductory education to basic echocardiography concepts without teaching directly to the online examination or specific test questions. The pediatric cardiology fellows (n = 24) also underwent a hands-on testing session proctored by a pediatric cardiologist using a model patient in which fellows were required to assess 21 echocardiographic objectives, graded on a seven-point scale (1 = minimum, 4 = satisfactory, 7 = proficient). Objectives included the attainment of standard echocardiographic views, assessment of valvar pathology, systolic function by various methods, assessment for arch obstruction and assessment for pericardial effusion consistent with similar assessments previously performed by Maskatia et al. at our institution [1]. Testing was performed on preselected model patients with normal cardiac anatomy and excellent acoustic windows (critical care fellows were excluded from this hands-on testing session). Cardiology fellows were assessed by a single observer with expertise in training fellows in echocardiography (C.A.A, T.R.S., or J.A.K.). The goal was to assess all fellows on all 21 objectives, but in cases of technical limitations and time constraints, some objectives may not have been completely evaluated and a score was not assigned.

Data Analysis

We compared pre and post-test scores for the online learning modules in the boot camp fellows and the online learning populations. For boot camp fellows, we compared their pre boot camp learning module test scores (pre and post module) as well as test scores following boot camp (second post-test following boot camp). For fellows undergoing hands-on testing the assigned scores were evaluated. Self-assessment scores were evaluated to assess learner confidence levels (using a 7-point Likert scale). Wilcoxon signed-rank and Friedman tests were used for analysis.

Results

A total of 124 learners participated in this study. Of this cohort, 84 were exclusively online learners. The learning cohort consisted of incoming first-year cardiology fellows (n = 53), residents (n = 31), medical students (n = 6) and individuals stating they had no prior cardiology training (n = 34). 40 fellows from this cohort (23 cardiology and 17 critical care fellows) also participated in an echo boot camp and underwent additional testing and self-assessment (Fig. 2).

Fig. 2
figure 2

Pie Chart depicting online learning cohort and boot camp cohort and respective subgroups within each cohort

Full size image

The mean pre-test score for the boot camp group was 48.2% ± 13.1% with an average post module test score of 68.5% ± 14.8%. The mean post boot camp test score was 75.7% ± 13.3%. The boot camp group demonstrated significant improvement in test scores following completion of the online modules (p < 0.01) with an additional incremental improvement following boot camp (p < 0.01) (Fig. 3). On self-assessment, boot camp fellows reported increased confidence scores in interpretive competency of normal anatomy prior to and following learning modules with an incremental improvement following boot camp. In addition, boot camp fellows reported increased confidence scores in perceived procedural competency following boot camp (Fig. 4).

Fig. 3
figure 3

Boot camp cohort exam scores. The figure demonstrates an improvement in test scores following online learning modules as well as further incremental improvement in test scores following boot camp. Standard deviation bars are included. All p values were ≤ 0.01

Full size image
Fig. 4
figure 4

Boot camp cohort confidence scores. Panel a demonstrates confidence scores in interpreting a normal echocardiogram prior to, following the online learning modules and following echo boot camp. The subsequent panel b depicts confidence scores in performing a normal echocardiogram prior to and following boot camp. Confidence scores were expressed using a 7-point Likert scale. All p values for both figures were ≤ 0.01

Full size image

Online learners (n = 124) had a mean pre-test score of 49.1% ± 15.3%. Interval improvement following completion of the learning modules was noted in the online learning group, with an average post-test score of 67.8% ± 17.4% (p < 0.01). All subgroups within the online learning group demonstrated a statistically significant (p < 0.01) improvement in exam scores following online learning. Confidence scores revealed a significant increase in interpretive confidence across all subgroups with an initial mean score for the entire cohort of 2.7 and a final mean score of 4.3 (Fig. 5).

Fig. 5
figure 5

Online learning cohort exam and confidence scores. Panel a demonstrates pre and post-test scores for the online learning cohort and subgroups within that group. The subsequent panel b depicts confidence scores in interpreting a normal echocardiogram prior to and following the online learning modules for the online learning cohort and subgroups within that group. Confidence scores were expressed using a 7-point Likert scale. All p values for both figures were ≤ 0.01

Full size image

A subset of cardiology fellows (n = 24) also underwent hands-on testing following completion of boot camp. Fellows were graded on a seven-point Likert scale for 21 separate objectives. In 12/21 objectives, ≥ 75% of fellows obtained a satisfactory score (defined as a score ≥ 4) or greater. The lowest objective scores were noted in the assessment of ejection fraction (Simpson’s and bullet), assessment of pulmonary veins, use of gain/time-gain compensation and use of focal zone (Table 1).

Table 1 Hands-on testing session objectives. This table outlines 21 echocardiographic objectives assessed during hands-on testing session for cardiology fellows (n = 24)
Full size table

Discussion

The mastery of echocardiography is a complex skill set which requires interpretive and procedural knowledge. It requires a detailed comprehension of normal anatomy and physiology as well as an understanding of ultrasound physics [1]. It also requires the development and reinforcement of fine motor manipulations in order to maneuver the echocardiographic probe and acquire the necessary images [1].

Our study demonstrates the benefits to online learning modules in the short and medium-term retention of echocardiographic learning content. Medical learners from around the world who had not been previously exposed to echocardiographic learning content exhibited a clear benefit from this online platform and learning materials. These differences appear to be consistent as a whole and across learner subgroups. Cardiology and critical care boot camp fellows also demonstrated the successful acquisition of echocardiographic knowledge following completion of the learning modules. In addition, echocardiography boot camp appeared to further reinforce these core educational echocardiographic concepts as evidenced by incremental increases in test scores following the second post-test. These findings are consistent with Newberry et al. and Windish et al. who both have demonstrated the benefit of a blended approach utilizing web-based resources and modules coupled with classroom learning to promote short and medium-term retention of knowledge [15, 19].

Our study re-affirms and supports existing data from Maskatia et al. on the feasibility and short-term effectiveness of echocardiography boot camp [1]. Boot camp appears to provide a good introduction reinforcing procedural echocardiographic skills as evidenced by the majority of fellows achieving satisfactory competence in the majority of echocardiographic core objectives. There were some challenges noted in more complex imaging (such as pulmonary veins), quantitative assessments (such as ejection fraction) and image optimization which require a higher degree of technical skill and repeated exposure and reinforcement for proper mastery. Our study emphasizes the importance of such high yield educational symposia to promote basic knowledge and skill acquisition in medical trainees.

Limitations

This study was focused on short and medium-term retention of an educational intervention. Fellows were given access to learning modules one month prior to the start of boot camp. Differences in when individual fellows completed the learning modules may introduce additional variability with some fellows having completed the modules weeks prior to the start of boot camp versus shortly before boot camp. In addition, the online only learning group post-test completion times varied considerably introducing additional potential bias of test scores which may have been affected by time from module completion. Consideration of additional follow up testing at longer time intervals would be useful to assess long term retention of learning content. Another limitation of this study is that it may be underpowered to detect more subtle differences as it relates to prior experience within echocardiography and cardiology. Finally, given that the same assessment was taken multiple times, it is possible that the post-test scores may have improved as a consequence of the learner using additional resources to gain insight into incorrect answers. Therefore, some degree of improvement in score may be secondary to test familiarity and outside study, although using outside resources was discouraged and the learners did not receive the answers and explanations to the test until the final post-test was completed in order to minimize these potential confounders.

Conclusion

Our innovative hybrid approach to echocardiography education appears to demonstrate significant benefit. Online tutorials in echocardiography coupled with echocardiography boot camp appear to promote additive short and medium-term retention of echocardiographic knowledge in this population of learners. Training programs should strongly consider a blended educational approach through the implementation of online learning platforms with lecture and procedural-based educational modalities like boot camps. Through this integrated learning method, learners will be better equipped to obtain a fundamental understanding of the basic principles of cardiac ultrasound and the application of these principles in clinical management of infants, children, adolescents, and adults with CHD. Self-directed learning utilizing an online education platform dedicated to pediatric echocardiography significantly promoted the acquisition of background knowledge and improved learner confidence. Online learning platforms can serve as powerful and effective tools in medical education.