Introduction

“See-one, do-one, teach-one” is a well-known colloquial statement for resident procedural training for bedside invasive procedures [1,2,3,4]. In Internal Medicine (IM), the transition from “see one” to “do one” is well supported with multiple educational tools including simulator models, procedure videos, and supervision [3]. The transition from “do-one” to “teach-one,” however, has considerably less support [5]. Although multiple systematic approaches to teaching procedures exist [1, 4, 6,7,8,9], best practices have not yet been identified. In a needs assessment at our hospital, 60% of residents responsible for teaching and supervising procedures did not have a systematic method for teaching procedures.

Simulated teaching encounters or objective structured teaching encounters (OSTEs) are successfully used in formative and summative feedback to improve teaching skills [10]. There is only one reported use of a procedural OSTE (PrOSTE) to assess teaching of procedural skills; however, PrOSTEs have never been used as an educational tool to improve procedural teaching [11].

USGIV placement is a skill that most incoming IM interns do not possess, locally and nationally [12, 13]. Providers with USGIV proficiency place less central lines, rendering it an important skill to provide quality care [14].

Thus, we sought to bridge the gap between “do one” and “teach one” by creating and evaluating a novel, comprehensive method of procedural teaching and disseminating this method via a PrOSTE. We specifically focused on USGIVs; however, the teaching framework could be applied to any bedside procedure.

Program Description

Development of Procedural Teaching Framework

We performed a literature review of methodology and theory behind teaching procedures. We sought to develop a framework that incorporated best practices. A highly regarded method of teaching procedural skills is Peyton’s four-step method: demonstration, deconstruction, comprehension, and performance [15]. Hashimoto added the gradual decrease in supervision in stages, referred to as “tight” and “loose” coaching [6]. We added the “flipped classroom” learning theory to utilize high-quality resources online [11, 16]. After the literature review, the method was piloted with learners and expert procedural teachers with no previous exposure to this framework. Based on expert feedback, the framework was iteratively updated, resulting in a comprehensive eight-step method with theoretical and evidence-based underpinnings outlined in Table 1.

Table 1 Description and justification for 8-step method of teaching procedures
Full size table

Development of Educational Intervention (“PrOSTE Session”)

The PrOSTE was developed in simulation education format, adapted from McSparron and colleagues [11]. Participants were given a prompt to teach standardized learners how to place an USGIV. Standardized learners were portrayed by volunteer residents and faculty after undergoing 30 min of training around the scenario [21]. All were previously trained in simulation debriefing. These standardized learners committed common errors, such as misidentifying the needle tip and struggling to advance the IV catheter. The scenario lasted 15 min, with a 5-min debrief immediately after the scenario.

After the individual debrief, all participants participated in a 45-min group session where our comprehensive eight-step method to teach procedures was introduced. Past experiences and stories supervising procedures were sought as well as discussion of the proposed method. Following the session, the participants were given the opportunity to re-do the PrOSTE using the techniques and eight-step method. Training materials, PrOSTE case, and debrief guide are available from the authors.

Setting and Participants

We recruited PGY 2–3 Internal Medicine residents via a residency-wide email for the PrOSTE training. Teachers who were able to attend both a PrOSTE training session and intern orientation were assigned to teach the intervention. Teachers who expressed interest in participating, but due to clinical duties were only able to attend intern orientation, underwent individualized procedural training. The authors had no control over clinical schedules. We conducted two PrOSTE sessions with rising PGY 2–4 residents in the IM program at our hospital in June 2017.

Individualized procedural training (“control” group) included orientation to the task trainers, review of USGIV steps, and hands-on practice without any training on how to teach procedures. These teachers were given ample practice time until they felt confident in teaching USGIV placement.

An ultrasound machine (Sonosite, Fujifilm, Bothell, WA), high fidelity ultrasoundable task trainers (Blue Phantom, CAE Healthcare, Sarasota, FL), and typical IV supplies were used for the PrOSTE. Each resident who participated was given a $25 gift card.

Each year, incoming interns are oriented to various common IM procedures during orientation. This training takes place in one large, crowded conference room with different stations for each procedure, with multiple learners and teachers at each. USGIV placement was a new addition the year of our intervention, 2017. Participation in the study was voluntary for incoming interns, and those who participated received a $5 gift card. Since the conclusion of the study, USGIV is a now a standard station at orientation.

Outcome Measures

Pre-PrOSTE session surveys were administered to the resident teachers to ascertain their confidence, experience, and current method of teaching procedures; post-surveys assessed their perception of 8-step method and PrOSTE session.

Teachers who underwent the PrOSTE curriculum taught alongside control teachers. There were four teaching stations equipped with an ultrasound, a high fidelity ultrasoundable arm trainer, gel, and typical supplies for IV catheter placement.

Each incoming intern complete identical pre- and post-surveys assessing their confidence in placing USGIV’s. The post-survey also included free response questions about their teacher’s behaviors. Upon completion of the teaching session, the learner proceeded to the assessment station to place an USGIV while being rated on a checklist that was modified from a previously validated checklist by a blinded expert observer (Appendix) [22].

Statistical Analysis

Results were tabulated from both teachers and incoming interns. Two-tailed t and chi-squared tests were used to compare baseline characteristics. For our primary outcome, we used a mixed-model repeated-measures analysis with teacher training as a fixed effect and the individual teacher as a random effect. Paired pre- and post-t tests were used to compare change in self-reported confidence among learners and teachers. Learners completed post-surveys with a section for comments, “What did your teacher do well?” and “What could your teacher improve upon?” The comments were analyzed for themes by two independent graders who were blinded to teacher training group. If applicable, comments were mapped to steps from the eight-step method. A section of comments was analyzed by two separate reviewers to ensure intra-rater agreement.

The Massachusetts General Hospital Institutional Review Board deemed this project exempt. All analyses were performed using JMP© Pro 13 (SAS Institute Inc., Cary, NC).

Program Evaluation

Teacher Perspectives

Twelve residents were enrolled, seven intervention and five control teachers. The pre-survey data from the control and intervention teachers showed there were no significant differences between baseline characteristics of teachers and learners based on the intervention group. Overall, teachers reported moderate comfort teaching USGIVs.

All seven residents who underwent the PrOSTE curriculum rated the case as realistic and would recommend the session. Of the 8 steps, all participants reported incorporating at least 1 new step (mean 3.1 new steps) with “verbalize” (6/7 participants), “demonstrate” and “debrief” (4/7 participants), and “loose coaching” (3/7 participants) being the most common cited new steps. Additionally, residents planned on utilizing 7.3 of the 8 steps on average in their next procedural teaching encounter, indicating strong uptake of our novel teaching method. Furthermore, as measured on a 5-point scale, self-reported confidence in teaching USGIVs increased for all PrOSTE curriculum participants (2.3 vs 3.7, p < 0.01).

Effects on Intern Learner

Sixty-seven of 70 interns consented to participate and 58/67 (87%) completed the pre-survey and post-survey. The assessment station was completed by 53 interns (79%). A complete data set (linked pre-, post- and assessment scores) was completed by 47 (70%) incoming interns. Learners taught by an intervention teacher scored 81.0% versus 74.8% by interns taught by a control teacher with an effect size of 0.50 (medium effect, Cohen’s d), which did not reach statistical significance (SD = 12.4; p = 0.22) (Fig. 1) [23]. On a 5-point Likert scale, all learners, regardless of their teacher’s training, reported increased confidence in placing USGIVs (1.9 to 3.3, p < 0.0001).

Fig. 1
figure 1

Qualitative results of intern post-survey with positive comments on the left of the chart (depicted by a plus sign) and negative comments on the right (depicted by a triangle). Bars depict percentage of respondents in each group that mentioned each theme. “Nothing” was only tabulated if the word was written explicitly

Full size image

Qualitative Data

On the post-orientation survey, 90% of interns entered at least 1 free response comment for both areas of strength and areas for improvement.

Thematic analysis translated into bar graph form is depicted in Fig. 1. Three themes outside of the 8-step method emerged: (1) a teacher clearly defining the steps of the procedure; (2) personality traits of the teacher such as “nice” and “welcoming”; and (3) time management. Within the 8-step framework, “Verbalize” and “coaching” were noted as positives more frequently for intervention teachers. Personality traits were mentioned as positives more frequently for control teachers. “Time” was more frequently mentioned on the negative feedback section for intervention teachers. Notably, “Steps” was noted as a positive for intervention teachers, but an area for improvement for control teachers (Fig. 1). This data was converted into a word clouds to better visualize the raw data, where larger font size correlates to higher frequency of usage of the phrase (Fig. 2).

Fig. 2
figure 2

Qualitative results from learner survey. Positive responses are depicted by the plus sign and negative responses are depicted by triangle. Size of font in word clouds represents relative frequency at which words were used in the learner survey. “Nothing” was only tabulated if the word was written explicitly

Full size image

Discussion

We developed and evaluated a comprehensive eight-step procedural teaching method that was successfully disseminated in a 90-min PrOSTE session. The PrOSTE session and 8-step method were highly rated, and all PrOSTE teachers planned to incorporate new teaching techniques, many of whom previously did not have a dedicated technique for teaching procedures.

These teachers identified “verbalize,” and “coaching” as effective teaching techniques, and those same themes distinguished trained versus control teachers on learner evaluation. This concordance supports that the 8-step method was beneficial from both the teacher and learner perspective, and that the PrOSTE delivery method was able to engrain these behaviors. Our results align well with prior work in educational theory as having residents participate in the PrOSTE prior to introducing the systematic eight-step method primed them to take in new knowledge [24].

Learners of intervention teachers mentioned “steps” as a positive quality more frequently than learners of control teachers. This is concordant with prior research showing that a teacher focusing on the steps of a procedure predicts learner performance [11]. Our eight-step method facilitates teaching complex steps of a procedure by having the learner either recite or perform the steps of a procedure three independent times. We believe this repetition is why learners felt they retained the steps of the procedure.

However, a few limitations to our study impeded the ability to demonstrate a statistical improvement downstream in learners. Twelve was the maximum number of resident teachers due to space limitations during orientation which reduced our statistical power. Orientation teaching took place in a crowded room with time pressure, and often to more than one learner at a time, both of which likely decreased the training’s impact. We did not assess intra-rater reliability on our checklist, which also may have skewed the results. Further studies are needed to see if the curriculum is more effective in a controlled teaching setting.

After completion of this study, a procedural service was set up at our institution. Given our collective findings, faculty who staff this service are trained in the 8-step method. Residents rotate on the service and this method is disseminated in a PrOSTE-like method (there is less standardization in the current system). This has allowed the opportunity to learn and teach procedures, and the tenets of our method are still used today. Given this new service, we were unable to repeat this study in subsequent years to increase our statistical power.

Despite this significant limitation, we believe our rigorous development of the 8-step method, the positive teacher PrOSTE surveys, the differences in teacher behavior, and modest effect size of downstream learning make it more likely than not that this method is an effective means to bridge the gap.

We chose USGIVs as this is an increasingly common procedure and a need within the residency program; however, the theoretical framework of our model is applicable across all IM procedures that can be simulated effectively using a task trainer. Thus, the PrOSTE as an educational tool has broad generalizability and can be used at any IM residency to train supervising residents how to teach procedures more effectively, with the goal of improving patient safety downstream.

Conclusion

“See one, do one, teach one” is a common framework for the progression of procedural skills in IM residency, and our intervention was aimed at addressing the often-overlooked gap between the last two. Our novel eight-step procedural teaching method disseminated in conjunction with a PrOSTE session is a well-received intervention to train procedural teachers. While learners who were taught by teachers who underwent this training did not perform statistically significantly better, we strongly believe that this effect was underestimated by the learner setting. As a pilot study, we demonstrated that the PrOSTE is a feasible, well-received intervention for training supervisors on procedural teaching, with broad generalizability to any IM residency and any procedure that can be simulated. Further studies will be needed to assess its impact in a more controlled teaching environment, as well as its downstream impact.