Abstract
Problem
Minimal formal training exists in teaching invasive bedside procedures during Internal Medicine (IM) residency despite the large role trainees have in instructing junior colleagues.
Objective and Methods
We investigated if using a Procedural Objective Structured Teaching Encounter (PrOSTE) to disseminate a novel method for teaching procedures would improve supervising residents’ (n = 7) ability to teach ultrasound-guided peripheral IV’s (USGIV) to incoming interns (n = 67) at a single, large academic IM residency. Supervising residents were assigned to receive the PrOSTE training versus standard procedure training, and then, both groups instructed incoming interns. The impact of the PrOSTE was measured by participant surveys, observed changes in teacher behavior, and performance of incoming interns on a USGIV blinded assessment station.
Measurement and Main Results
PrOSTE-trained residents reported high levels of satisfaction with the session and demonstrated increased desirable behaviors when teaching procedures. There was no statistical difference in incoming intern performance when placing USGIVs between intervention and standard groups (81.0% vs 74.8% items correct; difference 6.2; SD = 12.4; p = 0.22).
Conclusion
The PrOSTE is a feasible, well-received tool for training supervising residents in our novel teaching framework, as demonstrated in this pilot study. Despite not showing a difference in learner performance, qualitative data suggests the impact of the PrOSTE would be even greater in a more controlled teaching environment. Using a PrOSTE to deliver this teaching framework has broad applicability to any IM residency, and the tenets can be used with any bedside invasive procedure with an effective task trainer.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
“See-one, do-one, teach-one” is a well-known colloquial statement for resident procedural training for bedside invasive procedures [1,2,3,4]. In Internal Medicine (IM), the transition from “see one” to “do one” is well supported with multiple educational tools including simulator models, procedure videos, and supervision [3]. The transition from “do-one” to “teach-one,” however, has considerably less support [5]. Although multiple systematic approaches to teaching procedures exist [1, 4, 6,7,8,9], best practices have not yet been identified. In a needs assessment at our hospital, 60% of residents responsible for teaching and supervising procedures did not have a systematic method for teaching procedures.
Simulated teaching encounters or objective structured teaching encounters (OSTEs) are successfully used in formative and summative feedback to improve teaching skills [10]. There is only one reported use of a procedural OSTE (PrOSTE) to assess teaching of procedural skills; however, PrOSTEs have never been used as an educational tool to improve procedural teaching [11].
USGIV placement is a skill that most incoming IM interns do not possess, locally and nationally [12, 13]. Providers with USGIV proficiency place less central lines, rendering it an important skill to provide quality care [14].
Thus, we sought to bridge the gap between “do one” and “teach one” by creating and evaluating a novel, comprehensive method of procedural teaching and disseminating this method via a PrOSTE. We specifically focused on USGIVs; however, the teaching framework could be applied to any bedside procedure.
Program Description
Development of Procedural Teaching Framework
We performed a literature review of methodology and theory behind teaching procedures. We sought to develop a framework that incorporated best practices. A highly regarded method of teaching procedural skills is Peyton’s four-step method: demonstration, deconstruction, comprehension, and performance [15]. Hashimoto added the gradual decrease in supervision in stages, referred to as “tight” and “loose” coaching [6]. We added the “flipped classroom” learning theory to utilize high-quality resources online [11, 16]. After the literature review, the method was piloted with learners and expert procedural teachers with no previous exposure to this framework. Based on expert feedback, the framework was iteratively updated, resulting in a comprehensive eight-step method with theoretical and evidence-based underpinnings outlined in Table 1.
Development of Educational Intervention (“PrOSTE Session”)
The PrOSTE was developed in simulation education format, adapted from McSparron and colleagues [11]. Participants were given a prompt to teach standardized learners how to place an USGIV. Standardized learners were portrayed by volunteer residents and faculty after undergoing 30 min of training around the scenario [21]. All were previously trained in simulation debriefing. These standardized learners committed common errors, such as misidentifying the needle tip and struggling to advance the IV catheter. The scenario lasted 15 min, with a 5-min debrief immediately after the scenario.
After the individual debrief, all participants participated in a 45-min group session where our comprehensive eight-step method to teach procedures was introduced. Past experiences and stories supervising procedures were sought as well as discussion of the proposed method. Following the session, the participants were given the opportunity to re-do the PrOSTE using the techniques and eight-step method. Training materials, PrOSTE case, and debrief guide are available from the authors.
Setting and Participants
We recruited PGY 2–3 Internal Medicine residents via a residency-wide email for the PrOSTE training. Teachers who were able to attend both a PrOSTE training session and intern orientation were assigned to teach the intervention. Teachers who expressed interest in participating, but due to clinical duties were only able to attend intern orientation, underwent individualized procedural training. The authors had no control over clinical schedules. We conducted two PrOSTE sessions with rising PGY 2–4 residents in the IM program at our hospital in June 2017.
Individualized procedural training (“control” group) included orientation to the task trainers, review of USGIV steps, and hands-on practice without any training on how to teach procedures. These teachers were given ample practice time until they felt confident in teaching USGIV placement.
An ultrasound machine (Sonosite, Fujifilm, Bothell, WA), high fidelity ultrasoundable task trainers (Blue Phantom, CAE Healthcare, Sarasota, FL), and typical IV supplies were used for the PrOSTE. Each resident who participated was given a $25 gift card.
Each year, incoming interns are oriented to various common IM procedures during orientation. This training takes place in one large, crowded conference room with different stations for each procedure, with multiple learners and teachers at each. USGIV placement was a new addition the year of our intervention, 2017. Participation in the study was voluntary for incoming interns, and those who participated received a $5 gift card. Since the conclusion of the study, USGIV is a now a standard station at orientation.
Outcome Measures
Pre-PrOSTE session surveys were administered to the resident teachers to ascertain their confidence, experience, and current method of teaching procedures; post-surveys assessed their perception of 8-step method and PrOSTE session.
Teachers who underwent the PrOSTE curriculum taught alongside control teachers. There were four teaching stations equipped with an ultrasound, a high fidelity ultrasoundable arm trainer, gel, and typical supplies for IV catheter placement.
Each incoming intern complete identical pre- and post-surveys assessing their confidence in placing USGIV’s. The post-survey also included free response questions about their teacher’s behaviors. Upon completion of the teaching session, the learner proceeded to the assessment station to place an USGIV while being rated on a checklist that was modified from a previously validated checklist by a blinded expert observer (Appendix) [22].
Statistical Analysis
Results were tabulated from both teachers and incoming interns. Two-tailed t and chi-squared tests were used to compare baseline characteristics. For our primary outcome, we used a mixed-model repeated-measures analysis with teacher training as a fixed effect and the individual teacher as a random effect. Paired pre- and post-t tests were used to compare change in self-reported confidence among learners and teachers. Learners completed post-surveys with a section for comments, “What did your teacher do well?” and “What could your teacher improve upon?” The comments were analyzed for themes by two independent graders who were blinded to teacher training group. If applicable, comments were mapped to steps from the eight-step method. A section of comments was analyzed by two separate reviewers to ensure intra-rater agreement.
The Massachusetts General Hospital Institutional Review Board deemed this project exempt. All analyses were performed using JMP© Pro 13 (SAS Institute Inc., Cary, NC).
Program Evaluation
Teacher Perspectives
Twelve residents were enrolled, seven intervention and five control teachers. The pre-survey data from the control and intervention teachers showed there were no significant differences between baseline characteristics of teachers and learners based on the intervention group. Overall, teachers reported moderate comfort teaching USGIVs.
All seven residents who underwent the PrOSTE curriculum rated the case as realistic and would recommend the session. Of the 8 steps, all participants reported incorporating at least 1 new step (mean 3.1 new steps) with “verbalize” (6/7 participants), “demonstrate” and “debrief” (4/7 participants), and “loose coaching” (3/7 participants) being the most common cited new steps. Additionally, residents planned on utilizing 7.3 of the 8 steps on average in their next procedural teaching encounter, indicating strong uptake of our novel teaching method. Furthermore, as measured on a 5-point scale, self-reported confidence in teaching USGIVs increased for all PrOSTE curriculum participants (2.3 vs 3.7, p < 0.01).
Effects on Intern Learner
Sixty-seven of 70 interns consented to participate and 58/67 (87%) completed the pre-survey and post-survey. The assessment station was completed by 53 interns (79%). A complete data set (linked pre-, post- and assessment scores) was completed by 47 (70%) incoming interns. Learners taught by an intervention teacher scored 81.0% versus 74.8% by interns taught by a control teacher with an effect size of 0.50 (medium effect, Cohen’s d), which did not reach statistical significance (SD = 12.4; p = 0.22) (Fig. 1) [23]. On a 5-point Likert scale, all learners, regardless of their teacher’s training, reported increased confidence in placing USGIVs (1.9 to 3.3, p < 0.0001).
Qualitative Data
On the post-orientation survey, 90% of interns entered at least 1 free response comment for both areas of strength and areas for improvement.
Thematic analysis translated into bar graph form is depicted in Fig. 1. Three themes outside of the 8-step method emerged: (1) a teacher clearly defining the steps of the procedure; (2) personality traits of the teacher such as “nice” and “welcoming”; and (3) time management. Within the 8-step framework, “Verbalize” and “coaching” were noted as positives more frequently for intervention teachers. Personality traits were mentioned as positives more frequently for control teachers. “Time” was more frequently mentioned on the negative feedback section for intervention teachers. Notably, “Steps” was noted as a positive for intervention teachers, but an area for improvement for control teachers (Fig. 1). This data was converted into a word clouds to better visualize the raw data, where larger font size correlates to higher frequency of usage of the phrase (Fig. 2).
Discussion
We developed and evaluated a comprehensive eight-step procedural teaching method that was successfully disseminated in a 90-min PrOSTE session. The PrOSTE session and 8-step method were highly rated, and all PrOSTE teachers planned to incorporate new teaching techniques, many of whom previously did not have a dedicated technique for teaching procedures.
These teachers identified “verbalize,” and “coaching” as effective teaching techniques, and those same themes distinguished trained versus control teachers on learner evaluation. This concordance supports that the 8-step method was beneficial from both the teacher and learner perspective, and that the PrOSTE delivery method was able to engrain these behaviors. Our results align well with prior work in educational theory as having residents participate in the PrOSTE prior to introducing the systematic eight-step method primed them to take in new knowledge [24].
Learners of intervention teachers mentioned “steps” as a positive quality more frequently than learners of control teachers. This is concordant with prior research showing that a teacher focusing on the steps of a procedure predicts learner performance [11]. Our eight-step method facilitates teaching complex steps of a procedure by having the learner either recite or perform the steps of a procedure three independent times. We believe this repetition is why learners felt they retained the steps of the procedure.
However, a few limitations to our study impeded the ability to demonstrate a statistical improvement downstream in learners. Twelve was the maximum number of resident teachers due to space limitations during orientation which reduced our statistical power. Orientation teaching took place in a crowded room with time pressure, and often to more than one learner at a time, both of which likely decreased the training’s impact. We did not assess intra-rater reliability on our checklist, which also may have skewed the results. Further studies are needed to see if the curriculum is more effective in a controlled teaching setting.
After completion of this study, a procedural service was set up at our institution. Given our collective findings, faculty who staff this service are trained in the 8-step method. Residents rotate on the service and this method is disseminated in a PrOSTE-like method (there is less standardization in the current system). This has allowed the opportunity to learn and teach procedures, and the tenets of our method are still used today. Given this new service, we were unable to repeat this study in subsequent years to increase our statistical power.
Despite this significant limitation, we believe our rigorous development of the 8-step method, the positive teacher PrOSTE surveys, the differences in teacher behavior, and modest effect size of downstream learning make it more likely than not that this method is an effective means to bridge the gap.
We chose USGIVs as this is an increasingly common procedure and a need within the residency program; however, the theoretical framework of our model is applicable across all IM procedures that can be simulated effectively using a task trainer. Thus, the PrOSTE as an educational tool has broad generalizability and can be used at any IM residency to train supervising residents how to teach procedures more effectively, with the goal of improving patient safety downstream.
Conclusion
“See one, do one, teach one” is a common framework for the progression of procedural skills in IM residency, and our intervention was aimed at addressing the often-overlooked gap between the last two. Our novel eight-step procedural teaching method disseminated in conjunction with a PrOSTE session is a well-received intervention to train procedural teachers. While learners who were taught by teachers who underwent this training did not perform statistically significantly better, we strongly believe that this effect was underestimated by the learner setting. As a pilot study, we demonstrated that the PrOSTE is a feasible, well-received intervention for training supervisors on procedural teaching, with broad generalizability to any IM residency and any procedure that can be simulated. Further studies will be needed to assess its impact in a more controlled teaching environment, as well as its downstream impact.
References
Grantcharov TP, Reznick RK. Teaching procedural skills. BMJ. 2008;336(7653):1129–31. https://doi.org/10.1136/bmj.39517.686956.47.
McLeod PJ, Steinert Y, Trudel J, Gottesman R. Seven principles for teaching procedural and technical skills. Acad Med. 2001;76(10):1080. https://doi.org/10.1097/00001888-200110000-00023.
Lenchus JD. End of the “see one, do one, teach one” era: the next generation of invasive bedside procedural instruction. J Am Osteopath Assoc. 2010;110(6):340–6.
Sawyer T, White M, Zaveri P, Chang T, Ades A, French H, et al. Learn, see, practice, prove, do, maintain: an evidence-based pedagogical framework for procedural skill training in medicine. Acad Med. 2015;90(8):1025–33. https://doi.org/10.1097/ACM.0000000000000734.
Mourad M, Kohlwes J, Maselli J, Auerbach AD. Supervising the supervisors-procedural training and supervision in internal medicine residency. J Gen Intern Med. 2010;25(4):351–6. https://doi.org/10.1007/s11606-009-1226-z.
Hashimoto DA, Phitayakorn R. Procedural training.; 2014.
Garcia-Rodriguez JA. Teaching medical procedures at your workplace. Can Fam Physician 2016;62(4):351–354, e222-5. http://www.ncbi.nlm.nih.gov/pubmed/27076547. Accessed 25 May 2018.
George JH, Doto FX. A simple five-step method for teaching clinical skills. Fam Med. 2001;33(8):577–8.
Lake FR, Hamdorf JM. Teaching on the run tips 5: teaching a skill. Med J Aust 2004;181(6):327–328. http://www.ncbi.nlm.nih.gov/pubmed/15377246. Accessed 25 May 2018.
Trowbridge RL, Snydman LK, Skolfield J, Hafler J, Bing-You RG. A systematic review of the use and effectiveness of the Objective Structured Teaching Encounter. Med Teach. 2011;33(11):893–903. https://doi.org/10.3109/0142159X.2011.577463.
McSparron JI, Ricotta DN, Moskowitz A, et al. The proste: identifying key components of effective procedural teaching. Ann Am Thorac Soc. 2015;12(2):230–4. https://doi.org/10.1513/AnnalsATS.201406-237BC.
Promes SB, Chudgar SM, Grochowski COC, et al. Gaps in procedural experience and competency in medical school graduates. Acad Emerg Med. 2009;16(SUPPL. 2). doi:https://doi.org/10.1111/j.1553-2712.2009.00600.x
Huang GC, Smith CC, Gordon CE, et al. Beyond the comfort zone: residents assess their comfort performing inpatient medical procedures. Am J Med. 2006;119(1). doi:https://doi.org/10.1016/j.amjmed.2005.08.007
Au AK, Rotte MJ, Grzybowski RJ, Ku BS, Fields JM. Decrease in central venous catheter placement due to use of ultrasound guidance for peripheral intravenous catheters. Am J Emerg Med. 2012;30(9):1950–4. https://doi.org/10.1016/j.ajem.2012.04.016.
Peyton J. The learning cycle. Teach Learn Med Pract. 1998:13–9.
Hew KF, Lo CK. Flipped classroom improves student learning in health professions education: a meta-analysis. BMC Med Educ. 2018;18(1):38. https://doi.org/10.1186/s12909-018-1144-z.
McMahon GT, Ingelfinger JR, Campion EW. Videos in clinical medicine a new journal feature. N Engl J Med. 2006;354(15):1635.
Young JQ, Van Merrienboer J, Durning S, Ten Cate O. Cognitive load theory: implications for medical education: AMEE Guide No. 86. Med Teach. 2014;36(5):371–84. https://doi.org/10.3109/0142159X.2014.889290.
Weeks D, Anderson LP. The interaction of observational learning with overt practice: effects on motor skill learning. Acta Psychol. 2000;104:259–71. https://doi.org/10.1016/S0001-6918(00)00039-1.
Anders EK. Deliberate practice and acquisition of expert performance: a general overview. Acad Emerg Med. 2008;15(11):988–94. https://doi.org/10.1111/j.1553-2712.2008.00227.x.
Rudolph JW, Simon R, Rivard P, Dufresne RL, Raemer DB. Debriefing with good judgment: combining rigorous feedback with genuine inquiry. Anesthesiol Clin. 2007;25:361–76. https://doi.org/10.1016/j.anclin.2007.03.007.
Jung CF, Breaud AH, Sheng AY, Byrne MW, Muruganandan KM, Dhanani M, et al. Delphi method validation of a procedural performance checklist for insertion of an ultrasound-guided peripheral intravenous catheter. Am J Emerg Med. 2016;34(11):2227–30. https://doi.org/10.1016/j.ajem.2016.08.006.
Cohen J. Statistical power analysis for the behavioral sciences.; 1988. doi:https://doi.org/10.1234/12345678.
Zendejas B, Cook DA, Farley DR. Teaching first or teaching last: does the timing matter in simulation-based surgical scenarios? In: Journal of Surgical Education. ; 2010. https://doi.org/10.1016/j.jsurg.2010.05.001.
Funding
Support for this study was obtained from the Center for Educational Innovation and Scholarship at Massachusetts General Hospital.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare that have no conflicts of interest.
Ethical Approval
The Massachusetts General Hospital Institutional Review Board deemed this project exempt.
Informed Consent
All participants were informed of the structure and intent of the project via email as well as in person before each session. Given all participation was voluntary, consent was implied by participating in the project.
Disclosures
The authors have no financial interests to disclose.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic Supplementary Material
ESM 1
(DOCX 15.7 KB)
Rights and permissions
About this article
Cite this article
Jhaveri, V.V., Currier, P.F. & Johnson, J.H. Bridging the Gap Between “Do One” and “Teach One”: Impact of a Procedural Objective Structured Teaching Encounter on Resident Procedural Teaching Proficiency. Med.Sci.Educ. 30, 905–910 (2020). https://doi.org/10.1007/s40670-020-00972-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40670-020-00972-7