Providers of every specialty encounter alcohol-related problems and medical students need to be prepared to identify and intervene with patients with every level of alcohol use, from at risk drinking through pathological use. Though students may have their most systematic exposure to the treatment of moderate or severe alcohol problems during a psychiatry clerkship, only 4.6% of US medical students choose psychiatry as a specialty [1]. To disseminate important skills beyond the boundaries of our specialty, a clerkship curriculum should deepen students’ understanding of screening, brief intervention, and referral for treatment interventions (SBIRT) [2]. SBIRT is an evidence-supported, public health intervention with proven efficacy in decreasing risky substance use and preventing the development of dependence [3,4,5].

This study describes an evaluation of the impact of a video-modeled SBIRT on students’ performance on a standardized patient (SP) exam. We hypothesized that exposure to a video modeling SBIRT would make students more likely to identify and accurately characterize the alcohol use disclosed by an SP compared to performance on SP exams in previous years. Improving students’ ability to recognize and intervene with alcohol use has the potential to improve patient care and decrease long-term health care costs.

A wealth of literature supports video modeling and simulation in the teaching of discreet skills [6,7,8,9]. SBIRT Oregon [10, 11] has provided brief educational videos to teach SBIRT, specifically relating to the Alcohol Use Disorders Identification Test (AUDIT) [12] that quantifies the full range of drinking behaviors. Whether observing videos might duplicate, supplement, or even replace the more labor-intensive techniques of standardized patient (SP) encounters remains an open question.

Teaching students to accurately characterize and respond to alcohol-related problems is challenging. The curriculum surrounding this study included pre-clinical education in the physical effects of alcohol, the definition of alcohol use disorders, and instruction about screening questions to be used in all medical settings—especially the CAGE mnemonic. The clerkship sought to enhance these elements by reviewing the diagnosis of alcohol-related disorders, exposing students to complicated alcohol problems, especially as they appear in inpatient psychiatric settings, and having students participate in an elaborate SP exercise on these topics, before being tested in a similar protocol. Despite this investment of time and expensive educational resources, the rate at which students taking an SP exam either did not recognize an alcohol problem, characterized it inaccurately, or did not respond appropriately has remained around 30% over the past 5 years at George Washington University. Finding effective, low-cost educational resources is important to not only students’ performance in medical school but also their future practice as physicians.

Methods

This retrospective, cohort study was reviewed and approved by the Institutional Review Board at the George Washington University. Participants were third- and fourth-year medical students on the psychiatry clerkship in academic years 2013–2014 and 2014–2015; students who chose not to participate were excluded. The 8-week clerkship included three practice SP encounters; each SP encounter included an interview followed by a lecture on the patient’s presenting complaint and treatment options. Following the lecture, the third SP encounter involved a brief additional encounter in which students were expected to discuss the patients’ psychiatric diagnosis and present them with treatment recommendations. Students who completed the psychiatry clerkship between September 2014 and June 2015 watched one of two videos illustrating SBIRT with either a male physician/female patient or a female physician/male patient [10, 11], before conducting the brief, second encounter.

The SP exam included a 25-min interview and intervention with a female SP with a medical complaint and mild major depression, exacerbated by a mild alcohol use disorder. The actors portraying the patient evaluated the students on their interview behavior, including process items (i.e., the balance of questions, smooth transitions, expressing empathy, time management) and content items (i.e., assessing mood, anxiety and worry, substance use, and suicidal ideation, expressing concern about drinking patterns, explicitly advising reducing alcohol consumption, and making appropriate treatment recommendations). The actors were trained by the simulation center to ensure consistency among ratings. Students then had 25 min to write a summary note (narrative assessment plan [NAP]) on the encounter, including presenting the patient’s history, making a diagnosis, and recommending a treatment plan. A blinded reviewer scored the SP feedback and notes for research purposes. Methods of evaluation were held constant across both study groups.

We used standard descriptive statistics to tabulate variables of interest. The primary outcome measure analyzed the impact of the video exposure on students’ total exam scores. Secondary outcome measures analyzed examined specific items: students’ accurate characterization of the alcohol problem in their write-ups; the SPs’ ratings of students’ empathy and expressed concerns; and how systematically students applied the principles of brief intervention. Unpaired Z and t tests for all relevant variables comparing the two academic years were conducted. Sub-analyses comparing the video of a female patient, male patient, and the control group (no video) were also conducted. Statistical analysis was conducted using IBM SPSS Statistics [13].

Results

One hundred ninety-four students were assessed for eligibility in this study with none excluded. Ninety six participated in the interventional group while 94 participated in the control group. There was no significant difference between the mean overall exam scores for those who saw the video (41.29, SD = 4.61) and those who did not (40.93, SD = 3.55, p = 0.539). On the exam subscales, subjects who observed the video had a mean SP rating of 18.96 (SD = 2.91) compared to 18.93 (SD = 2.50, p = 0.924) for those who did not. Those who observed the video had a mean NAP exam score of 22.33 (SD = 2.79) while those who did not had a mean NAP exam score of 22.00 (SD = 2.34, p = 0.379). The gender of the patient in the video had no effect on the students’ performance on the SP or the NAP portions of the exam (see Tables 1 and 2).

Table 1 Comparison between no video and video
Full size table
Table 2 Comparison between female, male, and no video
Full size table

Secondary analysis revealed that compared to control students, those who saw a video were significantly more likely to be rated as empathetic in expressing concern for the SP’s drinking level when compared to those who did not see the video (81.63 vs 69.79%, Z score = −1.9246, p < 0.05). Students who did not see the videos were more likely to inappropriately identify alcohol dependence, compared to the students who saw the video (7.14 vs 17.71%; Z score = 2.2346, p < 0.05). However, the groups of students did not differ in their ability to make the correct diagnosis (i.e., alcohol use disorder). One third of students did not include any diagnostic information related to alcohol use in their interviews or write-ups. Students who saw the male or female patient video showed no significant differences on many variables (see Table 2). Students who watched the female patient video were significantly more likely to recommend decreasing use to female SPs during the exam, compared to those who saw the male patient video (72.92 vs 56.00%, female vs male; Z score = −1.7471, p < 0.05).

Discussion

Just as a brief intervention for alcohol use disorders may have unexpectedly strong benefits in clinical settings, we hoped that a brief educational intervention might have unexpected benefits. Unfortunately, the intervention did not achieve the primary objective of increasing students’ ability to recognize and accurately characterize an alcohol use disorder in primary care. Had the assessment of the intervention been tied to a specific objective at the outset, as current curriculum designs may require, the effort would have been deemed a failure.

However, our objectives were diffuse and partly determined after the fact. As a result, we were able to recognize the serendipitous finding that observing a video could be a valuable modeling experience for students, with an unexpectedly lasting effect on their ability to communicate empathetically. Outside of students being better able to express empathy, students who saw the female video were more likely to recommend decreasing consumption. The video may have served as a primer (the actor for the SP exam was female) for the students, making them more aware of the recommended guidelines for female alcohol consumption. Further study is needed to see whether this is a spurious finding.

The implication and significance of our findings are that educational interventions may have unintended consequences and that different educational techniques may have different effects. Observation may have benefits other than those afforded by direct participation, and formal didactic instruction may have yet other impact. The limitations of this study include the use of data from only one school, with its unique resources and curriculum. Our analysis was based on students’ use of DSM-IV criteria [14] for alcohol use disorder, which changed in DSM 5 [15]. We used a retrospective design, and our positive findings are based on secondary analyses. The generalizability and likely reproducibility of our findings are therefore quite limited. Our findings say little about ultimate value and purposes of the either the videos or the AUDIT and the motivational techniques they promulgate. Future directions will include a follow-up study using only the male video to see if the findings can be replicated and to distinguish the potential for a primed finding in those students who saw the female video.

This study highlights that simple and cost-effective educational interventions can have significant findings. As the majority of students transition into residency and non-psychiatric specialties, clerkship curriculums must focus on transferable skills, such as SBIRT, and the effective teaching of these skills.