Introduction

Telepsychiatry is increasingly meeting the mental healthcare needs of the general population (Shaver, 2022). Over the past decade, widespread adoption of smart devices and increased internet access have fueled an expansion of telepsychiatry from a niche care delivery model to a core component of mainstream healthcare. Prior to the COVID-19 pandemic, many clinics and larger healthcare organizations were exploring telepsychiatry as an emerging tool to enhance access and fill gaps in care for vulnerable or geographically isolated patient populations (Roth et al., 2019; Brunt & Gale-Grant, 2023; Tuckson et al., 2017; Fortney et al., 2021). During the COVID-19 pandemic, social distancing required a broader spectrum of patients and physicians to accept telepsychiatry as a legitimate and safe modality for healthcare delivery (Allaert et al., 2020; Hagi et al., 2023; Quinton et al., 2021; Sharma & Devan, 2023). Now, telepsychiatry is ubiquitous with some studies even demonstrating improved healthcare outcomes relative to in-person psychiatric care (Ettman et al., 2023).

To prepare physicians for the ongoing use of telepsychiatry in mainstream healthcare, training in core skills and best practices should begin in medical school. While medical students in the USA typically complete coursework and clinical training in psychiatry, there are unique skills associated within telepsychiatry and telemedicine more generally that are not necessarily included in standard medical school curricula.

In 2021, the AAMC published a report titled “Telehealth Competencies Across the Learning Continuum” which outlines educational goals for physician learners from medical school through residency and beyond (AAMC, 2021). The competencies include skills unique to remote care such as collecting data in the virtual environment and developing a “webside manner”—the way physicians present themselves within the camera frame and navigate discourse with patients. Mastery of such skills allows physicians to build rapport and foster meaningful connections with their patients in the virtual environment (Chua et al., 2020), which might not occur as organically as in traditional in-person encounters.

While formal training in telemedicine is nascent, several medical schools in the USA have existing and forthcoming training programs for students (Competency-Based Education in Telehealth Challenge Grant Program|AAMC). For example, the Medical College of Georgia at Augusta University offers a longitudinal telehealth curriculum exploring the connection between telehealth competencies and patient outcomes in chronic diseases; New York University is developing competency-based telehealth simulations; and Stanford and Stony Brook Universities are developing longitudinal, interprofessional telehealth education courses. Specialty-specific telemedicine training programs also exist, but they are less common (Liew et al., 2023; Curry et al., 2016) and tend to be geared towards the graduate medical education level (Shekunov et al., 2023).

For medical schools in locations with a paucity of psychiatrists, telepsychiatry can increase access to preceptors, reduce teaching ratios, and provide students with more individualized experiences. A telepsychiatry clerkship can also provide unique learning opportunities such as access to psychiatric subspecialties and exposure to unique practice settings such as assisted living facilities and rehabilitation centers. These features can improve the quality of the psychiatry clerkship for students and perhaps even bolster interest in the field. Despite these potential advantages, little is known about best practices and students’ perceptions regarding telepsychiatry education in medical school. In this study, we describe the development and implementation of a pilot telepsychiatry clerkship experience for third-year medical students at a multi-site medical training institution in 2022 and present survey data on students’ learning experiences.

Methods

From 2022 to 2023, a pilot educational telepsychiatry program was developed and incorporated into the neurosciences clerkship (psychiatry and neurology) completed by third-year medical students at Geisinger Commonwealth School of Medicine (GCSOM), an allopathic medical school in Scranton, Pennsylvania. Before the clerkship began, both preceptors and students were provided guidance about telemedicine workflow and general education about webside manner. As mentioned above, webside manner refers to the physician’s virtual presence and their capacity to employ relational skills such as “mindfulness, verbal empathy, and sentiment-congruent prosody” while also ensuring proper lighting, minimal distractions, and appropriate eye contact. Students were assigned a reading about clinic workflow and watched a pre-recorded lecture regarding the use of the telemedicine platform. They then attended an interactive virtual learning session which first focused on the types, advantages, and disadvantages of telepsychiatry and showed a video demonstration of webside manner. Second, students practiced these skills in small groups, and lastly returned as a large group for discussion and reflection. Preceptors were given a self-guided learning module, which included sections on webside manner, ways to incorporate medical students into the clinical workday in a telepsychiatry outpatient clinic, and the expected workflow with the students, along with quizzes to ensure comprehension and retention of the material. This was created by the clerkship director with the assistance of the GCSOM Curriculum Design and Development Team.

After the prework and training were completed by the preceptors and students, students were assigned to one preceptor during the three-week psychiatry portion of their neurosciences clerkship with whom they would remotely spend a half day in an outpatient telepsychiatry clinic via a video-based software system accessible by students, providers, and patients. The regional educational specialists for psychiatry assisted in coordinating the preceptors’ and students’ schedules to this end. The general workflow was as follows: observe while the attending sees the first two patients, chart review for the third patient, do the history and take ownership of the fourth patient, and then write the note for the fifth patient. For the rest of the day, the student can alternate with chart review and seeing the patients to help prevent cognitive overload and to help ease the additional workload of teaching for the preceptors. Tables 1 and 2 organize this workflow with clear expectations for preceptors and students depending on whether it is a new or follow-up patient, respectively.

Table 1 Template for remote telepsychiatry workflow for new patients
Full size table
Table 2 Template for remote telepsychiatry workflow for follow-up patients
Full size table

At the end of the half day in the telepsychiatry outpatient clinic, students and preceptors as a pair engaged in a personal feedback/teaching session focused on history gathering skills, note-writing skills, and general webside manner. Preceptors were provided with a performance rubric to help guide feedback towards AAMC telehealth competencies. At the end of the clerkship, students were sent a survey, which included both Likert scales (quantitative measure) and open-ended questions (qualitative measure), that evaluated their perceptions of the outpatient telepsychiatry component of the rotation. The results from the survey were used to report on the experience the third-year medical students had in the telepsychiatry portion of the clerkship. Both the performance rubric and Likert scale survey can be found in the appendix at the end of this report. Table 3 provides the Likert scale data used to analyze and consider the student perception of the telepsychiatry rotation.

Table 3 Likert scaled feedback of student perception of the telepsychiatry rotation
Full size table

Results

The quantitative portion of the survey consisted of two Likert scale questions, both of which were on a scale of 1 to 5. The first question asked if the outpatient telepsychiatry experience was a valuable addition to the clerkship, with 1 being “strongly agree” and 5 being “strongly disagree.” The second question asked how the telepsychiatrist supervision compared to the in-person psychiatry supervision, with 1 being “significantly better” and 5 being “significantly worse.” The qualitative portion of the survey elicited general comments and concerns about the telepsychiatry experience, which were also analyzed according to category or comment type. The complete qualitative dataset is included in the Appendix. The data from the Likert scales and open-ended questions were reported in percentages, and the Likert scale data was additionally represented visually in bar graphs and reported by sample size, mean, and standard deviation of the sample.

In total, there were 137 third-year medical students and nine attending preceptors who completed the telepsychiatry rotation from fall 2022 through spring 2023. For the first Likert scale question, a total of 88 students, which is more than half (64%), stated they “strongly agree” or “agree” the outpatient telepsychiatry experience was a valuable addition to the clerkship. Only eleven students (8%) “strongly disagreed” or “disagreed” with this statement. Thirty-eight students (28%) were neutral about this statement. For the second Likert scale question, a total of 37 students, which is slightly above a quarter (27%), stated they thought the supervision with the telepsychiatrist was “significantly better” or “slightly better” than their experience with the in-person psychiatry supervision. 20 students (15%) believed the telepsychiatry supervision was “significantly worse” or “slightly worse” than the in-person psychiatry supervision. Eighty students (58%) thought the supervision via telepsychiatry and in-person were equivalent.

Figure 1 uses a bar graph to illustrate visually the distribution between the opinions held by students about the outpatient telepsychiatry experience being a valuable addition to the psychiatry portion of the neurosciences clerkship, which was the first question posed to the students in a Likert scale. The mean of the Likert scale ratings on this question was 2.2, skewed to the side of the spectrum with “strongly agree,” while the standard deviation of this sample of ratings was 0.99 and the median was 2 (“agree”). Pearson’s first coefficient of skewness was calculated as 0.16. Figure 2 also uses a bar graph to demonstrate the distribution between the opinions of students comparing telepsychiatry supervision to in-person psychiatry supervision, the second Likert scale question on the survey that students completed at the end of the psychiatry rotation. The mean of the Likert scale ratings on this question was 2.8, not significantly skewed to any side of the spectrum, supported by the classic bell curve shape of the graph. The standard deviation of this sample of ratings was 0.85 and the median was 3 (“the same”). Pearson’s first coefficient of skewness for this question was − 0.19.

Fig. 1
figure 1

Outpatient telepsychiatry experience: valuable addition to clerkship? Response in the affirmative. Sample size, N = 137; mean = 2.2; median = 2; standard deviation of sample, SD = 0.99; Pearson’s first coefficient of skewness: 0.16

Full size image
Fig. 2
figure 2

Telepsychiatry supervision as compared to in-person psychiatry supervision: equivalency reported by students. Sample size, N = 137; mean = 2.8; median = 3; standard deviation of sample, SD = 0.85; Pearson’s first coefficient of skewness: − 0.19

Full size image

The qualitative portion of the feedback is processed as follows: a total of 56 comments were received, but 15 were excluded from the final analysis because content was irrelevant or non-contributory (i.e. “N/A,” “was unable to attend”); thus, 41 comments were analyzed in total. Over half of the comments (56%) mentioned the rotation as valuable and/or enjoyable, and 17% of the comments further requested more time devoted to telepsychiatry in the future. Specifically, students with a positive experience described it as “valuable,” “enjoyable,” and felt they had the opportunity to have “meaningful encounters” with patients. Similarly, 17% of respondents stated they received active feedback from preceptors and/or were highly engaged during their experience. For instance, one commenter explained they felt they had “more autonomy” and another received feedback after each patient encounter. There was, however, some negative feedback: about 20% of the comments cited technological issues, and 15% did not recommend this experience and/or described low engagement during their outpatient half-day telepsychiatry rotation. Furthermore, about 20% of commenters mentioned scheduling issues such as patient no-shows and interference with other components of their psychiatry clerkship. For instance, multiple comments noted their telepsychiatry experience was on the first day of their inpatient/outpatient clerkship. These same commenters also noted feeling uncomfortable informing their future preceptor they would be absent on the first day and feared it dampened the preceptor’s first impression of them.

Quantitative (64%) and qualitative (56%) measures demonstrate about half of the students believed this was a valuable addition to their clerkship experience. Quantitative results also support that the students generally felt there was no difference in supervision between the telepsychiatry and in-person psychiatry experience. Pearson’s first coefficient of skewness further supports these findings as data was skewed to the “agree” side regarding the added value of the experience and not skewed regarding the supervision quality. Therefore, there was a significant portion of the student population who felt they largely benefitted from the incorporation of this experience into their clerkship.

Discussion

To adequately prepare medical students for their future careers in healthcare, medical education must evolve to reflect changes in care delivery models. The increased use of telecommunication technologies has made it imperative for physicians to have telehealth expertise. Providing resources and opportunities to learn and develop these skills in medical school will allow for future physicians to be prepared as they enter the physician workforce. With the future in mind, a pilot program to incorporate telepsychiatry within a third-year curriculum has been evaluated with participating students. Overall, the feedback from the outpatient telepsychiatry pilot program was positive, with a majority of the students affirming that telepsychiatry supervision was at least or more effective than in-person psychiatry supervision, as illustrated in the graphics above. The experience was qualitatively extolled as being a valuable, enjoyable addition to the third year of medical school that enhanced the student-patient interaction. There were, however, technological challenges and scheduling issues that painted some students’ perspectives of this rotation in a more negative light.

A recurring theme described in the qualitative feedback was that there were challenges with and disruptions of the planned clinic workflow. The half-day schedule meant that any perturbation in the day would significantly impact the entire experience for the student. According to the “workflow” presented to preceptors, students were supposed to see at least five patients and be actively involved with at least one or two patients via chart review, history taking and note writing at a minimum. Yet, this could have gone awry for multiple reasons such as a preceptor forgetting to implement the workflow or a patient no showing for their appointment. As a specific example, one-quarter of the commenters mentioned technical issues which limited their patient time, whereas a momentary software update/delay for a student in a longer rotation would most likely be a mere passing inconvenience, in a short half-day rotation it may result in the learner missing nearly the entire experience. These are details to keep in mind for the reader interested in recreating this program. Some students simply did not appreciate the rotation due to the modality of care itself; one commenter specifically noted, perhaps cynically, they “just listened on the phone while calls were made.” Although the students, physicians, and patients interacted via a video-based software, some may feel this more passive modality hinders learning or perhaps this commenter’s preceptor did not adhere to the workflow. Thus, while a significant portion of students felt this was a valuable addition to their clerkship, improvements could be made to enhance this program.

The feedback received from students regarding the telepsychiatry rotation matches the outcomes found in reports from other institutions’ telehealth care and education models. Students have reported that having the capability to deliver care remotely is an important and necessary skill, and it should be included in medical education (El Kheir et al., 2023). These previous studies also demonstrated a majority of students who have had a telehealth opportunity found learning a “webside manner” helpful and the overall educational experience enjoyable (Bhatia et al., 2021; El Kheir et al., 2023; Ghaddaripouri et al., 2023). Furthermore, while this report did not collect feedback from preceptors, another report demonstrates that remote education provided improved educational outcomes for their students (Belakovskiy et al., 2022). Though this paper describes a pilot telepsychiatry rotation only, the literature shows there is currently ample interest, utility, and effort in establishing remote care learning opportunities within the medical school curriculum.

Technological challenges are unavoidable, but students and preceptors should be as prepared as possible to optimize this type of program. Some ideas for potential workarounds are described. First, ensure students use properly updated school-issued devices which would ensure all software is correctly installed. Second, if students are using their own devices, perform a technology check with a colleague the day before the rotation in order to ensure everything is working properly. Third, the attending can have a backup plan to call the patient by phone if the video does not work. It would then be necessary to ensure the physician knows how to place a conference call, a skill which can be taught as part of preparatory training. Fourth, if all else fails and the attending is unable to reach the patient, the student and attending can discuss a pre-made list of teaching topics during the scheduled patient appointment which could include charting, patient messaging, and correspondence etiquette.

In this particular pilot, 17% of students noted they would appreciate more time for this experience. Theoretically, this would give students the opportunity to see more patients, see a greater variety of patients, and mitigate any technological issues that do arise. A half-day is prone to errors that could skew a student’s perception about the experience from positive to neutral or neutral to negative. A longer pilot rotation would provide more opportunities for students to see a new patient encounter, reduce the impact of technical challenges, and to better understand the clinical nuances of telepsychiatry. However, since some students felt neutral or disagreed the experience was valuable, perhaps lengthening the program could be optional for the student in a future implementation of this project. This would allow those with greater interest to spend more time with this modality of care, while those with less interest would still have exposure and the opportunity to develop the unique skills required in this area of medicine.

Future areas of research could include adopting some of the potential improvements described in the limitations section and repeating the program with a future class of third-year students. A longer rotation could also serve as an opportunity to tease out other complicating factors which are not apparent in disparate half-day rotations. Ultimately, telehealth is a growing practice and the opportunity to generate meaningful contributions to the field is significant. Overall, this telepsychiatry pilot rotation for third-year medical students was largely met with positive feedback from participants and with understandable and potentially addressable resolutions for the negative feedback given by students. Since telepsychiatry dramatically increases the opportunity for more preceptors to teach students, it is feasible and well worth the effort for an undergraduate medical school program struggling to meet the clinical pedagogic needs of their medical students to consider a variation of this program in their own curriculum.

Limitations

While this report was able to identify the students’ general perception about this pilot program, there were limitations to the study. A few students were excused from the program due to personal emergencies or scheduling conflicts. Furthermore, qualitative analysis was limited since it was requested but not required, making it prone to response bias where perhaps participants with stronger opinions tended to comment, resulting in only 41 comments that met the analysis criteria. Although this study elucidated the students’ perspective on the clerkship, it would also be helpful to gather and incorporate formal feedback from the preceptors. This would be helpful to improve faculty participation, retention, and development opportunities.

Beyond the pedagogic utility and reported receptiveness of participating students, there are important practical challenges which must be acknowledged to every reader interested in preparing future pilots. While there was ultimately improved accessibility, the administrative burden of organizing and scheduling individual students with psychiatrists was significant. The physicians for example had dedicated time blocked out within their schedule so that they would be able to teach the students separate from the patient interviews. This is not a typical practice within the clerkship curriculum at large, and it is not likely sustainable within an established curriculum or busy outpatient practice. Additionally, in the event of a physician requiring a sudden change of schedule, students in a virtual setting will have less flexibility finding same day re-assignment with another virtual professional.

While GCSOM is not repeating this telepsychiatry program for its current cohort of third-year students, the student feedback and demonstrative pedagogic utility will ensure that it is considered a reasonable option with the aforementioned changes as both a curriculum addition as well as a means of expanding the clerkship capacity if needed in the future. The results of this study support that implementation of a telepsychiatry experience for medical students is feasible and can benefit learners. Important considerations include availability of a faculty champion for the effort, dedicated time in the curriculum for the training and experiential components of the program and having the administrative resources to assist with scheduling and technology challenges.

Conclusions

This manuscript presents a novel implementation of a telepsychiatry program for third-year medical students. Student feedback is demonstratively positive as a clinical modality; however, technical hurdles significantly impacted the experience for a large minority of learners. The difficulties experienced can be potentially addressed with dedicated administrative effort. Those medical schools which face a scarcity of available preceptors for students could avail themselves to the use of telepsychiatry technology for a clerkship to increase student exposure to the specialty, clerkship capacity, and interest in the field of psychiatry.