The residency application process is becoming increasingly competitive. Due to COVID-19, there were significant changes for applicants as they experienced an entirely virtual interview process. For matches 2021, 2022, and 2023, the American Association of Directors of Psychiatric Residency Training (AADPRT) recommended that programs offer virtual interviews [1,2,3]. While the transition to virtual interviews for match 2021 and match 2022 was secondary to concerns about COVID-19 transmission, it was recommended that match 2023 be virtual due to convenience, cost, and applicant preference [4].

As programs prepared for virtual recruitment, programs changed how they made information available to applicants. New digital resources were created and expanded, including video tours of institutions on program websites, virtual open houses, and social media accounts for residencies [5]. Conversely, due to organizational guidelines, away rotations were restricted [6]. The influence of these resources on recruitment remains largely unknown.

This study aims to understand the use of residency recruitment resources by psychiatry applicants who participated in match 2021 or match 2022, at two time points: when they apply and when they rank programs. While there is consensus that the shift to virtual recruitment has had major impacts on the match process, there have been no studies describing or quantifying these impacts. We hypothesize that most virtual recruitment cycle applicants will consider social media presence, virtual open houses, websites, and away rotations most influential when they apply and rank programs.

Methods

Between January 27, 2022, and February 24, 2022, a non-probabilistic sample of match 2018 to match 2022 psychiatry residency applicants were recruited to complete a survey via email and social media (Twitter and Instagram). A full list of 2021–2022 psychiatry programs and program directors was gathered from the Accreditation Council for Graduate Medical Education (ACGME) list of accredited programs. Emails were sent to 292 psychiatry program directors on the ACGME list requesting their help to disseminate the survey. No reminder emails were sent.

Due to the continuation of virtual recruitment post-pandemic, an analysis of psychiatry residents in the match 2021 and match 2022 cycles was conducted for this study. Participants had access to the survey either from social media or from their program director. We are unable to confirm that all program directors shared the survey link with their residents, though we know there were 1907 psychiatry spots available in match 2021 and 2047 in match 2022 [7, 8].

Survey responses were anonymous. An incentive raffle opportunity was offered to participants, and participation in the raffle required the use of an email address. Email addresses were stored in a separate survey database to ensure all survey responses remained fully de-identified. Raffle prizes include ten $50 gift cards and one $250 gift card.

The survey was created using Qualtrics software. It included demographic questions such as age, race and ethnicity, gender, and educational background. Respondents were also asked about the number of virtual open houses attended, the social media platforms they used to learn about residency programs, the social media content that they found most helpful, and the impact of the virtual interview season on the number of programs they applied to. Using Likert scales, with the option of “not applicable” if respondents did not access a resource, respondents answered questions about the influence of websites, virtual open houses, virtual video tours, in-person opportunities (i.e., away rotations), social media content, and the Fellowship and Residency Electronic and Interactive Database (FREIDA) on applicant application and ranking of programs.

Descriptive statistics were used to assess the data, and a chi-square analysis was used to assess the influence of various recruitment resources on applying for and ranking programs. The Washington University in St. Louis Institutional Review Board deemed this study exempt.

Results

Respondents included 605 psychiatry residents from the 2021 and 2022 match cycles. Considering the number of all psychiatry positions available in matches 2021 and 2022, this provides a response rate of 15.3%. Most respondents (n = 365, 60.3%) identified as female (Table 1). There were 288 (47.6%) responding US allopathic medicine doctors (US-MD), 178 (29.4%) international medical graduate doctors (IMG), and 139 (23.0%) doctors from an osteopathic medicine (D.O.) background. A majority of respondents (n = 353, 58.3%) were between the ages of 25 and 29 years.

Table 1 Demographics of match 2021 and 2022 survey respondents
Full size table

Among respondents, 57.4% (n = 347) reported that the virtual interview season led to an increase in the number of programs to which they intended to apply, whereas 42.6% (n = 258) reported no impact or applied to fewer programs than intended. Most respondents (88.3%) reported that they attended one or more psychiatry virtual open houses; 71 (11.7%) respondents attended 0 virtual open houses, 229 (37.9%) attended 1–3, 161 (26.6%) attended 4–6, 53 (8.8%) attended 7–9, and 91 (15.0%) attended > 10.

A majority of respondents (n = 512, 84.6%) followed psychiatry residency programs on at least one social media platform. Participants were most likely to use Instagram to learn about psychiatry programs (n = 398, 65.8%), and 40.3% reported using Twitter (n = 244). For those who followed content about residency programs on social media, 62.5% (n = 320) found posts that focused on news and updates for the program, activities like virtual open houses, or highlights of faculty and residents to be the most helpful. Respondents found social media posts focused on resident research and academic conferences (4.3%) to be the least helpful.

The influence of digital platforms on the decision to both apply and rank programs was measured using a 5-point Likert scale (from 0 = no influence on applying/ranking to 4 = a great deal of influence). Program websites were reported to be most influential for both applying and ranking (means 2.5434 ± 1.176 and 2.278 ± 1.201, respectively) among all digital platforms during virtual recruitment seasons (Table 2). Notably, the difference in influence between the two time points, applying and ranking, was significant (mean difference 0.265 ± 0.071, p = 0.0002). Conversely, participants reported the least influential digital platforms were social media sites, although there was no significant difference from applying to ranking (mean applying 1.382 ± 1.135 and ranking 1.411 ± 1.166, respectively; p = 0.681). The influences of program websites, virtual video tours, virtual open houses, and FREIDA were all found to be significantly different between application decisions and ranking decisions, with participants reporting all were less impactful on rank decisions.

Table 2 Influence of program resources on applying and ranking programs during match 2021 and match 2022 virtual recruitment cycles
Full size table

Discussion

Our study investigated the influence of residency recruitment resources on match 2021 and match 2022 psychiatry applicants. In part due to virtual interviews, the pandemic led many residency programs to spend time on non-traditional recruitment strategies, including social media, virtual open houses, and virtual video tours. Despite increased digital resources, traditional resources like program websites and away rotations appeared to be the most influential when deciding where to apply and rank programs.

Program websites had the largest influence on the application decision among the program resource options provided in this survey. This finding is consistent with studies performed in different fields of medicine. In a survey of match 2019 orthopedic surgery applicants, 96% of applicants used program websites to learn about a residency program [9]. With emergency medicine, applicants found information on a program website, like the curriculum and information about residents and faculty, helpful when making decisions about where to apply [10]. However, websites are inconsistent between programs and vary significantly. In 2020, an analysis of the content of psychiatry residency program websites found that a majority included a general overview of the program (99.2%), but fewer included a rotation schedule (43.9%), the number of trainee positions (42.0%), a diversity and inclusion statement (39.2%), and the employers of graduated residents (21.2%) [11]. Considering the influence websites may have on recruitment, residency programs should aim to keep their website up-to-date and complete.

Residency programs significantly increased their use of social media during virtual recruitment [12]. According to one study, in March 2021, 109 psychiatry programs had Instagram accounts, with 99 (90.8%) of these accounts being opened in 2020 [12]. Of the program resource options in this survey, social media had the least impact on applicant decisions for applying and ranking programs, despite most participants reporting the use of social media. Maintaining a social account takes time, energy, and often additional staff. While programs may benefit from ongoing use of social media, our data suggest efforts would be better directed to other resources for greater impact.

Virtual video tours provide an opportunity for applicants to see their future prospective work setting when they are not able to visit campuses. Respondents in our study found virtual video tours more impactful in the application process than the ranking process. While the number of psychiatry programs with virtual video tours on websites is unknown, one study of orthopedic surgery program websites between July 2020 and November 2020 found a 12.7% increase in the presence of virtual tours on program websites [13]. Programs without a virtual video tour on their website might consider adding one to help applicants make decisions about where to apply.

Virtual open houses have also become a fixture of virtual recruitment. The majority of respondents reported attending at least one virtual open house. Applicants reported that open houses were relatively more impactful on the decision to apply versus rank programs. A survey of urology applicants found that virtual open houses were beneficial, especially when applicants can learn the strengths and weaknesses of a program and get their questions answered [14]. Per AADPRT, virtual open houses and fairs were advised to end before the Electronic Residency Application Service submission date in September, likely contributing to the influence these have earlier in the application cycle. Future changes in AADPRT guidelines could allow programs to offer virtual open houses at the end of recruitment season to help applicants answer any last-minute questions when making a rank list, and programs could consider doing these earlier on in the season.

Away rotations provide applicants with an opportunity to learn about the program’s culture and city. Per ACGME recommendations, however, few respondents in our survey participated in away rotations during the match 2021 and 2022 application seasons due to COVID limitations [6]. Those who did found the experiences impacted applying and ranking, but there was no statistical difference between the two. Notably, away rotations were more impactful than other factors in this survey for application and ranking decisions. This suggests that despite virtual recruitment, applicants still found away rotations to be useful. Due to equity, it may be beneficial to permit no more than one away rotation per applicant. Given applicant financial barriers for away rotations, psychiatry programs can consider offering away rotation scholarships [15].

Outside of program-created resources, the centralized database FREIDA provides information about programs. Notably, FREIDA had the largest decrease in impact between influences on applying versus ranking programs. Applicants likely use FREIDA to decide where to apply when comparing program characteristics including demographics, geography, and visa status. Our findings suggest this information may be particularly valuable early on in the application cycle when differentiating psychiatry residency programs. Although there is no published data on psychiatry programs, studies in other specialties indicate that other specialties have missing information in the database [16, 17]. One study observed discrepancies with FREIDA data compared to program websites lead to uncertainty for applicants [16]. Based on the way FREIDA is set up, it can take time for newly accredited programs to appear on the database [18]. Keeping FREIDA updated with accurate information may improve transparency and reduce the number of applications per applicant. For example, if FREIDA is updated and complete, IMG physicians can more easily find information about visas.

A majority of respondents described how virtual interviews led them to apply to more programs than originally intended. Over the past 5 years, there has been an increase in applications per applicant [19]. Increased application numbers can be financially challenging for applicants and burdensome for programs to screen and select interviewees. Further discussion and proposals are needed to decrease overapplying, including interview caps, increasing transparency about a program’s review process, and the use of preference signaling in the match 2023 cycle. While ophthalmology has used interview caps, they are a smaller specialty and use a different application service platform and match system [20].

In terms of limitations, our survey study is subject to selection bias and recall bias. Preferably the response rate would be based on the total number eligible for the survey, independent of program directors disseminating the survey. Although small, the large number of responses received in this survey does provide needed information on the subject.

In conclusion, given the transition to virtual recruitment and the increase in the number of applications per person, understanding the utilization and influence of recruitment resources is critical. Programs have limited time and resources to dedicate to recruitment, and prioritizing efforts is necessary. Programs may consider ensuring that their website is updated, offer a combined virtual open house with other psychiatry programs, attend a residency fair, or spend fewer resources on their residency program social media accounts to maximize recruitment efforts. Finally, the development and adoption of best practices for recruitment might better inform applications and create consistency among programs. As virtual interviews continue, programs need to consider how recruitment is an ever-growing, changing, and evolving process.