Cognitive behavior therapy (CBT) is one of several empirically supported psychotherapeutic treatments for depression and anxiety disorders. Despite its clinical efficacy, only a fraction of those who could benefit are able to access and complete CBT-based treatments [1,2,3] due to a variety of factors, including inadequate insurance coverage and the resource-intensive nature of psychotherapy [4, 5]. Despite the passage of the Affordable Care Act [6], barriers remain to obtaining insurance of sufficient coverage and quality [7, 8]. Overall, the uninsured have the lowest utilization rates of mental healthcare services, including psychotherapy [9].

Student-run free clinics (SRFCs) are an important source of medical care for those with limited financial means and provide training opportunities at most major US medical schools [10]. There is a growing evidence base that students at SRFCs can effectively screen for psychiatric illnesses [11,12,13,14,15,16] and provide beneficial treatment services [12, 17,18,19,20,21,22,23,24,25]. Nevertheless, most SRFCs provide only screening, referral, and medication management–based services. While a few have reported preliminary findings of student-facilitated skills-based group therapy [25] or psychoeducational interventions [12], no SRFC to date has reported on the provision of individual psychotherapy treatments by medical student trainees.

To address psychiatric conditions more comprehensively in our clinical population, we designed a multipronged education program to train medical students in CBT. The program included didactic training and supervised individual psychotherapy sessions with patients who had depression, anxiety, and trauma-based disorders in an SRFC serving residents of the East Harlem borough of New York, New York [19]. We evaluated the effectiveness of the program in several domains.

Methods

All patients were Hispanic, Spanish-speaking, >21 years of age, and receiving treatment at the Mental Health Clinic of the East Harlem Health Outreach Partnership (EHHOP MHC, or E-MHC), a student-run and faculty-supervised free clinic affiliated with the Icahn School of Medicine at Mount Sinai (ISMMS) in New York, New York. The overall design of and outcomes in the clinic have been described previously in detail [18]. The study period took place from June 1, 2019, to March 1, 2020; during this period, all patients began receiving CBT treatments in addition to treatment as usual (TAU). It was also during this period that we analyzed Patient Health Questionnaire 9 (PHQ-9) [26] and Generalized Anxiety Disorder Scale 7 (GAD-7) [27] data in all patients to compare clinical outcomes.

All students in this study were medical (M.D. or M.D.-Ph.D.) students at ISMMS. We recruited trainees by word-of-mouth as part of annual volunteer recruitment; all trainees served as volunteers in the clinic during the study period. CBT trainees underwent didactic training during the summer in between their first and second years of medical school, provided supervised treatment during their second year, and did not have prior education or training in CBT. Students providing TAU were in their second, third, or fourth year of medical school or were M.D.-Ph.D. students in the graduate phase of the program.

Student clinicians in both treatment groups had completed the pre-clinical medical school course in psychiatry. Prior to any clinical work, all students completed a 2-hour orientation session that included review of psychiatric interviewing, the mental status exam, and suicide risk assessment. Students providing TAU began clinical work immediately after completion of the orientation session. Students providing CBT also completed a faculty-led course of eight weekly didactics consisting of approximately half lecture-based content and half role-play practice. The CBT curriculum was chosen based upon topics that are broadly taught in introductory CBT courses as well as those related to conditions affecting patients served in our clinic. The topics of the weekly didactic sessions were, in order, CBT theory and session structure, identifying and modifying cognitions, behavioral techniques and non-adherence, the first session in CBT, and CBT for depression, anxiety, post-traumatic stress disorder (PTSD), insomnia, and substance use.

The Cognitive Therapy Awareness Scale (CTAS) [28] is a 40-item assessment designed to measure knowledge of CBT. The maximum score is 40 points, and all items are based upon yes or no responses. Individuals with no CBT knowledge should receive a score of 20, and original testing data showed subjects scored an average of 24.2 that improved with CBT training [28]. The reliability of the CTAS was estimated to be moderate at 0.58 [28]. In this study, trainees completed paper copies of the CTAS one week prior, immediately after, and six months after conclusion of the didactic program. One of the study authors with prior CBT training (SKP) supervised test administration to ensure that trainees did not have access to any CBT materials while completing CTAS assessments. Trainees were not given copies of the CTAS nor the correct answers until after data from all three assessment timepoints had been collected.

Both CBT and TAU sessions were each approximately 60 min in duration. For cases in which the student was not fluent in Spanish, Pacific Interpreters, LLC, provided professional medical translation services over a landline telephone in the private clinic room. After sessions, students in both treatment groups presented to an attending psychiatrist or postgraduate year 4 psychiatry resident to discuss the case, the patient’s clinical status, and formulate a treatment plan. Then, the student and attending or resident met with the patient to answer questions, address any safety concerns, and review the next steps in management. All patients in this study received TAU, and the majority had been undergoing TAU prior to the initiation of CBT sessions. All patients were aware that their treatment providers were medical students who were supervised by senior-level psychiatry residents, psychiatrists, and/or clinical psychologists. The inclusion criteria in this study were patient ability and voluntary willingness to complete written study measures (e.g., PHQ-9) and completion of intake assessment and baseline study measures, as well as at least one endpoint measure (described below). We did not include in this study any patients with active psychotic disorders, evidence of intoxication during treatment sessions, those with dementia, or those with personality disorders of sufficient severity to warrant referral to more intensive treatments, as the clinic does not provide such services.

TAU sessions occurred monthly. At the first clinic visit, students conducted an intake psychiatric assessment that was reviewed with an attending psychiatrist to establish the presence of any psychiatric disorders using the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition, criteria and to formulate an initial treatment plan. All follow-up visits were predominantly focused on medication management if applicable and, as time permitted, included general, non-specific, non-manualized, supportive counseling.

After completion of the didactic program, CBT trainees took on one or two patients who wished to engage in CBT. Patients were offered CBT treatments if they were receiving ongoing TAU; had a depressive, anxiety, adjustment, or trauma-based disorder; had insufficient improvement in symptoms after at least 1 month of TAU; and did not have a record of frequent non-attendance to TAU sessions. We excluded individuals who had only minimal symptom severity, defined as <5 on each of the two scales [26, 27]. For analyses of clinical efficacy, we also excluded individuals in the CBT + TAU group who did not have at least four CBT sessions during the study period.

Trainees conducted individual, 60-min CBT sessions with their patients two to four times per month. Treatment sessions were non-manualized and were instead formulated by the trainee and the CBT supervisors based upon the conceptualization of the patient in the CBT model, focusing on problematic symptoms and related thoughts, feelings, and behaviors; level of functioning; and treatment goals. All CBT interventions were consistent with general treatment guidelines and techniques as detailed in Treatment Plans and Interventions for Depression and Anxiety Disorders, Second Edition, by Leahy, Holland, and McGinn [29].

Immediately after completion of the didactic program, all trainees began weekly supervision sessions in small-group format. Two Ph.D.-level clinical psychologists (VG, CA) led all sessions; study author VG served as the primary supervisor and oversaw more than half of all supervision sessions for each of the trainees. Consistent with current practices and guidelines [30, 31], CBT supervisors worked with trainees to establish presenting problems and treatment goals, conceptualize cognitive formulations, review progress in prior sessions, plan subsequent steps in CBT therapy, role-play therapeutic interventions, troubleshoot challenges, and provide a supportive role for trainees.

Four to six months after trainees completed the didactic program and began providing treatment sessions, one of the study authors (VG) completed the CBT Supervision Checklist (available from the authors upon request) for each trainee based upon their weekly reports of the contents of CBT sessions, case formulations, technical adherence during sessions, and utilization of appropriate CBT interventions.

We used the PHQ-9 and the GAD-7 to measure depressive [27] and anxious [32] symptoms, respectively. Patients completed the PHQ-9 and GAD-7 after checking in for their clinical appointments at each visit during the study period. Student providers reviewed completed PHQ-9 and GAD-7 forms and recorded results in the electronic medical record.

We employed a modified version of the Patient Feedback Survey described previously [18]. Only those receiving CBT + TAU completed the surveys after CBT treatment sessions. All surveys were completed voluntarily: individuals were provided with the surveys at the end of their visits and had the option to complete them and turn them in. The survey contains two sections: the first asks a series of questions regarding feedback on medical student CBT providers, while the second seeks to ascertain patient perceptions of improvements across broad domains of psychosocial functioning. The questions in the first section assessed patients’ feedback of CBT trainees in how often they performed five components of care and utilized answer choices ranging from never to always. The questions in the second section asked about how much better they felt in broad domains of psychosocial functioning and employed a 5-point Likert scale ranging from strongly disagree to strongly agree.

We employed R version 4.2.1 for all statistical analyses. We used Fisher exact tests to evaluate whether there were any differences in demographic and clinical characteristics in the two treatment groups. We employed repeated-measures one-way ANOVA with Bonferroni correction to analyze changes in CTAS composite scores across the three timepoints. To evaluate whether the addition of CBT to TAU was associated with improved outcomes, we utilized PHQ-9 and GAD-7 data collected from patients as part of routine clinical monitoring. We defined baseline scores as those recorded at the first available treatment session (whether CBT or TAU) within the study period. The endpoint score was defined as the last score recorded before the end of the study period. We first tested whether receipt of CBT in addition to TAU during the study period was associated with differences in the magnitude of improvement in PHQ-9 and GAD-7 using mixed within-subjects (baseline versus endpoint scores) and between-subjects (TAU versus TAU + CBT) two-way robust (non-parametric) ANOVA tests with the WRS package [33]. We then used multiple regression to explore whether any associations observed were simply due to an increase in the overall number of treatment sessions or could be attributed to the addition of CBT per se to TAU. Finally, we evaluated whether the specific number of CBT sessions received was associated with endpoint symptom severity using multiple regression with baseline symptom scores as a covariate. We provide only descriptive statistics for the results of the CBT Supervision Checklist assessments and the patient feedback surveys.

The Institutional Review Board at ISMMS approved of the research conducted herein.

Results

We included 16 patients who received TAU and 22 who received TAU + CBT, for a total of 38 (Table 1). The mean age was 45.7 (12.8) and ranged from 23 to 76. The proportion of male and female patients did not differ between the two treatment groups (OR = 0.89 (0.15–5.5), p > 0.99). Because of sample size limitations, diagnoses of major depressive disorder, persistent depressive disorder, and complicated grief were grouped into the composite diagnostic term “depression,” and diagnoses of generalized anxiety disorder, social anxiety disorder, and panic disorder were grouped into the composite diagnostic term “anxiety.” As shown in Table 1, all patients had depression, anxiety, adjustment, and/or trauma-based disorders. The proportions of patients with these diagnoses did not differ between the two treatment groups (Table 1). Sixteen (42.1%) patients had more than one psychiatric diagnosis (TAU: n = 8, TAU + CBT: n = 8), and these patients were not more likely to be in one treatment group than another (OR = 0.58 (0.12–2.56), p = 0.51). Finally, we did not find evidence that the proportions of patients on any psychotropic medication (vs. none), selective serotonin reuptake inhibitors, and specific medications differed between the two groups (Table 1).

Table 1 Clinical and demographic characteristics of patient population
Full size table

All eight trainees completed the CTAS at each of the three timepoints. Composite scores prior to the didactic program had a mean of 29.0 and ranged from 25 to 35. Immediately after completion of the didactic program, the mean composite score increased to 32.9 (range: 26–37). Six months later, the mean composite score was 34.1 (range: 32–38). Results of repeated-measures one-way ANOVA demonstrated that composite CTAS scores differed significantly across the three timepoints (F(2,14) = 8.47, p = 0.004). Post hoc analyses revealed that scores immediately after the didactic program increased significantly from baseline by a mean of 3.9 points (adjusted p = 0.02). At the third and final assessment six months after completion of the didactic program, CTAS scores remained increased over baseline assessment by 5.1 points (adjusted p = 0.04). Scores at the two post-didactic assessments did not differ significantly (adjusted p > 0.99).

Across all competencies, most medical student CBT trainees attained a rating of “satisfactory” (Table 2). Five of the eight trainees were rated as “did not attempt or N/A” on the “Can utilize exposure and response prevention or graded task assignment” competency, which either reflected the fact that not all students were treating individuals with conditions, such as PTSD, for which this approach is frequently utilized or a genuine weakness in which the trainees should have utilized the competency but did not do so. All eight trainees attained a rating of “did not attempt or N/A” on the competency of “Can utilize CBT relapse prevention methods” because it was too early during the overall treatment courses for the trainees to begin utilizing these techniques with their patients (Table 2).

Table 2 Evaluation of cognitive behavior therapy (CBT) treatment competencies with the CBT Supervision Checklist
Full size table

PHQ-9 data were available on 34 patients (TAU: n = 14, TAU + CBT: n = 20). The median baseline score was 9.5 in the TAU group (range: 5–22, maximum of 27 possible) and 10 in the TAU + CBT group (range: 5–24). Depressive symptom severity did not differ between the two groups at baseline (W = 141, p = 0.99), and the two groups did not differ in the number of TAU sessions received prior to the beginning of the study period (W = 100, p = 0.15). There was a significant interaction effect between treatment group and timepoint (p = 0.01), and post hoc analysis demonstrated that individuals in the TAU + CBT group improved by an average of 5.4 points more than did those receiving TAU alone (p = 0.04). In the multivariate regression modeling, the addition of CBT to TAU was associated with a reduction in endpoint PHQ-9 score by 5.7 (1.9) points (p = 0.006), but the total number of visits (TAU visits plus CBT visits) during the study period was not associated with differences in endpoint PHQ-9 score (β = 0.16, p = 0.52). We did not find evidence that the number of CBT sessions received by those in the TAU + CBT group related to the endpoint depression severity after adjusting for the baseline PHQ-9 score (β = −0.32 (−0.56 to −0.08), p = 0.20).

GAD-7 data were available on 32 individuals (TAU: n = 13, TAU + CBT: n = 19). The median score was 7.0 (range = 7.0–18.0, maximum of 21 possible) in the TAU group and 9.0 (range = 5.0–21.0) in the TAU + CBT group. Anxious symptom severity did not differ between the two groups at baseline (W = 104, p = 0.47), and the two groups did not differ in the number of TAU sessions received prior to the beginning of the study period (W = 104, p = 0.44). There was no evidence of an effect for treatment group on the magnitude of symptom reduction (p = 0.36). The model relating the addition of CBT to TAU on endpoint GAD-7 scores with the baseline score and total number of treatment sessions as covariates did not find an association between receipt of CBT in addition to TAU and endpoint anxiety severity (β = −1.7 (−3.5 to 0.1), p = 0.34). We then tested whether the specific number of CBT sessions received by those in the TAU + CBT group related to endpoint anxiety severity after adjusting for baseline GAD-7. Every additional CBT session was associated with a decrease in endpoint GAD-7 by 0.47 points (p = 0.02).

A total of 24 patients completed the first part of the feedback survey. Overall, 95% (n = 21/22) of respondents reported that their CBT trainee always spent enough time with them, and one indicated that this was “sometimes” the case (on 2 of the 24 surveys, this question was left blank). Respondents reported that they were always treated with respect and that they felt safe during the session in 100% of the surveys collected (n = 23/23, with 1 left blank). Furthermore, 100% (n = 24/24) of respondents indicated that their CBT trainee always listened carefully and explained things well to them.

Twenty-two respondents completed the second part of the survey, and on a few of the surveys, certain questions were left unanswered. Overall, 95% (n = 21/22) indicated that they better understood their personal problems, while 100% (n = 22/22) agreed or strongly agreed that they were satisfied with their overall improvement, that they were better at solving their personal problems, that they had improved relationships, and that they had made progress toward goals. The majority (90% (n = 20/22)) indicated that they were more sensitive to differences in others, and the remaining 10% (n = 2/22) were neutral. An identical proportion (90% (n = 20/22)) of respondents reported that they were better able to handle their feelings and communicate their emotions with others; the remaining 10% (n = 2/22) were neutral.

Discussion

In this study, medical student trainee knowledge of CBT improved following the eight-week didactic program. The magnitude of improvement in our sample was smaller than that reported in the original CTAS study [28], where scores improved by 8.3 points. Differences in the sample demographics between the two studies likely contributed. Consistent with this possibility is the finding that our baseline pretest scores had a mean of 29.0, whereas the mean baseline score in [28] was 24.2. To our knowledge, no prior study of an SRFC mental health care service has directly tested trainee knowledge of the relevant content area(s) and demonstrated improvement. We believe that longitudinal evaluation of student knowledge is an important component of ensuring the provision of high-quality, ethical care, particularly when provided by a heretofore atypical group of trainees to individuals with numerous psychosocial and economic disadvantages. In this scenario, ethical principles [34] require that clinical care is competent, consistent with patient values, and provides clearly demonstrable benefits.

We assessed the extent to which trainees adhered to proper clinical technique and provided competent CBT treatments. Trainees performed at a predominantly “satisfactory” level, which is expected of those who are newcomers to a unique set of clinical skills. Evaluations were conducted by a single rater based upon aggregated information self-reported by trainees over four to six months of weekly supervision sessions, and we did not record the clinical treatment sessions in any way, as this was beyond the technical resources available to our clinic. This constitutes the primary weakness of this aspect of our study; the information self-reported by trainees is likely biased and cannot be verified through audio(visual) recordings of treatment sessions. Furthermore, there is limited data on the psychometric properties of the Cognitive-behavior Therapy Supervision Checklist, which was developed for training purposes and has not undergone extensive validation studies (Donna Sudak, M.D., personal communication). It will be imperative that the competencies of this new trainee population are evaluated using instruments such as the CTRS [35] in future studies. Importantly, our feedback data indicate that patients felt that they were treated well by their CBT trainees.

In the primary analysis on the association between the addition of CBT to TAU and endpoint symptom severity, we found that those in the CBT + TAU group improved by an average of 5.4 points more than did those receiving TAU alone in depressive symptom severity; we did not detect such an association for anxiety symptoms. Because of the study design, we cannot reach any conclusions about causal relationships or lack thereof. Despite the lack of association between the addition of CBT to TAU and improved anxiety symptom severity, we did find an association between the specific number of CBT sessions received and the magnitude of improvement in anxiety among the TAU + CBT group. Importantly, we note that our findings only pertain to patients grouped together across all diagnoses and are not particular to those with specific diagnoses, as we were not adequately powered to undertake such analyses. Taken together, these findings suggest a potential clinical benefit of CBT provided by medical students and showcase the promise of this trainee population in the provision of psychotherapy.

SRFCs serve a key role both in providing much needed treatments to underserved patients and in generating formative clinical experiences for early-stage medical trainees. Our study reflects the utilization of a thus far untapped trainee pool, namely, medical students, to provide supervised psychotherapy to patients in need. To our knowledge, this study is the first report showing that medical students can be trained to provide competent and clinically beneficial individual CBT treatments to an entirely Spanish-speaking, uninsured population. This represents an important advancement in addressing unmet psychotherapy needs in underserved populations, especially given the evidence that a greater proportion of Hispanic individuals and members of other racial and ethnic minorities may prefer psychotherapy to pharmacologic treatments of psychiatric conditions [36]. In parallel, we note that prior reports of mental healthcare services in SRFCs show benefits in bolstering student knowledge of, skills in, and attitudes toward clinical psychiatry [37,38,39,40]. We expect that our program and training model represents a novel advancement in promoting medical student knowledge and competency in psychotherapy. Future studies implementing psychotherapy training and supervised practice in medical education settings are needed to further evaluate the feasibility, utility, and clinical effectiveness of our model.