Introduction

Over the last decade, there has been an increased awareness of inequitable access to quality health care and the resultant need to improve the widespread dissemination of evidence-based treatments (EBT; Harvey & Gumport, 2015; Nadeem et al., 2016). This has been amplified by the rise in mental health problems and concerns resulting from the COVID-19 pandemic, leading to increased need for mental health services that were already at capacity prior to the pandemic. As one way to address these concerns, research has increasingly focused on identifying and evaluating strategies to train mental health providers (MHPs) (e.g., Edmunds et al., 2013; Hanson et al., 2016; Lang et al., 2015). While there have been some discrepant findings regarding the effectiveness of different training strategies (Valenstein-Mah et al., 2020), general consensus across the field is that multi-component strategies that include opportunities for skill-building, practice, and active coaching/consultation appear to achieve the most effective training outcomes (Edmunds et al., 2013; Fixsen et al., 2009; Herschell et al., 2015). One specific example of a multicomponent training strategy is the Learning Collaborative (LC). LCs include several components (e.g., didactics, readings, consultation, coaching) to train clinicians in a specific EBT, while also providing ongoing support to address therapist- and organizational-level implementation barriers (Amaya-Jackson et al., 2018; Ebert et al., 2012; Hanson et al., 2019; Herschell et al., 2015). Studies evaluating the effectiveness of LCs have found positive client treatment outcomes, as well as improved treatment adherence and competence, pre- to post-LC (Amaya-Jackson et al., 2018; Ebert et al., 2012; Helseth et al., 2020; Espeleta et al., 2021; Hanson at al., 2019). In addition to the noted increase in mental health problems and need for additional trained MHPs, the COVID-19 pandemic also required adaptations and changes in the delivery of treatments and the ways in which MHPs are trained, including remote work and an increased use of telehealth (Molfenter et al., 2021; Whaibeh et al., 2020). However, because of the sudden onset of COVID, there is limited information on how pandemic-related changes (e.g., quarantines, social distancing, increased financial and other personal stressors) impacted MHPs and their abilities to participate in training or to deliver evidence-based care to their clients.

Impact of COVID-19

It is common for MHPs to experience work-related stress (Simionato & Simpson, 2018), particularly those that work with populations exposed to trauma (Makadia, Sabin-Farrell, & Turpin, 2017). Most recently, the COVID-19 pandemic has posed additional challenges and stressors for MHPs including the societal impact of pandemic-related restrictions (e.g., quarantine, social distancing, financial losses, remote work), the personal impact of MHPs’ own losses and health concerns, and compassion fatigue related to increased stress and mental health concern and need of clients. Recent research conducted during the COVID-19 pandemic suggests that MHPs experienced high levels of COVID-related stress (Aafjes-van Doorn et al., 2020, 2021; Ledesma & Fernandez, 2021; Probst et al. ,2020). For example, Aafjes-van Doorn and colleagues (2020) surveyed 339 therapists about their professional practices and experiences during the pandemic and found that on average, therapists experienced moderate levels of stress and that about 15% experienced high levels of stress. What remains less known is how these challenges may have impacted MHP engagement in training and treatment delivery.

In addition to the impact of COVID-19 on MHP stress and burnout, COVID-19 led to many changes in MHPs’ roles and responsibilities. In March 2020, most agencies and organizations delivering mental health services pivoted to telehealth (Molfenter et al., 2021; Whaibeh et al., 2020). Further, as a result of health and safety restrictions imposed by health authorities, many MHPs transitioned to remote work and subsequently had shifted or decreased work hours. Although telehealth has allowed MHPs and clients to continue ongoing treatment, the involuntary and sudden shift to an online platform likely impacted MHP stress levels and their ability to deliver treatment (Aafjes-van Doorn et al., 2021). Likewise, the pandemic forced MHPs to complete trainings and training-related requirements while adhering to health and safety protocols. Thus, ongoing training efforts (e.g., training sessions, consultation, and supervision meetings) also transitioned to online or virtual formats. Although there is an emerging body of literature on the impact of COVID-19 on MHPs’ mental health and stress, to the authors’ knowledge, there is not yet research examining the impact of these changes on MHPs’ abilities to participate in professional trainings and subsequent treatment delivery.

The Current Study

Overall, the impact of pandemic-related stressors may be exacerbated for MHPs engaged in a comprehensive training initiative, such as a Learning Collaborative, that requires extensive time for training sessions and coaching/consultation calls, in addition to treatment delivery. Thus, the current mixed-methods study includes a sample of MHPs who engaged in one of five Learning Collaboratives conducted annually as part of a statewide initiative from 2017 to 2021. The LCs focused on training MHPs in the delivery of trauma-focused cognitive behavioral therapy (TF-CBT; Cohen, Mannarino, & Deblinger, 2006), an evidence-based mental health treatment for trauma-related symptoms in children and adolescents. Study aims were to examine whether MHPs’ experienced COVID-related stressors and the impact of these challenges on their perceived competence in TF-CBT delivery and completion of training requirements. Quantitative surveys were administered to all participants before and after the LC. Qualitative interviews were conducted with 15 MHPs who participated in an LC that occurred during the COVID-19 pandemic (during 2020 or 2021), with the goal of expanding upon quantitative findings by exploring barriers and resources that may have impacted the completion of training requirements and delivery of TF-CBT with their clients.

Study I. Quantitative Data

Methods

Participants

Participants (N = 269 included masters’ and doctoral-level MHPs across a Southeastern state, who participated in one of five learning collaborative cohorts (cohort 1 n = 55, cohort 2 n = 60, cohort 3 n = 58, cohort 4 n = 67, cohort 5 n = 29) conducted before (pre-COVID; 2017–2019) and during the COVID-19 pandemic (peri-COVID; 2020–2021). Given that data were gathered originally as part of a training and implementation initiative versus a research study, participant demographics (e.g., age, race, gender) were not collected.

Procedures

Project X (removed for blinded review) is an ongoing statewide initiative that started in 2007 with foundation funding from X. The overall mission of Project X is to expand the reach and impact of trauma-focused EBTs for youth and families across the state. Thus, as part of this initiative, Project X provides annual statewide LCs to train mental health providers to deliver TF-CBT. Annual LCs are open to community mental health agencies and MHPs across the state who have obtained at least a master’s degree. Five cohorts of TF-CBT LCs were conducted between 2017 and 2021.

Each LC lasted approximately 10 months and consisted of four main components: a pre-work phase, two learning sessions, action periods, and weekly metrics. In the pre-work phase, MHPs completed the web-based TF-CBT training course (TF-CBTWeb2.0) and attended a webinar orientation session to learn about the LC training requirements. MHPs then attended two training workshops (i.e., learning sessions) led by TF-CBT master trainers; these learning sessions included didactics and practice opportunities to enhance MHPs’ TF-CBT-specific skills and to identify and reduce barriers to implementation. Learning Sessions for Cohorts 1 and 2 (pre-COVID) were conducted in person for two full days; Cohorts 3–5, held during the COVID-19 pandemic, were conducted virtually (via Zoom) over three six-hour days. During the 3–4-month action periods between and after the two learning sessions, MHPs identified training cases to implement TF-CBT and participated in biweekly group coaching/consultation calls led by nationally approved TF-CBT trainers and certified TF-CBT supervisors to address barriers to treatment delivery.

Participating MHPs also completed weekly online questionnaires (i.e., metrics) about each of their TF-CBT training cases. These metrics assessed MHP self-reported use and perceived competence in the delivery of the TF-CBT components, caregiver engagement in treatment, and barriers to TF-CBT treatment model adherence (e.g., new events arising, client disengagement, etc.) in that week’s session. Weekly metric surveys were collected following the first learning session and then throughout the LC training initiative. Training faculty shared group-level metrics to MHPs to update them on group progress and tailor LC training activities. MHPs completed pre- and post-online surveys that measured clinical practice information, TF-CBT knowledge, perceived TF-CBT competence, TF-CBT practices, community practices, and organizational climate. MHPs who attended both learning sessions, participated in at least 12 consultation calls, and completed a minimum of two TF-CBT training cases were added to the Project X roster of trained professionals. All authors certify their responsibility for this contribution and declare that there are no known conflicts of interest.

Measures

TF-CBT Clinical Skills Questionnaire (TCSQ)

The TCSQ (MUSC, 2010) consists of 33 items assessing therapist perceived competence in delivering and adapting different TF-CBT components across various cultural, developmental, and familial backgrounds. MHPs rated their competence for each item on a 5-point Likert scale (1 = not at all, 2 = a little bit, 3 = somewhat, 4 = very much, 5 = exceptionally). There are six subscales: Psychoeducation (two items), Coping (four items), Exposure (six items), Caregiving (three items), General (seven items), and Tailoring (12 items). The total competency score is an average of all 33 items and provides an overall score of MHPs’ perceived competency in delivering TF-CBT. Total scores on the TCSQ ranged from 31 to 160. The TCSQ was administered to all 5 cohorts. Internal consistency for the sample was excellent (α 0.95).

LC Training Requirements

LC training requirements were tracked by the project coordinator. This included number of TF-CBT cases identified, number of TF-CBT cases completed, attendance at both learning sessions, percent of weekly metrics completed, number of consultation calls completed, and whether roster requirements were completed.

COVID-19 Questionnaire

These 6 items, developed for Project X and administered at the post-assessment for cohorts 2020 and 2021, assessed the degree to which MHP respondents were impacted by COVID-19 and COVID-19 related stressors (e.g., “How much has the COVID-19 pandemic increased your stress level?”). MHPs indicated how much news media coverage related to COVID-19 they consumed using a 5-choice, Likert-type format ranging from 1 (not at all) to 5 (5 or more hours per day). Then, MHPs indicated how much the COVID-19 pandemic impacted their stress level, their daily work schedule, their ability to deliver TF-CBT, and how concerned they were for the safety of their clients using a 100-point scale ranging from 0 (not at all) to 100 (significantly impacted). Lastly, MHPs were asked to indicate on a dichotomous scale (yes or no) which changes they made in their work schedule because of the COVID-19 pandemic (i.e., reduced hours, changes in workday such as morning or evening hours, changes in workday such as weekend hours).

Data Analysis

All analyses were conducted in IBM SPSS Statistics, Version 28 (IBM Corp., 2021). Descriptive statistics, independent samples t-tests, and chi-square tests of independence were computed to examine differences in training requirement completion and reported competence between pre-COVID and peri-COVID cohorts. Mann-Whitney U tests were computed to analyze differences between variables with non-normally distributed data. There were no missing data for all training completion variables; however, data completion varied regarding the measure of MHP-reported competence. Approximately 98% of MHPs across all cohorts completed the pre-TCSQ, whereas 72% completed the post-TCSQ. Given the significant number of participants who did not complete the post-TCSQ, a two-tailed independent samples t-test was performed and subsequently indicated that post-TCSQ survey completers and non-completers did not vary significantly based on pre-TCSQ responses t(p = .79). Additionally, Little’s test indicated data among pre- and post-TCSQ surveys were missing completely at random (MCAR; χ2[1, N = 268] = 2739.85, p = .96). In the peri-COVID cohorts, completion for the post-COVID-19 related questions ranged from 73 to 78%. Given the descriptive nature of these data, imputation was not used to correct for missingness.

Results

MHP-reported COVID-19 Stress

When asked how much news coverage about the COVID-19 pandemic do you watch, read, or listen to, 17.6% of MHPs reported none, 55.4% of MHPs reported 1–2 h per week, 10.8% of MHPs reported 3–4 h per week, 13.5% of MHPs reported 1–2 h per day, and 2.7% reported 3–4 h per day. MHP-reported impact of COVID-19 varied, but most of the respondents’ scores fell above the mean (i.e., M > 50 on the 100 point scale). Specifically, MHPs reported that COVID-19 affected their work schedule (M = 70.19, SD = 25.36), their ability to provide TF-CBT (M = 64.93, SD = 24.68) and their stress level (M = 59.64, SD = 24.86), Nearly one-third (28.6%) indicated that their stress level was higher than 75 (top quartile) on the 100-point scale. Further, MHP’s reports of concern for their clients’ safety due to COVID-19 averaged at the midpoint of the scale (M = 48.14, SD = 25.57).

Comparison of Pre-COVID and Peri-COVID Cohorts

An independent paired samples t-test was performed and indicated that perceived competence in TF-CBT did not vary between the pre-and peri-COVID cohorts at pre- (t[226] = -1.56, p = .152) or post- (t[183] = -2.46, p = .683) LC assessment. Lastly, there was no significant difference between groups in perceived competence change scores from pre to post, t(154) = 0.803, p = .467. Overall, MHPs who participated in a peri-COVID LC completed fewer training requirements compared to MHPs who completed the LC prior to the COVID-19 pandemic (see Table 1). A Mann-Whitney U test indicated that the number of cases identified between groups was non-significant, U = 7926.500 p = .533. A chi-square test of independence showed a significant association between pre and peri-COVID groups and number of cases completed, X2 (df = 2, N = 269) = 6.58, p = .037. Additionally, there was a significant association between pre and peri-COVID groups and number of consultation calls completed, X2(df = 2, N = 269) = 6.34, p = .042. These results suggest that although MHPs across cohorts were able to identify the same number of TF-CBT cases, MHPs that participated in a peri-COVID LC completed their identified cases and participated in consultation calls less often compared to MHPs participating in a pre-COVID LC. Further, a Mann-Whitney U test indicated that there was a significant difference in the percentage of weekly metrics completed between pre and peri-COVID groups, U = 6141.00, p = .000. This finding indicates that MHPs in the pre-COVID cohort completed more of weekly metrics (e.g., assessments of treatment progress and barriers and facilitators) compared to MHPs in the peri-COVID group. Lastly, despite significant differences across specific individual training requirements, a chi-square test of independence showed that there was no significant association between pre-COVID and peri-COVID cohorts and the percentage of MHPs that rostered, X2 (df = 1, N = 269) = 1.23, p = .171.

Table 1 MHP LC training requirements
Full size table

Study II. Qualitative Data

Participants

Participants (N = 15) included masters’ level MHPs across a Southeastern state, who participated in one of two LC cohorts during the COVID-19 pandemic (i.e., peri-COVID). MHPs were on average, 40 years old (SD = 10.58), 93.3% female, and 60% black. On average, MHPs had a little over 2 years of experience providing TF-CBT (M = 2.37; SD = 1.86) and a little over 6 years of experience as a MHP (M = 6.33; SD = 3.83). Participating MHPs were in cohort 4 (n = 9) which occurred in 2020 and cohort 5 (n = 6) which occurred in 2021.

Procedures

The first and second authors collaborated with the senior author in the development of interview questions. Questions were designed to expand upon quantitative data and further examine the impact of COVID-19 on the MHPs, their abilities to complete training, and to deliver TF-CBT. Following the formalization of the semi-structured interview, the first author piloted the interview with two MHPs experienced in delivering TF-CBT via telehealth during the pandemic. Of note, these two MHPs were not participants of these Learning Collaboratives.

The first author sent out an email to all MHPs that participated in the 2020 or 2021 LCs about the opportunity to participate in qualitative interviews. All MHPs were offered a $15 Amazon gift card as compensation for their time. Fifteen MHPs agreed to participate. The first author conducted semi-structured interviews with all 15 MHPs. Interviews were audio-recorded using Zoom, as well as a secondary recording device for back-up. Interviews used open-ended questions followed by probes to generate conversation, as recommended by Creswell et al. (2011).

Audio files from conducted interviews were sent to Ubiquos for transcription. Interviews were transcribed into Microsoft Word documents and uploaded into NVivo 11.1 (2015), the qualitative data software that was used for analyses. All identifying information was redacted during the transcription process.

Data Analysis

Qualitative data analysis was conducted by clinical psychologists (first, second, and last author) with expertise in qualitative methods and consisted of a qualitative deductive content analysis with a priori domains determined, while also allowing for the development of inductive categories that emerged through coding (Elo & Kynagas, 2008). This method is utilized to test categories via identification of themes/patterns within the qualitative data. Specifically, a three-step approach was utilized, in which each participant’s responses (i.e., raw data) were carefully examined line-by-line to develop a comprehensive codebook to capture all possible themes emerging from the data. Themes were refined, merged, and/or subdivided into sub-themes via collaborative discussion in multiple in-person meetings until a comprehensive codebook was developed. The codebook was then used by two independent coders (i.e., first and second authors) to code and analyze each participant’s responses to the discussion questions (Elo & Kynagas, 2008; DeCuir-Gunby, Marshal, & McCulloch, 2012). Coders were able to apply more than one code to participant responses if applicable. A subset of interview transcript (i.e., 30%) were coded to ensure interrater reliability. The interrater reliability for the double-coded interview transcripts was 88% and ranged from 86 to 91%. Inter-rater discrepancies were discussed and resolved by the two independent coders, in consultation with the senior (last) author. Demographics and background variables were analyzed using SPSS 28 (IBM Corp, 2021).

Results

For the qualitative analyses, five overarching themes, each with their own sub-themes, emerged from the participants’ answers to the semi-structured interview questions: (1) Challenges in completing training requirements (in general and related to the COVID pandemic; (2) Challenges in delivering TF-CBT (in general and related to the COVID pandemic); (3) Strengths and challenges of telehealth delivery of TF-CBT; (4) Resources to support completion of training requirements; and (5) Strategies to support TF-CBT delivery. See Table 2 for percentages related to themes and subthemes.

Table 2 Percentages of themes
Full size table

Challenges in Completing Training Requirements

Throughout the interviews, most participants discussed challenges in completing the training requirements, with responses highlighting both general challenges and those specific to the COVID-19 pandemic. The general challenges in completing training requirements included identifying appropriate TF-CBT cases and delivering the parenting component.

All participants discussed challenges to meeting the training requirements that were specifically related to the COVID-19 pandemic, including problems completing cases due to the transition to telehealth, challenges with training virtually, additional stress related to the virus and pandemic, and requirements taking additional time due to the pandemic. For example, one participant stated, “When COVID happened and the whole world shut down, I was primarily seeing kids in school. So everything shut down. We started doing telehealth and the kids were already starting their narratives it was really hard for them to engage in telehealth sessions.” Regarding the training itself, participants stated, “You had to like be fully engaged and still try to maneuver around what you needed to do at home and with your own kids. That was kind of difficult and stressing;” and “It was hard to participate fully in the discussion.” Most participants discussed additional stress related to the virus and pandemic. For example, “[The pandemic] really made those crisis-of-the-week a lot bigger, and a lot of the kids were pretty much fixated on not being able to see family. It took longer to go back and reteach a lot of coping strategies with this brand-new problem, especially for the ones that were wrapping up. Then they no longer had access to their support, so it put a damper on some things.”

Challenges in Delivering TF-CBT

Most participants reported challenges in delivering TF-CBT, with some challenges being more general and some related to the COVID-19 pandemic. General challenges included engaging caregivers, placement changes and stability, lack of acknowledging trauma or need for treatment by the client, and complex trauma or cases.

All participants discussed challenges to the delivery of TF-CBT that were specifically related to COVID-19, including that the client or provider had COVID-19 or concerns about COVID-19, experienced shutdowns or quarantine due to COVID-19, and other pandemic-related challenges. For example, one participant stated, “We had people who were getting sick. We had some people who work in the medical field who just were not available to bring kids;” while another participant mentioned, “Having to make sure that you’re kept safe because of the patients when you did see them it’s like oh, that fear of if you are ok. Have they been tested? Are they around people?” A participant that discussed shutdowns or quarantines said, “There were times when students were getting quarantined. Typically, TF-CBT should be at least weekly, and there were times where we would have to go two weeks without, which meant that follow-up session we had a lot more to cover and felt less effective.” Finally, several other COVID-related challenges were reported, including “We were dealing with trauma anyway, but then, to some degree, I guess you could consider COVID traumatic, because a lot of them dealt with either some financial issues or death of relatives;” and “We were given specific days that we could come in so, we had to try to juggle that schedule, and all of that is from COVID. Everybody had to have a mask when they came in the building. We had to clean the office after each visit. So, it just entailed a lot to try to deliver services period.”

Telehealth Delivery of TF-CBT

All participants reported using telehealth during the onset of the pandemic, and 40% reported current telehealth use due to the pandemic. Participants discussed challenges to telehealth delivery of TF-CBT, including virtual delivery being less personal, engagement concerns, environmental challenges, technology problems, difficulty completing virtual treatment with younger children, and trauma narrative-related challenges. For example, one participant stated, “My biggest issue, I really believe it was just because we didn’t have that interaction, personal, you know, face-to-face, up close and personal interaction.” Regarding engagement challenges, one participant reported, “Sometimes it would surprise me that a lot of kids, they would cover their camera or they wouldn’t be seen;” while another participant mentioned, “Probably the most difficult part is you don’t have cards or toys or kinetic sand or playdough or anything like that. So you can’t distract them or help engage.”

Some participants describing environmental challenges stated, “A lot of clients, especially older ones, didn’t feel comfortable because they were at home. And so they might not want to say something because somebody might be listening…” and “I had a couple of cases where the abuse did happen right in the home. So doing a trauma narrative and therapy in the same space was difficult.” Further, a few participants discussed specific challenges with completing the trauma narrative (i.e., gradual exposure) component of TF-CBT. For example, “When I do TF-CBT with kids, I would have this huge role of paper where we would do the trauma narrative in this kind of storybook pictures and words and can’t do that over the computer.” Regarding technology problems, one participant reported “Not having good enough reception. You might drop a call or the family doesn’t have a good enough or well enough service, or even just they don’t have the internet.” Several participants also mentioned difficulties with younger kids, such as “I found it really hard with specifically younger kids. It was more difficult for them attention-wise and they’re at home and they have distractions.”

Most of the participants also discussed strengths of delivering TF-CBT via telehealth, including addressing access barriers and increased comfort with technology. Specifically, a participant stated, “I live in a very rural area so transportation is a huge issue for our clients. Some clients have an issue getting to our office or getting to appointments and their parents are working or don’t have a car, we can work with them on that and that’s been really, really helpful.” Some participants mentioned that patients feel more comfortable discussing trauma via technology rather than in-person, “Especially with older kids, I think sometimes they felt more comfortable using the technology because they’re so used to it and it’s a way to be more open and freer to talk about certain things.”

Resources to Support Completion of Training Requirements

Participants discussed several resources to support the completion of training requirements during COVID-19, including training team consultation calls, training team support outside of calls, in-house supervision and peer consultation, and other training-related resources. Specifically, participants shared that ongoing support while managing increased stress resulting from the pandemic was helpful. For example, one participant said “I guess the overall support was helpful knowing that you’re not alone and that you’re not the only one going through these difficult, these obstacles of everything. So that gave some, a little comfort level to get through. And that was from all agencies and the training.” Another participant stated, “We were meeting twice a month. Being able to hear everybody’s issues that they were dealing with and getting feedback from them. Those were the biggest helpful things that I received.” A participant mentioning in-house supervision and peer consultation described, “…I had a lot of experienced peers around me who had done TF-CBT training at some point. So it was really helpful in having their support and tips.” Several participants also mentioned that the extended time allowed for the completion of training requirements was helpful.

Strategies to Support TF-CBT Delivery

All participants discussed strategies that assisted in the delivery of TF-CBT, including resources to assist with telehealth delivery, lessons learned from trainers and facilitators, consultation calls, telehealth-specific trainings, and in-house support and consultation from colleagues. Some resources discussed by participants included, “Sharing ideas like activities that you could screen share or games or rapport building things, different ways to do the narrative with kids virtually. That was really cool like brainstorming and hearing what other people had done that was helpful for them.” Another participant said, “Links that were shared were helpful. They gave great ideas on how to share with our parents and give them information for the parents to have at home and some things you could share with the parents.” In discussing consultation calls, one participant noted, “During our monthly check-ins, it was helpful to have the trainer really give some insight on things and hearing others how they maneuvered and did certain things and you taking those and try to implement it into your session to see what works and doesn’t work.” Participants also discussed trainings that were beneficial. Regarding telehealth-specific trainings, a participant reported, “The trainers did a training with us very early on as we had to transition to doing tele-visits with our clients that was very beneficial.”

Discussion

Rising mental health concerns following the onset of the COVID-19 pandemic have increased the need for accessible and evidence-based mental health treatments. While recent research indicates that the multi-component LC training/implementation model may achieve the most effective outcomes (Hanson et al., 2019; Herschell et al., 2015), there is limited research on how pandemic-related changes and stressors impact MHPs and their abilities to participate in training or deliver EBTs. The current study contributes to the literature by identifying MHP-reported COVID-related stress and examining differences in the completion of training requirements between MHP cohorts that completed the LC before and during the pandemic. MHP perceptions of barriers and facilitators to TF-CBT delivery, training requirement completion, and telehealth use were also explored.

As anticipated, findings showed that MHPs that participated in an LC during the COVID-19 pandemic completed fewer cases and fewer consultation calls compared to MHPs that completed the LC prior to the pandemic. These results are unsurprising based on MHPs’ discussion of difficulties completing cases following the transition to telehealth. Although prior research is limited, results are consistent with other studies suggesting that MHPs’ caseloads decreased following the onset of COVID-19 (Probst et al., 2020), which may be related to reservations against telehealth (Schuster et al., 2018), or as noted in our qualitative interviews, it is also possible that the lower case completion rate stemmed from challenges in engaging youth and families during the pandemic.

Contrary to expectations, there were no significant differences in the percentage of weekly metrics completed or percentage of MHPs that met training requirements to roster. Additionally, there were no significant differences in perceived competence in delivering TF-CBT between MHPs that completed the LC prior to and during the COVID-19 pandemic. This is inconsistent with previous literature that indicates MHPs report higher levels of self-doubt when providing treatment via telehealth compared to in-person (Nissen-Lie et al., 2013; Odyniec et al., 2019). However, these results are promising in that they suggest that both the in-person (pre-COVID-19) and virtual (peri-COVID) learning sessions may be equally effective in providing MHPs with the necessary skills and resources to feel competent in TF-CBT delivery. Additionally, despite MHPs completing fewer calls and cases, MHPs still met the training requirements at similar rates pre and peri-COVID. Given the potential cost savings of virtual trainings (i.e., eliminates travel expenses, space rental), future research is needed to determine whether virtual trainings do in fact yield similar outcomes to in-person. Such studies should focus on immediate post training outcomes, such as EBP adoption and fidelity, along with long-term sustainment, and also elucidate factors at the provider and agency levels that may influence the benefits and ultimate selection of one training format over another.

MHPs described several challenges to completing LC training requirements both generally and specific to the COVID-19 pandemic. The most shared challenges were difficulty completing cases following the transition to telehealth and difficulty managing stress related to the virus and pandemic. These findings are consistent with the limited research that has emerged on MHPs working during the pandemic. For instance, numerous studies conducted since the onset of the pandemic have found that MHPs are reporting increased levels of stress and burnout related to COVID-19 (Aafjes-van Doorn et al., 2020; Ledesma & Fernandez, 2021; Probst et al., 2020). Further, as noted previously, at least one other study has found that MHPs’ number of clients decreased following the onset of the COVID-19 pandemic. These findings highlight the importance of including strategies in trainings to address factors that may attenuate positive outcomes. For example, inclusion of topics related to self-care/managing secondary trauma can be included as part of Learning Sessions and on consultation calls to address stress and burn-out (CITATION). Additionally, prior research has highlighted the importance of including senior leaders and supervisors as part of an LC, given their critical role in supporting and sustaining EBPs within an organization. (e.g., Aarons & Sommerfeld, 2012; Aarons, Ehrhart & Farahnak, 2014).

MHPs also discussed several challenges specific to the delivery of TF-CBT. All MHPs in the current study reported transitioning to telehealth during the onset of the pandemic. Overall, MHPs reported that the biggest barrier to TF-CBT delivery was engaging caregivers in treatment. This finding is consistent with the extant literature on TF-CBT delivery, which indicates that difficulty engaging caregivers in treatment is a common challenge (Ascienzo et al., 2020; Cohen et al., 2012). This is concerning given the substantial research that links caregiver involvement in treatment to positive outcomes for both the caregiver and the child (Deblinger et al., 2001; Domhardt et al., 2015). Caregiver involvement is likely of particular importance during a time of heightened stress, such as the COVID-19 pandemic. Thus, future research should continue to identify effective strategies to increase caregiver buy-in and participation. When asked about challenges specific to COVID-19, the most common challenge was the concern that the provider or client may be carrying the virus. This concern was particularly distressing for MHPs that still saw a portion of their clients in person and faced the risk of transmission. However, MHPs also expressed concern about the COVID-19 pandemic for clients seen via telehealth due to the anxiety of possibly navigating an extended hiatus in treatment and/or their clients becoming seriously ill. Future research should focus on identifying individual and contextual factors that may influence these anxieties (e.g., the presence or absence of health and safety protocols).

MHPs also identified several factors that influenced their ability to deliver TF-CBT via telehealth. The most common concern was that MHPs perceived telehealth as being less personal compared to in-person therapy. Research on MHP perceptions toward telehealth prior to the pandemic found that challenges regarding personal connectedness with clients via telehealth have been a major concern (Connolly et al., 2020; Roesler, 2017). The current study highlights that despite growing empirical evidence that the therapeutic alliance in telehealth is just as strong as in face-to-face settings (see review by Norwood et al., 2018), especially when rated by clients (Ruwaard et al., 2009), MHPs still feel challenged by the lack of connectedness associated with treatment delivered via telehealth. Future research should examine this further and specifically investigate whether this differs based upon the type of treatment being delivered. Relatedly, MHPs mentioned a wide range of other telehealth-related engagement challenges, including youth not wanting to be on camera or getting distracted during session. Unsurprisingly, technology was another commonly mentioned concern, including issues with the internet and client access to an appropriate device (e.g., laptop or computer). These challenges are consistent with those identified in a recent study conducted with a sample of over 1,200 MHPs providing mental health treatment during the pandemic (Békés et al., 2021). Future research should continue to focus on identifying effective strategies for building client engagement and the therapeutic alliance via telehealth.

In addition to the many identified challenges, MHPs also described several resources and strategies that supported the completion of LC training requirements and the delivery of TF-CBT. The most frequently identified supports were resources that assisted with telehealth delivery, including providing information to MHPs about online materials/games to engage youth during Learning Sessions and on consultation calls; as well as encouraging participants to share ideas with one another throughout the training. MHPs also identified the importance of in-house supervision, which has been highlighted in prior research as a critical strategy to promote successful uptake of an EBP (e.g., Meza, Alrasheed, R., Pullmann, & Dorsey, S, 2023). Consistent with prior research (Stewart et al., 2020; Villalobos et al., 2023), findings from the present study suggest that telehealth delivery of TF-CBT is feasible, and further, that the strategies included as part of the LC model, such as ample opportunities for peer and expert consultation, coaching via didactic-based learning sessions, and routine consultation calls can support MHPs as they learn to deliver complex EBPs. Future research should aim to empirically examine how these supports may buffer against an array of other stressors, beyond the COVID pandemic.

Despite noteworthy findings, there were several limitations to the current study. First, findings may not be generalizable given the fact that MHPs elected to participate in the training, and all were participating in an LC conducted within one state. Research would benefit from a larger and more inclusive sample of MHPs as well as multimodal measurements (i.e., observational, multi-informant, etc.) of outcome variables (e.g., therapist competence, fidelity, client outcomes). The use of multidimensional assessments of therapist and client outcomes may provide valuable information regarding differences in virtual and in-person trainings. Second, pre-COVID cohorts were those who completed the LC during 2017, 2018 and 2019, and peri-COVID cohorts as those participating in the 2020 and 2021 LCs. Given the onset of COVID and variable timing for its impact across the state, we were unable to determine that cohort differences were exclusively related to the onset of COVID-19; there have been several other societal changes and stressors over the last 5 years (e.g., the presidential election, uprise in racial justice movements) that may have contributed to challenges noted by participants. Third, the current study was a training initiative, not a research study, so there was no random assignment or comparison condition.

The current study examined how MHP participation in an LC was impacted by the current COVID-19 pandemic. We found that MHPs participating in LCs during the pandemic completed fewer cases and fewer consultation calls compared to before the pandemic. However, no differences were found between cohorts in the percentage of MHPs that completed all training requirements and MHP-reported competence in TF-CBT. Qualitatively, MHPs described several barriers and facilitators to the completion of training requirements and treatment delivery, which expanded upon the quantitative findings. Collectively, the study highlights the value of the LC training/implementation model in assisting MHPs to navigate unexpected challenges and concerns that may arise while delivering EBTs.