Introduction

Though HIV interventions including treatment as prevention [1, 2] and pre-exposure prophylaxis (PrEP) [3,4,5] have demonstrated individual-level effectiveness when adherence to these biomedical interventions is met, evidence for effectiveness in reducing HIV incidence at the population level remains elusive, with large studies demonstrating mixed results [6, 7]. The missing link between effective interventions and population-level impact likely lies in optimal implementation and scale-up. Policy recommendations and guidelines are often guided by results from internally valid randomized clinical trials (RCTs) run by research teams, which frequently encounter implementation challenges when translated to typical practice settings that impede scale-up of evidence-based interventions [8].

The translation of evidence to impact includes clearly specifying effective implementation strategies and delineating actors, actions, and targets beyond patient-level behaviors—including providers’ actions, health systems, delivery mechanisms, and timing and intensity of supports [9]. Implementation research inherently challenges predominant evidence hierarchy assumptions in search of generalizable knowledge, prioritizing real-world data and/or pragmatic designs which emphasize congruence between study and target populations. Further, recognition is growing that adapting to context is critical for tailoring effective implementation strategies [10,11,12]. Accordingly, the last few years have witnessed an increase in the use and prioritization of implementation research methods in HIV-related research [10, 13, 14]. Indeed, implementation science (IS), a multidisciplinary approach which seeks generalizable knowledge through assessing and addressing gaps in evidence and practice across multiple stakeholder levels, is listed as a National Institute of Health (NIH) high-priority topic area for AIDS-designated research funds [13, 15, 16].

Despite expansions in funded HIV-related implementation research grants and publications in recent years, there is a need to advance the rigor of IS through shared understanding and common language [13]. Opportunities exist to increase the use of IS frameworks, implementation outcomes, and pragmatic study designs, and to expand IS work further along the translational pathway given the recent preponderance of studies focused on implementation preparation [17,18,19]. Improving rigor in HIV implementation research requires expanding the workforce through IS training and mentorship, alongside practice-based partnerships [20]. Thus, our objective is to review current training opportunities for HIV-related researchers, particularly those in the USA, and assess performance metrics and lessons learned from a recent HIV-related implementation research training initiative.

Training Demand Exceeds the Supply

To meet the growing demand for HIV-related IS work, an expanded workforce of implementation scientists is required [21]. Training opportunities in IS exist—spanning from individual courses within degree seeking programs or multi-day institutes, a growing array of doctoral or master’s levels training programs specifically in implementation research, and dissemination and implementation (D&I) institutes targeting health professionals [21,22,23,24,25,26,27]. The US Training Institute for Dissemination and Implementation Research in Health (TIDIRH) was run from 2011 to 2020 by the NIH, first as an on-site program and then as a hybrid online/on-site program for post-doctoral investigators and was successful in its mission of building a cadre of D&I researchers [22, 28]. The Implementation Research Institute (IRI) has also trained multiple cohorts of implementation researchers focused on mental health and substance use and has reported impact in terms of publication and grant performance of trainees [29, 30]. Available spots within these training opportunities have been limited and demand currently outstrips supply [22, 29]. The TIDIRH and IRI models were not offered in 2020, although TIDIRH material has been integrated into the Mentored Training for Dissemination and Implementation Research in Cancer (MT-DIRC) program applied to cancer training, and a competing renewal was submitted by the IRI to offer it again in the future. The MT-DIRC program includes online freely available modules, as well as a more space limited, mentored 2-year experience that has demonstrated success [31,32,33,34]. Key features of these successful models and others, including the King’s College Masterclass in London, have been the cross-institutional collaboration of faculty mentors [35]. Increasingly, however, programs have become narrower in focus, such as the MT-DIRC program, chronic disease preventative training at the Institute for Implementation Science at Washington University in St. Louis, or focused on specific populations, such as care related to US veterans [36, 37]. Other non-degree-based attempts to expand opportunities have focused on local initiatives within geographic or institution specific public health and academic communities, which could be further expanded through greater integration of IS training into Clinical and Translational Science Awards (CTSAs) [38,39,40,41,42,43]. There currently are few available IS training opportunities for HIV researchers, and until recently, none was focused exclusively in this area.

HIV-specific training opportunities in IS have typically been informal or accessed through institutional or individual awards. For example, the NIH is currently funding 11 HIV-related institutional training programs (T32s) that include varying degrees of IS elements, five individual F awards related to HIV and IS, 12 research education awards (R25 grants) that have HIV and IS elements, and 65 individual research career development awards (K) that include both HIV and implementation research [44]. International training opportunities for researchers in low- and middle-income countries are also offered through 26 NIH-funded D43 awards which include some training in HIV and IS, 13 of which IS is the primary training focus [44]. The Centers for AIDS Research (CFAR) which focus on HIV-related institutional scientific development and junior faculty career development has provided another source of IS opportunities. Of the 17 CFARs nationwide in the USA, three have scientific cores in IS, two have IS-focused scientific working groups, and an additional two have inter-disciplinary research or interest groups. A 1-day IS mini-course for researchers conducting HIV and sexually transmitted infections research is offered by the University of Washington (UW) CFAR, alongside UW’s IS short course [21]. At least four other NIH-funded HIV research centers have IS cores or methods cores with IS components, and the Center for Prevention Implementation Methodology focuses on the development of IS methodology for application to HIV [45]. Additionally, the inter-CFAR working group [46], established in 2018, brings together HIV and IS experts across institutions and works collaboratively with the NIH; the Implementation Science Coordination, Consultation, and Collaboration Initiative (ISC3I) [47]; and IS regional consultation Hubs to support improved rigor of implementation research in the USA through webinars, coaching, and resource collation [48, 49]. However, many of the IS training and consultation services provided by ISC3I, and the new IS Hubs are largely limited to grantees of the CFAR/ARC Ending the HIV Epidemic Supplement Awards, limiting the reach of the potential health impacts [50]. Overall, there remains a gap between training needs and available training opportunities.

Creating a Training Program and Agenda for HIV Researchers in Implementation Research

In response to the underlying training gap, the inter-CFAR IS working group, with NIH support through the Ending the HIV Epidemic Initiative [50], launched a training fellowship in IS for early-stage investigators (ESIs) conducting HIV-related research in 2019 [51]. Contextualizing IS within broader scientific training and supporting simplification of concepts has been noted as an effective way to help investigators new to the D&I space to understand the jargon [52]. Embedding IS language within investigators’ content-specific area, HIV, further enhances relatability of the concepts and simplifies for learners the comparisons and contrasts of more traditional clinical effectiveness research with implementation research and designs. Supporting training of core methods and measures in IS, alongside mentorship in engaging in practice partnerships to understand context and identifying relevance in research questions, is also critical [20]. Thus, the goals of the new training program were to fill an IS training gap among emergent HIV-focused investigators, maintain and expand the mentoring network of HIV researchers conducting IS research, and improve the quantity and quality of IS-related NIH grants submitted.

The Approach

The approach of the inter-CFAR IS Fellowship for ESIs builds upon several of the lessons learned through the IS training programs described above, as well as the lessons learned through our experience during the first year of the fellowship. Four complementary and intersecting components are encompassed in our training program, including didactic learning, paired mentoring, networking, and grant-writing (Fig. 1).

Fig. 1
figure 1

HIV-related Implementation Science Fellowship training components

Full size image

Similar to the evolution of the TIDIRH program, our HIV-focused IS training utilized an online, didactic component, accompanied by a 2-day in-person meeting [28]. In the second year the approach has evolved to include pre-recorded asynchronous modules of the foundational material (as opposed to synchronous modules the first year), but added synchronous discussions of more granular details, reinforced by real-world examples, related to the application of frameworks, methods, and IS practice. These changes are also in alignment with lessons learned through the Canadian knowledge translation training program [53].

Core topic areas covered through the didactic training modules are outlined in Fig. 2. The curriculum was developed based on knowledge of existing training programs, review of introductory IS course syllabi, and faculty input drawing from training gaps common across the HIV field. HIV researchers are commonly exposed to behavioral science, appreciating the importance of behavior change theories and the role of social and external forces on uptake and adherence to evidence-based interventions. This is an important foundation to build upon. However, whereas traditional HIV behavioral science research largely focuses on efforts to influence individual (patient) behavior, IS emphasizes the importance of adopting strategies to address service delivery barriers across patient, provider, and health systems levels. Further, the IS approach is less interested in controlling for contextual factors but is explicitly interested in evaluating the role of context to moderate effects. This reframing of the question and the approach from traditional HIV scientific training underlies elements emphasized in the curriculum. Thus, our intention with the curriculum was to ensure a common foundational training in the building blocks of IS, utilizing synchronous sessions and mentoring to expand upon this foundation. The last two modules (the Implementation Research Logic Model [54] and Technology and IS), as well as adjustments to asynchronous versus synchronous components were added to the second year curricula based on faculty and Fellow feedback from year one implementation.

Fig. 2
figure 2

Core topic areas covered through the didactic training modules

Full size image

Akin to lessons learned in other institutes, our approach includes an emphasis on mentoring [33, 55]. The fellowship pairs one faculty mentor to two Fellows, with the intention that mentors provide specific feedback on the development of grant ideas and components, while also expanding the IS networks of trainees. The mentoring team is comprised of 12 faculty who have experience and funded grants in the HIV IS space. Increased structure in terms of mentoring guidance, including pre-specified expectations of monthly meetings over an expanded number of fellowship months (6-month duration) prior to the in-person meeting, are lessons learned based on the initial year of feedback and reinforce experiences of other D&I programs [56, 57]. In the second year, we will also leverage engagement from fellowship alumni for additional coaching and interaction with the Fellows.

Grant-writing is a priority area for this program. Grant considerations are integrated into the didactic lectures; grant components are the key focus of assignments which are sequenced in relation to the lecture schedule and are the chief focus of mentor-Fellow engagement, as well as emphasis of the 2-day culminating meeting. Given this emphasis, Fellows are only accepted into the program if they will be submitting a grant; based on the experience of the first year, specific aims were requested and reviewed as part of the trainee application process in the second year.

Finally, networking is a core area of the program which is intended to support ESIs in their development while also generating a community of practice among investigators in the HIV IS space. A 2-day meeting toward the end of the fellowship allows for Fellows to build networks with one another and their mentors and fosters learning by hearing and being a participant in the presentation-based grant feedback process. Further, opportunities to foster development of collaborative networks are promoted through synchronous sessions and group mentoring meetings. Additionally, faculty mentors have been intentionally selected across an array of CFAR institutions, creating opportunities for expanded professional networks and the creation of a broader community of practice for both the mentors as well as the Fellow trainees. This model pulling from resources across institutions builds on the success of models such as TIDIRH, IRI, and the master’s class approach [35].

Our theory of change, including the underlying context and assumptions, is shown in Fig. 3. Using this model, minimal financial inputs have been leveraged to create multiple outputs and a proximal outcome of an IS-focused grant submission. Our long-term goal is to facilitate increased high-quality grant submissions and evidence generation in the HIV-related IS space. We endeavor to develop a community of HIV researchers to further progress the field and build cross-institutional collaborations.

Fig. 3
figure 3

Implementation Science Fellowship theory of change

Full size image

Selection

ESIs have been eligible to apply if they were nearing or have completed doctoral training, have not yet received an NIH R01 grant or equivalent, and have not been promoted to the associate professor level. Recruitment focuses on faculty and post-doctoral fellows from CFAR institutes, but selection is not limited to those individuals. A demonstrated interest in IS is considered, alongside the incorporation of IS concepts into the proposed grant to be developed. Each application is reviewed by two faculty reviewers, and if there is a substantive difference in scores, assessment by a third reviewer is sought. During the first year of the program, 53 eligible applicants applied for 27 spots (51% acceptance); the final cohort included Fellows across 14 CFARs or institutions. The application requirements were increased during the second year (inclusion of specific aims), and the slots reduced to ensure a 2:1 ratio of mentors to Fellows; a total of 55 applications were reviewed and 24 accepted across 13 CFARS or institutions, decreasing the acceptance rate to 44%.

Nearly three-quarters of selected applicants are cisgender women in year one, decreasing to 54% in year 2 (Table 1). The majority of applicants held PhDs, with just over one-quarter having completed medical degrees. Public health (e.g., epidemiology, statistics, health policy, health behavior) training was the most common training discipline among selected Fellows, followed by medicine and then mental health/psychology. The preponderance of proposed grants were new submissions; however, grant mechanisms being targeted (e.g., R01, R21, R34, K awards) were heterogenous. On average, selected ESIs had completed 4 years of training since the completion of their terminal degrees. At the time of application, Fellows had published a median of 22 papers [IQR 13–42] and had led as the principal investigator or program director a median of two grants [IQR 1–4] in the inaugural year, and had published a median of 19 papers [IQR 13–28.5] and received funding for three grants [IQR 1–4.5] in the second cohort.

Table 1 Characteristics of selected applicants for the Implementation Science Fellowship
Full size table

The majority of applicants proposed internationally based grants in year one; however, this balance shifted and was predominantly domestically (USA) focused during year 2, perhaps reflecting the emphasis on domestic research under the continued Ending the HIV Epidemic. While there was heterogeneity in the HIV-related focus of the grants within and across years, the largest number of grants was focused on PrEP in both years; in terms of treatment, emphasis shifted from ART initiation in year one to adherence and viral suppression in year 2. A focus on co-morbidities was common in both application cycles.

Performance

Performance results were informed by the first year of the program, with the second year having been recently launched. Evaluations were completed by the 24/26 Fellows attending the culminating in-person meeting at the end of the first training year. All Fellows reported gaining new skills or understanding of IS, with the majority (71%) reporting gaining many new skills (Table 2). On average, Fellows ranked the median value of the entire fellowship experience (1 lowest to 10 highest) as a 9 [IQR: 8–9, range 7–10], and all reported that they would recommend the fellowship to a peer colleague. The online didactic sessions were rated as highly useful (92%), though the majority would have preferred more time or sessions (82%). In particular, Fellows commonly identified IS frameworks, pragmatic study designs, more applied examples, and IS presentation skills as areas for which additional or more in-depth training was desirable. These suggestions, alongside recommendations to use synchronous time to get into greater depth of topics, were echoed by the mentorship faculty and incorporated into a refined approach to the second year training curricula. Though nearly all Fellows engaged with their mentors prior to the culminating meeting, the need for more structure was commonly expressed. Fellows met a median of two times before the in-person meeting but reported desiring an increased number of pre-meeting engagements. Further, based on Fellow and mentor feedback, the duration of the fellowship has been increased from 4 to 6 months. Overall, Fellows were satisfied with the mentorship received and the grant-focused nature of the fellowship, as well as the networking opportunities. Fellows were largely satisfied with the quality of the grant feedback and perceived it to be useful, though improvements in structuring to this regard could also be made.

Table 2 Fellows’ perceptions of the implementation science (IS) training program (n = 24)
Full size table

Overall, of the 27 Fellows initially selected, 22 (81%) submitted their grants to the NIH or other funders by 12-month post-Fellowship, of which at 1 year out a total of 13 grants have been funded among 10 of the 22 Fellows that had submitted (45%), or 37% overall.

Using the Kirkpatrick model as a guide for framing program success and lessons learned, evaluation of the training performed well [58]. Firstly, Fellows reported satisfaction with value and utility of the training and would recommend it to others. Secondly, new skills were self-reported by the Fellows, which was reflected in the evolution of the quality of assignments over time. Next, nearly three-quarters demonstrated expected behavior—grant submission. Finally, substantial impact in terms of funded IS grants was achieved in less than a year from Fellowship completion; results we anticipate will increase in future years among this cohort. Overall, the return on expectations has been substantial.

Identifying Training Priorities Moving Forward

The mentorship team was surveyed to identify their perspectives of the fellowship and IS training priorities for ESIs based on mentorship experience during the first fellowship year, as well as among other trainees. Mentors recommended a median of 10.5 didactic lectures [IQR: 9–12] be included going forward, representing an increase from the nine offered in year 1 and in alignment with recommendations from the Fellows. In a ranking exercise, mentors identified framing of implementation research questions as the highest priority area for IS training among ESIs, alongside specification of implementation strategies, IS frameworks, IR outcomes, mechanisms and study design which were all identified as priority areas (Fig. 4). In their evaluations, Fellows identified frameworks, implementation strategies, and alignment of study design to research questions as areas in which they desired more time and attention, also noting the need for more applied examples and discussion, as well as analytic guidance. Framing of research questions, identified as the top priority among mentors, was not perceived by Fellows as an area in which further training was needed; however, it is possible that this reflects the strong focus and integration of this focus across sessions.

Fig. 4
figure 4

Mentors’ perceptions of training priorities in implementation science for early-stage HIV investigators

Full size image

In terms of perceived benefits of the training program, mentors ranked the didactic component as having the greatest benefit, followed by mentoring, grant-writing support, and finally the facilitation of networking (Fig. 5). Seventy-five percent of mentors noted limited bandwidth among the faculty mentors as a key challenge to the program, alongside the need for more structure to the mentorship guidance. Qualitatively, Fellows reinforced the benefit of the didactic training components, though there was a mix between Fellows who wanted more sessions and readings and those who wanted a more curated mix.

I really got a lot out of the webinars and the readings. Could have had more!

There is a lot of reading and for those who work clinically (and probably those who don't!). It was a challenge to keep up. I really fell off during my weeks on service. Would be useful to provide a "priority list" of most relevant reading material.

Fig. 5
figure 5

Mentors’ rankings of perceived benefits of the training components

Full size image

Time constraints raised by mentors in the evaluation were also noted by Fellows. Similarly, the desire for more structured mentorship guidelines and mentor interaction was commonly expressed by Fellows, though Fellows were empathetic that mentorship was one of many mentor responsibilities and was unfunded.

This overall was an excellent training. It would be great to increase the support for the mentors to facilitate additional engagement opportunities.

More ongoing communication with mentors and other fellows; the online sessions could be more interactive.

Finally, though networking ranked as the least important perceived benefit by mentors, Fellows frequently expressed desire for more prolonged engagement, both to reinforce networking with other Fellows and mentors, as well as further time to support grant development and growth in the IS space. The majority of Fellows was interested in playing a mentorship role themselves moving forward.

[I’d recommend] continued connection among fellows after the Baltimore meeting in an asynchronous way, such as discussion forums.

Conclusions

There is a need to demystify the methods and measures used in IS for emerging D&I researchers [52], expanding the IS knowledge base, increasing researchers’ abilities to link concepts to previous training, and providing mentorship from IS experts. A shared language and IS lens compared to a more traditional epidemiological/clinical lens have not fully penetrated HIV IS research but increasingly will be important to advance generalizable knowledge. Expanding the general knowledge of the utility of IS may support more grants focused on the optimal implementation of diagnostic, prevention, and treatment strategies in HIV, without expectation of the assessment of specific clinical or population-level health outcomes. Beyond the modalities of this fellowship, trainees would benefit from in vivo experience with ongoing IS projects in HIV. Our experience piloting and refining an HIV-specific training agenda has incorporated the lessons learned from well-regarded training programs and fills an important gap for ESIs conducting HIV research. Both Fellows and mentors perceived the training program to add value, and the Fellows’ successful submission of grants in this space supports this view. We anticipate that this benefit will continue to unfold in future years, increasing the quality and quantity of HIV-related IS grant submissions and awards.