Introduction

HIV prevention and care peer education interventions have demonstrated effectiveness at changing HIV risk and care behaviors among a variety of at-risk populations in different settings [1,2,3,4]. Peer education interventions are complex because they are intended to change the behavior of both peer educators (through direct participation in the intervention) and people in peers’ networks (through their engagement with their naturally occurring social relationships [5,6,7].

As a result, the reach of the intervention may be widespread as information from the intervention is disseminated to more people than those directly intervened upon by the program.

Since many community organizations have limited resources, evidence-based, low-cost peer educator interventions have great appeal [8]. Peer educators promote norms of risk reduction and disseminate HIV prevention resources, information, and skills. By interacting with a sample of individuals, peer educator interventions have the capacity to change behaviors of hard-to-reach populations. However, little is known about the implementation of this type of intervention in community-based settings. Further, there is limited information available regarding the facilitators and barriers to implementing peer education interventions in community-based settings. One issue encountered by community-based programs is that interventions tend to target populations who have precarious lives due to lack of employment, homelessness, and substance use [9]. This dynamic can make it difficult to recruit and retain individuals into behavior change programs through community-based organizations (CBOs) that may not have the same resources as a randomized controlled trial (RCT) through which programs were initially evaluated.

As agencies begin to implement evidence-based interventions, several multi-level factors arise that facilitate implementation and sustainability, while others conversely hinder the process [10]. Buy-in from staff and leadership across an organization has been widely noted as a facilitator of program implementation [11]. Having one person within the organization who champions a program by supporting and promoting it is key to increasing buy-in among the rest of the team [12, 13]. Pinto et al. found that providers who have prior experience with implementing other evidence-based interventions are more likely to achieve the desired outcomes of a new intervention [14]. Additionally, implementers’ professional knowledge and training is key to successful implementation of an intervention [15]. Cohesion between the goals of the intervention and the needs of organization and target population has also been noted as a facilitator [16].

A major challenge for organizations is funding, which is exacerbated because agencies do not have enough information about the actual costs required to implement the interventions [17]. In an analysis of 34 agencies implementing evidence-based interventions (EBI), investigators examined implementation challenges. Pre-implementation challenges included limited detail of EBI implementation information as well as mismatch between intervention and agency clients; full implementation challenges were retention of clients and staff turnover; and post-implementation challenges included evaluation and funding [18].

The SHIELD intervention is a 6-session intervention designed to train people who have a history of drug use to be peer educators for members of their social networks. SHIELD has demonstrated success in changing sex and drug behaviors that increase HIV risk [19]. The SHIELD intervention was packaged and disseminated through the Centers for Disease Control and Prevention’s (CDC) initiative called the Diffusion of Effective Interventions (DEBI). The goal of the DEBI initiative was to assist community organizations and state and local health departments in identifying effective public health programs and provide local organizations with the tools and resources to facilitate the implementation [20,21,22]. No previous studies have discussed implementation challenges and facilitators for this intervention.

The purpose of this study was to examine facilitators and barriers during implementation of an evidence-based peer education program for people who use drugs. To our knowledge, this is the first study to examine facilitators and barriers in community settings (rather than research settings) of a peer education program for people who use drugs. In addition, we explored the steps organizations took to overcome these challenges throughout the implementation process. We examined facilitators and barriers using the Consolidated Framework for Implementation Research (CFIR). The CFIR has been applied to numerous evidence-based interventions (EBIs), including HIV and substance use evidence-based interventions [23,24,25]. This framework includes five domains of constructs that impact implementation: (1) intervention characteristics, or various features of the intervention; (2) inner setting, or the context within an organization that a program is implemented; (3) outer setting, or the context within which the organizations operate; (4) characteristics of individuals who deliver the interventions or receive them; and (5) implementation process that leads to successful implementation of a program [26]. Like several other studies, we focused on the domains to guide interpretation of the findings [27,28,29].

Methods

This study explored implementation of the Self-Help In Eliminating Life-Threatening Diseases (SHIELD) intervention in agencies who were trained by the CDC to carry out this behavioral intervention. For the purpose of this study, we defined implementation as having completed at least one cycle of the SHIELD Intervention (6 sessions).

Using a set of rigorous standards for study design, outcomes, and implementation, the CDC’s “Prevention Research Synthesis Project” identified SHIELD as a “Best Evidence Intervention” [30]. The intent of this designation is to assist CBOs and health departments to identify effective intervention models for strategic planning of prevention services. Many health departments tie their HIV prevention and care funding to interventions recognized by the CDC. Thus, agencies often have a choice among a select few interventions. The SHIELD intervention was packaged into an implementation manual and widely disseminated through the DEBI program beginning in 2009 [31]. From 2010 to 2014, the CDC offered free regional trainings on this intervention through the HIP program (formerly the DEBI program) in various cities in the US. These trainings were required for agencies funded by the CDC to implement the intervention and agencies who wanted access to the intervention package. While these trainings were free, agencies did incur costs associated with travel and time spent away from the agency. In addition to the training, the CDC contracted with several organizations to provide support and technical assistance before and during implementation to agencies who requested this support.

The study team obtained the list of individuals, representing 34 agencies, who attended a training offered through the CDC. We initially contacted individuals through email and then followed-up with a phone call to describe the purpose of the study. The study team contacted staff from all 34 agencies on the list, which included community-based organizations, harm reduction centers, and drug treatment programs. When we called each agency, we requested to speak to the person who was named on the CDC list. If none of these individuals were still at the agency, we asked to be connected to the individuals who were involved in the SHIELD intervention. Guided by similar studies, we prioritized staff who were facilitators of the intervention or held a leadership role such as Executive Director, Project Manager, etc. [32, 33]. Representatives (i.e., facilitators, supervisors, and other project staff) from eighteen agencies took part in the study. Of the remaining 16 agencies on the list, 5 agencies were no longer in existence and representatives from 11 agencies did not return our repeated attempts or declined participation after repeated outreach.

Based on the definition of implementation, 12 agencies were implementers and 6 agencies were non-implementers. The present study focuses on the 19 respondents from 12 agencies that implemented the intervention. Interviews were conducted with 1–2 staff members with a focus on a group facilitator or administrator, such as a Program Manager or Executive Director. After providing verbal consent, interested individuals took part in a telephone interview with a member of the research team. All interviews were audio-recorded and transcribed by an independent third party. Participants received a $25 gift card as well as a $50 gift card for their agency. The study was reviewed by the Johns Hopkins School of Public Health Institutional Review Board and was deemed to be exempt.

Analysis

The process analysis incorporated data from both Project Directors/Administrators and Facilitators. The process analysis began by compiling all materials associated with each implementing site. Next, two reviewers explored the material for specific implementation facilitators and successes, implementation barriers, and the solutions developed by the agency to overcome these barriers. These components were analyzed based on the five factors that comprise the CFIR: intervention characteristics, inner setting, outer setting, characteristics of individuals, and implementation process. Each facilitator, barrier, and solution were categorized as one level of the CFIR by one reviewer. Two other coders reviewed the CFIR coding and came to a consensus. Process analysis explored intervention specific barriers and facilitators across agencies. Analyses also sought to identify whether implementing agencies encountered challenges specific to working with people who use drugs and the strategies they developed to overcome these barriers. The results of this analysis were synthesized to identify specific types of barriers and the strategies agencies can use to overcome these barriers.

Results

In Table 1, a brief description of the geographic location, type of agency, intervention setting, and population for each of the implementing agencies is presented.

Table 1 Description of agencies that implemented SHIELD
Full size table

Respondents for the current study included staff in leadership positions (i.e., Directors, Supervisors, and Program Managers) and facilitators of the intervention. Respondents were represented the major geographic divisions of the United States and the majority were community-based organizations. As shown in Table 1, there was a diversity of implementation settings and populations who went through the SHIELD intervention. Funding for SHIELD implementation came from a variety of sources: 4 organizations received funding from the CDC, 4 organizations received funding through SAMHA, and five organizations received funding from a variety of sources (i.e., internal funding, local funding, foundations) that did not include federal funding.

Guided by the CFIR, Table 2 outlines facilitators and barriers at the outer, inner, individuals, and intervention level.

Table 2 Facilitators and barriers of SHIELD implementation by consolidated framework for implementation research domains
Full size table

Outer Setting

Facilitators

Availability and quality of the initial training is an important first step in the successful implementation of a new intervention. One organization noted that support from their partners and trainers who provided their initial SHIELD training was important to implementation. Further, several organizations often built collaborative relationships with other organizations that served similar populations and used these relationships to identify potential participant pools and recruit people into SHIELD:

I think just the fact that we have a variety of partners that we work well with and I think the support from the CDC trainers was overall pretty strong. Facilitator, Agency 1

Barriers

Implementation and sustainability of interventions are challenges for many organizations and highly dependent on budgets after the initial funding cycle. Several outer setting barriers were noted that greatly impact implementation and sustainability of the intervention. Several organizations reported that budget factors influenced program implementation by limiting incentives for participants, which ultimately decreased recruitment and retention.

Mostly, the unfortunate sea of budgeting. We started out giving away a lot more and word on the street—this is a small town. So if somebody graduates and they get on the bus and they have this really pretty bag that says SHIELD on it and everybody goes, “Oh! Where did you get that?” And as things went downhill with that, we were unable to give people as much things. So that was a main reason for the decline. Facilitator, Agency 8

An organization noted a reduction to their budget led to insufficient incentives to encourage program participants. This lack of incentives further impeded program enrollment. As noted by one facilitator, they would prefer that clients from the intervention come for the information but in reality, they come for the money.

Well I guess with the—I need more incentives- more incentives I should say to get them to come and know that it’s important enough for them to come for the sessions; that it’s urgent that they should come to take advantage of the free information. Facilitator, Agency 26

As noted previously, many health departments and organizations choose interventions based on the recommendation of the CDC. Federal recommendations change for a variety of reasons; it is not uncommon for an organization to start with one intervention and shift to another due to changes in CDC guidelines [34]. During the time that SHIELD was disseminated, the CDC began to prioritize interventions with only one or two sessions as compared to interventions with multiple group-based sessions. This change in priorities was illustrated by one agency (who was CDC funded) being compelled to choose a different intervention for subsequent years of their funding cycle.

The new strategy requires moving away from group interventions to more individualized interventions, so we were informed that this funding year we would not be using SHIELD. Project Manager, Agency 10

Inner Setting

Facilitators

Intervention staff and facilitators noted organization and staff characteristics that fostered successful implementation of the SHIELD intervention. Several organizations had prior experience implementing complex, multi-session interventions based on an externally developed manual. Implementing organizations developed strong internal infrastructure based on previous DEBI experience and leveraged these resources to implement SHIELD. Strong internal support for intervention facilitators and program staff were noted, including an education department for information sharing and to assist intervention facilitators with increasing their knowledge.

I think we as an agency have a pretty decent history of implementing sort of preexisting interventions that—I think it started 15 years ago or so with Mpowerment. [Mpowerment is an HIV prevention intervention for young gay and bisexual men] I think it was one of the first ones we implemented, and we have a pretty strong prevention base in terms of knowledge. And a lot of kind of internal support. Our education department, for example, is always helpful. Whenever I get a question that I don't immediately have the answer to or need more information on, usually, I can walk over to their cubes and say, "What do you know about this?" Facilitator, Agency 6

In a few organizations, previous experience with these multi-session interventions led to existing infrastructure and partnerships to aid in recruitment and retention of participants. Further, these partnerships allowed organizations to have a respected presence in the community and access to the population.

One thing that helped us is that our agency is already so established in the community. People know our name, and so that helped us when we went out that people had a larger agency to identify us with.” Facilitator, Agency 9

Being accessible is just as important as having access to the population. One organization noted that their location was convenient and accessible, and their services and facility were consistent with the needs of the population. Thus, the content of the intervention was less important than where the agency is located and what the agency had to offer.

Summers are extremely hot, extremely, extremely hot. We offer SHIELD in our building. It's air-conditioned. We recruit some clients that are homeless. So the idea of being out of the sun—because, like I said, we do it before or after lunch, and that's when it's most hot out here. To be out of the sun in an environment that is welcoming, we have cold water, we're going to feed them, there's a shower if they want to use it, that has been a huge asset for us. And it's, again, something that we really were not planning or expecting but has proven to be really good. Executive Director, Agency 8

Barriers

The example above highlights a key aspect of successful implementation of this type of intervention: the necessity of the right venue. Other organizations had difficulty identifying appropriate spaces to conduct the intervention. For example, agencies that were small did not have a space large enough to conduct the intervention or that were safe and welcoming for participants.

We’re very limited in space here in the Prevention office. And, you know, also try to just, because that we were mostly getting women, is trying to get more like a safe space where the women can feel comfortable. So, you know, ‘cause maybe have to look at different space locations to ensure that one is warm safe, friendly, inviting, and welcoming for the women. Supervisor, Agency 15

Individuals

Facilitators

Program clients and agency staff are key to successful implementation of a program. People who delivered the interventions played critical roles. Sometimes an organizational staff has high buy-in and support of a program, other times, one individual—a “champion”—is the motivating factor in getting a program implemented.

Like I said, it’s really a combination of the strength and passion and drive of our staff. But also it is when people come through the door at our office they know that they’re going to get a different treatment than many of the places they go for service. Director, Agency 22.

Having a staff member who champions and promotes a new program may influence others and increase buy-in. Further, it contributes to successful implementation.

I think it's the person that actually has the position. You have to have a person that's a go-getter, that doesn't mind hitting the pavement, and really knowing what SHIELD is. Program Manager/Supervisor, Agency 15

The intervention is designed to train peer educators to share information about HIV prevention and risk reduction. By taking part in the intervention and gaining communication skills for HIV prevention, peer educators translated theses skills to recruitment of other potential clients.

Our peers are instrumental in disseminating the information into the community. So we were pretty good in giving out general information. But then our peers did like in the trenches wherever they go to places and their hours of service are different than hours when we do traditional hours. And because these are peers, they come close in contact with their peer network at different hours, nontraditional hours, nontraditional days. So our peers they played a really, really big part in recruiting for the SHIELD intervention. Facilitator, Agency 10

Barriers

According to several respondents, working with people who use drugs can be very challenging. Several agencies noted recruitment and retention of their populations was a barrier to successful implementation. One agency noted that some people who use drugs have different priorities, which may limit their participation in the intervention.

Sometimes it's recruiting and retention because we can get a lot of people that say yes, yes, they want to do it. But the retention part sometimes can be a little challenging.

Project manager/Facilitator, Agency 15. Director, Agency 8

A few organizations planned to deliver the intervention to a specific subset of people who use drugs. This narrow scope of participants presented additional challenges, as noted by an agency that served young adults who used drugs.

The main difficulty we have is the recruiting, being that our age group is so limited. That's the main problem we have, because we find that—at least I've found that during recruiting SHIELD it's the older current or former drug users that are really thinking "I want to be involved in talking to younger current drug users." So you have to find the 18 to 24-year-olds that have that passion and they want to be a peer educator. So a lot of 18 to 24-year-olds don't even want to tell you that they use drugs, so that's the main difficult we've had, is the recruiting. Facilitator, Agency 9

Intervention

Facilitators

While introducing a new program may bring some challenges, a new program may raise people’s curiosity and bring new clients to an agency as noted by one organization.

I think because it’s something that it’s new that nobody had ever heard of. My agency has been above and beyond when it comes to presenting new ideas to the community for things that get picked up later on, if not by other agencies at least by people that want to keep it going so to speak. …I have a good time bringing people in here who had never foot in this agency. Facilitator, Agency 8.

A key element in implementation success is the fit between the program content and goals and the target population’s needs and interests. One agency noted this important factor as they considered expanding their services.

You know, we wanted to identify something that would be a nice fit, that targets the population that we mostly see walking in, and to help support with the growth of our Women Services program. Supervisor, Agency 15

One aspect of small-group interventions is the development of accountability and social support among participants and the expectation that we are in this together. One organization noted that the group intervention makes participants accountable and increases their willingness to attend.

The other piece that has proven successful to us is that once a cohort begins the members of that group kind of hold each other accountable. And sometimes people come with a friend or someone they know, so they kind of help each other out, like "This is where we need to be. We need to go.” Director, Agency 8

Barriers

As noted previously, the SHIELD intervention consisted of six sessions. Multiple sessions were noted as a facilitator because it meant more accountability and support among clients. However, given the instability among the population, six sessions was incompatible with some the participants' lives. Getting the word out about a new program and retaining the clients over multiple sessions can be challenging.

I would say primarily it is—there’s two things. Getting the word out about a new program or project always takes time. And then the most difficult part of SHIELD in my evaluation of it is retainment of people, to get people to come to something six times, all six times on a particular schedule is difficult for anybody, let alone for somebody who has no schedule, doesn’t live by a schedule or doesn’t know how they’re getting there. So I think that initially getting the word out and then retaining people that’s where our employees have to work the hardest. Director, Agency 22.

Managing Barriers

Encountering barriers to program implementation is common; however, the agencies took action to manage or overcome these challenges. All organizations noted challenges with recruitment and retention of people who use drugs. Recruitment and retention for a 6-session intervention was a challenge cutting across multiple levels of the CFIR. Many solutions focused on creative strategies to manage recruitment and retention of participants. One agency started with two sessions a week but changed the format to a weekend retreat after dismal retention. This format change helped the agency meet its program enrollment goals.

Oh sure. Well, as you know the DEBI is six sessions. Well, we chose to do it six sessions over three weeks so two sessions a week for three weeks. And in talking with our project officer because our one program again that we had been a little challenged because it's such a niche population to recruit that we chose. That he suggested "How about trying a retreat format to see if you can adapt the curriculum to kind of a weekend format." When we've done that it's been very successful in recruiting and of course retention has been better too for completion. Director, Agency 6

After several clients missed some sessions, one agency prioritized sharing the missed information as a make-up session, as well as reaching out to clients for continued engagement so that they came back for additional sessions.

Well, sometimes we have to maybe go over a session with them if they miss it. We try not to do that too much because we don't want to make a habit of it, but that seems to allow people to like, “Okay, so I can continue.” So just doing that. Reviewing with them, and doing a follow up call to find out what happened because I think that just letting them know that, “Okay, you've been coming to the session, and you stopped for whatever reason, so how can we help to make sure that you will continue to do that?” So just doing a follow up call, or sending a letter, and if they can't complete it telling them that they can start the next one. That we're not forgetting about you, we want you to receive this information because we think it's very important. Facilitator, Agency 15

Several agencies used creative strategies to increase recruitment and retention. One agency emphasized the importance of coming to the intervention to get services.

I was just telling my supervisor that today. I was thinking that we will iterate. ‘Cause a lot of ‘em, from my particular department—what I do, they come in and get what they need; and then they fail to make good on their promise to participate. And I provide ‘em with a service; and then they—they’re really slow in- you know, in responding to what I need them to do. So my thing is I got to reiterate that: “Hey we made a deal; that’s what you’re getting here. And this is not for me, it’s for you; you know, this is information that you need to do- need to know in order to help you and your—well help others, peers in the same situation that you’re in. You know? And this is saving your life. And you got to make good. You got to be a person of your word. You know? If you’re really serious about changing your life and your lifestyle, you have to be a person of your word and take on your responsibilities. And your responsibility is to participate and learn all you can; and also spread it amongst your peers.” Facilitator, Agency 26

Finally, during intake, one agency completed an extensive locator form for each client to provide multiple types of contact information as a way to increase retention efforts.

So we're really strong on at intake completing a locator form, and we try to of course "Where do you hang-out? What are your nicknames? If we're trying to reach you who can we contact?" that kind of stuff.

Discussion

Implementation of evidence-based interventions is a complex process that is influenced by multi-level factors [35,36,37]. The results of this study show several facilitators and barriers that organizations encountered in the implementation of a peer educator intervention for people who use drugs. Some of the identified factors were consistent with previous literature. Others such as (1) challenges of recruiting and retaining people who use drugs for a multi-session peer education program; (2) how organizations pivoted in the midst of changing priorities at the CDC; and (3) logistics of holding multiple sessions of a sensitive nature in a safe, confidential space have not been discussed.

At the outer level, facilitators included support from partners and trainers and barriers included budgets and changes in funder priorities. As noted by Dolcini et al., initial training and support throughout implementation were critical to delivering the SHIELD intervention [38]. Funding is often the most cited barrier to implementation of new programs. Also, having funders or governing bodies change priorities leads to change in the intervention. As noted previously, SHIELD was disseminated through the DEBI program. After 2 years of dissemination, the CDC shifted their efforts into interventions that were single session and focused on individuals. This shift in priorities was called the High Impact Prevention approach [39]. As a result of this shift, some organizations changed the format of the intervention (such as a weekend retreat), while others had to select a different intervention during their implementation. As the CDC priorities shifted, SHIELD and similar interventions were no longer promoted by the CDC. This shift meant that the CDC no longer funded SHIELD implementation or provided trainings and technical assistance to organizations who were interested in SHIELD. While CDC ceased support for SHIELD implementation, many organizations continue to fund organizations in implementing SHIELD. One aspect that is missing is training and technical assistance to support implementation. At the time of the study, going to a CDC training was a common approach for agencies to learn how to implement an intervention. Further, many organizations have contacted the study team for a copy of the SHIELD manual  which indicates interest in this program. Without training and support from the CDC, it is even more imperative to prepare organizations for what they will encounter with implementing a peer education intervention for SHIELD.

At the inner or organizational level, facilitators included previous experience with evidence-based interventions, having a presence in the community, access to the population, and delivering the session at their site. Barriers included identifying an alternative site that was a safe space. Some organizational factors met the basic needs of clients such as access to a shower or hot meals during the sessions [40]. Having experience with other evidence-based intervention as well as strong infrastructure are often cited as facilitators [15]. One agency referenced their education department, which allowed the organization to have easy access to updated information that may not be common knowledge among staff. Continuing educational resources, such as an education department, are key to the success of long-term sustainable EBIs [38, 41].

Individual-related facilitators included having dedicated staff, a program champion, and current SHIELD participants. Noted barriers were recruitment and retention of people who use drugs or a specific subset of this population. Having support among staff or a champion to promote the program has led to successful implementation [42]. As noted in the study findings, working with people who use drugs can be challenging but support from engaged participants was a facilitator. Recruitment and retention were identified as particularly challenging which is consistent with other studies [43, 44]. The added difficulty of poverty and transience among people who use drugs emphasizes the need for organizations to be accommodating and to ensure that the implemented public health program can address the pressing and relevant needs of the populations. As discussed previously, peer educators play an important role in telling others about the benefits of the program [45]. Training participants to disseminate information within social networks allows the intervention to reach individuals that may previously have been inaccessible. Finally, program participants often encouraged their peers to attend the program and make other participants accountable. This finding has been reported in other peer-based interventions [46].

At the intervention level, facilitators were novelty of the program, program that meets client’s needs, and multiple sessions which made participants accountable. Barriers were getting the word out about a new program and retention of participants. At the time when SHIELD was disseminated by the CDC, there was only one other intervention promoted for people who used drugs, Safety Counts [47]. Organizations were excited to implement SHIELD because it was an opportunity to bring a new program to their site. As noted by one organization, this enthusiasm was felt among participants as well as staff.

Although many barriers were identified, the agency staff used creative solutions to address many of these barriers. Technical assistance can also be critical to assist programs, especially those having less experience with the target population or are hesitant to adapt the programs.

One limitation to this study was that the respondents were interviewed after they implemented the intervention. For some agencies, implementation took place one to two years prior introducing recall bias as a limitation. Our sample was based on agencies who were trained by the CDC; thus, the sample may not be representative of all agencies who have implemented SHIELD. Also, since we contacted organizations based on the list we received from the CDC, agencies that implemented SHIELD may have been more likely to respond to our calls leading to potential selection bias.

The study’s findings also suggest the importance of linking public health and social services programs to address additional needs in the participants lives. As mentioned above, the program will not be a priority among participants until their immediate needs have been met. Ensuring that the physical space and appointment times are feasible for clients are also imperative to improving recruitment and retention among participants. Future evidence-based programs should, in addition to addressing effectiveness, be relevant to the needs and lives of clients so we can better integrate public health programs with social services.