Introduction

Implementing interventions that improve outcomes along the HIV care continuum for persons living with HIV (PLWH) are necessary to reducing mortality, improving viral suppression and decreasing transmission of HIV in the United States [1]. The Health Resources and Services Administration’s (HRSA) Special Projects of National Significance (SPNS) developed the Systems Linkages and Access to Care for Populations at High Risk of HIV Infection Initiative (SLAC) in 2011 [2] to improve access to and retention in care for persons who are at high risk for and/or infected with HIV but are unaware of their HIV status; are aware of their HIV infection, but have never been referred to care; or have been referred, but have not linked to care. In response to the initiative, New York State’s (NYS) SPNS-funded project known as New York Links (NYLinks) [3], developed and distributed implementation manuals for three evidence-informed interventions: Peer Support [4,5,6], Appointment Procedures [7,8,9], and an adapted version of the Anti-Retroviral Treatment and Access to Services (ARTAS) [10, 11] case management intervention. These interventions were selected due to evidence of effectiveness found in the literature, and the feasibility and costs of implementation. HIV care providers participating in NYLinks across NYS elected to implement one of the interventions at their site.

While published evidence shows that these three interventions improve linkage and retention, interventions found to be effective in research studies may fail to provide the same successes in real world clinical and supportive service settings [12, 13]. Factors including organizational structure, processes and culture, as well as the adaptability, complexity, cost, resources, and packaging of interventions, may influence both their successful implementation and ultimately their effectiveness and impact [12, 14, 15]. As sustainability and replicability of the interventions were goals of SLAC, drawing upon Damschroder’s model [16], we explored factors shaping implementation processes.

Using a convergent mixed methods design [17], we present quantitative findings of the process measures describing intervention delivery, in conjunction with qualitative findings that explain the barriers and facilitators to achieving full implementation, as well as adaptations made during implementation.

Methods

Intervention Development and Implementation Through SLAC

As one of six demonstration states, SLAC support for NYLinks took place over 5 years. The first 2 years consisted of a preparation phase in which interventions were designed and tested, followed by wider-scale statewide dissemination and evaluation of tested interventions during years 3–5.

NYLinks staff consisting of program managers, quality improvement expert coaches and program evaluators, invited NYS providers at HIV testing and clinical care sites to implement an intervention based on a set of criteria. This included the feasibility of implementing the intervention at their site, and linkage and retention rates with room for improvement based on self-reported performance measurement data [18, 19]. Performance data reviewed included linkage to care: percentage of newly diagnosed patients who had their first HIV clinical care visit within 30 days of the date of their confirmatory HIV test result; retention in care: percentage of patients with at least one HIV clinical care visit in each 6-month period over 2 years; and new patient retention: percentage of patients new to the clinic with at least one HIV clinical care visit in each 4-month period over the course of a year. To participate in the study, sites ideally had to have linkage rates below 70%, retention rates of less than 75%, new patient retention rates of less than 65%, and a sizeable patient volume (e.g. HIV clinic population of 200 or more patients per site for the appointment reminders intervention).

In total, 12 out of 13 volunteer sites (11 clinical care providers and one HIV testing/supportive service provider) implemented one of the three NYLinks interventions over the course of the entire year. Four sites from Brooklyn, Upper Manhattan (UM), Long Island (LI), and the Hudson Valley (HV) implemented Peer Support; seven sites from Western New York (WNY), Queens, UM and the HV implemented Appointment Procedures, and two sites from UM and WNY implemented ARTAS. One Appointment Procedures site dropped out of the project after 5 months. This site’s implementation experience is described in the qualitative assessment. Sites that agreed to implement an intervention received non-monetary support including technical assistance from NYLinks through a dedicated implementation coach (HIV quality improvement specialists) and access to an implementation manual and various trainings (i.e. ARTAS, motivational interviewing) for their staff. All sites took part in multiple in-person meetings with NYLinks coaches and evaluation staff, and received assistance and check-ins by phone or through email. All participating sites were asked to submit intervention process measure data monthly for at least a year and participate in interviews and focus groups regarding implementation processes.

The majority of participating sites began implementation in the fall of 2014. Eligible patients were either automatically enrolled (Appointment Procedures intervention) or offered the intervention (ARTAS, Peer Support interventions) as part of routine service delivery and quality improvement activities. Patients could refuse enrollment or opt-out at any time. The protocol for the NYLinks evaluation was approved by the Hunter College Institutional Review Board.

Interventions

NYLinks staff developed supporting intervention implementation manuals, implementation readiness assessments, fidelity to the intervention check-lists, data collection tools, and other resources to aid implementation (www.newyorklinks.org/interventions). The three NYLinks interventions consisted of the following:

Peer Support

The aim of the NYLinks Peer Support intervention was for trained peers to engage patients early and routinely to establish a foundation and relationship that enables regular HIV medical care. A peer was defined as an HIV positive person who receives care at the clinic, adheres to their ART medication regimen, and is virally suppressed. Peers were paid staff (some of whom were former volunteers), who understood how the clinics functioned, what services it provided, and were familiar with clinic staff. Eligible patients for the intervention included those who were newly diagnosed, new transfers to the clinic, and those returning to care after being out of care for more than 6 months. Peers were responsible for meeting and greeting new patients either prior to, on the same day of, or within 3 days after the patient’s first medical visit. They were also responsible for introducing the patient to clinic staff, sharing information about available services and educational materials, informing case managers about needed referrals, and providing social support.

Appointment Procedures

Care coordination often involves the use of multiple interventions and the coordination of two or more persons to facilitate a patient’s care [20]. NYLinks Appointment Procedures intervention was adapted from a module in the New York City Department of Health and Mental Hygiene (NYC DOHMH) Care Coordination program [7, 21]. The Appointment Procedures intervention standardized procedures to facilitate scheduling and reminding all patients of their upcoming appointments. The aim was to reduce no-show rates and to increase the likelihood that patients were successfully engaged and retained in ongoing care. For this intervention, two reminder calls were made: one within 5 days before the appointment, and the second within 1–2 days before the appointment. If a patient missed an appointment, daily follow-up calls for up to 5 days were made to reschedule the appointment and ensure continuity of care. Although all HIV positive patients were eligible for the intervention, two sites modified their intervention target population. One site focused on those with upcoming appointments who had missed two or more appointments in the past year, while the other site targeted approximately 300 of 1300 HIV positive patients who fell under the caseload of five physicians. This site selected a mix of physicians with high and low caseload numbers. Both sites made these modifications to keep the intervention more manageable due to limited staff capacity.

ARTAS

ARTAS provided brief strengths-based case management to newly diagnosed persons. ARTAS sessions focused on a patient’s strengths, identified their needs and barriers to accessing care, and helped patients create an action plan towards obtaining their goals. It was essential that the case manager built a relationship and trust with the patient. The aim of the intervention was to increase patients’ abilities to address and overcome barriers and promote early linkage to HIV care. The intervention included up to five sessions with a case manager over a 90-day period, or until the patient had at least one HIV care visit with a medical provider. NYLinks adapted the existing ARTAS manual [22] by expanding the patient eligibility criteria from focusing solely on newly diagnosed patients, to including individuals returning to care after a lapse (greater than 6 months out of care) [23]. Sites implementing ARTAS received the original manual, as well as a condensed implementation manual developed specifically for NYLinks. An official free in-person ARTAS training conducted by the NYC DOHMH was also provided.

Quantitative Methods

Process measure data collected on intervention implementation were used to assess whether the interventions were being delivered as designed and reaching targeted patients with adequate coverage and engagement (i.e., percentage of eligible patients enrolled, percentage of intervention components successfully delivered). Each intervention manual contained a section on patient-level and aggregate process measure data elements. Data collection tools developed using Excel spreadsheets were also provided. Only de-identified aggregate-level process measure data were reported by implementation sites on a monthly basis and used as part of the NYLinks evaluation. We analyzed process measure data reported from September 2014 through September 2015 (Table 1).

Table 1 Process measures collected monthly from sites for each intervention
Full size table

NYLinks coaches and evaluators performed data quality reviews for each submission and potential inaccuracies were returned to the sites for review, discussion, and correction. Process data from each site were combined for each intervention to produce NYLinks-wide numbers.

Qualitative Methods

From July 2014 to August 2015, NYLinks qualitative researchers (ABL, LS) collected data through recorded observations of in-person technical assistance meetings and key informant interviews with intervention implementation staff at most of the participating sites (Table 2). Semi-structured interviews and field observations focused on implementation processes, as well as contextual factors facilitating or creating barriers to implementation. Adaptations were also explored. Interviews were conducted in person or by phone. They were digitally recorded, transcribed, and securely stored in password-protected files to maintain confidentiality.

Table 2 Qualitative data collection methods
Full size table

Convergent Mixed Methods Analysis Process

Consistent with Fetters et al. [17], we used a two-phase, convergent analysis process. In Phase I of the quantitative and qualitative arms of the study (beginning in late 2014), we conducted parallel data gathering, which by February 2015 produced initial process measure data assessments, and a qualitative findings codebook and report highlighting processes shaping site selection of interventions. Phase II began in March 2015 consisted of ongoing review and analysis of process data, along with additional interviews with implementation sites based on the initial codebook as well as newly emerging questions from the quantitative analysis. This phase identified additional barriers and facilitators to implementation, as well as adaptations sites made during implementation. The coding framework was further refined, and focused thematic memos on each intervention were developed.

Results

Peer Support

Peer Support intervention sites (n = 4) with caseloads of approximately 400-1450 HIV positive patients yearly, identified 396 patients eligible for the intervention from November 2014 to September 2015. Eligible patients included those who were newly diagnosed, new transfers to the clinic, and those returning to care after being out of care for more than 6 months. Of those eligible, 255 (64%) elected to receive the intervention or were automatically enrolled, of whom 91% (n = 233) met with a peer either prior to, on the same day of, or within 3 days after their first medical visit (Fig. 1).

Fig. 1
figure 1

Peer support sites (n = 4): cumulative number of patients eligible, enrolled, and met with a peer from November 2014 to September 2015

Full size image

An important factor that helped with the successful implementation of the Peer Support intervention was that participating sites had pre-established peer programs into which they could incorporate the core components of the NYLinks Peer Support intervention. Peers who could perform the core functions of the intervention were easily identified and had already received many of the necessary trainings. This allowed for an easy expansion of their roles to provide the intervention to the target population. Although sites had existing peer programs, the NYLinks Peer Support intervention provided structure to these programs by creating a time frame that linked peers with patients at initial medical visits. The intervention formalized meet and greet procedures and improved integration of peers into the clinic workflow. Peers became more active member of the clinic team with clear functions to engage and support patients in care. Peers also strongly believed in the intervention and wanted it to be successful. They felt they were building unique and special relationships with patients, who shared useful information that clinical staff would be unable to obtain otherwise.

Implementation staff noted several factors accounting for why only 64% of eligible patients were enrolled in the intervention. In spite of the pre-existing peer infrastructure, one barrier to the successful enrollment of eligible patients was due to peers not being fully integrated into service delivery processes. For example, sometimes staff would schedule an eligible patient to come in for an appointment when the peer was unavailable, or the peer would miss connecting with the patient while he/she met with various clinic staff. Some sites tried to overcome these barriers by making a more concerted effort to integrate peers into the service delivery flow:

“We talked about how this would blend into our team process, and just all agreed that when a new referral to our program happened that the case managers, or whoever got that call, would schedule it on a day that the [peer] was working and if the [peer] was there, that they could take the call for that initial contact.” (Director of HIV Services at one of the sites)

At one site, the peer was stationed at the front desk, thereby increasing the likelihood of making contact with eligible patients. At another, the peer printed daily schedules and cross-checked them with Electronic Medical Records (EMRs) to diligently track down patients with visits that day and, in the peer’s words, “grab him before anyone else grabs him.”

Further, staff described two characteristically distinct target groups of the intervention: newly diagnosed and returning to care. The intervention was suited to newly diagnosed patients, but for those out of care for 6 or more months, the intervention’s ‘meet and greet’ focus was not usually needed since many of the patients were already familiar with the clinic and didn’t require additional orientation. Further, while the intervention was better suited to those new to the clinic, some newly diagnosed persons were also challenging to enroll if they were not ready to meet a peer while processing a recently received HIV diagnosis. As a result, peers reported applying the intervention with some flexibility, using their experience to determine when and how to engage patients most meaningfully.

Appointment Procedures

Approximately 1300 patients in six sites were enrolled in the Appointment Procedures intervention over the course of a year. While one reminder call was a standard procedure for all sites, two reminder calls were made for only 81% of appointments (Fig. 2). Over the course of the year, on average, there were 130 patients with missed appointments during a period of a month across all sites combined. Patients who missed appointments received daily follow-up contact for a minimum of 5 working days or until they were reached for 92% of the missed appointments. 81% of patients who missed appointments were reached after follow-up, with 99% of patients rescheduling their appointment once reached (data not shown).

Fig. 2
figure 2

Appointment Procedures sites (n = 6): number of patients with scheduled appointments, number of appointments, and number to receive two reminder calls/contacts from September 2014 to September 2015

Full size image

Sites developed new procedures to increase the number of calls made each day, and some sites relied upon their information technology (IT) department to create new data tracking systems and extract data as needed. Although most patients received the two reminder calls for upcoming appointments, there were some notable barriers to full intervention delivery. Six sites had their staff conduct the reminder calls prior to a patient’s appointment, with only one site using automated phone calls. The site with an automated reminder system had approximately 700 patients (54% of total patients targeted in this intervention). Sites without automated systems had approximately 50–350 patients targeted for the intervention. With one reminder call often being the standard procedure prior to intervention implementation, sites without automated call systems struggled with the additional workload of making the second reminder call. Staff at most of these sites reported that making the additional calls was taxing to their already full schedules. Competing demands and staff turnover contributed to implementation challenges. As one implementation staff lead noted:

“Our plates are very full. I have to do this [intervention in addition to all of my other work]. And it’s not something I can [delegate] to others.” (HIV Program Implementation Lead at one site)

One site ceased implementation entirely after 5 months due to the burden of making the additional calls. Others modified intervention delivery to decrease the volume of reminder and follow-up calls required to still achieve some degree of implementation. This included reducing the frequency from five to one or two calls after a missed appointment. While deviating from the intervention protocol, these modifications were thoughtfully executed for greatest impact, while rendering the intervention more manageable:

“[We went with] the group needing the most attention. It would not only be important to help getting them in and hopefully impact them medically, but was a little more doable for us.” (Program Lead at one site)

Another barrier pertained to sites’ scheduling procedures. At two sites with open access appointments, patients regularly missed scheduled appointments and dropped in at a later date. In these situations, the new procedures put into place through the intervention were even more onerous and a poor fit with patient use of the clinic:

“They just kind of walk in with the – at least the hope, but not the expectation that they’ll be seen that day. It was just sort of a little bit more of a challenge…because if somebody tends not to make appointments, [we] can’t really give reminders and then try to see an increase in their attendance.” (Clinical Director at one site)

ARTAS

ARTAS intervention sites (one a supportive service site, the other provided HIV clinical care) approached 20 patients who were eligible for the intervention, of which 19 (95%) were enrolled. Of those enrolled, 16 (84%) were linked to care within 90 days of enrollment (Fig. 3). Although the eligibility criteria for ARTAS included both newly diagnosed and hard-to-reach patients (defined as out of care for 6 or more months), it is unknown how many patients served at each site were eligible for the intervention, but not offered enrollment.

Fig. 3
figure 3

ARTAS sites (n = 2): cumulative number of patients eligible, enrolled, and linked to care within 90 days from September 2014 to September 2015

Full size image

Similar to the other interventions, ARTAS required some changes in processes of care, which the participating sites achieved through staff communication and planning. The high rate of enrollment at participating sites may have been influenced by the value clinical staff placed on the intervention’s content and utility. They felt the intervention helped staff forge a deep connection with patients, and helped gather information that could inform many aspects of a patient’s care. Implementation staff at one site spread this message and improved buy-in and support by expanding training of ARTAS to all clinical staff and not just those directly involved in the implementation process.

Although successful in enrolling patients who were offered the intervention, sites implementing ARTAS noted challenges focusing on hard-to-reach patients which resulted in a time-consuming engagement process of active outreach. Allowing flexibility in staff schedules, and the ability for staff to travel outside of the site to meet with patients, were critical to the delivery of the intervention and ultimately its success. Staff at sites also reported that the ability to conduct half of an ARTAS session one day, and the rest at another time, provided flexibility that best fit with patients’ needs.

Discussion

Our findings on the factors shaping implementation, resulting in barriers, facilitators and adaptations, were consistent with those found in the literature [24, 25]. Three common themes were identified across all of the interventions and were found to be critical components for successful intervention implementation (Table 3). Factors that facilitated implementation included concerted buy-in and coordination of staff, building upon existing infrastructure including ensuring sufficient staff capacity, and allowing adaptability of certain parts of the intervention to better fit patient needs and clinical settings.

Table 3 Major summative themes to achieve implementation success
Full size table

First, however seemingly straightforward a new intervention might be, its implementation and delivery require coordinated staff effort, and careful procedural integration into pre-existing delivery systems. This entails the deliberate identification of where and how intervention components can be incorporated, alongside taking steps to make adjustments based on new processes. When this did not occur, inadequate integration of the intervention into existing organizational processes and workflows was found. Clearly defining staff roles and educating all staff on the intervention and how it will be conducted within the site’s settings appear to be critical to this process. Gaining the buy-in of staff who are not directly implementing, but whose working patterns may influence and be influenced by the intervention, enhances chances of success. As reported, one way to attain buy-in is to open up intervention trainings to non-intervention staff as happened with an ARTAS site. Another, as was the case with the Peer Support intervention, is to collectively determine when and where intervention staff can best connect with patients in the flow of patients through routine clinic procedures. Strong leadership buy-in has also been found to be an important factor influencing successful implementation. Our findings suggest that the value of buy-in comes from diffusing the intervention’s importance across clinical staff at all levels.

A second theme identified in this study is the importance of having the appropriate infrastructure, including staffing levels, to take up and implement the intervention. This can be significantly aided by building off of pre-existing program components. For example, Peer Support sites were greatly aided by building upon pre-existing peer programs. The Appointment Procedures intervention also required pre-existing IT systems for some sites to contend with the extensive amount of data management to track patient appointments and missed appointment follow-up. ARTAS implementation also needed the staffing capacity and clinical procedures that would allow intensive community-based outreach. When interventions were being implemented without robust structures in place (e.g., a higher degree of staff turnover and competing priorities), they created a workload burden and its implementation suffered as a consequence.

The third theme pertains to the importance of allowing for modifications of certain intervention components, or processes of delivery, to fit into clinical settings and patient use of services, so long as core, evidence-based components of the intervention are uncompromised. Consistent with the literature [16], patient-level and site-specific factors influence the need to tailor the intervention. For example, findings from the Appointment Procedures intervention indicate that when the appointment reminder system matched how patients actually used the clinic, intervention delivery was more effective; as a result, the intervention was also more manageable for staff to execute. The same was true for Peer Support, in which peers modified content and contact based on their nuanced understandings of patient needs. For peers, as well as in the ARTAS intervention, the intervention allowed staff to be attuned to patients’ specific needs, which helped build rapport with the hardest to reach patients. Intervention dissemination strategies that allow for ongoing clinic-level adaptations are critical to successful implementation. This keeps the interventions current and promotes sustainability, particularly when new target populations or contexts of delivery emerge.

All sites were assisted by dedicated NYLinks implementation coaches who provided access to various resources, as well as solutions to challenges in implementation and data systems to track measures. We do not know the extent to which coaches may have influenced implementation. Although coaches served as a technical assistance resource, internal and external factors were still identified, which created challenges to successful intervention implementation. Primary internal or clinic-level factors included, for example: complex existing service delivery processes and limited workforce or IT capacity. External or structural factors included: funding limitations resulting in staff needing to integrate new requirements into their already full scopes of work, and contending with hard-to-reach patients. These factors should be carefully assessed to determine the optimal methods of intervention delivery.

Although long term sustainability of the interventions has not been evaluated, half of Appointment Procedures sites confirmed wanting to continue the intervention beyond the evaluation year, as did two out of three Peer Support sites, and both ARTAS sites. With some experience delivering the interventions as part of NYLinks, some sites that confirmed continuation expressed the desire to make minor alterations to the interventions going forward to reduce staff burden or increase reach of the intervention. This included, for example, targeting specific patients who have high viral loads instead of all patients for Appointment Procedures, and expanding the ARTAS intervention to help individuals with adherence challenges. Organizations wanting to implement one of the NYLinks interventions in the future would need to budget resources to continue monitoring and evaluation of intervention reach and effectiveness. This includes allocating funds or staff time towards routine collection and review of process and outcome data, or working with IT staff to track and report data from electronic health record systems.

This study has limitations. Process measure data reported monthly by sites were aggregated, therefore we were unable to examine patient-level differences. For example, although gaps in enrollment in and receipt of the intervention were found in the Peer Support intervention, we were unable to determine if the majority of the gaps occurred with new patients versus those returning to care although clear differences were noted during staff interviews. Another limitation was that there were very few patients eligible for the ARTAS intervention over time (sites had a small caseload), which limited the assessment of process data trends. Although successful in enrolling patients who were offered the ARTAS intervention, the number of patients who were out of care and eligible for the intervention is unknown.

Our study does however contribute to the growing literature of factors influencing implementation of evidence-based interventions in real world settings particularly without additional funding. By highlighting factors influencing implementation, and in particular the planning and ongoing monitoring required when implementing a new approach to care delivery, our findings may be used by program developers to inform the design and implementation procedures for other interventions. Our study supports the need to incorporate the examination of implementation processes in evaluations and to not solely focus on outcomes. Quality improvement methodology is also ideally suited for this purpose, involving the ongoing testing of adaptations, observing the results, and using real-time feedback to improve intervention effectiveness across different patient groups and in a variety of settings [26]. Findings from this study have also been used to develop and further refine NYLinks intervention manuals, and will be used to contextualize outcomes when data become available.