Introduction

People living with HIV (PLWH) require long-term care to maximize health outcomes and achieve viral suppression (VS). Consistent engagement, or retention, in HIV care is one of the strongest predictors of initiating ART [1, 2] and achieving viral suppression for PLWH [1, 3,4,5]. Conversely, poor retention in care is associated with higher viral loads (VL) and decreased rates of VS for PLWH [1, 4, 6], as well as other adverse outcomes, including increased hospitalizations, development of AIDS-defining illnesses [2, 7, 8], and increased risk of mortality [2, 4, 9, 10]. Therefore, improving retention in HIV care is a priority of national significance, especially in the Southern Region of the US, which is disproportionately impacted by HIV. The South possesses the highest HIV incidence rates, accounting for half of all new HIV diagnoses in the US, while having just over one-third of the country’s population [11]. Furthermore, compared to other regions, HIV and AIDS case-fatality rates are significantly higher while survival rates are significantly lower in the South [12, 13].

North Carolina’s population of PLWH mirrors that of the rest of the South. As of 2015, there were 33,388 PLWH in NC. However, estimates indicate that when including those who are undiagnosed, this number was closer to 36,800 PLWH, of which only approximately 21,300 were retained in care, defined as at least two care visits 3 months apart in 1 year or demonstrating viral suppression within the year [14]. The unique social and demographic factors of the South may offer a partial explanation of these disparities in HIV incidence and mortality. The South experiences higher rates of poverty, unemployment, lack of health insurance, and stigma around HIV infection than other regions in the US, all of which may work as barriers to accessing HIV care [15]. The South also has a higher proportion of individuals with HIV living in rural areas [13], lacking public resources, creating a significant barrier to accessing and maintaining HIV care [15,16,17]. This is true in North Carolina [18], where transportation options are solely dependent on private resources [19]. In addition to lack of material resources, stigma is highly prevalent, and subjects reported avoiding relationships because they believe they are at risk of being judged if they disclose their HIV status. Both of these beliefs make it difficult to access resources that do exist [20].

Some of the most widely-used interventions to increase retention in care include using patient navigators, care coordinators, and medical case managers to work closely with newly diagnosed patients and those at-risk for being out of care [21,22,23,24]. Prior to 2014, many of these interventions focused on linking and retaining newly diagnosed PLWH to care and most used a dedicated person(s) to staff the intervention [25]. Outreach interventions have demonstrated success in re-engagement of those considered out-of-care [26]. In a systematic review of methods to increase linkage, retention and reengagement in care, only three of the ten identified interventions focused on delivering interventions to a general clinic population; only one described methodical monitoring of the clinic population for retention [25]. Since systematic monitoring in the international retention guidelines is a recommendation, there was a need for effective strategies to improve retention and re-engagement in care for PLWH [27]. As part of the Systems Linkages Initiative, the NC LINK Project developed a clinic-based protocol that five Ryan-White clinics used to retain people living with HIV (PLWH) in care. The NC-LINK Retention Project utilized a systematic approach to identify, locate, and attempt to return to care out-of-care patients who did not attend a clinic appointment for 6 or 9 months. This paper addresses the following research questions for the NC LINK Retention Intervention: What are the demographic and clinical characteristics of PLWH who are retained in care after receiving this intervention? What proportion of PLWH returned to care and achieved VS following the intervention?

Methods

Setting

The Division of Public Health’s AIDS Care Program divides NC into 11 geographic regions for HIV service delivery. Each region participated in the NC-LINK Learning Collaborative, the process used to develop the NC-LINK interventions and were offered the opportunity to participate in this intervention. Five clinics in four different regions of North Carolina participated. All were Health Resources and Services Administration (HRSA) Ryan White funded clinics in central and eastern NC and ranged in size from approximately 400–1950 total HIV patients. Sites A and B were academic Infectious Disease clinics, while Site C included two small rural clinics and Site D was a clinic in a regional medical center.

Out-of-Care Participants

Patients were eligible for a retention intervention referral if they were identified as potentially out-of-care based on lack of attendance at a medical visit in the prior 6–9 months and had no known future appointment scheduled with an HIV medical provider. Lists of these out-of-care patients were generated monthly in each of the clinics using CAREWare and electronic medical records. Persons with a future HIV medical provider appointment were removed from the list. The remaining persons were considered as potentially out of care.

Intervention

Clinic-Based Out-of-Care Intervention

The NC-LINK Retention Intervention has been described previously [28]. The intent of the intervention design was to create a low-intensity intervention that would prevent gaps in care and could be incorporated into the usual activities of clinic staff. Briefly, NC CAREWare and/or clinic electronic medical records (see below) were used to generate a list of active clinic patients who had not had a clinic visit in at least the previous 6 or 9 months (time frame was clinic-specific). This intervention was based solely on clinic appointment attendance and was not a surveillance data-based intervention. PLWH who were identified for this intervention between January 1, 2013 and December 31, 2014 were included in this analysis, (Fig. 1). The initial out-of-care lists were manually reviewed by clinic staff to remove patients with future scheduled appointments or who were already known by staff to have moved, died, or become incarcerated. Then, clinic-based retention staff used letters, phone calls, and database searches to locate, contact, and reschedule the out-of-care patients with the goal of retaining the patient in active care. This is inherently a low-touch [29] intervention where clinic staff used only phone calls to try to reach and reschedule patients. If all strategies were used and the patient was either unable to be located or located but not retained at the current clinic, referrals were then made from the clinics to public health team members, called State Bridge Counselors (SBC), for further intervention including home visits. The SBC intervention and program outcomes have been reported elsewhere [30, 31].

Fig. 1
figure 1

Out-of-care patients referred for clinic-based retention services and final intervention status

Full size image

Intervention Staffing

The five clinics divided the activities amongst existing staff. List generation was usually performed by the person managing the data and reporting for Ryan White funding requirements. The time for this activity was approximately one-half a day per month. The follow-up with patients was usually performed by patient navigators or case managers already working with the clinic as employees or in affiliated agencies. The time and effort required ranged from 7 to 19 min/week (average 12 min) for every ten patients in the clinic census. The staff assigned to patient follow-up were already responsible for reaching out to patients found by providers to be missing from the clinic. However, the  previous efforts were ad hoc requests by providers or in large batches at infrequent intervals. The use of the retention intervention provided a structure to ensure that patients were looked for systematically and at shorter intervals, the work was distributed throughout the staff, and identification of out-of-care patients was comprehensive rather than based on providers’ personal lists or memories. The clinics with the lowest amount of time per patient did report that sometimes they were not able to complete the list within 30 days if other work or staffing demands occurred (i.e. annual enrollments into the state AIDS drug assistance program).

Documentation of Services

Clinic-based retention staff tracked their retention efforts in CAREWare to record clinical data, generate HRSA-required service report data, and monitor clinical quality indicators. These clinics all had prior experience using CAREWare, and a unique table was created for the purpose of recording time spent on retention efforts, methods used, and the 30 day outcome. Once communication was made with the patient, actions taken to aid the patient with care engagement were tracked as ‘services’ in CAREWare. Retention staff documented the status (see below for variable definitions) of the patient after 30 days of retention efforts from the perspective of the clinic.

Lost-to-Care Intervention

Persons who were not located with the retention intervention services were considered as “lost-to-care” and referred to a Division of Public Health team of State Bridge Counselors. This intervention and the outcomes have been described elsewhere [30, 31] and are not included in the analyses reported here.

Ethical Issues

Data Sharing

In 2013, a joint statement was issued by the NC Division of Public Health Legal and Regulatory affairs and the State AIDS Director allowing for the release of information under HIPAA for the purposes of treatment and health care without requiring a specific data sharing agreement between entities. This resulted in changes that allowed clinics to see information for their patients in any domain in which that patient exists. Prior to this, the Ryan White clinics in North Carolina used CAREWare, but each agency only had access to their own service information. The ability to see all patient care information and determine if care was being provided in another clinic improved coordination of care services and allowed efforts to be focused on those patients truly in need of retention services [28].

Waiver of Consent

The study was performed under the direction of the Institutional Review Boards at Duke University and NC Division of Public Health. The intervention was considered quality improvement activity and informed consent was not required.

Methods of Evaluation

Implementation Reporting

Implementation science has as a goal the modification of human behavior in the delivery of services rather than the generation of new generalizable knowledge. Consequently, the standard elements of scientific publications are not always well aligned with the presentation of this data, leading to inconsistent descriptions [32]. SQUIRE guidelines have been developed to present program implementation and quality improvement data and form a knowledge base for implementation science. This article has been prepared using the SQUIRE guidelines to allow an accurate presentation of quality improvement work.

NC CAREWare

CAREWare is HRSA’s electronic health and social support services information system for Ryan White HIV/AIDS Program grant recipients. NC CAREWare is a repository of NC Ryan White service data recorded by agencies receiving Ryan White PART B funding from the North Carolina AIDS Care Program. The program is shared by all funded service agencies in the state, allowing staff to see if patients have received services from other providers. The referral tool in CAREWare was used to record a referral for out-of-care services, but also allowed for the designated staff member and the outcome of the retention service to be recorded. These data were the source for the referral outcome variable. Measures regarding retention work for those identified on the out-of-care list were recorded in CAREWare and included referrals, services and process outcomes.

NC eHARS

All CD4 and HIV RNA results performed on patients living in NC are currently reported to the Division of Public Health and are stored in the NC Enhanced HIV AIDS Reporting System (eHARs), the CDC’s HIV surveillance information system storing demographic characteristics and lab data for VL and CD4 lab results for all persons identified in NC with HIV infection. All the data for analyses were retrieved from this surveillance data set except for the referral and outcome data retrieved from CAREWare. Individuals identified as receiving a referral for retention services were matched to a record in eHARS. A lag period of at least 90 days after any given follow-up period was used for all laboratory results from this source to account for delays in reporting, with all data being extracted after March 31, 2016.

Data Limitations

The surveillance data was initially limited by reporting regulations that did not require reporting of undetectable HIV RNA or CD4 counts over 200 c/ml. Data improved over the course of this intervention as the state implemented required CD4 and VL reporting from all labs, beginning in July 2013. CAREWare laboratory data was available, but only for Ryan White funded clinics, so it did not capture outcomes for patients who moved to a non-Ryan White site. Therefore, the HIV RNA data for using VL as a proxy for medical care were obtained from both eHARS and CAREWare to capture the most complete records available. We distinguished when patients moved to new clinics, because leaving a Ryan White clinic would mean that a patient could be more likely to have missing data.

Outcomes

Referral Outcomes

Possible outcome statuses included: “located, maintained clinic” (patient was successfully contacted and known to be living in the clinic service area), “located, new clinic or relocated” (patient was successfully contacted but known to have moved outside the clinic service area), “not located” (could not be contacted), or “ineligible” (known to be deceased or incarcerated). PLWH were excluded from further analysis if their 30 day outcome status was deceased, incarcerated, or missing an outcome.

Return to Care Outcome

The return to care outcome measure was defined by the presence of a VL result in eHARS within 90 days of the referral. For those who were located, the presence of a VL test result was used as a proxy to indicate care both prior to and following the intervention. (This differs from the use of clinic visits to identify out-of-care patients initially for the intervention). CD4 lab results were also considered, but we found that they did not provide any additional information compared to VL alone.

Retention Outcomes

Outcomes for time periods of 180–365 days were also reported to provide information about the long-term return to care of the PLWH. The proportion that met the HIV/AIDS Bureau (HAB) retention measure of two lab markers within the follow-up year at least 90 days apart following referral for retention services were also reported. PLWH with missing VL results were considered to be not in care and not virally suppressed. These measures were not used in the analytic model because they were not anticipated to be attributable to the intervention. While this limited intervention may not be responsible for outcomes at the 1 year mark, we report these numbers to give a fuller description of our study population.

Viral Suppression Outcome

Using the VL results, we also measured the proportion of the population achieving viral suppression (VS, HIV RNA < 200 copes/ml) by 180 days. VS at 1 year was also reported but not considered a primary outcome.

Analyses

Prevalence ratios with 95% confidence intervals were calculated to assess relationships between demographic and clinical characteristics and whether a patient was located. Then, adjusted prevalence ratios (aPR) were calculated to adjust for characteristics found to be associated with the outcomes in the univariate models. These were insurance, prior care evidence, prior VS, clinic site, age and time from diagnosis. Prevalence ratios with 95% confidence intervals were calculated to evaluate the association between demographic and clinical characteristics and returning to care within 90 days and achieving VS with 180 days. These prevalence ratios were adjusted for characteristics found to be associated with the outcomes in the univariate models. These were insurance, relocating care, prior care evidence, and prior VS, age and gender. To build models and calculate prevalence ratios, we chose 90 days as the outcome for returning to care to assess what happened immediately after receiving the intervention. We chose the slightly longer time period, 180 days, to assess VS as it would likely be take addition time to achieve  following an initial return to care. All analyses were performed using SAS version 9.4.

Results

Patient Characteristics

Between January 1, 2013 and December 31, 2014, 1118 patients were identified as out-of-care and were referred to retention staff at the participating clinics (Table 1). More than half (61%) of the out-of-care patients were identified at Site B, the largest site participating in the program. The out-of-care population was young; 41% of the patients were between 18 and 29 years of age and, overall, 73% were less than 40 years of age. The majority of the patients were black/African American (71%) and male (70%). Half (49%) of the patients reported high risk heterosexual sex as their HIV transmission risk factor and 40% reported male-to-male sexual contact. Approximately half (54%) of patients had some form of health insurance at their last medical appointment. The majority of patients (83%) had received medical care in the 12 months prior to their referral to retention staff and 63% were virally suppressed at the last VL test before being identified for the retention intervention.

Table 1 Socio-demographic characteristics of the out-of-care persons living with HIV (PLWH) and associations with the ability to be located for retention services 2013–2014
Full size table

Characteristics of Located Patients Versus Those Not Located

Of the 1118 patients identified as out-of-care, 80 (7%) were found to be ineligible (26 deceased, 40 incarcerated, 14 with missing outcomes), 326 (29%) were not located, and 712 (64%) were located by retention staff at the clinics. Patients receiving medical care within the last 12 months (aPR 1.43, 95% CI 1.25, 1.66) and patients with VS at the last known HIV RNA measurement (aPR 1.28, 95% CI 1.16, 1.41) were more likely to be located compared to those who did not receive care in the last 12 months or were not virally suppressed at the last known test, respectively. Patients 50 years of age and older were more likely to be found than those under 30 (aPR 1.17, 95% CI 1.02, 1.33).

There was considerable variation in the proportion of out-of-care patients found at each intervention site. In comparison to out-of-care patients in region B (the largest site), patients in region A were more likely to be found (aPR 1.13, 95% CI 1.04, 1.24) while those in regions C and D were less likely to be found by retention staff (aPR 0.62, 95% CI 0.42, 0.91 and aPR 0.84, 95% CI 0.84, 95% CI 073, 0.97 respectively). The proportion of patients found by retention staff was similar between gender and racial/ethnic groups and between those with and without insurance.

Association Between Return to Care and Viral Load Suppression

Of those who were found by retention staff, 26% had relocated, either by moving out of state, to a different region in North Carolina, or transferring to another clinic, while 74% were not known to have moved from the clinic vicinity or transferred their care to a different clinic (Table 2). Fifty-eight percent of out-of-care patients who were located by retention staff returned in care within 90 days of their retention referral, 75% within 180 days and 83% within 1 year (Table 2). However, only 55% met the HAB definition of retention in care 1 year after referral (2 visits/year, > 90 days apart). More than half (51%) were virally suppressed within 180 days of referral, and the majority (64%) were suppressed within 1 year (Fig. 2).

Table 2 Outcomes for all out-of-care PLWH referred for retention services 2013–2014 and care outcomes for all located PLWH
Full size table
Fig. 2
figure 2

Clinical outcomes of all patients referred to clinic-based retention in care intervention. Proportion of patients demonstrating viral suppression prior to referral for retention services, followed by proportion of patients demonstrating viral suppression at 90, 180 and 365 days after referral, based on outcome groups reported by retention staff

Full size image

Predictors of Return to Care and Viral Load Suppression

Those patients who were not known to have relocated had more evidence of care at 90 days (68 vs. 31%, aPR 2.15 95% CI 1.72–2.69) and a higher percentage of VS at 180 days (58 vs. 31%, aPR 1.77 95% CI 1.432–2.21) compared with those who were known to have relocated (Table 3). A higher percentage of 40–49 year-olds achieved VS at 180 days (aPR 1.19 95% CI 1.01–1.40) than 18–29 year olds, but no other demographic factors predicted care at 90 days or VS at 180 days. Having insurance at prior care (56 vs. 38% aPR 1.19 95% CI 1.03–1.38), having a care visit in the year prior to referral (87 vs. 13% aPR 1.37 95% CI 1.05–1.77) and having VS at prior care visit (69 vs. 31% aPR 2.54 95% CI 1.98–3.25) were associated with VS at 180 days of follow-up. None of the patient-level characteristics were associated with returning to care within 90 days of the referral. In the bivariate models, site was predictive of return to care at 90 days and VS at 180 days; however, this association was not seen in adjusted models.

Table 3 Association between characteristics of located PLWH and return to care within 90 days or viral load suppression within 180 days
Full size table

Discussion

This paper describes the outcomes of a simple, sustainable retention intervention that can be performed with existing clinic staff in most HIV clinical settings with access to Ryan White funds. An important finding showed that prior to being identified as out-of-care, most patients had achieved VS (69%) but even 6 months later, VS in the out of care group was only 51%. It was not until the 12th month of follow-up that the proportion of patients with VS approached this same frequency (64% at 12 months post-intervention). This suggests that gaps in viral suppression are occurring frequently and that annual assessments may not be sufficiently nuanced to capture them. In fact, retention-in-care rates have been shown to fall when longer periods of time are examined [33, 34]. This is an important observation because viral copy-years, or the time patients spend with viral loads over the suppression threshold, are associated with added morbidity and mortality [22, 35].

Out-of-care patients on the final referral list in two sites in this study had a lower likelihood of being located, while patients at one site had a slightly higher likelihood of being located compared with the largest site (Site B). The two sites with lower success at finding patients differed by having smaller clinic populations, not being associated with an academic medical center, and the retention staff were actually employed by an external case management agency. A smaller patient volume at these sites may have enhanced the ability to provide increased personal contact to all clients and improve overall clinic retention [36], leading to a smaller out-of-care list consisting of harder-to-find individuals. Additionally, while all of the sites adopted the same outreach protocol once patients were placed on the out-of-care list, steps taken to create and “clean” the list were not standardized. We did not formally assess the cleaning of the preliminary list or variations in the use of brief outreach efforts made prior to “officially” referring a patient to retention staff. If employed, any outreach activities prior to the list finalization and hand-off to retention staff would also produce a smaller list with harder-to-find patients. We did adjust our models for clinic site to account for these differences, noting also that a prior study of standardized clinic-wide messaging was stratified by care site and variations in effectiveness were not seen, suggesting that clinic structure is not always a confounder in retention interventions [37].

Recent past care (< 1 year) was associated with both returning to care and achieving subsequent VS. One likely explanation is that the many barriers to care are often not eliminated even if they are able to be addressed and the same barriers that caused prior disengagement in care may create challenges for future retention in care. Life chaos, psychiatric illness, financial vulnerabilities, etc. are frequently chronic and recurrent. The association between past engagement in care and improved clinical outcomes has been noted in other settings [38]. Additionally, regardless of the cause, this suggests that retention efforts focused on early identification of people at risk of disengagement, or shortly after disengagement, might be more successful at achieving retention in care.

A final study observation was that patients returned to care following the intervention. The intervention was characterized as a “light touch” intervention to distinguish it from intense CM services with weekly contacts for a year—a “heavy touch” intervention [29]. The patient contact in the current intervention was by phone only, and referrals for in-person navigation services, provided outside of the intervention, were made only if a specific need was identified. While an intervention of this sort would benefit from a controlled analysis to assess the results, the findings reported here support the continued development and testing of low-intensity interventions. This intervention was incorporated into clinics as a routine activity and existing funding streams were used for this work. Consequently, this intervention has a high degree of sustainability, confirmed by the continuation of the intervention at all of the participating sites.

A strength of our work is the use of two state-wide data systems which helped limit the effects of movement between care regions. Missing out-of-jurisdiction data is a significant limitation for all retention in care work [39]. To address this, statewide clinical data-sharing for patients at Ryan White funded clinics was implemented. This allowed each site to look across the state for evidence of care prior to placing an individual on the intervention referral list. Additionally, we used state-wide surveillance data for the return to care and VS outcomes for our follow-up analysis. As discussed above, both of these systems still had limitations, but using them together improved the data availability.

A second unique aspect to this report is the description of a systematic monitoring approach for clinics to use to support retention. The CDC reports that despite the recommendation for systematic monitoring, only about half of clinics providing care to PLWH use systematic monitoring [23]. This approach has had limited description in the literature. An exploratory investigation of the out-of-care status of PLWH in a single San Francisco clinic reported that most persons were found to be in care by using surveillance data [39]. Using a similar combined clinic and surveillance data approach in Seattle, investigators reported that 79% of patients were found to be in care [40]. Our study protocol excluded persons in care in other Ryan White locations prior to investigation, allowing the clinic staff to focus on the persons who were more likely to be truly out-of-care.

Limitations

Generalizability of this study is limited to locations where some level of regional or statewide data sharing is present. Our results are expected to be generalizable to other Ryan White clinics if their data-sharing is maximized. Ryan White funding was used to support the time of existing staff to conduct the intervention; clinics without Ryan White funding may not have the funds to support these activities. While CAREWare data-sharing and the use of state-wide surveillance data greatly helped eliminate gaps in data, we know that missing data is still a strong limitation. In the instances that a patient could not be found and/or had no evidence of care in follow-up, we cannot know if this represents failure to engage in care, or simply that they are receiving care in a setting that did not report lab results, such as out of the state. Therefore, we are likely identifying some individuals for intervention who are already in care elsewhere, as well as underestimating the frequency at which patients return to care and achieve VS, a commonly reported issue in other interventions targeting potentially out-of-care patients [39, 41].

We were unable to distinguish between patients who may have moved within NC without returning to care and those who moved out of state because the information was not collected when patient contact was made. Patients found living in the same region as their initial referral were 40% more likely to return to care than those who had relocated (aPR 1.43 95% CI 1.22–1.67), but we are unable to determine what percent of that difference is attributable to missing data in the relocated group. When we excluded all patients with no VL data from the analysis, relocation was no longer associated with care or VS outcomes, but this still does not address how many of those “missing” results are truly missing, and how many are due to relocation out of state or gaps in our surveillance data.

Another limitation is the lack of information on why patients were out-of-care. A variety of barriers to care have been described [19, 42] and in our study both structural (site) and personal (age, insurance) characteristics were associated with outcomes. What is not known is the proportional effect of these barriers on a population that is out-of-care, as we were not able to fully capture all potential barriers to care. Finally, we were unable to identify a comparison group for analysis as a variety of retention in care actives were being actively adopted by all regions in North Carolina over this time. Historical data were not available as complete VL and CD4 data reporting was not mandatory until mid-2013.

Conclusions

Clinic-based retention efforts may be effective at helping PLWH maintain consistent care and decrease or eliminate out-of-care periods, leading to better overall outcomes. Despite having access to state-wide surveillance data for outcomes, relocation remained associated with no evidence of care. This is likely due to disruption in care in conjunction with changing residences and care sites, as well our as lack of access to out of state information for analysis. If identified in surveillance data as entering care in a new location within the state, PLWH were as likely to have VS as those who remained at the same care location. Continued expansion of data sharing will be necessary to increase the accuracy and the utility of out-of-care lists.