Introduction

By year-end 2016, national prevalence in the United States of diagnosed HIV was 306.6 per 100,000 people, or 991,447 people living with HIV (PLWH), 525,374 of whom had ever survived a stage-3 (AIDS) infection [1].While the U.S. South and Midwest regions continue to carry a substantial burden of HIV, the majority of PLWH in the U.S. reside in urban centers, [1] with an estimated 127,823 in New York City (NYC) as of mid-2019 [2]. Mortality rates have fallen substantially over the course of the epidemic (e.g., in 2018, only 1 in 3 deaths among PLWH were attributable to HIV [3, 4]).

Such improved prognoses among PLWH follows the advent of effective antiretroviral therapies (ART). With ART enabling the reliable suppression of HIV, individual morbidity, mortality, and forward transmission can be reliably managed and reduced in turn [5]. But long-term adherence to a daily ART regimen can be challenging for many PLWH [6], and access to HIV treatment services sporadic [7], imposing substantial logistical and self-management demands on PLWH. By 2015, within the 39 states (plus Washington, DC) reporting CD4 and viral load (VL) results to the CDC, even as 73.4% of diagnosed PLWH were able to access some form of HIV care, only 57.2% were retained in care, and only 59.8% had achieved viral suppression [8]. In NYC, the benchmarks of the UNAIDS 90-90-90 initiative [9] have only been surpassed among White PLWH [10]. These racial and ethnic disparities can be attributed to a range of factors, including the abovementioned socio-structural burdens, many of them reflective of broader racial inequities in the U.S. [11,12,13], and some of them, such as historically-grounded medical mistrust [14], largely unique to PLWH of color.

Rates of alcohol and drug use are elevated among otherwise heterogeneous communities of PLWH, with 48% of PLWH in treatment evidencing disordered use of at least one substance, and 20% evidencing disordered use of multiple substances, according to a recent national multisite estimate [15]. Alcohol use is common among PLWH in treatment and is reliably linked to lapses in ART adherence [16,17,18]. Illegal drug use—injectable opiates and cocaine, in particular [19]—are linked consistently to treatment non-adherence [20, 21].Similarly, the use, and especially the hazardous misuse, of alcohol is linked to a range of detriments to PLWH’s health and wellbeing: sporadic medical appointment-keeping [22], sub-optimal treatment adherence [23, 24], and virologic rebound, via inhibited ART adherence [17]. In addition, alcohol is linked independently to increased morbidity and mortality among PLWH, due to the direct effects of disordered drinking [16],CD4 decline independent of ART lapses [25] damaged immune function [26, 27], and increased viral replication [28].

As a significant driver of treatment non-adherence for PLWH, substance use presents a key target for intervention. Approaches grounded in motivational interviewing (MI)—which aims to activate individuals’ inherent capabilities to enact positive behavior change [29]—and cognitive-behavioral skills training (CBST)—which aims to instill insights on the feedback between individuals’ environments, emotions, and decision-making processes [30]—show early promise toward advancing self-management skills among substance-using PLWH. Alone [31,32,33,34], and in combination [35] including in multi-component theoretically integrated interventions [36], MI and CBST have demonstrated efficacy in reducing sexual risk and substance misuse, and in enhancing treatment adherence among PLWH.

Project PLUS is one such intervention. It uses MI and CBST techniques to individualize the provision of tailored informational supports and elicit client-specific motivations for positive behavior change, consistent with the stages of the Information-Motivation-Behavioral Skills (IMB) model [37, 38]. Its initial efficacy was demonstrated in a randomized controlled trial that enrolled 143 racially diverse PLWH on ART who met screening criteria for habitually hazardous drinking. Participants receiving the intervention showed greater VL reductions and boosts in self-reported treatment adherence and laboratory-extracted CD4 counts at three months compared to an educational control condition—though these gains were not sustained longitudinally [38]. Nevertheless, Project PLUS, to our knowledge, was the first intervention to show significant benefits in advancing HIV-related self-management skills among heavy-drinking adult PLWH when evaluated in the controlled setting of an academic research center. A necessary next step for the intervention was to test its real-world effectiveness, particularly within the context of the complex and overlapping needs of an ethnically and racially diverse sample in an NYC-based outpatient clinic network.

In the present study, we tested the effectiveness of an adapted version of Project PLUS, a 6-session combined MI + CBST intervention, as delivered by existing practicing clinicians in a NYC-based network of outpatient clinics serving PLWH, to reduce problematic alcohol and drug use and to improve ART medication adherence and HIV-related immune functioning (VL and CD4 count). The purpose of the current study was to evaluate the effectiveness of the intervention when delivered in real-world settings. It was hypothesized that receipt of the intervention would be associated with decreases in substance use and substance use-related problems at follow-up. Furthermore, it was anticipated that receipt of the intervention would be associated with improvements in ART adherence as well as immune functioning (decreased VL and improved CD4 count) relative to waitlist and cohort control conditions.

Methods

Participants

Recruited participants were eligible if they were: (1) at least 18 years old, (2) living with HIV (3) English-speaking (4) active patients at the clinic (5) currently prescribed an ART regimen but (6) had a VL documented in the medical record of more than 200 copies/mL within the past 12 months and (7) reported drinking alcohol at hazardous levels—operationalized as exceeding 14 standard drinks per week for men or exceeding seven standard drinks per week for women within the past three months—or reported use of illicit drugs (exclusive of marijuana) or illicit use of prescription opioids within the past three months.

Study Design

The Project PLUS study utilized data from participants at a network of hospital-based HIV outpatient clinics in New York City between March 2015 and May 2017. The original efficacy trial evaluated the Project PLUS intervention relative to a time- and content-matched education control condition. This represented a highly competitive comparator condition that could be construed as an alternative intervention in and of itself because both ART adherence and hazardous drinking were addressed in education content. The original study authors speculated this may have contributed to challenges in detecting effects on alcohol consumption [38]. Consistent with the goals of an implementation study, we compared the Project PLUS intervention to the standard of care typically experienced by patients at the participating clinics—and diminish the likelihood that specialized content in a comparison condition would obfuscate the benefits of the intervention relative to treatment as usual. This was achieved through two methodological elements of the study—a Waitlist control condition and the collection of electronic medical record (EMR) data from matched patients not participating in the trial (hereafter referred to as No-Intervention control).

The three study conditions (Immediate delivery, Waitlist control, and No-Intervention comparison cohort) were distributed across these four sites in a quasi-experimental design. In the Immediate delivery condition, participants were offered the intervention immediately after their baseline assessment, and sessions were completed before their next assessment point (at 3 months post-baseline). Participants then completed follow-up assessments at 3, 6, 9, 12, 15, and 18 months post-baseline. In the Waitlist condition, participants received an enhanced treatment-as-usual condition (eTAU)—which, in addition to TAU, included two handouts with printed information about HIV, the importance of ART adherence, and problematic alcohol use and HIV disease progression—during the first 12 months of their participation, and then received the intervention after their 12-month follow-up. In the No-Intervention comparison cohort condition, participants received TAU with no research study contact. Participant data for this TAU site were drawn from EMR.

Sites 1 and 2 delivered the Immediate condition with an intended target of enrolling 120 participants, while Site 3 enrolled participants into the Waitlist condition. However, due to challenges in participant accrual at Sites 1 and 2, Site 3 shifted to enrolling participants in the Immediate intervention condition during the course of the study. In other words, Site 3 implemented the Waitlist control from March 26, 2015 to October 31, 2016, and then the Immediate condition from November 1, 2016 through the end of the study. Site 4 provided the No-Intervention comparison cohort condition.

Selected sites were all affiliated with the same clinic network. Therefore, their organizational traits, services, and staff training are comparable. Many of these practitioners also work at more than one of these clinic sites, further standardizing levels of experience and skillsets between these clinics. This clinic network spans the New York City area and therefore does have slight differences in patient demographics. All sites serve predominantly publicly-insured patients with the average patient age in the mid-40 s. Sites 1, 2, and 4 are based in predominantly white neighborhoods but see an economically and racially diverse patient population from across the city. Sites 1, 2, and 4 predominantly serve individuals identifying as sexual minority men, or men whose sexual histories include encounters with other men. Site 3 is located in a more racially diverse neighborhood and serves a larger percentage of heterosexual-identifying patients and a larger proportion of women.

Enrollment in Immediate and Waitlist Control Conditions

Potential participants in the Immediate and Waitlist control conditions were identified through a prescreening review of the EMR, which provided study staff members with lists of patients with scheduled clinic visits who had one or more detectable VL within the past year. Study staff approached these patients during clinic visits to introduce the study and invited them to participate in screening. Participants provided verbal consent for screening and provided written informed consent if they were eligible and interested in enrolling. They also granted permission for the study team to extract their EMR data for study screening and enrollment purposes.

Once consent to use the EMR was granted, eligibility criteria were verified for ART regimen and VL. Participant self-report was used to determine alcohol and substance use criteria on the screening questionnaire. Eligible participants who enrolled in the study were scheduled for a baseline visit, which could take place the same day post-screening or be scheduled within the ensuing month. The full protocol was approved by the CUNY University Integrated Institutional Review Board (CUNY-UI IRB) and the IRB for the Institute for Advanced Medicine, Mount Sinai St. Luke’s and Mount Sinai West Hospitals.

Selection of No-Intervention Control Patients

For the purposes of providing a natural history comparison group for the outcomes of VL and CD4, we accessed the de-identified EMRs of 120 randomly-selected patients who received care at Site 4 (i.e., a 4th clinic that did not receive any intervention or eTAU). The 120 patients were randomly selected from a larger pool of patients who were deemed eligible if they: (1) had an HIV diagnosis, (2) attended at least one primary care visit at Site 4, (3) received ART, and (4) had a VL greater than 200 copies/mL at any point during the enrollment period from March 26, 2015 to March 16, 2017. EMRs of 20 patients who were enrolled in the study and possibly transferred to receive HIV care at this 4th clinic during the study period were excluded. Additionally, records of nine patients with only one primary care visit and no follow-up visits during the study period were excluded from the pool of patient records.

Using EMR data for 686 patients at Site 4 meeting the criteria described above, case control matching was conducted to generate a pool of 157 patients from which we randomly selected the comparison cohort of 120 patients. Exact matching with the intervention sample was conducted using key variables: age group (with ranges 18–29, 30–39, 40–49, and 50 or older), racial and ethnic identity (Black, Hispanic/Latinx, White, Asian/Pacific Islander, Other/Multiracial), and gender (recorded as male or female). Additionally, we matched on indicators of substance misuse based on EMRs with documentation of substance use problems or ICD-10 diagnostic codes, such as documentation of alcohol abuse or dependence, having had a substance use diagnosis, or any indication of substance misuse during the study period. EMRs were then extracted into a de-identified dataset that linked patient EMRs to a unique study ID number with the following variables: demographic characteristics, HIV diagnosis, HIV clinical outcomes, ART regimen, ICD-10 diagnoses of AIDS-defining illness, substance use disorders, psychiatric disorders, non-AIDS defining infections (bacterial pneumonia, cellulitis, sepsis), cardiovascular, gastrointestinal, liver, kidney, pulmonary, endocrine, immune, and oncologic diseases, as well as sexually transmitted infections. In addition, records about services received and type of outpatient clinic visits were extracted, such as whether the visit was an HIV primary care visit, a specialty or non-HIV primary care visit type (dentist, OB/GYN, hematology), mental health visit, or substance use-related visit.

Intervention

The Project PLUS intervention was adapted from the previous iteration that proved to be efficacious in reducing VL and improving CD4 and ART adherence [38]. Adaptations to the intervention were informed by the results of the initial efficacy trial [38] as well as pre-study interviews and feedback groups with the existing mental health personnel who would be trained as intervention clinicians. In the initial efficacy trial, the effect of treatment attenuated at the 6-month follow-up. Parsons et al. [38] speculated that the addition of booster sessions would increase effects on longer-term outcomes. In addition, they suggested that the intervention could be expanded to encompass a focus on drug use in addition to hazardous drinking.

In the pre-trial phase of the study, research staff undertook the process of evaluating and refining the original intervention for use in the specific context of the clinics involved in this trial. These results are detailed elsewhere [39]. Staff interviewed 12 mental health providers, 8 administrative staff, and 11 non-clinicians involved in resource provision for patients. In these pre-study interviews and feedback sessions, mental health providers concurred with the Parsons et al. [38]. suggestion that the intervention’s scope be expanded to include a focus on reducing drug use [39]. The manual and training materials were modified to encompass this expansion. Feedback from providers regarding the acceptable dose of treatment contradicted the recommendations of Parsons et al. [38]. The providers involved in the implementation trial felt strongly that delivery of 8-sessions would present challenges within the context of their system of care. Boosters were perceived as increasing, rather than alleviating, the burden of treatment delivery. Their recommendation to reduce, rather than expand, the session content was based on four emergent insights: (1) a longer treatment diminished staff’s recordable output (which was evaluated based on number of different patients seen in a given period); (2) 8-sessions would be impractical for patients who typically attend clinic approximately once every 3 months; (3) the burden of 8-sessions would deter patients from enrolling given the level of commitment required, and 4) an 8-session intervention was perceived by some therapists as sufficiently intensive that it exceeded what they were ready to deliver [39]. Based on their feedback, the original intervention was reduced from eight 1-h sessions to six 45-min sessions, in order to improve scalability in a clinic setting and to address concerns that the original 8-session iteration may appear too burdensome for prospective participants or clinicians.

The sessions followed a uniform schedule across sites and clinicians. Session 1 utilized motivational interviewing to establish treatment goals related to the participant’s substance use and ART adherence. During Session 2, clinicians and the participant collaborated on a functional analysis of the participant’s substance use and missed ART doses. Sessions 3–5 utilized CBST modules selected to match the participant’s goals and to address the skills deficits identified in the functional analysis. Session 6 focused on relapse prevention.

Modules covered in Sessions 3–5 were selected by clinician and participant consensus from a menu of 11 modules created in the original efficacy trial that centered on alcohol/drug use and adherence [38]. Though the session number was reduced for the present trial, the complete menu of available modules, and the content they contained, were not reduced. Modules tailored to alcohol/drug use included: basic skills to avoid drinking or drug use, moderating substance use, managing cravings, managing thoughts, and drinking/drug refusal skills. Adherence-centered modules included: basic skills to avoid missing medications, communication skills with providers, managing thoughts about adherence, managing side effects, making time for yourself, and treatment as prevention. These modules utilized a handout that provided education on the topic and an activity or exercise for the clinician and participant to work through together. For example, the managing cravings module discussed how cravings and urges occur and how a participant could minimize triggers and cope more effectively with those urges. The clinician and participant would discuss techniques together, often write them on the handout, and discuss the potential for utilizing them in the future. Participants left the session with the handout, including the written plan.

Intervention Fidelity

All sessions were audio-recorded with the participant’s consent, and randomly selected sessions were reviewed to provide feedback to the clinicians about their fidelity to treatment delivery (i.e., using MI and delivering the CBST content). Independent coders utilized the Motivational Interview Treatment Integrity (MITI) coding system to evaluate the clinician’s use of MI [40]. CBST fidelity was monitored by use of a checklist to ensure that components were delivered within each session (either yes, and in detail; yes, but only cursory; or no), such as the reviewing of previous session’s homework, and the applying of content to the participant’s specific circumstance. MITI coding was applied to a random selection of 15% (1-in-4) of the recordings of Sessions 1, 2, and 6, and CBST monitoring was applied to a random selection of 15% (1-in-4) of the recordings of Sessions 3, 4, and 5. Clinicians received group supervision, and individualized feedback was provided if fidelity coding indicated inadequate delivery.

In total, 141 sessions were eligible for MITI coding and 21 were reviewed. Summary scores uniformly indicated fair proficiency in MI delivery. The average Technical Global score was 3.6 (SD = 0.50), which is above the threshold of 3.0 that characterizes fair performance. Similarly, the average Relational Global score was 3.8 (SD = 0.61), also above the threshold of 3.5 for fair performance. Likewise, the average reflection to question ratio was 1.09 (SD = 0.74), just exceeding the threshold of 1.0 for fair performance and 41.9% (SD = 24.7) of providers’ reflections were complex on average, just exceeding the threshold of 40% for fair performance.

Overall, 83 sessions were eligible for CBST coding, of which 13 were reviewed. Of these, 3 sessions were rated as delivered in full. An additional 7 sessions had minor deviations from protocol. The most common deviations were failure to remind participants of the diary card (observed in 4 sessions) and missed opportunities to connect module content to participant’s lives (also observed in 4 sessions). In 3 sessions, extensive deviations from CBST module adherence were observed.

Measures

Demographic Information

Participants reported various demographic characteristics in their baseline survey, including their age, education level, race and ethnicity, gender identity, sexual orientation, and years since HIV diagnosis. For participants in the No-Intervention control group, age and gender were extracted from their EMRs.

Biological Indicators of Immune Functioning

For all four sites (i.e., both study-enrolled participants and the No-Intervention control), standard-of-care HIV-1 VL and CD4 cell counts were run using local laboratories and were abstracted from EMR data. VL was log-transformed for the purposes of analysis.

Survey Measures Completed by Participants in Immediate and Waitlist Control Conditions ART Adherence

The visual analogue rating scale (VAS) was used to ask participants to estimate on a scale from 0 to 100%, how much of the time they had taken their ART medications as prescribed in the past 30 days [41]. This single-item scale has been used in previous studies of ART adherence [42].

Alcohol Use Disorders Identification Test (AUDIT)

The AUDIT is a 10-item screening questionnaire consisting of three questions related to alcohol drinking frequency, three questions on dependence, and four on problems caused by alcohol use [43]. Scores range from 0 to 40, with higher scores indicating greater alcohol use. The AUDIT was administered at screening and during follow-up assessments and displayed strong internal consistency at baseline (α = 0.88).

Drug Abuse Screening Test-10 (DAST-10)

Drug use severity was assessed by the 10-item version of the DAST [44]. The items ask participants whether they have experienced a range of problems associated with their drug use (e.g., “do people you are close to ever complain about your involvement with drugs?), with response options of yes or no. The total score ranges from 0 to 10 and displayed acceptable internal consistency at baseline (α = 0.78). The DAST was administered at screening and during follow-up assessments.

Statistical Analyses

To assess the effectiveness of our randomization procedure, we used χ2 tests and one-way ANOVA’s to investigate the equivalence of possible cofounders across each of the study-enrolled groups (Site 1 Immediate, Site 2 Immediate, Site 3 Immediate, and Site 3 Waitlist) at baseline. Specifically, we examined proportional differences across demographic characteristics (race and ethnicity, sexual and gender identity, and education), and mean differences across age and baseline outcome values (alcohol severity, medication adherence, VL, and CD4). For DAST-10 scores, pairwise tests of estimated marginal means were conducted using the generalized linear model function in SPSS to specify a negative binomial distribution. Where cell sizes were too small (n < 5) to permit χ2 tests, we utilized a Fisher’s exact test. The matching of the No-Intervention control group at Site 4 (N = 120) was assessed by bivariate comparisons with the entire study-enrolled sample (N = 174) on the available variables of age, race and ethnicity, and gender.

We also conducted analyses to test for the presence of differential attrition between conditions and sites where appropriate. First, we conducted χ2 analyses to test for differential attrition by site and condition (i.e., Site 1 Immediate, Site 2 Immediate, Site 3 Immediate, and Site 3 Waitlist) at each follow-up, as well as several demographic characteristics (race and ethnicity, sexual and gender identity, and education). In addition, we conducted a series of independent samples t-tests to examine attrition at each follow-up by age and baseline outcome values (alcohol severity, medication adherence, VL, and CD4). Finally, the generalized linear model component of SPSS was utilized to test whether baseline DAST-10 scores were significantly associated with follow-up completion specifying a negative binomial distribution.

Outcome analyses were conducted using piece-wise longitudinal growth curve models following procedures outlined by Chou et al. [45], adjusting for baseline scores on the corresponding outcome. For the models predicting medication adherence, AUDIT, and DAST, we adjusted for gender identity and sexual orientation, whereas for the models predicting VL and CD4 we adjusted for gender (male, female) as sexual orientation was not indicated in the EMR records for the matched cohort. DAST scores were treated as having a negative binomial distribution. The latent intercept represented the initial 3-month follow-up time point. Slope 1 quantified linear change across the 3-, 6-, 9- and 12-months follow-ups. Where possible for outcomes variables extracted from EMR (i.e., VL and CD4 count), Slope 2 quantified changes in trajectory at the 15- and 18-month follow-up corresponding to the point at which the Waitlist control condition received the intervention. The fixed effect of site and condition was entered as a 4-category predictor that was dummy-coded. For VL and CD4 models—where data were taken from EMR records—the No-Intervention control patients were included as an additional subgroup and served as the referent category. All models were estimated using full-information maximum likelihood estimation in Mplus Version 8.0 [46]. Good model fit was assumed when the χ2/df ratio was 3 or less, root-mean square error of approximation (RMSEA) ≤ 0.05, Tucker-Lewis fit index (TLI) > 0.95, and comparative fit index (CFI) > 0.95.

Results

Demographic data for the 174 participants enrolled in the study are presented in Table 1, as well as for the 120 No-Intervention patients on the variables available in their EMRs. The consort diagram for enrolment into the trial is depicted in Fig. 1. Tests of between-group differences indicated no significant differences across sites and conditions with respect to race and ethnicity, age, education, and four of the five outcome variables measured at baseline (AUDIT score, DAST score, CD4, and VL)—however, medication adherence and composition of gender and sexual identity did differ among the clinics. For medication adherence, both of the conditions at Site 3 comprised participants who, on average, scored higher on medication adherence than at Sites 1 and 2; F(3, 170) = 5.20, p = 0.002. Differences by site were also indicated for gender and sexual orientation, whereby a greater proportion of participants at the Site 1 identified as sexual minority men, compared to the two conditions at Site 3; χ2 (6) = 22.07, p = 0.001. In subsequent analyses of the medication adherence, AUDIT, and DAST outcomes, sexual orientation and gender identity was included in the growth curve models, with women as the referent category. In addition, Table 1 also displays the comparisons on age, racial and ethnic identity, and gender between the entire study-enrolled sample and the matched No-Intervention control group, whereby one significant difference was observed: the No-Intervention control group had, on average, a higher VL; t(288) = − 3.60, p < 0.001.

Table 1 Demographic characteristics and baseline outcome values across conditions and sites
Full size table
Fig. 1
figure 1

CONSORT Flow Diagram

Full size image

There was no evidence of systematic differential attrition across the 3-, 6-, 9-, and 12-month follow-ups. The probability of follow-up completion was not significantly associated with site and condition, nor was it significantly associated with any demographic covariates. Likewise, neither AUDIT, DAST, nor medication adherence scores were significantly associated with follow-up completion through 12-months post-baseline. No demographic characteristics were associated with 15- or 18- month follow-up completion. Neither were AUDIT or DAST scores; however, participants who did not complete the 15-month follow-up reported significantly lower ART adherence at baseline. Data for the analyses of CD4 and VL were provided by EMR and did not rely on attendance at follow-ups.

Medication Adherence

For self-reported ART medication adherence, intervention condition among the 174 study-enrolled participants was not significantly associated with differences in adherence scores immediately post-intervention. With regard to between-group differences in trajectory over time (Slope 1), Site 1 showed a significantly more positive trajectory over time, relative to the Site 3 Waitlist control condition (B = 6.34; p = 0.03); however, the other two groups (Site 2 and Site 3 Immediate) did not differ significantly from Site 3 Waitlist. One-between group difference at 12-months was noted whereby Site 2 had significantly greater adherence scores at the 12-month follow-up compared to Site 3 Waitlist (B = 20.32; p = 0.01), as depicted in Fig. 2. Table 2 contains full model results for this, and for AUDIT and DAST models.

Fig. 2
figure 2

Trajectories of ART Medication Adherence, N = 174

Full size image
Table 2 Results of latent growth curve models predicting ART medication adherence, AUDIT, and DAST (N = 174)
Full size table

Alcohol Use

For alcohol use severity (AUDIT), intervention condition was significantly associated with differences in AUDIT scores immediately post-intervention, but in a direction opposite to what was expected for intervention delivery. Sites 1 and 3 (where the intervention was immediately delivered) had higher AUDIT scores at 3-months relative to Site 3 Waitlist (B = 3.19, p = 0.02 and B = 4.17, p = 0.02, respectively). Intervention condition was not significantly associated with trajectory over time post-intervention. As a result of non-significant drifts in trajectory over time, between-group differences attenuated over the follow-up period. Groups did not differ significantly on AUDIT scores at 12-months, as depicted in Fig. 3.

Fig. 3
figure 3

Trajectories of Alcohol Use Severity (AUDIT) scores, N = 174

Full size image

Drug Use

For drug use severity (DAST), intervention condition was not significantly associated with differences in DAST scores post-intervention for the sites receiving immediate delivery relative to Site 3 Waitlist. However, Site 2 showed greater reduction across time from 3- to 12-months (Slope 1) relative to Site 3 Waitlist (B = -0.32, p = 0.01), and lower DAST scores at 12-months compared to Site 3 Waitlist (B -1.53, p < 0.001). The trajectories of the remaining sites did not differ significantly from the referent Site 3 Waitlist (Slope 1) and they did not differ in DAST scores at 12-months, as depicted in Fig. 4.

Fig. 4
figure 4

Trajectories of Drug Use Severity (DAST) scores, N = 174

Full size image

CD4 and Viral Load

In the models predicting CD4 and log-VL, the four study-enrolled groups were compared against the matched No-Intervention comparison cohort who did not receive the intervention (see Table 3). CD4 models provided no indication of between-group differences at 3-month follow-up, nor was there indication of between-group differences in the trajectory over the first 12 months, or the final 6-months of follow-up relative to the No-Intervention comparison cohort. Further, this was also true when comparing the Immediate delivery groups (Site 1, Site 2, and Site 3 Immediate) to Site 3 Waitlist.

Table 3 Results of latent growth curve models predicting CD4 and viral load outcomes (N = 294)
Full size table

Similarly, VL models provided no indication of between-group differences at 3-month follow-up nor in trajectory over the first 12-months of follow-up. Between-group differences at 18-month follow-up were uniformly non-significant and no significant change occurred for the Waitlist participants relative to the other groups after their receipt of the intervention at 12 months (depicted in Figs. 5, 6).

Fig. 5
figure 5

Trajectories of CD4 count, N = 294

Full size image
Fig. 6
figure 6

Trajectories of log-transformed Viral Load, N = 294

Full size image

Discussion

Across outcomes, no hypothesized treatment effects were observed for the Project PLUS intervention. Results of this implementation trial therefore provide only limited evidence that the Project PLUS intervention was associated with improvements in HIV-related outcomes or reductions in substance use relative to standard-of-care services at the clinics involved. There was evidence of some modest improvement over time observed in self-reported medication adherence at one of the three immediate intervention sites relative to Site 3 Waitlist. Similarly, there was evidence of modest improvements in DAST scores over time at one immediate intervention site relative to Site 3 Waitlist. Results for other primary outcomes (AUDIT, CD4, and VL) suggested that trajectories were uniform across conditions—and although AUDIT scores differed at 3-months for two sites, no differences remained at 12-month follow-up. These findings point to the importance of clinic and community context when considering the impact of intervention implementation on health outcomes. They also highlight the utility of incorporating No-Intervention control conditions to quantify the effects of history in these contexts.

A range of implementation factors might account for this absence of substantial between-condition differences. Interpreted via the Consolidated Framework for Implementation Research (CFIR) [47], these factors include: (a) an outer setting characterized by prevailing population-level successes in adherence and viral control among PLWH at the time of the trial [47, 48], (b) an organizational climate at sites in which perceptions of the intervention’s relative advantage may have been tempered, (c) a shortened course of treatment that may have inadvertently attenuated a core component of the intervention and (d) modest proficiency in MI and fidelity to CBST modules.

The role of unmodifiable outer-setting factors in determining the success of demonstrated interventions within specific contexts remains, broadly, under-studied [49, 50]. But the influence of the relatively robust public health infrastructure aiding PLWH and those newly diagnosed with HIV in NYC and New York State at the time of the trial is likely to have undergirded key policy-level, fiscal, and cross-sectoral facilitators of individual treatment success [48]. The presumptive achievement of NYC in reaching the UNAIDS 90-90-90 goals has occurred against the backdrop of this sustained population-level HIV response, alongside of which stage-3 AIDS diagnoses continued a steady year-over-year decline [2, 9] and historical gains in viral suppression among PLWH in care were sustained [2, 48, 51]. In short, non-study care and resources, as well as increasingly simplified ART regimens (e.g., one pill per day), may have contributed to a “rising tide” of adherence outcomes within the broader implementation setting, uniformly benefiting individuals enrolled across study sites and conditions. Though this trend may have dampened or obscured the active ingredients of the intervention relative to eTAU provided in the Waitlist condition and TAU provided in the No-Intervention comparison cohort, it is, of course, ultimately to the advantage of many PLWH.

Simultaneously, there are two plausible provider-level factors that may have mitigated the effects of the Project PLUS intervention relative to both the Waitlist and No-Intervention comparison cohorts. First, fidelity monitoring data indicated fair MI proficiency and modest adherence to CBST module protocols. Inconsistency in intervention delivery may have attenuated effectiveness. Second, the competitiveness of the Waitlist and No-Intervention comparison cohort may have been bolstered by the interlocking range of skillsets that site-affiliated practitioners brought to all conditions. These practitioners contributed substantial experience in clinical psychology, psychiatry, social work, counseling, and treatments for substance misuse, with TAU non-study care often open-ended and individually responsive to emergent patient needs [39]. These organizational traits almost certainly aided in the legitimacy, compatibility, and fit of the intervention [47]. But they may, too, have tempered interventionists’ expectancies of the intervention’s relative advantage over the prevailing standard of care [52,53,54]. In contrast to trials in which the MI + CBST active ingredients were assiduously avoided in otherwise intensive attention-matched controls [34, 38], some background interference—though not direct cross-condition contamination[53, 55]—may have muted any distinct effects otherwise attributable to the immediate delivery. Indeed, most participants (84%) in the Site 3 Waitlist condition continued to receive some level of mental health support at their clinic during their 12-month waitlist period as part of their non-study care. On average, participants received 6.2 visits (median = 3) that were coded in their EMRs as either psychiatry, psychology, mental health, or social work. Similarly, expectancy may have been strong among the Waitlist group, or individual predispositions among those participants amenable to waitlisting, such as enhanced agreeableness, may have encouraged responsivity to the eTAU components [52, 53].

Any number of these inner-setting dynamics could operate in tandem with the abovementioned external-setting factors. Of course, these factors cannot be assumed of all—or even most—implementation settings. Indeed, it is plausible that, in a more under-resourced setting, the relative advantage of the active MI + CBST ingredients would be more plainly evident. Rural settings, in particular, in which a climate of interagency collaboration (a construct the CFIR defines as “cosmopolitanism” [47]) can be lacking, and workforce constraints acute [56, 57], might be especially receptive to adaptations of the intervention.

Such adaptations would benefit from the findings of finer-grained components analysis, an important future direction of research, which might aid in delineating the core components from the adaptable peripheries of the intervention [47, 58]. Among the elements of the intervention that were handled as peripheral, and adapted to the implementation setting, the reduction in session length and total number (and hence, the dose) may have attenuated its effectiveness. The decision to abbreviate the intervention was made in response to provider feedback. While the modification ran counter to the recommendations of the original efficacy trial [38], it was deemed essential in order to reduce anticipated provider and patient barriers to implementation. In this way, the adaptation embodies the tension between acceptability and potential effectiveness. No intervention can be effective it is not sufficiently acceptable to be implemented; however, in shortening the intervention, mechanisms of its efficacy in a controlled academic setting may have been lost or further diminished [38]. Session number has been shown to influence the effect size of depression treatments on ART adherence among PLWH [59]. In a recent meta-analysis of psychosocial interventions focused on sexual minority men [60], stronger effects on mental health were noted for interventions of 9 or more sessions, compared to briefer interventions, but this difference in strength of effect was not noted for drinking, drug use, and adherence behaviors.

It is plausible that, while the intangible “spirit” that animates MI may survive adaptation [61,62,63], the concrete skills-building content that CBST aims to impart may be more brittle to adaptation. The problem-solving strategies that infuse well-demonstrated CBT-aligned interventions, such as those that incorporate and adapt the single-session Life-Steps program [30, 64,65,66], may be particularly inelastic to change. Again, these hypotheses point to the promise of standardized components analyses for complex behavior-change interventions [58], in order to more closely discern their effects from the influences of the implementation context.

The findings of this study must be understood in the context of the methodological limitations of the study design itself. The internal validity of quasi-experimental designs—the extent to which they provide evidence that can support strong causal inferences—is diminished relative to experimental trials in which random assignment to condition serves to equate groups [67]. In this trial, participants were not randomly assigned to condition or to clinic site. Furthermore, clinic site and condition were fully conflated in this design. Sites 1 and 2 implemented only the immediate intervention condition; whereas Site 3 enrolled only into the Waitlist control condition initially and then switched to Immediate intervention condition midway through the trial. These methodological challenges were unanticipated at the time of study design and were necessary to respond to developments in the clinic-context that were beyond the control of the research team. It is therefore possible that unmeasured differences at the patient, provider or site level as well as historical events impacting outcomes at each site may have obscured treatment effects.

A number of additional limitations should also be noted. Alcohol and drug use were self-reported, and not objectively measured. Accordingly, it is possible that some degree of social desirability may have biased survey responses, and future studies should consider objective testing. Future studies should also consider further efforts to minimize the substantial attrition in our study of participants across the follow-up assessment period. In future multi-site studies, researchers should ensure greater comparability of the populations recruited across the sites, given some of the noted demographic differences we observed between our clinics (e.g., some having fewer women than others) as displayed in Table 1.

Conclusions

While delivery of the intervention was not associated with substantive sustained improvements in the outcomes assessed in this trial, the study illustrates several methodological elements of potential utility to future intervention research. First, the employment of a comparison cohort provides a low-cost opportunity to quantify the impact of assessment effects on participant behavior. Second, these results highlight the importance of considering the quality of standard of care treatment delivery when selecting an environment for implementation research.