Introduction

Detecting and addressing sub-optimal adherence to antiretroviral medications (ARVs) is an essential component of HIV clinical care. Adherence to ARVs is one of the most important determinants of durable HIV viral suppression and is critical in the prevention of drug resistance, disease progression, and death [1–11]. Inadequate adherence to ARV medication regimens is under-detected by providers [3, 12, 13] and is often recognized only after virologic failure. Currently, there is no widely accepted and easily implemented standardized approach to the routine assessment of ARV adherence in routine clinical care settings [14].

Objective measures of medication adherence used in research settings include detection of drugs or metabolites in body fluids, pill counts, and electronic drug monitoring devices [15]. Each has advantages and disadvantages, but due to expense, complexity and intrusiveness, they are typically not practical for routine use in clinical care [16, 17]. Self-reported ARV adherence measures are the most practical method of measuring adherence at the point of care [15, 16, 18–20]. Self-reported measures have many advantages including low cost, minimal patient burden, ease of administration, flexibility in timing and mode of administration, and they are associated with both objective adherence measures and most importantly, HIV-1 viral load [14, 16, 21–32]. However, self-reported adherence measures have several potential limitations, including a requirement for accurate recall of past events and concerns that patients may over-report their actual level of ARV adherence [16, 18, 23, 32–39] (for a review, see Wilson et al. [40]), which can lead to ceiling effects in which the majority of patients report perfect adherence [14].

Despite an extensive literature on self-reported ARV adherence measures, critical questions remain regarding their use, particularly in the context of routine clinical care [39]. Previous studies used measures with very short time frames that may not represent overall behavior or used complicated measures that may be too cumbersome in clinical care settings (i.e. repeating a series of items for each individual ARV in a regimen) [27]. A clinically useful self-reported adherence measure should be brief, easy to administer, and result in accurate responses. It should have simple-to-understand response options that make general, not detailed, memory demands of the respondent. The validity of self-reported ARV adherence measures meeting these criteria has been a focus of recent research [39, 41, 42]. In the context of a clinical trial, Lu and colleagues [39] compared several self-reported adherence items to adherence assessed with medication event monitoring system (MEMS) caps, which track each time a pill bottle is opened. They found that, compared with the MEMS caps, the self-rating scale item (SRSI), a single-item Likert-type self-report rating of ARV adherence over a 1-month period, seemed to fare best in that it yielded less over-reporting of adherence than other self-report measures [39]. A second study found that among 46 predominantly Hispanic HIV-infected patients, the SRSI did not have the highest correlation with MEMS caps among the instruments compared, including the visual analogue scale (VAS) [43]. Another study of 53 methadone-maintained individuals found that the SRSI was one of several measures that correlated with viral load, but that in addition, it had the smallest ceiling effect of the measures compared [44]. This study highlighted the importance of item selection as estimates of adherence can vary based on the individual items asked [44]. These studies provide important information regarding the validity of this measure but are limited by their small sizes and focus on somewhat unique populations of patients within the context of clinical trials.

The purpose of this study was to build on prior validation work on the SRSI and extend the evaluation of validity to routine clinical care. To accomplish this, we used data collected on touch-screen tablets or personal computers as part of clinical assessments of patient reported data or outcomes (PROs) in two routine clinical care settings [45]. We compared responses from the SRSI with responses from other adherence measures to assess concurrent validity. We also compared associations of the SRSI and other adherence measures with barriers to adherence such as substance use and depression as a further measure of concurrent validity. To assess predictive validity, we compared associations of adherence measures, including the SRSI, with adherence-related clinical outcomes such as viral load and CD4 cell count. Because of the importance of respondent burden in clinical care settings, we also compared completion times for the SRSI and the VAS, a single-item measure that is commonly used when respondent burden must be minimal. We hypothesized that the SRSI would not require substantially more time to complete than the VAS. We also predicted that the SRSI would correlate well with other adherence items, would be inversely correlated with known barriers to adherence such as substance use or depression, and would be as good as or better than other adherence items at predicting viral load and CD4 cell count.

Methods

Study Setting

This study was conducted among patients from the Centers for AIDS Research Network of Integrated Clinical Systems (CNICS) cohort. CNICS is a longitudinal observational study of HIV-infected patients receiving primary care between 1/1/1995 and the present [46]. CNICS comprises eight clinical sites, with a large and diverse population of HIV-infected patients [46]. Data from the University of Washington (UW) and the University of Alabama, Birmingham (UAB) were included in these analyses. Study procedures were approved by both Institutional Review Boards.

Data Collection

The CNICS data repository captures longitudinal data on the CNICS cohort. It integrates comprehensive clinical data from all outpatient and inpatient encounters, including a routine clinical assessment of PROs. The data repository incorporates demographic, clinical, laboratory, medication, and socioeconomic information obtained from each site’s electronic health record and other institutional data sources. Medication data are entered into the electronic health records by clinicians, or prescription fill/refill data are uploaded directly from pharmacy systems and verified through health record review. Laboratory data are uploaded directly from the clinical laboratory systems at each site.

Since 2006, consenting patients have completed clinical assessments of PROs prior to routine physician visits at approximately 4- to 6-month intervals. Patients use touch-screen tablet or PC computers and a web-based survey software application developed specifically for PROs [47–49]. Patients who do not speak English or Spanish, or who have obvious cognitive impairment, are excluded from the assessment.

Study Participants

HIV-infected patients over 18 years of age, receiving ARVs, who attended routine clinical care appointments between December 2006 and April 2011 and completed the clinical assessment were eligible for the study. For analyses that used a single time-point, data from the most recent assessment were used for patients who completed multiple assessments during the study period. For longitudinal analyses, all assessments during the study period were included.

Clinical Assessment Measures

ARV Medication Adherence

In addition to the SRSI [39], patients responded to a number of additional widely used medication adherence items including a 30-day VAS [27, 50], and several Adult AIDS Clinical Trial Group (AACTG) adherence items [27, 51].

  1. (1)

    SRSI To complete the SRSI, patients rated their ability to take their HIV medications over the past 4 weeks. The six response options are “very poor,” “poor,” “fair,” “good,” “very good,” and “excellent.”

  2. (2)

    VAS Individuals were asked to rate the percentage of their past month’s adherence by marking a point on a scale from 0 to 100 %.

  3. (3)

    AACTG The AACTG adherence items included inquiries about the number of missed doses over the past 4 days (6 categories ranging from “0” to “more than 4”) and the previous weekend (“yes” or “no”), as well as the time of the last missed dose (six categories ranging from “within the past week” to “never miss doses”).

Illicit Substance Use

We used the Alcohol, Smoking, and Substance Involvement Screening Test (ASSIST) to assess patients’ illicit substance use [52, 53]. There are several ways to score the ASSIST [52, 53]. Because of the association between illicit substance use and adherence [54, 55], we were interested in current use, which we analyzed as a binary variable indicating any illicit substance use (excluding marijuana) in the past 3 months. In addition, we looked specifically at any cocaine/crack use, any amphetamine use, and any opiate use in the past 3 months.

Alcohol Use

We used the abbreviated version of the Alcohol Use Disorders Identification Test consumption questions (AUDIT-C) to identify patients’ at-risk alcohol use [56, 57]. Consistent with previous use, we used a score of 5 or higher for men and 4 or higher for women to define at-risk alcohol consumption [58].

Depression

The 9-item patient health questionnaire (PHQ-9) from the PRIME-MD [59, 60] measures depressive symptoms experienced over the previous 2 weeks (e.g., “Please indicate how often over the last 2 weeks you have been bothered by little interest or pleasure in doing things”) with three response options ranging from “Not at all” to “Nearly every day”. We scored the PHQ-9 as a severity measure with total scores ranging from 0 to 27 [60].

Current CD4+ Cell Count and HIV-1 RNA Levels (Viral Load)

CD4+ cell counts and HIV viral loads were measured as part of routine clinical care. We considered the first laboratory value at the time of, or up to 60 days after, the clinical assessment to be current and conducted sensitivity analyses using 30- and 90-day cutoffs. We categorized CD4+ cell count results as <200, 200–349, and ≥350 cells/mm3, and HIV viral load results as detectable or >400 copies/ml vs. undetectable or ≤400 copies/ml.

Statistical Analyses

To ensure that study samples represented the overall clinic populations, we performed bivariate analyses comparing study participant characteristics to the UW and UAB HIV-infected cohorts using Chi-squared tests for categorical variables and t tests for continuous variables. We used Spearman nonparametric correlations to investigate the bivariate associations of the SRSI and other adherence items with each other, as well as with barriers to adherence such as depression and substance use. We based correlations on data from individuals’ most recent clinical assessments. Each correlation used all individuals with data for the given pair of variables, with sample sizes ranging from 2,301 to 2,399; neither listwise deletion nor imputation was used. We also conducted sensitivity analyses using the subset of patients who had complete data on all adherence items (N = 2,176).

We examined the completion times for the VAS and the SRSI. We selected the VAS as the primary comparison measure for the SRSI because it is also a single-item measure, and it is commonly used when respondent burden must be minimal.

To assess the association of adherence items with HIV viral load and CD4+ cell count, we conducted unadjusted and adjusted generalized estimating equation (GEE) analyses. GEE employs robust standard errors to account for repeated clinical and adherence measures within individuals. Adjusted analyses included patients’ sex, age, race, and risk factor for HIV transmission. For the GEE analyses, we dichotomized HIV viral load (detectable or >400 copies/ml vs. undetectable or ≤400 copies/ml) and CD4+ cell count (≥200 cells/mm3 vs. <200 cells/mm3).

Finally, using individuals with responses on all of the adherence items at their most recent assessment, we employed ROC curves with detectable (versus undetectable) HIV viral load as the outcome to compare the five adherence items with respect to their predictive validity. For the adjusted analyses, we used individual predicted values (\( \widehat{{y_{i} }} \)) from logistic regression. Stata 11 [61] was used for all analyses.

Results

Overall, 2,399 patients completed a total of 6,160 assessments. At the most recent assessment, the mean age of study patients was 45 (SD 10) years, 81 % were men, and mean current CD4+ cell count was 514 (SD 290) cells/mm3. Characteristics of patients in the study were similar to those of all patients on ARVs receiving care in the two clinics during the study period (data not shown). Table 1 shows the demographic and clinical characteristics of the study patient population by SRSI responses. Most patients rated their adherence at or near the top of the scale; very few rated their adherence as “poor” or “very poor.” Among those reporting poor or very poor adherence, 51 % had a detectable viral load. This can be compared with 42 % of those reporting fair adherence, 23 % of those reporting good adherence, 13 % of those reporting very good adherence, and 9 % of those reporting excellent adherence. Somewhat larger proportions of white patients reported excellent adherence than patients in other racial categories, somewhat larger proportions of men who had sex with men reported excellent adherence than individuals with other HIV transmission risk factors, and there appears to be a trend toward older age groups reporting better adherence than younger age groups (see Table 1).

Table 1 Characteristics of study patients at most recent PRO assessment (N = 2399)
Full size table

The box and whiskers plot in Fig. 1 shows the distribution of the VAS continuous scale at each level of the SRSI at individuals’ most recent assessment. There is a great deal of variability in VAS scores in the lower categories of the SRSI. Figure 1 also shows a nonlinear relationship between the median VAS responses and the SRSI response categories. The median of the VAS was slightly less than 60 % at each of the two lowest levels of the SRSI, and reached a maximum value of 99 % at the highest level of the SRSI (Fig. 1).

Fig. 1
figure 1

Box and whiskers plot of the distribution of VAS results at each level of the SRSI. The gray box shows the inter-quartile range of the VAS, and the horizontal line in the middle shows the median value. The whiskers show 1½ times the inter-quartile range, and individual observations more extreme than those values are shown as dots

Full size image

Both the SRSI and the VAS required little time to complete. The mean time for patients to complete the VAS was 18.6 s (median 15 s) and the mean time for the SRSI was 13.5 s (median 11 s).

Table 2 shows Spearman correlations between the SRSI, the VAS, and each of the included AACTG adherence items. Because of the infrequent endorsement of the bottom category (see Table 1), we combined the two lowest categories and present results from both the original six-category SRSI and the reduced five-category version. All correlations are statistically significant and moderately large in magnitude. The five-category SRSI had correlations with other adherence items at the same magnitude as the original six-category variable, so we used the five-category SRSI in all subsequent analyses. Results did not change in sensitivity analyses of the Spearman correlations among the subset of individuals who had complete adherence data for all items (data not shown).

Table 2 Spearman correlations between SRSI and other adherence items
Full size table

Spearman correlations between the adherence items and selected adherence predictors and barriers are shown in Table 3. The correlation between current illicit substance use and SRSI (r = −0.15, p < 0.001) is similar in magnitude to analogous correlations with other adherence items (r s’s range from −0.10 to −0.19, all p values <0.001). The correlations between depressive symptom burden and adherence also had similar magnitudes across all of the adherence items (r s’s range from −0.10 to −0.21, all p values <0.001). Correlations between adherence items and sex (r s’s range from −0.02, p = 0.3, to 0.04, p values 0.3–0.03) were small across all adherence items, and correlations with non-white race (−0.17 to 0.03, p values <0.001–0.1) were inconsistent across adherence items.

Table 3 Spearman correlations between adherence items and predictors and barriers to adherence
Full size table

When evaluating the relationship of the five adherence items with viral load and CD4+ cell count, we used GEE to analyze each adherence item separately in unadjusted and adjusted analyses. We treated the highest value (best adherence) as the reference category (see Table 4; Fig. 2). When the adherence items were used to predict CD4+ cell count <200 cells/mm3, only the SRSI (and the binary “past-weekend” item) showed a consistently strong and statistically significant relationship for each category in comparison with the highest level of adherence, in both unadjusted and adjusted analyses. The categorized VAS was significantly associated with CD4+ count <200 cells/mm3 at its lower levels of adherence in unadjusted and adjusted analyses, and the remaining items had variable and inconsistent relationships with CD4+ cell count (Table 4). The associations between adherence items and detectable viral load all showed a general trend in which worse reported adherence was associated with a greater odds of a detectable viral load (>400 copies/ml; Fig. 2; Table 4). Sensitivity analyses, in which 30 and 90 days were used to define current lab values, had similar results.

Table 4 Unadjusted and adjusted odds ratios for the associations between adherence items and detectable HIV viral load and CD4+ cell counts using GEE analyses
Full size table
Fig. 2
figure 2

Unadjusted and adjusted odds ratios for the associations between adherence items and detectable HIV viral loads using separate GEE analyses. SRSI-5 cat SRSI with the bottom two categories combined, VAS visual analogue scale, Last 4 days “How many doses of your medications did you miss in the last 4 days?”, Last weekend “Did you miss any of your medication last weekend…?, Last missed dose “When was the last time you missed any of your medications?” Adjusted analyses included patients’ sex, age, race, and HIV transmission risk factor as covariates

Full size image

Finally, we used ROC curves to compare the success of the five ordinal adherence items in predicting undetectable viral load. Both unadjusted and adjusted ROC curves are shown in Fig. 3. In unadjusted pairwise analyses, the SRSI was a better predictor of undetectable viral load than any other single item except, possibly, for the VAS (p = 0.07; Table 5). In adjusted analyses, the difference between the SRSI and the VAS was no longer statistically significant and the comparison of the SRSI with the “Past 4 days” item was marginally significant (p = 0.07; Table 5).

Fig. 3
figure 3

ROC curves comparing adherence items with respect to the strength of their association with undetectable viral load. SRSI-5 cat SRSI with the bottom two categories combined, VAS visual analogue scale, Last 4 days “How many doses of your medications did you miss in the last 4 days?”, Last weekend “Did you miss any of your medication last weekend…?, Last missed dose “When was the last time you missed any of your medications?” Adjusted analyses included patients’ sex, age, race, and HIV-transmission risk factor as covariates

Full size image
Table 5 Comparison of the area under the curve (AUC) for adherence items predicting undetectable viral load (N = 1,914)
Full size table

Discussion

This study examined the validity of the SRSI when used in routine clinical care. The SRSI required very little time to complete, making it practical for HIV clinical care. It was significantly correlated with other self-reported adherence items and inversely correlated with known predictors of ARV medication non-adherence, such as illicit substance use and depression, indicating the concurrent validity of the SRSI for measuring adherence. Finally, the SRSI predicted CD4+ cell count and viral load as well as, or better than, other adherence items, indicating good predictive validity.

Much of the emphasis of this study focused on comparisons between the SRSI and the VAS as two of the items commonly used to measure adherence when longer instruments are not feasible. In contrast to an early, small study [43], which found that the VAS had higher correlations with adherence as measured by MEMS caps than the SRSI, we found that the SRSI was either similar to or outperformed the VAS when considering each of the study measures including concurrent validity, predictive validity, and respondent burden. Other factors should also be considered when comparing these measures. A concern with the SRSI has been the possibly judgmental response category terms of poor, or very poor [43]. In contrast, long-standing concerns with the VAS include numeracy issues, and more recently, a study demonstrated that patients frequently need assistance with the VAS raising even more feasibility concerns when considering a measure for wide-spread use in busy clinical care settings [43]. These issues demonstrate the complexity of selecting a brief, feasible, and valid adherence instrument for routine clinical care. However, our findings suggest that the SRSI may be an effective, brief way to routinely measure self-reported adherence in clinical settings, and may do so with minimal workflow disruption and patient burden, where other longer adherence measures may not be practical.

Our finding of a nonlinear relationship between the SRSI and the VAS is worth noting. Although numeric values of 0, 20, 40, 60, 80, and 100 have been assigned to the SRSI categories in some studies [39, 43], our comparisons of the SRSI with the VAS suggest this may not be an ideal approach for this categorical variable. If numeric values are going to be assigned to SRSI categories, our analyses suggest that 20, 60, 70, 90, 95, and 99 would be more appropriate values, however this is based on the VAS with all of its limitations. Better estimates should be obtained using a more rigorous approach. In addition, the extreme variability in the VAS scores at the low end of the SRSI suggested the possibility of misunderstanding of the direction of the VAS or individual differences in the interpretation of the descriptive words used in the SRSI.

One of our key findings was that the SRSI worked well for analyses with the bottom two categories collapsed. We think that based on our findings, it may be entirely appropriate to continue to collapse these two categories for analyses in the future. However, we note that we are not suggesting that one of these two categories should be dropped from the question. Patients might adjust their responses based on the categories available to them. Before dropping a category, empirical data comparing a five-category and a six-category response option should be compared.

Strengths and Limitations

This study is limited by the lack of existence of a true gold standard for measuring adherence. Objective measures of adherence, such as direct observation or unannounced home-based pill counts, are impractical for large clinic-based samples. We did have objective measures of HIV viral load, a clinical marker strongly influenced by adherence, and the SRSI performed well, as did other adherence items, in predicting these values. Another potential limitation of self-reported items, such as the SRSI, is the impact of social desirability bias. Patients taking the assessment were aware that their providers could view the results. As the purpose of this study was to determine how this measure would perform in routine clinical care, provider knowledge was a necessary component. Social desirability could lead to over-reporting adherence, which would be expected to attenuate the strength of association between self-reported adherence and clinical measures such as HIV viral loads. Associations of these clinical characteristics with the SRSI were largely as strong as, or stronger than, the associations with other self-reported adherence items.

Single-item measures of adherence may not capture the complex patterns of adherence behavior found among HIV-infected patients. However, they allow for routine measurement of adherence in settings where more complex measures may not be feasible, including resource- or time-constrained environments such as busy clinical care settings.

The generalizability of the findings may be limited because this study was conducted only among English and Spanish-speaking patients from two clinical sites. Unfortunately, the number of Spanish speakers included in these analyses was not large enough to permit a subset analysis of these findings only among Spanish speakers. However, additional sites with large numbers of Spanish speakers have recently initiated the CNICS assessment, therefore we hope to be able to address whether these relationships are the same among Spanish speakers in the future. We also did not address the further complications of use in additional languages and cultures, which would include the difficulties of equivalent translations of the SRSI adjectival scale and the numeracy issues of the VAS. Further investigation would be appropriate, particularly among groups not represented in our sample.

Strengths of this study include a larger sample size than prior evaluations. The study was conducted within CNICS, which has comprehensive clinical data including a routinely collected self-reported clinical assessment with instruments measuring adherence, substance use, and depression. In addition, this study evaluated patients in two geographically distant routine clinical care settings and therefore evaluated the use of the SRSI in more “real world” settings than has been done previously.

Conclusions

Best practices for HIV treatment include adherence assessment and counseling at every visit for patients on ARVs [62]. Brief valid measures of adherence such as the SRSI that are feasible in clinical care settings have the potential to improve clinical care and outcomes for patients with HIV. Of the single-item adherence measures examined here, the SRSI appears to be a more accurate predictor of clinical outcomes, including HIV viral load levels. It also may be the most practical single-item adherence measure for clinical use, given its minimal patient memory demands, low time burden for completion, and simplicity. Further analyses will include longer-term longitudinal analyses to strengthen causal assumptions and determine whether the SRSI can be used to measure changes in adherence over time.

Patient self-report has proven beneficial for detection of health conditions and behaviors that are traditionally difficult to measure accurately in the context of a clinic visit [47, 63–65]. Among HIV-infected patients, self-report assessments have detected high levels of depression, suicidality, poor adherence, sexual risk behavior, and substance use [47]. Low-burden, easy-to-integrate, and low-cost self-report measures, such as the SRSI, may help streamline and facilitate patient-provider communication. This can allow for more targeted discussion of patient needs, as well as real-time adherence intervention before patients suffer virological failure and the potential consequences of ARV resistance mutations. These findings underscore the benefit of further study of self-report measures with the goal of improving HIV care.