Introduction

Adherence to HIV antiretroviral therapy (ART) is the strongest known predictor of viral suppression, drug resistance, disease progression, and death [1–4]. There is, however, no gold standard for adherence monitoring. Several approaches are associated with HIV viral suppression, but each has limitations. Patient interview is imprecise [5], pharmacy refill reflects maximum possible adherence [6], unannounced pill counts are expensive and labor-intensive [2], and electronic monitoring precludes the use of pill box organizers, which are effective, simple, and inexpensive adherence support tools [7]. Moreover, all these approaches monitor adherence retrospectively, often detecting adherence lapses weeks to months after they occurred.

Real-time electronic adherence monitoring creates the opportunity to detect missed doses as they happen, potentially allowing for intervention before virologic rebound occurs [8]. A second-generation electronic medication container, called the Med-eMonitor®, has multiple compartments similar to a pill box organizer and can transmit data in real-time through a telephone connection. It has been used successfully in the treatment of schizophrenia [9, 10] and congestive heart failure [11].

This study examines the feasibility, validity, and acceptability of real-time ART adherence monitoring with the Med-eMonitor® among HIV-infected patients in San Francisco.

Methods

Participant Description

Participants were recruited consecutively between August 2006 and January 2008 from (1) the Positive Health Program HIV Clinic at San Francisco General Hospital, which serves a publicly insured urban population, and (2) the Research in Access to Care (REACH) cohort of HIV-infected homeless and marginally housed individuals. The REACH cohort is described elsewhere [12]. Briefly, REACH participants were followed prospectively with monthly unannounced pill counts, as well as socio-demographic questionnaires and HIV RNA determination, every 3 months.

Inclusion criteria for the current study were age 18–64 years, current use of ART, personal ownership of a landline telephone, and residence within 20 km of San Francisco. The only exclusion criterion was cognitive impairment severe enough to prevent informed consent. Participants were followed for approximately 3 months and received a $20 incentive at the beginning and at the end of this study.

Med-eMonitor Description

The Med-eMonitor (Fig. 1; www.informedix.com) stores medications and electronically records the time and date of every opening of its five child-resistant compartments. This adherence data is stored in flash memory and downloaded to a secure web site during pre-set time windows when Med-eMonitor is placed into a modem cradle connected to a telephone line. The device prompts users to take their medication by sounding a chime. Additionally, an arrow on a liquid crystal display (LCD) points to the container that should be opened for each specific dose. Further customized information can also be displayed on the LCD screen (e.g. a pill description for medication verification, reasons for taking the pill). The device alerts users if they are taking the wrong medication (i.e. opening the wrong compartment) or taking a medication off schedule.

Fig. 1
figure 1

The Med-eMonitor device

Full size image

Other programmable features include the ability to query the user about confirmed pill ingestion, side effects, and/or symptoms on a regular basis. The device also provides for branching logic, whereby the response to a question leads to additional questions or instructions specific to that response. For example, patients with high symptom severity (e.g. elevated body temperature) can be prompted to call or simply place the device in its modem cradle to be automatically connected with support personal.

All participants in this study were trained on the use of the Med-eMonitor device at enrollment and as needed during the first 4 weeks of the study, including refill instructions, use of prompts, and insertion of the device into a modem cradle. On average, participants had three training sessions lasting 15–30 min each. No run-in period was included in the study design. All antiretroviral (ARV) drug names, dosing instructions, pharmacy dispensing information, and clinician names were collected and confirmed. A licensed pharmacist dispensed all medications into Med-eMonitor cassettes monthly. Participants were provided with instructions for ongoing technical assistance, if needed. The frequency of data transmission was set to once daily in this study.

Feasibility Assessment

Feasibility of Med-eMonitor adherence monitoring was determined by the receipt of Med-eMonitor adherence data from each participant via the Internet, as well as the percent of data transmitted daily. Types and frequency of transmission errors were also recorded.

Adherence Measurements

Participants were monitored prospectively for 3 months. Unadjusted Med-eMonitor adherence was defined as the number of compartment openings for all ARVs divided by the number of prescribed doses. Adjusted Med-eMonitor adherence is the number of compartment openings for all ARVs with confirmed doses divided by the number of prescribed doses. Participants confirmed doses by pressing a button on the device to indicate that the dose was ingested, thus allowing for identification of compartment openings without dosing. The number of treatment interruptions of all ARVs lasting greater than 48 h was also determined. The 48 h time period was chosen, because it represents a plausible length of time in which measurable viral rebound could begin to occur [13].

Participants received unannounced monthly home visits, as previously described [14]. Briefly, on a random day without advance notification, a research assistant performed a medication inventory and counted pills. Participants from the REACH cohort had received at least 3 months of unannounced pill count monitoring prior to enrollment in this study, whereas participants from the Positive Health Program had not received prior unannounced pill counts. Unannounced pill count adherence was defined as the number of pills prescribed minus the number of pills counted, all divided by the number of pills prescribed for the interval between home visits.

Med-eMonitor Acceptability

Participants received an in person quantitative interview at the end of the 3-month study period to assess device acceptability. Questions aimed to understand the participants’ perception of the device as a useful tool in remembering to take their medications, likes and dislikes of the device, whether they would recommend the device to a friend, and whether they would use it again. Responses were recorded on five-item Likert scales ranging from very strongly disagree to very strongly agree.

Laboratory Testing

CD4 cell count and HIV RNA testing were performed at baseline and 3 months using flow cytometry and the Amplicor ultrasensitive assay, version 1.5 (detection limit of 50 copies per milliliter; Roche Diagnostics, USA), respectively.

Statistical Analysis

Participant characteristics, feasibility, adherence distributions, and acceptability data were assessed by descriptive statistics. Monthly adherence values were averaged for each individual over the 3-month period for comparisons, and adherence distributions were tested for normality by the Shapiro–Wilk test. Unadjusted and adjusted Med-eMonitor and unannounced pill count adherence measurements were compared with Wilcoxon signed rank test and Spearman’s correlations. Spearman’s correlation was also used to assess the relationship between the each adherence measure and log HIV RNA. For participants who also participated in the REACH cohort, a paired t-test and Wilcoxon signed rank test were used to compare unannounced pill count adherence for the 3 months prior to this study with unannounced pill count adherence for the 3 months during the study to determine if Med-eMonitor use may have had an effect on adherence behavior. Univariate logistic regression analysis was used to assess for an association between each of the three adherence measures and the acceptability question about the likelihood of using the Med-eMonitor again (i.e. very strongly disagree, disagree, or neither disagree or agree versus somewhat or very strongly agree that the participant is likely to use the Med-eMonitor again). All analyses were conducted in SAS 9.0 (Cary, North Carolina).

Ethical Considerations

This study was approved by the ethical review board of the University of California San Francisco, and all participants provided written, informed consent.

Results

A total of 23 individuals were recruited from the Positive Health Program HIV Clinic and 29 individuals were recruited from the REACH Cohort. No participants were excluded due to cognitive impairment. As shown in Table 1, the cohort characteristics consisted largely of middle-aged black and white men who had relatively high levels of education. The majority were men who have sex with men and injection drug users, and 69% had undetectable viral load at enrollment. Median follow-up time was 85 (IQR 72–90) days. Data on Med-eMonitor adherence was missing for two participants due to lack of interest in participating after enrollment, and HIV RNA was missing for another participant due to a missed study visit. Complete data was available for all other participants. Additionally, no devices were lost, stolen, or broken. At least one transmission error occurred in all but two participants, and errors consisted of the lack of a detected cradle connection, lack of a dial tone, a modem initialization error, and a busy signal. These errors, however, were typically transient, and a median of 92.4% (IQR 72.8–98.4%) of data was transmitted daily during the follow-up period.

Table 1 Cohort characteristics (N = 52)
Full size table

Adherence values are shown in Table 2. Median adherence was 97.1% (IQR 99.7–105.5%) for unannounced pill count, 96.8% (IQR 70.2–123.4%) for unadjusted Med-eMonitor, and 89.8% (IQR 77.1–102.5%) for adjusted Med-eMonitor. Unannounced pill count adherence was similar to unadjusted Med-eMonitor adherence, although it trended toward a difference (Wilcoxon signed rank test, P = 0.07), and was significantly higher than adjusted Med-eMonitor adherence (P = 0.01). Unannounced pill count adherence was not significantly correlated with unadjusted Med-eMonitor adherence (Spearman r = 0.20; P = 0.17), but was significantly correlated with adjusted Med-eMonitor adherence (r = 0.29; P = 0.04).

Table 2 Monthly adherence levels over the 3-month study period
Full size table

Figure 2a, b provide scatter plots presenting each participant’s 3-month average unannounced pill count adherence with his or her 3-month average unadjusted and adjusted Med-eMonitor adherence, respectively. The discrepancies seen on the left side of these plots likely indicate non-use of the Med-eMonitor device while still taking the pills, whereas discrepancies on the right side likely reflect extra openings without removal of pills (also known as curiosity events). Fewer right-sided discrepancies are seen with the adjusted Med-eMonitor data compared to the unadjusted Med-eMonitor data, as confirmed dosing should remove most curiosity events.

Fig. 2
figure 2

Scatter plots of unannounced pill count versus adjusted (a) and unadjusted (b) Med-eMonitor adherence. Data from the same participant are aligned vertically and are ordered from lowest to highest unannounced pill count adherence

Full size image

Fifty-three percent of participants had one or more interruptions of at least 48 h according to both unadjusted and adjusted Med-eMonitor data. Among participants with any interruptions of at least 48 h, the median duration of interruptions was 93.6 (IQR 57.6–151.2) hours for unadjusted data and 103.2 (IQR 67.2–158.4) hours for adjusted data. For the subset of participants who were also in the REACH cohort, the mean adherence level by unannounced pill count was 5% higher in the 3 months of the study period compared to the prior 3 months, with this difference nearing statistical significance (91.0 ± SD 12.4% versus 94.6 ± SD 7.8%; paired t-test, P = 0.06). Median values, however, were more similar (96.0% [IQR 88.9–99.2%] versus 97.1% (IQR 99.7–105.5%], respectively; Wilcoxon signed rank test, P = 0.26).

Figure 3 presents the distribution of adherence by unannounced pill count and unadjusted and adjusted Med-eMonitor, revealing wider variation in both types of Med-eMonitor adherence measurements compared to the unannounced pill count.

Fig. 3
figure 3

Distribution of adherence by unannounced pill count, unadjusted Med-eMonitor and adjusted Med-eMonitor

Full size image

The number of participants with viral suppression remained constant during the 3-month study period (N = 36). HIV RNA suppression was significantly correlated with unannounced pill count adherence (Spearman r = −0.35, P = 0.01), trended toward significance with unadjusted Med-eMonitor adherence (r = −0.24; P = 0.10), and was not significantly correlated with adjusted Med-eMonitor adherence (r = −0.18, P = 0.22).

As shown in Table 3, subjects generally found the Med-eMonitor device acceptable and useful; however, not all features were as appealing as others. For example, 79% stated that the device made it easy to remember when to take their pills, while 65% found the chimes annoying. Although not statistically significant, all three adherence measures were associated with a higher likelihood of using the Med-eMonitor again (OR 8.43 for unannounced pill count, P = 0.71; OR 4.8 for unadjusted Med-eMonitor, P = 0.52; and OR 68.8 for adjusted Med-eMonitor, P = 0.29).

Table 3 Acceptability of the Med-eMonitor device
Full size table

Discussion

The results of this proof-of-concept study indicate that the Med-eMonitor is a feasible adherence measure in that data were obtained from all but two participants; no devices were lost, stolen, or broken; and the majority of data were transmitted at least daily. Its validity is supported by similarity in median adherence values between the unadjusted data and the unannounced pill count, as well as the correlation between the adjusted data and the unannounced pill count. Moreover, the trend toward significance in the correlation between HIV RNA suppression and unadjusted Med-eMonitor adherence is of similar strength as the significant correlation between HIV RNA suppression and unannounced pill count adherence.

The variations in similarity and correlation among the three adherence measures and HIV RNA in this study reflect the inherent challenges of adherence monitoring. Namely, all adherence measurements have limitations and true adherence behavior can only be estimated through comparison of multiple measures and adjustment for known shortcomings [14]. For instance, adherence values >100% by both unannounced pill count and Med-eMonitor data suggest that participants may have lost pills or opened the compartment without taking out a pill, respectively. Such curiosity events have been reported in first generation electronic medication event monitors [15]. Moreover, a small number of participants had adjusted Med-eMonitor adherence values >100%. These participants may have mistakenly pressed the indicator button because of curiosity or a misunderstanding of its intended use. This finding highlights the importance of proper training and re-training, which can be accomplished in real-time through active monitoring of the received data. Such unintended use of the indicator button may have lead to under or over-estimation of adherence with the Med-eMonitor adjustment and its lack of association with viral load. Each participant’s true adherence behavior is not known, but likely lies somewhere between the unadjusted and adjusted estimates.

Additional variability is seen in the distribution of adherence. Both unadjusted and adjusted Med-eMonitor adherence had a wider distribution of adherence measurements compared to that found with unannounced pill counts. Less precise measures of adherence by definition have a limited ability to distinguish high and low adherers. The wider range of adherence values with Med-eMonitor compared to those with unannounced pill count therefore suggests that the Med-eMonitor may better identify individuals with low adherence.

Medi-eMonitor has two critical advantages over unannounced pill counts. First, it does not require time-, labor-, and cost-intensive home visits. Second, Med-eMonitor has the capacity to transmit adherence information in real time when it is linked to a phone modem, thus enabling intervention prior to the loss of viral suppression. The devices in this study were set to transmit data on a daily basis rather than after each dose, although multiple data transmissions widows are possible. Either way, logistical considerations, such as busy clinical or research schedules, may limit the practical ability to act on adherence information to once daily. Moreover, once daily transmission provides data frequently enough to identify treatment interruptions with the potential for viral rebound [13]. Such treatment interruptions were indeed not uncommon in this study, emphasizing the need for timely intervention.

The trend toward a statistically significant improvement in mean adherence of 5% after the introduction of the Med-eMonitor could suggest a beneficial intervention effect comparable to that observed with pill box organizers [7]. This finding highlights the potential of this device for intervention, as well as better monitoring and patient education. Additional studies with a larger sample size and randomization, however, would be needed to determine intervention effects.

Med-eMonitor was also found to be acceptable to most participants, although many did not agree that it was easy to get pills in and out of the compartments, would not recommend it to a friend, and were not likely to use the device again. Further qualitative research drawing on technology acceptance models would be useful to optimize the design features in various target populations [16]. This study involved urban poor individuals, and other populations likely have differing opinions. Acceptability may be increased through additional training, including help desk-style support to assist users when needed. Such “just in time” assistance has been previously shown to be effective for device training [17]. The educational and reminder features should also be explored for their impact as interventions.

This study has several other limitations. First, it has a relatively small sample, which limits statistical analysis. Second, the sample was biased in that all participants owned landline telephones and none had significant cognitive impairment, which is common in individuals with HIV-related co-morbidities and/or substance abuse. Indeed, the levels of adherence seen in this subset of individuals are higher than those in the larger REACH cohort [12]. Generalizability of the results may therefore be limited, and other measures, such as unannounced pill counts, may be needed for these populations. Second, the high levels of adherence limit the ability to detect associations among measures and viral load (known as a ceiling effect). Third, while the unannounced pill count is considered an objective measure of adherence, it is possible that it is not a sufficiently precise referent measure and may also change adherence behavior. Finally, pharmacists dispensed the medications for this study and self-dispensing outside of a research context may introduce more medication error.

This study shows promise for real-time antiretroviral therapy adherence monitoring and potentially intervention. eHealth, or the use of electronic devices to support the delivery of health care, enables real-time adherence monitoring and is gaining momentum in a variety of applications [18, 19], although little data exists on the feasibility and acceptability of specific devices and approaches. This study provides evidence that the Med-eMonitor can be used successfully to monitor adherence in an urban population and is acceptable to most participants. While additional studies are warranted to test Med-eMonitor in diverse populations and at scale, real-time monitoring and intervention may help mitigate the high cost of poor adherence in both developed and developing settings [20, 21].