Introduction

Over four million people in resource-limited settings currently take HIV antiretroviral therapy (ART) [1]. Sustained adherence to ART is critical for durable viral suppression to prevent AIDS-related morbidity and mortality [2, 3]. The most common form of adherence monitoring is structured patient interview (commonly referred to as self-report), which correlates moderately well with virologic suppression [4, 5]. When self-report is obtained during routine clinic visits, adherence problems are typically identified weeks to months after they occur. Moreover, no adherence data is collected if the patient misses the visit. Loss of viral suppression may occur in as little as a few days after interrupting ART and can rapidly lead to drug resistance to the most commonly used regimens [6]. More frequent, proactive identification of adherence problems could prompt intervention to prevent viral rebound and the development of drug resistance.

mHealth refers to the use of mobile devices, such as phones, to support the practice of medicine and public health; it is a rapidly growing field with potential applications for frequent self-reported adherence data collection. Over 60% of the world owns mobile phones, with mobile phone subscriptions increasing from one billion in 2002 to 4.1 billion in 2008 [7]. Growth in mobile phone usage has allowed for new innovations such as mobile banking, mass dissemination of health and market information, and verification of drug authenticity [8]. Two technologies are readily available for automated data collection on standard mobile phones: interactive voice response (IVR) and short message service (SMS) text messages. IVR uses recorded voice messages to collect information through voice or keypad inputs and has been used in the research context for the collection of health-related behaviors, such as alcohol use and condom use, in developed countries [9–13]. SMS can be used to collect similar data through text messages, and short-term benefit has been seen for chronic disease management in developed countries [14]. Other applications with expanded data collection and delivery capacity are available, but require Java-equipped [15] or Linux or Android-based smart phones [16].

Adherence self-report is particularly challenging in the pediatric population because the data refers to the child, but is commonly provided by the caregiver [17]. IVR and SMS may help lessen problems in accuracy by decreasing the caregiver’s recall period and increasing the number of sampled time periods through frequent data collection. Moreover, the technologies may be particularly appealing to adolescents who are often early adopters of technology [18].

Several HIV/AIDS programs are currently exploring mHealth technology for data collection and algorithm support, primarily through community health workers [19–21]. Other programs are using mobile phones to deliver automated SMS to remind patients to take their ART [22] or to send preventions messages [18, 23]. Mobile technology also has the advantage of covering rural populations with poor transportation infrastructure in resource-limited settings. The vast majority of the population has access to wireless telecommunication networks in Uganda [24, 25]. Rigorous evaluations of these mHealth programs are pending [26], and there is limited experience with patient level data collection. This study examined the feasibility of using IVR and SMS for automated collection of weekly individual-level ART adherence data from July to August 2009 in rural Uganda. The primary outcomes were adherence query data completion rates and participant perceptions of the technologies.

Methods

Participant Recruitment

All participants were caregivers who owned phones and were involved with a parent study, the CHildren’s ART Adherence (CHARTA) Study, which consists of a cohort of 121 HIV-infected children prospectively followed with multiple measures of adherence over a 1-year time period in Mbarara, Uganda. The median age of the children in the cohort is 6.4 (interquartile range [IQR] 4.7–8.3) years, 47% are female, 71% are cared for by a biological parent, 15% initiated ART at enrollment to the cohort, the median enrollment CD4% for all children was 40% (IQR 28–48%) cells/ml, and 47% of children taking ART for at least 3 months had <400 HIV RNA copies/ml at baseline. All children were recruited from an outpatient HIV clinic in a regional referral hospital in a rural setting. Risks of potential unintended disclosure of HIV status through IVR calls and SMS, as well as the multiple other measures of adherence, were explained and consent was obtained at enrollment into the cohort. The initial goal of this sub-study was to compare IVR with SMS. Accordingly, literate caregivers were randomized to IVR or SMS (1:1) at enrollment to the parent study. All illiterate caregivers were assigned to IVR.

IVR and SMS System Specifications

Yo! Voice Solutions and Software Development in Kampala (www.yo.co.ug) developed the IVR system, which utilizes INTELLIVR software to manage interactions with participants and YBS software for billing functions. Dimagi (www.dimagi.com) developed the SMS system using an open-source RapidSMS application, which is based on the open-source python web framework, Django. A new RapidSMS application was written to model the survey logic, time the initiation of surveys each day, and collect and collate the survey responses. The application was deployed on an Amazon EC2 server based in Europe and hosted in an Apache web server. Yo! managed the gateway for both IVR and SMS over the local MTN mobile phone network (www.mtn.co.ug). Protection against power outages included a back-up service provider and gas-powered electrical generator. Initial set up costs consisted of approximately US $1,900 for custom IVR programming, $1,000 for custom SMS programming, and $300 to establish both systems with MTN. On-going costs were $113 for airtime ($0.23 per minute and $0.05 per SMS) during the 4-week period.

IVR Call and SMS Algorithm

Algorithms for the IVR calls and SMS were initiated once a week. The content for both technologies was nearly identical. Each IVR call or SMS was conducted in the local language, Runyankole, and began with a brief greeting and a request for a four-digit personal identification number (PIN). Participants were then asked one of the following randomly selected research questions:

  • How many doses of medicine did your child miss in the past 7 days?

  • How many doses of medicine did your child miss in the past 30 days?

  • How many doses of medicine did your child miss since your last pharmacy refill?

The term “HIV” was not mentioned to protect confidentiality; however, participants were instructed that the question referred to ART during the training sessions. IVR calls and SMS included a response example (e.g. for IVR, “two followed by the hash symbol”) with each question and gave participants instructions on how to indicate they did not know the answer. For IVR calls, participants were given the opportunity to repeat all statements and/or the research question. A successful answer was defined as a typed numerical response.

During the first week, the IVR calls and SMS were made randomly during daytime hours and only one attempt was made. During subsequent weeks, they were made at the same day and time (e.g. Mondays at 4 pm) as specified by the participants and repeated if a numerical response was not received. IVR calls were repeated after 20 min and again 20 min later, as well as up to three times on each of the following 2 days starting at the specified time (e.g. Tuesday and Wednesday at 4 pm). SMS were repeated on each of the following 2 days at the specified time. If the IVR calls and SMS were not successful after the nine and three attempts, respectively, no further attempts were made for that week. SMS were only repeated three times, because unlike IVR calls, the text remained on the phone until it was answered or deleted. A new cycle was then attempted the following week.

Participant Education on Use of IVR and SMS

Participants received approximately 20 min of individual instruction prior to beginning the study. Research assistants described the background on the assigned technology in detail and went through the IVR call or SMS algorithm step-by-step. They explained the concept of a PIN and gave an example of the participant’s year of birth. Participants practiced receiving IVR calls or SMS several times, at first with the assistance of the research assistant and then independently. Research assistants offered up to two additional training sessions if they were concerned about the participant’s understanding of the technology or at the request of the participant.

Qualitative Data Collection on IVR/SMS Acceptability

After the completion of the IVR calls and SMS, research assistants conducted individual qualitative interviews with all participants. Questions and appropriate probes were asked to determine typical usage of mobile phones; explore experiences, perceptions, and acceptability of the IVR calls or SMS; and potential means for improving the number of successful responses. Interviews were transcribed into English [27].

Concurrent Measures of Adherence

In the parent study, the child’s adherence is measured through monthly caregiver report (3-day recall and 30-day visual analog scale [VAS]), monthly unannounced pill count or liquid weight, and ongoing electronic monitoring with the medication event monitoring system (MEMS). These data are reported here for approximately the same time period during which the caregiver-reported adherence data was collected through IVR and SMS. The median number of days of overlap in the measurements was 17 (IQR 15–22). Values are truncated at 100%.

Analysis

Interactive voice response call and SMS outcomes were analyzed using descriptive statistics. Logistic regression was used to explore potential characteristics associated with successful responses. These analyses were performed using SAS 9.2 (SAS Institute Inc., Cary, North Carolina, USA). Qualitative interview data was systematically examined using NVivo 8 (QSR International, Cambridge, Massachusetts, USA). All interviews were included in the analysis, and thematic saturation was achieved.

Human Subjects

The study protocol was approved by the Mbarara University of Science and Technology Research Committee, Partners Human Research Committee at Massachusetts General Hospital, and the Uganda National Council for Science and Technology. The parent CHARTA Study is registered with ClinicalTrials.gov (NCT00868257).

Results

All 31 of the 121 caregivers in the CHARTA Study who owned their own phones were offered participation. One participant decided to withdraw from the parent study because she was concerned that her child’s HIV status might be disclosed to other household members. Ten other participants either no longer had their own phones or had non-functional phones at the time the sub-study began. One participant had a landline and was not included in the analysis; all others had mobile phones, yielding a sample of 19 participants. The median age of participants was 34.0 (IQR 28.5–39.0) years, 90% were female, 79% had completed more than primary school education, and 63% were literate as defined by being able to read and write a sentence in their native language. Randomization of the 12 literate caregivers resulted in six receiving IVR calls and six receiving SMS; all seven illiterate caregivers received IVR calls.

A total of 48 weekly IVR call cycles were initiated over the 4-week period (mean of 3.7 weekly IVR calls per participant). Only three IVR calls were made to four participants due to misunderstandings about participant phone numbers or preferred calling times. Six (12.5%) IVR call cycles were incomplete for the following reasons: a persistently busy signal (n = 1); a busy signal and network congestion (n = 1); a busy signal and intentional call rejection (n = 1); and non-answer (n = 3). Of the completed IVR call cycles, only six (12.5%) resulted in a successful response. The most common reason for completed yet unsuccessful calls was a hang up during the PIN stage (46%), followed by a hang up or inappropriate response to a research question (29%). One participant changed phone numbers during the study. The median adherence rate obtained was 82% (IQR 71–92%) based on the six successful IVR calls (three IVR calls referred to doses missed in the past 7 days; three referred to doses missed in the past 30 days).

Twenty-four SMS cycles were initiated during the study, with four text messages sent to each participant. A total of 76% of the SMS cycles were not answered, meaning that no response was received to the greeting and prompt for the PIN. Detailed reasons for the lack of a response were not available. The median adherence rate obtained was 100% (IQR 99–100%), based on the four successful SMS (two SMS referred to doses missed in the previous 30 days; two referred to doses missed since the last pharmacy refill). One additional response was “do not know”.

See Fig. 1 for the types of responses to the IVR calls and SMS by week. There were no detected power outages in the IVR or SMS systems during the study.

Fig. 1
figure 1

Types of responses to the research questions asked during weekly IVR call and SMS cycles

Full size image

The concurrent median adherence rates by caregiver 3-day report, 30-day VAS, unannounced pill count/liquid weight, and MEMS are shown in Table 1.

Table 1 Median adherence rates (IQR) by different measures
Full size table

The following characteristics were assessed as potential predictors of successful IVR calls and SMS: age, sex, primary school education, and literacy. None were statistically significant, although this study was not powered for such an analysis. Age trended toward significance (p = 0.16) with a 0.9 odds of a successful call or text message for every year of increasing age.

Qualitative interviews revealed that mobile phones in this population are typically used for personal arrangements, such as making healthcare appointments, brief conversations with family, and setting up business interactions. Calls and/or text messages are both made and received, and most participants share their phones with others. Three participants reported technical challenges with their phones: two had trouble with keeping the battery charged, and the third’s phone frequently dropped calls for unclear reasons. Two participants also reported not keeping their phones with them at all times (e.g. when working in the fields). One participant needed the assistance of a family member to use her mobile phone, but none reported that someone else responded to the study IVR calls or SMS. Most participants had used the dial pad on their phones to load airtime in a pay-as-you-go system; however, none of these participants had experience with PINs or data collection through IVR or SMS prior to this study.

The first theme that emerged from the qualitative interviews was a poor understanding how to respond to the IVR and SMS prompts. Participants expressed confusion and a fear of making mistakes. Several used an incorrect PIN (e.g. their child’s year of birth or their phone number) or hung up on the IVR call or SMS without any input. One participant rejected the IVR calls. Two IVR participants stated:

At first those questions confused me. They would ask you for your year [PIN]. You would still be trying to respond and the call goes off. Then, they say that we shall try to call you back. Yet, the person who had trained me had repeated for me and told me what to do. It really greatly disturbed me. They were hard.

They asked me how many times your child has missed taking his drugs. I got confused with this question because my child had not missed any drug… I did not know that when the child misses a dose, you press two on the mobile phone and when the child has not missed, you press zero. So there I got confused because I did not know what to reply and because of this, they could call me and then I refuse to answer it as I could know that I can’t answer questions I am asked.

Some participants wanted to erase numbers they had entered at one stage of the call or SMS (e.g. the PIN) before entering a response to the next stage (e.g. the research question). Another IVR participant indicated:

Well, for the first time I had forgotten what to do, and when trying to click yes the call went off. They came and trained me again and I learnt that when the call comes and you respond, you don’t first erase, because when I would try doing so the call would go off.

A second theme related to challenges in training. Several participants reported feeling hesitant to acknowledge a lack of understanding during the training sessions. One SMS participant stated:

The one who taught me thought that I understand this thing so much which was not the case. I don’t blame [the research assistant] because she took me to know but I did not know what to do.

Incomplete understanding was also evident in that several participants reported knowing how to complete the calls and text messages appropriately; however, no successful responses were received from these participants.

Participants suggested the following strategies to improve response rates: repeated trainings over time, training in groups so shy participants can learn from each other, testing knowledge from the trainings, having a dedicated mobile phone, calling from an anticipated number at an anticipated time, writing down the PIN, and allowing more time for responses. An SMS participant said:

At first this call was hard where I used to put a wrong PIN number but since I came to know the PIN number, things changed a lot where I could answer the questions asked. So as time went on, this call became easier. I can tell you that now there is no question I can not answer.

Despite these challenges, all participants said that the technologies were acceptable and wanted to provide proper responses. Some participants also felt that the calls and text messages served as an adherence reminder. An IVR participant reported:

It even increased in my memory that the child is supposed to take her drugs every morning and every night.

Discussion

This study suggests that the use of IVR and SMS in resource-limited settings is technically feasible. There were minimal problems with mobile phone coverage, and no power or network outages affected the IVR and SMS systems during the study period. While electrical outages do occur frequently in rural Uganda, the IVR and SMS systems were protected through a robust back-up system. The adherence rates obtained through IVR were lower than with other measures, while those obtained through SMS were more similar. The low weekly data completion rate, however, limits this comparison and indicates that important challenges must be overcome.

The PIN caused the most confusion. Such security measures are critical for data fidelity in settings where mobile phones are shared, as well as protecting the confidentiality of individuals for any health-related issues. Confidentiality is especially important for data collection regarding HIV/AIDS due to stigma and discrimination [28, 29]. Given the prevalence of pay-as-you-go systems for airtime in developing countries [7, 8], the problems with the PIN were unexpected. The study results, however, suggest that the concept of a PIN is unfamiliar and poorly understood. One potential explanation is that PINs for airtime are used one time and discarded, while PINs in this context must be remembered and used repeatedly. Additionally, it is possible the problems with the PIN resulted from individuals other than the participants trying to respond to the query in the absence of a correct PIN. The participants, however, knew when to expect the IVR calls or SMS and could avoid sharing their phones during this time. Indeed, no participants reported this scenario. If an individual did try to respond to an IVR call or SMS, the PIN served its intended purpose of protecting the confidentiality of the participants and their children.

Several possible solutions exist for the problems encountered in this study. The first approach addresses the skills necessary to use mobile phone technologies. A detailed, multi-session training protocol over time including group participation could help improve successful IVR or SMS data collection. In the qualitative interviews, the participants indicated a need for repetition over the first several uses of both technologies. This training method gives participants time to get accustomed to the technology and learn from each other, as well as provide reminders if the initial understanding fades outside of the training session. The training should also include verification of understanding with active practice and confirmation of the desired behavior, as some participants appeared to understand, but in reality did not. Others thought they understood the instructions, but provided inappropriate responses. Another potential method for improving training could involve real-time SMS notification of unsuccessful attempts. That is, a research assistant could be notified by SMS when a participant is having problems and call him or her immediately after the problem occurs. Such “just in time”, teachable moments may help overcome barriers of understanding [30, 31].

A second approach to the problems in this study involves increased motivation. One potential explanation for the low completion rates is a lack of desire to respond to the IVR calls and SMS texts. However, all of the participants expressed a high level of interest in both IVR and SMS during the qualitative interviews. Additionally, only one IVR call cycle was unsuccessful because the participant intentionally rejected the call (i.e. turned off the phone), and adherence data was successfully gathered with all other concurrent adherence measures. Because initial enthusiasm may wane with participants outside of the research context and over longer time periods, incentives, such as airtime credited at the time of the response, could be used to motivate participants to learn to use IVR and SMS successfully and continue it over time. Any monetary incentives would have to be nominal in nature to allow for large-scale implementation; however, prior studies have shown that modest monetary compensation can increase study participation rates, as well as ART adherence [32, 33]. Moreover, it is possible that an improvement in skills may result in increased motivation, as success at completing the IVR calls or SMS could lend confidence and enthusiasm to participate.

Other potential limits to the feasibility include the low prevalence of mobile phones in this population at the current time (19 of 121 caregivers) and the fact that not all participants had the phone with them at all times. Mobile phone ownership, however, is rising rapidly [7] and proper training and incentives as discussed above may encourage constant possession of the phone by the participant.

Finally, cost must be considered when implementing technology in resource-limited settings. The initial set up costs of $3,200 are considerable, although they would likely be much lower for replication of existing systems, many of which are open source, with minor modifications to suit individual cultural contexts. Moreover, once the system has been established, it can be easily scaled to spread the cost over many individuals on therapy. The ongoing costs of airtime and text messages would be a potential burden; however, a recent cost analysis from southern Africa [34] showed that poor adherence to ART costs approximately $85 per individual per month. While median adherence levels for both adults and children in resource-limited settings [35, 36] suggest this degree of cost savings applies to a minority of individuals, the modest costs associated with IVR and SMS seem reasonable and should be assessed through formal cost effectiveness studies.

The swift expansion of mobile phone technology in the developing world has created rapid development and deployment of a wide array of potential applications, including mobile banking, data collection, patient tracking, and behavioral interventions. To date, little evidence has been published to support the efficacy of mHealth interventions [37]. This study is one of the first to present a detailed assessment of the challenges facing the use of mobile phones for individual level collection of healthcare data in a developing setting. While the data completion rates were lower than anticipated, the technology functioned well and the identified challenges may be overcome through improved skill training and increased motivation. The lessons learned with this small sample of participants also emphasize the importance of field-testing any introduction of novel technology into a new setting. Given the potential for timely adherence data to improve health through prolonged use of affordable first-line ART, efforts to make IVR and SMS effective are clearly warranted. Rapidly growing enthusiasm, however, must be tempered with careful identification of barriers to reliable and complete data collection.