Designing a Personal Health Application for Older Adults to Manage Medications: A Comprehensive Case Study

Abstract

Older adults with multiple chronic conditions often go through care transitions where they move between care facilities or providers during their treatment. These transitions are often uncoordinated and can imperil patients by omitted, duplicative, or contradictory care plans. Older adults sometimes feel overwhelmed with the new responsibility of coordinating the care plan with providers and changing their medication regimes. In response, we developed a Lesser General Public License (LGPL) open source, web-based Personal Health Application (PHA) using an iterative participatory design process that provided older adults and their caregivers the ability to manage their personal health information. In this paper, we document the PHA design process from low-fidelity prototypes to high-fidelity prototypes over the course of six user studies. Our findings establish the imperative need for interdisciplinary research and collaboration among all stakeholders to create effective PHAs. We conclude with design guidelines that encourage researchers to gradually increase functionality as users become more proficient.

Introduction

Older adults with multiple chronic conditions often experience care transitions where, depending on their health, they transition between outpatient care facilities, hospitals, skilled nursing facilities, and home care with non-professional caregivers. During these transitions, it is critical that an older adult’s Personal Health Information (PHI) is easily shared and communicated among all of the care providers and facilities. When the older adult transitions to his home, he and his non-professional caregiver must be able to understand and implement his care plan to avoid rehospitalization.

Unfortunately, in the United States, care is poorly coordinated between care facilities, care providers, and the older adult. Care facilities do not have the time, resources [7], or interoperable record systems to coordinate care among different facilities. In these situations, the older adult and caregiver become the information conduit between care providers [6], however they do not feel comfortable communicating or executing the care plans [7]. Indeed, the care plans often involve complex medication regimes prescribed by multiple doctors with different dosing frequencies which can overwhelm older adults with declining cognitive abilities [30]. When care is poorly coordinated, older adults are especially susceptible to medication errors with adverse health consequences [10, 13, 15, 21]. It is estimated that medication management errors cost billions of dollars in the United States each year [2].

The health informatics community has responded to these problems by assessing user needs for medication management [26] and developing medication management technologies [11, 14, 38]. The aforementioned systems would not meet the needs of older adults during care transitions because they do not provide older adults with the functionality necessary to manage and share medication regimes in fragmented systems of care. The “Care Transitions Intervention” has successfully improved care transitions for older adults with paper-based Personal Health Records (PHRs) [5, 6]. We were motivated to technologically enhance the paper-based PHR to provide older adults and caregivers with easier ways to maintain a medication list, find authoritative medication information, and communicate with healthcare providers. In this paper, we extend our previous work [16] and present a comprehensive case study tracing the iterative design process to create a functional Personal Health Application (PHA) prototype—the Colorado Care Tablet (CCT).

The CCT evolved from multiple paper-based low-fidelity prototypes to a high-fidelity functional prototype over the course of six user studies. In this case study, we discuss the design decisions that we made while designing the CCT and provide the community with a set of guidelines to help design future PHAs. We recommend participatory design to create PHAs for older adults with basic PHI management functionality and provide options for advanced PHI management functions as users become comfortable with the basic features. More generalizable recommendations include designing PHAs with interdisciplinary collaborations to ensure all stakeholders have a voice in the design process.

CCT background

The CCT is a tablet PC-based PHA that was developed over a two-year period concurrently with eight other PHAs as part of the Robert Wood Johnson Foundation Project HealthDesign [4]. We designed CCT for a tablet PC because it provided portability along with a large, touchscreen interface. In addition, patients can use CCT wherever they feel comfortable using a tablet PC and have Internet accessibility. We split our development into four design cycle components: user needs assessment, low-fidelity prototyping, high-fidelity prototyping, and functional prototyping. Deliverables from each design cycle component were reviewed by the target user group and a panel of experts as shown in Fig. 1.

Fig. 1

CCT iterative development timeline

User needs assessment

During the first six months of the CCT project, we conducted a user needs assessment with four focus groups (2 exploratory and 2 confirmatory) and twenty-one in situ interviews to explore the issues older adults and caregivers experience when managing medications and Personal Health Information (PHI) during care transitions. We found that older adults and caregivers sought: (1) medication information from multiple sources depending on the urgency of their informational needs; (2) autonomy over their medication regime; (3) a better way to integrate conventional and alternative medications into their regimes; and (4) reasons for why they were taking so many medications (e.g., taking two medications for high blood pressure) [29]. The findings from this needs assessment and recommendations from the expert review informed the design of CCT. The comprehensive needs assessment findings are out of scope of this paper, but are presented in detail elsewhere [29]. In this paper, we detail the results of the six subsequent user studies that iteratively designed the PHA. Specific to this manuscript, we extend our prior manuscript [16] by discussing two new studies where we evaluated appropriate medication scheduling interfaces and compared CCT with a mainstream PHA.

Expert review

The main research team was composed of human computer interaction researchers, medical informatician-practitioners, and social scientists. In addition, we had an expert review panel that consisted of an older adult patient and four experts in the areas of care transitions, Health Information Technology (HIT) interoperability, behavioral science, and patient-centered HIT. Initially, the expert review provided us insights into what older adults and caregivers experience during care transitions and assisted us in the design of the user needs assessment. We reviewed findings with the expert review panel from each design cycle and discussed next steps in the prototype development. During the low-fidelity prototype stage, we conducted a design workshop with the expert review panel to brainstorm ideas on medication reconciliation, the act of comparing a patient’s medication list with her doctors’ medication lists to identify inconsistencies. An example of an artifact from the design workshop can be seen in Fig. 2c. Pairing the expert review with user studies provided us with the opportunity to develop a PHA that would assist older adults with their medication management and provide healthcare providers enough information to help older adults make informed decisions.

Fig. 2

Collaborative design exercise: a flower clock prototype from study 1; b redesigned clock prototype; and c example of a design workshop artifact made by a member of the expert review panel iterating on the clock prototype

Overview of CCT

The final CCT design was based on the Care Transitions Intervention [5, 6], findings from the user needs assessment, expert review, and iterative design findings. We used a Lenovo ThinkPad X60 Tablet PC running Windows XP Tablet edition that had a finger-touch sensitive screen. A Socket Mobile Bluetooth Cordless Hand Scanner Series 7 was used to scan barcodes as an alternative input mechanism. Based on our resources and time constraints, our major design requirements were to:

• Assist participants in keeping an up-to-date personal medication list

• Provide authoritative medication information

• Help participants effectively discuss concerns about medication regimes and conditions with healthcare professionals

To this end, we created the CCT prototype, shown in Fig. 3, that provides participants the ability to easily complete four main functions: (1) medication list creation and management (designed in studies 1–6); (2) medication information retrieval (designed in studies 1, 4, and 6); (3) doctor visit preparation (designed in studies 4 and 5); and (4) information on when to seek assistance (designed in studies 5 and 6). The final Lesser General Public License (LGPL) open source, web-based prototype is available for the community to use and extend (http://www.projecthealthdesign.org/resources/pastprojects-products-open#Colorado). A more thorough description of the final CCT design is available elsewhere [33].

Fig. 3

Main application screen of the final CCT prototype with an overview of the four main functions

Related work

In this section, we briefly differentiate between Electronic Medical Records (EMRs) and Personal Health Records (PHRs). Furthermore, we examine the reasons for older adults’ inability to follow medication regimes and how CCT can help them in addressing these challenges. Finally, we explore current medication management technologies.

EMR vs PHR

EMRs are digital patient medical records that are intended for doctors and generally owned by healthcare institutions. Patients may get paper copies of the information in their EMRs, but cannot alter them. In addition, in the United States, EMRs are typically not interoperable among healthcare institutions. Conversely, PHRs are intended for patients. Electronic PHAs connect to PHRs to provide patients the ability to access, manage, and share their health information with trusted parties who may include doctors, nurses, or caregivers [36].

Medication regimes and adherence

Research suggests that patients have difficulty adhering to intended medication regimes because they: (1) find medication regimes are too complex [25, 35]; (2) forget to take medications [35, 40]; (3) do not have sufficient medication information [25]; and (4) feel they cannot communicate effectively with their doctors [25].

Complexity of medication regime

Older adults with multiple chronic medical conditions often visit multiple doctors for treatment. Each doctor can potentially provide multiple medications that vary in dosage and frequency which results in a complex medication regime [25]. We designed a single consolidated medication list in CCT that provides users the ability to view, add, and delete medications prescribed by various doctors to address the medication regime complexity.

Forgetting medication dose

Older adults who take multiple medications often miss a medication dose due to forgetfulness [40]. Forgetfulness can take two forms: (1) forgetting the correct way of taking the medication dose resulting in over-dosing or under-dosing; and (2) forgetting the medication dose all together [19]. This raises the need for a reminder system that prompts older adults whenever a medication dose is due. Through a common PHR platform developed by Project HealthDesign, the CCT was able to incorporate a mobile phone application developed by fellow grantees at Vanderbilt University Department of Biomedical Informatics that could provide users the ability to schedule medications, set up reminders, and send alerts when medications were missed [34].

Lack of medication information

Older adults are more likely to not adhere to medication regimes because they have insufficient information about medications and are therefore unaware about the purpose of a medication or the consequences of missing a dose [25]. Prior to designing CCT, we conducted in-home interviews and found that older adults desired easily accessible, authoritative information about medications. We found that older adults had medication management areas in their homes where they kept file cabinets containing binders and booklets about medications. Older adults infrequently utilized the medication management areas because searching for particular medication information was tedious. We designed an interface in CCT to provide a convenient way to obtain authoritative medication information.

Poor patient/doctor communication

Another factor that contributes to nonadherence of medication is poor patient/doctor communication. Researchers have found that patients receive inadequate information on the benefits and side effects of medications from their doctors [25]. Our needs analysis confirmed this finding [29]. We created an easy way for older adults to prepare for appointments by reviewing their medication lists and selecting questions to ask their doctors.

Medication management systems

We found that older adults with multiple chronic conditions employed several methods to manage their medications—including pill boxes and paper-based medication lists [29]. Although current paper-based systems are inexpensive, they are often illegible, out-of-date, and difficult to share with multiple providers or remote caregivers. In our needs assessment, one couple discussed moving file cabinets between their two homes to share medical information with providers [29]. Patients created stand-alone medication lists using Microsoft Notepad and Excel [29]. Other researchers have created electronic pill boxes [11] and medicine cabinets [38] that automatically monitor nonadherence and medication errors. Although these systems can be helpful, an older adult must have the ability to create an accurate medication list before being concerned with adherence to the medication regime.

Web-based PHAs have the potential to address all of the preferred functions of the target user group. The Surgeon General of the United States has recommended the My Family Health Portrait PHA (familyhistory.hhs.gov). This PHA is not interoperable and does not provide an online repository to store patient data. Thus, the user is still the information conduit—responsible for manually saving and sharing a XML file with all of the PHI. More recently though, there has been a rise in web-based, interoperable PHAs including Google Health and Microsoft HealthVault, however our prior work utilizing the cognitive walkthrough method has shown that older adults will face considerable challenges in using these systems [32].

Study overview

We were motivated to use participatory design because researchers have successfully used this method to create applications for older adults [9, 31]. In addition, HIT is mostly designed from a doctor’s perspective—health informaticians must integrate the citizens’ perspective into healthcare technologies [3].

Participant recruitment

After we received Institutional Review Board (IRB) approval, we recruited participants for the user studies. We did a brief cognitive screen-–where we asked the participants to provide their name, year of birth, age, and telephone number–-to ensure they were able to participate. Participants were recruited from a large urban area for the first four user studies (studies 1–4). The first site was a residential facility for older adults in a medium sized city that was a combination assisted living and independent living facility. The site housed predominantly highly educated older adults—many who had advanced degrees. The second site was a senior citizen center in a large metropolitan city that catered mainly to the surrounding working class community. The third site was a hospital clinic that predominantly served patients from a highly educated community in the large metropolitan city. The fourth site was a smaller, independent living residential facility for older adults in a suburb of the large metropolitan city.

While conducting the first four studies, we learned that caregivers were more likely to use the proposed technology and assist older adults with using the PHA. Thus, we recruited caregiver participants for the latter two studies by emailing recruitment notices to a university mailing list. Our final recruitment criteria to participate in the studies included individuals who were at least 60 years old or caregivers of such individuals where the older adult: (1) was hospitalized at least once in the past three years; (2) regularly saw two or more medical providers; (3) took three or more prescription medications; and (4) was willing to use a computer application to manage health information. All participants were able to write and speak in English.

Participants’ demographics

The six user studies involved a total of 31 participants summarized in Table 1. Nine participants self-defined themselves as caregivers and 18 participants self-defined themselves as older adults. Four participants were older adults and caregivers but were categorized as older adults because they mostly discussed their own experiences. The average age of the 22 older adults was 76.4 years old (s.d. = 7.3 years). Out of 18 older adults,¹ 14 had a computer in their house with Internet access. Four older adults did not have access to a computer. Of this subset, two older adults had never used a computer and the other two had used a computer in the past. Out of the 14 older adults that had a computer, ten older adults used it daily, one older adult used it 4 days a week, and three mentioned they used it rarely. Eight older adults accessed health information on the Internet. Other primary uses of computers included email (N = 10), word processing (N = 9), and financial applications (N = 5).

Table 1 Participant demographics

The average age of the nine caregivers was 52.7 years old (s.d. = 6.9 years). Eight caregivers had a computer with an Internet connection that they used daily. The remaining caregiver had used a computer in the past, but did not have access to a computer on a regular basis. The eight caregivers frequently used a computer primarily for accessing health information on the Internet and emailing. Caregivers also used computers for word processing (N = 5), photo editing (N = 5), and shopping (N = 3).

Methods

We designed CCT using an iterative participatory design methodology informed by Rapid Iterative Testing and Evaluation (RITE) [41] and Instant Data Analysis (IDA) [17]. Typically, researchers who use RITE methodology discuss any problems the participant had during a study session and fix any major prototype deficiencies before the next participant study session. The IDA methodology is similar to RITE, however it involves evaluation of data at the end of the user study day that could involve multiple user study sessions.

We conducted RITE studies for the first two CCT user studies. Since we were evaluating low-fidelity, paper-based prototypes, it was easy to modify major prototype issues between participant sessions if there was a definite need. During the high-fidelity prototyping studies (studies 3–5), we listed issues and possible changes in between the participant sessions, but could not modify the prototype given the time constraints. The expert review panel provided us with example PHI data to use for the studies. The expert review panel assessed possible changes before any edits were made to the prototype.

We performed user studies with 3–8 participants per user study. The exact number of participants in each study is provided in Fig. 1. Each participant session lasted about 1 h. The interval between user studies varied between 2 to 6 months. We confirmed previous findings during each iterative user study to ensure the changes made between studies were appropriate for the target population. The user studies were held in private rooms at different locations including a public library, a senior center, and a medical school campus. The data obtained from the six user studies included background questionnaires, videos of participants’ interaction with the prototypes, and researchers’ notes.

User study protocol

The user study sessions were facilitated by two researchers: one from health sciences and another trained in user study techniques. The researchers obtained informed consent from each participant and explained what was going to be recorded during the study. Participants were briefed about CCT and the purpose of the user study. We used a think aloud protocol during the study and modeled a practice example of thinking aloud to show participants what was expected. We provided participants with scenarios that were based on common medication management tasks identified in our needs assessment [29]. The participants were asked to perform the scenario tasks while their interactions with the prototypes were recorded. The health sciences researcher documented participants’ interactions while the usability researcher facilitated the study. Participants received a $20 supermarket gift card for their participation in the study.

Iterative design of CCT

In this section, we examine how an iterative participatory design approach helped us transform CCT from a needs assessment analysis to paper-based prototypes to high-fidelity prototypes and finally to a functional prototype that older adults and caregivers found helpful in managing medications. An overview of the study goals and findings is provided in Table 2.

Table 2 Overview of iterative design cycle findings

Low-fidelity prototype studies

During the early iterations of our user studies, we wanted to evaluate multiple prototypes. Therefore, we rapidly designed paper-based low-fidelity prototypes [28] for the first two user studies. From these studies, we found that although participants liked the idea of having pictorial representation of different features, it was difficult for them to recognize what feature the picture represented. Furthermore, we were able to discover the complexities involved in designing an interface for medication reconciliation.

We realized CCT could benefit from using some of the MyMediHealth PHA components from fellow Project HealthDesign grantees at Vanderbilt University to help remind older adults when to take their medications with text messages sent to their mobile phones. Since the MyMediHealth scheduling component was designed for caregivers to schedule medications on a traditional computer for their children with cystic fibrosis [34], we had to investigate what would be the best way to help older adults and caregivers schedule medications on a touchscreen tablet. Thus, we integrated tasks into user studies 4 and 5 in which users interacted with low-fidelity prototypes for medication scheduling. We found that participants performed the best on prototypes that showed the least amount of time.

Study 1: The overall look

Goal

Identify the look and feel of CCT.

Study design

The study design was informed by the CCT qualitative needs assessment results [29]. We found that older adults were most interested in learning about their medications and receiving reminders to take some medications. These questions led us to design three low-fidelity prototypes. The first prototype consisted of an analog clock, shown in Fig. 2b, that had some aesthetics borrowed from the flower prototype (Fig. 2a). The clock had medication images near the corresponding time they should be taken. Some screens on the clock prototype had a menu bar on top that contained links to CCT features.

In addition, we found that the target population organized their PHI around the house based on context and routine. For example, a calendar was prominently featured in participants’ homes and was typically located near a well defined PHI management area. Based on these findings, we created the kitchen counter prototype, shown in Fig. 4. Participants accessed information about their medication management regimes by clicking on the various objects on the counter. The final interface, shown in Fig. 5a, was informed by common information management interfaces that are easy to develop. This prototype used two menus (horizontal and vertical) and had a lot of text.

Fig. 4

Kitchen counter prototype

Fig. 5

Evolution of CCT

During our first expert review session, we discovered that doctors’ number one concern for older adults during care transitions was medication reconciliation. We explored what medication reconciliation meant to patients by conducting semi-structured interviews.

Results

We recruited four participants—three older adults and one caregiver. Study 1 showed that participants liked the visual qualities of the clock and kitchen counter prototypes. They found the textual prototype confusing. Furthermore, although participants liked the idea of having pictures of different objects in the kitchen counter prototype, they did not understand the meaning of the pictures. Participants wanted directions on each screen. In addition, participants had difficulty telling the current time on the clock because it presented all 24 h instead of 12 h.

The results of semi-structured interviews were particularly interesting because we found that patients believed doctors did the medication reconciliation since the doctors had EMRs. Conversely, doctors knew that they did not have complete medication lists because EMRs are not interoperable across hospitals. In addition, doctors believed that patients did medical reconciliation because patients are expected to keep their medication list. When we described medication reconciliation further to participants, one participant remarked that it sounded like breaking into his doctors’ files.

Study 2: Refine medication management

Goal

Refine the look of the prototype and define medication management interfaces.

Study design

Findings from the first study revealed that the interfaces we developed were either too abstract or had too much information on the screen. Therefore, for study 2, we further investigated how to integrate pictures and text on a single organized menu bar. We developed a basic medication management prototype that provided participants with an interface to create a medication list. Finally, we designed and evaluated multiple prototypes for medication reconciliation. We started study 2 with a card sorting exercise to identify what pictures and associated text to put on the menu. Participants were asked to sort twenty picture cards containing various health related images. The first sorting exercise had participants sort the pictures into any piles they wanted to and describe how they created each pile. For the second sorting exercise, we told participants the categories (e.g., medication information) and asked participants to sort cards based on these categories. At the end of each sorting exercise, participants ranked the cards within each category.

Since the primary aim of CCT was to provide users with a way to manage their medications, we created a prototype for basic medication management. The prototype provided participants an interface to create a medication list. Based on the feedback from the first study, we designed a medication list creation wizard where participants could read instructions on each screen and navigate between wizard screens to add their medications. Medications were added by typing in the name of the medication.

We designed four different interfaces to study medication reconciliation. Although study 1 showed that participants did not believe they had to do this activity, the expert review panel feedback encouraged us to present medication reconciliation in different ways to explore if it would resonate with participants. The first interface (Fig. 6a) provided participants the opportunity to see what medications were on each doctor’s list. The second interface (Fig. 6b) was informed by the feedback from a participant in study 1 and looked as if the participants were viewing their doctors’ files. The third interface (Fig. 6c) was borrowed from standard database list interfaces and had the participant add or delete medications to make the lists match. Finally, the fourth interface (Fig. 6d) was a simplified design where participants answered simple yes/no questions about list inconsistencies.

Fig. 6

Medication reconciliation prototypes

Results

We recruited four older adult participants. The card sorting exercise helped us identify what pictures represented the appropriate CCT functionality for the menu structure. For the medication input wizard part of the study, we found that participants had enough difficulty adding medication names and did not want to be burdened by inputting dose and schedule information while creating their initial medication list. In addition, participants wanted a less textual way of adding medications—typing was too time consuming.

The results of testing medication reconciliation prototypes revealed that participants did not want to reconcile their medication lists even if they knew that their doctors could not perform medication reconciliation. Interestingly, an overarching concern for participants during the medication list creation and medication reconciliation was how their PHA list would affect their doctors’ medication lists. Participants were concerned that somehow they would modify their doctors’ medication lists. We found that if the participants had to choose one medication reconciliation interface, they would prefer the fourth interface because the computer did most of the reconciliation for them and they only had to answer a few questions instead of clicking on each doctor to compare lists.

Studies 4 and 5: Medication scheduling

Goal

Identify appropriate medication scheduling interfaces for caregivers and older adults.

Study design

Medication scheduling requires two basic tasks: (1) determining the current time and (2) scheduling a medication at a desired time. The clock prototype from study 1 was not intuitive to users because they generally found it difficult to determine the time on the 24-h clock. The expert review panel reminded us that for scheduling medications, people may have to schedule medications throughout a 24-h period. Thus we had to accommodate the users’ natural perceptions of time while providing the ability to schedule times throughout a day. We developed seven different prototypes in three basic themes—circular, sliding, and digital—that could be used for displaying the current time and scheduling medications.

The first circular “sun dial” metaphor prototype, as shown in Fig. 7a, consisted of a a pie chart equally divided into 24 pieces that was color coded from yellow to orange to blue depending on what time of day was represented. Only eight pieces of the dial were displayed in a semi-circle form—participants could drag the clock dial to see the other sections of the sun dial. The current time was indicated by a maroon striped clock arm. Patients could drag medication pictures from a medication list onto the sun dial to schedule that medication.

Fig. 7

Circular medication scheduling prototypes: a Sun dial and b 24-h clock

Although the 24-h clock from study 1 was not well received, we continued to iterate on a 24-h clock prototype to provide users the ability to see an entire day’s worth of medication scheduling. For the second circular prototype, we attempted to better delineate each part of the day such that each hour was represented by a line or a small circle on the perimeter of the clock as shown in Fig. 7b. The current time was displayed by a maroon striped clock arm.

The horizontal time prototype (Fig. 8) consisted of a horizontal bar with 24 equally divided vertical sections. The sections were labeled from 1 to 12 and then again 1–12 representing hours of a day. Similar to the sun dial prototype, we used colors to represent which part of the day a strip represented. The current time was represented by a black vertical bar. Participants interacted with the prototype by sliding it side-to-side (sliders) to view specific times of the day. Users could drag the medication picture and place it on the appropriate strip to schedule the medication at that time. We also designed two variations of the horizontal time prototype, one that grayed out the time of day that had already passed and the other that was a vertical time sliding prototype.

Fig. 8

Horizontal slider clock medication scheduling prototype

The final two prototypes were digital clock prototypes as shown in Fig. 9. In the first prototype, we displayed the current time in digital form along with AM and PM. An adjacent box contained pictures of medications along with the time they were scheduled to be taken. In the second variation, when the scheduled time for the medication was within an hour of the current time, the time remaining was shown as a count down of minutes.

Fig. 9

Digital clock medication scheduling prototypes: a schedule of upcoming medications and b count down until next medication

For all of the medication schedule prototypes, if there were more than one medication scheduled at the same time, a small red circle appeared (Figs. 7a and 9a, b) that contained the number of medications scheduled for a particular time. When the users pressed the circle, they would see the list of the medications scheduled for that time. The medication list contained the name of the medication and a picture of it.

The prototypes were integrated into a paper-prototype mock-up of the CCT home screen interface. Participants were shown the prototypes in semi-random order so that no participant viewed the same order of prototypes. Two older adult participants wanted to complete the study together, so they did see the same order of prototypes but were given equal time to comment on the prototypes. For each prototype, participants were asked to identify the current time, next medication scheduled, how much time was represented, and preferred interaction with the prototype (e.g., dragging, clicking). In addition, participants discussed how they would schedule medications with the prototype.

Results

Overall, the eight participants (five older adult and three caregivers) performed better when interacting with the digital clock prototypes. Participants did not think the multiple medications icon—a small red circle—was easy to understand, but the participants were able to identify what it meant after the first time it was introduced to them. The color selection for time of day (orange to blue) was not intuitive—instead participants wanted to see AM and PM clearly delineated. In addition, the colors selected for time of day made some of the orange and blue pills difficult to view. Most of the participants liked interacting with the system using drag-and-drop.

Older Adults were able to complete all of the identification tasks when using the count down digital clock. The second best performer interface was the scheduled upcoming medications list digital clock, but two older adults found it difficult to schedule medications with this interface. These findings are interesting when considering that most older adult participants preferred the scheduled medications list digital clock prototype. Two participants liked the 24-h clock prototype because it showed the entire day. Overall, older adult participants were able to identify the next medication and three older adult participants were able to schedule medications with the 24-h clock prototype. Older adults did not like the sliding prototypes, although most older adult participants were able to identify the current and next scheduled medication time.

Caregivers were able to complete all of the tasks when using the digital clock with the scheduled medications listed. Two caregivers struggled to schedule medications on the second best performer interface—the count down digital clock. Most of the caregivers preferred the 24-h clock or the “sliding” prototypes (horizontal or vertical prototypes) because they were more concerned with scheduling medications for the person they cared for, however only one participant was able to successfully identify the next medication and schedule medications with these prototypes. Caregivers did not like the sun dial prototype, although most participants were able to identify the next medication time and schedule medications with the prototype. In addition to the current prototypes, caregivers wanted more information next to each medication, such as the dosage. One caregiver was concerned how the interfaces would look if multiple medications were scheduled for a specific time period. The caregivers thought dragging and dropping medications was intuitive. One caregiver mentioned how her preference was influenced by the type of pill box she used for her mother and thus selected the round, 24-h clock because it was similar to her mother’s round pill box.

High-fidelity prototype studies

After obtaining sufficient information about user needs and interface expectations from the first two studies, we were able to design and evaluate high-fidelity prototypes for the rest of the studies. Initially, we developed a high-fidelity prototype using images and HTML for study 3, while for studies 4 and 5, we used Adobe Flex. Finally, the prototype for study 6 was developed using PHP, JavaScript, and HTML.

Study 3: Refine medication list management

Goal

Refine medication list management.

Study design

The results of study 2 indicated that the interface must convey to participants that the information was their own personal information and that it was independent from their doctors’ records. While brainstorming the design of the medication list, we noticed that during our needs assessment, the patients mostly had hand written medication lists, whereas the lists they were given from their doctors were printed. Hence, we designed the medication list interface (Fig. 5b) to display the participant’s medication list in comic sans handwriting superimposed on a notebook page image—borrowing from artifacts the participants’ used in their everyday lives [33]. We designed an organized menu bar that contained pictures and text to represent the appropriate CCT functionality based on findings from study 2.

We also brainstormed new ways to input medications to decrease the input time. We included two more methods for adding medications: pharmacy fulfillment and barcode scanning. For the pharmacy fulfillment interface, we assumed that CCT was able to connect to the pharmacy system and obtain the list of medications that a participant had recently picked-up. These medications were displayed on a screen where participants were asked to select the medications that they wanted to add to their medication list. The barcode scanning method required the user to scan the barcode on the medication bottle by using a cordless barcode scanner. Alternatively, the user could enter the barcode number to add the medication. In addition to these modifications, we further simplified the interface for adding and removing medications. Finally, since study 2 results showed that older adults were not willing to do medication reconciliation, we automated it and provided participants with a way to communicate these issues to their doctors.

Results

We recruited eight participants—seven older adults and one caregiver. Study 3 findings were particularly interesting because it was the first time we used the tablet PC-based prototype. Although none of the participants had used a touchscreen device before, everyone liked the idea of using a touchscreen application to manage medications. Participants were interested in how much the system would cost. We found that the touchscreen did not react well to participants’ fingers—we had to hold the screen rigid during the sessions.

The participants liked the idea of adding medications by scanning the barcode, but did not think typing barcode numbers on the touchscreen was convenient. Participants also liked the pharmacy fulfillment interface as it required very few input steps. In addition, participants understood that they were editing their own medication list and not their doctors’ medication list. All of the participants found the menu bar easy to use.

Study 4: Common questions

Goal

Verify medication management and identify common questions patients have during care transitions.

Study design

After analyzing the results of study 3, we developed a more robust high-fidelity prototype to evaluate how older adults could add and remove medications, navigate through the interfaces, and edit an established medication list. Apart from modifying the medication management interface, we worked on addressing another issue that had emerged during the needs assessment study where patients had difficulty communicating with their doctors. Consequently, we conducted semi-structured interviews to explore what common questions and concerns the participants had and how would they like to share this information with their doctors.

Results

We recruited five participants—four older adults and one caregiver. Study 4 results showed that generally, the participants performed the basic medication management tasks comfortably. As in study 3, participants liked adding medications by scanning the barcode. Participants had difficulty editing the medication list when the medication item and action was not explicitly linked. For example, participants were not sure if they deleted medications properly because the medication was not highlighted when selected and appropriate feedback was not presented once the delete action was completed.

Participants were enthusiastic about communicating with their doctors by asking questions using the CCT. The participants provided different questions that they would ask their doctors. For example, participants wanted to ask “What will be the side effect of this medicine?” All the participants said that they asked their doctors questions during their appointments, but they usually forgot to ask important questions.

Study 5: Sharing concerns

Goal

Confirm medication management enhancements and evaluate the appointment preparation module.

Study design

In study 4, once we confirmed that participants could perform the basic medication management tasks, we developed a set of wizard screens for study 5 that would help a participant set-up CCT the first time they started the application. In addition, we created a wizard that would be linked to a hospital system and prompt the participant to confirm their medications after they were released from the hospital. We created a “Prepare For Appointments” wizard, based on the semi-structured interview results from study 4, where participants could verify their medication list, select common questions, and share this information with their healthcare providers. The common questions and concerns were “stubs”—incomplete sentences that captured the question, but did not provide specific information. They were designed to help remind the participant about their question without requiring too much typing input. For example, a participant could select, “Is there something I can take besides...” We created these stubs because in previous studies, participants wanted to have minimal interaction with a keyboard.

Results

The three participants (one older adult and two caregivers) were able to easily complete the two set-up wizards and modify medication lists by adding and deleting medications. They were concerned with the simplified pharmacy fulfillment input because they did not get all of their medications or supplements from pharmacies. Additionally, participants liked the idea of preparing for appointments and thought the stubs could provide them with enough information to remember what they wanted to ask their doctor. They also wanted the ability to fill in more information in case they did not have an appointment in the near future. When we presented these results to the expert panel, we found that the doctors also wanted more information so that they could look at common questions and identify possible complications that should be evaluated before the scheduled appointment.

Functioning prototype studies

The key difference between the high-fidelity prototypes and the functioning prototype was that the high-fidelity prototypes (studies 3–5) used a local MySQL database for data storage and information access (e.g., mocked-up medication information databases), whereas the functional prototype (study 6) was integrated into an interoperable PHR system and linked to authoritative information. The CCT functional prototype interconnected with four different systems: (1) a common, interoperable platform PHR (projecthealthdesign.org/resources/common_platform); (2) RxNav web services; (3) Micromedex; and (4) a local MySQL database. The Common Platform provided a PHR repository to store medications present in the medication list. The RxNav web services were used to convert medication names into codified entries, including National Drug Codes (NDCs), that specified unit strength, dosage form, brand names, and generic ingredient. The Micromedex database provided authoritative medication information and medication images. The local MySQL database stored information about patients’ fake pharmacy fulfillment data, worsening symptoms, and common questions.

Study 6 had two parts—the first part was to evaluate the enhancements made to CCT for adding medications and tracking participant symptoms, whereas the second part was designed to assess how participants could use CCT in comparison with a mainstream PHA, Google Health. Although both parts were conducted within the same RITE session, we divided the studies into two parts to better describe the methods for each study.

Study 6.1: Enhanced PHA management

Goal

Evaluate the entire system design.

Study design

Previous iterations of the CCT prototype had two main medication input mechanisms: pharmacy fulfillment and barcode scanning. In the real world, however both the aforementioned methods cannot be easily implemented since most of the pharmacy systems are not interoperable and medication barcodes are not standardized. This argument was further bolstered during study 5 when participants raised concerns that they may want to add medications that do not come from the pharmacy, such as herbals or over-the-counter medications. Consequently, we designed a wizard where users could add a medication by entering the medication name using a touchscreen keyboard. If the users spelled the medication name incorrectly, CCT would suggest the correct spelling or alternative medications. Otherwise, CCT displayed the different strength and forms of the queried medication. Once the users selected the desired strength and form, CCT would display a set of images associated with that medication. Alternatively, we provided users with an option to select a generic medication bottle image in case none of the images matched the medication they had. When the users selected the image, they were shown the medication name, strength and form, and image so that they could confirm whether it was the correct medication to add. Furthermore, from our qualitative studies, we found that often times older adults remembered the physical appearance of their medications, such as the “blue pill” rather than the actual medication name. Therefore, we provided an option to add medications by entering free text to accommodate these nicknames.

Another major enhancement we made in CCT was informed by earlier work on paper-based PHRs during transitions of care [7] where providers and patients wanted a mechanism that could be used to understand what symptoms require immediate medical assistance. Thus, we developed a wizard for “Red Flags.” Red flags consisted of different statements such as “I developed a fever of more than __ degrees.” We gathered seven common red flags statements from doctors and provided an “Other __” option where participants could provide any symptom they thought should be recorded.

Results

The seven participants (three older adults and four caregivers) found that adding medications by entering the medication name was more complex than the other two methods since the former involved multiple interaction steps. Some participants mentioned that the instructions on different screens of CCT could be improved. The participants easily navigated the red flags interface and expressed that it would definitely help them monitor their own or loved one’s worsening symptoms and share it with their doctors. Additionally, the ease with which the participants performed common medication management tasks further verified the design of our medication management interface. Similar to study 3, some participants found it difficult to use the touchscreen of the tablet PC because it did not respond very well to their fingers.

Study 6.2: Comparison study

Goal

Compare the CCT system design with a mainstream PHA.

Study design

Based on the collective experiences from the needs assessment, expert review panel, and feedback from participants in the previous iterative studies, we identified the top four most common tasks that people performed while managing medications: (1) viewing a medication list, (2) adding a medication to the medication list, (3) deleting a medication from the medication list, and (4) obtaining information about a medication. We wanted to compare how participants interacted with CCT and another mainstream PHA, however we did not want to overwhelm the participants with multiple PHAs. Thus, we evaluated two freely available, Internet-based PHRs, Google Health and Microsoft HealthVault, in October 2008 to identify which PHR would be easiest to use by the target population. We assumed that Google Health would be easier for older adults to use in comparison with Microsoft HealthVault because the tasks took fewer clicks to complete in Google Health than in Microsoft HealthVault. In addition, the interface had a more simplified navigation structure, although at the time there was not an easy way to share medication information. We later confirmed these results when we conducted a cognitive walkthrough on Google Health, Microsoft HealthVault, and CCT [32].

Based on these findings, we asked participants to complete the four common tasks on both—CCT and Google Health. Six out of seven participants self reported that they used a computer regularly. Based on our study observations, we have categorized three participants as inexperienced with using computers because they struggled to complete basic navigation tasks within the web browser and on web sites. Half of the participants interacted with CCT first and half interacted with Google Health first to complete the tasks. Here we provide a brief overview of the overall design and navigation structure of both PHAs.

Google Health

This overview of Google Health is from the version that was available in January 2009. We acknowledge that the Google Health interface has changed significantly since we conducted this study, however some interface components have remained.

When the user logs into Google Health (see Fig. 10a), she is presented with the Google Health home screen. This screen consists of a three-column information layout. The medications taken by the user are shown on the right most column. The medications are ordered vertically and in alphabetical order underneath the user’s health data (e.g., age, height, conditions, etc.)—thus the vertical position will vary as the number of health conditions listed in the PHR are modified.

Fig. 10

Google Health interface screens: a home screen with a three-column information layout; b medication list interface with a two-column information layout; c deletion confirmation box; d adding a medication through predictive text input; and e medication addition confirmation screen

A user can click the “Medications” link in left-most or right-most column to view the detailed medication list (task 1) shown in Fig. 10b. As one can see, the information layout changes from a three-column layout to a two-column layout. Each entry in the medication list has an associated “Add record,” “Edit,” and “Delete” link on the right side of the screen. To add a medication (task 2) from the home screen, a user can press the “Medications” link on left or right column, and then click the “Add medications to profile” link on top of the screen that is highlighted with a double green line in Fig. 10b. Alternatively, the user can click “Add to this Google Health profile” link on the home screen and then click the “Medications” link in the center column. Users can add a medication in two ways: (1) selecting from an alphabetical list of medications or (2) typing in a medication name. When typing the name of the medication in the text field, Google Health provides medication name suggestions corresponding to the text entered as shown in Fig. 10d. As soon as the user clicks the “Add” link next to the medication name text field, the medication is added to the medication list, and the confirmation is displayed by highlighting the newly added medication on the right-most column of the screen (Fig. 10e). If the user wants to obtain more information about a medication (task 4), she can click the “More info > >” link present at the bottom of each medication name in the medication list. Clicking this link opens up a new web-page (a new tab if the browser supports multiple tabs) that contains authoritative information about the medications, e.g., drug interactions, overdose, precautions. To delete a medication (task 3), the user clicks the “Delete” button link associated with that medication entry row. Upon clicking the “Delete” link, a confirmation box is displayed which asks “Are you sure you want to delete this record?” as shown in Fig. 10c. Upon clicking “OK,” the medication is removed from the medication list.

Colorado Care Tablet

The home screen of CCT consists of a two column layout as shown in Fig. 11a. The bottom of the screen contains a navigation menu with links to more CCT functionality. The medication list is displayed (task 1) on the left column of the screen with accompanying medication pictures, medication names, unit strengths, and generic ingredients. The right column contains a large picture of the medication selected in the medication list along with the buttons to (1) obtain information about the medication (task 4), (2) modify the medication schedule, (3) delete the medication (task 3), and (4) add a medication (task 2). Similar to Google Health, the user only has to confirm that she would like to delete the medication through a pop-up confirmation box (Fig. 11b). If the user would like to obtain more information about a medication, she selects the medication from the medication list and then selects the Drug Facts button to navigate to the drug facts overview screen (Fig. 11c). When the user selects a drug fact topic button, she can view more information specific to that topic (Fig. 11d).

Fig. 11

CCT interface screens: a home screen with a two-column information layout; b deletion confirmation box; c medication information screen; d example screen with information for “Possible Side Effects”

We designed a step-by-step, wizard-based interface to provide users the ability to easily add medications into CCT as shown in Fig. 12. Users can choose to add a medication in three ways: (1) entering a barcode or prescription number; (2) entering the medication name; or (3) selecting from pharmacy fulfillment data. The most complicated input mechanism is entering the medication by name, thus we provide an overview of this wizard in Fig. 12. Since CCT has a touchscreen interface, the user enters the medication name with an on-screen keyboard. After entering the desired medication name, the user is presented with a list of medications derived from the user’s query. Once the user selects the desired unit strength, a confirmation screen with a large picture of the medication is displayed as shown in Fig. 12f.

Fig. 12

CCT wizard to add medications by typing in the medication name

Results

Generally, the seven participants (three older adults and four caregivers) found CCT easier to use than Google Health because the interface was simple to understand and use. One participant mentioned that CCT only showed what was needed. Most of the participants were more comfortable navigating CCT in comparison to navigating the multiple menus in Google Health.

Participants who had experience with using computers (N = 4) were able to easily perform medication management tasks and perceived that they were able to complete the task in fewer steps than CCT. Despite the positive impressions by those with computer experience, participants with limited computer experience had difficulty understanding the Google Health interface. Two participants explicitly mentioned that Google Health had a long learning curve, while another participant, who had the least computer experience, failed to perform a single medication management task in Google Health. Participants with limited computer experience were overwhelmed with all of the options presented to them and were unsure of where to click first.

Viewing medication lists and adding a medication

In Google Health, participants struggled to navigate to the medication list screen. Once they navigated to the medication screen, they found it relatively easy to add the medications. Participants had mixed feedback regarding the predictive drop down text feature for adding medications shown in Fig. 10d. While some participants thought it was useful, others got confused in selecting the medication of their choice. One participant complained that she was unsure whether the medication had been added because, at the time of the study, Google Health simply added the medication to a list on the right of the screen as shown in Fig. 10e. Indeed, participants inadvertently entered the same medication multiple times before they noticed that the medication was successfully added. Participants preferred to type in the name of the medication instead of searching for a medication name in the alphabetical list because the list was too complex and time consuming.

Participants easily viewed their medication list in CCT. When adding medications in CCT, most participants liked that the images were paired with medications, however some were concerned about the possibility that the images provided on the Medication Image Selection Screen (Fig. 12e) would not match their specific medication they were adding. Although CCT provided users the option to select a generic pill bottle instead of a medication image, participants did not select the generic pill bottle. Some participants had difficulty using the touchscreen keyboard. For example, one participant could not find the backspace button on the touchscreen keyboard even though the keyboard design had a standard US English keyboard layout. Generally, participants did not have difficulty in adding medications in CCT.

Deleting a medication

Google Health provided a link to delete a medication associated with each item in the medication list. Although most of the participants deleted medications easily, two participants had difficulty determining how to delete a medication. In addition, one participant voiced some confusion about the confirmation box because it called the medication a record and did not note the medication name (it read, “Are you sure you want to delete this record?”). Similarly, in CCT, one participant had difficulty in locating the link to delete the medication.

Obtaining more information about a medication

When the participants were asked to obtain more information about a medication in Google Health, most of them entered the medication name in the “Search the web” text field. This navigated them away from Google Health to a new window or tab. We had to guide most of the participants about how to look for more information about a medication within Google Health.

Participants found CCT easy to use when searching for information about a medication. They appreciated how the information was broken down into various subcategories. Participants pointed out that this feature was much easier in CCT as compared to Google Health.

Discussion

We set out to design a PHA that was iteratively designed by all stakeholders—older adults, caregivers, and doctors—that could assist older adults with managing their complex medication regimes. The six studies described in this paper gave us a better understanding of their needs, wants, and realistic expectations for medication management. In addition, we were able to share our findings with medical experts in care transitions and alert them about misconceptions in their own expectations of patients’ responsibilities (e.g., medication reconciliation). Here we present design guidelines to help future health informaticians design effective PHAs and PHI management systems. We conclude with a brief discussion on the limitations associated with our studies, a retrospective, and future work.

Guidelines

During the studies, we observed that the participants initially wanted the ability to perform basic PHI management tasks. Once they were able to easily perform basic tasks, they desired additional capabilities for an improved PHI management system. For example, our preliminary prototype provided the participants with the ability to manage medications, associated dosages, and scheduling information. The participants were mainly concerned with recording their medication list first and then iterating on the list to include dosage and scheduling information at a later time. Consequently, we focused on the basic medication list design. In later studies, the participants could easily create medication lists and inquired about the dosage and scheduling information. Similarly, we designed the common questions stubs because the participants wanted minimal interaction with the keyboard. Once they understood the stubs and were comfortable with interacting with CCT, the participants sought the ability to fill in more information regarding their health. Based on these findings, we urge the community to design a basic PHA with the option to add advanced PHI management functions as users grow accustomed to basic features. These findings also highlight the importance of the iterative participatory design approach—participants’ active involvement in the studies facilitated our identification of their increasing PHI management needs.

When designers provide participants the ability to select additional functionality, they must provide understandable instructions on how to add the new features and confirm the additions. In study 1, participants told us that they wanted instructions on each page. The research team noted that the participants had difficulties understanding the instructions provided on the subsequent CCT prototypes. Indeed, even in the last study, participants continued to provide suggestions on how to reword the instructions. During studies 2–6, participants frequently asked questions that were answered by the instructions. Consequently, we adjusted the instruction layout and altered the instruction wordings. This helped the participants locate and understand the instructions. In addition, we found that participants were unsure if medications were deleted (study 4) or added properly (study 6.2 with Google Health), because the interface did not provide sufficient feedback to participants through pop-up windows or confirmation of a successful action on the main area of the screen layout. Based on these findings, we suggest that the community pay special attention to placement, readability, and wordings of instructions and confirmations on PHA interfaces.

In addition, we saw that there was sometimes a clear divide in preferences between older adult and caregiver participants. For example, in studies 4 and 5: medication scheduling (Section “Studies 4 and 5: Medication scheduling”), older adults preferred the digital clock prototype, whereas caregivers preferred the 24-h clock or sliding prototypes. If we were to design the scheduling interface for CCT, we would have decided on the digital clock prototype because all of the participants were able to perform the tasks with the digital prototypes, whereas some older adults struggled with the 24-h and slider prototypes. If we could develop CCT further and differentiate the interfaces between older adult and caregiver users, then we would accommodate the preferences of each group in their respective applications. Currently, CCT uses a 24-h scheduling interface, designed by fellow grantees at Vanderbilt University Department of Biomedical Informatics [34], where 12-h blocks of time are stacked horizontally. Our findings for caregiver preference confirm their design because their 24-h design was created for caregivers to schedule medications for their children. Our general guideline was to prioritize performance over preference when deciding which interface to select for the PHA. We do note, however that if participants strongly prefer an interface that they do not perform well on, the designers can choose to iterate on that concept.

One of the emergent themes from our qualitative studies was similar to Leysia Palen and Stinne Aalökke’s [26], and Anne Moen and Patricia Brennan’s [22] work that physical reminders were important for medication management. Based on this theme, we developed the kitchen counter prototype where we anticipated that pictures of various objects on the kitchen counter would remind participants about different medication management activities. The results of study 1, however revealed that participants could not derive meaning from interfaces that were too abstract. In contrast, we found the target population felt overwhelmed when viewing the textual interface. We found the right balance by displaying intuitive, participant selected, health-related images paired with meaningful, informational text. From these findings, we suggest that health informaticians work with all stakeholders to find the right balance between metaphor and textual information to effectively present health information. Our guideline complements earlier research done by Roger Morrell and Denise Park [23] where it was shown that older adults made fewer errors following instructions composed of text and images as compared to text only instructions. Interestingly, Suzanne Prior and colleagues [27] reported that older adults found an instant messaging interface based on a cafe setting more intuitive than a traditional chatting interface. These findings differ from our findings where older adults did not understand the meaning of the pictures in the kitchen counter prototype. The differences in findings may be attributed to the different application mediums (health versus everyday communication).

We found the balance between metaphor and textual interfaces by conducting specific exercises during studies. For example, card sorting exercises identified the most meaningful metaphors for menu icons. The notebook image, informed by our prior work [29], helped participants understand that they were editing their own medication list and not their doctors’ lists. Once the participants had seen the medication list with the notebook image, they easily understood that it was their list being modified even if displayed differently later in the prototype as shown in Fig. 5c [33]. We envision future systems where participants can select their own images to specifically help them identify icons. For example, they can use an image of their calendar or pill box to denote scheduling functionality. Of course, a trade-off with this customizability is that it increases the amount of input—something that was not favorable with the target population.

An important finding during our studies was the participants’ preference for automated mechanisms. Barcode scanning was preferred over pharmacy fulfillment and entering a medication name because it was the modality that required the least interaction. Similarly, the participants favored automated medication reconciliation over self-comparison of medication lists. This suggests the design of PHAs for older adults should have automated mechanisms that require minimal interaction steps to perform basic medication management tasks. Kerrie Laguna and Renée Babcock [18] reported that computer anxiety in older adults increases with the amount of time required for older adults to make a decision. From this, we surmise that if there are more interaction steps, then older adults will require more decision time resulting in higher anxiety. Marquié et al. [20] showed that older adults underestimate their actual computer knowledge. Thus, we argue that if PHAs are designed to automate basic tasks with minimal interaction steps, older adults will require less decision time resulting in less computer anxiety and higher self-efficacy in completion of medication management tasks. One may wonder, “why not use paper-based medication lists?” As mentioned earlier, paper-based medication lists are often illegible, out-of-date, and difficult to share with multiple providers or remote caregivers.

One of the major findings that affected the participants and expert panel members was the obvious conflict between doctors’ and patients’ understanding of medication management. Doctors in our expert review panel assumed patients compared their medication lists with their doctors’ lists and notified the doctors of any list discrepancies, whereas patients assumed doctors had all of the medication list information and did the comparisons themselves. Indeed, the reactions from the two groups were quite interesting—the doctors were surprised participants did not already reconcile their medications—especially since participants’ safety was at risk—and thought perhaps more educational programs were needed to alert patients of medication reconciliation importance. Whereas, participants were surprised to hear that EMRs were not interoperable. In addition, they were not interested in doing medication reconciliation because it was too much work—if the doctor did not do it, then the computer should do it.

Based on these findings, we encourage the community to explore PHI management issues with all the stakeholders to avoid any preconceived notions from creeping into the design of the PHA. This guideline requires effective communication between participants, health experts, interaction designers, and health informaticians to understand each other’s perspective about PHI management. This may reduce the transfer of real world interdisciplinary misunderstandings into the PHAs.

We draw another guideline from studies 2–5 where the participants expressed the need for multiple, easy ways to input medications. We iteratively designed new input mechanisms—some not possible in today’s diverse, non-interoperable healthcare systems—that provided participants an easy way of creating medication lists. As we discussed in study 6.2, experienced computer users perceived that they were able to add medications in fewer steps, although both Google Health and CCT took the same amount of steps to add a medication, delete a medication, and find medication information [32]. This perception could be attributed to the use of a conventional keyboard when using Google Health because the participants could enter information and change text fields with keystrokes instead of selecting information with the touchscreen on CCT. From our experience, we propose that the community should design PHAs for the future, while at the same time acknowledge the current limitations and opportunities.

We utilized this guideline from the beginning of our CCT design process. We initially decided to use a touchscreen computer because we envisioned a future similar to Mark Weiser’s ubiquitous computing vision where people could interact with any surface to access a computer [39]. Despite designing for the future, we had to deal with a touchscreen computer that most participants had difficulty using during the studies. Fortunately, the future started to catch-up during our design process with the release of the touch sensitive, touchscreen iPhone in 2007 and the continued release of inexpensive touchscreen mobile phones. At the time, we decided the current touchscreen mobile phones did not have enough screen space to accommodate the CCT interface for easy use by older adults, although CCT could be accessed from a mobile phone with Internet access. More recently, touch sensitive touchscreen tablet computers have been introduced to the market. We are encouraged by the continued innovations in touch sensitive touchscreen computers and believe that interactions with CCT on these tablets would be easier for older adults.

The final design guideline is drawn from participants’ interaction with the functional prototype. We found that they used different techniques for entering medication information. Some participants entered just the medication name; some entered the Rx number, while others entered the medication name with strength. The order of these attributes varied among participants. Although the CCT recognized some of the combinations, it failed to interpret the others. Based on this experience, we advise the community to explore the target populations’ different practices regarding specific PHI management tasks. The success of future PHAs will lie in their ability to correctly interpret different input permutations.

This guideline complements Rikke Aarhus and Stinne Ballegaard’s work that suggests the role of a patient changes from hospital to home [1]. At a hospital, the patient will be directed about how to perform PHI management tasks. But at home, the patient will perform PHI management tasks in his own unique way. Furthermore, iteratively working with participants revealed that participants wanted to add health supplements that were not present in pharmacies. Therefore, it is important for the researchers to understand individuals’ unique practices and design future PHAs accordingly.

This guideline is further bolstered by our low-fidelity prototype medication scheduling findings where a caregiver chose a round prototype that showed 24 h worth of medications because it reminded her of the medication pill box she prepared for her mother. Similarly, we can reflect on the roles of caregivers and older adults—caregivers are primarily responsible for scheduling medications, whereas the older adult is more interested in integrating medication adherence with her everyday life [29]. When we reexamine the roles of participants and their preferred medication scheduling prototype, it is not surprising that caregivers preferred prototypes that showed an entire day for scheduling medications and older adults preferred prototypes that showed when their next medication was scheduled. The two older adults who preferred the 24-h clock prototype were also caregivers for their respective partners.

Limitations

Although we successfully designed a PHA for older adults to manage their complex medication regimes in a laboratory setting, we acknowledge limitations to our research—namely small sample sizes and differences in participant roles for managing medications. We had only four users in most of the user studies. Although these user study numbers may seem small, researchers have found that conducting usability studies with as few as 4–6 participants can provide enough data to determine the effectiveness and usability of a system [8, 37]. In addition, the iterative nature of our study design provided us ample opportunity to confirm previous findings with participants in follow-up studies.

Another limitation is that during the last two studies, there were only three older adults while the rest of the participants were caregivers. In most cases, the caregivers were younger than the older adults, thus this may have skewed our results for confirming the system with caregivers and future older adults.

Finally, in study 6.2, we compared CCT, a PHA specifically designed for medication management, with Google Health, a general purpose PHR, thus this may have skewed our results towards CCT. We contend that the findings (e.g., confusion using predictive text; insufficient confirmation for adding medications) are generalizable for future PHA improvements.

Retrospective

Based on our time and resource constraints, using an iterative participatory design methodology informed by RITE and IDA fit our needs well. We could get feedback from participants, review the results with the expert review panel, and continue iterating on a functional CCT prototype in 1.5 years with only two graduate research assistants. Low-fidelity prototyping was also a good choice early on in our design cycle because we could easily mock-up multiple interface ideas without fully programming the interface. Finally, the expert review panel was extremely valuable—it was beneficial to have a group of people interested in CCT, but not committed to the design process. Thus, they could pose insightful questions and suggestions after reflecting on our designs and results.

If we were to design CCT from scratch again, we would make the user sessions more meaningful to the individual participants by preloading the participant’s (or participant’s loved one’s) medication information and tailoring the scenarios accordingly. For example, participants were asked to add Premarin to CCT during a study. Premarin is a medication used to treat vaginal dryness. A male participant commented that he would not use Premarin because it was for women. Although we tried to create scenarios that were realistic, they sometimes did not resonate with participants. If we had used real user data in CCT, we would have needed better security measures on the prototype and coordinated with participants’ clinics to get the data. In addition, our IRB approval was for an expedited study. If we had used real participant data, we would have had to have a full board IRB review which would have impacted how much time we could iterate on the prototype depending on IRB protocol review times.

Where do we go from here?

From the perspective of designing PHAs for medication management, we urge the community to research alternative ways to input medications. For example, providing voice input or entering medications by taking pictures. The former example has some limitations because some medications are difficult to pronounce, but the free-form recorded text could easily capture participants concerns when recovering from a care transition. In addition, researchers have had some success in creating systems to recognize a subset of medication names [24], although speech recognition for drug names is difficult and has safety issues [12]. The picture input could also assist caregivers. For example, caregivers often discussed with us the problem of suddenly becoming a caregiver after an independent loved one becomes ill and needs immediate help. In these situations, caregivers would like tools to assist them to quickly transition from outside family member to in-the-trenches caregiver. In this situation, we envision a tool where new caregivers could take pictures of their loved one’s medications and receive information about the medications to assist the caregiver and older adult modify the medication regime per doctor’s prescription. Unfortunately, before any of these ideas can be realized, we must address some of the previously mentioned challenges in this area. Namely, we need a freely available standard library of pictures and medication information. In addition, this repository must have digital input signatures for voice input recognition.

Finally, this paper addresses a small, albeit important, part of personal health information management. The health informatics community needs to look at how to design PHI management systems for different user segments of the population with different conditions. For example, medication management is important for many chronic conditions, but tweens and caregivers dealing with cystic fibrosis medication management will need different applications and tools to help the tween transition from dependent child to informed, responsible young adult—all the while providing the caregiver the piece-of-mind needed to ensure their tween is successfully managing their illness. Thus, we need more longitudinal in situ studies to test the efficacy of PHAs across a spectrum of users with varied medical conditions.