Keywords

1 Introduction and Background

A central tenet of many Human-Computer Interaction methods is ensuring the population for which a technology is designed is involved in the design process. In working with people with dementia, this is especially key, for reasons ranging from the reliance on caregivers as proxies [1] to the challenges in supporting technology adoption and a sense of empowerment [2].

Yet, user-centered design approaches often require modification when being applied with populations with dementia. Researchers have devised new approaches to overcome some of the barriers to designing in dementia, such as the wide gap in the everyday life experiences of designers to those with people with dementia. The KITE approach has been designed to foster empathy in relationships between designers and people with dementia [3]. Similarly, the OASIS method was used to establish “an appropriate atmosphere in the meetings” [3] to bridge the gap between younger designers’ technological insight and need areas for future technology in older participants’ daily lives [4]. The OASIS method consists of four stages: (1) stakeholder identification and recruitment (2) presenting participants with a video illustrating a scenario with future intervention for the problem domain (3) exploratory meetings which define the domain of the project and user requirements, and (4) low-fidelity prototyping sessions [3, 4]. In the work we present in this paper, we took a similar approach of integrating prototyping with video presentations—but in an entirely remote approach.

Past work has found that some of the conventional approaches to brainstorming and generating design ideas can be difficult for people with dementia. Bourazeri attempted to use fill in the blank cards to spur inspiration for the design of sensor technology in participant’s homes [5]. However, this method proved unsuccessful as participants had difficulty understanding how the sensors would work and were unable to think abstractly on the subject [5]. This finding aligns with Hendrik et al.’s work which found that participants with dementia had a hard time making abstractions from visuals of how technology would work [6]. Though this work points to the deficiencies of a visual-based approach in co-design with people with dementia, other researchers have found people with dementia are able to think abstractly using symbols and metaphors during co-design [7]. Although not tested with people with dementia, one method to overcome the potential difficulty of abstraction is to use high-fidelity prototypes which reduce the amount of abstraction needed, which was shown to be useful for users with aphasia—a condition which affects people’s ability to communicate verbally [8,9,10].

In addition to challenges with abstract thought in co-design with people with dementia, challenges with verbal communication has been a subject of interest for research that questions the usefulness of traditional approaches to co-design sessions with this population [11,12,13]. Foley et al. suggest using more embodied interactions in the design process to overcome these verbal speaking challenges [12, 14]. Neate et al. found that using the personas allowed people with aphasia to “optimize their use of language rather than having to use extensive language to articulate a particular thought” [9]. In contrast, Bourazeri et al. had their co-designers with dementia describe their design ideas out loud while the researcher would draw out the ideas and the group would actively critique the design being drawn [5]. To our knowledge, this is one of the only examples of how low-fidelity prototyping in co-design has been adapted for users with dementia [5].

In the study we report on here, we leveraged past findings about engaging in user-centered design with people with dementia to test a novel remote approach. The need for remote approaches is especially applicable for people with dementia as many individuals living with the condition are over the age of 65 and therefore at a much higher risk of suffering severe illness from COVID-19, making in-person research unsafe for the foreseeable future. Additionally, remote methods can be beneficial as a way to allow participants the ability to complete studies in their natural work or home environment rather than an academic or laboratory environment [15, 16]—this is particularly useful in dementia given the importance of contextual cues and routines [17, 18]. When it comes to technology design, researchers do at times include people with dementia in remote interviews and field trials (e.g., [19, 20]). However, co-design, which often centers around interactions with design materials and prototypes, is much more challenging to translate online for dementia.

This paper presents a case-study of the use of remote methods for a co-design process which included interviews, low-fidelity prototyping sessions, and asynchronous evaluation of prototypes with tech-savvy individuals with dementia. Our work focuses on people with mild to moderate dementia, the stage of the condition where individuals experience difficulty with remembering names, word finding, performing tasks, and planning [21] but with the right support are generally still able to manage activities of daily living. The main contribution of this paper includes a discussion of the benefits and challenges of using this method with tech-savvy people with mild to moderate dementia.

2 Method

Below, we describe our approach to remote data collection and analysis. We also provide information on participant recruitment and demographics.

2.1 Data Collection and Analysis

All procedures were approved by the university’s Institutional Review Board. For this study we took a general User-Centered Design approach [22]. See Fig. 1 for a visual of the process we followed and who was involved in each step.

Fig. 1.
figure 1

Study process. phases that are bolded are those that people with dementia are included

Full size image

We first followed the approach of learning more about participants’ lives and needs through “empathizing interviews”—remote semi-structured interviews to understand current technology usage and any issues participants experience. Our goals for conducting these interviews was to scope the scenarios which we would design for in the co-design sessions as well as get participants’ ideas for future technologies. Each interview and observation session was between thirty minutes and an hour in length. The semi-structured format [23] allowed us to guide the participants into relevant discussions while making sure we were not controlling the narratives of conversations. We asked probing, open-ended questions to uncover deeper insights into their experiences. All four interviews were conducted remotely and were audio/video recorded. Observation notes were taken throughout each interview. The audio recordings were later transcribed for analysis using Otter.ai. Participants were compensated with a $10 Amazon gift card.

The research team then used affinity mapping to identify the emergent themes among the findings and insights we gained in the interviews [24]. The research team conducted interpretation sessions on each interview, taking notes from the transcribed interviews and translating each insight to individual sticky-notes. The team then grouped the notes, to reveal emerging themes most relevant to the design problem [24]. The affinity map was created on Miro where we grouped the participants’ needs into three major focus areas: social needs, sensory needs, and technology needs. The research team gathered for a preliminary brainstorming design session where we discussed design ideas for each of these major need areas. We created storyboards that illustrated commonly occurring problem scenarios in the lives of people with dementia, as well as designs for the interfaces of the proposed solutions. Several of these storyboards included sketches of future technology based on ideas participants shared during the empathizing interviews, taking a similar approach to co-design as in past work (e.g., [5]). These designs were then internally critiqued by our research team, where only those designs which best addressed the identified user needs from the empathizing interview and were most novel (in terms of gaps in research literature) were documented in Google Docs.

Different research projects take different approaches to co-design, from end users doing most of the sketching [3] to researchers playing more of the design role with participants generating ideas and feedback [5]. We took the latter, by sketching design ideas previously described by participants. As a way to contextualize and provide structure to these preliminary design ideas, a Google Doc was created to be shared with participants during the co-design session. We started this document with an initial scenario for which the entire design session would focus to provide participants with a concrete example to focus their design ideas. Three primary design questions, which arose during the preliminary design session, were then presented to participants to provide structure for the co-design sessions. For each of these three design questions we presented participants with two example designs, in the form of sketches and story-boards, for their critique and to iterate on. The document was formatted to include one design per page so as to avoid visual overstimulation.

Two members of the research team then conducted a remote co-design session with one participant with dementia via Zoom video-conferencing based on these preliminary designs. This session was approximately one hour long and was audio/video recorded. Before the session, the participant was asked to have a pen and paper handy and was encouraged to sketch and explore their own ideas during the session. During the video call, we shared our screen with the participant (see Fig. 2) to walk them through the various scenarios using our storyboards. The participant was encouraged to ask questions, critique the solutions, and also suggest their own ideas and insights into how the solutions could be further improved. The participant was compensated with a $20 Amazon gift card.

Fig. 2.
figure 2

Google document of remote co-design session

Full size image

Incorporating the feedback gained in the remote co-design session, we developed a Wizard of Oz prototype of our design solution using Adobe XD. We then used the prototype to create a video-enactment to showcase the problem scenario and to show how the various features of the proposed prototype solution would help to solve the problem. This video was then shared with two of the original interview participants for discussion and evaluation.

2.2 Participants

In order to qualify for the study, participants had to have some form of dementia and use technology on a regular basis. Four individuals with mild to moderate dementia participated in this study (age range 57–67, average age 61.5; three identified as female and one as male; all identified as Caucasian). The age range of participants indicates that they likely have younger onset dementia [25]. Table 1 has additional details on demographics, including: age, gender, type of dementia, stage of dementia (which was self-reported), and the specific phases of the study each participant participated in. Jessica, Amelia, and Rose all resided in the US and Joe in the UK.

Table 1. Demographic information
Full size table

In regards to the phase of the study each participant participated in, two of the four initial participants were recruited to participate in remote co-design sessions, Rose and Joe. These participants were chosen after scoping the project to location based reminder systems because of the relevance of this scope to their current reminder systems. Unfortunately, due to medical complications, Rose was not able to complete the co-design session. Therefore, only Joe completed the remote co-design session, but Rose and Joe both completed the evaluation phase of the video-enactment of the wizard-of-oz prototype.

Given the small number of participants and the specifics of their demographics, the goal of this paper is not to advance our understanding of reminder systems for dementia: rather, it is to understand some of the benefits and challenges that arise when working towards remote approaches to engage people with dementia in design.

3 Findings

Below, we describe the benefits and challenges in taking a remote approach to design.

3.1 Benefits

Remotely conducting sessions was helpful for enabling us to meet our recruitment criteria, as we could include participants who were not geographically located near us. We were able to be selective with recruitment by only including participants who were avid technology users.

A remote approach also helped with the successful facilitation of the study. Participants were able to talk to us from a setting in which they were comfortable with the environment. Additionally, by conducting the study with participants in their own homes, using their own technology, there were no barriers to using the digital tools that we selected for collaboration. In other words, they could access Zoom and Google Docs on their preferred devices rather than needing to use a device that we supply and might be less familiar. For example, Amelia strictly uses Apple products due to the interconnectivity between devices (e.g. iPhone, iPad, Mac laptops) and the “unlimited support” she receives from “go[ing] to the Mac store down the street.” If we had asked her to use Windows products for a co-design session this could have been a barrier for participation as she is not used to this system.

A remote approach that allowed individuals to be in their own homes was most essential when it came to asking individuals to talk about their current technology workflows. Because they were situated in their home environment, participants did not have to rely on their memory to recall the various technical and non-technical aspects of their scheduling systems, but instead could just look around, recognize, and discuss them with us. For example, when a researcher asked Joe what other systems he used, he first looked around the room, as he was completing the interview remotely from his study, and then responded: “I have on the wall in the study. A white board that is divided up into days of the week and weeks of the month and there is a cork board for pinning live letters and things like that things that are waiting to be concluded.” Had Joe not been in his home environment he may not have remembered these non-technical strategies for reminders.

While several of the above benefits could also be derived from conducting co-design in an individuals’ home, if we had been engaging multiple people in design at once as we had originally intended, a remote approach would be the only way to facilitate participants in taking advantage of their facility with their home environments. Further, our remote approach is able to include multiple people with dementia in a single co-design workshop with their technical accessibility configurations in their home environment, such as Joe’s use of a mounted tv attached to his computer to magnify his screen to his needed level. This level of accessibility configuration for several people in a group in-person co-design session would be costly and difficult for researchers to recreate in a lab setting.

3.2 Challenges

Although there were positive aspects of remote interviews and observation sessions, which went smoothly overall, there were several downsides of this method. While participants were able to access their own devices and setups in their homes, it was difficult for us to observe some kinds of technology use. In particular, it was difficult to see how people used their phone applications for scheduling because they could not use the applications and simultaneously show us their screen of what they were doing with that application. They had to keep flipping the phone towards the computer camera to show us their use, which was not ideal.

A second challenge arose when, in sharing their screen to demonstrate their scheduling systems, participants unknowingly shared personal information of both their own and their friends. For instance, Joe keeps a record of interactions he has with other people, details about their lives and families, as well as when he was going to meet them again, all in a spreadsheet database to act as a memory aid. Although this was a very interesting and thought provoking example of a personal scheduling system, we were unable to keep or analyze the video-recording of this interaction to ensure the privacy and anonymity of this data as those other people had not consented for this information to be shared.

A third challenge arose in attempting to engage Joe in the co-design session to comment on our preliminary design ideas. We opened the discussion asking him to feel free to ideate and sketch out his own designs/solutions. Although we were able to capture Joe’s thoughts on each design, he did not sketch anything of his own. When we asked why he didn’t sketch his ideas out, Joe explained “if we’d hit a point where I was trying to put something across and my words were capable of giving you a clear enough picture of what I was seeing in my head, then yeah, maybe being able to draw.” He did however still engage in co-design through critiquing certain aspects of the preliminary designs, as well as elaborating on and suggesting new features for our original designs. At the end of the co-design session we asked Joe to critique the remote co-design method. He described the remote co-design session as: “wonderful” and increasing his “motivation, and inspiration” [Joe]. The largely verbal remote co-design session worked for him because “I’m comfortable with technology. And I can, therefore visualize what it is you’re saying and showing and that it’s comfortable for me.” Though we appreciate that Joe found verbal approaches sufficient, we recognize that this is not an approach that will work for many, and we are iterating on our approach to engage people with dementia more actively in the design process.

A final challenge arose when we had participants evaluate the video walkthrough of our Wizard of Oz prototype which incorporated the critiques and suggestions from Joe during the co-design session. Using the prototype, we produced a video consisting of an enactment of the problem scenario as well as how the proposed solution could mitigate the problem. A video enactment of the prototype was used rather than a click-through prototype due to scheduling conflicts with both Rose and Joe, making asynchronous evaluations the most viable alternative. This video was seen as overly complicated by both participants, “even for me that knows what it is about” [Joe]. In the future, we will follow Joe’s suggestion to “Pick one specific example of a daily use” and then “Show the example journey” from the need for the system, to set up, to one example in operation [Joe].

4 Discussion

In this paper, we present a case study of engaging people with mild to moderate dementia in a remote co-design process. It is important to note that the relative smoothness of our process may have stemmed from our inclusion criteria, which restricted the population to people with dementia who were tech-savvy. This likely made it easier for them to use the videoconferencing system that we selected, but also may have made it more feasible to discuss abstract design ideas (in contrast to past work [5, 6]). Additionally, it’s important to note our participants were in the mild to moderate stages of dementia where they were all able to communicate verbally to express and critique design ideas. Even with the consideration for stage and technological literacy, our findings demonstrate that it is entirely feasible for tech-savvy people with mild to moderate dementia to translate visual representations to technology concepts and critique/improve upon these concepts to tailor them for their own use.

Though past work stresses the importance of providing embodied and sensory-rich ways for people with dementia to engage in design, two instances in our data seem to contrast with these past findings. First, the participant who was a part of the interactive design session preferred to relay their ideas verbally, rather than use the approach we offered to draw. This may be due to the stages of people with dementia participating in this work, as embodied interactions may be more feasible as people advance in dementia. Further research is needed to uncover methods that may better invite people at all stages of dementia to remotely engage in hands-on activity during the design process (for one approach, see [26]). A second issue arose when we attempted to provide a sensory rich depiction of how the prototype system might work, with a video rather than a verbal description. The video was seen as overly complicated and confusing, demonstrating the limitations of video prototypes. We offer considerations to improve this piece below.

The design process involved both synchronous portions and an asynchronous component, when individuals viewed the video that we created. Interestingly, the most negative and critical feedback we received on the proposed solution and the process came during the asynchronous video evaluation. We believe this could be due to the fact that participants were more comfortable sharing their criticisms because they didn’t need to say them directly to the designer. This provides one benefit to asynchronous interaction, which we had previously tried to avoid due to the importance of real time interaction in dementia.

However, additional iterations of this video-based evaluation stage are necessary, as the majority of the criticisms were on the accessibility and understandability of the video itself, rather than the prototype. The video was 4 min long, was very detailed, and was narrated by a non-native English speaker. It is possible that the speed of the video or the lack of subtitles made the video inaccessible and difficult to understand. Potentially this could have been due to the multi-sensory format of the video causing sensory overload [27] as the video included audio descriptions paired with text to be read on the screen of what was being input into the system, as well as visual displays of the system. Additionally, participants expressed they wanted a quick overview of the product rather than us show them every function we designed. However, this may not provide the detailed evaluation that designers need to be sure the product is usable by users with dementia. For this reason, having participants themselves use the click through prototype or conduct a cognitive walkthrough of the high-fidelity prototype themselves rather than watching a video tutorial may be better for evaluation of each of the functions of a prototype—as shown in prior research with users with Aphasia [8,9,10]. In a remote session this could be done by participants taking remote control of the researchers computer to evaluate a click-through prototype.

In this paper we have presented a case-study of the use of remote methods for a co-design process, including interviews, a low-fidelity prototyping session, and asynchronous video evaluation of the resulting prototype with tech-savvy individuals with mild to moderate dementia. The main contribution of this paper includes a discussion of the benefits and challenges of using this remote method. Our work indicates that remote methods may be a feasible approach to designing with some people with mild to moderate dementia, though more work is needed to understand how to remotely test working prototypes, facilitate hands-on activities during the design process, and broaden participation beyond tech-savvy individuals with dementia. With consideration for the next generation of people with dementia who will be familiar with remote working tools, our findings provide a glimpse into the future feasibility of remote co-design methods with people with mild to moderate dementia.