Keywords

1 Motivation

Due to the demographic transition and societal changes, 5.8 million adults over the age of 65 were living alone in Germany in 2019 [7]. Many of these older adults want to continue living in their own homes while retaining their autonomy. While digital technologies play an increasingly prominent role in our lives and can support our autonomy, many older adults struggle to use them [11]. These technologies include smartphones and computers, but also assistive devices like smart home speakers or social robots. An approach to increasing the digital competence of older adults is the participatory development of such technologies [16]. This leads not only to user-centered products and developments but also to users who are more adept at using these developed technologies. At the same time, for such user-centered development to be successful, developers need to be sensitized to their users’ needs - which can be especially challenging when the user group is from an entirely different demographic.

Tackling these challenges is the goal of the project GeneRobot - in which engineering students develop applications for a social robot with and for older adults in assisted living. In the project, older adults can contribute to the development of apps that are designed specifically for them while becoming more comfortable with their use. At the same time, students come into direct contact with their users to ensure the successful development of user-centered apps. The developed apps are deployed into the assisted living communities in which the participating older adults live for continuous optimization and expansion. Overall, this also serves to investigate how older adults want to lead their daily lives and which roles social robots can and may play in them. In the following, we present our project, including related research, the different participant groups, as well as the first learnings and findings in the teaching and research domains.

2 Related Work

The design and development of assistive technologies for health and well-being has been an innovative field of research for years, one of the promising sub-areas being social robotics [1, 22]. As opposed to virtual agents, the physical presence of robots has been shown to have positive impacts on user interaction [10]. This applies in particular to providing social company and combating loneliness [15].

Due to innovations in automation and sensor technology, socially assistive robots (SARs) are gradually being introduced into real-world scenarios like nursing homes or private households of older adults [4, 9]. One of these robotic systems is Pepper (Aldebaran, France), described in [17]. The potential of Pepper as an SAR for older adults is already being tested in projects within and outside the academic community [2, 20]. Since the robot’s design includes social cues like gaze and gestures, Pepper is an ideal robot to develop and test the benefits of robotic social interactions. Current use cases include cognitive training through tasks, gymnastic exercises or games [20, 21]. While several research projects present results of testing Pepper applications with older adults [6], few point out the process of how the tested behaviors or applications were chosen.

The age of the target group has to be considered when designing the behavior of a social robot [13]. One way to include the needs of older adults in product development is the participatory design method [14]. Some studies use user-centered design (UCD) methods for SARs, but most focus on the ideation or conceptual phase [5, 19]. One study that displays an extensive participatory design process is the development and evaluation of the social and assistive robot HOBBIT [9]. However, since the robot was initially mainly designed for fall prevention, the study was less focused on co-creating social interactions. Using the methods of UCD holds great untapped potential for developing SAR applications and behaviors.

Furthermore, most studies only involve researchers, users, and sometimes other stakeholders in the co-creation process when developing robotic applications for older adults. This presents a missed chance for application-oriented teaching formats, especially when considering the benefits of intergenerational exchange [3]. There are successful university projects that bring together students and older adults to develop products collaboratively, some of which even deal with designing SARs [18]. But since these classes are held for design students, the joint development of robotic solutions mostly stops after the prototyping. Including engineering students in the co-creation process of SARs can expand the learning experience of the whole UCD process. Therefore, our approach of bringing generations together via participatory design aims to not only result in user-centered robotic applications but also to offer a meaningful contribution to the education of young engineers. Complementing this approach is the interdisciplinary research team, which acts as an interface between the older adults and students while integrating both groups into the participatory design process and its resulting learnings.

Fig. 1.
figure 1

The project participants are shown interacting with Pepper in one of their assisted living communities. The applications in use are described in Sect. 4.

Full size image

3 Methods

GeneRobot is a cooperative project between the Cologne Cobots Lab at the TH Köln - University of Applied Sciences and the Diakonie Michaelshoven, a provider of adult day care and assisted living facilities. One of these facilities are shared apartment assisted living communities in the Cologne area. The four communities include 24 one-bedroom apartments in total, each community with a shared common room. We enquired which residents would like to participate in the project after an event during which they had the opportunity to get to know and interact with the social robot Pepper [17], which we chose as the focus of the project is applications for social interactions. Six residents, subsequently referred to as participants, volunteered to take part in our project (see Fig. 1).

The six participants have been diagnosed with varying health impairments. However, we specifically looked for participants for the study who neither have a diagnosis of any type of dementia nor mild cognitive impairment. As the participants’ ages range from 55 to 95 and their impairments differ widely, their expectations and wishes for the robot also vary. To investigate this, the project is accompanied by continuous qualitative research assessments carried out by the interdisciplinary research team with backgrounds in engineering, social sciences, and gerontology. The applications for Pepper are developed by students in the TH Köln’s Mechanical Engineering Master’s program. Each semester, ten to 20 students in groups of four to five are involved in the project.

3.1 Technical Infrastructure and Data Privacy

Pepper is designed as a social companion and communicates mainly via natural language. Applications for the robot are programmed as Android apps. To design the verbal interaction, the rule-based dialogue system QiChat was used, which is included in the robot’s framework. Additionally, everything Pepper says is displayed as text on its tablet, often also supported by additional images, to support the speech output.

Privacy is also an important component of this research project, as the robot lives with the participants and thus invades the privacy of their homes. Informed consent was ensured and obtained by providing information about what happens to the analyzed data via a detailed exposition of the project-related data protection guidelines according to current regulations, such as the General Data Protection Regulation (GDPR). The participants were informed through workshops in which they could practice how to use and interact with Pepper.

3.2 Participatory Development

Every semester, new apps or functionalities for Pepper are developed by groups of students and the participating residents in a co-creation process, derived from [8], and shown in Fig. 2. For the course “Developing anthropomorphic machines" (DAM) in the summer semester, each group is tasked to develop an application, subsequently referred to as project apps. Following the UCD method [12], the students conduct semi-structured interviews with the residents to understand the context of use. Based on interview recordings and their notes, the students specify the user requirements. After brainstorming possible ideas, each group chooses one solution they proceed to design and develop. Due to time limitations and Covid-19 restrictions, the ideas and designs are not evaluated by the users before the project apps are developed. However, evaluation and feedback are given by the research team so that the students have the possibility to rework their individual apps. General observations made by the project team concerning the usability of the project apps serve as impulses for the course “Human-machine interaction" (HMI) which is held during the winter semester. In this course, the groups of students are given predefined assignments that are tailored to the needs of the specific user group. The developed functions resulting from these projects are subsequently referred to as conceptual solutions. The project apps and conceptual solutions are combined and optimized by the research team (step 2). Finally, the optimized versions are deployed in the assisted living facility, where they are tested by the older adults for whom they were developed (step 3). Periodic participant observations by the research team serve as the basis to evaluate and further optimize the developed project apps and conceptual solutions against the user requirements. All findings are also being used to further develop the teaching methods in the following semesters.

Fig. 2.
figure 2

The participatory development process includes three steps, while the findings from steps 2 and 3 are integrated into courses for following semesters.

Full size image

4 Results and Learnings for the Future

In the following, we show the results of the students’ work and the optimizations made by the project team. We address the challenges of participatory technology development and our findings on the integration of the project into teaching.

4.1 Step 1: Participatory Design and Development

In the context of the DAM module, most of the wishes expressed by the participants were related to entertainment or support in everyday life, such as fitness exercises or recipes. Furthermore, one participant, having lost the partial ability of speech due to a medical condition, wished to use the robot for logopedic exercises as an addition to regular therapy. A total of three project apps were created within the first course: A speech training with logopedic exercises and a quiz, a fitness application, and an organizer app, which includes weather and an alarm clock function, notes, and recipes (see Fig. 4). Further features resulted from conceptual solutions students developed in the course HMI. The assignments consisted of optimizing an existing app with respect to users with either hearing or vision impairments, as shown in Fig. 3.

Fig. 3.
figure 3

Features for vision-impaired users (left) include a user interface in green with higher contrast and larger fonts; features for hearing-impaired users (right) include a slider to adjust the volume. Screenshots were translated into English. (Color figure online)

Full size image

In the context of vision-impaired users, the students focused on implementing verbal menu navigation in addition to the existing buttons, promoting learnability by means of an interactive self-description of the system and restructuring the user interface (UI). The students researched the influence of colors and contrast on the perceptibility of written content and pre-selected several color combinations. These were subsequently presented in an online survey where participants rated their perceptibility. Based on the results, the students created a new color scheme. For better readability of text displayed on the tablet, the font Atkinson Hyperlegible (Braille Institute of America, USA), which is recommended for inclusive app and website development, was used. Also, an option was implemented that allows users to incrementally change the font size on the tablet via a button or voice command. Regarding features for hearing-impaired users, the students created an option to change the system volume within the app using a slider. Additionally, to help users understand the robot’s behavior and ensure safety in human-robot interaction, the students implemented visual warnings about excessive robot movements, e.g., before the robot dances.

4.2 Step 2: Combination and Optimization

Within the first iteration, based on the evaluation by the project team, fundamental changes were made to the three project apps, as shown in Fig. 4. Since each of the apps was based on the ideas of one participant, partial functions and activities differing thematically from one another were grouped together. We developed a new structure, which takes the thematic content of each function into account and incorporates them into new apps accordingly:

  • Fun and games: quiz (sights or flags) and fun facts about various countries

  • Kitchen companion: recipes and facts about the cuisines of different countries

  • Planer: notes, alarm clock, and weather function

  • Fitness: various exercises and workouts for older adults

  • Speech training: logopedic exercises on words, sentences, letters and numbers.

Fig. 4.
figure 4

The old and new designs of exemplary applications and features are shown: the weather feature and the recipes feature. A clearer and more consistent design with larger buttons and fonts was chosen.

Full size image

The design of the project apps, as received from the students, differed widely. Targeting good learnability and ease of use of the complete system, we created and implemented uniform design guidelines, specifying, e.g., colors, contrasts, font type and sizes to be used, and the size and design of clickable items. The UIs designed following the guidelines maintain the corporate identity (CI) of TH Köln. In most apps, the students used synchronous chats, thus preventing further user input from being processed while the robot is still providing verbal output. Throughout all project apps, we replaced these synchronous calls with asynchronous ones, thus adhering to the principle of increased user control over the system. When reviewing the dialogues, we removed or revised inappropriate content where necessary and shortened lengthy robot utterings in favor of a better user experience.

4.3 Step 3: Tests and Evaluations

The results of the first participatory observations were collected and analyzed by the research team. The two most prominent issues were the navigation from the single project apps to the menu overview and minor bugs that impact the user experience. In the first case, the default wiping motion was replaced by classical x-buttons to close the app. Additionally, the first bugs were fixed and the user experience is continuously optimized to get users more involved in the interaction. Minor usability issues as well as the results from future observations and usability tests will be used to continuously optimize the apps.

4.4 Learnings and Benefits for Older Adults, Students, Research Team

One of the challenges of integrating participatory design into teaching is the limited time frame. Due to the university schedule, all projects lasted three months and needed to be graded at the end of the semester. For some students, this was their first practical experience in app development. Since the goal was to develop a working application, there was little time for design iterations or prototypes. This also affected the contact time between students and users. The Covid-19 pandemic further aggravated this circumstance since the user group belonged to a vulnerable demographic. Since the TH Köln switched to online teaching, the students also had few possibilities to experience the robot they developed their apps for face-to-face. The simulation software the students used at home allowed them to test their apps and Pepper’s movements but lacked speech input or output. This was a challenge for designing natural dialogues.

The overall learning from the first two semesters of the project was that when integrating co-creation methods into university classes, the research team plays an important role at the intersection between users and students. In the following semesters, more guidance and structure were introduced to compensate for the limited time frame. To help with the technical challenges of app development, more coding tutorials were introduced into the course schedule. We now also provide the code for a demo app so that students have a framework they can use and modify. To achieve a coherent look of the final product, the students are provided with a style guide that helps with questions of UI design concerning font, size, color, and spacing. Since the experience has shown that testing prototypes or apps with end-users during the semester presents a challenge, expert analysis will continue to play a role. For that purpose, standardized evaluation methods like heuristic evaluation are being introduced into the project.

To compensate for the limited contact time with the robot and to make first improvements and bug fixes, the students are shown recordings of Pepper’s demo application as part of the programming lessons. This helps to give the students a better understanding of the timing and intonation in their dialogues and helps create more natural dialogues. The students then have the opportunity to test their apps in person for final adjustments. Furthermore, the importance of the original user interviews is stressed by letting the students transcribe the dialogues.

We also asked both the participants and the students for feedback. The participants stated that they were curious to see what benefit the robot can really provide in their daily lives, as it had not yet moved into the shared apartment community when we enquired for feedback. The students said they learned a lot, specifically about UCD and programming. They were also surprised that the participants they interviewed offered different ideas about what Pepper should do than they initially expected. This shows that it is essential that developments for older adults are developed together with them, as the projection of user requirements often does not match the actual expectations of the target users.

The next step in the project is to investigate how we can ensure that Pepper has a long-term benefit as a new roommate for the participants after having permanently moved in a few weeks ago. This includes continuous participatory developments and optimizations to make sure that all apps work as intended for the participants they were designed for while developing new apps and solutions based on the ideas they come up with. Additionally, the project will continue to explore different methodical approaches to co-creation, focusing on developing real-world applications through a complete user-centered design process.