Keywords

1 Introduction

Currently, robots of different style exist. They are heavily involved in domains like industrial production [12] or healthcare [1]. The reader might know many other application domains. Therefore, we mention only two of them here.

For production lines, robots are very functional. They are constructed or a specific purpose and might have arms like a crane only. They do not look like humans and have no head or eyes. They behave and look like machines. Humans do not have empathy with them. There is no desire to communicate with such robots. However, sometimes robots look like animals play the role of pats. Communication and interaction is especially important for elderly people with dementia [3, 6]. Robots that look like humans are characterized as humanoid robots.

Within our project E-BRAiN (Evidence-based Robot Assistance in Neurorehabilitation), we want to use a humanoid robot to support patients after a stroke with their training aiming to restore brain function.

Platz and Lotze [8] report about the clinical effectiveness of specific exercises. We will provide an overview of the exercises and discuss the digitalization of one of them.

2 Arm Ability Training (AAT)

The Arm Ability Training was designed to promote manual dexterity recovery for stroke patients who have mild to moderate arm paresis [7]. Platz and Lotze report in [8] about its design, clinical effectiveness, and the neurobiology of the actions. The idea of the AAT goes back to the identification of sensor motoric deficits of stroke survivors in [9] and [10]. Figure 1 provides an overview of the suggested training activities.

Fig. 1.
figure 1

(from [8]).

Training tasks of AAT

Full size image

The clinical effectiveness of the arm ability training was discussed in [8] on the basis of two single-blind randomized controlled studies (RCT) that included 125 patients who were eligible for the AAT. In one of these two RCT the AAT had superior effectiveness compared to therapeutic time-equivalent “best conventional” therapy.

However, the training is resource intensive because one therapist is necessary to observe and support the exercises of a patient. The idea arose to use a robot for assisting the patients. The robot is intended to lead a stroke survivor through the training and to act as a motivator. It was analyzed which existing kind of robot fits to the requirements of the arm ability training. As a result, the humanoid robot Pepper was selected for first experiments.

3 The Humanoid Robot Pepper

Pepper is a humanoid robot from the company SoftBank Robotics [11]. It is already used in shopping centers, railway stations or airports to give support to customers by providing information (Fig. 2).

Fig. 2.
figure 2

Humanoid robot pepper from softbanks robotics [11].

Full size image

Pepper has a very nice facial expression, can talk and move around. The robot can also move its head, blink with eyes and ears. Additionally, Pepper can move its arms and fingers.

We assume that a socially interactive robot can be helpful for stroke survivors when performing their individual training and hence for their recreation. However, the feasibility to use a humanoid robot for neurorehabilitation has yet to be analyzed within an appropriate research setting. Our E-BRAiN project has the objective to implement neurorehabilitation training in a digital form using humanoid robot technology.

4 Challenges in Interaction Design

The design of the interaction of the robot and the patient will likely be a relevant success factor of the application. Among the many questions to be addressed are the following:

  • What kind and how much information (e.g. instructions, feedback) has to be provided in which way?

  • When should the robot take corrective actions?

  • How can the interaction design be modelled in an appropriate way?

Currently, we assume that the arm ability training is introduced by a trained therapist to the patient. The therapist explains all tasks and observes the first executions by the patient. Later, short introductions and the supervision of training sessions are planned to be provided by Pepper. On the tablet upon his chest videos of the task execution can be provided, verbal instructions can be provided via loudspeakers. They can be personalized by clinical assessment and the experience made during a first training session with the “human coach”.

The robot can motivate patients during their exercises with general supporting comments. However, it would be preferable if such comments could be provided related to the shown performance, as feedback. Pepper has a lot of sensors. Nevertheless, it is sometimes difficult to analyze manual tasks performed using paper and pencil. Therefore, a digitalization of training tasks could help a lot to analyze performance and provide feedback. We will demonstrate this by an example of the arm ability training. The respective task is called “hitting targets”. A patient has to hit circular targets of different size from left to right and afterwards from right to left (see left part of Fig. 3.). Before the next circle can be hit a circle on the table surface (“home position”) has to be hit. This can be seen on the right part of Fig. 3. During a pre-specified timespan (1 min) a patient has to hit as many targets as possible. The therapist interrupts the task execution if a target is missed. The patient has to try again to hit that target.

Fig. 3.
figure 3

Manual and digitalized training task.

Full size image

The overall task “hitting targets” (1-min intervals) is repeated four times within a training session. The goal of hitting more targets within the given time span can only be reached when the individual level of performance (i.e. a combination of speed and accuracy) is improved by training, e.g. when speed can be increased without losing accuracy. Patients have to train the tasks at their performance limit and by repetition they will eventually improve their performance in an incremental way (motor learning). In that way they reduce their performance deficits caused by stroke and regain dexterity in everyday life.

It was already mentioned that performance of a manual task is difficult to observe by a humanoid robot. Therefore, we implemented an application using two tablets (see Fig. 3 right hand side).

To implement the manual procedure straight away would trigger a comment from Pepper when a target was not hit in a correct way. However, we felt that it would be better if the application on the tablet itself provides an appropriate feedback. In this way the robot could not be perceived as a kind of opponent and the feedback could more directly be linked to task execution and promote movement correction.

We considered the following types of feedback for the task “hitting targets” on the tablet:

  • The target that has to be hit next in the sequence is blinking

  • A properly hit target becomes green until the next target is properly hit

  • A specific acoustic sound is provided for correct and incorrect attempts to hit

In addition, the suggested forms of feedback could be used as redundant information. It will be evaluated in the future which kind of patients would like to have which feedback and when redundant feedback could be clinically warranted. Maybe, some patients might prefer and/or might benefit from the redundant feedback while others might even feel distracted by an “overload” of visuo-acoustic information when concentrating on their motor performance. As an example, for the former, stroke survivors with visuospatial attentional deficits performing the training might more adequately be supported with multimodal and redundant sensory information, because they have difficulties to orient their attention and hence their movement in visual space.

We are also yet not able to answer the question: “When should the robot take corrective actions?” Experiments with different versions of our application will show which solution is appreciated by patients and which solution does not assist, but puts more pressure on the patients.

5 Summary and Outlook

The paper introduced the arm ability training (AAT) for patients after a stroke. More details can be found in the provided references. Some of these references demonstrate the clinical effectiveness of the training. In addition, it was discussed whether and how a social humanoid robot like Pepper can assist patients during the AAT. Such a robot could play the role of a coach, motivator and supporter. While a humanoid robot cannot and is not intended to replace a human therapist, it might be suitable to support a training situation that might otherwise only be possible in an unsupported patient-led way.

For such a scenario, the AAT itself would need a kind of digitalization. For one training task it was shown how training can be supported based on two tablets and a corresponding application. Solutions for other training tasks will be developed as well.

Because of lack of space, we did not discuss the aspect of modelling of collaborative activities of patients and robots. Some general ideas about the specification of collaborative activities can be found in [2] and [5]. Robot-specific models are discussed in [4]. A comprehensive domain-specific language (DSL) for robot actions will be a challenge for the future.

Evaluation of the success of the digitalized AAT in comparison to conventional AAT with a group of patients are currently planned for the near future.