Keywords

1 Introduction and Aim

Assistive robots are an upcoming approach to support the independence, safety and social connectedness of older persons [1].

This paper describes mobile video communication as it was developed in the KSERA Project [2–5]. A LED projector module, which is carried on the back of a NAO robot [6], is used as output device for visual content. In the following, a short overview on the development of the module is provided, followed by details of the implementation and preliminary evaluation of the mobile video-communication prototype.

2 User Requirements

The project deliverable D1.1 Scenarios, Use cases and Requirements [7] describes the user-requirements towards the system with means of scenarios and specific use cases. Regarding mobile video communication and the projector module which is carried on the robot’s back, the following requirements are the most important ones:

  • Solutions for users who feel isolated and miss social contact, support of ‘social connectedness’, contact to family members and friends;

  • Supporting communication with medical authorities and professionals, alarm calls in case of emergency (e.g. falls);

  • Motivation to do physical exercises;

  • Mobile interface which approaches the user autonomously in the living environment.

These points show the necessity of an audio and video connection between primary user (old persons and persons with COPD in their own home) and secondary users (e.g. health care professionals, family members and friends).

3 Mobile Projector

The humanoid robot NAO [6] is used as an interface between the users and the rest of the KSERA system [3]. The robot is capable of moving independently towards the users wherever they are in their home.

In the KSERA project the robot is additionally integrated into an assistive living environment. Monitoring the activity and the health status of the user is possible as well as measuring and analysing environmental parameters (e.g. air quality outside the home) [8]. There is also an interface provided to eHome, an intelligent assistive living environment system [9, 10]. This interface was used during the tests described below, to adjust the lighting in the test-room in order to conform to the needs of the projection.

Since the used robot is rather small compared to a human being (58 cm in height) it is not able to transport devices for human–machine-interaction such as touch-screens or tablet-PCs as were used in similar projects [11–16]. The NAO robot is also not strong and large enough to hand a smart-phone or PDA to a sitting or standing person.

The robot has built-in cameras which can be used for a uni-directional video connection (cf. work of Bäck et al. [17]). To our knowledge there is up to now no satisfactory solution for bi-directional video communication using such a small humanoid robot. The innovation presented in this paper is a newly developed LED-projector module which is carried by the robot on its back [18] (see Fig. 1).

Fig. 1
figure 1

Prototype of the KSERA LED-projector module (left); projector module mounted on back of NAO (right) [19]

Full size image

The path of rays of the LED-projector that is used in the module (Fig. 2, light grey) is deflected by a small mirror due to space constraints in the module housing. Therefore, the projected image is a mirrored image of the original visual content. Dedicated software takes care of pre-processing the video information in order to project a correct image.

Fig. 2
figure 2

Projector-module (with vertically mounted integrated LED pico-projector) and path of rays of the projection

Full size image

The most important parameters of the developed module (Fig. 1) are shown in Table 1 (more details can be found in [19]). Based on the user requirements the LED projector module is used to present the following information:

Table 1 Updated specification of LED-projector module [19]
Full size table

  • Videos, e.g. physical training;

  • Text and pictures, e.g. weather forecast, appointments, air quality;

  • Video communication with friends, relatives, emergency centres.

The information that is to be projected and the information flow from and to the projector module is controlled by the intelligent KSERA Server [3] via an appropriate interface [19]. The projector component is responsible for visualisation of the information.

4 Mobile Video Communication

The developed research prototype allows for video communication between the KSERA user at home and a service centre, other private homes, mobile phones and landlines and thus fulfils the need for remote support from physicians or care persons and offers extended possibilities to foster social contacts (friends, family members). The open SIP standard for internet telephony is used. An external SIP server provides the functionality to connect to the public telephone network (Fig. 3).

Fig. 3
figure 3

Simplified architecture of the mobile video communication in the KSERA project

Full size image

The LED-projector module prototype was tested with experts and end users in laboratory settings [20] before it was provided to the project partners for extended tests under real life conditions. Figures 4, 5 and 6 give an impression how the LED-projector module was used in a controlled setting in the laboratory.

Fig. 4
figure 4

Set up of test-room in laboratory (see also Fig. 5). Couch (for user), chair (for supervisor of test), NAO facing test-person (green) and direction of projection (orange). Table with touchscreen for stationary video communication

Full size image
Fig. 5
figure 5

Mobile Video communication in laboratory setting: NAO carries LED projector module on its back and uses it together with its integrated head-camera for video communication between user (sitting) and service centre

Full size image
Fig. 6
figure 6

NAO robot projects image of service center on a wall next to the user

Full size image

The robot is steered by the KSERA system to a place in the apartment which is known to the system and offers enough free space for projection. After the robot has been aligned by the system (distance to projection area 1 m ± 10 cm, perpendicular to it ±5°) projection of relevant content can be started.

5 Operation Modes

The prototype allows for different modes of operation [21]. These modes are chosen according the specific needs of the users:

  • Visualisation of text, graphics and videos: pre-recorded videos for physical training (callisthenics etc.), complex textual messages (e.g. personal reminder, day’s schedule, air quality) are projected onto a wall next to the user. Tests in laboratory setting have proven that projected information provides additional benefit compared to speech output [20], especially if the content is complex.

  • Bi-directional video communication: The projector module and the integrated camera in the robot’s head serve as output and input for a video telephone (see Figs. 4, 5). Added value exists in particular in emergency situations—e.g. if the user has fallen. In this case the video-phone can walk up to the user on its own.

In order to be able to compare to the ‘state of the art’ and to demonstrate more applications of the KSERA system, an additional communication module without a projector was used during the laboratory trials:

  • Stationary video communication: Only the stationary touchscreen-terminal (without NAO) is used.

  • Mixed mode video-communication: (a) a stationary touchscreen-terminal is used for communication and (b) the NAO robot is placed next to it to act as a motivator whose task it is to motivate the user to do some video calls (Fig. 7).

    Fig. 7
    figure 7

    Stationary touchscreen terminal with NAO robot as ‘motivator’

    Full size image

  • Uni-directional video communication: This is similar to the bi-directional video communication. The head camera of the robot is used to pick up the image of the user and transmit it to a service centre, however the beamer module is not available, and thus the user cannot see the operator of the service centre (cf. work of Bäck et al. [17]).

6 Workshop with Experts

In addition to the trials in a laboratory setting with old persons and experts [20] a workshop was done with experts from the care domain [21]. The aim of this workshop was to evaluate the quality, user friendliness and added value of the mobile video communication of the KSERA system.

The workshop was held in a laboratory at Vienna University of Technology. Three high ranking experts from mobile care, occupational therapy and daily care of chronically ill persons took part. After a short introduction of the state of the art of assistive robots and the overall goals of the KSERA project, the possibilities of video communications were demonstrated. The experts gave their input regarding the system and the video-communication in particular in a free round of discussion. During this discussion also ideas about usage of the system or enhancing possibilities were presented.

The experts rated the audio and video quality of the connection as sufficient for the presented emergency scenario and for general video communication. This coincides with the results of the laboratory tests [20] where also end user evaluated the image quality as sufficient. The possibility of getting a better impression of the emotional status of a user in case of an emergency compared to an audio only connection was rated as particularly useful.

Additionally it is possible for an operator in a service centre to get a better idea of the severity of an emergency situation even if the user is no longer able to communicate. Detection of an emergency and establishing the connection to the service centre was not part of the workshop and was considered fully functioning.

Applications which were rated very useful are reminders to do physical exercise, calendar function for the daily schedule, as well as the capability of the system to motivate to social contacts. Another application which was rated very useful was to motivate users with film and/or movement of the robot to perform regular physical exercises or to guide them through such exercises.

7 Discussion

The developed LED projector module and the mobile video communication functionality that could be realized with this module, result in enhanced abilities of the NAO robot regarding human robot interaction (HRI) and support that can be provided to primary users.

At the beginning of the project, focus was on development of the module and its contribution to satisfy the user needs motivate for physical exercise and mobile interface which is able to meet the user wherever they actually are [18].

Based on this first prototype the extended second prototype also contributes to the user needs for social contact with friends and family members and for simple possibilities for emergency calls or for communication with professionals or health care organisations [21].

One obvious limitation of the projector module is the low brightness of the projected image. The luminous flux of the used pico-projector is 30 lm and thus much lower than the one of a standard projector (e.g. 2,500 lm). For this reason pico-projectors can only be used effectively inside a building, without direct sunlight on the projection surface and with an overall low level of ambient lighting. Since the KSERA system offers also an interface to eHome, an assistive environmental control system [9, 10], this was used to lower room lighting level during the time of projection. It can be assumed that future projectors with higher luminous flux will overcome this limitation. It will be possible to project overlay information on real objects and therefore enhance the ‘reality’ of the user [22]. This augmented reality potential could also be useful for the KSERA target users.

8 Conclusion

A prototype of a mobile projector module which is carried by a small humanoid robot was built and tested. Within the KSERA system this module makes it possible to project information onto a wall while the robot is communicating with the user of the KSERA system. The persistence of the visual information (compared to speech output) could be evaluated by end users and experts as being useful and relevant for the AAL context. The second prototype successfully integrated video communication to enable the users to get in contact with friends, family members, informal carers or a service centre. As video input for this the head camera of the robot us used, output is provided by the developed projector module on NAO’s back.

Tests and workshops with users and experts from the care domain confirmed the added value of the mobile solution for video communication as provided by the NAO robot in the KSERA system.

Three LED projector modules were built and provided to the project partners for further tests of the whole KSERA system in a near to daily life setting.