Keywords

1 Introduction

The increasingly exponential growth of ageing population and the lack of resources to manage their needs has stimulated the creation of new frameworks and solutions, where the Information and Communication Technologies (ICT) have an important role in them. The pandemic of covid-19 has also served to reinforce the fact that the ICT can help to avoid isolation, permitting social interaction when the mobility of population is reduced or restrictions by lockdown are applied. But ICT are also promoted in ageing population for helping them in healthcare needs, e.g. to combat cognitive ageing. In this regard, immersive technologies like Augmented Reality (AR), Virtual Reality (VR) or Mixed Reality (MR) offer older people the opportunity to be stimulated, and to live, in a virtual reality environment, simulating activities or physical exercises that they used to do or cannot do now due to illness or physical/mental impairments.

VR, MR and AR are immersive media technologies that provide new scenarios and applications to education, healthcare, business and society, enabling new ways of learning, interacting, communicating, enjoying and working. Although numerous applications have been or are being developed for the immersive media AR, MR and VR, uniform technical and legal standards and the methods for workflows to develop content and products are still missing for mass market maturity. Distribution channels through which larger user groups can use these media are just emerging as well. In short, AR, MR and VR are innovative technologies that are still in an early stage of development [1] (Fig. 1).

Fig. 1.
figure 1

Differentiation of VR, AR and MR [2]

Full size image

One of the biggest differences between AR/MR and VR (at the current time) is that AR or MR is more of an individual experience, whereas in VR enables an easier communication and collaboration with other avatars, like in multiplayer video games. However, according to recent reports [3,4,5], AR will also soon move towards collaboration. Nevertheless, AR/MR and VR should not be seen as competing products, as both have individual potential uses with advantages and disadvantages.

In this work, the AR, VR and MR technologies are introduced in depth, identifying the state of art, research, innovations and challenges of these for ageing population. The research is focused on the software/hardware solutions existing that can be applied in an extended Ambient Assisted Living (AAL) where intelligent furniture and living habitat of the ageing population is expected. The research also includes an experimental study carried out with AR and VR, to show the benefits of using these technologies in ageing population.

This contribution is organized as follows: Sect. 1 introduces the work. Section 2 defines VR, AR and MR technologies, reviews the most popular devices and applications that are found in the state of art and market. Section 3 explores the development and application tools of AR, VR and MR that could be useful for ageing population. Section 4 explains the experimental study carried out to make a first assessment of the suitability of using AR and VR technologies for stimulation in elderly people. Finally, Sect. 5 presents the conclusions and future work.

2 AR, VR and MR: Definition, Devices and Development Tools

2.1 Virtual Reality

VR, from a technology-centered perspective, is a set of computer systems that create immersive and interactive environments through appropriate hardware such as stereo displays. VR is also defined as a methodology for giving users the experience of inclusion in an illusory reality. The goal is not necessarily to achieve a perfect VR in which virtuality and reality can no longer be distinguished. Peculiarities of human perception such as the “Suspension of Disbelief” can be used to create effective virtual environments for people and to give them the feeling of being present in VR. This can serve different purposes: research (e.g. human perception), education, entertainment, supporting communication, visualising simulation results or economic goals (e.g. prototyping to increase efficiency or save costs). The goal of VR is mainly focused on creating innovative interfaces between humans and computers [6].

VR experiences can have different levels of immersion, understood as the capacity of a system to generate an environment that emulates experience of presence in the real world. Depending on the degree of immersion, VR are classified as:

  • Non-immersive: They show the virtual world through a combination of three-dimensional images on screen, sounds and a high degree of interaction with the simulated virtual world. The hardware used is low cost and easy to install. A clear example is the desktop personal computer and 3D video games. Although they are not immersive systems per se, they are capable of generating a high degree of attraction of the user’s attention, producing strong emotional responses.

  • Semi-immersive: These generally comprise a projection system that displays the virtual environment on the walls and floor of a room. They also have a system that tracks the user’s head movements in order to adjust the simulation accordingly, and in most cases, they incorporate a handheld device to interact with the virtual world. This type of system has a multi-user capability, where several people can enjoy the virtual experience, which makes them very interesting in collaborative work environments. These systems require a large space for their installation and their cost is high.

  • Immersive: They offer the highest degree of immersion, making the user’s perception of the virtual world as close as possible to the human relationship with the real world, blurring the line between the physical world and the digital or simulated world. These systems are generally comprised of a helmet with a stereoscopic vision system that allows the user to visually perceive the three-dimensional virtual world in a way that is identical to how the real world is perceived. The helmet also incorporates position and movement sensors that synchronise the user’s position and perspective with the virtual world in real time, as well as headphones that reproduce a surround sound environment and various input devices, such as joysticks or gloves, that allow interaction with the virtual world. The cost of these systems varies from medium to high, depending on the system chosen and its peripherals.

The most popular devices that enable a VR experience are the VR goggles with headphones. They enable virtual or visual experiences. In virtual experiences, the user does not interact with the real world, and is immersed in a virtual environment with an avatar. The user can manage it with gestures, controllers or a special suit via motion tracking. In visual experiences, the user is immersed in the virtual environment only with visual role, e.g. when playing Minecraft or visiting a virtual world such as Second Life by means of a computer screen [7] (Figs. 2, 3, 4 and 5).

Fig. 2.
figure 2

VR traking [8]

Full size image
Fig. 3.
figure 3

HTC vive tracker [9]

Full size image
Fig. 4.
figure 4

Cybershoes [10]

Full size image
Fig. 5.
figure 5

VR glasses [38]

Full size image

2.2 Augmented Reality

AR is defined as a set of tools that offer a user experience in which the real world that the user perceives is enriched by additional information, usually generated by a computer. Then, AR can be understood as an integration of the virtual world into the real world. The real world can be augmented by one or several senses, which means that visual, sound, tactile and even olfactory AR experiences can be achieved, separately or a combination of them.

AR experiences with a visual component can be classified according to their levels of immersion, with the following degrees or levels:

  • Non-immersive: These monitor the user’s orientation by means of sensors and capture the real world through a camera, which in turn is shown to the user through a screen. It is in this interface where the integration of the real world with the virtual elements takes place, which are oriented according to the data received by the sensor system. Currently, most of mobile devices in the market (smartphones and tablets) are compatible with these developments, and are used as tools for capturing, processing and projecting the augmented experience. This is a low cost solution for enjoying the AR experience.

  • Immersive: These are the same as in VR experiences, but with the use of a set of cameras that capture the real world and transfers it to the helmet’s vision system. The experience reproduced has a strong immersion, by combining its stereoscopic vision system, together with the advanced sensor system and controls for interaction with the virtual elements.

The most popular devices for VR are the smart glasses such as Google Glass or slightly futuristic-looking glasses like Microsoft’s HoloLens, which also make it possible to superimpose virtual elements in the real world [7].

AR can be enjoyed also through smartphones with camera and apps such as Aurasma [11], which superimposes content on the smartphone screen over the images that come from the camera; e.g. info about historical monuments placed in the street, or translating text captured by the camera in real time. One popular AR example is the smartphone game Pokémon GO, in which Pokémons are superimposed on the user’s current surroundings [1] (Fig. 6).

Fig. 6.
figure 6

Aurasma [8]

Full size image

2.3 Mixed Reality

MR belongs to the continuous spectrum of virtuality, integrating the virtual world with the real world but one step further, allowing the physical elements of the user’s environment to be components of interaction with the virtual elements. This allows the creation and modification of virtual objects through data obtained from the real world and a better integration of the virtual elements in the real world, by being able to calculate how they are affected by the real physical environment in which they are incorporated, e.g. to adapt the shadows and reflections to the physical environment, to modify the lighting of the object depending on the luminosity of the scene, to limit their movements to the real environment, etc.

MR systems can be classified analogously to AR systems: non-immersive systems and immersive systems, as AR systems were taken as the basis for further research and development in MR technologies.

The term MR is often used synonymously with AR and associated with data glasses such as the HoloLens. Since both cases involve the expansion of the real-physical environment through the integration and overlapping of virtual objects, the terms MR and AR are used synonymously in this work.

MR is currently in a development phase and requires a boost in research at both software and hardware level, making it necessary to explore improvements in the algorithms for interpreting the real world and perfecting the capabilities of components and systems for immersive technologies.

3 AR and VR Tools for Stimulating and Motivating Ageing People

AR and VR can be used as a driver technology for avoiding isolation, stimulating and motivating ageing population, tools to support active ageing [12].

With AR, it is possible to superimpose a lot of information in text, graphics, video and audio into a real time environment. Elderly people can participate interactively with the environment, exploring and learning details of each significant area of the event site [13]. With AR is also easy to design interactive paper worksheets to be used in workshops for adults and ageing population. The work is done by uploading the triggers (objects such as pictures that are recognized by an AR app when users look at it through the camera). The AR app (e.g. Aurasma) enables to add overlays. When a user scans the worksheet with the AR app, tips, learning videos or additional tasks are plotted [14].

The AR interface can reduce cognitive load and provide correct spatial information, promoting the spatial visualisation ability of older adults [15]. Some AR gaming applications have also proven to be useful for early detection of dementia and cognitive training [16].

At present, we can find some interesting applications and services for creating interactive AR contents for different purposes. Thinking on elderly people, a set of tools are identified as suitable candidates for cognitive and physical stimulation. They are listed in Table 1.

On the other hand, VR allows users to interact with all manner of objects and systems, including those that are too small, too large, or perhaps too dangerous to experience in real life. With VR, elderly people can work in stimulating tasks, such as assemble, disassemble, manipulate and modify objects and environments in ways that have not been possible previously. VR can also help ageing population to expand their knowledge about complex concepts that cannot be explained otherwise.

VR enables to engage the senses, emotions and cognitive functions of the brain, harnessing the most powerful aspects of retention. Now, from manufacturing to customer service, organizations are jumping on the bandwagon, realizing that virtual visits in training can really affect their bottom line. VR, delivered through immersive headsets, offers the opportunity for interventions to improve physical, mental and psychosocial health outcomes in older adults [17].

Nowadays we can find applications that allow health professionals and caregivers to create their own content for VR glasses, focused on ageing population. One of the most popular is the tool provided by Google, which enables virtual field trips with its Google Cardboard and Google Expeditions apps. Others are also coming onto the market with higher-priced glasses such as the Oculus Rift or the HTC Vive [18].

A set of interesting applications have been identified that could be beneficial for stimulating elderly people, even although some of them were primarily designed for educational purposes. They are summarized in Table 1.

Table 1. List of suitable AR/VR tools and apps for being used for stimulating elderly people
Full size table

4 Experimental Study of Immersive Experience of Ageing Population with AR/VR

There are numerous works that present different applications of AR and VR to improve the quality of life of older people, e.g., on how to improve spatial vision [15], medication management [34], early detection of dementia and cognitive training [16], fall prevention [35], etc. But only a few show results of using these technologies already implemented, freely available and easily accessible tools.

In this context, in this work an experimental study was carried out with elderly people to measure if immersive experiences using AR and VR could be beneficial to them. The candidates were 10 people over range of 73–80 years old, without mental/physical impairments or diseases, living alone or with a partner. Six of them are women and four are men.

This study was conducted during the month of April 2021 in Spain, so the global covid-19 pandemic situation prevented a larger number of participants, as the study candidates belong to the at-risk population. In addition, the restrictions and security measures resulting from the pandemic also influenced both the development and the number of participants. It should be noted that all necessary distancing and hygiene measures were complied with and the activities ran smoothly.

The candidates were invited to enjoy two different experiences: to use AR app using smartphones and to use VR glasses in which they watched 360 videos on topics of interest to them and to try out. The VR experience consisted of watching or visiting one of the following list of possibilities: (1) Machu Picchu, (2) The Maldives, (3) Manhattan, (4) Seeing a pride of lions and (5) Wild dolphins. Six of them decided to enjoy an experience of virtual travel and four of them to watch animals. For the VR experience, REDSTORM VR glasses [36] with built-in headphones for a more immersive experience, and adjustable grip were used. The ARLOOPA [37] application was used for AR, giving them the opportunity to overlap some virtual elements on their real environment.

After enjoying both experiences, the users filled a questionnaire to gather key information that could help us to primarily evaluate the suitability in the use of immersive technologies in the elderly population. The questionnaire was designed to capture also two KPIs:

  • Positive or negative feelings with the experience.

  • Weaknesses in the use of the devices, technologies and environment (Figs. 7, 8, 9, 10, 11, 12, 13 and 14).

Fig. 7.
figure 7

Virtual experience Machu Picchu.

Full size image
Fig. 8.
figure 8

Virtual experience Pride of lions.

Full size image
Fig. 9.
figure 9

Virtual experience Manhattan.

Full size image
Fig. 10.
figure 10

Virtual experience Maldives.

Full size image
Fig. 11.
figure 11

Virtual experience wild dolphins.

Full size image
Fig. 12.
figure 12

Virtual visit to Machu Picchu.

Full size image
Fig. 13.
figure 13

Virtual visit to the bottom of the sea with wild dolphins.

Full size image
Fig. 14.
figure 14

ARLOOPA app – screenshots.

Full size image

For reasons of data protection, only the gender and age of the respondents were collected as personal data. The age of the respondents is between 73 and 80 years old, 40% man, 60% woman.

They were asked about overall rating of the AR experience. The results were very positive, 60% 9 points over 10, 40% 10 points over 10 (Fig. 15). The users were also asked about their emotions during the AR experience. The answers (Fig. 16) reflected that they felt closer to the young population, being enthusiastic and excited to be able to use these technologies.

Fig. 15.
figure 15

Rating of AR experience.

Full size image
Fig. 16.
figure 16

Feeling during the AR experience.

Full size image

They were asked about the AR experience lived. Most of them declared it was fun and moving. They also stated that to see the information superimposed on their environment was impressive and exciting. They were entertained and showed interest in continuing to play with the application and its different functions. A summary of the answers gathered is shown in Fig. 17.

Fig. 17.
figure 17

Synthesis of the AR experience.

Full size image

On the other hand, the overall evaluation of the VR experience was also very positive, with 50% giving the experience the highest score and the other 50% giving it a 9 out of 10 (Fig. 18).

Fig. 18.
figure 18

Rating of VR experience.

Full size image

In order to synthesize the emotions experienced with VR, we provided them with a list of 6 statements (Fig. 19). Most of them stated that they felt disconnected from their everyday life, felt that for a short period of time they were away from home and had interest in the places they were visiting virtually. The experience was fun both for learning and getting to know something new and for feeling integrated by being able to use this technology.

Fig. 19.
figure 19

Feelings during the immersive experience.

Full size image

They were also asked for the positive things of using VR. All of them answered to have had an enriching and immersive experience. The feeling of doing something different for the first time and feeling integrated, both for the experience and for the use of the devices and technology. The list of the answers gathered is shown in Fig. 20.

Fig. 20.
figure 20

Synthesis of the experience with VR.

Full size image

Some problems were also detected during the experiment. Eight of the ten participants encountered some problems with the use of the glasses due to the weight of the glasses and the lack of sharpness in the images. With more modern and higher quality devices these sensations can be reduced considerably. Figure 21 summarizes the list of drawbacks.

Fig. 21.
figure 21

Discomfort while using VR glasses.

Full size image

Figure 22 shows participants’ ratings of their experiences with VR and AR according to gender. As can be seen in the graph in Fig. 23, there is hardly any difference, despite the fact that women rated them higher, both sexes rated the experiences above 9.

Fig. 22.
figure 22

Data collected on VR and AR experience ratings.

Full size image
Fig. 23.
figure 23

Average rating of VR and AR experiences by gender.

Full size image

Moreover, the trainers of the study also detected that most of candidates presented other problems during the experiment. Most of them were related to the difficulty candidates had in handling the devices, visualizing buttons or commands, need of training and help in its use. Finally, the high cost of some of the devices and applications for AR and VR, not mentioned during the experimental study, could be a handicap for their massive adoption for ageing population.

5 Conclusions

In this work, the AR, VR and MR technologies have been introduced as suitable tools for promoting cognitive and physical activity and for avoiding social isolation in ageing population. The work has included a review of the most popular hardware and software solutions found in the state of art and market and the use of them for specific activities focused on elderly people. An experimental study has been also carried out to get a preliminary feedback about the benefits and drawbacks of these technologies for elderly. Although the technologies used in the study were very welcome by the candidates, reflecting positive feelings and experiences, some drawbacks were exposed, most of them related to the convenience of the devices. Trainers in the study also detected that elderly population need specific training for the use of these technologies. The positive balance in the experimental study opens the door to extend the study with specific KPIs that can better measure the improvements of cognitive and physical activity through continuous training, and e.g. to study the benefits for ageing population with specific impairments or degenerative diseases like dementia, Parkinson or Alzheimer.