Keywords

1 Introduction

Digital technologies, nowadays, are used substantially in several industrial and healthcare applications to display and approach environments which are physically inaccessible. Such technologies are based on the effective real-virtual interactions for the users. Virtual reality (VR) creates a simulated environment using digital technology, where the users are “immersed” in the experience and are able to interact with 3D scenarios. VR systems are commonly characterized by a headgear apparatus which exploits senses of vision and hearing; however, few advanced VR systems include haptic feedback technology to provide the impression of touch by employing forces, motions, or vibrations (Rizzo and Galen Buckwalter 1997; Riva 1997). These systems involve high computational power and intelligent sensors to position the subject’s eyes within the surrounding such that the graphics react relatively to the user’s movements. Meanwhile, Augmented Reality (AR), another form of digital technology, relates the virtual data to the actual world by overlaying simulated objects or sights produced through the computer to the actual scenario (Liu et al. 2017). This regulates the location and orientation of a camera using sensors and related algorithms. Graphics are rendered by superimposing simulated images over a subject’s view from the real scenario. Milgram and Kishino (1994) define Mixed Reality (MR) as the merger of real and virtual elements where users are allowed to interact with both the elements through a single display screen. MR interfaces enhance the functionality of actual world instead of switching it entirely using the combined features of both AR and VR. On the other hand, the extended reality (XR) amalgamates the features of different digital technologies to improve the real and un-real experiences, collectively. It can have elements of immersion (VR), augmentation (AR), or both (MR). Figure 1 presents the classification of digital technologies.

Fig. 1
figure 1

Categories of digital technology

Full size image

VR is practically omnipresent in every industry-education,  healthcare, tourism, construction, architecture, entertainment, sports, art and design, event management, marketing, law enforcement industries, to name a few (Helsel 1992; Portman et al. 2015; Alcañiz et al. 2019; Bates 1992). Studies have shown that using VR in classrooms accelerates dynamic rendering, closed-loop interaction, and enhanced sensory feedback having a beneficial effect on retention (Helsel 1992). VR environments used in architecture and environmental planning can help plot designs, maps, and access remote territories (Portman et al. 2015). Marketing experts are showing keen interest in Extended Reality (XRs), technology similar to VR, to produce copacetic experiences for the consumer by reflecting those practiced in physical shops (Alcañiz et al. 2019). AR is generally maneuvered for applications like training, path planning, remote collaboration, warehouse logistics in manufacturing, tourism, medicine, and military services (Bates 1992; Ćuković et al. 2020; Petruse et al. 2019; Wei et al. 2014). Haptic and audio displays for tourism applications have shown an increased effect on interaction (Wei et al. 2014). Apart from training purposes, AR is used in military research for simulation of equipment like unmanned aerial vehicles (UAVs) in unknown areas (Ma’Sum et al. 2013). Similarly, MR interfaces are mainly designed for manufacturing and visualization processes. For instance, a “Needle biopsy” setup developed by Bajura et al. (1992) uses MR to overlay virtual ultrasound images onto a patient's body, allowing doctors to understand exactly where and how to insert the needle. Figure 2 indicates a video image presented to head mounted display (HMD), illustrating a sight of the subject’s abdomen along with a superposed 2D ultrasound image. Therefore, these reality interfaces can enable a person to see, connect, and interact with different worlds in seemingly impossible ways. The research discussed in this paper, however, is limited to VR and AR technologies in the field of medical rehabilitation.

Fig. 2
figure 2

MR system showing 2D ultrasound image superimposed on a subject’s abdomen (Bajura et al. 1992)

Full size image

VR and AR are without doubt powerful tools to monitor, replicate, and alter the healthcare activities in a safe environment without changing anything on the user end and are therefore potential for improving recovery and aiding medical care. These tools provide optimized functional results and enhanced clinical benefits in post-surgery rehabilitation. For example, VR and AR technologies have proven to be practical and beneficial interventions in cases of vestibular rehabilitation, primary total knee arthroplasty (TKA), and Chronic Obstructive Pulmonary Disease (Stankiewicz et al. 2021; Gianola et al. 2020; Rutkowski et al. 2019). VR Exposure (VRE) therapy is a psychological treatment where doctors create a safe environment and expose the patients to things they fear or avoid in a controlled manner. VRE is used extensively to overcome clinical phobias and disorders like acrophobia, claustrophobia, Post-Traumatic Stress Disorder (PTSD), substance use disorders, social anxiety disorder, panic, generalized anxiety disorder, obsessive compulsive disorder, schizophrenia, psychosis, pain, addiction, eating disorders, and autism (Rus-Calafell et al. 2018; Maples-Keller et al. 2017; Sun et al. 2014).

Over last few years, the usage of VR and AR has been started in the domain of neurological reintegration, specifically in subjects with brain injury, Stroke, Parkinson's disease (PD), Multiple Sclerosis (MS), and Cerebral Palsy. Conventional therapy (CT) in view of stroke, PD, MS includes physiotherapy and kinesiotherapy sessions to reduce the difficulties regarding spasticity, pain, and fatigue in motor impairments (Maggio et al. 2019). However, these traditional approaches are not very effective due to reduced motivation, boredom, and lack of support, leading to decreased participation. Conversely, VR and AR therapy (VRT and ART) approaches cover the four fundamental aspects of rehabilitation: intensity, task-oriented training, biofeedback, and motivation. Such therapy approaches are repetitive and designed in accordance to tasks relevant to upper extremity or lower extremity. It is often used with CT, therefore increasing the intensity of traditional exercise. To compare the results of CT and AR/VR therapy, specific outcome measures are used. Outcomes like Box & Block Test (BBT) measures, Functional independence measures (FIM), Fugl-Meyer assessment (FMA) scores, modified Barthel index (MBI), Balance Performance Monitor (BPM), Berg Balance Scale (BBS) tests, 6 min walk test (6mwt), 10 min walk test(10mwt), GAITRite are crucial to determine the quality of the findings. Studies and clinical trials have shown the effectiveness of VR therapy to carefully detect the explicit FOG triggers and balance debilities in patients suffering from PD (Li et al. 2011). In the case of MS and post-stroke patients, VRT is oriented toward the reconstitution of both motor and cognitive dysfunction to favor the ADL, augmenting the enduring abilities and learning of fresh strategies for spasticity, pain, and fatigue (Maggio et al. 2019; Calabrò et al. 2017). This technology can also be adaptive by modifying itself in accordance with patients’ feedback.

As several VR and AR systems have been designed and developed for rehabilitation purposes, there is an emergent need to review such systems comprehensively to understand their functionality and clinical efficacies. This paper aims to review VR and AR therapy approaches designed for rehabilitation purposes. The paper is organized as follows. Cases of VR and AR in health care, particularly post-stroke, MS, PD rehabilitation are first presented in Sect. 1. The adopted methodology along with a PRISMA report is presented, in Sect. 2, to support the inclusion-inclusion criteria of articles. Thereafter, the working and types of VR, AR systems are explained in brief in Sects. 3 and 4. Section 5 presents VR and AR therapy approaches for UE motor  rehabilitation. Section 6 discusses VR and AR therapy applications for LE rehabilitation. The existing shortcomings and related areas of technological improvement are discussed in Sect. 7. At last, concluding remarks of this review work are presented in Sect. 8.

2 Methodology Adopted for Systematic Review

A comprehensive literature was searched within various electronic databases such as Scopus, PubMed, Google Scholar, Institute of Electrical and Electronics Engineers (IEEE), and ScienceDirect was searched. The searched key inputs were like “(virtual reality OR game-based virtual reality OR computer-based virtual reality OR VR-based rehabilitation) AND (augmented reality or game-based augmented reality OR AR-based rehabilitation) AND (stroke OR PD OR hemiplegia OR brain injury OR multiple sclerosis OR traumatic brain injury) AND (Upper extremity OR cognitive OR motor OR Lower extremity OR executive function).” Using these key inputs, 72 out of 533 full-text articles are realized to be appropriate and, thereafter, studied in an exhaustive manner. The stepwise identification, screening, eligibility, and inclusion of 72 relevant articles are depicted in Fig. 3, utilizing a PRISMA flowchart (Moher et al. 2009). The final articles are included to refer VR, AR devices for UE, and LE rehabilitation for subjects suffering from Stroke, PD, MS (Tables 3, 4, 5 and 6). Exclusion criteria were (1) subjects without stroke or PD or MS; (2) subjects who were animals or children; (3) studies that did not affect physical UE/ LE rehabilitation; (4) studies that used methods other than AR/ VR for rehabilitation; (5) devices that did not provide feedback of any kind; (6) only design ideas were projected, no actual device included; (7) papers before year 2004.

Fig. 3
figure 3

Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) flowchart (Moher et al. 2009)

Full size image

The main study of this paper is AR and VR-based rehabilitation for patients suffering from deficits in upper limb and lower limb movements. Although few quality review works are already available in the literature (Schultheis and Rizzo 2001; Sveistrup 2004; Howard 2017; Kim 2005); however, either they were published more than a decade (Schultheis and Rizzo 2001; Sveistrup 2004; Kim 2005) or having non-systematic presentation in view of upper extremity and lower extremity dedicated devices (Howard 2017). In case of other review papers published recently (Penn et al. 2018; Huang et al. 2018a; Dunn et al. 2017), the VR and AR technologies for the rehabilitation purposes are not presented exhaustively and discussed either specific to a certain disease or specific to an extremity. To the authors’ best knowledge, this review work has explored all the possible design features and functionalities of VR and AR solutions for the rehabilitation of upper and lower extremity.

3 Virtual Reality Systems

Immersion in VR is the effect caused by a situation, environment, or graphic representation which makes the user perceive the projected environment as reality. Jennett et al. (2008) define VR in terms of user involvement; and consider it as the reason behind lack of awareness of time and actual world, along with a feeling of “being” in the work surroundings. While talking about immersion in general VR cases, the term “spatial immersion” is used, which means being physically present in a fabricated environment. This occurs when a user’s senses are partially/ fully stimulated by a VR system using images, sound, and other feedback sources to feel the said world as real. Considering an important element of a VR system, the levels of immersions can be varied for different purposes. There are three primary categories of VR simulations, differentiated on the degree of immersion as seen in Fig. 1.

3.1 Non-Immersive VR

Non-Immersive VR allows the subject to interact in a simulated world and can be straightforwardly deployed using input devices like joystick, monitor, keyboard, or a mouse (Robertson et al. 1993). Even though it is a computer simulated world, the user is well-aware about the surroundings and can control aspects of this environment. Non-immersive systems are also considered economic and are generally easier to set up as compared to immersive VR. Video game systems or movie systems are common examples of this system. Non-immersive VR is also used in rehabilitation, for instance, a RAPAEL smart glove experience developed by Lee et al. (2020) proved beneficial to improve the upper limb function of stroke patients.

To reduce fall risk and improve gait rehabilitation in older adults, a non-immersive VR system with a motion-capture (MOCAP) camera setup and a computer-aided simulation  demonstrated positive results (Mirelman et al. 2016). The VR system consisted of a motion-capture camera and a computer-generated simulation projected on to a large screen, which was specifically designed to reduce fall risk in older adults by including real-life challenges such as obstacles, multiple pathways, and distracters that required continual adjustment of steps. Figure 4 illustrates this VR system. Non-immersive VR systems are therefore considered as a powerful tool to improve the neurological disorder-related symptoms and to elevate cerebral and motor function of the brain.

Fig. 4
figure 4

Treadmill training using a non-immersive VR system (Mirelman et al. 2016)

Full size image

3.2 Semi Immersive VR

Semi-immersive simulated practices provide the feeling of presence in a different certainty while still staying aware of the surroundings. Quality details of the graphic along with the feedback provided by the system are directly proportional to the immersive feeling. Hardware for these systems generally includes high-resolution screens, powerful processors, and projectors to partly imitate the design and functionality of practical real-world scenarios. This class of VR is often utilized for educational or vocational training. Studies suggest that semi-immersive VR could be a beneficial approach for therapy of patients with traumatic brain injury, potentially leading to better cognitive and behavioral outcomes (Luca et al. 2019).

3.3 Fully Immersive VR

Fully immersive is the most realistic simulation experience to perceive and indulge with complete immersion-based virtual reality, where the operator needs the relevant supporting tools. VR headsets are most commonly used to offer high-resolution data with a varied field of view for a surreal immersive VR experience. The display creates a stereoscopic 3D effect and follows with the input tracing and feedback to create an authentic experience. A Cave Automatic Virtual Environment (CAVE) is a completely immersive VE wherein the user wears 3D glasses and is surrounded by projection screens or flat displays. It is widely used for education and training purposes (Ott and Pozzi 2008). VR glasses and Head Mounted Displays (HMD) deliver visual and auditory cues in the form of detailed graphics and auditory information. In addition to the benefits mentioned above, VR systems like Oculus Rift also allow for precise tracking of the user’s movements, thus making it very useful for education, training, and rehabilitation purposes (Basu and Johnsen 2014). While CAVE systems are more expensive and difficult to move, HMD, VR glasses, and Oculus Rift are relatively cheaper and easy to handle.

3.4 Tools Supporting VR Technologies

VR systems are commonly used for medical training purposes to perform tasks, enhance skills and simulate complicated procedures. Lap Mentor is a frequently used semi-immersive system, to simulate laparoscopic surgery (Alaker et al. 2016). Khalifa et al. (2006) review the Eyesi VR system to train prospective students for cataract and vitreoretinal surgery training. A non-immersive VR-based tool, named as modified IREX program, was designed by Thornton et al. (2005) to assist in brain traumatic injury. Subramanian et al. (2007) discussed CAREN VR, a CAVE type immersive-based simulation system, to improve hand impairments in stroke patients effectively. Another immersive system-nVisorSX was used in a work by Sharar et al. (2007) while providing therapy for severe burn patients. The Novint Falcon along with a leap motion controller assisted in upper limb VRT for patients suffering from hand motor impairments (Ramírez-Fernández et al. 2015). The key details of the VR-based medical devices are enlisted in Table 1).

Table 1 VR-based devices used in medicine
Full size table

4 Augmented Reality Systems

AR can be explained as a modification of the actual environment by adding visual or sound or other stimuli to it. The user interacts with the digital world and the system does the changes to the world by augmenting elements to it. Edwards-Stewart et al. (2016) classify AR systems into two main categories-triggered and view-based augmentation; shown in Fig. 1. Triggers refer to characteristics like object markers, GPS location, and dynamic augmentations of objects that initiate the augmentation.

4.1 Triggered-Based Augmentation

Trigger-based AR comprises Marker-based AR, Location-based AR, Dynamic Augmentation, and View-based AR. Marker-based AR can be either object based or paper/ image based. The object or image containing the marker is called the trigger object and it can be recognized by the AR system upon scanning. The scan triggers an additional sequence where more relevant content can be displayed on the device. Marker-based AR has been instituted successfully with patients suffering from animal phobias. Location-based AR is geo based and marker-less—it relies on GPS, accelerometer, digital compass, and other technologies to accurately identify a device’s location. Dynamic AR, usually included with motion tracking, is receptive to the object’s view as it alters. Lastly, the fourth kind of triggered AR is complex augmentation, defined as a hybrid form of location-based AR and dynamic amplification. A popular example of this is Google Glass, where users can access information regarding local spots depending upon their GPS location (Edwards-Stewart et al. 2016).

4.2 View-Based Augmentation

View-based AR consists of Indirect AR and Non-specific Digital AR. Indirect AR means augmenting static images as per the user’s preference. For example, trying on clothes virtually by superimposing clothes onto an existing image of the person. Non-Specific Digital AR refers to digitize a dynamic outlook of the environment without having any reference to what is being perceived (Edwards-Stewart et al. 2016). This is a common policy to be found in mobile games. The operator intermingles with the augmentation like tapping the augmented scenarios upon viewing without having a reference to the operator’s surroundings. However, it is pertinent to mention that view-based augmentation is not considered to be a part of AR in accordance with Milgram et al. (1994).

4.3 Tools Supporting AR Technologies

Some AR devices used in health care for visualization and training purposes are listed in Table 2. For anatomy education, a “Magic mirror” is used, where the system contains a sensor which tracks the user and displays all the anatomical organs and parts of the user on a LCD display (Ma et al. 2016). A projector-based MRI system enables simulated navigation of tracked interments on pre-defined routes and conception of risk structured on the subject undergoing MRI (Mewes et al. 2019). The Endosight system is a guidance system that assists in oncology procedures by visualizing 3D anatomical structures, tumor targets, and interventional tools on subject’s body (Solbiati et al. 2018). Sutherland et al. (Sutherland et al. 2012) explore an AR Haptic simulation system which uses an optical tracking system, a haptic device, and a GUI to offer visual feedback for spinal needle insertion process. AR BOOK is an educational tool with AR modules concentrating on the lower limb’s anatomy (Ferrer-Torregrosa et al. 2015). In another case, smart glasses were used as an educational tool to provide visual feedback for AR simulation of central venous catheters (CVCs) to train novice operators (Huang, et al. 2018b).

Table 2 AR-based devices used in medicine
Full size table

5 Upper Extremity (UE) Rehabilitation

Damage or impairment of motor function in the UE of patients compels to not move their upper extremities flexibly and accurately. Therefore, a system for UE rehabilitation needs to be developed to help the patients to retain these motor functions and improve the quality of their life (Narayan et al. 2021). Traditionally, for these cases, CT primarily consists of repeated movements involving upper or lower limbs, which makes the patient disinterested and reduces the effects of rehabilitation (Ying and Aimin 2017). However, applying digital technology to CT provides an interactive experience for the users, enhancing the rehabilitation quality and results.

5.1 VR Technology-Based Upper Extremity Rehabilitation

VR techniques allow for repetitive learning, well-rounded feedback to all the senses, augmented practice and can be paired with robotic devices/ exoskeletons to increase effectiveness (Cameirão et al. 2008). Users react with virtual objects in a directly using hand gestures and body movements or via devices like glove, joystick, and mouse. Table 3 discusses and provides evidence concerning current applications of VR Therapy for UE motor recovery.

Table 3 VR tools for UE Rehabilitation
Full size table

In a clinical trial conducted by Yin et al. (2014), the feasibility of VR training on early stroke subjects was investigated. Substantial improvement in FMA was obtained when participants were subjected to 30 min of VR therapy for weeks, 5 times each week, in addition to CT. VRT consisted of a Sixense unit, an electromagnetic sensor system that identifies the movement in 3D and a customized training program that consisted of highly repetitive tasks and different difficulty levels. Afsar et al. (2018) used the Microsoft Xbox 360 Kinect video game system to provide 30 min of VR therapy per day in addition to 60 min of CT for 4 weeks. The delta-BBT score for the experimental group has shown the significant improvement as compared to the control group (p = 0.007), proving that the Kinect-based game system may have added advantage for stroke patients. Figure 5 illustrates Jintronix, a virtual reality exergame system used to improve motor function in stroke survivors (Norouzi-Gheidari et al. 2020). Conducting VRT in addition to CT, post-intervention improvements were observed in ADL measures. Choi et al. (2016) used convenient VR via a mobile phone for 10–30 min of VRT sessions for 2 weeks. Notable results were seen in the FMA-UE, B-stage, and MMT after treated with the MoU-Rehab as compared to the conventional therapy.

Fig. 5
figure 5

The Jintronix rehabilitation exergaming system (Norouzi-Gheidari et al. 2020)

Full size image

Rutgers arm is a system involving a low-friction table with a 3D tracker and a library of virtual reality (VR) exercises. A telerehabilitation extension of this was developed by Kuttuva et al. (2006). The device was examined on a chronic stroke patient for over 5 weeks and improved FMA test scores were recorded for shoulder range of motion. In a similar study by Burdea et al. (2011), Rutgers arm II was introduced to sense and support the arm movement and thereafter, tilted to resist or assist reach. The VR games adapted automatically according to each individual's motor abilities and significant positive FMA scores were obtained along with self-reported changes in the participants' ADL. Improvements were also reported in active ROM and grasp strength. Kang et al. (2012) used virtual mirrors with visual modulation in a study including healthy and stroke patients. The study presented positive results for the virtual mirror task and proved that visual modulation is an effective form of therapy for UE rehabilitation in stroke patients. Another VR visual feedback therapy via HTC Vive HMD was explored in the butterfly project by Elor et al. (2019). The users experienced physiotherapy by following and guarding a virtual butterfly, with the help of a robot-based wearable device to assist the subject’s UE movements. Shin et al. (2016) introduced a biofeedback system containing a glove-shaped sensor device and a software application, called the RAPAEL Smart Glove which indicated improvements in the FMA, JTT, and SIS scores of patients with problems of distal UE function.

Sánchez-Herrera-Baeza et al. (2020) conducted a study for PD patients, using immersive VR technology—Oculus Rift 2 and a leap motion controller—OR2-LMC. They observed significant improvements in strength, fine and gross coordination dexterity, and mobility speed in the impaired side with an outstanding agreement. In a novel experimental setup by Maggio et al. (2019), a semi-immersive therapy system called BTS-N was used by PD patients. The results indicated an improvement in cognitive functioning pertaining to executive and visuospatial activities among the subjects. Cuesta-Gómez et al. (2020) used a Leap Motion Controller (LMC) System, which incorporates non-wearable sensors to capture the movement of the forearms and hands. Concluding results showed significant improvements in the post-treatment examination for coordination, locomotion speed, fine and gross UE dexterity.

5.2 AR Technology-Based Upper Extremity Rehabilitation

In most cases of AR therapy assisting UE recovery, gaming systems including virtual and real objects are used. Markers are attached to real objects and the systems track the moment, position, and orientation of these objects using a webcam. The system then seamlessly augments the real environment with the virtual world to present different tasks to the user which engages the user’s UE movements. Some cases mentioned below include robotic devices and exoskeletons to assist UE rehabilitation. Table 4 lists the development of AR-based UE rehabilitation prototypes.

Table 4 AR tools for UE Rehabilitation
Full size table

Bank et al. (2018) implemented three AR games that used a HMD and tracked the user’s hand and body without any contact. Figure 6 demonstrates the setup of this system. The game objectives included speed of movements, adjustment of hand opening, and obstacle avoidance. For the first game, maximum reach distance was slightly greater in controls (98.0 ± 2.9%) than in PD patients (96.8 ± 2.9%, p = 0.04) and stroke patients (95.5 ± 2.9%, p = 0.06). In the second game, it was observed that PD patients moved slower than controls. In the third game, success rate did not differ much between controls (100 [100–100] %) and PD patients (100 [75–100] %, p = 0.21) or stroke patients (100 [75–100] %, p = 0.09). Thus, results obtained were almost similar for the CG, PD, and Stroke patients.

Fig. 6
figure 6

Participant with AIRO II system (Bank et al. 2018)

Full size image

In a case study for post-stroke patients, Luo et al. (2005) created a training environment that integrated augmented reality (AR) and virtual objects with assistive devices like gloves containing body-powered orthosis (BPO) or pneumatic-powered device (PPD). This method demonstrated beneficial results. A NeuroR system developed by Assis et al. (2016) works by providing visual feedback of the illusion of injured UE movements while the affected limb is resting, resulting in increased FM scores. In an interactive game proposed by Van der Meulen et al. (2016) for PD patients, the system engages participants’ UE movements by employing AR to show simulated motion targets, (like candies drop from a conveyor belt, and a haptic game controller to grab the candies).

6 Lower Extremity (LE) and Gait Rehabilitation

After reviewing the VR and AR devices for UE rehabilitation, the design of such devices for rehabilitation of lower extremity is discussed in this section. Diseases such as stroke, PD, MS affect the motor controls of the body and as a consequence, the patient's ability to walk is impaired. Therefore, the primary focus of rehabilitation is gait recovery and balance. In recent years, VR, AR technology (with or without Robotic interventions) have shown improved locomotion evidence in patients with motor defects. Kalita et al. (2020), Narayan and Dwivedy (2021, 2020). To find the effectiveness of these technologies, VR/ AR digital therapies are often tested in clinical trials along with patients receiving only CT. The most frequently used outcome measures in the different trials are gait speed, balance, and improvement of the motor function.

6.1 VR Technology-Based Lower Extremity Rehabilitation

Several studies have shown positive effects of VR-based treatments for LE motor rehabilitation. Specific VR, interactive video games, reflection therapy, and robot-assisted VR are some interesting approaches to the patient's rehabilitation. Table 5 categorizes VR-based rehabilitation therapy for neurological conditions and also summarizes current research in LE application.

Table 5 VR tools for LE Rehabilitation
Full size table

Jaffe et al. (2004) conducted a trial wherein post-stroke subjects were requested to step over virtual objects or real objects on a 10 m treadmill (TM). The VR system provided visual, auditory, and vibrotactile feedback. The subjects expressed improvement in gait velocity, stride length, walking endurance, and obstacle clearance capacity, proving the effectiveness of obstacle training for improving gait velocity. Mirelman et al. (2009) used a robotic VR system to improve balance, speed, step time, step length, and stride in patients suffering from stroke. In a separate trial, the Rutgers Ankle, a 6-DOF robot with a VR simulation interface was used. Results from this trial intimated improved motor control of the ankle-ankle ROM changed by 19.5% (Mirelman et al. 2010). A VR gait training program for stroke patients was designed by Cho et al. (2013) by exploiting a video recording of the actual environment. The findings exhibited a greater improvement of the VR group on the BBS and TUG test, suggesting that this program may be a valid approach to improve gait performance. A virtual reality rehabilitation system (VRRS), appended to a MOCAP system, was successfully used (Luque-Moreno et al. 2016) to improve leg stance and walking speed in two post-stroke individuals.

Virtual reality reflection therapy (VRRT) was used effectively in a study performed by In et al. (2016) among post-stroke patients. The setup used is illustrated in Fig. 7. Significant improvements were observed in BBS, FRT, TUG, and postural sway outcomes among the concerned patients. VRRT can also be applied at home along with CT to improve affected LE function. Bergmann et al. (2017) introduced VR-augmented robot-assisted gait training (RAGT) to induce balance and gait recovery in stroke patients. The intervention manifested positive results like high acceptability and motivation, and slashed dropout rate, and an extended training period as compared to the standard control group. Fifteen PD patients with FOG took part in a trial designed by Janeh et al. (2019) that included VR-based gait modulation tasks on a GAITRite walkway system. The tasks effectively improved step width and swing interval parameters, proving to be a beneficial for manipulating gait characteristics in PD. Figure 8a, b demonstrate the experimental setup for the GAITRite walkway system. In a study conducted by Mendes et al. (2012), subjects with PD took part in Wii Fit training along with warm up exercises specifically designed to improve motor and cognitive skills. After more than 7 games, PD patients were capable to transfer the motor functionality attained through the games to an equivalent new task. In a similar study, Liao et al. (2015) used virtual reality-oriented Wii Fit exercise (VRWii) to enhance obstacle avoiding features and dynamic stability of PD patients. Wii Fit training is, therefore, a potential training method to improve motor controls and reduce FOG in PD patients.

Fig. 7
figure 7

Setting for virtual reality reflection therapy (In et al. 2016)

Full size image
Fig. 8
figure 8

Experimental system: a subject walking with HMD over the actual surface of GAITRite® b subject’s view of the simulated environment on the HMD (Janeh et al. 2019)

Full size image

Studies have shown that VR-based TM training tasks have managed to improve factors like speed, cadence, stride length, walking endurance, and lower limb joint ROMs in MS patients (Peruzzi et al. 2017). In a case study of an individual lady suffering from MS, a body weight support (BWS) with a treadmill and over ground walking activity was conducted in association with VR-based balance intervention for 12 weeks. In addition to high motivation, improved test results were observed for her during the LE rehabilitation tests (Fulk 2005). Another VRT (Fung et al. 2006) consisted of a locomotor training system involving a self-paced TM mounted onto a 6-DOF motion platform. Different scenario VEs were woven into the gait training program that provided various levels of complexity. With practice, patients could effectively adjust their walking style and speed pertaining to changes in the game and as required for the task. This study, therefore, demonstrated that patients with stroke were capable to regulate themselves as per the VR system and were immersed effectively for gait training.

6.2 AR Technology-Based Lower Extremity Rehabilitation

AR Therapy includes visual and auditory augmentation using systems like smart glasses, HOLOLens, smart treadmills for postural training, gait and balance stabilization to enhance LE rehabilitation. Table 6 presents the experiences of tools based on AR focusing on LE rehabilitation.

Table 6 AR tools for LE Rehabilitation
Full size table

AR-based postural balance activity for stroke patients in addition to CT indicated positive results on the TUG test, BBS, cadence, velocity, step length, and stride length of impaired and non-impaired sides (Lee et al. 2014). A recent study by Held et al. (2020) introduces the ARISE (Augmented Reality for gait Impairments after StrokE) system which provides evidence of gait adaptation via visual and auditory augmentation. This approach is a combination of HoloLens 2 glasses and a sensor-oriented MOCAP system, as shown in Fig. 9. It was used by one post-stroke patient where he completed gait assignments and an AR parkour program. HoloLens with auditory and visual cues was used in a study for PD patients with FOG, where the subjects performed 180° turns with two control settings (one with auditory cues and one without cues). The study showed that visual cues worked better than auditory cues and the AR visual cues reduced the peak angular velocity and step height compared to both control conditions (Janssen et al. 2020). In another study by Janssen et al. (2017), PD patients with FOG performed walking tasks under different conditions. Three-dimensional AR visual cues were conveyed by smart glasses and auditory cueing via a metronome was also included depending on the control conditions. However, augmented visual cues conveyed by smart glasses were not found to be advantageous for subjects with PD and FOG.

Fig. 9
figure 9

Post-stroke patient wearing the ARISE system (Held et al. 2020)

Full size image

7 Discussion and Future Opportunities

VR and AR Technologies are extensively employed in the area of medicine, showing positive effects especially for disability management, rehabilitation, surgical training, and psychological diseases therapy. In this review work, VR- and AR-based therapy for both UE and LE rehabilitation purposes, especially in case of brain injury, stroke, PD, and MS. Recent devices and intervention techniques used for therapy are listed in tables and compared, keeping in mind-feasibility, effectiveness, and result outcomes. Using VR, AR systems for rehabilitation therapy promotes neuroplasticity and motor learning while making sure that challenging tasks are practiced in a safe environment. The motor cognitive or limbic challenges can be tailored to fit specific patient needs and can follow customized straining strategies. The therapies, in most cases, are adaptive, i.e., task variation and progression happens in accordance with the patient and his/ her performance. Some of these systems also have an added advantage of portability, accessibility, ease of use, and no need of professional supervision. Almost all cases of VR, AR systems mentioned in this paper have shown increased motivation, enjoyment, and acceptability among patients-leading them to complete the therapy and a significant reduction in dropout rates (Canning et al. 2020).

However, intense nature of physical and cognitive challenges may cause unwarranted fatigue and might cause dizziness, eyestrain, motion sickness, and loss of coordination. Feedback can also sometimes play an adverse role. Excessive feedback may confuse the patient; discouraging feedback might put a damper on spirits; incomplete or inaccurate feedback does not tell the patient how to proceed precisely. Apart from this, VR and AR systems are not always feasible in the sense that sophisticated systems are usually costly and inaccessible. VR- and AR-based interventions are therefore considered inevitable steps toward revolutionizing the digital technology-based approach toward neuro-rehabilitation. Some points to be considered to make these interventions even more effective are:

  • Most of the control groups are very exclusive and take into account very specific details of a disease. Control groups need to be broadened, bit by bit, to include a larger group of people with some variations. Researchers need to be thorough and clearly define the intervention factors such as frequency, dosing, number of repetitions, and ardency (Proffitt and Lange 2015).

  • Methods to track the patients’ movements or gestures should be error free. More than one process can be used in a single tracking system. Feedback provided by the system should be accurate, should provide further steps to improvement (Ying and Aimin 2017).

  • AR, VR game therapies should include a wider range of tasks which can guide the patients accurately (Ying and Aimin 2017). Increased number of tasks should be made available on easily accessible hardware. Currently, in many cases, standard controllers are used; these need to be modified as per need of PD/ Stroke patients. VR and AR technology can be made “wearable” so it can be used more easily for ADL.

  • Furthermore, the research and development should be more focused to address concerns regarding standardization, power consumption, measurement validity, interoperability, and discretion of devices.

  • At home, low-cost therapy options can make VR and AR approaches accessible for everyone. In addition to this, professional supervision will not be needed everywhere and the patient can practice therapy by himself.

8 Conclusion

In the last decade, extraordinary improvements have been made regarding the development of virtual and augmented reality systems for motor rehabilitation. Several target populations have been considered, especially stroke PD and MS patients suffering from UE and LE defects. In this work, at first, the knowledge base of different digital technologies has been established. Thereafter, in the context of VR and AR applied to the therapy of UE and LE, this chapter has reviewed some of the main developed systems and described their major findings. Related paradigms and therapy concepts have been  grouped in four different categories: VR-based therapy for UE, AR-based therapy for UE, VR-based therapy for LE, and AR-based therapy for LE. All these techniques have a few common concepts like learning by imitation, reinforced feedback, haptic feedback, augmented practice and repetition, video capture virtual reality, exoskeletons, mental practice, and action execution/observation. VR- and AR-based approaches allow us to add to the conventional therapy to make it more effective, in a short period of time. In general, the patients that used VR and AR environments have experienced significant improvements in several performance parameters which directly impacts the activities of daily living. This review  will act as a guide for research communities to use digital technologies for rehabilitation purposes.