Background

Musculoskeletal disorders and injury are leading causes of disability worldwide [1]. Workplace injury often leads to disability and substantial time loss from work. Improved assessment and rehabilitation strategies are needed to reduce the burden of work disability due to musculoskeletal conditions. The standard practice in occupational rehabilitation is to first assess injured workers’ functional ability, often via Functional Capacity Evaluation (FCE) [2]. This is done by assessing a worker’s performance during a set of simulated work tasks. This often includes low-level lifting, a task that is an important predictor of recovery and return to work [3, 4]. A variety of FCE protocols and systems are available, but most include functional tasks that require different sets of equipment that are specific to a single task. For example, lifting and carrying are often tested using a crate and free weights on adjustable shelving, pushing and pulling may be done with a weighted sled or wall-mounted strain gage, hand coordination is tested with some form of manual dexterity equipment, etc.

FCE is widely used to assess injured workers before, during and after rehabilitation. Several studies have been carried out to evaluate FCE validity for assessment of work ability. Peppers et al. showed that combining clinical evaluation with FCE improves physicians’ assessments of the patient’s skills and work capacities [5]. FCE has been found to significantly predict return to work [4], and functional assessment is an integral component of graded activity and functional rehabilitation programs [6,7,8]. However, Edelaar et al. concluded that further research is needed in FCE, especially on the incorporation of computer technology (including robotics and digital sensors) [9].

Currently, there exists no single, streamlined solution to simulate all or a large number of standard work tasks. Also, assessment metrics for these tasks are limited by what a therapist can observe qualitatively. This means that assessments are based only on clinical metrics such as how much weight was lifted, the height it was lifted to, the completion time, judgments of the patient’s effort, etc. More complex and quantitative assessments are still unavailable, although wearable sensors and other technological measuring devices are rapidly developing [10, 11]. Moreover, injured workers residing in rural or remote areas may not be able to receive timely assessment or rehabilitation given a lack of resources or services in their areas. Creative solutions are needed to address these issues.

Innovative solutions may be found in the use of rehabilitation robotics, an area that has seen important development over the last three decades. The ability of robots to provide repetitive, high-intensity interactions without fatiguing makes them a useful method for providing the repetitive tasks that are fundamental to FCE and occupational rehabilitation, but that may be found tedious or fatiguing for human assessors [12]. Moreover, with the advent of robots with internet connectivity, there is potential for assessments and treatment to take place at a distance. Research in the rehabilitation robotics area has sought to improve the stability of these robots to make them patient-safe, as well as to provide them with the ability to adapt their behaviors based on feedback to assist or resist patient activity during assessment and exercise.

One goal of rehabilitation robotics research has been to improve the assessment of injured workers by providing methods for easily simulating workplace tasks using intelligent robotic systems [13]. Such a system would provide a single, streamlined solution for both FCE and functional rehabilitation and be able to simulate reaching maneuvers as well as more dexterous functional tasks that are typical of workplace activities. Utilizing machine learning-based approaches, we have attempted to combine the benefits of robotic systems with the expertise and experience of human therapists by allowing a robot to learn based on a therapist’s motions (“demonstrations”) what the desired workplace activity (“task”) is and how to recreate it for a worker with injury (“patient”). This paper will provide a brief history of rehabilitation robotics and review efforts to incorporate machine learning algorithms into robotic solutions for FCE.

Robotics in Rehabilitation

Initially, most robots used in rehabilitation were for assistive purposes [14]. These robots did not aim to help regain lost motor function or abilities of the patient, but rather they aimed to assist the patient in performing activities of daily living. These were commonly seen as robots attached to wheelchairs to assist with eating and drinking, grabbing objects, and power mobility [15]. It was not until the late 1980s that researchers pursued rehabilitation robotics for use in therapy and neurological rehabilitation [16]. Research in rehabilitation robotics started to search for solutions that would produce more efficient and effective rehabilitation techniques.

In 1988, two double-link planar robots were coupled with a patient’s lower limb to provide continuous passive motion for rehabilitation [17]. This was soon followed by an upper-limb rehabilitation device in 1992, the MIT-MANUS, which was used for planar shoulder-and-elbow therapy [18]. Upper limb rehabilitative devices were further developed after the advent of the MIT-MANUS. These include devices such as the Mirror-Image Movement Enabler (MIME) robotic device, which improved muscle movements through mirror-image training [19], and the Assisted Rehabilitation and Measurement (ARM) Guide, which functions both as an assessment and rehabilitation tool [20]. Robotic rehabilitation that targeted other areas of the body surfaced in the 2000s. These robotic devices allowed rehabilitation for the wrist [21], hand, and finger [22] in the upper limb, and gait and ankle training for the lower limb [23, 24]. Robots have also been developed for training patients to perform activities of daily living [25, 26].

While the majority of research in rehabilitation robotics has been in neurological rehabilitation, there has recently been interest in robots for occupational rehabilitation [13]. Including robots in therapy to provide therapist-robot-patient interactions presents potential advantages over conventional therapist-patient interactions within occupational rehabilitation. Current FCE practice is performed by using standardized assessments of simulated work tasks in which therapists observe performance and make judgments about maximal effort and abilities. While specific assessments have some demonstrated evidence of validity and interrater reliability [4, 27, 28], there can never be complete certainty in the results they provide due to the idiosyncratic nature of human raters leading to limited precision and reliability of assessment results. To overcome this problem, sensors in robotic systems can provide direct numerical measurements that can accurately describe a patient’s performance on a variety of metrics. This could be ideal for supplementing the typical rater judgments during FCE.

The ability of robots to be automated is one of their most important strengths and provides important advantages over existing work simulator devices that are available on the market [29, 30]. The automation of rehabilitation robots provides an opportunity to streamline assessment and therapy to make it more efficient. For example, the ability to time-share a single therapist across multiple patients using robot systems becomes possible. As mentioned, FCE is inherently restricted by distance when conducted in a traditional (i.e., face-to-face) manner. Patients must attend a clinic or rehabilitation center, or a therapist must visit the patient at their workplace or home. In cases where patients are situated in remote or otherwise difficult to access locations, providing services may be exceedingly challenging and cost-inefficient [31, 32]. Telerehabilitation with robotics is a potential solution [33] Telerehabilitation is the concept of providing rehabilitation support, assessment and intervention over a distance using internet-based communication as a medium for therapist-patient interaction [34]. This can take the form of purely audio or video communication, audiovisual communication with patient-robot (unilateral) interaction with performance communicated over the internet, or true telerobotic therapy involving haptic (bilateral) interaction between a therapist side robot and a patient-side robot [35,36,37]. Through telerehabilitation robotics, remote access to patients is a possibility and this opportunity has received significant focus in research [38]. Early indications from longitudinal studies have highlighted that telerehabilitation can lead to modest cost savings despite the up-front cost of necessary technology [39].

An important consideration is that the field of rehabilitation robotics should focus on the use of robots as supplementary to conventional assessments and as enabling tools in the hands of therapists, instead of as replacements for them [40]. Providing semi-autonomy is one solution: semi-autonomy maintains the therapist in charge of the rehabilitation and assessment process while allowing them to save time and effort since the robot or automated system takes a share of the required tasks. Autonomy in robotics implies the existence of machine intelligence, which demands the domain of machine learning research.

Machine Learning in Rehabilitation Robotics

The incorporation of machine learning algorithms in rehabilitation (robotic or conventional) has increased in the past two decades. The vast majority of research focuses on the use of machine learning algorithms for classification, prediction, and treatment planning, but not for learning the actions and interventions demonstrated by a therapist. The use of support vector machines and random forest algorithms for learning and recognizing general human activities was evaluated by Leightley et al.[41]. Li et al. also used support vector machines as well as K-nearest neighbors classifiers to recognize gestures for hand rehabilitation exercises [42]. The use of K-nearest neighbors, logistic regression, and decision trees for identifying upper body posture using a flexible sensor system integrated into the patient’s clothes was assessed by Giorgino et al. [43]. The use of logistic regression, naive Bayes classification, and a decision tree wave have been compared by McLeod et al. for discriminating between functional upper limb movements and those associated with walking [44].

The power of machine learning models is not limited to only classifying movements. They also have the potential to provide predictions of a patient’s work status, which may inform rehabilitation planning. Zhu et al. trained a support vector machine and K-nearest neighbors classifier to predict a patient’s rehabilitation potential, both of which provided better predictive abilities than an assessment protocol currently used in the field [45]. A support vector machine was also used by Yeh et al. to classify balance in able-bodied individuals and those with vestibular dysfunction [46]. Begg et al. also used a support vector machine to classify gait in younger, healthy participants as well as elderly participants [47]. Lastly, a support vector machine was implemented by LeMoyne et al. for classification of normal and hemiplegic ankle movement [48]. In the area of occupational rehabilitation, Gross et al. used Repeated Incremental Pruning to Produce Error Reduction techniques to develop an algorithm for selecting rehabilitation interventions for injured workers [49].

More recent applications of machine learning expand on these works classifying both a patient’s movements and their health condition and attempt to build intelligent rehabilitation systems that can adjust assessment tasks or provided interventions based on features of a patient. Barzilay et al. trained a neural network to adjust an upper limb rehabilitation task’s difficulty based on upper limb kinematics and electromyography (EMG) signals [50]. The use of K-nearest neighbors, neural networks, and discriminant analysis techniques were evaluated by Shirzad et al. for adjusting task difficulty in relation to a patient’s motor performance and physiological features, with neural networks providing the best predictive abilities with a success rate of 78% [51]. Badesa et al. performed a similar evaluation for perceptron learning algorithms, logistic regression, discriminant analysis, support vector machines, neural networks, K-nearest neighbors, and K-center classifiers in which support vector machines were able to best predict a user’s physiological state [52]. A fuzzy logic algorithm was used by Garate et al. to relate a patient’s joint kinematics to the primitive motor outputs of a Central Pattern Generator, which effectively assists during gait through the control of an exoskeleton’s torques [53]. Gui et al. took a similar approach, using electromyographic measurements as the input to a discriminant analysis algorithm that provides assistive exoskeleton trajectories through a Central Pattern Generator [54]. It is important to note that in each of these projects, the adaptation learned by the algorithms was not learned from demonstrations. Rather, these interactions were generated from predetermined models relating patient performance to task difficulty or desired assistance.

Learning from Demonstration for Haptic Interaction

Learning from Demonstration (LfD) describes a family of machine learning techniques in which a robot observes task demonstration by a human operator (the “demonstration” phase) and learns rules to describe the desired task-oriented actions, which then may or may not be acted upon by the robot in a later “reproduction” phase [55]. The synonymous terms “programming by demonstration” or “imitation learning” refer to the same concept. The rules learned through LfD techniques is a central point to its innovation, and has seen implementation through mapping functions (classification and regression) or through system models (reinforcement learning) [56].

Using LfD techniques to program robots provides several advantages. After the initial challenge of making the machine intelligent (i.e., teachable), programming the robot can easily be done by physically holding the robot and moving it through a desired trajectory. This is known as kinesthetic teaching and is advantageous since users do not require knowledge of computer programming. The capabilities of the robot are completely dependent on the level of sophistication of the underlying learning algorithms and the number of sensors used to characterize a behavior. It is possible to teach robots more complex aspects of tasks (e.g., understanding a user’s intent) with highly sophisticated algorithms and sufficient sensors. LfD methodology requires a human user to be involved in the programming process, meaning the aspect of interacting with an actual human is preserved and conveyed by means of imitation. Importantly, like any other implementation of machine learning for robotics, LfD allows for automation and can translate into time and cost savings.

The concept of semi-autonomous systems and LfD has seen extensive research in the past few decades. Application of LfD principles to human-robot interaction has naturally led to the exploration of cooperative tasks such as those required during work activity and assessment. Calinon et al. taught a robot to cooperatively lift a beam [57]. Gribovskaya et al. built upon the same work to ensure complete stability of the robot throughout its entire workspace [58]. A variant to learn motion and compliance during a highly dynamic cooperative sawing task was created by Peternel et al. [59].

Learning Haptic Interactions Provided by a Therapist

Our group has investigated LfD since 2015 as its advantages make it an ideal method for introducing semi-autonomy into the field of rehabilitation robotics. This stems from the ability of LfD to provide a plausible method for therapists with minimum programming experience to customize assessment and rehabilitation regimes. Therapists can easily adjust not only the level of therapeutic assistance or resistance provided to a patient but also set up any number of different assessment or therapy tasks (See Fig. 1). This aspect of mutual adaptation, where users can explore and train robotic aides themselves, is an important step for rehabilitation robotics [60]. It is proposed as a viable method of making robotic-assisted assessment and therapy more cost-effective and personalized.

Fig. 1
figure 1

An example of Learning from Demonstration (LfD) for training a robot to imitate a therapist’s activities. In phases 1 and 2, the therapist provides haptic interaction for the patient when performing a simulated task (in this case opening a door) while the rehabilitation robot observes the intervention through kinesthetic teaching. The LfD algorithm is trained after phase 2. In phases 3 and 4, the robot will imitate the haptic interaction demonstrated by the therapist so as to allow the patient to practice in the absence of the therapist while still receiving haptic guidance. Reproduced with permission from Fong and Tavakoli [72]

Full size image

Few groups have applied LfD-based machine learning techniques towards the practice of occupational rehabilitation, but some research has been conducted in physical therapy more generally. An adaptive logic network was used by Hansen et al. to learn a model relating eloctromyographic signals and the timing of a patient’s activation of an assistive Functional Electrical Stimulation device during gait, which was successfully applied in daily activity over the length of a year [61]. Kostov et al. performed a similar project comparing adaptive logic networks and inductive learning algorithms, but instead related foot pressure recordings with Functional Electrical Stimulation activation timing [62]. Adaptive logic networks were found to have marginally better gait recognition abilities, with the authors concluding that the amount of training data provided matters more than the classification method used. Strazzulla et al. used ridge regression techniques to learn myoelectric prosthetic control during a user’s demonstrations, characterized by EMG signals, and showed that retraining the learned model during performance of a task is intuitive for experienced and naïve users alike [63].

Research that uses LfD to specifically learn and reproduce the haptic interaction provided by a therapist during assessment or interventions represents one branch of the current state of the art in robotic rehabilitation. The merging of these two technologies exploits the hands-on nature of LfD-based robotic systems and addresses some of the shortcomings of robotic rehabilitation as mentioned earlier (i.e., the enabling of cost-savings and ease of programming). Lauretti et al. optimized a system built on dynamic motor primitives for learning therapist-demonstrated paths for activities of daily living, which successfully learned the time-sensitive nature of the tasks [64]. Atashzar et al. proposed a framework for both electomyographic and haptics-based LfD, where the learning of the therapeutic behaviors for an upper limb task was successfully facilitated with a neural network using a computational model of motor disability for a patient who had experienced a stroke [65].

Tao utilized a method based on linear least squares regression to provide a simple estimation of the impedance (i.e., stiffness, damping, and inertia) inherent to a therapist’s intervention during the cooperative performance of upper limb activities of daily living with a patient [66]. Maaref et al. described the use of Gaussian Mixture Model-based LfD as the underlying mechanism for an assist-as-needed paradigm, evaluating the system for providing haptic interaction for assistance in various upper limb activities of daily living [67]. Assistance-as-needed describes the practice of providing patients with enough assistance to complete a task and maintain motivation, but not so much that an insignificant amount of effort is required on their part. Najafi et al. learned the ideal task trajectory and interaction impedance provided by an able-bodied user with a Gaussian Mixture Model and provided user experiment evaluations for an upper limb movement therapy task [68]. Martinez et al. extended the Stable Estimator of Dynamical Systems learning algorithm developed by Khansari-Zadeh and Billard [69]. to learn both motion and force-based therapist interventions [70].

Most recently, Fong et al. applied kinesthetic teaching principles to a robotic system in order to allow it to first learn and then imitate a therapist’s behavior when assisting a patient in a lower limb therapy task [71]. A therapist’s assistance in lifting a patient during treadmill-based gait therapy was statistically encoded by the system using a Gaussian Mixture Model. Later, the therapist’s assistance was imitated by the robot, allowing the patient to continue practicing in the absence of the therapist. Preliminary experiments were performed by inexperienced users who took the role of an assisting therapist with healthy participants playing the role of a patient by wearing an elastic cord to simulate foot drop. The system provided sufficient lifting assistance, but highlighted the importance of learning haptic interactions in the form of the therapist’s impedance as opposed to only their movement trajectories.

We then applied a similar method of kinesthetic teaching for learning the impedance-based haptic interaction provided by a therapist during the intervention in an upper limb activity of daily living [72]. The kinesthetic teaching process proposed that during performance of the task, the interaction forces exerted on the robot end-effector by each of the agents (task environment, patient, therapist) could be simplified as a set of spring forces, linearized about spatial points of the demonstration. An estimate of the impedance-based interaction provided by the therapist could then be obtained by measuring the “performance differential” (i.e., differences in forces along the trajectory), between the patient practicing the task when assisted by the therapist and when attempting the task alone. Experimental validation of the system showed that the interaction impedance was faithfully reproduced, although the resolution of the learnt interaction model briefly produced inaccurate haptic interaction. Similar procedures have also been used to simulate work-related tasks via robotic systems (See Fig. 2) [13].

Fig. 2
figure 2

Robotic simulation of work-related tasks (painting)

Full size image

The Gaussian Mixture Model-based LfD system was also applied to a bilateral telerobotic setup to enable telerobotic rehabilitation for home-based delivery [73]. A Gaussian Mixture Model and GMR-based approach to LfD was implemented with the purpose of learning therapeutic interactions in a collaborative activity of daily living (which simulated lifting weighted crates), where the intervention was dependent on the patient’s upper limb position and velocity. By training the Gaussian Mixture Model with patient performance (represented by their limb velocity) as a model input, the LfD algorithm inherently learned the adaptive nature of the therapist’s intervention with respect to a patient’s level of ability.

Lastly, we compared the single robot and telerobotic modalities previously implemented (referred to as Robot- and Telerobotic-Mediated Kinesthetic Teaching) for implementing LfD in robotic rehabilitation (See Fig. 3) [74]. The study provided incentive for rehabilitation-oriented systems to pursue Robot-Mediated Kinesthetic Teaching designs, as the demonstrations provided through that modality were found to be more consistent.

Fig. 3
figure 3

Illustrations of the telerehabilitation system with LfD proposed by our group. The demonstration phase is shown in a where the patient interacts with the therapist via a distant robot, and the reproduction phase in b where the patient interacts with a robot that emulates the therapist’s behavior

Full size image

Future Directions for LfD-Enhanced Rehabilitation

Despite their advantages, integrating robotic rehabilitation into FCE and occupational rehabilitation is in the early stages and faces several limitations. First and foremost is that analyses of the efficacy of robotic rehabilitation are largely inconclusive as to whether robotic rehabilitation will be as effective as “conventional” assessment processes. Research in this area is in its infancy and when put in context with the high initial costs of purchasing robots, acceptance of robotic assessment and rehabilitation remains relatively low in clinical settings. The programming of rehabilitation robots has always been done such that the robots provide interactions associated with a specific set of tasks, with no easy method of changing these tasks. As a result, the kinds of interactions a therapist can provide through the robotic medium are limited unless the therapist or a technician are familiar with computer programming principles and can change the task and/or task-oriented behavior of the robot. Low patient motivation remains an issue even with the addition of robotics. As robots allow for reduced therapist interaction, the patients themselves may lose motivation due to the lack of encouragement, entertainment, and human interaction [75, 76].

Despite these limitations, more research is needed to evaluate the use of machine learning and rehabilitation robotics in the area of FCE and occupational rehabilitation. The incorporation of machine learning techniques is still relatively new in the field of rehabilitation robotics. A wide range of learning algorithms is present in the literature, but none of these are a definitive best option. A possible future direction would be to explore and compare LfD algorithms so as to create guidelines that are optimal for FCE assessment tasks and the human-robot interaction learning for assessment. Algorithms that generate global models from demonstrations (i.e., that cover the entire task workspace) may represent a good starting point. In these models, desired haptic interactions would be defined for all patient behaviors, which is desirable for safety and ease of programming. This could be performed through simple methods such as surface fitting, but could also be extended to explore more advanced concepts such as fitting Riemannian manifolds [53], or the SEDS algorithm [70].

A common limitation of the majority of the technologies that have been presented in this paper is that they present proof-of-concept systems or have not been validated for patient-safe interaction. It is crucial to validate the systems by conducting longitudinal studies on actual patients with work disability. Systems incorporating the proposed technologies should be compared with traditional FCE by analyzing the outcomes of patient satisfaction and return to work in order to determine their effectiveness against current methods. Emphasis should also be placed on recruiting large sample sizes, as the majority of rehabilitation robotics studies to date have been done with relatively small samples.

Conclusions

The ultimate goal of this research area is to improve the assessment and rehabilitation of injured workers by providing methods for easily simulating workplace tasks using intelligent robotic systems. Such a system would provide a single, streamlined solution for both FCE assessment and rehabilitation. The system would be able to simulate reaching maneuvers as well as more dexterous functional tasks that are typical of workplace activities. Utilizing machine learning approaches, the benefits of robotic systems could be combined with the expertise and experience of human therapists. While the research is in the early stages, it has great potential for overcoming several limitations of traditional FCE practice.