Keywords

As research continues to support prolonged ­practice of functionally relevant activities for restoration of function, interactions with technology have revealed new prospects in the areas of motor teaching. The compelling question many researchers are currently pursuing is whether such new applications of technology can go further than simply giving a higher intensity or more prolonged care. This chapter will focus on how robotic devices combined with computer displays can augment error in order to speed up, enhance, or trigger motor relearning. Below, we outline the sources of this rationale, as well as present some early examples.

1 Experience Enables Prediction of Consequences

While neurorehabilitation science is still in early stages with numerous debates, nearly all agree that a key mode of recovery is the nervous system’s natural capacity to change in response to experience – neuroplasticity of neural control. Although for brain injuries such as stroke, there are many deficits that may not be related (contractures, weakness, cognitive deficits, attentional deficits, etc.), neuroplasticity is believed to be one of the most powerful and can be leveraged to foster functional recovery through the proper conditions of training, feedback, encouragement, motivation, and time.

Early exploration of training-induced neuroplasticity is hinged on studies of sensorimotor adaptation in healthy individuals. Tasks such as reaching for a cup are thought to be trivial but extremely difficult and frustrating to patients. We often take for granted the challenges of coupled nonlinear arm dynamics [1], long feedback delays [2], and slow activation times for muscle [3], must rely on sophisticated control by the nervous system. Consequently, rapid movements must be preplanned using a prediction or “neural representation” of the outcomes. These representations, also called internal models, are typically acquired via experience [4]. Research has shown that distorting sensory-motor relationships in a variety of ways can alter these representations. For example, mechanical distortions such as holding a heavy weight in one’s hand causes errors in reaching accuracy, but people adapt and recover their ability to move normally within a single motion [5]. More complex loads can take hundreds of movements [6–8]. People often stiffen (i.e., co-contract their muscles) as a first strategy [9, 10], but stiffness quickly fades as they learn to counteract the forces, leading to aftereffects when forces are unexpectedly removed (Fig. 5.1) [11, 12]. It is important to note that both the adaptation and aftereffects can occur implicitly with minimal conscious attention to any goal. We have shown that this type of training can be used constructively to teach new movements [13, 14].

Fig. 5.1
figure 1

A classic adaptation experiment in which a robot exerts a mechanical distortion. The subject attempted reaching movements to targets in eight different directions. (a) Subject seated at the robot, (b) initial exposure to the force field, (c) at the end of training, movements appear normal. (d) Removing the force field unexpectedly results in aftereffects (Adapted from Shadmehr and Mussa-Ivaldi [8])

Full size image

Motor learning is strongly driven to reduce performance errors [15, 16] and, in particular, deviations from a straight-line hand path in ­targeted reaching [17, 18]. Experiments have demonstrated that it is possible to train subjects to produce new arm movements [19, 20] or legs [21] by accentuating trajectory errors using robotic forces. Subjects in those studies were exposed to custom-designed force fields that promoted the learning of specific movements by exploiting short-term adaptive processes [22].

1.1 The Nervous System Responds Dramatically to Visual and Mechanical Distortions

Similar adaptation can occur when exposed to a visuomotor distortion. The robotic approaches above can be grouped with an older body of research on visuomotor adaptations, such as those induced by prisms (see [23] for a review), rotations, stretches, and other distortions of the conventional hand-to-screen mapping [17, 24, 25]. All of these distortions appear to induce learning and can reduce sensory dysfunction such as hemispatial neglect [26].

1.2 Neuroplasticity, Learning, Adaptation, and Recovery

Such adaptation described above, however, might not necessarily reflect long-term learning. There is strong evidence that when a person experiences more than one training experience, the latter experience tends to disrupt or interfere with the former [27–29]. One key premise of robot-mediated training is that adaptation will be retained if the resulting behaviors have functional utility. Our studies and the work of others have demonstrated permanent effects after training in the presence of visuomotor distortions [27, 30, 31]. Hence, individuals de-adapt if conditions require it, but also some motor memory is preserved well beyond the training phase. Here, we use the term “learning,” since our ultimate goal is permanence. Further work is needed to understand what neural processes mediate the successful evolution between adaptation and long-term retention, and it may be that the two share many common neural resources, with a continuum between short and long-term neuroplasticity.

Quite importantly, these adaptive responses can also be observed in stroke patients. Evidence is found in the oculomotor [32] and limb motor systems [20, 33, 34]. In fact, errors seen in individuals who have suffered a stroke are similar to simulation models that try to imitate the pathology with poor compensation for interaction torques [35] and resemble the problems seen in healthy subjects when they are exposed to force fields. At least part of the impairment has been attributed to “learned nonuse” that can be reversed by encouraging individuals to practice and relearn how to move their arm [36].

2 Multiple Forms of Neuroplasticity

Plasticity comes in many forms across many time scales making it difficult to fully identify all underlying mechanisms. Changes can range from very temporary shifts in neurotransmitter concentrations, facilitation or inhibition from collateral neurons, neural growth to establish synapses, or to actual neurogenesis where entire neurons are established. Making this more complicated, neuroplasticity can be seen as residing within a much larger spectrum of mechanisms with overlapping time scales that span short-term ­adaptation in milliseconds, long-term potentiation over ­minutes, permanent leaning, muscle hypertrophy, healing, or degeneration of whole tissue structures through development and aging. Finally, there are also aspects of the nervous system’s control apparatus that can be seen as hierarchical agents, where people learn to learn, and learn to make decisions to learn. There are many ways in which the nervous system alters its behavior in response to new experiences, and many of these mechanisms are driven by error (Fig. 5.2).

Fig. 5.2
figure 2

A schematic flowchart that illustrates the believed error-mediated adaptation for the control of movement. News of outcome movements is fed back to the central nervous system to calculate errors, e, that is used for adjusting (adapting). Several known mechanisms exist that use error (red lines) to make alterations, such as recalibration of the proprioceptive system, alterations in preplanned inverse dynamics, impedance, and the intended trajectory

Full size image

There has been recent debate over whether the neural resources used are the same for adaptation to kinetic and kinematic distortions. Krakauer et al. [28] suggested that learning of kinematic distortions (a 30° rotation of visual display) and kinetic distortions (distortions of added mass) were independent processes because learning one did not interfere with the other. It would appear that these are separate processes (different red lines of Fig. 5.2). Flanagan and colleagues also showed similar results with a visuomotor rotation and a viscous force field [37]. However, Tong and colleagues argued that these studies should not show interference because the kinetic and kinematic distortions involved different variables, and the kinematic rotation depended on position while the kinetic mass depended on acceleration [29]. They demonstrated that when both the force field and the visuomotor rotation depended on position (or on acceleration), interference was observed. These results strongly suggest that kinetic and kinematic adaptation occupy common neural resources in motor-working memory. One can take this one step further to test and facilitate rather than interfere, whereby experiencing a mix of force and visual feedback distortions can enhance learning even further [38].

3 The Crutch Effect

What is clear is that human–machine interactions have the extremely powerful ability to foster learning, but it is not clear precisely how to program them for therapeutic benefit. One possibility would be to have a system that guides one’s actions to help one learn. This enables the patient to visit the positions and velocities of a task, being “shown the way” as a template. This template may offer the added benefits of keeping the joint mobile through the range of motion and preventing secondary effects such as contractures from immobility. While this may be an answer for people entirely paralyzed, this provides the correct kinematics without the correct kinetics. While there have been a few studies that have shown a benefit for haptic guidance in learning motions [39–41], it may be that such interaction forces do not ensure that the limb makes the correct motion. In one study on healthy people, simply watching the robot make a template motion caused subjects to learn about as well as the people that practiced with robotic guidance [42].

One problem may be that such guidance algorithms generate unnatural forces unless individuals actively make the desired motion, which renders the guiding robot unnecessary. Guidance interactions are not only unnatural; they may encourage unwanted resistance, promote laziness, or reduce the subject to inattention. This can remove any desire to learn and lead the individual to simply rely on guidance like one might rely on a crutch. People could literally fall asleep practicing.

4 Guidance Versus Anti-guidance

The opposite line of attack – systematically altering the movement to enhance error – may be one possible answer. In an early study of error augmentation, our group focused on the chronic stroke population and compared error-magnifying forces to error-reducing forces in a short therapy session. We exposed hemiparetic stroke survivors and healthy age-matched controls to a pattern of disturbing forces that has been found by previous studies to induce dramatic aftereffects in healthy individuals. Eighteen stroke survivors made 834 movements on a manipulandum robot in the presence of a robot-generated force field. The force field pushed proportional to hand speed, perpendicular to movement direction – either clockwise or counterclockwise (Fig. 5.3a–c). We found significant aftereffects from the stroke-surviving participants, indicating the presence of a reserve capacity for neuroplasticity in these patients that has very little or nothing to do with stroke severity [20]. Significant improvements occurred only when the training forces magnified the original errors and not when the training forces reduced the errors, or when the there were no forces (Fig. 5.3d). Such adaptive capacity in stroke survivors is also supported by evidence that the nervous system is able to reorganize with practice [43]. These results point to a unifying concept: errors induce motor learning, and judicious manipulation of error can lead to lasting desired changes.

Fig. 5.3
figure 3

(a) One stroke survivor’s response to training forces that amplify the original counterclockwise movement error. The force field during training (arrows in b) resulted in a reduction of error following training that was sustained until the end of the experiment (c). (d) Cross plot of all subjects’ final performance improvements vs the amount of error magnification/reduction in training. Error magnification was determined by calculating the dot product between the average training force direction and the average movement error direction. Performance improvement was calculated by measuring the reduction of initial direction error from the baseline phase to the final phase of the experiment. Boxes represent mean and 95% confidence intervals, and whiskers indicate two standard deviations (Adapted From Patton et al. [20]; used with permission)

Full size image

5 Error Augmentation for Leveraging Neuroplasticity

The great enlightenment philosopher George Berkeley pioneered the idea “Esse est percipi” (to be is to be perceived). Rather than using immersive environments for mere entertainment, technology has recently allowed us to constructively alter behavior through new perceptual distortions, essentially creating a “lie” to the interacting subject in a variety of ways. This is a bright prospect, not only in the world of engineering for rehabilitation but also in many areas in which people must learn to make new actions. One aspect is error augmentation, where we isolate and selectively enhance the perceived error.

There are several lines of support for error augmentation approaches for enhancing learning. Simulation models and artificial learning systems can show that learning can be enhanced when feedback error is larger [22, 44–46]. Subjects learning how to counteract a force disturbance in a walking study increased their rate of learning by approximately 26% when a disturbance was transiently amplified [21]. In another study, artificially giving smaller feedback on force production has caused subjects to apply larger forces to compensate [47]. Several studies have shown how the nervous system can be “tricked” by giving altered sensory feedback [17, 48–53]. Conversely, suppression of visual feedback has slowed the unlearning process [14]. It is clear that feedback that provides an error signal can influence learning and that the truth can be stretched for greater effect.

Nevertheless, not all kinds of augmented feedback on practice conditions have proven to be therapeutically beneficial in stroke [54]. It may be that there are limits to the amount of error augmentation that is useful [55, 56]. More error might mean more learning, but it would not seem logical for error augmentation to work in a limitless fashion.

6 Choices: Does More Error Mean More Learning?

The optimal method for error augmentation is not yet known and may depend on a number of contexts. We conducted a simple evaluation of the rate change of hand-path error while subjects made point-to-point reaching movements of the unseen arm [57]. Error deviations from a straight-line trajectory were visually augmented with either a magnification of 2, a magnification of 3.1, or by an offset angular deviation. The smaller time constants (fitting performance changes to an exponential curve) for the *2 and offset groups demonstrated that error augmentation could increase the rate of learning (Fig. 5.4). However the *3.1 group showed no benefit. This result was observed in a similar study where there was diminishing effectiveness from larger errors, causing smaller changes from one movement to the next [58].

Fig. 5.4
figure 4

Time constant of error decay during a visual error augmentation trial on healthy subjects, revealing a breakdown in higher gain of error augmentation 3.1. Error bars indicate 95% confidence intervals. Horizontal lines indicate significant differences (post hoc) between groups

Full size image

The offset group above represents another type of error augmentation via the addition of constant error offset. This is in contrast to error magnification, where learning could become unstable if it causes the subject to overcompensate. Because of motor variability, sensor inaccuracies, and other uncertainties that influence learning [49, 56, 59], error magnification may be practicably limited to small gains. On the other hand, adding a constant bias to augment error may be equally or more effective because noise and other confounding factors would not also be magnified. A constant offset presents persistent errors throughout training, even as the learner improves. This technique may motivate learning longer during practice and hence cause the amount of learning to increase. However, each approach (biasing or magnifying) has their benefits and potential pitfalls: gain augmentation is vulnerable to feedback instability, whereas the biasing approach risks learning beyond the goal.

There are a variety of compelling aspects of error augmentation that arise from the fact that we often evaluate and adjust our control based on the error of previous movements rather than the current one – we learn to walk by repeatedly falling down and trying again. Such postmovement evaluations imply that we often are able to gain insights into the nature of the learning process from one attempt to the next. We can also more easily use what is known about how someone responds to prior environmental changes to customize a training environment for the subject. Such co-learning is a compelling new prospect in many areas that include rehabilitation, where the machine encouraging the patient to adapt is itself adapting as learning progresses.

7 Free Exploration and Destabilizing Forces

Beyond manipulation of force and trajectory signals, the concept of error augmentation can be further extended to training environments that amplify motor actions. Instead of error with respect to a specified movement, robot-guided training can exaggerate movements in real time, effectively augmenting the dynamic behavior of the arm. Robot assistance can certainly expand human capabilities through assistance as a function of applied forces or speed [60, 61]. Such approaches use active impedance such as ­negative damping. Beyond altering online performance, such augmentations can increase awareness of deviations from expected behavior – information critical for driving adaptation. Furthermore, a major advantage to this form of augmentation is allowing access to coordination training even when weakness limits voluntary motion. Most importantly, however, such augmented environments must both facilitate training and still allow easy transition to unassisted conditions.

To test this form of environment ­augmentation, we investigated the efficacy of manual skill training with destabilizing forces, presented by a robotic interface. One key feature of our approach was to allow self-directed movement during training. While goal-directed movement focuses on kinematic performance, we expected that allowing training via exploratory movements would emphasize relevant force and motion relationships. Training on a variety of actions provides better improvement in overall function than repetitions of the same task [62, 63]. The free training paradigm also served as an excellent measure of learning generalization, since the structured evaluations after training (making circles) differed from the practice.

We found that improvements in performance persisted even when destabilizing forces were removed, and that training with combined negative viscosity and inertia resulted in superior learning when tested in the isolated inertial conditions [64]. In a follow-up study with stroke survivors (Fig. 5.5), similar training with negative viscosity resulted in improved coordination skill within a training session, while no improvement was observed in the control group where no forces were administered. It is important to emphasize that each group was evaluated in the absence of applied forces, which demonstrates that patients’ training with negative viscosity does transfer to positive skills in the real world.

Fig. 5.5
figure 5

Patients benefit from free exploration training with robot-applied negative viscosity to augment error. (a) The robot interfaced to the arm about a free pivot at the wrist. Subjects were allowed to freely interact with each load in a “motor exploration” stage. Following exploration, subjects made counterclockwise circular movements during task performance trials at random starting locations of a 0.1-m radius circular track. (b) The virtual arm augmented the existing dynamics of the human arm with negative viscosity in the elbow and shoulder and/or positive inertia to the upper and forearm. (c) Stroke survivors (n  =  10) perform motor exploration with no load, revealing average baseline distribution with evident asymmetry in range. Negative viscosity training prompted significant increases (indicated as x’s) especially in elbow flexion-extension. (d) Tests of learning show error decreased (−19.1  ±  0.1%, p  =  1.3e − 2) from negative viscosity training, while no change was found from inertia  +  negative viscosity training (+5.1  ±  16.2%, p  =  4.3e − 1)

Full size image

8 Making Error Augmentation Therapy Functionally Relevant

When a robotic device is coupled with a three-dimensional graphic display, the sensorimotor system is able to engage all the types of visual and motor learning described above [65, 66]. The haptic actuator is typically a specially designed robot to allow the user to easily move ­(back-drive) and may also exert forces that render the sense of touch. The augmented reality graphic display presents images in stereo, in first person, and using head tracking to appropriately correspond to the current eye location (Fig. 5.6). Images can be superimposed on the real world.

Fig. 5.6
figure 6

A subject seated at a large workspace haptic/graphic display

Full size image

These haptic and graphic virtual environments offer several advantages. First, properties of objects can be changed in an instant with no setup and breakdown time. This element of surprise is critical for studying how the sensorimotor system reacts and learns to move in new situations. For rehabilitation, friction or mass can be suppressed, or mass can be reduced during the early stages of recovery.

A few studies have explored such virtual reality for rehabilitation [67–75] although many other studies on virtual reality applications for rehabilitation fail to effectively test how this technology can offer added benefit in clinically facilitating motor recovery. One concern is whether any training benefits are retained. Evidence from studies of healthy individuals shows little retention beyond the time that adaptation typically “washes out.” Such findings, taken in isolation, would suggest reasons not to treat with error augmentation. Recent work, however, reflects a more careful approach to understanding retention and, more importantly, the accumulation of benefit from repeated visits [76].

In this recent study, stroke survivors with chronic hemiparesis simultaneously employed the trio of patient, the therapist, and machine. Error augmentation treatment, where haptic (robotic forces) and graphic (visual display) distortions are used to enhance the feedback of error, was compared to comparable practice without such a treatment. The 6-week randomized crossover design involved approximately 60 min of daily treatment three times per week for 2 weeks, followed by 1 week of rest, then another 2 weeks of the other treatment. A therapist teleoperated the patient using a tracking device that moved a cursor in front of the patient, who was instructed to match it with their hand’s cursor (Fig. 5.7a). Error augmentation, using both haptic (F  =  100[N/m] • e) and visual (x  =  1.5 • e) exaggeration of instantaneous error, was employed for one of the 2-week periods without being disclosed explicitly to anyone (thus blinding the patient, therapist, technician-operator, and rater). Several clinical measures gauged outcome at the beginning and end of each 2-week epoch and 1 week post training. Results showed incremental benefit across most but not all days, abrupt gains in performance (Fig. 5.7b), and most importantly, a significant increase in benefit to error augmentation training in final evaluations. This application of interactive technology may be a compelling new method for enhancing a therapist’s productivity in stroke functional restoration.

Fig. 5.7
figure 7

(a) An error augmentation application for stroke rehabilitation where a subject and therapist work together, seated and using the large workspace haptic/graphic display to practice movement. The therapist provides a cue for the subject, and can tailor conditioning to the needs of the patient. The robot provides forces that push the limb away from the target, and the visual feedback system enhances the error of the cursor. (b) Typical chronic stroke patient improvement from day to day, each dot representing the median error measured for a 2-min block of stereotypical functional movements. While the patient shows progress across the 2-week period and final benefit, this person did not always improve each day

Full size image

9 Why Might Error Augmentation Work?

While there are several mechanisms for how error augmentation might work, a full understanding of the sources is not known. One possible mechanism is that elevating error simply motivates ­subjects to persistently try to reduce error until they see an acceptably small (perhaps zero) error. A number of modeling and experimental systems have demonstrated better and faster learning if error is larger [15, 44, 77, 78]. Error bias, such as in the offset condition mentioned above, can lead a subject to “overlearn” beyond the desired goal, but this technique may be otherwise beneficial in situations where subjects do not fully learn. Based on our findings, we speculate that mixtures of force and visual distortions, combined with offset-based and gain-based error augmentation, might be optimal. However, optimal parameters governing such a mixture are not yet known and are likely to differ from patient to patient.

Another possible reason why error augmentation may lead to benefits is that the impaired nervous system is not as sensitive to error and hence does not react to small errors. Error augmentation might make errors noticeable by raising signal-to-noise ratios in sensory feedback. It may heighten motivation, attention, or anxiety, which has been suggested to correlate with learning [79]. Errors that are more noticeable may trigger responses that would otherwise remain dormant.

Error perception appears to be on a continuum that is not yet understood. Error reduction appears to stifle learning [80], and suppression of visual feedback has been shown to slow down the de-adaptive process [14]. This suggests that less perceived error could reduce learning. Considering the other extreme, too much error augmentation appears to dampen results, thus suggesting that there is a sweet spot of error augmentation intensities. The nervous system may react to excessively large error signals by decreasing learning so that there is little change in response to subsequent performance errors. Large errors thus may be regarded as outliers by a nonlinear “loss function” that governs motor adaptation [56]. These and other studies that induce sensorimotor conflict suggest that the nervous system can quickly “adapt its adaptation” by reweighing the interpretation of sensory information if it no longer is perceived reliable [49, 81].

Regardless of the mechanism, the bioengineering community is now observing successes with error augmentation, and the clinical research world calls for more studies on its optimal ­application. These new studies should also reveal new insights on how the nervous system learns and recovers after injury. There is a clear advantage to such distorted reality feedback, where judicious manipulations of visual information can lead to practical improvements in the extent and rate of learning. Research also suggests that these training approaches may be broadly effective in facilitating motor learning in sports, piloting, performing arts, teleoperation, or in any other training situation requiring repetitive practice and feedback.