Smart and Assistive Walker – ASBGo: Rehabilitation Robotics: A Smart–Walker to Assist Ataxic Patients

Abstract

Locomotion is an important human faculty that affects an individual’s life, bringing not only physical and psychosocial implications but also heavy social-economic consequences. Thus, it becomes paramount to find means (augmentative/assistive devices) to empower the user’s residual capacities and promote functional recovery.

In this context, a smart walker (SW) is explored for further clinical evaluation of ataxic patients during walker-assisted and to serve as a functional compensation and assist-as-needed personalized/customized rehabilitation tool, autonomously adapting assistance to the users’ needs, through innovative combination of real-time multimodal sensory information from SW built-in sensors. To meet the users’ needs, its design was weighed, considering to whom it is intended.

Thereby, this paper presents the system overview, focusing on design considerations, mechanical structure (frame and main components), electronic and mechatronic components, followed by its functionalities. Lastly, it presents results regarding the main functionalities, addressing clinical evidence.

2.1 Introduction

Locomotion/mobility plays an important role in daily living activities of individuals [1]. However, different types of pathologies, such as poliomyelitis, degenerative joint diseases, spinal cord injuries, multiple sclerosis, traumas, musculoskeletal deformities and ataxia, greatly impair human mobility at different levels provoking partial or complete loss of this faculty, consequently compromising performance of daily tasks with ease [1, 2].

Mobility restrictions induce not only profound physical and psychosocial implications but also heavy social economic consequences, which further jeopardize quality of life [2,3,4,5]: early retirement, increasingly third-party and informal caregiver’s dependence costs and in more serious cases, palliative care or institutionalization. Ataxia is one of the most startling mobility disorders, induced by focal neurological deficits in the cerebellum and afferent proprioceptive pathways, responsible for ensuring a proper voluntary movement coordination and control [6]. World Health Organization reports an annual global prevalence of 1,3–20,2 cases per 100.000 population, and in Spain is estimated an annual health cost per patient with spinocerebellar ataxia of around 18.776€ [7]. Additionally, lesions in the cerebellum can result in postural sway and backward balance reactions, being those subjects extremely prone to falls and fall-related injuries, which greatly jeopardizes their independence and exacerbates social-economic consequences [8]. In fact, 73.6% and 74% of spinocerebellar ataxic patients involved in a 1-year study reported falls occurrences and fall-related injuries, respectively [9].

There is neither a pharmacological or surgical solution which can fully reverse motor disturbances, being Physical Medicine and Rehabilitation (PMR) interventions key to promote functional recovery in ataxic patients [6, 10, 11]. Intensive coordinative physiotherapeutic training is supreme by harnessing functional residual capacities though stimulating of motor learning capabilities of the cerebellum [12]. Rehabilitation programmes found in current literature including balance, coordination, postural reactions training, occupational therapy and hydrotherapy have shown good results in terms of muscle strengthening, physical resilience increase, postural stability and motor coordination [11]. Recently, techniques such as trans-cranial magnetic stimulation, virtual reality, biofeedback (e.g. PhysioSensing platform), treadmill exercises with supported bodyweight and torso weighting have also shown to have potential [11].

Thus, researchers have been addressing the need of therapeutic alternatives including the selection and prescription of assistive/augmentative devices to provide adequate functional compensation and recovery [5]. Augmentative devices aim to empower the user’s natural means of locomotion, taking advantage of the remaining motor capabilities. In this sense, emphasis has been placed on conventional walkers, as a promising solution for ambulatory rehabilitation, once it supports bipedalism and dynamic gait stability. Conventional walkers promote an increase of the base of support and a decrease in the weight-bearing on the lower limbs [13, 14], relegating to secondary alternative devices, including wheelchairs or canes/crutches, strongly discouraged for their limited lower limbs weight support and lateral stability (base of support) [13]. Negative experiences (e.g. falls) arising from canes/crutches use, often cause patients to adopt a more sedentary life, not walking as much as recommended, which progressively deteriorate their motor condition and promote the onset of other diseases (e,g. diabetes mellitus).

Conventional walkers mechanical structure promotes simultaneously, muscle strengthening and physical resilience increase, considering it reliefs weight bearing on the lower limbs and compensates the decrepitude of postural stability of ataxic patients as a result of a wider base of support [13]. Conventional walkers are classified accordingly to the type of support with the ground: (1) standard, with rubber tips, which must be lifted off the ground and place it forward while walking, (2) front-wheeled walker for patients who have difficulty lifting a standard walker, and (3) four-wheeled walker that promote the most natural gait patterns and can be used if the patient does not rely on the walker to bear weight [5, 15]. Nevertheless, prescription of a walker to patients implies taking into account not only their locomotion deficits, but also cognitive or sensory impairments [16]. This way, deficits related to dexterity, cognitive ability, motor coordination and maneuverability of conventional walkers still provoke destabilization of biomechanical forces leading to a lack of balance and potential falls [13]. As a result, many patients prefer not to use any walking aid, progressively deteriorating their motor condition and social-economic consequences [16]. In this sense, to solve this paradigm, incorporation of robotic technologies has started to emerge, in order to achieve a multifunctional powertrain smart walker, which promotes a better service, by combining safety, ease of use and low cognitive requirements and allowing patients to fully concentrate on their gait rehabilitation.

Recently, several multifunctional powertrain smart walkers have been developed. Even so, the majority of those do not assemble a set of comprehensive functionalities in a single smart walker, promoting a generic functional compensation and recovery tool [17]. Further, they often promote early onset of fatigue, a higher cognitive workload and often result in destabilization of biomechanical forces, considering the majority of them deprive the user of an assisted-as-needed experience, tailored to each user’s needs and motion intentions towards assisted living environments [18]. This chapter describes a new PMR intervention with a smart walker – ASBGo∗ (Smart Walker for Mobility Assistance and monitoring System Aid) , towards efficient rehabilitation by promoting ambulatory daily exercises enhancing the quality of life of people in general, and particularly, cerebellar ataxic patients. The prototype includes distinct functionalities to be adjustable to different users’ needs. Among those, physical support, autonomous guidance by interpreting the surrounding environment, manual guidance through an advanced human-machine interface designed to extract users’ intentions to modulate the SW behavior accordingly [19] and lastly, user’s state monitoring to track the patients motor condition through a set of rehabilitation sessions.

Thus, they deprive the user of an assisted-as-needed experience, demand high cognitive effort, promote early onset of fatigue and often provoke destabilization of biomechanical forces leading to falls [5].

In the following sections, a brief literature review of SWs will be presented. In Sect. 2.3 an overview of the ASBGo∗ prototype is presented, extending from its mechanical and electrical features, to its design and ergonomics considerations. Next, Sect. 2.4 presents the results obtained from the ASBGo∗‘s functionalities validations, culminating with the presentation of hospital trials results and its discussion for each functionality proposed. Lastly, Sect. 2.5 presents the major conclusions withdrawn.

2.2 Related Work

Bateni et al. reports that most conventional walkers are neither tailored or recommended for individuals with mobility dysfunctions plus visual and cognitive impairments, as well for patients suffering from balance issues, including ataxic patients [20]. The emergence of SW seeks to fill the gaps of conventional walkers, but its design presents unique challenges to researchers in this area, related to specific demands of people with impaired mobility and balance. Up to the authors knowledge, current smart walker-based systems have not yet addressed the required adaptation of the device to the patient’s disorder [21, 22]. To successfully outline/design a walker, it is paramount to look beyond motor disabilities and take into consideration users’ cognitive or sensory impairments as well. Additionally, usability issues including safety, comfort, simplicity of use and low cognitive demand must not be discarded, in order to attain a patient-oriented system design [16].

Thus, SW can be classified accordingly to their main functionalities: (1) physical support, (2) manual guidance, (3) autonomous and shared-control guidance (4) security and status monitoring of the user(s).

2.2.1 Physical Support

Physical support functionality on a SW addresses mechanical enhancements to the standard four-wheeled walker, in order to ensure dynamic and static stability. Its mechanical architecture must be adaptable to different needs and requirements. For instance, Simbiosis, i-Walker, Yoshihiro et. al and JARoW (JAIST Active Robotic Walker) devices provide physical support by offering support assistance while ambulating, through the implementation of forearm support platform, contrarily to conventional handlebars [23, 24]. Forearm supports are also recommended for patients with hand and/or wrist pathology [25]. As a result, muscle fatigue and weight bearing on the lower extremity joints (e.g. knee, ankle, or/and hip joints) are severely diminished [15, 4, 26]. In addition, forearm supports provide an increased degree of support and stability, although they do not allow correct centralization and verticalization of the user’s trunk once there is no height adjustable mechanism. Adjustment of the devices’ height and width must be taken into consideration. As an example, Ye et al. proposed a SW with a width-adjustable base mechanism, based on an electric cylinder to stretch out or contract the rods in the presence of spacious environments narrower spaces, respectively [27]. On the other hand, i-go walker presents an handle’s height adjustable mechanism [28]. Sit-to-stand support is equally essential in a walker. As is the case with Chugo Takese et al. mobile walker and Walkmate [29,30,29].

ASBGo∗ seeks to fill a gap in the literature by including a wooden table with a curvature in the removable trunk contact area, which allows a postural control, increased stability and decreased tremors and dysmetria in patients with ataxia. The ASBGo walker also includes a similar system of [29,30,29] through an electric lifting columns and the support provided by the wooden table, allowing assistance to patients in the transition from sitting to standing.

2.2.2 Manual Guidance

Other research concern relates to manual guidance, being vital a user-friendly and low cognitive effort human-machine interface, adaptable to different users’ levels of physical and mental abilities [32, 33], in order to promote an adequate maneuverability. These interfaces must be able to anticipate the users command intentions to drive the device accordingly and effectively contribute to the user rehabilitation training. Once, SW guidance is controlled by the user without retrieving any feedback from the controller to aid in decision-making, this guidance mode seeks to provide merely power assistance for patients that are cognitively able to make command decisions but lack proper control over their walking velocity and gait pattern. This mode is only recommended for patients without cognitive or visual limitations and, simultaneously, motor and strength coordination to manipulate the handlebar.

To this end, some sensory modalities are frequently implemented to establish human-machine interfaces by relying on physical interaction [34]. Several models of robotic walkers, including Morris et al. [21], GUIDO [24], Walkmate [31], RMP (Robotic Mobility Platform) [35], explored force sensors accoupled with handlebars of the device, being users’ intentions transmitted through physical interaction with ease. Force signals are converted into guidance commands through filtering and classification strategies [10]. Simbiosis [36] also explored 3D upper-body force interaction concept through force sensors installed under the forearm supporting supports [36]. Forearm supports promote a better posture and stability during gait [37]. Additionally, i-go Walker and Ye et al. also exploits upper-body force interaction for manual control [28]. More recently, Huang et al. exploited Lasso model and PCA algorithms to stablish a nexus between the measured forces by piezoresistive force sensors and user intentions and consequently explored a fuzzy-neural network controller to predict the SW motion in accordance. An alternative low-cost approach is based on a joystick mechanically coupled with a spring in the SW handlebars, whose movement is in agreement with users’ manipulation. Nonetheless, interfaces including joysticks integrated into handlebars present an unreliable behavior (delay and hysteresis) due to walking vibration. Switches, buttons and touch screens have also been explored, but all of these sensory interface modalities translate into discrete and unnatural movements [22, 33, 34, 37] and, particularly, demand a high mental workload and may cause confusion. Recently, interfaces using voice communication have emerged for visually impaired users, as a bilateral communication tool for transferring effective high-level commands [33]. Yet, verbal instructions are not optimal, considering voice recognition variability and restriction in the presence of noisy environments. Considering the concerns still posing regarding this topic, interfaces independent of physical interaction, including ultrasonic sensors and laser sensor sought to actively control the walker motion based on gait pattern recognition, which imposes a low mental workload [37]. Yet, evidences involving generation of an adequate motion based on pathological gait pattern is still scarce.

Similarly, ASBGo∗ integrates manual guidance functionality, but it seeks to accomplish a manual guidance approach adequate for ataxic individuals, through an intuitive, user-friendly and easy to handle handlebar, capable of transmitting the intentions of the user.

2.2.3 Autonomous & Shared-Control Guidance

Deficits in spatial orientation and wayfinding within unfamiliar and familiar environments are known to deteriorate with age [38]. In this sense, incorporating a smart spatial orientation functionality in the SW may be highly beneficial to promote mobility in individuals suffering from cognitive disturbance, sensory degradation and loss of memory that are associated with neuro-degenerative diseases such as Alzheimer or Parkinson [10, 38]. In addition, from a rehabilitation point of view, an autonomous guidance functionality is paramount for SW to provide safe navigational decisions and collision avoidance [39], consequently, allowing the patient to fully concentrate on their gait pattern and mobility. This feature is also known as sensorial and cognitive assistance, once this type of interface does not require the user to produce any specific command to produce walker motion, being particularly, suitable for individuals with visual or cognitive dysfunctions [10]. Particularly, PAMM-AID walker takes advantage of sensorial information to assist blind people guidance [17]. Autonomous navigation is accomplished either by exploring ultrasonic, vision or infrared sensors for real-time detection and obstacles avoidance [40] or following predetermined paths inside indoors settings, to achieve a certain location specified in the map [41]. Following a predefined map is ideal for indoor environments, particularly homes or clinical settings, whereas the obstacle negotiation is recommended for open-space and dynamic environments [10].

Several robotic walkers integrate cognitive and sensorial assistance, including [39]. JARoW, Ye et al., PAMM-AID (Personal Adaptive Mobility AID), i-Walker developed by Cortés et al., i-Go and Huang et al. walker, provide cognitive assistance [17]. Huang et al. proposed a CO-Operative Locomotion Aide (COOL Aide) SW directed to safety, by predicting falls and adjusting the SW motion accordingly to accommodate [17]. As for Cortés et al., and PAMM SW, their studies address safety, by incorporating inclinometers on its system, which can identify positive and negative slopes and adjust the velocity to ensure the SW do not uncontrollably accelerates downwards and provokes falls [17]. Ye et al. [27] presents a walking aid robot embedded with an obstacle avoidance system with width-changeable to adapt to different environments. The robotic walker has ultrasonic sensors placed around it to detect the surrounding range, leading the rods of the rear casters to contract when passing through narrow spaces and to be in stretch mode in spacious environments [27].

Other robotic walker examples which explores sonar, infrared sensors, wheel encoders and laser range finder sensor data for obstacle avoidance are MARC [42] and RT walker [43]. Also, they entail a control system to guide the user along a predefined path delineated accordingly to a map. Other relevant examples of SWs with path planning control system are CAIROW (Context-aware Assisted Interactive Robotic Walker) [44] and Omni RT walker-II (ORTW-II) [45]. In addition, interaction often occurs through visual, auditory or haptic feedback, to provide instructions or alerts to the user regarding the environment [10].

In addition, some research works conducted with SW adopted the shared-control concept. SWs exploring simultaneously, a navigation system (autonomous) and a user-interaction system demand a shared-control system to determine whether the user or SW yields control, once their intentions often misalign. COOL Aide passive SW [32] sought to accomplish a shared navigational control strategy, based on real-time obstacle negotiation and user interaction for propulsion. Cortés et al. and PAMM Smart Walker also includes a shared adaptive controller to determine whether the machine or user detains control of the motion. Yet, its use is limited to specialized indoors environments.

Recently, research works conducted with smart walkers have adopted preliminary strategies to attempt an autonomous guidance based on implicit user intentions inferred from human gait patterns [33], which implies a change of the paradigm of force-sensing interfaces for a vision-based one [39]. Nonetheless, neither one of these studies considers the deficits of conscious proprioceptive sensory system present in ataxia, which causes ataxic gait patterns to be peculiarity and unpredictable, which compromises the effective walker navigation strategy based on lower limbs patterns explored by those.

Similarly to those presented, ASBGo∗ includes sonars, cameras and Laser Range Finders sensors for navigating and interpreting the user’s intentions during assisted-ambulation. Novelty arises from closed-loop assist-as-needed motion control strategies able to adapt autonomously, through innovative combination of real-time multimodal sensory information from SW built-in sensors to predict users’ motion intentions, without imposing a cognitive load.

2.2.4 User’s State Monitoring and Security

Through interactive and embedded SW sensors, quantitative and continuous relevant features regarding user activity can be extracted. These features can assist clinicians tracking patients’ status throughout a set of rehabilitation sessions and actively contribute to aid in treatment decision-making and consequently customize rehabilitation programmes [10]. The final aim would be to alleviate the burden on physiotherapists by reducing the direct assistance, once it is unfeasible to continuously attend/monitor patients rehabilitation interventions and self-assessments of patients are often unreliable. In addition, this also promotes the possibility of ambulatory user activity monitoring [10]. An accurate recognition of the human locomotion implies a well-established real-time lower and upper limb tracking algorithm, potentially based on statistical methods/heuristic rules or machine learning methods.

Initially, some authors hypothesized that through the analysis of the distribution of the weight load applied over the handlebars, inferences could be done regarding the gait cycle, once those cyclic changes reflect the gait cycle. Once determined those load changes, the corresponding gait features could be easily identified. Results have reported peaks in vertical direction to be related to heel initial contact and in the forward direction to the toe-off event cycle [31, 46, 47]. As an example, Alwan et al. [22] extracted gait features, including heel strikes and toe-off events, as well as double support and right/left single support phases using only two 6-DOF load cells. Another study exploring force interaction (direct interaction) between user’s upper-body and walker is Abellanas et al., which can extract cadence based on Weighted Frequency Fourier Linear Combiner and precisely identify heel-strike (HS) and toe-off (TO) events. HS and TO estimation methods present mean errors of 1.35 and 0.55% respect to gait cycle. For continuous cadence estimation, the Mean Square Error is below 3.3 steps per minute. Given these gait features, not only are they able to infer the user’s state, but also adopt a motion control strategy, promoting safety. Extraction of gait features based on force sensors imposes a limitation, once weight load merely provides temporal patterns.

Next, a paradigm transition occurred, going from direct interaction (force sensors) to indirect interaction approaches. Exploring on board SWs sensors to acquire clinical insight by tracking feet and legs trajectory revealed promising, since a robotic walker beyond being primarily an assistive locomotion device, it is also a device capable of stimulating an active participation of patients in functional recovery (rehabilitation) [10].

By means of lower limbs’ segments tracking with sonar sensors [4], accelerometers, laser range finder sensors (LRF) [48], infra-red sensors [33] or cameras [49], it is possible to extract surrogate markers of incipient disease manifestation or identify abnormal gait patterns associated with disease progression [50], in order to proper customize/adjust treatment or rehabilitation program.

In addition, tracking lower limbs not only offers the ability to compute gait parameters but also addresses safety measures. In autonomous and shared-control guidance section, it was described safety measures adopted to ensure a safe interaction and navigation of the user with the surrounding environment, through obstacle avoidance (e.g. ultrasounds).

Recently, by means of analyzing human and SW interaction, instead of the interaction of the SW with the surrounding environment, some works have adopted distinct preliminary safety measures. For instance, CAIROW robotic walker is equipped with a laser range finder sensor to recognize human locomotion, and simultaneously adopt a safety strategy by adjusting the walker velocity accordingly to the distance measured between the walker and user legs [44].

Another robotic walker that explores lower limbs tracking to simultaneously extract gait features and promote safety is Frizera et al. [4]. Based on direct transmission technique between two sonar transmitters placed on each leg and one sonar receiver accoupled to the walker frame, it is feasible to extract gait features. Moreover, the information obtained can be used to modulate the velocity of the motors of the device accordingly to prevent potential falls [4]. Wu et al. [51] is another example of a robotic walker study, exploring ultrasonic receiver and transmitters tied to each leg, making it feasible to infer the position and orientation of the users’ feet based on straightforward ultrasonic wave propagation math. Given the 3D feet pose, the authors propose a motion control strategy in agreement to promote safety.

Next, studies exploring infrared sensors and laser range finder have been widely addressed. For instance, the RT-Walker [52] is also equipped with an LRF and performs an estimation of the kinematics of a 7-link human model. The model is only used to estimate the position of the user centre of gravity (CoG) in 3D. The LRF acquires the position of the knee with regards to the walker. Despite the model have been tested in the real-world environment, no real walker users tested this system. In addition, JAIST active robotic walker (JARoW) tackles the challenge of accomplishing a natural user interface between a user and JARoW for ambulatory application. It proposes a particle filtered interface function to estimate and predict lower body segments location based on a pair of rotating infrared and laser range finder inputs [33]. The feedback motion control strategy acts accordingly to lower body segments location to adapt the pretended motion (e.g. velocity). Yet, its potential application is still scarce, considering issues were reported due to human gait variability [48].

Up to now, infrared sensors and LRF systems have been widely explored, but relying on these sensors often lead to false detections, resulting in an impracticable algorithm. Pallejà et al. [53] reported miscalculations of spatiotemporal gait parameters. Recently, the paradigm for 3D feet pose tracking have evolved to technologies using depth/RGB images [33]. As an example, Hu et al. [49] employs a probabilistic approach based on particle filtering to obtain an accurate 3D pose estimation of users’ lower limbs segments by exploring a depth sensor data along the coronal plane (Kinect), accoupled on the lower part of a four-wheeled walker. Nonetheless, position errors reported by the authors (less than 60 mm) are considerably biggerthan the ones obtained with markers (27 mm) with VICON [54]. Similarly, Chung Lim et al. [55] proposes a markerless gait tracking analysis system based on exploring depth image sensor onboard the robotic walker and explores the previously mentioned probabilistic approach as well.

Lastly, Joly et al. [56] proposes a standard 4 wheeled-walker with an accoupled Kinect in the axial plane for biomechanical gait analysis through feet position estimation during assisted-walking based on the same probabilistic approach of [49]. Authors reported orientation errors to be less than 15°.

Concerns still pose around vision-based capturing systems for biomechanical analysis, being the major concern human gait variability, particularly high for ataxic patients. Neither one of these studies address pathological gait patterns. ASBGo tackles innovation for combination of real-time multimodal sensory information from SW built-in sensors (e.g. camera, infrared, force sensors, etc) for a proper pathological human locomotion recognition.

2.3 ASBGo∗ Smart Walker

A Smart Walker is intended to be a device that can act as a versatile rehabilitation and functional compensation tool. It should be adaptive considering the necessities of its user and its use should be safe. Patients present different necessities according to their intrinsic characteristics, their disorder and therapies. In order to help them, a Smart Walker should provide different functionalities.

For the creation and development of a medical device such as a device, it should be taken into account for whom it is intended (end-user). This brings crucial characteristics and limitations to the development of the final prototype. Therefore, it is important that first of all a list of goals is specified before any other point of prototype creation is set.

The first goal is to guarantee the safety of the device to its user. The walker should be robust and reliable in order to reduce to the maximum any risk of injury to its user. Second goal is the attractiveness of the device, which means that it has to be economic and comfortable. Other goal is to provide multifunctionality to the walker, being adjustable to the user and able to incorporate and solve various problems such as being motorized and help its user in various tasks (e.g. sit and stand from a chair). Also, the SW’s design must be suitable to the aim of use, i.e. as a functional compensation and rehabilitation tool. Thus, the device must have an ergonomic design that can provide the necessary support for the patient’s treatment. Finally, its use must be practical, easy to transport, store and adjust.

This section aims to present the project in general, focusing on design considerations, mechanical structure (frame and main components), walker’s system, the electronic and mechatronic components, including the sensorial system, and finally its functionalities as a gait assessment tool.

2.3.1 System Overview

The development of an ASBGo (Assistance and monitoring system aid) 4-wheeled motorized walker aims to provide safety, a natural maneuverability and a certain degree of intelligence in assisting with the use of multiple sensors. In Fig. 2.1, it is presented the main functions that are proposed to integrate in the SW. These main functions are structure, motor connection, sensor location, adjustments, extra-help components. Each main function has several sub-functions that were considered throughout the project. In this way, several options were considered to be designed and developed, so the designer could get a better sense of the most reliable option for the final prototype.

Fig. 2.1

Main functions proposed to the SmartW prototype. The dark boxes represent the final decisions

In design process, the first design proposal is subject to evaluation against the goals, analysis, refinement and development. Sometimes the analysis and evaluation show up fundamental flaws and improvements to be made, and thus, the initial conjecture has to be abounded, a new concept generated, and the cycle starts again. ASGo project is no exception and three different prototypes were developed until the final version, and its evolution is presented in Fig. 2.2.

Fig. 2.2

ASBGo prototypes’ iteration. From left to right the frist, second and third models

Then, the initial version of the ASBGo was projected as a proof of concept in order to verify some requirements and functions. The structure was too rudimental, needing improvements, and composed by iron materials, which are very heavy. Sensor locations were tested however the components were fixed, and a more adjustable position was required.

The second version was designed with a circular tube base with a parallel structure that can pass through any environment (elevators, doors, etc) and to have a small area to have an easy storage. However, this latter characteristic turned out to be a bad option for its users since most of them present a gait with a wide base of support, making them to trip over the walker structure. The motorization system of this SmartW was the same as the first prototype. This second prototype was tested with different patients and important modifications were set for the third prototype.

In the third prototype, a more robust and stable structure was manufactured to give greater sense of confidence and safety to the user. In terms of the base structure, oval tubes were designed and instead of parallel tubes, they were angled in 10° for each side. This prototype was intensely used in clinical trials with gait disabled patients. During that time flaws were identified such as the buckling of the length adjustment handlebar’s systems, low robustness of the height adjustment system, low forearms’ ergonomics and the support base was not wide enough for the user who presented disordered coordination between trunk and legs and dysmetria and were constantly hitting the low frame of the walker (i.e. box compartment, and tube structure).

After extensive field research and several discussions with medical staff and physiotherapists of the Hospital of Braga and respective patients, it was possible to conclude that the users of walkers, especially users with ataxia and cerebellum lesions, tend to have a wider gait base of support. Another aspect that was observed in some patients was the asymmetry of support in the walker. They have a tendency to choose one of the arms and therefore have decentralized gait forcing on one of the upper limbs, creating an incorrect and harmful posture.

Therefore, it was needed a fourth model (Fig. 2.3), ASBGo∗ that includes improvements considering mechanical, electronic and software architecture. This device should integrate all the sensors embedded in the previous prototypes and be based on a modular SW architecture in order to an engineer easily integrate new functionalities, operating modes, sensors and adjust any mechanical and electronic necessary modifications. Afterall, this device was specially design to take into consideration a rehabilitation treatment for patients with ataxia. For example, an abdominal surface area with a curvature in the contact area with the user was added in the fourth prototype to center the user and correct his posture, independently of his anatomy. Such surface was built of wood because it is a cheap material and attractive. Most of the SW weight (electronics and heavy components) was placed in the lower part to reduce the risk of instability and provide a better general equilibrium. An electric lifting system was installed with a load capacity of 800 N, less prone to buckling. Some extra-help components were also added. In order to give more autonomy and safety to patients it was added two bars with handles on the back of the walker to assist the transition of sit-to-stand. Finally, the box compartment that accommodate all the electronic part was designed with a good aesthetic, functionality and a structure to enable a wide support base for ataxic patients.

Fig. 2.3

ASBGo∗ system overview: (a) mechanical frame and its subsystems and (b) fourth prototype

In summary, the ASBGo∗ has a mechanical structure that allows the installation of motors, sensors and other electronic components. The functionalities and characteristics of the device are presented in Table 2.1. ASBGo∗ has four wheels and a supporting structure that holds the user. Its front casters can freely rotate. Two motors drive its right and left rear wheels independently. Each rear wheel is installed with an encoder.

Table 2.1 ASBGo∗ characteristics and functionalities

For rehabilitation purposes, the SW must provide adequate physical stability and safety, that is required in early stage treatments and be able to aid in the progression of the patient, as the users become more independent to control the walkers handling. The configuration of the handles can provide adequate stability levels and may also be used in man-machine interactions, such as detection of user’s movement intentions. Thus, the ASBGo∗ walker design provides a handlebar as a direct interface and is composed by potentiometers, and laser range finder and ultrasonic range finder sensors to be used for navigation and autonomous operation. For safety measures, force sensors were installed in the forearm supports and infrared sensor below the wooden support. Also, in order to monitor and work as a gait assessment tool the SW has an IMU and two Real Sense Camara to collect data regarding gait parameters, stability, equilibrium and posture of the patient during the therapy session. All these features will be presented in detail in the following sections.

As mentioned, the fourth design iteration proposed improvements and upgrades on a mechanical and design perspective on a third prototype of the ASBGo smart walker. Nonetheless, electronic and software improvements were also considered. In this step, it was fundamental to ensure the successful implementation of the associated electronics from previous prototypes; to have easy access to the various electronic components; to have easy and intuitive use and implement a unified modular system architecture, ensuring robust and user-friendly solutions, in order to engineer a trustworthy device that will establish a new rehabilitation concept.

The integration of embedded sensors, the development of a modular software architecture (Fig. 2.4) capable of merge all the different algorithms, which will assure the device’s autonomy and enhance the user’s monitoring and assessment, and the implementation of user-friendly and robust solutions capable of working without failure were achieved steps.

Fig. 2.4

ASBGo∗ Smart Walker system architecture

Given its generic nature tools, inference on modular development, documentation, message-passing infrastructure and positive evolution, revealed to be an asset for the new system, the software architecture was built using ROS, a robotic software platform. The system is centralized in a Inter NUC computer which runs the Main Controller of the system over ROS layer. The Low-Level Controller takes care of the low-level part of the ASBGo∗ and is built upon a Real Time Operating System-RTOS (FreeRTOS) which consists in a program that schedules execution in a timely manner, manages system resources, and provides a solid base for developing application code in a multitasking environment.

Gathering the main functional requirements for a robotic smart device’s modular software and hardware, the new architecture was outlined and can be achieved with the embedded sensors and their interaction through well designed algorithms:

• 1. Acquire data from a set of sensors in real-time;

• 2. Monitor the battery’s voltage;

• 3. Stop in case of emergency;

• 4. Control walker motion based on user’s physical manipulation through a handlebar, or a remote control;

• 5. Monitor in real time the patient’s lower limb motion (gait parameters);

• 6. Monitor in real time the patient’s posture and balance;

• 7. Ensure a safe movement and provide alert errors (e.g. processing unit failure, motor errors, sensor acquisition communication failure, etc);

• 8. Provide biofeedback through a local graphical user interface;

• 9. Achieve intuitive human-walker interaction, without demanding cognitive efforts;

• 10. Include a database containing the clinical information of patients and their sessions, and thus enable to monitor user’s progress.

Sensors Data Acquisition and Motors’ Control are included in Block 1, Block 2 corresponds to the Computer Vision for both lower limb and posture, Block 3 through a USB communication will be used for the assistive autonomous navigation functionality, and finally Block 4 with different type of communication has the local interface GUI (monitor) and the control remote. The blue shaded squares represent the modules directly connected to the Main Controller.

Next, all the sensors and actuators presented in each block will be presented along with their role on the system. Firstly, it is important to describe the mechanical frame and main mechanical components that shelter all the electronics.

2.3.2 Mechanical Frame and Main Components

This document is based on the idea that a SW should provide support whenever required, and it should be an easy-to-use device, monitoring and rehabilitation tool, presenting the following features: (i) provide dynamic support – whenever the user is walking, standing or sitting, a walker should provide a relatively stable support for the user to recover from losing balance; (ii) demand little or no effort to use – i.e. to move and change direction; (iii) be user-friendly – the movement speed and direction is controlled by the user subconsciously, not requiring special training. To achieve all the listed features, it is important to have a structural frame to withstand not only all the robotic components (sensors, computers, hardware) but also the patient and his/her body weight. Hereupon, in this section it will described the mechanical frame and main components composing the total ASBGo∗ SW.

The structure is divided into three main parts: the lower section, the middle section and the top section. The lower section is identified on Fig. 2.5 as points one and two (orange and green shaded parts), the middle is solely the point three (blue shaded part), and the top section is identified by points four, five and six (pink, yellow and grey shaded parts, respectively).

Fig. 2.5

Identification of ASBGo∗ mechanical frame main parts

2.3.2.1 Low Section

SW base systems are mainly concerned with balance and stability of the device’s frame. The system must be secure and stable, not putting the user’s health and physical state in danger. By this, we addressed this challenge by relying on significant weight to lend stability to the system and by putting the electronics and hardware on the lower section of the structure.

The structure (1) is composed of oval tubes whose shape provides the necessary space, especially width, for the mid stance phase of gait for patients with ataxia and cerebellum lesions [57]. Two motors drive its right and left rear wheels independently and have 24 V DC with nominal speed of 40 rpm and nominal torque of 5 Nm. The front wheels are smaller than the rear wheels to enable a better control at maneuverability since the perimeter of the wheel, thus the space covered, is smaller than a greater wheel. Also, the front wheels are caster and thus freely rotate enabling a better device maneuverability. The box (2) is used to store the electronics and hardware. The lower base of the box houses two 12 V rechargeable batteries with a capacity of 36 Ah. The right side comprises the hardware (Low-Level Controller) and all the acquisition system of the sensors from Block 1 of system architecture. On the left side are attached the motor drivers and the power plug connector.

2.3.2.2 Middle Section

The middle section (3) comprises mainly the two support frames for the active depth sensor camera and the electric lifting columns. The lower support frame is directed to the user’s lower limb to acquire gait data parameters, and the upper support frame has the sensor for the posture and equilibrium assessment. As for the electric lifting columns, this system is capable of supporting 800 N of vertical load and providing a stroke length of 0.650 m. This system is used either to adapt the SW height to the user’s height and thus provide a comfortable maneuverability, and to assist the user in sit-to-stand moments.

2.3.2.3 Top Section

The three points that comprise the top section are top frame (4), the handlebar (5) and the handlebar and monitor frame (5). The top frame is mostly the structure that supports all the top components but also the connection to the middle and thus lower section. At the mechanical level, the physical support is provided by a wood table in which patients can sustain the major part of their weight. This technical aspect gives a real and secure sensation to the patients, as they have a large surface where they can grab on to, increasing their support and reducing the risk of falling. Moreover, the wood table as also an abdominal surface is with a curvature in the contact area with the user to center him/her and correct his/her posture, independently of his/her anatomy. Besides the that, the wooden table has two comfortable and ergonomic forearm-supports, with foam filling, with length and width adjustment by Velcro system. These supports have also the possibility to be integrated with sensors, as it will be detailed in the next section. A simple normally closed emergency button was also installed on the top section. When pressed, it opens the circuit, thus stopping the ASBGo∗ SW in the middle of its use, in case of emergency events: falls, system failure and/or uncontrolled guidance by the patient.

It is thus essential to be able to anticipate the user’s intentions such that the walker might proceed accordingly and verify if the device is effectively helping the user in his rehabilitation training. In order to do this, it has to be developed an interface that establishes a bridge of interaction between user and walker. This interface should be able to adapt to users with different levels of physical and cognitive capacities. This adaption should be done in a user-friendly, natural and transparent manner to the users not being demanding at their cognitive level. Thus, these interfaces have to be able to read and interpret the user’s command intentions to drive the device accordingly. There are many types of interfaces that have been used in smart walkers as we have seen in the state-of-the-art section. Despite all these advances in the current state-of-the-art user-walker interaction field, there are still many unsolved questions and key areas in determining user-friendly and efficient interfaces. Especially, it was not found in the literature an interface with a user-oriented design, that is, an intuitive-use device with low cognitive effort for patients, such as ataxic individuals, capable of responding to the low motor coordination.

To acquire user’s commands, the proposed handlebar needs two-axis sensors to detect the forward and turning forces [19]. These forces are detected with two potentiometers. Thus, two commercial potentiometers were embedded into the handlebar: a linear potentiometer (0-10kΩ linear) to detect directional changes in speed and a rotary potentiometer (0-470kΩ linear) to detect forward changes in speed. With this system, the user can intuitively manipulate the SW at his own pace. The SW interprets these two basic motions and controls the motors speed and direction, accordingly. It is not allowed to walk backwards. Since abduction movement of the wrists should not be allowed a movement greater than 20° and the rotary potentiometer has a 300° range, two mechanical battens were integrated to limit this rotation movement. On the other hand, the translational movement is limited by springs, placed on the center of the axle tube, but with the trade-off of not forcing the flexion/extension of the users’ wrists. In addition, the handle-bar is characterized by its balance, i.e. when not actuated by the user it remains in its zero position, which corresponds to when the device is stopped. This balance is extremely important because user safety must be assured, since users are mostly people with physical weakness.

The part number six is the sheet metal that supports the handlebar’s frame and holds the monitor to the top section structure. The monitor is placed with a tilt angle of 25° to provide an ergonomic visibility of the GUI to the user.

2.3.3 Sensors and Actuators

The final version of ASBGo walker (Fig. 2.3) was integrated with multiple sensors and other electronic components given it different functionalities and characteristics. It will be now presented the structure of the sensorial system as well as the different modules and their relations.

The smart walker collects and processes information from a series of embedded sensors, allowing it to understand the surrounding environment, infer the user’s intentions and act accordingly.

Starting from Block 1 – Sensors Data Acquisition and Motors’ Control, as referred previously, is mainly focused on the box compartment. A STM32F4 Discovery development board is used to implement the low-level control, i.e., sensors data acquisition, data’s management and motor’s actuation. Besides that, it handles the emergency events and monitors the battery’s voltage checking if it is in critical state (discharged). The application runs in FreeRTOS.

The linear and angular potentiometers are used to acquire user’s commands from the handlebar and thus control the angular and linear velocities of the SW, respectively. This information is then processed and send to the motors that act the device accordingly. The velocity and traveled distance of the motors is calculated with two encoders, coupled in the device, one in each rear wheel.

Strain gauges (load cells) and the infrared sensor are mainly used to monitoring the risk of fall by detecting possible falls, instabilities or imbalance moments from the user. The uni-axial force sensors (one in each forearm) were installed in the forearm and they detect if patient is correctly supported and if load on the walker is to heavy or null may mean that the subject is in a dangerous situation. As for the infrared sensor, localized beneath the abdominal surface table, it measures distance between the device and the patient and two conditions can be perceived: greater or lesser proximity of the user to the walker, which can be too close or too far meaning that the subject is in a dangerous situation. Depending on the detected state, the walker will perform a different action. Therefore, each state will provide enough information so that the walker can make a decision. This multi-sensor system comes to meet one of the ASBGo∗ project main goals to ensure user safety, monitor different states of a person and extract patterns and behaviors of the user along his/her gait during assisted gait with the SW.

Besides this monitoring system, the safety mode is also characterized by a warning system that alerts the presence of obstacles in front of the walker. This is done with 9 sonar sensors distributed in a three-layer configuration, on box compartment, to maximize the detection area. A low ring of 6 sonars mounted forward oriented detects the majority of ordinary obstacles, like people, walls or other low obstacles. High obstacles such as tables or shelves are detected by a high ring of 2 sonars pointing upwards is mounted to detect high obstacles. These 8 sonars are meant specifically for obstacle avoidance. An extra sonar pointing downwards is mounted on the walker to detect stairs. This sonar does not contribute to the obstacle avoidance task, but stops the walker when changes in the ground, such as stairs or holes are detected. Results demonstrated that the sonar configuration mounted on the SW had successfully detected several types of hospital obstacles, including dynamic obstacles [58]. When all the sonar sensors measure a distance greater than a predefined minimum distance no alert is given. When ASBGo∗ is at a distance of less than a pre-defined maximum distance, a sonorous/visual alert is activated to warn the patient that there is an obstacle near the SW.

Other important requirement of a SW is the possibility of doing clinical evaluation during walker-assisted gait. Therefore, the final sensor integrated in Block 1, the IMU (Inertial Measurement Unit) is mostly used to indicate the stability of the user regarding his center of mass (COM) position, giving posture and balance information. The sensor is placed at the trunk of the patient and measures the COM’s displacement. Another good interpretation of this parameter is the capability to detect fall risk situations. If the body is unstable, the probability of falling increases substantially [59, 60].

Clinical gait analysis is the process by which quantitative information is collected to aid in understanding the etiology of gait abnormalities and in treatment decision-making. Advances in robotics made it possible to integrate a gait analysis tool on a walker to enrich the existing rehabilitation tests with new sets of objective gait parameters. Further, these systems allow evaluating the evolution of some disorders and enhance diagnostics in ambulatory conditions. The team of this study developed a feet detection method to estimate feet position during assisted gait and implemented an application to extract patient’s upper body motion, assess the balance, posture and the risk of fall. More details will be presented in Sect. 2.3.4 along with the corresponding integrated biofeedback application. One active depth sensor (ADS) will track the feet, to provide position and orientation of the feet center and is localized in the middle section of the SW, the lower support frame, with 60 mm of height and 40 mm of distance to the patient. Another ADS is placed, also in the middle section, on the upper support frame, pointing upwards to the user’s trunk and shoulders. This last sensor is solely intended to be used when the user is driven by the walker (remote or autonomous maneuverability) while supporting his/her weight on the handle grips at the back of the ASBGo∗ SW, and thus the wooden table is provisionally removed to allow visibility to the user’s upper body. All this sensory system is integrated in Block 2 – Computer Vision.

Path modulation and generation are classical issues in navigation architectures for SW since intentions of the system are taken into account to compute a final locomotion command, used for gait training sessions. Thus, Block 3 corresponds to the autonomous navigation and integrates a Hokuyo URG-04LX-UG01 Scanning Laser Rangefinder sensor. Consists in a small and accurate laser scanner with a detectable range of 200 millimeters to 5.6 meters. Also, it scans in a 240°area with a 0.36°angular resolution and a scanning time of 100 ms/scan. This autonomous navigation mode allows the user or physiotherapist to define the desired position coordinates while guiding the SW in the environment.

The local Graphical User Interface (GUI) provide means to configure parameters related to the locomotion, perform rehabilitation sessions, check the state of the device and configure and run application-level features. The QML language was used fort the design of the interface that is included in Block 4 – User Interaction. This user-SW interaction will be further discussed in Sect. 2.3.4. Finally, as to the remote controller, if the patient is not able to drive the device using the handlebar, the medical specialist can drive it remotely. Also, the buttons may be convenient to start or stop the system.

All the collected data from the sensorial system is constantly stored on an embedded database, with the patient ID identified. The information is all gathered and organized in folders according to the patient and the gait training sessions’ date and time.

Thereafter, it will be discussed the functionalities, maneuverability modes and main functions.

2.3.4 Functionalities

As mentioned previously, the ASBGo∗ SW has the possibility of doing clinical evaluation during walker-assisted gait, acquire biomechanical data collection, offering real interactive applications such as the biofeedback real-time interaction and a multitasking game. Also, besides acting as a gait assessment tool, ASBGo∗ SW enables a smooth and secure maneuverability to support the patient in a dynamic and intuive rehabilitation. With such functionalities, ASBGo is versatile, adaptive and safe as a rehabilitation and functional compensation device for patients with mobility problems prescribed for its use. Versatile since it can be used for a variety of patients that present difficulties in mobility associated with other personal limitations such as visual problems and/or cognitive). Adaptive since it allows adapting the parameters of control systems (such as minimum and maximum speeds) depending on the physical limitations of the patient. Safe because the structure of the presented SmartW was developed with a design that provides for a more stable movement and safety for the patient.

Therefore, the ASBGo∗ SW functionalities and operation modes are structured as seen in Fig. 2.6 and through main modules: Maneuverability, Clinical Evaluation, which comprises the Gait and Posture Assessment Tool, Biofeedback and Multitasking. All these features are integrated in a safety module system, with patient-oriented considerations. The red arrows correspond to a direct interaction and parallel use between the four main modules. The blue dashed arrows indicate the intrinsic topics of the module. Finally, the black dashed arrow demonstrates the dependency of the Biofeedback from the Clinical Evaluation data. Each one of the listed modules will be sequentially described.

Fig. 2.6

ASBGo∗ SW functionalities main modules

2.3.4.1 Maneuverability

The main goal of the developed SW (ASBGo∗) is the rehabilitation and functional compensation of patients with mobility and balance problems. Since patients can present different types of difficulties and disorders associated with locomotion, the SW has to be adaptable to these different limitations. Thus, through three modes of maneuverability is possible to adapt the operation of ASBGo∗ depending on the difficulties of the patient and provide a safer, comfortable and efficient rehabilitation.

The manual operation consists in a double-task training and is recommended for patients with visual and cognitive capabilities and enough motor coordination to conduct, independently, the walker under the guidance of the commands defined on the handlebar. In this way, the patient is responsible for supervising the ASBGo∗ movement while avoiding the obstacles in front of the smart walker, thus constituting a well synchronized double-task training.

After placing the hands on the two handgrips, the user will act on them accordingly to the command he wants to perform: start to walk, accelerate, slow down and turn left or right. Thus, if the user intends to: (1) increase the walking speed, he has to turn the handlebar in a counterclockwise direction; (2) decrease the walking speed, he has to turn the handlebar in a clockwise direction; (3) turn to the right, he must move the handlebar to the right side; (4) turn to the left, he must move the handlebar to the left side. Safety is a feature always present in any of the modules. For example, during manual operation the user is supported by the abdominal wooden table and the forearms’ support. The latter has embedded load cells that acquire data related to the applied load. If load on the walker is to heavy or null may mean that the subject is in a dangerous situation. Also, it is possible to assess and correct the patients’ posture accordingly to the values given by each sensor and provide a visual message, so the patient compensates his/her load on the least supported body side. In a similar way, the infrared sensor, below the wooden support, measures distance between the device and the patient and thus monitor the risk of fall.

The other mode of maneuverability is remote control, developed to allow the physiotherapist to monitor the user behavior and control the velocity and orientation of the SW accordingly. In this mode, the physiotherapist can analyze the behavior, compensations and reactions of the patient against sudden changes in speed and orientations and defines the commands to control the SW’s movement. In addition, it allows the patient to focus on his gait pattern and balance and not on the guidance of the SW. It is advice the use of this mode in parallel with the biofeedback to augment the correction and focus of the patient’s locomotion and feet position. With such visual information the patient demonstrated will have the opportunity to auto-correct his movements, having the sense of his problem and solving it automatically. In this mode the patient has the option of either support himself/herself on the forearms or use the handle grips at the back of the walker.

To decrease physiotherapist work effort to monitor the patient’s behavior and focus solely on the patient’s locomotion rehabilitation an autonomous navigation mode is proposed. Autonomous mode allows the user or physiotherapist to define a target location for the walker [61]. Neither the patient nor the physiotherapist have control over this mode. Decisions are made by the walker itself. This operation mode is suitable for patients with visual and/or cognitive limitations, or/and cannot control the SW manually due to weakness or lack of upper and lower limbs coordination.

This mode it is intended to complement the SmartWalker ASBGo∗ with a lower cognitive maneuverability than the previous ones, taking into account the needs of the patient using it. So, the functions of this mode are:

• 1. Have a map of the environment, where the user should be able to choose a destination;

• 2. With an endpoint selected by the user the walker should be capable of calculate a route from an initial position, that is the location where the walker was positioned when a destination was selected;

• 3. With a route planed the walker should be capable of guide the patient through that path while: avoiding obstacles, like walls, plants and similar; not disturbing the surrounding people; take the intention of the user into consideration.

During ASBGo∗ SW maneuverability, for any chosen mode, while taking into account the safety of the user, the walker will alert about or avoid obstacles that appear along the way. The monitoring of the environment is characterized by a warning system that alerts the presence of obstacles in front of the SmartW and this is done through the nine sonar sensors placed on the front of the box compartment, as discussed in Sect. 2.1 (System Overview) and Sect. 2.3 (Sensors and Actuators).

2.3.4.2 Clinical Evaluation

A SmartW is not only a device to give support and guide its user. It should also have the functionality of evaluating the recovery of its user. In order to well diagnose and follow rehabilitation with the use of a walker, a gait assessment system has to be accurate but also affordable to reduce unequal access to health care and to improve clinical follow-up (i.e. to allow to be used in physiotherapists’/physicians’ office). Similarly, the gait assessment system should be portable and adaptive to the majority of walkers. The system should be contactless to be used in daily routine, improve comfort and decrease the time of analysis. If the system enables real-time analysis, data could be used directly during the consultation by the physician and eventually at home for motivational purposes or monitoring the quality of walk (to predict any forthcoming deterioration of a user’s gait). Such availability of equipment is important to allow an objective assessment of a person’s functional physical state. Thus, inclusion of embedded and portable systems on the walker seems to be more appropriate for building a gait assessment system to characterize and analyze walker-assisted gait. Advances in robotics made it possible to integrate sensors on walkers to act as portable gait assessment systems, thus ADS (active depth sensor) were used to acquire biomechanically data of patient’s body motion.

The definition of main concepts and the software applications are presented below:

• Lower limb monitoring. Assess the patient’s gait pattern and storing the patient’s data: spatial and temporal metrics related to the gait cycle, using one Active Depth Sensor (ADS) to track the feet (position and orientation of the feet center). This allows generating easy-to-read plots relative to the symmetry of gait in terms of size and time of passage, average width between feet, distances traveled, among other spatiotemporal gait parameters. With this data, the medical team can objectively measure the patient’s evolution and prescribe an individualized treatment. Also, being a clinical tool, it is also a motivating agent which leads patients to achieve better results by improving their gait symmetry.

• Upper limb monitoring. Monitoring and real-time assessment of posture with an ADS oriented towards the user’s upper body to evaluate patient’s posture, equilibrium and stability, in real-time. The data is collected and stored in the database which can also be used, later, by the medical team to measure the patient’s evolution.

This algorithm is based in the detection and use of the patient’s feet’s centroids (Lower Limb Monitoring) and identifies user’s central points (shoulder, hips and upper body), the velocity of the upper body’s center and the patient’s hands (Upper Limb Monitoring). The data captured by the camera are disposed through a 3-dimensional system (coordinates in (x,y,z)).

Besides, other system is considered for the clinical evaluation: use of one IMU placed at the trunk, level of the sternum to estimate the 3D orientation of the trunk, the stability of the user regarding his center of mass (COM) position, posture and balance information, by applying an efficient algorithm. Since the common problem of walker users is usually lack of balance, such assessment is fundamental through their recovery. The assessment of posture stability and risk of fall of this work will be based on the study conducted by Doheny et al. [60].

2.3.4.3 Biofeedback

Most ataxic patients are distinguished by weakness in their proprioceptive system. Moreover, during the gait, it is common that the patient’s legs bump against one another. Due to the degradation of their proprioceptive system, the individuals with ataxia are unable to identify precisely the localization of his/her legs and sense their position relative to the ground, resulting in an unsteady gait [62]. Moreover, ataxic gait has been characterized by a widened base of support, inappropriate timing of foot placement, reduced step frequency, increased step width, and prolonged time in double-limb support. Both impaired postural stability and decomposition of multi-joint leg movements appear to be factors in cerebellar gait ataxia [63].

Thereby, the development of a tool which can give biofeedback related to lower limbs performance can be extremely useful for assisted gait training. Hence, the insertion of biofeedback to self-correct gait is one of the main goals of ASBGo∗ SW. This module is intrinsically connected to the Clinical Evaluation module, since it uses the data collected from the ADS sensor and compute it as alerts and warnings for foot drag, consequence of inappropriate timing of foot placement/foot drop, and for leg bumping. If the distance is lower than the threshold defined, a warning is activated. The warning consists in a continuous audible alarm and/or visual image which indicates the undesired limb position. The other alert is activated always when the length distance between the feet are above a pre-defined threshold consider as the desired standard for the patient’s anatomy.

In addition to the monitorization of the pattern followed by feet, the variables to evaluate body balance of the user are also acquired. In a similar way, the upper body biofeedback module acquires data from the Clinical Evaluation and a warning is issued if the patient is incorrectly holding on the handle grips, preventing the risk of fall. This biofeedback helps the individual to correct his/her posture in order to prevent a possible fall.

Results regarding both topics of the Biofeedback module will be provided in the corresponding Section.

2.3.4.4 Multitasking Game

The relevance of multitasking in gait ambulation is widely acknowledged by medical community [64, 65]. Multitasking training is a plus-value advantage in physical rehabilitation and can significantly impact recovery of functional waking in people with neurological disorders. For this, individuals, the sessions become more attractive while at the same time motivate the subjects to get better results for his/her progression. In this context and according to medical request, a multitasking tool that can measure the reaction time of patients to specific stimulus was designed and implemented.

The strategy adopted followed the main listed aspects, considering the literature [66, 67]:

• 1. The application must be adaptable to different levels of cognitive ability;

• 2. It must also include audible and visual stimulus to be suitable to patients with hearing or vision impairment;

• 3. Visual stimulus should include basic geometric shapes as circles, triangles and squares;

• 4. In order to design different visual levels, the geometric figures should be represented in distinct colors;

• 5. Audible warnings should include sounds that are easy identifiable, such as train horn;

• 6. The audible mode must have different difficulty levels and other sounds should be included;

• 7. Patients without cognitive, audible and visual disabilities must perform a multitasking training with audible and visible stimulus interaction.

2.4 Results and Discussion

The potential of using walking aids in patients with ataxia is promising but lacks clinical evidence. The combination of imbalance and low coordination of the lower limbs suggests a strong rationale to use of an intelligent walker in gait training. The ASBGo∗ is based on the assumption that the following characteristics favor this assistance: (1) the actuators (motors) enables systematic speed control; (2) improvement of walking pattern through repetition of gait cycles using accurate sensory information; (3) stimulate a normal gait pattern; (4) performing tasks in parallel by stimulating cognitive and motor skills; (5) gait training in real environments avoiding obstacles; (6) patient biofeedback of their body, thus encouraging their active participation in therapy, speeding up the locomotion recovery; (7) axial support reducing tremor and asymmetry; and (8) human-machine interaction is natural and without cognitive effort.

In this section, are demonstrated the obtained results with the ASBGo∗ SW since the beginning of this project. Results consider the main functionalities and modules well integrated in a safe usage environment: low cognitive effort maneuverability adaptable for different conditions and needs (manual, remote or autonomous); Biofeedback of the body in real-time; and clinical evaluation with biomechanical metrics determination.

Hereinafter, the results will be discussed.

2.4.1 Safety

A very important aspect of SWs is to provide for security/safety such that the user feels safe while controlling the SW, mostly in manual maneuverability. On the ASBGo∗, the patient guides the SW and a warning system is activated if a dangerous situation is detected. Both the environment and the patient are monitored. The monitoring of the environment is characterized by a warning system that alerts the presence of obstacles in front of the SW. The warning system of this operation mode consists of three lights: green, yellow and red. The green light is lightened in the absence of obstacles in front of the SW (Fig. 2.7 situation A), i.e. when all the sonar sensors measure a distance greater than a predefined minimum distance (mindist = 1.1 m). In situation B, the yellow light signal is connected because there are obstacles in a distance between mindist and maxdist = 0.2 m. When SW is at a distance of less than a pre-defined maximum distance (maxdist), the red light is activated to warn the patient that there is an obstacle near the SW, such as in situation C. Additionally, an audible alarm system, with different sound frequencies associated to these different distances, may also be triggered. Note that the parameters (mindist and maxdist) that define the distance of detection of obstacles can be calibrated in order to adapt the warning system to the type of patient in the ASBGo∗.

Fig. 2.7

Three situations detected by the URF sensors used to provide safety features for the ASBGo∗ SW

In terms of detecting the risk of fall of the user, two sensors are used: infrared sensor and load cells. In Fig. 2.8 it is depicted the infrared sensor (IR) signal of a walking user with the ASBGo∗. The IR signal decreases accordingly to the user approximation to the ASBGo∗. An algorithm was developed to detect abrupt changes on the signal, to then detect if the user was falling forward. When such situation is detected, the ASBGo stops immediately. The situation of falling backwards is similar, but the IR signal decreases in Fig. 2.8b.

Fig. 2.8

Fall risk detection sensors identified on the black dashed rectangle: (a) IR signal for a forward fall, (b) IR fall event for a backward fall, and (c) load cells detecting unload event (fall event)

Secondly, two strain gauges, one on each forearm support, are used to verify if the user is with his arms properly supported on the forearm supports. On one hand, if the user relies on both supports, the measured force signal increases, and the ASBGo∗ is enabled to move. On the other hand, if the user is not loading the sensor the output signal decreases until it reaches zero (Fig. 2.8c) and the ASBGo∗ immediately stops. Therefore, the safety mode implemented in the ASBGo∗ allows detecting the obstacles present in the environment and advising the patient of their presence. In addition, it warns the physiotherapist if the user is falling.

2.4.2 Biofeedback

In previous sections we discussed that assisted gait and posture monitoring could benefit from the measurement of feet position and orientation and also the body COM. Several works have been dedicated to the detection of lower limbs. The proposed methods are usually fast, but only detect the position of the legs. Also, they often use markers attached on the feet, which is unsuitable in daily routine. In this work, and to be imbedded in the assistive and monitoring ASBGo∗ SW we proposed a new method to extract feet position and orientation data from a camera depth sensor. The main advantages of the presented method are that it is marker less, faster than using 3D models, robust against clothing variations and that continuously detect orientations of the feet. The precision of the presented method is better than the other marker less methods and seems sufficient for gait analysis.

From the ADS sensor we are able to extract color and depth images that will be used for visual and audible feedback and/or gait and fall risk assessment analysis, respectively. Depth information is obtained through the projection of an infrared light and its deformation by ADS own software. The algorithm developed for gait analysis and posture assessment process the depth frames and calculates the position of the feet, or upper body, relative to the camera, the distances between them and also for the detection of steps, strides and user’s central points (shoulder, hips and upper body), the velocity of the upper body’s center and the patient’s hands.

The extraction of the feet, or upper body, from the depth frame is done through background removal techniques. This, for instance, is obtained by calculating the minimum depth value for each pixel of a predefined number of background frames. This process is shown in Fig. 2.9. From feet distance comparison between consecutive frames it is possible to deduce the different phases of gait of biomechanical metrics. Regarding the upper body with collected data, the software will expose the coronal, axial and sagittal views of the patient, giving important information regarding posture.

Fig. 2.9

Color, depth and depth with background removed

As mentioned, the majority of ataxic patients exhibits small support distances which increase the instability and the risk of fall. Moreover, during the gait they may present moments when the legs bump against one another or there is foot dragging/drop. Thus and due to the degradation of their proprioceptive system, a biofeedback strategy was implemented in order to correct the patient’s gait training performance. During the execution of the lower limb monitoring a window with the patient’s gait state information is shown, along with warnings consisting in a continuous audible alarm, which can be disabled, and a visual image which indicates the wrong position (see Fig. 2.10). And more importantly, the continuous transmission of the color frame to provide the real and online biofeedback of the patient’s performance. Besides patient motivation increases, we also hypothesize that the gait’s symmetry will increase with the use of the lower limb biofeedback.

Fig. 2.10

Patient undergoing gait training session with the SW and the biofeedback module of the lower limb

Regarding upper limb biofeedback, the main goal of this application is the development of a monitoring tool to be used in real time and help the SW’s users prevent risk events such as falls. For that purpose, it was necessary to firstly perform the detection of upper body main sections: neck, shoulders, waist and hips. The method uses border extraction (Canny Edge Detector) in the depth frames and the results of the detection are shown in Fig. 2.11. The picture present in Fig. 2.11a depicts the points used to determine the central point between the shoulders and the neck point. As for the hand support, to correctly and securely use the SW, the user has to place and support his/her body in the respective handle grip. Failure in one hand’s detection is considered has a risk situation (fall), the patient is alerted, and the SW should act accordingly, i.e. stop the movement.

Fig. 2.11

Detection of points of interest of the upper body: (a) points of the neck and shoulders; (b) hip point and waist center determination with an accuracy of 80%; (c) definition of hand’s region

To assess user posture and balance, it is required the determination of the user’s COM. The position of COM can be roughly assumed to be by the midway point between the two central points that mediate the extremities of the upper body (midway point of the shoulders and central point of the waist/hips). To automatically find the waist’s position, the algorithm starts by determining the first possible waist points on each side of the hips. Experimental trials have shown an accuracy higher than 80%.

2.4.3 Clinical Evaluation and Hospital Trials

The manual maneuverability is characterized by controlling the movement of the ASBGo∗ SW under guidance of commands defined on the handlebar by the user. In this mode, the patient is responsible for taking the decisions regarding the ASBGo∗ movement. However, this mode is prescribed by the physician or physiotherapist only for patients without visual and/or cognitive difficulties, with motor coordination and sufficient strength for the SW manipulation handlebar.

Most of the validations to verify the potential of assistive technology like the ASBGo∗ SW and their long-term effects in rehabilitation therapies were done considering the manual maneuverability of the device since it constitutes a well synchronized double-task training.

The validation study, here presented, introduced the smart walker in the rehabilitation of three ataxic patients. Their gait patterns and postural stability was acquired and clinically evaluated. Great improvements in gait parameters as well as in postural stability were observed in all three cases. Important outcomes were highlighted in order to assess the improvement of the three case studies: stride-to-stride variability, symmetry index and COM displacement.

Before beginning the gait training with the SW, all baseline data was collected. Patients were evaluated by the application of BBS and with static and dynamic tests the information was gathered by several sensors integrated in the device, which allowed characterizing the assisted gait and stability. Static and dynamic tests consisted on 4 conditions: (1) static stance, (2) static semi-tandem stance, (3) walk with the SW and (4) walk alone and/or with an alternative assistive device. In each condition several parameters were acquired, as we will see. Conditions (2.1) and (2) consisted on 3 trials with 1 min of duration each and in conditions (3) and (4) the patient had to walk 20 meters. It is noteworthy that condition (4) is done in order to verify which gait and postural modifications have been made with the SW training.

The total average number of gait training sessions with the SW were 20, each one lasting 15 min, in a velocity comfortable to the patient. Before the beginning of the sessions generic evaluations of balance were assessed, however they will not be referred in this document as it is not the focus of this work. Clinical evaluation during walker-assisted gait is the first step to assess the evolution of a patient during rehabilitation and to identify his needs and difficulties. Advances in robotics made it possible to integrate a gait analysis tool on our SW to enrich the existing rehabilitation tests with new sets of objective gait parameters. As already mentioned, the team of this study developed a feet detection method to estimate the position of the lower limbs during assisted walking, using the active depth sensor (ADS). Gait events were identified to calculate the following spatiotemporal parameters, previously determined [68]: step and stride length (STP and STR) for each side, stride width (WIDTH), gait cycle (GC), cadence (CAD), velocity (VEL), stance and swing phase duration (STAD and SWD), double support duration (DS) and step time (STPT), for each side.

With these spatiotemporal parameters, it is possible to calculate stride-to-stride variability. This is a strong indicator of risk of fall. Other important indicator is the symmetry of parameters. This can tell us if the coordination between legs is improving or not. Symmetry indices (SI) [69] were calculated for each feature using the formula:

$$ SI=\frac{U_R-{U}_L}{U_L} $$

(2.1)

Where 𝑈𝑅 and 𝑈𝐿 are any aforementioned features for the right (R) and left (L) leg, respectively. Perfect symmetry results if SI is zero, larger positive and negative deviations would indicate a greater symmetry towards the right or left leg.

The results obtained for the SI regarding the spatiotemporal parameters are following disclosed. We will discuss the SI and stability of the three cases along the training sessions. Starting with the Case 1, Fig. 2.12 presents the gait parameter’ results in terms of symmetry index (SI) of the evaluations done with the ASBGo. As it can be seen all parameters had a good evolution for the improvement of the patient’s gait pattern. Since most parameters present negative asymmetry, the left leg is the one responsible for the asymmetric gait. Looking for the evolution of SI, one can see that SI of all parameters tend to zero week to week.

Fig. 2.12

Case 1 study SI evaluation while in gait training with ASBGo

In a similar way, looking at Fig. 2.13, it is obvious that the symmetry tends to zero across the evaluations with ASBGo. In terms of symmetry, in Fig. 2.14, it can be observed that in the first sessions, the patient presented great asymmetry on gait, improving over time. Over time, the patient improved coordination and symmetry.

Fig. 2.13

Case 2 study SI evaluation while in gait training with ASBGo

Fig. 2.14

Case 3 study SI evaluation while in gait training with ASBGo

Postural stability parameters were calculated during static and dynamic positions using an IMU sensor as was pointed in Sect. 2.3.3. Two stance conditions were evaluated, a comfortable stance (CS) and a more unstable and challenging position semi-tandem stance (for each side, SSL and SSR) [60]. COM displacement was acquired for all conditions (CS, SSL, SSR and ASBGo). In order to have a better visualization of the evolution, in time, of the patient in terms of stability, the COM displacement was approximated to an ellipse. Taking the outside margins of the COM displacement, an ellipse was drawn in Figs. 2.15, 2.16, and 2.17. For simplicity of this work, the results obtained for the stability will only consider the ASBGo walking support.

Fig. 2.15

Case 1 study postural stability results using the SW

Fig. 2.16

Case 2 study postural stability results using the SW

Fig. 2.17

Case 3 study postural stability results using the SW

From this analysis, we conclude that the patients had a large medial-lateral displacement, meaning that he/she presented a lateral displacement that could cause instability while walking, having the tendency to fall sideways. However, this instability was reduced over the weeks, allowing the patients to better control posture while walking with SW. It is noteworthy, that the ML displacement has reduced more than AP, showing greater improvements. In all cases the ellipses decreased their radius, meaning a significant enhancement in COM displacement.

In this study, three different ataxic patients performed gait training with ASBGo. Different improvements, in different recovery times, with different functional gains were achieved by the patients. However, similar measures, intervention and protocol were performed. Symmetry index, stride-to-stride variability and COM displacement were considered the best outcomes to evaluate the evolution of these type of patients, giving quantitative information about their improvements.

Findings of this study show that gait training equipment can be improved and, consequently, better functional gains can be achieved if quantitative methods for evaluating walking performance are developed, such that it can be possible to establish baseline training parameters for each patient and then progress each patient in an optimal recover.

The team is now improving the algorithm for spatiotemporal parameters detection, using improved cameras, calculating additional gait parameters, and in parallel integrate the biofeedback in the new prototype and modular architecture. Besides, the data will be display automatically in a graphical database to be better analyzed by the medical team and the patient as a conscientious and motivating evaluation.

2.5 Conclusions

The smart and assistive walker ASBGo∗ – A contribution to ataxic patients and neurological diseases, is an intelligent, motorized and adaptative walker with interoperability between functions, which pretends to offer gait assistance, either in hospitals or clinics, to imbalance and lack of limb coordination individuals, especially the ataxic population. This device works as a rehabilitation procedure and is generic enough to help people with different locomotion and neurological disorders.

ASBGo∗ SW is an ergonomic prototype, with an aluminium and steel structure especially designed for the ataxic gait, with four wheels, sized for a wide range of users in terms of weight and height. Its mechanical structure includes a support base for the upper limbs implemented with forearm and trunk support, that reduces tremor and asymmetry, a very relevant feature for these patients. The information collected by several embedded sensors is used to characterize assisted gait and user-walker interaction. An instrumented handlebar enhances the user’s manual and intuitive manoeuvrability and interprets the intentions of the user allowing the walker to act accordingly without the cognitive and weight effort to push it.

ASBGo∗ SW acts as a support tool for the rehabilitation of gait and for the diagnosis of gait spatiotemporal parameters and stability of the patient. This is done through the interactive functionalities and embedded tools, now integrated in a new ROS architecture, that allow a real-time analysis of locomotion parameters and posture, and thus assess the evolution of the patient and adjust his/her treatment. Finally, it integrates a system of biofeedback, hence achieving an effective participation of the patient is his/her own rehabilitation.

In this specific document we described the evolution of the SW until nowadays. We explore the different maneuverability options that make the walker a rehabilitation device and functional compensation tool for patients with mobility problems. The autonomous module (navigation) allows the user or physiotherapist to define the desired position coordinates of the walker and autonomously moves to the position avoiding any obstacles in the environment. The manual mode is characterized by the walkers’ movement under the guidance of commands defined by the user. The safety of the user is always in consideration and during any maneuverability mode a warning system alerts the presence of obstacles in front of the walker. The remote maneuverability is called remote control mode and has been developed in order to allow the physiotherapist to control the movement of the walker. Results suggest that these different modes are sufficient for this kind of therapy. A great positive feedback was given by the patients and physiotherapists. This ASBGo∗ showed to be versatile, adaptive and a secure rehabilitation and functional compensation device for patients with mobility problems prescribed for the use of ASBGo∗. Versatile since it can be used for a variety of patients that present difficulties in mobility associated with other personal limitations such as visual problems and/or cognitive). Adaptive since it allows adapting the parameters of control systems (such as minimum and maximum speeds) depending on the physical limitations of the patient. Safe because the structure of the presented SW was developed with a design that provides for a more stable movement and safety for the patient.

In summary, this multifunctional SW, whose project is user centered, guarantees safety, stability, low cognitive effort usability, natural maneuverability and provides an end-user oriented adjusted assistance, in a way to improve the comfort and recovery in rehabilitation sessions. The device intends:

• A safety training for ataxic patients through a cyclic and regular training, enabling an improvement of stability, balance and gait pattern;

• Delaying the early use of a wheelchair;

• Promote the ataxic and neurological impaired patients’ autonomy;

• Provide a means to monitor gait and posture parameters without the need of a gait laboratory, during gait training sessions, and thus allowing the summative assessment of the functional gains of each patient.

The ASBGo∗ smart walker introduces a new relevant concept in terms of rehabilitation and clinical follow-up. A prototype of the ASBGo∗ is already in operation at the Hospital de Braga, in the Department of Physical Medicine and Rehabilitation, where some selected patients perform their physiotherapy treatments and are followed by the physician and physiotherapist involved. At the same time, we are improving and implementing new functionalities to integrate in the modular system of this device. Also, future studies will address more experimental studies with other types of patients, actuation modules in a Hospital environment.