Keywords

1 Introduction

Measurement of human motion has a long history since J.E. Marey and E. Muybridge have proposed chronophotography to record animals in motion [20]. Motion was then expressed as a sequence of poses which is still widely used to describe human motion nowadays. Numerous sensors have been introduced to capture human motion and especially human walking. In this chapter we propose a summary of the most studied parameters in human walking and direct/indirect methods to obtain them.

Human walking is one of the most commonly used motion in everyday life and it has been studied from a long time. However it is still a very active field of research in many scientific domains, including biomechanics, neurosciences, robotics and computer animation. One of the key points is to extract the most relevant parameters of human walking according to the specific requirements of a given application. In this section, we propose a summary of the most popular parameters and give some methods to retrieve them with various sensors.

Nowadays, there are many possible devices to measure human motion and, consequently, human walking. Most of them are generic but some devices have been specifically designed for human walking. In virtual reality, capturing the intentions of the user is necessary in order to navigate properly in the simulated environment or to animate an avatar. While motion capture devices were very expensive and difficult to use in the past, it is now possible to use cheap and easy-to-use systems such as the Wii-mote (product of Nintendo) or the Kinect (product of Microsoft) devices. Whatever the system, the key issue is to design algorithms to extract the relevant parameters of the user’s gait that enable the system to react realistically.

The first part of this chapter is dedicated to global parameters such as walking speed, step length, frequency and global walking trajectory. The second part of the chapter focuses on kinematic and dynamic parameters such as joint angles, torques and muscle activity. We then conclude and give a summary of the studied parameters and their potential use in walking in VR.

2 Sensing and Interpreting Global Gait Parameters

Human being can be represented by more or less complex models. The simplest one consists in considering human being as a point which corresponds to his center of mass. Analyzing human walking with such a model consists in dealing with global gait parameters such as trajectory, velocity, step length, and frequency. It provides us with relevant information about the global performance of gait. As described in Chap. 3 those parameters enable us to associate the real-time performance of the user with multisensory feedbacks, such as adapting the movement of the virtual camera according to velocity when navigating in virtual environments [35]. In this section, we describe how these parameters are defined and measured.

2.1 Step Length and Frequency

When dealing with human walking, one can focus on global parameters such as step length \(\mathrm{{S}}_{\mathrm{{L}}}\) and frequency \(\mathrm{{S}}_{\mathrm{{F}}}\). These two parameters are in relation with walking speed V:

$$\begin{aligned} V=S_L *S_F \end{aligned}$$

As the model is restricted to the user’s center of mass, walking speed can be approximated by integrating the signal delivered by an accelerometer placed on the pelvis (assuming that the center of mass is close to the Pelvis). It’s thus possible to deduce the instantaneous velocity and to drive a virtual camera in the virtual world in navigation tasks [35]. However this approach has two main limitations. Firstly it assumes that acceleration is noise-free which is not true with common accelerometers. As a consequence velocity computed this way may become false after a moment. Secondly the initial velocity is required when computing \(V(t)\):

$$\begin{aligned} V\!\left( t \right) =\left( {\mathop \sum \limits _{i=0}^n a\!\left( {t_i } \right) *\varDelta t} \right) +V\!(0) \end{aligned}$$

where \(\varDelta t\) is the sample time, \(V({ 0})\) is the initial velocity, n is the number of samples (\(t_{n}\) corresponds to t) and \(a\) stands for accelerations provided by the accelerometer. Any error in estimating \(V({ 0})\) could thus lead to an error in \(V(t)\). One has to notice here that users generally have a limited space to move whereas the virtual environment could be very large. To solve this problem, one of the most famous solutions consists in using an instrumented treadmill. In that case, forward speed could be delivered by the treadmill while the other components could be given by the accelerometer.

An alternative consists in detecting footstrikes in the signal delivered by the accelerometer and to deduce step frequency \(\mathrm{{S}}_{\mathrm{{F}}}\). If we assume that the step length is constant and relative to the user’s size, it is thus possible to deduce speed. The resulting speed is less noisy than integrating acceleration and is not subject to deviations in time. However it does not provide accurate speed as the step length is not actually measured.

Other systems such as the GAITRite (see http://www.gaitrite.com) have been introduced to measure the step length and frequency when walking along limited distances. It consists of a cable which is attached to the user’s ankles and which length is measured at a predetermined sampling frequency. It is widely used in medicine because of its simplicity (especially no calibration is required). In addition to \(\mathrm{{S}}_{\mathrm{{L}}}\) and \(\mathrm{{S}}_{\mathrm{{F}}}\), the system returns the instantaneous distance between a fixed reference frame and the two ankles. It is thus possible to analyse the longitudinal trajectory of the ankles within the gait cycle. However, it is limited to straight line walking in a limited space (generally a few meters).

Recording step length and step width is more difficult in curved walks. As described in Chap. 3 these parameters are still difficult to define in a strict manner.

2.2 Curvature and Non-linear Walking

Retrieving the curvature of non-linear walking is still a complex problem. Indeed, the instantaneous global orientation of the body is difficult to define. As explained in Chap. 3 some authors focuses on the footprints, the orientation of the pelvis, the torso or the head. It leads to different results for determining the global trajectory of non-linear walking. Hence, positioning a unique sensor on the body to accurately analyze the instantaneous orientation of the body in curved walking is still debated. The most popular approach consists in tracking the orientation of the pelvis. Hence placing a sensor on the pelvis, such as inertial, magnetic or gyroscope sensors (or a combination of these sensors), could provide a good compromise. Indeed, some authors have shown that the pelvis orientation can provide an early prediction of the running orientation of a rugby player even in deceptive motions where the subject tries to provide fake information to an opponent [4]. The pelvis’ trajectory is thus strongly linked to the actual orientation of the user while other body parts may also perform other tasks while walking.

In the same way, detecting a change in walking direction is very difficult. Indeed, the trajectory of the center of mass and of a point placed on the pelvis could be viewed as a series of arcs of circles, even for straight-line walking. Some authors [24] have proposed to model the natural sinusoidal trajectory of the center of mass as a sequence of arcs of circles (see Fig. 8.1). This model assumes that the body could be viewed as an inverted pendulum which basis is the contact foot.

Fig. 8.1
figure 1

The trajectory of the center of mass is modeled as a sequence of arcs of circles which parameters are the radius of gyration \(\mathrm{{R}}_{\mathrm{{i}}}\) and the velocity (represented here by the angle \(\uptheta _{i}=(\mathrm{{V}}_{\mathrm{{i}}}*{\text{ duration }})/\mathrm{{R}}_{\mathrm{{i}}}\) (where \(\mathrm{{V}}_{\mathrm{{i}}}\) is the average speed within the ith arc). Adapted from [24]

Full size image

The authors defined a statistical relation between speed and curvature for straight-line walking. Thus turning could be determined when a new point in the speed-curvature space does not fit this relation.

This is a very important issue in immersive environments as we need to detect a subtle change in direction when the subject is walking in order to react in a proper manner when he wishes to change the walking direction. In some works in virtual reality, it leads to exaggerating some indices such as the head inclination in Walk-In-Place interfaces [34].

As shown by many authors [28] the head is viewed as a stable inertial platform which acceleration profile is unchanged even for various gait styles [11] and for uneven terrains [21]. To stabilize this inertial platform, the orientation of the head changes in direction 200 ms before the remaining of the body turns. In case of natural walking, the coordination between head and pelvis could enable us to early determine the orientation of the next step. However, in VR, this could be a problem if the screen is static and placed in front of the user. As the user will orient his head in the direction of the screen this natural behavior disappears. Because of constraints due to the visual feedbacks and other interaction devices, it seems to be difficult to address this problem without introducing metaphors.

An alternative might be using a force plate under the feet which can bring relevant information of the user’s gait. In biomechanics, force plates are used to measure the ground reaction force (GRF) below the feet, the moment of this force around the main ground axes and the location of the instantaneous center of pressure (COP). The ground reaction force is used to compute the acceleration of the center of mass if no other force than gravity and GRF occur. For the global mechanical system (restricted to its center of mass):

$$\begin{aligned} W + { GRF} = m\ddot{q} \end{aligned}$$

where W stands for the body weight, m is the mass and q is the center of mass position. If we can estimate q and its derivative at some times (especially the initial value but it is sometimes possible to get these values for each foot-strikes event) it is thus possible to integrate the signal:

$$\begin{aligned} q(t)=\frac{1}{m}\int \!\!\int _{t_0 }^t W+{ GRF}(\tau )d\tau ^{2} \end{aligned}$$

where \(\mathrm{t}_{0}\) states for the beginning of the studied sequence. Practically, the initial center of mass velocity \( \dot{q}(t_{0})\) and position \(q(t_0)\) are required to compute \(q(t)\). As a consequence these two values should be either measured or imposed at the beginning of the motion (such as starting straight with a null velocity). With a simple Euler integration scheme, this equation becomes:

$$\begin{aligned} \dot{q}\!\left( n \right)&=\dot{q}\!\left( 0 \right) +\mathop \sum \limits _{i=1}^n \left( {\frac{1}{m}\left( {W+{ GRF}\!\left( i \right) } \right) \Delta t} \right) \\ q\!\left( n \right)&=q\!\left( 0 \right) +\mathop \sum \limits _{i=1}^n \dot{q} (i)\Delta t \end{aligned}$$

where \(\Delta \mathrm{t}\) stands for the sampling frequency.

A force plate is generally calibrated at the beginning of the sequence so that the value is set to zero when nothing is placed over it. However, if the user jumps over the forceplate or goes out and in several times, the initial calibration could be inappropriate. From the numerical point of view, we obtain:

$$\begin{aligned} \dot{q}\!\left( n \right) =\left\{ {{\begin{array}{c} {\dot{q}\!\left( 0 \right) +\mathop \sum \limits _{i=1}^{t_s -1} \frac{1}{m}\left( {W_0 +{ GRF}\left( i \right) } \right) \times \Delta t \quad if \ t<t_s } \\ {\dot{q}\!\left( 0 \right) +\mathop \sum \limits _{i=1}^{t_s -1} \frac{1}{m}\left( {W_0 +{ GRF}\left( i \right) } \right) \times \Delta t+\mathop \sum \limits _{i=t_s }^n \frac{1}{m}\left( {W_{t_s } +{ GRF}\!\left( i \right) } \right) \times \Delta t} \\ \end{array} }} \right. \end{aligned}$$

where \(\mathrm{{W}}_{0}\) is the initial body weight measured prior to an important strike, \(\mathrm{{t}}_{\mathrm{{s}}}\) is the frame number when the strike occurs, and \(\mathrm{{W}}_{\mathrm{{ts}}}\) is the body weight after foot trike. It is thus possible to deduce oscillations of the center of mass. It could be used in Walk-In-Place interfaces based on GRF measurements (such as using a Nintendo Wii Balance Board). Information about other axes provides interesting information for navigation, such as the velocity in the forward direction or lateral displacements involved in rotations. However, they are more difficult to use in virtual reality as they require the subject to move in all the directions which generally leads to going outside the measurement area in a very near future (generally the next step).

One has to notice here that the two feet must be placed over the forceplate surface to enable correct estimation of the above displacements. It is also possible to use footscan devices (from RSscan company http://www.rsscan.com) to measure the pressure below the feet. It consists in introducing a sole in the shoe which enables the user to move freely in the real environment. This type of device delivers the pressure in space and time which enables us to compute the resulting vertical component of GRF. The other components cannot be accurately deduced but the location of the center of pressure (COP) may help to guess how this force is oriented in the other main directions.

COP can also be deduced with a forceplate if this latter provides 3D GRF and the corresponding global momentum M over the ground. M and GRF are linked by:

$$\begin{aligned} \left\{ {{\begin{array}{l} {M_x ={ GRF}\!_z \times y_{{ COP}} -{ GRF}\!_y \times z_0 } \\ {M_y =-{ GRF}\!_z \times x_{{ COP}} +{ GRF}\!_x \times z_0 } \\ \end{array} }} \right. \end{aligned}$$

where \(\mathrm{{x}}_{\mathrm{{COP}}}\) and \(\mathrm{{y}}_{\mathrm{{COP}}}\) stand for the local coordinates of the COP in the forceplate reference frame, and \(z_{0}\) stands for the vertical coordinate of the forceplate surface. It becomes:

$$\begin{aligned} \left\{ {{\begin{array}{l} {x_{{ COP}} =-\frac{(M_y -{ GRF}\!_x \times z_0 )}{{ GRF}\!_z }} \\ {y_{{ COP}} =\frac{(M_x +{ GRF}\!_y \times z_0 )}{{ GRF}\!_z }} \\ \end{array} }} \right. \end{aligned}$$

In quasi-static condition, it is possible to assume that the average value of COP is the projection of the center of mass on the ground. Hence if COP moves the pose of the user has changed, for example to lean in a given direction. This type of information has been used in biomechanics to determine if a subject was turning while walking [24, 34]. It can thus be used to indicate a main direction so that the user could be viewed as a kind of joystick. It is used in some Nintendo Wii games based on the Balance Board device (one has to notice that this device is not a force plate as it does not provide the system with 3D GRF and Momentums ; it provides the vertical component of the GRF and the location of the COP). The Joyman interface [19, 26] is an extension of the idea of using the user’s body as a joystick.

2.3 Gait Asymmetry and Regularity

As stated above, walking is a quasi-cyclic and almost symmetrical motion. Measuring gait asymmetry and loss of regularity is widely used for diagnosis [17], especially in double-tasks protocols with cognitive loads (such as counting down while walking). There are two main methods to measure symmetry. The simplest one consists in computing a ratio between left and right values [30]:

$$\begin{aligned} \left[ {\frac{\left( {R-L} \right) }{0.5}\times (R+L)} \right] \times 100 \end{aligned}$$

where R and L are respectively measurements performed on the right and the left respectively (such as step length or step frequency). The other approach consists in computing auto-correlation of the studied signal [2]. This signal is generally twice the frequency of the stride. Hence auto-correlation of this signal between one step and the following will provide information on asymmetry. Auto-correlation between several strides will provide information on regularity.

As described above irregularity and asymmetry mainly occur when the user has to perform a cognitive activity while walking. In immersive systems based on metaphors, the cognitive load of the user may be different compared to natural walking. Regularity and symmetry may then be affected even if it has not been demonstrated in VR yet.

3 Joint Angles, Torques and Muscle Activity

In the previous section human body was modelled as a point (his center of mass). We have seen that it could provide relevant information about the main gait pattern, such as speed, step length and frequency. In VR this type of information could be used to globally adapt the motion of the virtual camera or to capture basic gait parameters to drive the simulation. However more accurate information could be required in order to drive an avatar or capture more accurate information about the user’s gait. To this end, it is necessary to have multi-point measurements and to capture more complex parameters, such as joint angles and position, but also to compute dynamic parameters such as joint torques or muscle activation patterns.

3.1 Measuring Joint Displacements

The most popular method to get joint position depending on time is to track visual markers placed over standardized anatomical landmarks. The International Society of Biomechanics (ISB) has standardized this markers’ placement in order to enable people to compare their results to previously published ones [39, 40]. The key idea is to place markers on accurate anatomical position where skin markers do not significantly slide over the bones. The problem is then to retrieve the location of the internal joints according to the external position of the markers. In this chapter we describe the methods commonly used to process motion capture data in order to compute joint centers and angles.

Let us consider that the position of all the markers \(\mathrm{{m}}_{\mathrm{{i}}}\) is known (assuming that there is no missing information due to occlusions) as shown in Fig. 8.2. There are mostly three approaches to deduce the joint centers according to this marker placement. The first approach consists in applying regressions that express joint centers as a linear function of external markers’ positions [12, 29]. For example, the right shoulder joint rShoJC could be expressed as:

$$\begin{aligned} \left\{ {{\begin{array}{c} {{ rShoJC}_x ={ RSHO}_x } \\ {{ rShoJC}_y ={ RSHO}_y +0.43 \cos (\dfrac{11\pi }{180})||{ CLAV}-C7||} \\ {{ rShoJC}_z ={ RSHO}_z -0.43 \cos (\dfrac{11\pi }{180})||{ CLAV}-C7||} \\ \end{array} }} \right. \end{aligned}$$

where RSHO, CLAV and C7 are markers depicted in Fig. 8.2.

Fig. 8.2
figure 2

Common marker placement used in Biomechanics [39, 40] to retrieve joint centers and to use anthropometric tables

Full size image

The main advantage of this type of method is simplicity. However this approach is not very accurate as it is based on average values while anthropometric data in humans can vary very significantly from one user to another.

The other approach, named functional approach, consists in searching for the joint centers that would generate the observable displacements [5, 7, 9]. For example, the right hip joint (rHip) is assumed to be a ball and socket. Its joint center should be the center of a sphere that covers the various positions of a point of the femur expressed in the pelvis reference frame. For example, any position of RKNE should satisfy the following constraint in the pelvis reference frame:

$$\begin{aligned} \left( {{ RKNE}_x -{ rHip}_x} \right) ^{2}+\left( {{ RKNE}_y -{ rHip}_y} \right) ^{2}+\left( {{ RKNE}_z -{ rHip}_z} \right) ^{2}-l^{2}=0 \end{aligned}$$

where l is the distance between the hip joint center and RKNE (length of the femur). Thus recovering the joint center consists in solving an optimization problem:

$$\begin{aligned} argmin_{{ rHip},l}&\left[ \left( {{ RKNE}_x (i)-{ rHip}_x } \right) ^{2} +\left( {{ RKNE}_y (i)-{ rHip}_y } \right) ^{2}\right. \\&\quad +\left. \left( {{ RKNE}_z (i)-{ rHip}_z } \right) ^{2}-l^{2} \right] ^{2} \end{aligned}$$

Of course, to find a good solution, RKNE should have large displacements in all the possible directions, in order to cover most of the sphere’s surface. As a consequence, this method is generally applied to “range of motion” protocols where the user is moving each joint in all directions and with large displacements. However if the hip joint is actually not a ball and socket joint, the result could be inaccurate. Because of skin displacements artefacts, this method will also provide the system with virtual bones that could change in length at each frame.

Another approach consists in generalizing the above idea to the whole skeleton. Let us consider that there exists a model of the skeleton based on rigid bodies and perfect joints (such as ball and socket and pivot joints). Knowing the external markers \(\mathrm{{m}}_{\mathrm{{i}}}^{*}\) in each local reference frame, the distance between \(\mathrm{{m}}_{\mathrm{{i}}}^{*}\) and \(\mathrm{{m}}_{\mathrm{{i}}}\) depends on the size of the body segments and the angles \(\uptheta =\{\uptheta _{1}\ldots \mathrm{{n}}\}\) (where n is the number of degrees of freedom of the model). If there are enough markers on the body compared to the number of unknowns (i.e. number of degrees of freedom and of body segments), the problem can again be rewritten as a global optimization problem [15]:

$$\begin{aligned} argmin_{\theta ,l} \frac{1}{2}\left( {m-m^{*}} \right) ^{T}\left( {m-m^{*}} \right) \ with \ m^{*}=f(\theta ,l) \end{aligned}$$

where l is the vector containing the lengths of all the body segments, m is the number of markers and f is the kinematic function that computes the estimation of a marker placement according to the angles and the lengths of each body segment. As a result, joint angles are known together with the length of each body segment. Again the hypothesis here is that the joint are perfect mechanical joints.

In most of the applications in VR high accuracy is not needed when measuring joint positions in time. However animating avatars with such inaccurate data generally leads to artifacts, such as foot skating, flying avatars or collisions with the ground.

3.2 Measuring Joint Angles

In most applications involving motion capture data, measuring joint position is only a first step. Avatars are driven with joint angles and not with positions. Magnetic or inertial motion capture systems provide the user with global orientation of sensors attached to body segments. However, because of inaccuracies, local displacements of the sensor on the body segments and external perturbations, the data provided by these systems should be corrected (see [3, 22, 33] for the specific case of magnetic sensors).

Let us consider now how to compute joint angles according to local reference frames defined either thanks to the ISB recommendations [39, 40] or the H-ANIM norm (see http://www.h-anim.org for a description). If we use the Euler-like angles for a ball-and-socket joint, the problem consists in finding the three angles \(\uptheta _{\mathrm{{x}}},\uptheta _{\mathrm{{y}}}\) and \(\uptheta _{\mathrm{{z}}}\) that transform the father reference frame \(\mathrm{{F}}_{\mathrm{{i}}}\) (such as the one attached to the pelvis) to the child one \(\mathrm{{F}}_{\mathrm{{j}}}\) (such as the one attached to the thigh), as depicted in Fig. 8.3.

Fig. 8.3
figure 3

Local reference frames associated with two adjacent body segments: \(\mathrm{{F}}_{\mathrm{{i}}}\) the parent segment (such as the pelvis) and \(\mathrm{{F}}_{\mathrm{{j}}}\) the child segment (such as the thigh)

Full size image

Let \(\Omega \) be the global reference frame. Let \(\mathrm{{X}}_{\mathrm{{i}}}, \ \mathrm{{Y}}_{\mathrm{{i}}}\) and \(\mathrm{{Z}}_{\mathrm{{i}}}\) (resp. \(\mathrm{{X}}_{\mathrm{{j}}},\ \mathrm{{Y}}_{\mathrm{{j}}}\) and \(\mathrm{{Z}}_{\mathrm{{j}}})\) be the three axis of reference \(\mathrm{{F}}_{\mathrm{{i}}}\) (resp. \(\mathrm{{F}}_{\mathrm{{j}}}\)). Let us define:

$$\begin{aligned} T_{i\rightarrow {\Omega }} =\left( {{\begin{array}{lll} {X_i }&{} {Y_i }&{} {Z_i } \\ \end{array} }} \right) \end{aligned}$$
$$\begin{aligned} T_{j\rightarrow {\Omega }} =\left( {{\begin{array}{lll} {X_j }&{} {Y_j }&{} {Z_j } \\ \end{array} }} \right) \end{aligned}$$

where \(\mathrm{T}_{\mathrm{{i}}\rightarrow \Omega }\) (resp. \(\mathrm{T}_{\mathrm{{j}}\rightarrow \Omega })\) stands for the transformation matrix from the local frame \(\mathrm{{F}}_{\mathrm{{i}}}\) (resp. \(\mathrm{{F}}_{\mathrm{{j}}})\) to the world \(\Omega \). These matrixes can be computed according to the external markers and joint centers for each time:

$$\begin{aligned} T_{j\rightarrow {\Omega }} =T_{j\rightarrow {i}} \times T_{i\rightarrow {\Omega }} \end{aligned}$$

where \(\mathrm{T}_{\mathrm{{j}}\rightarrow \mathrm{{i}} }\) stands for the transformation matrix from \(\mathrm{{F}}_{\mathrm{{j}}}\) to \(\mathrm{{F}}_{\mathrm{{i}}}\), i.e. the transformation due to the action of the joint between \(\mathrm{{F}}_{\mathrm{{i}}}\) and \(\mathrm{{F}}_{\mathrm{{j}}}.\,\mathrm{{T}}_{\mathrm{{j}}\rightarrow \mathrm{{i}}}\) is thus given by:

$$\begin{aligned} T_{j\rightarrow {i}} =T_{j\rightarrow {\Omega }} \times {}^T T_{i\rightarrow {\Omega }} =\left( {{\begin{array}{lll} {t_{11} }&{} {t_{12} }&{} {t_{13} } \\ {t_{21} }&{} {t_{22} }&{} {t_{23} } \\ {t_{31} }&{} {t_{32} }&{} {t_{33} } \\ \end{array} }} \right) \end{aligned}$$

In the numerical model, this matrix is the product of three elementary symbolic matrixes associated with each degree of freedom and which depends on the chosen sequence of Euler angles. Let \(\mathrm{{T}}_{\mathrm{{x}}},\mathrm{{T}}_{\mathrm{{y}}}\) and \(\mathrm{{T}}_{\mathrm{{z}}}\) be the elementary matrixes for a rotation along the X, Y and Z axes respectively. From the theoretical point of view, for a ZYX sequence (recommended by the International Society of Biomechanics), \(\mathrm{{T}}_{\mathrm{{j}}\rightarrow \mathrm{{i}}}\) could also be expressed as the product:

$$\begin{aligned} T_{j\rightarrow i}^*=T_x \times T_y \times T_z \end{aligned}$$

The resulting matrix has the following shape:

$$\begin{aligned} T_{j\rightarrow i}^{*} =\left( {{\begin{array}{ccc} {\cos \, \left( {\theta _y } \right) \cos \, (\theta _z )}&{} {-\cos \, (\theta _y )*\sin \, (\theta _z )}&{} {\sin \, (\theta _y )} \\ \cdots &{} \cdots &{} {-\cos \, \left( {\theta _y } \right) \sin \, (\theta _x )} \\ \cdots &{} \cdots &{} {\cos \, \left( {\theta _x } \right) \cos \, (\theta _y )} \\ \end{array} }} \right) \end{aligned}$$

As \(\mathrm{{T}}_{\mathrm{{j}}\rightarrow \mathrm{{i}}}^{*}\) should be equal to \(\mathrm{{T}}_{\mathrm{{j}}\rightarrow \mathrm{{i}}}\):

$$\begin{aligned} {\theta _z}&=-\mathrm{{a}}{\tan } \left( {\frac{t_{12} }{t_{11} }} \right) \\ \theta _x&=-\mathrm{{a}}{\tan } \left( {\frac{t_{23} }{t_{33} }} \right) \\ \theta _y&= \mathrm{{a}}{\tan } \left( {\frac{t_{13} \times {\cos }(\theta _z )}{t_{11}}} \right) \end{aligned}$$

When using this type of approach, the results are very sensitive to the well-known Gimble-Lock problem. In the above equation, if \(\uptheta _{y}=\uppi /2,\) then the denominator of the three equations is zero which leads to numerical problems. Other researchers use quaternions or exponential maps to avoid this problem. Whatever the selected formalism for the angles, noise or inaccuracies have a high impact on the result. In biomechanics more and more researchers propose to use a higher number of markers (named cloud of markers) and to apply the method described in the previous section: global optimization to fit the simulated markers \(\mathrm{{m}}_{\mathrm{{i}}}^{*}\) to the measured ones \(\mathrm{{m}}_{\mathrm{{i}}}\) [10].

Whatever the method, joint angles give a large quantity of information about the user’s gait. These angles are directly used to animate avatars or to analyse if a user is avoiding a virtual obstacles, which is almost impossible if just considering parameters linked to the user’s center of mass. Another application is to evaluate the quality of the user’s gait when walking in virtual environments. For example, it has been used to demonstrate that treadmill walking in VR affects gait [31]. Moreover some authors have shown that joint angles were also adapted during nonlinear walking [25]. It could thus be used to detect turning in walk-in-place interfaces or to adapt the motion of an avatar in real-time when turning.

3.3 Estimating Joint Torques with Inverse Dynamics

Some authors have demonstrated that even if kinematic gait parameters were similar in ground and treadmill walking after some training, the joint torques and muscular activity remains different [14]. Taking this type of parameter into account could thus help to evaluate the performance of the user’s gait in virtual environment, using or not a treadmill. Indeed, this type of parameter seems to enable us to highlight unperceivable modifications of gait even if joint angles look similar. However, there is no direct method for measuring joint forces and torques. Indirect methods named “inverse dynamics” have been introduced to compute these forces and torques according to kinematic data associated with a physical model of the skeleton. There are merely two main approaches to address this problem: isolated segments and use of controllers.

4 Isolated Segments

The first approach is based on the Newton formalism and consists in considering each body segment separately, assuming that the mechanical system is limited to body segment \(\mathrm{{S}}_{\mathrm{{i}}}\) [16, 38]. For each body segment \(\mathrm{{S}}_{\mathrm{{i}}}\) external forces could be separated into two families: external forces and torques (\(\mathrm{{F}}_{\mathrm{{e}}}^{\mathrm{{i}}},\uptau _{\mathrm{{e}}}^{\mathrm{{i}}})\) exerted by the environment (such as gravity and contacts), and the internal ones due to muscles (\(\mathrm{{F}}_{\mathrm{{m}}}^{\mathrm{{i}}},\uptau _{\mathrm{{m}}}^{\mathrm{{i}}})\). Thanks to kinematic data it is possible to compute the body segment center of mass acceleration. The Newton equation gives:

$$\begin{aligned} \left\{ {{\begin{array}{c} {m_i \gamma _i =F_e^i +F_m^i } \\ {\frac{dL_i }{dt}=\mathcal{M}_{F_e^i } +\mathcal{M}_{F_m^i } +\tau _e^i +\tau _m^i } \\ \end{array} }} \right. \end{aligned}$$

where \(\upgamma _{i},{\mathrm{{m}}}_{\mathrm{{i}}}\) and \({\mathrm{{L}}}_{\mathrm{{i}}}\) stand for the acceleration of segment \(\mathrm{{S}}_{\mathrm{{i}}}\) center of mass, its mass and its angular momentum respectively. Hence knowing the external forces, torques, the mass and the acceleration of the body segment it is possible to deduce (\(\mathrm{{F}}_{\mathrm{{m}}}^{\mathrm{{i}}}, \uptau _{\mathrm{{m}}}^{\mathrm{{i}}})\) if there is only one unknown for each equation. It means that there is only one force \(\mathrm{{F}}_{\mathrm{{m}}}^{\mathrm{{i}}}\) and only one muscle torque \(\uptau _{\mathrm{{m}}}^{\mathrm{{i}}}\). However for a given segment \(\mathrm{{S}}_{\mathrm{{i}}}\) with two adjacent segments \(\mathrm{{S}}_{\mathrm{{i-1}}}\) and \(\mathrm{{S}}_{\mathrm{{i+1}}},\mathrm{{F}}_{\mathrm{{m}}}^{\mathrm{{i}}}\) is the result of two forces: the forces exerted respectively by each neighboured segment at contact point.

To solve this problem, there are two main methods. The first one consists in solving the system from the extremities (without contact, such as the hands and the head) to the ground. For body segments placed at the extremities of the skeleton, there is only one unknown for \(\mathrm{{F}}_{\mathrm{{m}}}^{\mathrm{{i}}}\) and \(\uptau _{\mathrm{{m}}}^{\mathrm{{i}}}\) (associated with the proximal joint attached to the body segment). \(\mathrm{{F}}_{\mathrm{{m}}}^{\mathrm{{i}}}\) and \(\uptau _{\mathrm{{m}}}^{\mathrm{{i}}}\) could thus be rewritten \(\mathrm{{F}}_{\mathrm{{m}}}^{\mathrm{{j}}\rightarrow \mathrm{{j}}}\) and \(\uptau _{\mathrm{{m}}}^{\mathrm{{j}}\rightarrow \mathrm{{i}}}\) to express their relation to the joint attached to segments i and j. The problem can then easily be solved by inverting the above equation:

$$\begin{aligned} \left\{ {{\begin{array}{c} {F_m^{j\rightarrow i} =F_e^i -m_i \gamma _i } \\ {\tau _m^{j\rightarrow i} =\mathcal{M}_{F_e^i } +\mathcal{M}_{F_m^i } +\tau _e^i -\frac{dL_i }{dt}} \\ \end{array} }} \right. \end{aligned}$$

When dealing with segment \(\mathrm{{S}}_{\mathrm{{j}}}\), we can then reuse these results as known forces \(\mathrm{{F}}_{\mathrm{{m}}}^{\mathrm{{j}}\rightarrow \mathrm{{i}}}=-\mathrm{{F}}_{\mathrm{{m}}}^{\mathrm{{i}}\rightarrow \mathrm{{j}}}\) and torques \(\uptau _{\mathrm{{m}}}^{\mathrm{{j}}\rightarrow \mathrm{{i}}} =-\uptau _{\mathrm{{m}}}^{\mathrm{{i}}\rightarrow \mathrm{{j}}}\). The method is applied until the feet so that the GRF could be deduced at contact point with the ground. Comparison with measured GRF is a common method to estimate the errors due to this process. The second method, named bottom-up method, consists in starting with the segments which are in contact with the environment (and which external forces are known) and finishing with the extremities free of any contact. A mixed method can also be used.

5 Global System and Controllers

A second approach to solve inverse dynamics problems is to model the global system with the Lagrangian formalism (based on the principle of virtual works) in order to obtain the motion equations, including the torques at any joints \(\uptau _{\mathrm{{i}}}\):

$$\begin{aligned} \frac{d}{dt}\frac{\partial C}{\partial \dot{q}_{i} }-\frac{\partial C}{\partial {q}_{i} }-Q_i^{\prime } =0 \ with \ i=1\ldots n \end{aligned}$$

where C is the difference between the potential and kinetic energies, and \(\mathrm{{q}}_{\mathrm{{i}}}\) is the ith state of the system. The applied forces and torques \(\mathrm{{Q}}_{\mathrm{{i}}}\) are expressed in the generalized coordinates as generalized forces,

Lagrangian multipliers are added to this equation in order to express that body segment displacement should correspond to motion capture data (see [8] for details). The model must be associated with a robust representation of contact forces with the ground to deliver realistic results. When simulating the system with a dynamic solver, Lagrangian multipliers will naturally compute forces and torques that are necessary to ensure that the resulting simulation is compatible with the imposed motion (generally motion capture data).

A famous solution consists in modelling the joint torques as proportional-derivative (PD) controllers which merely consists in associating damped springs to each joint:

$$\begin{aligned} \tau _i =k_p \left( {\theta _i^d -\theta _i } \right) -k_d \dot{\theta }_{i} \end{aligned}$$

where \(\uptheta _{\mathrm{i}}^{\mathrm{d}}\) stands for the desired joint angle for joint i, \(\mathrm{{k}}_{\mathrm{{p}}}\) and \(\mathrm{{k}}_{\mathrm{{d}}}\) are the proportionnal and derivative gains of the controller. If we consider that the desired joint angles correspond to those measured by the motion capture system, and if the gains are correctly tuned, it is thus possible to compute the joint torques that are required to perform the measured motion [42]. Torques obtained with the PD controller is applied to a physical model of the human body. This physical model can be obtained thanks to commercial software or opensource packages, such as OpenDynamicEngine (http://www.ode.org). This type of software provides us with a simulator of a physical model which inputs are internal and external forces applied to the system. In our case, external forces are obtained either by direct measurements with gauges or by using the above inverse dynamic method applied to the global whole-body system (human body is modelled by its center of mass). Internal forces and torques are computed using the PD controllers.

This approach is very difficult to tune, especially the values of the PD gains. However for well-known motions such as walking, many researchers have proposed semi-automatic methods to estimate these gains.

6 Conclusion About Inverse Dynamic Approaches

Joint torques and forces are mainly used in biomechanics and computer animation. In biomechanics it enables to distinguish different motor strategies that kinematic data fail to differentiate. In computer animation, it is mainly used to check if the square of joint torques of a given simulated motion is minimized, assuming that it corresponds to natural motions. When walking in VR, it could also help to evaluate gait efficiency as human walking is supposed to be associated with low energy consumption (see Chap. 3). For example, it has been used as a relevant criterion to distinguish overground and treadmill walking, as explained above [31]. Moreover animating the user’s avatar walking on uneven terrain implies to adapt the joint trajectories performed by the user who is walking on a treadmill or a flat ground. Hence this motion adaptation is necessary to compensate differences between the constraints imposed in the virtual environment and those actually linked to the real movement of the user. In that case methods based on physical models enable to automatically adapt the motion to different kinematic and physical constraints [41].

However inverse dynamics is sensitive to noise or inaccuracies especially when computing joint accelerations. Another limitation is that it remains difficult to deal with closed-loop systems, such as dealing with the double support phase in walking where the two feet are in contact with the ground. In that case, it is very difficult to strictly separate the forces exerted below each foot.

6.1 Measuring or Estimating Muscle Activities

In some very specific applications, joint torques is not accurate enough to understand motion strategies. Indeed, joint torques provides us with the resulting action of a group of muscles whereas control strategies could have a direct link with the action of one isolated muscle. Slightly changing the axis of rotation of a motion may recruit different muscle groups even if the resulting joint torque looks the same.

The direct approach to measure muscle activity consists in sensing the electromyograms (EMG) of targeted muscles. EMG is a measurement of the electrical activity of skeletal muscles recorded with the placement of small electrodes over the skin (there exist more invasive electrodes but they are unusable for large movements). Thus EMG is limited to surface muscles. The signal returned by the EMG system is noisy and required heavy signal processing to estimate muscle tensions in Newtons. However, it gives an interesting point of view about muscle coordination if several muscles are measured concurrently. See [23] (among many others) for more details on EMG.

Some researchers have proposed to use indirect methods to retrieve the tension of all the muscles involved in the studied motion (including deeper muscles). Musculoskeletal models have been introduced in the early nineties [6] thanks to the increase of computation power of computers. The key idea is to model muscles thanks to action lines acting along an axis determined by to muscle insertions. Knowing the accurate location of each of these muscles and tendons insertion on bones, it is possible to retrieve these action lines, as shown in Fig. 8.4.

Fig. 8.4
figure 4

Action line joining two adjacent segments \(\mathrm{{S}}_{\mathrm{{i}}}\) and \(\mathrm{{S}}_{\mathrm{{j}}}\). The insertions of this action line in both segments are given by anthropometric tables in each local reference frame

Full size image

A muscle is supposed to work only by applying a positive tension (leading to contractions) and cannot push the bone. Hence for each muscle i, its tension \(\mathrm{{T}}_{\mathrm{{i}}}\) is positive. Each muscle is also limited to a maximum voluntary contraction (MVC) which is generally evaluated in isometric condition (i.e. no displacement of the bones but exertion of a force against an external load). If we consider the surface of the cross section area (perpendicular to the muscle fibers), this MVC is given by:

$$\begin{aligned} F_i^{max} =K\!(l)\times K_0 \times { PCSA} \end{aligned}$$

where \(\mathrm{{K}}_{0}\) is a constant ranging from 15 to 33 N/cm\(^{2}\) according to the authors, and K(l) is a value that depends on the muscle length l. Indeed there exists a relation between the maximum delivered force and the length of a muscle (named Force–Length relation). This relation is given in the biomechanical literature. Hence a muscle tension \(\mathrm{{F}}_{\mathrm{{i}}}\) for muscle i is constrained by:

$$\begin{aligned} 0\le F_i \le K(l)\times K_0 \times { PCSA} \end{aligned}$$

l is then the distance between the two insertion points of the action line. Motion capture data enable to associate a local reference frame to each body segment, as shown previously for the computation of joint angles. It is thus possible to apply anthropometric tables that give the local coordinates of each line action insertion in each bone. These tables are generally embedded in dedicated software, such as OpenSim (see http://simtk.org/home/opensim for an example).

Knowing these insertion points for each action line, we can deduce the length of each action line and consequently its MVC in the current pose. If external torques \(\uptau _{\mathrm{{ext}}}\) are also known, such as gravity and the momentum of all the contact forces, the torques applied by the muscles \(\uptau _{\mathrm{{musc}}}\) in static condition are given by:

$$\begin{aligned} \tau _{ext} =-\tau _{musc} \end{aligned}$$

In the remaining of this section we assume that the system is static for simplification but extension to dynamic situation simply involves adding to this formulae the terms linked to the derivative of the angular momentum of each body segment.

\(\uptau _\mathrm{ext}\) could be expressed for each body segment i as:

$$\begin{aligned} \forall i\in \left[ {1,n} \right] \!,\mathop \sum \limits _{j=1}^m M_i^j =-\tau _{ext}^i \end{aligned}$$

where n is the number of body segments, m is the number of independent action lines, and \(\mathrm{{M}}_{\mathrm{{i}}}^{\mathrm{{j}}}\) stands for the moment of action line j on the ith body segment. If the local reference frame \(\mathrm{{O}}_{\mathrm{{i}}}\) of segment i is placed on the proximal joint:

$$\begin{aligned} M_i^j =O_i P_i^j \times F_j \end{aligned}$$

where \(\mathrm{{P}}_{\mathrm{{i}}}^{\mathrm{{j}}}\) is the insertion point of muscle j on body segment i and \(\mathrm{{F}}_{\mathrm{{j}}}\) stands for the force applied by this muscle. Let us consider that the norm of \(\mathrm{{F}}_{\mathrm{{j}}}\) is denoted \(\mathrm{{T}}_{\mathrm{{j}}}\). Using the superposition theorem we can express \(\mathrm{{M}}_{\mathrm{{i}}}^{\mathrm{{j}}}\) as the product of \(\mathrm{{Tj}}\) and \(\mathrm{{I}}_{\mathrm{{i}}}^{\mathrm{{j}}}\) where \(\mathrm{{I}}_{\mathrm{{i}}}^{\mathrm{{j}}}\) is the theoretical Momentum associated with a unit force vector \(\mathrm{{Fj}}\) (\(\parallel \mathrm{{F}}_{\mathrm{{j}}} \parallel = 1\!N\)):

$$\begin{aligned} M_i^j =I_i^j *T_j \end{aligned}$$

and

$$\begin{aligned} \forall i \in \left[ {1,n} \right] ~\quad \mathop \sum \limits _{{j = 1}}^{m} I_{i}^{j} *T_{j} = - \tau _{{ext}}^{i} \end{aligned}$$

which can be rewritten in a matrix form:

$$\begin{aligned} \beth .T_{{musc}} = - \tau _{{ext}} \end{aligned}$$

where \(\beth \) is the \(\mathrm{{n}}\times \mathrm{{m}}\) matrix containing \(\mathrm{{I}}_{\mathrm{{i}}}^{\mathrm{{j}}}\) for all i (body segment) and j (action line), \(\mathrm{{T}}_{\mathrm{{musc}}}\) is the m-vector containing the forces of all the action lines and \(\uptau _{\mathrm{{ext}}}\) is the n-vector of the external torques and moment applied to all the body segments.

This linear system is not invertible because n is not equal to m. In general m is greater than n leading to redundancy of the actuators (the action lines). It consequently leads to an optimization problem with a space of solutions. In addition to this equality constraint, it is possible to add inequality constraints such as:

  • A muscle force is positive and its maximum value is equal to K(l)*\(\mathrm{{K}}_{0}\)*PCSA as shown previously,

  • People naturally tend to minimize energy and torques (\(\sum _{{{j}} = 1}^{{{m}}} {{F}}_{{{i}}} ^{2}\) in many motions [32] (or the normalized effort \((\sum _{{{j} = 1}}^{{{m}}} {\left( {\frac{{{F}}_{i}}{{{F}_{{ i}}^{{{max}}}}}} \right) ^{2} })\) [18]),

  • Taking some EMG signals into account in the solving process [1]

This domain is very active in biomechanics and many researchers try to improve the quality and to validate the results. One of the most important problems is to be able to take contraction of antagonist into account. Using EMG signals to check if the antagonist muscles are active is a very promising approach.

Table 8.1 Summary of gait parameters and their potential use in walking in VR
Full size table

The approach presented in this section is named inverse dynamics but there exist two other approaches to solve this type of problem using direct dynamics within an optimization loop or directly exploiting EMG signals. Whatever the method, it is still difficult to validate the results. However, it is a promising contribution to analyse more accurately the actions performed by the user in a virtual environment, as it has been shown in ergonomics [27]. VR is more and more used to evaluate and train disable people as it provides a safe and reproducible environment. In this type of application EMG and muscle contraction feedbacks are very important.

As for the joint torques and forces, muscle tensions are good indicators of gait efficiency and can thus be used to determine subtle changes in gait patterns that kinematic data fail to identify. It could thus be used to evaluate if the user’s gait is as efficient in VR compared to natural walking.

7 Conclusion

J. E. Marey and E. Muybridge were the first researchers who proposed objective measurements of animal and human locomotion [20]. Nowadays, numerous systems exist to analyse and measure human gait. The biomechanics community is still very active in this domain and collaboration with other domains will certainly lead to new systems, such as using depth-cameras (Kinect of Microsoft) or inertial sensors. It is difficult to predict what will be the future in this domain, but many researchers tend to propose non-invasive and light systems associated with more and more sophisticated numerical models in order to access to new parameters. Musculoskeletal models are clearly a step forward in this domain but many researches have to be carried-out in this domain for calibration and validation. The Table below summarizes the parameters introduced in this chapter and their potential use in walking in VR (Table 8.1):

  • sensing the user’s motion,

  • delivering the most appropriate and accurate multisensory feedbacks,

  • and evaluating the naturalness of the interaction for the user compared to reference values (as those reported in Chapter “Biomechanics of walking in real world”).

Systems deliver more and more data (global parameters, joint positions, angles, torques and muscle forces) and a new problem occur: how to deal with all these parameters? In virtual reality researchers try to provide the user with more realistic feedbacks which could rely on using such big amount of knowledge. Getting more accurate data on the user’s gait could help to select and compute the corresponding feedbacks. Let us consider the walk-in-place approach currently used in VR [36]. In this approach, signal processing is applied to the orientation of the head depending time. Extending this approach to more complex parameters would enable to deal with more complex situations and behaviours, such as getting up and down stairs, avoiding obstacles or taking specific gait style into account. As for future steps, one main challenge therefore remains to be able to compile all the available data and to compute the most appropriate feedbacks in real-time.