Keywords

1 Introduction and Literature Review

The rich behaviors of the hand derive in major part from the five flexible fingers. Only a few natural haptic interactions involve just one finger, but because of the complexity of multi-finger grasping and perception there is less research on the properties of the fingers working together or their individual differences.

In this chapter we will highlight just two aspects of multi-finger haptics: the spatial stiffness properties of pen-type multi-finger grasping, and perception of small haptic features using all four fingers. The former is of interest because of increasing focus on performance of tasks using the pen-grasp, such as surgery. The later is significant for designers of multi-finger haptic devices and algorithms whose specifications are ideally derived from human perceptual characteristics.

The material in this chapter on pen-grasp has appeared in [1]. The material on multi-finger perception has appeared in [2].

1.1 Multi-finger Haptic Displays

Many engineers have tackled the challenge of multi-finger haptic devices (see Burdea [3] for a comprehensive 1996 review). These devices tend to be mechanically very complex as structure, sensing, and actuation needs to be provided for a large number of coupled degrees of freedom (DOF) in a small space. The following review is not meant to be comprehensive, but instead to convey the common and necessary mechanical tradeoffs.

The SARCOS dexterous master [4] provided force sensing and hydraulic drive to the thumb and one finger in a 3 DOF configuration optimized for grasping and tool use. The U. Tokyo Sensing Glove II [5] was a tendon driven exoskeleton, with 20 DOF, aimed at manipulation of virtual objects. The “Tactuator” [6], was a very high bandwidth device designed and used for psychophysical threshold measurements on a single DOF to each of three fingers. With disk drive flat coil actuators, the Tactuator achieved bandwidths of over 200 Hz and up to 25 mm displacement. The motion axes drove the thumb, index finger, and middle finger in a relaxed cup-shaped posture. The Rutgers Master [3] used four custom pneumatic pistons on gimbal mounts to generate internal forces between the palm and the tips of the thumb and three fingers. The Cyberglove/Cyberforce system [7] was a multi-finger glove and wrist gimbal mounted in a haptic device. The finger actuators were removed to ground (for mass and volume reduction) by tendon drives. Kron and Schmidt [8] designed compact fingertip tactile actuators to overcome some of the bandwidth limitations of the Cyberglove’s tendon drives. Gosselin et al. [9] developed a two-finger spatial device worn on the wrist which had three actuated degrees of freedom. Gillespie [10] studied a piano keyboard haptic device capable of simulating the dynamics of linkages (such as piano mechanisms). High bandwidth and multi-finger display was achieved in one degree of freedom per finger. Casiez et al. developed a 3-finger skin-slip device [11] and two recent devices [12, 13] supplied three high-bandwidth DOF to two fingers. The Spidar-8 [14], supplied 3DOF to 8 fingers through tension cables.

The human hand gives us at least 26 DOF (including the wrist) inside a very compact space (estimate: 17.2 ml per DOF). This complexity makes it inevitable that many compromises are made by engineers of haptic devices.

All of the above devices, as well as the devices we describe below, trade away many desireable properties. High bandwidth (e.g. [6]) can be achieved with only 3 degrees of freedom while high degrees of freedom (e.g. [7]) can be achieved with high friction tendon drives which limit force feedback fidelity.

1.2 Mechanical Impedance of the Hand and Fingers

In most of the related experiments found in literature, a haptic device is used to measure the mechanical properties of the human arm/hand. The subject is constrained to the device. Then, either the device applies a signal of force, and position is measured, or the device position is displacement controlled, and the force applied by the operator onto the device is measured. A model of the arm/hand can than be identified by the data collected.

Hogan [15, 16] stated and experimentally confirmed that the human arm dynamics can be described as a modulated, lumped parameter, linear impedance varying with the amount of muscular contraction, position of the arm, and difficulty of the task. He experimentally observed that the human arm can be described as an adaptive second order system, composed of mass, damper and spring. Impedance can be defined as the dynamic relation between force exerted by the muscle and imposed stretch, or as the relation between imposed force and stretch of the muscle.

$$ F \to {\text{admittance}} \to \Updelta x $$
$$ \Updelta x \to {\text{impedance}} \to F $$

Whenever this relation can be modeled by a lumped-parameter linear system, the dynamics of the muscle-limb can be represented as a Laplace Transform in the frequency domain:

$$ Z\left( s \right) = \frac{F\left( s \right)}{\Updelta X\left( s \right)} $$

where \( Z\left( s \right) \) is the muscle-limb impedance.

Mussa-Ivaldi [17] modeled the human arm, for tasks constrained to the horizontal plane, as a pair of springs and dampers, oriented in the plane. The notion of mechanical impedance is expressed in 2 DOF, as:

$$ \left[ {\begin{array}{*{20}c} {F_{1} \left( s \right)} \\ {F_{2} \left( s \right)} \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {Z_{11} \left( s \right)} & {Z_{12} \left( s \right)} \\ {Z_{21} \left( s \right)} & {Z_{22} \left( s \right)} \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {\Updelta X_{1} \left( s \right)} \\ {\Updelta X_{2} \left( s \right)} \\ \end{array} } \right] $$

where \( \left[ {\begin{array}{*{20}c} {F_{1} \left( s \right)} & {F_{2} \left( s \right)} \\ \end{array} } \right]^{T} \) and \( \left[ {\begin{array}{*{20}c} {\Updelta X_{1} \left( s \right)} & {\Updelta X_{2} \left( s \right)} \\ \end{array} } \right]^{T} \) are the force and position vectors respectively. The matrix impedance is a 2nd order linear model of the impedance in the plane at the limb end point:

$$ \left[ {\begin{array}{*{20}c} {Z_{11} \left( s \right)} & {Z_{12} \left( s \right)} \\ {Z_{21} \left( s \right)} & {Z_{22} \left( s \right)} \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {M_{11} } & {M_{12} } \\ {M_{21} } & {M_{22} } \\ \end{array} } \right]s^{2} + \left[ {\begin{array}{*{20}c} {B_{11} } & {B_{12} } \\ {B_{21} } & {B_{22} } \\ \end{array} } \right]s + \left[ {\begin{array}{*{20}c} {K_{11} } & {K_{12} } \\ {K_{21} } & {K_{22} } \\ \end{array} } \right] $$

where the matrix \( K \) is symmetric such that \( K_{21} = K_{12} \).

If we plot the set of points \( F \) obtained from mapping a circle in \( X \) around the equilibrium,

$$ \left[ {\begin{array}{*{20}c} {F_{1} \left( s \right)} \\ {F_{2} \left( s \right)} \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {K_{11} } & {K_{12} } \\ {K_{21} } & {K_{22} } \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {\Updelta \bar{X}_{1} } \\ {\Updelta \bar{X}_{2} } \\ \end{array} } \right] $$

where \( ||\Updelta \bar{X}|| = 1 \), we obtain a stiffness ellipse in the Cartesian plane, that tell us the stiffness along different directions. The ellipse is characterized by size (principal axis), shape (ratio of axis sizes) and orientation (angle that the principal axis makes with the reference frame) Mussa-Ivaldi’s experimental results showed that the stiffness ellipse size and orientation are to an extent under control of the subject according to co-contraction and the posture of the arm.

Tsuji [18], qualitatively confirmed the results of Mussa-Ivaldi, but found differences in the size of the ellipses, probably due to the differences in the experimental setup. Tsuji’s experimental data shows that:

  • The human hand inertia characteristics can be explained from basic biomechanics of the passive effects (stiffness, damping, and inertia).

  • Increasing the grip force increases the size of the stiffness and viscosity ellipses. However, the published data consist only of two different types of grasp: relaxed and tight;

  • The orientation and shape characteristics of the stiffness and viscosity ellipses are mostly explained from the kinematics of the human arm.

Haijan and Howe [19] measured the impedance of the metacarpal-phalangeal joint of the index finger, constraining the distal and the proximal interphalangeal joints. They found that the joint can be reasonably modeled as a linear spring, with stiffness a function of how much the subject is trying to co-activate the muscles of the finger. Milner [20], modeled the stiffness of the full finger as a 2 × 2 matrix, as was done by Mussa-Ivaldi and Tsuji for the arm impedance. However, for a given posture, he measured five different ellipses that were different not only in size, as measured by Mussa-Ivaldi, but also in shape and orientation, depending on the presence and orientation of an additional constant bias of force. Stiffness maximum eigenvalues ranged from about 8–22 N/cm, for the extended finger, and from about 4–20 N/cm for the flexed finger. The ratio between maximum and minimum eigenvalues ranged from about 2.45–8 for the extended finger, and from about 1.3–2.44 for the flexed finger. Force measurements were made 70–90 ms. after displacement onset, so that measurements included a contribution from reflexes activated by the displacement.

In a manual about surgical technique, Anderson [21] describes how to handle a scalpel according to the particular task to be performed, and he suggests the best direction to execute a clean cut. This information is the result of centuries of practical evidence. At Berkeley, Tendick [22] used a model of the kinematic structure of the human hand, including the stiffness of the muscles, to see if there is a mechanical advantage, in terms of impedance, in the current surgical technique. The parallel configuration of the grasp is a very stiff structure, in particular along some specific directions, depending on the fingers’ orientation. If we are to perform a clean cut, it is better to move orthogonally to the direction of maximum stiffness. This is for two reasons: first of all, the low stiffness allows force control in the direction of the cut; second, the high stiffness improves position control in the direction orthogonal to the cut, filtering mechanical noise that could cause to deviate from the straight cut trajectory. Tendick partially confirms the correctness of surgical techniques using an analytical model of the human grasp, and experimentally measured the impedance at the tip of a firmly grasped scalpel.

1.3 Psychophysical Thresholds

Psychophysical literature relevant to sensory thresholds at the fingertips has been reviewed in detail in [23]. A review from the point of view of haptic interface design is available in [24]. Physiological responses can be detected from stimuli as high as 10 kHz, and these perceptions have been linked to specific neural discharges and receptor types [25, 26]. Tan and Rabinowitz’s device [6] confirmed earlier measurements of a declining vibrotactile threshold (indicating increased sensitivity) up to 200 Hz.

Other researchers have quantified the spatial acuity of human tactile perception with the bare finger [27] as well as perception of textures via a rigid probe [28, 29]. A study of Braille perception contributed adaptive thresholding algorithms to the study of tactile perception [30]. In terms of amplitude, Jones [31] measured a 6 % ability (Weber fraction \( {{\Updelta F} \mathord{\left/ {\vphantom {{\Updelta F} F}} \right. \kern-0pt} F} \)) to haptically discriminate forces applied to the extended finger. Allin et al. [32] got a Just Noticeable Difference (JND) of 9.9 % in a similar experiment.

To our knowledge, the only similar work with multiple fingers has used vibrotactile stimulation. Yuan et al. [33] studied ability to detect onset time differences between the thumb and index finger. They found a threshold of 34 ms below which onset order could not be distinguished. Craig [34] measured about 2 dB drop in threshold when 100 Hz vibrotactile stimuli were applied to two fingertips simultaneously. This spatial summation disappeared when the frequency of vibration was 9 Hz. A similar result, when fingers contacted a vibrating cylinder, was obtained in [35]. However, Refshauge et al. [36] found that tonic stimulation of adjacent fingers did not reduce thresholds for detection of passive movements. Physiological mechanisms for aspects of these sensations are explored by Collins et al. [37].

What these studies have in common is an input/output view of perception. One or more physical stimulus variables (e.g. texture roughness or orientation, vibration frequency or energy) is rigorously controlled, and the human central nervous system returns information. This is important knowledge in its own right, but when humans interact with haptic devices, as well as with most of the physical world, no single variable is held constant. There is a bi-directional flow of energy and information between the human and external environment. Furthermore, the haptic device is designed with different goals and constraints than psychophysical experimental apparatus. It is not straightforward to predict whether or not a haptic effect is detectable on a given device from psychophysical thresholds alone. Nor is it straightforward to use psychophysical data to predict the effect of design variations in a haptic device on feature detection performance.

West and Cutkosky [38] compared the bare finger, hand-held stylus, and stylus/haptic device in terms of users’ ability to detect sinusoidal gratings along one dimension and count the number of cycles present. They found that detection performance with the haptic device was inferior to the bare finger or stylus, and depended on the stiffness parameter of the virtual surface model.

Venema and Hannaford [39] compared haptic feature detection performance with a single finger haptic device and found optimal values of stiffness and damping gains. The variable of interest in this experiment was the magnitude of C1 discontinuity between two linear segments. Thus, in most cases where general purpose haptic devices have been used in psychophysical research (e.g. [29, 38]), only the gain of the haptic rendering has been varied. Other variables, notably the number of fingers supported, have been held constant and we do not learn how performance varies with these variables.

2 Pen-Based Haptic Device

In this section we describe a haptic display designed to measure the mechanical impedance of the human hand [1, 40, 41] (Fig. 1). The design specifications were to obtain a large stiffness, low damping and low inertia, and to achieve a spatially invariant dynamic response. The operator interacts with the manipulator using the tip of a real scalpel, other pointed tools, or the fingertip.

Fig. 1
figure 1

Pen-based force display, a high-bandwidth 3 DOF device for application of force to the tip of a pen-like instrument

Full size image

The pen grasp, in which individual digits jointly support a rod-like tool, is used for many high precision manipulation tasks including writing, drawing, soldering and surgery. While it is clear that the multiple fingers provide stability to the pen grasp, a quantitative framework would be useful to answer questions such as ``Which direction has the maximal stiffness for a give type of pen-like grasp?”.

The device is composed of a 2 DOF, actuation redundant, planar parallel structure, rotated around a horizontal axis by an additional pair of actuators to create vertical motion. Because of the limited motion range, it is possible to consider the rotation around the up and down axis as a linear motion along the vertical z axis. The actuators are direct-drive, flat-coil actuators from hard disk drives [42]. The device is characterized by a high force output bandwidth (over 100 Hz), low friction and no backlash. The maximum torque output of the 3 actuators on the planar structure is about 0.01 Nm, while the maximum torque output of the other 2 actuators, connected to the up and down axis is about 0.05 Nm. The kinematic structure of the device is shown in Fig. 2. We can consider the 2 DOF parallel structure as three 2 DOF serial Cartesian manipulators connected together at the end-effector. Each serial chain is composed of an inner and an outer link.

Fig. 2
figure 2

Schematic representation of the pen-based force display (reprinted with permission from [1]). The shaded circles represent actuated joints. The non-shaded circles represents non-actuated joints. The small filled circle at the center represents the end effector

Full size image

3 Mechanical Impedance of the Pen-Grasp

3.1 Theory

We focus here on the relationship between displacement and force provided instantaneously by the biomechanics of a particular grasp:

$$ F = K_{0} \Updelta X_{0} $$
(1)

To determine the stiffness matrix \( K_{0} \), we will use the grip transform analysis. This method has been extensively analyzed in [22, 43, 44]. It consists of three steps, during which we will determine:

  • the relation between force at the tip, and torques at the joints of each single finger;

  • the relation between forces at the tip of the finger and net force applied to the pen;

  • the relation between finger joints displacement and pen tip displacement.

However, the data obtained in these three steps alone it is not enough to obtain the stiffness matrix in the Cartesian space, \( K_{0} \). We also need to know the stiffness matrix in the joint space, that relates the displacement induced in the angular position of the joint due to applied torque. Unfortunately, there is not much data in the literature about finger joint stiffness. Most of the literature reports experiments which were performed to obtain the stiffness of a single joint, since inter joint stiffness is very difficult to obtain [19, 20]. Since we will use incomplete data for the stiffness matrix in the joint space, the model in this section is intended to be qualitative, to get a better overall idea of the pen grasp. The experimental setup described later will be used to determine quantitative data.

Before the detailed grip transform analysis, we will briefly describe the hand kinematic model and introduce some notation [45, 46] (Fig. 3). The model permits grips consisting of one to four fingers, with points of contact only at the fingertips. In reality, a one finger grip is impossible. However, in our model we make the assumption that the tool is the distal phalanx of the finger, or it is rigidly connected to it, such as in the Phantom haptic interface [47]. We rely on the Jacobian Matrix, \( J_{i} \):

Fig. 3
figure 3

3-finger pencil-grasp model (reprinted with permission from [1]). Left screen capture of simulation software package. Each finger is modeled as three moving links plus a fixed link. Each finger has three degrees of motion. Shown is the index finger (light color), the middle finger (intermediate color), the thumb (dark color), and the pencil, held at the tip (black). Right, kinematic representation of single finger: reference frames, joint angles, dimensions and rotation axis. Thumb phalange names are on the plot on the left

Full size image
$$ \tau_{i} = J_{i}^{T} C_{i} $$
(2)

where \( C_{i} \) is the generalized contact force and \( \tau_{i} \) is the joint torque vector for the ith finger. \( J_{i} \) can be determined using Craig’s approach as

$$ J_{i}^{T} = \left[ {\begin{array}{*{20}c} 0 & 0 & {\left( {s_{34} + c_{34} } \right)s_{2} l_{3} - c_{23} l_{2} - c_{2} l_{1} } \\ {l_{2} s_{4} + l_{1} s_{34} } & {l_{3} + l_{2} c_{4} + l_{1} c_{34} } & 0 \\ {l_{2} s_{4} } & {l_{3} + l_{2} c_{4} } & 0 \\ 0 & {l_{3} } & 0 \\ \end{array} } \right] $$
(3)

considering all n fingers together,

$$ \tau = \left[ {\begin{array}{*{20}c} {J_{1}^{T} } & 0 & \ldots & 0 \\ 0 & {J_{2}^{T} } & \ldots & 0 \\ \ldots & \ldots & \ldots & \ldots \\ 0 & 0 & \ldots & {J_{n}^{T} } \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {C_{1} } \\ {C_{2} } \\ \ldots \\ {C_{n} } \\ \end{array} } \right] = J^{T} C $$
(4)

The contact forces produce a net force, \( F \), on the object related by

$$ F = WC $$

where \( W \) is the grasp matrix which depends on grasp configuration.

Figure 4 shows the relation between a fingertip frame and the object (pen) frame. On the lower right is the sequence of rotations (\( \theta_{xi} ,\theta_{yi} ,\theta_{zi} \)) that defines the orientation of the finger frame with respect to the pen. All rotations are around the fixed pen frame, with the fingertip lying on the x axis. The sequence of rotation is \( ZYX \). \( \theta_{pi} \) is used to define the point of contact of the finger on the pen and is the angle between the y axis and the point of contact. \( \Updelta cp_{i} \) is the offset of the contact position from the tip of the pen.

Fig. 4
figure 4

Fingertip-Pen Orientation. Left representation of the fingertip and pen (object) frames. Right Definition of pen offset and rotations

Full size image

The four angles \( \theta_{xi} ,\theta_{yi} ,\theta_{zi} ,\theta_{pi} \) and the offset \( \Updelta cp_{i} \) completely define the position and orientation of the finger relative to the pen. However, since the effect of the offsets is a torque applied onto the pen, they will not be used on our model which only studies forces in a 3 DOF space. Alternatively, it can be assumed that the \( i \) forces and offsets always produce zero torque in combination or that the pen is at rotational equilibrium.

For each finger, the relation between the contact force expressed in the fingertip frame and the same force expressed in the pen frame is given by

$$ F_{i} = W_{i} C_{i} = rot\left( {\hat{x},\theta_{xi} } \right)rot\left( {\hat{y},\theta_{yi} } \right)rot\left( {\hat{z},\theta_{zi} } \right)C_{i} $$

Multiplying the rotations together we get:

$$ W_{i} = \left[ {\begin{array}{*{20}c} {c_{z} c_{y} } & { - s_{z} c_{y} } & {s_{y} } \\ {s_{z} c_{x} + c_{z} s_{y} s_{x} } & {c_{z} c_{x} - s_{z} s_{y} s_{x} } & { - c_{y} s_{x} } \\ {s_{z} s_{x} - c_{z} s_{y} c_{x} } & {c_{z} s_{x} + s_{z} s_{y} c_{x} } & {c_{y} c_{x} } \\ \end{array} } \right] $$
(5)

The total force on the object becomes:

$$ F = WC = \sum\limits_{i} W_{i} C_{i} = \left[ {W_{1} W_{2} \ldots W_{n} } \right]\left[ {\begin{array}{*{20}c} {C_{1} } \\ {C_{2} } \\ \ldots \\ {C_{n} } \\ \end{array} } \right] $$
(6)

By the principle of virtual work, infinitesimal displacements of the finger joints, fingertips, and grasped object, (\( \Updelta \Upphi ,\Updelta X_{C} ,\Updelta X_{0} \)) are related by the following equations.

$$ \Updelta X_{C} = J\Updelta \Upphi $$
$$ \Updelta X_{C} = W^{T} \Updelta X_{0} $$

Linear stiffness relationships between generalized displacements and forces can be expressed either in joint or pen spaces:

$$ \tau = K_{\theta } \Updelta \Upphi $$
$$ F = K_{0} \Updelta X_{0} $$

where \( K_{\theta } \) and \( K_{0} \) are stiffness matrices. Using the previous equations, the stiffness in the two spaces is related by

$$ K_{0} = WJ^{ - T} K_{\theta } J^{ - 1} W^{T} $$

To simplify the analysis, we will consider joint stiffness matrices with non-zero elements only on the diagonal.

Note that \( J_{i}^{T} \) are 3 × 4 matrices, and therefore \( J^{T} \) is a 12 × 16 non-square matrix. Therefore, the notation \( J^{ - 1} \) and \( J^{ - T} \) in the above equation are not formally correct. We will assume these two matrices to be the pseudo inverses of \( J^{T} \) and \( J \). An interesting consequence of the kinematic redundancy of the hand is that, in theory, there could be infinite stiffness matrices in the pen frame \( K_{0} \), for a given grasp configuration and joint stiffness matrix \( K_{\theta } \), depending on how the pseudo inverses are computed [48].

Once it is identified, \( K_{0} \) is the stiffness matrix in the 3 DOF Cartesian space. In our study we are interested in determining the stiffness at the tip of the pencil when the motion is constrained to be on a specific plane, such as, for example, the surface of a table during the action of writing. We choose the table frame so that \( X \) is perpendicular to the surface and points straight down, \( Y \) lies on the surface and points toward the subject, and \( Z \) lies on the surface and points to the right of the subject. We will call this plane the ``horizontal plane”. Because the motion is constrained to be on a plane, we determine the 2 DOF stiffness ellipse by computing the section obtained by cutting the 3 DOF stiffness ellipsoid by the \( \{ Y,Z\} \) plane.

3.2 Results: Experimental Measurements

In this section we will experimentally compare the pen-grasp and the single finger configurations. We will first describe the experimental protocol, and the analysis method. Then we will discuss the results.

Experimental setup and protocol The pen-based force display was used to apply a series of 32 force steps \( \Updelta F \), of random intensity and orientation. The two components of \( \Updelta F \) were uniformly distributed in the interval [0.1N, 0.5N]:

$$ \Updelta F_{y} = U\left( {0.1N,0.5N} \right) $$
$$ \Updelta F_{z} = U\left( {0.1N,0.5N} \right) $$
$$ \vec{\Updelta }F = \left[ {\Updelta F_{y} ,\Updelta F_{z} } \right]^{T} $$

The subjects interacted with the device first using only the index finger, and then grasping a stylus. In both configurations the subjects were asked to ground the heel of their hand on the flat surface in front of the haptic display to eliminate their arm and wrist dynamics from the measurements. Each force step lasted 100 ms. and was followed by 1 s during which no force was applied, to allow the pen tip to go back to its at rest position. The force applied and the pen tip displacement induced were recorded 30 ms. after the beginning of the step. This should guarantee that the measured response is not influenced by spinal reflexes, and is only the mechanical impedance of the pen grasp. Because the pen-based force display has a very linear force response [49], and almost no friction and moving mass, we can assume for the purposes of this experiment that the force applied to the pencil tip is equal to that commanded by the controller. The position of the end-effector is measured with a resolution of about 20 μm. Due to the particular design, when force is applied on the end-effector there is no significant backlash. The subjects were asked to keep their grasp strength as constant as possible throughout each single experiment. For this reason a 4 mm force sensing resistorFootnote 1 was interposed between the thumb and the pencil. The pressure sensor signal was displayed as a yellow bar on top of the computer display. The pressure sensor has a non-linear response, so calibration was necessary. Each experiment was repeated twice. The subjects were first asked to start trying to resist force as much as possible, and then stay relaxed.

Analysis Data from three subjects were acquired and used to identify the parameters of the following 2 DOF linear system, representing the stiffness of the pen grasp at the tip of the pencil

$$ \left[ {\begin{array}{*{20}c} {\Updelta F_{y} } \\ {\Updelta F_{z} } \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {K_{yy} } & {K_{yz} } \\ {K_{zy} } & {K_{zz} } \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {\Updelta y} \\ {\Updelta z} \\ \end{array} } \right] $$

where \( \left[ {\Updelta y,\Updelta z} \right]^{T} \) is the position displacement induced by the force step.

An estimate, \( \hat{K} \) of the \( K \) matrix was computed from the force step response data. We estimated the symmetric component of \( \hat{K} \) by singular value decomposition. The singular value decomposition of \( \hat{K} \) yields two singular values and a rotation matrix which encode the maximum and minimum directions of stiffness in the plane perpendicular to the fingertip/pen-tip: \( K_{y} ,K_{z} ,\alpha \). These parameters are conveniently plotted as a ellipse whose size and direction convey the spatial stiffness of the grip.

In data from the three subjects (Fig. 5) stiffness magnitude varied between 1.15 and 27.99 N/mm for the various conditions and multiple regression coefficients varied from 0.74 to 0.98. The angle of peak stiffness was an average of \( 27.5^{ \circ } \) for the index finger conditions and \( 61.2^{ \circ } \) for the pen grasp conditions.

Fig. 5
figure 5

Stiffness elipsoids measured from three subjects for single finger (left) and pen-grasp (right) (reprinted with permission from [1]). In each figure, the smaller ellipse was measured when subjects were instructed to use a relaxed pose/grip, and the larger ellipse resulted from a firm pose/grip

Full size image

Sources of error in this computation include asymmetric stiffness component (non-conservative forces, which were no more than 10 % of the symmetric stiffness) and variance which was about 10 % of the peak stiffness.

3.3 Results: Different Grasp Types

In this section we will characterize different types of grasp, to see if there are any obvious advantages or disadvantages in particular kinematic configurations. We are particularly interested in grasps in which the index finger can be freely detached from the pen without a significant loss of stability. These grasps would allow the user, for example, to press a micro-switch during surgery to control an additional function of an advanced tool. We are also interested in grasps whose stiffness ellipse is almost a circle in which

$$ \frac{{\lambda_{\hbox{max} } }}{{\lambda_{\hbox{min} } }} \approx 1 $$

These grasps would guarantee similar user capabilities, such as force control and position control, in all directions. We will consider four different grasps (Fig. 6). We will first analyze the differences in shape, size and orientation of the stiffness ellipses using the theoretical model. Then we will experimentally measure the stiffness ellipses, as subjects assumed different grasp types.

Fig. 6
figure 6

Four grasp types studied for relative stiffness (reprinted with permission from [1]) Top to Bottom Surgeon Precision Grasp, Four-Finger Grasp, Modified Four Finger Grasp, “Chopsticks” Grasp (see text). Left Kinematic model (looking down on plane of table-top) Right Photograph

Full size image

3.3.1 Four Grasp Types

Surgeon Precision Grasp The first grasp is the surgeon precision grasp [22]. The three fingers (index, middle, thumb) contact the pen at approximately the same distance from the pen-tip. The contact points are equally spaced around the pen. Figure 6 shows views from the CAD model (left), and a photograph (right) taken looking at the hand from the side. The stiffness ellipse was computed using the model described in Sect. 3.1.

Four-Finger Grasp The middle and ring fingers and thumb contact the pen at approximately the same distance from the pen-tip, while the index is about 1 cm further up. The contact points for the middle and ring fingers and thumb are equally spaced around the pen. The index finger contact point is in between the middle finger and thumb. We observed that this second grasp is routinely used for writing and drawing by some of the subjects. The stiffness ellipse, obtained again from the theoretical model, has a more elongated shape, and it is rotated counter-clockwise, compared to the previous grasps.

This grasp is particularly interesting because it is possible to lift the index finger, and keep a stable grip with the remaining three fingers. The index finger is therefore free to move around and touch optional sensors and micro-switches. When the index finger is not touching the pen, the stiffness ellipse slightly shrinks in size, but the orientation does not change significantly.

Modified Four Finger Grasp This is a four-finger configuration similar to the four-finger grasp, but with the index finger moved further away from the tip of the pen. We observed that this grasp is routinely used for writing and drawing by one subject. As with the four-finger grasp, the index finger can be freely moved around without perturbing the grip.

“Chopsticks Grasp” The index and middle fingers strongly grip the pen from both sides, as if they were holding chop-sticks. The thumb presses against the pen further away from the tip. We obtained this particular three finger grasp trying different kinematic configurations on the model, until we obtained a more “circular” shape for the stiffness ellipse. As can be seen in Fig. 7 (dotted lines), the ratio \( {{\lambda_{\hbox{max} } } \mathord{\left/ {\vphantom {{\lambda_{\hbox{max} } } {\lambda_{\hbox{min} } }}} \right. \kern-0pt} {\lambda_{\hbox{min} } }} = 1.2 \) is very close to one.

Fig. 7
figure 7

Experimental stiffness ellipses measured from four subjects using four different grasps (reprinted with permission from [1]): Surgeon Precision grasp (continuous line), Four Finger grasp (dashed line), Modified Four Finger grasp (dashed-dotted line), “Chopsticks” grasp (dotted line)

Full size image

Measured Stiffness Elipses with Different Grasp Types The step perturbations described above were applied to pen tips held by four subjects in the four grasp types above.

Because of the variability of the experimental results (Fig. 7), we cannot draw general conclusions for all subjects about typical stiffness ellipse characteristics for each grasp type. From these preliminary results it seems like there is not a universally advantageous grasp, valid for every human. Each person, depending on their own hand structure, will perform better, in terms of stiffness, with a particular grasp. The experimental setup can be effectively used to assess the characteristics of different types of grasp for a single subject. As an example, this could be used for the training of surgeons, allowing trainees to check among different type of grasps, measuring the stiffness ellipses. It would be possible to choose the type of grasp that best fit the specific individual or, an individual could choose different grasps for different tasks.

4 Multi-finger Haptic Device

The University of Washington Biorobotics Lab built a four finger, eight DOF haptic device, the MultiFinger Haptic Device (MFHD) [50] by making four copies of its 1997 single-finger device [39, 51] (Fig. 8). The device support planar motion of the four fingertips. Each finger contains two custom wound flat-coil actuators driven by Neodymium-Iron-Boron permanent magnets and having 90° of motion range. Thermal modeling enables peak torques of up to 0.6 Nm—equivalent to about 6 N finger tip force. The lightweight linkages were computer synthesized to match human finger anthropometric data. In the MFHD, miniature interferometric optical encoders from Micro-E Inc. were integrated inside to allow the fingers close proximity and to increase position sensing resolution. We installed two grades of this encoder, the M1500 in three fingers and the M2000 in one finger, to allow testing at extremely high resolutions (see Table 1).

Fig. 8
figure 8

Multi-finger Haptic Device completed in 2005. Each finger is a two-DOF planar mechanism, computer optimized to cover the flexion–extension workspace of human fingers. Each finger is driven by two hand-built low friction, low inertia actuators. The base contains all electronics and interfaces to the computer through a single USB 2.0 cable

Full size image
Table 1 Specifications of position sensors from Micro-E Inc. used in the Multi-finger Device (Fig. 8). In the earlier single finger version reported in [39, 52], angular resolution was 48 μrad
Full size table

The user’s fingers are attached to the end of each mechanism using a fitted glove with plastic clips on each fingertip [52].

5 Multi-finger Haptic Thresholds

5.1 Related Previous Work

A successful haptic interaction involves a haptic device in contact with the human operator in a bi-directional exchange of information and energy. Thus it is difficult to predict haptic feature detection ability from psychophysical thresholds. Here we find the point at which a human can perceive meaningful information from a realistic haptic device, and additionally explore how that performance changes as we vary one or more engineering parameters. Our measurement depends on the specific haptic device as well as on the user and so is not a threshold in the classical psychophysical sense. Our prior work in this area involved only a single finger. This section looks at how results change as stimuli are presented to multiple fingers (Fig. 9).

Fig. 9
figure 9

a Illustration of the rendering environment shows planar workspace of the fingertip mechanisms, and icon locations on a horizontal plane in the workspace. b Screenshot of a multi-finger trial. A blue sphere indicates the subject’s position in the environment, and the icon turns from yellow to pink when in contact (used with permission from [2])

Full size image

5.2 Haptic Target Rendering

For this experiment haptic icons are presented to the user on a horizontal plane. The horizontal plane lies perpendicular to the fingertips’ workspace, and rendering of the plane is described by:

$$ F_{y} = \left\{ {\begin{array}{ll} {k_{p} \left( {y_{plane} -y} \right) - k_{d} v_{y} } & {y \le y_{plane} } \\ 0 & {y> y_{plane}} \\ \end{array} } \right. $$
(7)
$$ F_{x} = 0; $$
(8)

\( F_{x} \) and \( F_{y} \) are forces rendered in the \( x \) and \( y \) directions. \( k_{p} \) and \( k_{d} \) describe the stiffness and damping of the surface, \( y \) and \( v_{y} \) are the position and velocity of the fingertip in the \( y \), vertical, direction. \( y_{plane} \) is the vertical coordinate of the horizontal plane.

The haptic plane forms lines of intersection with the vertical plane at each of the fingertips’ workspaces. Haptic icons are presented to the subject on these lines. Along the line for each finger two potential locations for a 5 mm sawtooth icon are spaced 30 mm apart, center-to-center. One of the two locations has a sawtooth force icon, while the other location has no force. The sawtooth is described by:

$$ F_{x} = \left\{ \begin{array}{ll} 0 & x - x_{0} < - \frac{\Delta }{2} \hfill \\ F_{\hbox{max} } \left( {x - x_{0} } \right)& - \frac{\Delta }{2} \le x - x_{0} \le \frac{\Delta }{2} \hfill \\ 0 & \frac{\Delta }{2} < x - x_{0} \hfill \\ \end{array} \right. $$
(9)

This is illustrated in Fig. 10 (left). For all active fingers, the active icon was in the same location in the workspace.

Fig. 10
figure 10

Left Sawtooth profile of tangential force inside the icon (5 mm width). Amplitude of sawtooth was adaptively converged to subject’s threshold. Right Convergence path of adaptive thresholding for 8 subjects—75–85 iterations to convergence. Average threshold was 28.6 mN (about the weight of a dime) [54]. (used with permission from [2])

Full size image

5.3 Detecting Small Haptic Effects

In this section we briefly review our related work (reported in more detail in [53, 54]) using the FHD with a single finger to study limiting factors for perception and usefulness of small haptic effects. Pairs of targets (icons) were presented to the subjects in which one randomly selected target had force feedback and the other had none. The targets consisted of two 5 mm regions, 30 mm apart. The subject’s fingertip was supported by a virtual plane approximately mid-workspace, y \( \ge \) 100 mm. The attractive force profile (Fig. 10, left) was a linear function of the end effector’s distance from the center of the target. Subjects were allowed to sample both targets indefinitely before indicating which target they perceived to contain a force and indicated their choice by pressing a button held in their opposite hand and wore noise reducing ear protection with masking music to eliminate possible sound queues.

An adaptive thresholding method [30] converged the force output to a level at which approximately 71 % of responses observed from the subject were correct. Two or more correct responses decreased the force by 5 %, while a single incorrect response increased the force back to its previous value. Each deviation from the current trend in force was termed a ``force reversal’’. The experiment started at 500 mN of force (a very easily detected amplitude) and continued with initially declining force amplitudes until twelve force reversals occurred.

5.4 Results: Active Exploration

Figure 10, right, shows force-adaptation paths taken by seven test subjects [54]. Each line represents the peak magnitude of force present for each trial during the experiment (the first 40 trials are not plotted). Subjects took 75–85 trials per experiment before incurring the necessary twelve force-path reversals. The mean of all converged values was 28.6 mN.

Device Friction It is interesting to compare this value with the static friction of the device. By slowly increasing the applied force and watching for end effector motion (without finger contact), we measured the static friction level at 100 mN. However, observing the convergence pathways, we see that most of the force reversals occur with \( F < 100\;{\text{mN}} \). Because the subject kept the device in nearly constant motion, substantially smaller force levels could be detected at the 71 % reliability level. This illustrates how the haptic effect detection threshold is in fact a combined human–machine property so neither machine or human properties should be considered alone.

It is possible that differences between the mechanisms could account for some observed effects. To assess this possibility, subjects performed a single-finger experiment using their index finger on each of the four mechanisms (Fig. 11. The differences in thresholds (means between 30 and 35 mn) were not statistically significant.

Fig. 11
figure 11

Detection thresholds in which the subjects used their index finger singly on each mechanism. There is no statistically significant difference between the devices thresholds. (used with permission from [2])

Full size image

5.5 Results: Finger/Device Variations

Figure 11 shows the mean and distribution of 71 % detection thresholds of the index finger on each mechanism. A one-way ANOVA for repeated measures was used to compare results. Analysis showed no significant difference between fingertip mechanisms (F = 0.83, P = 0.48).

Single finger thresholds were compared with the multi-finger threshold. The distribution is shown in Fig. 12, and mean values for each trial are shown in Table 2. The overall mean and standard deviation for the middle and ring finger are 32.1/15.5 and 43.6/20.4 respectively. For the index finger, the force threshold mean and standard deviation from mechanism 0 (31.8/10.3) are used. We see in the figure that all fingers and the multi-finger case appear to have comparable mean value, except for the ring finger which appears to be higher. A one-way ANOVA for repeated-measures demonstrated statistical significance with F = 3.8, P = 0.008.

Fig. 12
figure 12

71 % detection threshold statistics for each individual finger (index, middle, ring, pinkie) on one fingertip mechanism, with the multi-finger result. (used with permission from [2])

Full size image
Table 2 The mean 71 % detection threshold, standard deviation, and number of subjects in each of the eight trial conditions. (used with permission from [2])
Full size table

Four paired t-tests compared each individual finger’s threshold to the multi-finger threshold using the Bonferroni correction over four trials. This correction requires a \( p < 0.0125 \) for rejection of the null hypothesis at the 0.05 confidence level. The t-values showed no significant difference between the multi-finger and the index, middle and pinkie fingers (p = 0.12, 0.27, 0.18 respectively). The ring finger showed a statistically significant difference compared to the multi-finger with p = 0.006.

6 Conclusion

In this chapter we have demonstrated (1) some aspects of how the multi-finger pen-like grasp affects the spatial stiffness which is achieved at the instrument tip, and (2) variations in sensitivity to detect haptic features as various combinations of fingers are used.

Our pen-based haptic device was a suitable experimental platform to determine mechanical properties of pen-like grasps. Using a theoretical model and experimental data, we found that pen grasp manipulation is superior to single finger manipulation, both in terms of mechanical impedance and accuracy of motion control. Moreover, because of its parallel structure, the pen grasp stiffness ellipse is relatively insensitive to changes in the kinematic configuration of the fingers.

We analyzed four different types of grasp, with particular attention to four-fingered grasps, and found similar stiffness ellipses for all of them. Some of these grasps were chosen by observing different people interacting with a pen, others by trying different configuration on a custom computer animated model.

A new Multi-finger haptic device, specialized for supplying two high bandwidth DOF to each finger’s flexion–extension plane, allowed us to measure sensitivities of the different fingers and compare them to all four fingers working together to detect a line stimulus. Note that the multi-finger stimulus presented here occured in a straight line, yet the fingertips more naturally form a curve. If the subjects used a more natural finger pose, each finger would in general encounter the stimulus at a different time. Our ongoing work addresses the possible affects of time synchronization versus spatial alignment on perception thresholds.

Many challenges remain in scientific understanding and technological innovation in multi-finger haptics. New mechanisms, actuators, sensing, and control techniques must be developed to convey kinesthetic sensations effectively to the whole hand. Teleoperation systems, especially those which support precision and highly dexterous interactive tasks like surgery could take advantage of the operator’s grasp characteristics to accomplish tasks more accurately and robustly.