A Fabric-Based Approach for Softness Rendering

Abstract

In this chapter we describe a softness display based on the contact area spread rate (CASR) paradigm. This device uses a stretchable fabric as a substrate that can be touched by users, while contact area is directly measured via an optical system. By varying the stretching state of the fabric, different stiffness values can be conveyed to users. We describe a first technological implementation of the display and compare its performance in rendering various levels of stiffness with the one exhibited by a pneumatic CASR-based device. Psychophysical experiments are reported and discussed. Afterwards, we present a new technological implementation for the fabric-based display, with reduced dimensions and faster actuation, which enables rapid changes in the fabric stretching state. These changes are mandatory to properly track typical force/area curves of real materials. System performance in mimicking force-area curves obtained from real objects exhibits a high degree of reliability, also in eliciting overall discriminable levels of softness.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

The reproduction of material properties like softness is a crucial component for a compelling and realistic experience of tactile interaction. Softness is a property specifically related to tactile information and hence to the semantic representation of objects (Klatzky et al. 1991; Lederman and Klatzky 1987; Newman et al. 2005). Indeed, tactile signals about softness are among the most accessible sources of information after the initial phases of contact (Lederman and Klatzky 1997b) since their coding does not require any geometrical description of the object (Klatzky et al. 1989). Softness perception relies on two types of sensory signals: cutaneous (tactile) information (which is mainly related to the mechanical deformation of the skin) and proprioception/kinaesthesia [which can be regarded as the internal sensing of forces, displacements and postures processed inside joints, muscles, tendons and skin (Bastian 1888)], even though other modalities could also contribute to some extent like vision and audition, see Chaps. 2 and 4 respectively. Both types of information are necessary to have a softness perception for compliant objects with rigid surfaces. However, during normal interaction with the world, tactile sensory information is predominant, as the cutaneous contribution alone is sufficient for softness discrimination of objects with deformable surfaces (Srinivasan and LaMotte 1995). Most haptic devices which are currently available  (Hannaford and Okamura 2008) act primarily as force displays, although a cutaneous sensation is nevertheless provided through the contact with the device tool, but it is not modulated by the device. By contrast, tactile displays stimulate skin by conveying force with both contact and shape information. To convey cutaneous information with these devices, it is necessary to reproduce on the finger pad the complex mechanical interaction and stress/strain distribution which originates from the contact between the finger and the external object. Such tactile displays stimulate the mechanoreceptors that basically react to the strains of the skin in a manner proportional to the velocity, acceleration, or elongation (Kern 2009). Trying to measure and reproduce the tensor distribution produced by the human skin (itself a dishomogeneous, anelastic material), is a difficult task. The challenge for research is to reduce the complexity of the tactile information to a meaningful approximation, while considering design limitations such as feasibility, costs and quality of the rendering of the haptic stimuli.

2 Taming the Complexity of Haptic Information

To identify models which harness the complexity of tactile sensing, Bianchi (2012) proposed a geometrical reduction method, mapping a high dimensional space of perceptual elemental variables (such as information provided by sensory receptors) to a low dimensional space comprised of perceptual primitives and performance variables. The goal was to use these primitives to drive the design of haptic devices and artificial systems, which might enable a more reliable human-machine interaction. This approach draws inspiration from the neuroscientific studies on the biomechanical and neural apparatus of the human hand, which demonstrate that, despite the hands complexity, the simultaneous motion and force of the fingers is characterised by coordination patterns that reduce the number of independent Degrees of Freedom (DoFs) to be controlled (Schieber and Santello 2004). This experimental evidence, which can be explained in terms of central and peripheral constraints in the neuromuscular apparatus, describes well the concept of hand synergies (synergy, from Greek work together), i.e. the aforementioned covariation schemes observed in digit movements and contact forces. For a complete review on the concept of synergies (see Santello 2013).

A valuable conclusion can be drawn from these results: for a wide range of hand behaviours the kinematic space of the hand has a smaller dimensionality than the one represented by its mechanical degrees of freedom. Synergies can be regarded as maps (Latash 2008) between the higher dimensional complexity of purely mechanical architecture of the human hand and the lower dimensional control space of the action and performance. In the latter it is possible to individuate kinematic and kinetic primitives. In robotics, the concept of synergies has been used to design robotic hands in a simplified manner, and has lead to promising initial results [from the early attempts, e.g. (Brown and Asada 2007; Ciocarlie et al. 2007; Ciocarlie and Allen 2009), to the most recent applications (Catalano et al. 2012)]. Furthermore, synergies were also used to guarantee optimal performance and design for glove-based Hand Pose Reconstruction (HPR) systems (Bianchi et al. 2013a, b).

As previously mentioned, Bianchi (2012) extended the aforementioned motoric concepts to the (dual) haptic sensing domain, to find a mapping between the higher-dimensional redundant space of elemental sensory variables involved in the mechanics of touch and the lower dimensional space of perceptual primitives, i.e. “what we actually feel”. An attempt to accomplish this goal was pioneered in Bicchi et al. (2011), where authors hypothesised the existence of a sort of sensory synergy basis. The elements of this basis can be regarded as tactual perception manifold projections onto constrained subspaces, where the subspaces increasingly individuate refined approximations of the full spectrum of haptic information. Hayward (2011) investigated the existence of a plenhaptic function for determining the dimensionality of haptic perception. This function is the haptic counterpart of the plenoptic function (Adelson and Bergen 1991), defined in the visual domain to indicate the number of coordinates necessary to describe all possible sensorimotor interactions. From a mechanical point of view, the plenhaptic function can be regarded as the complete characterisation of haptic experience. In terms of vector basis it comprises all the elements necessary for an exhaustive description. In Hayward (2011) it was also noted that even if the number of dimensions needed to describe mechanical interactions in haptics is larger than three or four, human touch-related experience seems to take place in a lower dimensional space; i.e. the nervous system produces nearly instantaneous reductions of dimensions, to convert a complex problem into a manageable set of computational tasks. Tactile illusions, for example, can be interpreted as the results of these low dimensional simplifications of the plenhaptic function—sampled in time and space—related to motoric and sensory capabilities.

3 The Contact Area Spread Rate Approach for Softness Rendering

In softness discrimination, a possible reduction of the dynamic, force-varying tactile information operated by the nervous system might be described by the tactile flow paradigm (Bicchi et al. 2005, 2008), which extends Horn and Schunk’s equation (Horn and Schunk 1981) for image brightness to three-dimensional strain tensor distributions. Tactile flow equation suggests that, in dynamic conditions, a large part of contact sensing on the finger pad can be described by the flow of Strain Energy Density (SED) (or Equivalent von Mises Stress), since Merkel-SA1 afferents, which are primarily responsible for dynamic form in tactile scanning, were proven to be selectively sensitive to these scalar quantities (Johnson 2001). Moreover, the integral version of the tactile flow equation can be used to explain the Contact Area Spread Rate (CASR) (Bicchi et al. 2000) experimental observation, which affirms that a considerable part of tactile ability in object softness discrimination is retained in the relationship between the contact area growth over an indenting probe (e.g. the finger pad that presses the object) and the indenting force itself. These suitable approximations and reductions of the haptic information manifold suggest new strategies for building haptic interfaces. For example, recognizing that a simple force-area relation describes a large amount of the cutaneous information involved in softness discrimination by probing has inspired the development of simpler and more effective softness displays for human-machine interaction.

3.1 The First Discrete CASR-Based Display

Bicchi et al. (2000) presented the first prototype of a CASR-based display. Its role is to mimic the rate at which the contact area of the probed material grows over the surface of the probing finger pad. The implementation proposed in Bicchi et al. (2000) consists of a set of cylinders of different radii, assembled in telescopic arrangement (see Fig. 11.1).

Fig. 11.1

The discrete CASR display (a). A finger interacting with the discrete CASR device (b)

As a result of the discontinuity in the structure due to the cylinders, this CASR display will be referred to as the discrete CASR display. Regulated air pressure acts on one end of the cylinders according to the desired force to be perceived by users during the indentation (see Fig. 11.1). Pressure is applied on all the cylinders. When the user finger probes the display, pushing down against the cylinders, it comes into contact with a surface depending on the height of the cylinders themselves and perceives a resultant force correlated to the pressure. The length of the cylinders is chosen such that, when the operator exerts no force, the active surface of the display can be approximated in a stepwise manner as a cone whose vertex has a total angle of $2a$. After the finger is pushed down by an amount of $\delta$, the contact area $A$ can be approximately computed as $A(\delta )=\pi \,\delta ^2\,\tan (\delta )^2$. Consequently, the resultant force $F$ that is opposed to the finger is $F(\delta )=pA\,(\delta )$, where $p$ is the pressure to be provided to the inner chamber of the device by the external pressure regulator. In this prototype, the displacement $\delta$ can be measured with an optoelectronic sensor or a proportional Hall sensor placed at the bottom of the inner chamber. In Bicchi et al. (2000) psychophysical experiments proved that the discrete CASR display is able to provide better performance in softness discrimination than the one achieved using a purely kinaesthetic display (i.e. the discrete CASR display covered with a hollow rigid cylinder).

The experimental curves $F/A$ (or CASR curves) obtained for the discrete CASR display, at fixed pressure levels, are linear; to mimic CASR curves of real objects, typically nonlinear, the display has to be controlled acting on the variable $p$.

4 Fabric-Based Displays

The discrete CASR display was proven to be able to replicate desired force-area curves and to enable a more realistic softness perception compared with the one achievable with a purely kinaesthetic device. However, the structure of this display does not provide users with a continuously deformable surface, thus producing edge effects. This fact might lead to a not completely immersive experience, which can destructively affect the transparency and reliability of the perception, e.g. in tele-operation tasks. Moreover, the contact area involved in the interaction can be known only after some geometric considerations related to the measured displacement. This can represent a limitation for correctly mimicking real CASR curves, for which a real-time accurate measurement of the contact region is mandatory.

To overcome these limits, we propose a new concept of displays based on a bi-elastic fabric, hereinafter defined as Fabric Yielding Displays (FYDs). Bi-elastic means that the fabric exhibits properties that render it elastic in at least two substantially perpendicular directions. After preliminary tests on different materials, we decided to use the Superbiflex by Mectex (Erba, Como, Italy) since it offers both good elastic behaviour within a large range of elasticity, and a high resistance to traction. By changing the elasticity of the fabric, users are able to feel different levels of softness by touching a deformable surface. At the same time the contact area on the finger pad can be measured via an optical system.

4.1 Introduction to the First FYD Prototype

The first prototype of the FYD consists of a hollow plastic cylinder (ABS 3D printed, $195\times 50$ mm) containing a linear actuator (Linear Actuator L16-100-35-12-P by Firgelli, Victoria BC, Canada), which is a compact DC motor geared to push or pull loads along the stroke ($100$ mm). This actuator comes with a built-in potentiometer and control (maximum positional error of $0.4$ mm), allowing one to monitor and set the actuator position. On the top of the cylinder a rectangular shaped piece of fabric ($200\times 200$ mm) is placed and tied to a circular crown. The crown is attached to the motor stroke and it can run alongside the cylinder with minimal friction. When the motor pulls down the crown, the fabric is stretched and its apparent stiffness increases. Conversely, when the motor pushes up the crown, the fabric is relaxed and feels softer (Bianchi et al. 2010). The linear actuator is able to move at a rate of $32$ mm/s and can exert up to $50$ N of force at lower speeds, for a range of motion of $30$ mm. The FYD also behaves like a contact area display. A web camera (Hercules web cam, resolution of $320\times 240$ pixels at $30$ frames/s) is placed inside the hollow cylinder, at the centre of the mechanical interface, just beneath the fabric (at a distance of $\simeq 300$ mm). The camera is equipped with high luminosity Light Emission Diodes (LEDs) and frames the lower surface of the fabric, in particular the image of the strained fabric after the indentation. During tactual probing, the fabric is strained and the fabric area in contact with the fingertip changes in proportion to the applied force. The contact area can be estimated using the algorithms described below. The prototype as a whole is connected to a base and enclosed within a protective shell, with total dimensions of $195\times 115\times 115$ mm. An overview of the system and an exploded drawing view are reported in Figs. 11.2 and 11.3, respectively. For further details on the architecture of the first FYD prototype the reader can refer to (Bianchi et al. 2009, 2010).

Fig. 11.2

The FYD first prototype, an overview (a). A finger interacting with the display. For the sake of clarity the FYD is shown without and with the cover in (b) and in (c), respectively

Fig. 11.3

The FYD first prototype: exploded drawing view

Fig. 11.4

The measurement of the contact area

4.2 Area Acquisition

The contact area between the fabric and the finger can be estimated and visually displayed by means of a suitable segmentation algorithm. The contact area acquisition algorithm is based on RGB image binarisation. More properly, only one image band (the R band, which is a $320\times 240$ matrix of integer numbers) out of three is involved in the area detection algorithm to reduce the computational workload and allow for fast processing. The underlying idea is quite simple; while the fabric is probed, the indented fabric surface is closer to the camera with respect to the outer region, see Fig. 11.4. Consequently, this area will be more brightly lit by the LEDs. The difference between background luminosity and contact area luminosity is discriminated via binarisation thresholds, heuristically calculated. Using a linear interpolation, a binarisation threshold is associated to each vertical position of the crown. In this manner, the pixels in the image that belong to the contact area can be individuated and displayed (see also Fig. 11.5). In order to guarantee uniform and repeatable luminosity conditions, a cover is placed on the top of the external shell of the device (see Fig. 11.2). The contact area $A_\mathrm{contact }$ expressed in ($\text {cm}^2$) is estimated as

$$\begin{aligned} A_{\text {contact}}=N_{c}\times \frac{A_f}{N_w}, \end{aligned}$$

(11.1)

where $N_c$ is the number of pixels belonging to the contact area; $A_f$ is the area captured by the web camera (frame area) in [$\text {cm}^2$] and $N_w$ is the web camera resolution (i.e. $320\times 240$ pixels).

Fig. 11.5

An image of the fabric indented by the finger recorded by the camera (a). The result of the area detection algorithm (for the sake of clarity the contact area is shown in black) (b). The number of pixels belonging to the contact area on the right is $N_c$ while the total number of pixels is $N_w$

4.3 Characterisation, Interpolation and Experiments

The FYD prototype is controlled in order to simulate mechanical compliance of materials having specific (stiffness) force/displacement ($F(\delta )$) and force/area ($F(A)$) curves. Since force ($F$) and displacement/indentation ($\delta$) are the primary objects of kinaesthesia, $F(\delta )$ profiles can be regarded as useful abstractions and approximations of the kinaesthetic behaviour of materials; at the same time, based on CASR assumption, $F(A)$ curve contains a large part of the cutaneous information useful for softness discrimination.

In Fig. 11.6, the $F(\delta )$ and $F(A)$ curves of the fabric at different levels of stretching are reported. More specifically these levels were obtained changing the position of the crown, in a range between $0$ mm ($0$ mm was chosen near the top of the cylinder) and $30$ mm, with an incremental step of $5$ mm.

For the characterisation phase we used a compressional rigid indenter shaped as a human finger, driven by an electromagnetic actuator and capable of applying a maximum controlled displacement of $10$ mm in the axial direction, while the contact area, the displacement(indentation) and the contact force were measured, by the web camera, a magnetic linear transducer and a load cell, respectively. The shape of the wood indenter—$15$ mm in diameter and $100$ mm in length—represents a first geometrical approximation of the human fingertip (Dandekar et al. 2003). The load-cell was mounted on the indenter to measure the force applied on the fabric during the indentation. What is noticeable is that $F(A)$ and $F(\delta )$ curves are linear over all the positions of the crown. During the characterisation phase, only a finite set of positions was acquired. For intermediate values, a piecewise linear interpolant was adopted. It is possible to mimic a given material with a specific stiffness coefficient (which can be regarded as the angular coefficient of the $F(\delta )$ linear curve), by suitably identifying the corresponding position of the crown. Moreover, from the actual measurement of the contact area, since $F(\delta )$ and $F(A)$ curves are coupled, an indirect estimation of the indented force and hence of the displacement can be obtained. Notice that these estimations are strongly related to the nature of the contact. Indeed, in order to obtain coherent values, the user should touch the fabric in the same manner as the wood indenter did during the characterisation phase.

Fig. 11.6

Force/indentation characteristics at different fixed positions of the crown (a). The indentation is measured using a magnetic linear transducer. Force/area characteristics at different fixed positions of the crown (b). Notice that the position at $0$ mm is close to the top of the external cylinder

4.4 Evaluation Experiments

We designed an experimental session to evaluate the performance of the FYD, comparatively with the discrete CASR display, using five simulated levels of stiffness ($SS1$, $SS2$, $SS3$, $SS4$ and $SS5$). Table 11.1 shows the input parameters for the FYD and for the discrete CASR device necessary to reproduce the five simulated stiffness values.

After providing written consent, 10 right-handed volunteers participated in the study (7 males and 3 females, their age ranged from $23$ to $40$) (Bianchi et al. 2010). None had a history of limitations that could affect experimental outcomes. They performed the tests blindfolded and with ear plugs, to prevent the possible use of any other sensory cues and eliminate any diversion from the task. They were presented with different levels of stiffness and asked to judge them by pressing vertically or tapping their index finger against the displays. In ranking tests, which were conducted independently with each device, participants were presented with new stimuli for less than one second and each trial was repeated three times per participant. They were asked to probe the set of five levels $SS1$ to $SS5$, presented in random order, and sort in terms of softness,. Participants were asked to rank the stimuli from 1 to 5, where 1 corresponded to the softest.

Table 11.1 Parameters used to control the discrete CASR display (third column) and the FYD (fourth column) in order to render the same level of stiffness (first and second columns)

Table 11.2 Confusion matrix of ranking experiments with the discrete CASR display

Table 11.3 Confusion matrix of ranking experiments with the FYD

Results from ranking experiments are shown in Tables 11.2 and 11.3, where subjective softness is reported versus objective compliance in a confusion matrix structure (Srinivasan and LaMotte 1995) for the five levels, using both the devices. Values on the diagonal express the number of correct answers. The percentage of total accuracy is calculated considering the sum of all correct answers for all the levels of stiffness. The correspondence between an objective estimation of the compliance and the subjective evaluation in terms of numerical values in a given scale was already used e.g. in Srinivasan and LaMotte (1995), and Friedman et al. (2008). The results obtained with the discrete CASR display exhibits a percentage of total accuracy of $61\;\%$, while with the FYD the percentage of total accuracy is $82\;\%$. Notice that the chance level is $20\;\%$.

Results show that the FYD enables a better softness perception than the discrete CASR display. This enhancement is probably due to the absence of edge effects during the interaction between the fingertip and the fabric surface; indeed, the FYD provides cues for a more reliable and realistic perception, since the fabric is deformable in a controlled manner under the finger pad. This fact might also help to develop in a more effective manner the haptic memory required for multiple comparisons.

5 The Second Version of the FYD: The FYD-2

Although the first FYD prototype was proven to be able to enhance softness discrimination accuracy in participants, by conveying contact area cues in an intuitive and efficient manner, there were still some design aspects to be improved. Indeed, although the contact area was actively measured, no contact-area feedback for dynamic tracking was implemented. Furthermore, the physical dimensions of the device can potentially prevent it from integration in multi-device systems such as in Scilingo et al. (2010), for tasks where space constraints are mandatory. For these reasons, we created a second version of the display, hereinafter referred to as FYD-2 (Serio et al. 2013) (cfr. Fig. 11.7). The main advantages of the FYD-2 design are: the reduced dimensions ($70\times 70\times 100$ mm), which enable possible integrations with other devices and wearability; an actuation system based on two fast motors and a more effective sensorisation scheme, which consists of a web camera, for real-time measurement, and a force sensor mounted at the base of the device, to record the normal contact force exchanged between the finger pad and the fabric. Other approaches found in literature lack real-time area measurement, and this severely limits the reliability of tracking $F(A)$ curves by introducing edge effects and discretisation (Bicchi et al. 2000) or allowing the control of the fingertip contact area only for a finite set of constructed and stored in advance “numerical models” as in Fujita and Ohmori (2001). Here, the proposed actuation and sensorisation scheme realises a closed-loop control, based on the actual measurement of the contact area, which allows to track arbitrary force/area characteristics.

Fig. 11.7

The FYD-2, an overview (a). A finger interacting with the display. For the sake of clarity the FYD-2 is shown without and with the cover in (b) and in (c), respectively

Furthermore, the actuation scheme endows the system with an additional degree of freedom, which can be used to convey supplementary haptic cues, such as directional information, for a more compelling and immersive haptic experience.

5.1 Mechanical Design

In the second version of the device the extremities of a rectangular strip of the fabric are connected to two rollers. These rollers are independently moved by two DC Maxon Motors REmax—$256:1$, $3$ Watt—(Maxon Motor ag, Sachseln, Switzerland) through two pulleys placed on motor shafts.

Motor positions can be controlled by processing the signals from two absolute magnetic encoders (12 bit magnetic encoder by Austria Microsystems—Unterpremstaetten, Austria—AS5045 with a resolution of 0.0875$^\circ$), read by a custom made electronic board (PSoC-based electronic board with RS–$485$ communication protocol).

A level of softness is generated by appropriately stretching the fabric using the two motors; i.e. when motor 1 rotates in a counter-clockwise direction and motor 2 rotates in a clockwise direction they stretch the fabric thus increasing its apparent stiffness. When motor 1 rotates in a clockwise direction and motor 2 rotates in a counter-clockwise direction they relax the fabric thus reducing its apparent stiffness, see Fig. 11.8. Furthermore, it is important to notice that the two-motor configuration allows one to implement and exploit an additional degree of freedom. Indeed, when the two motors coherently rotate in the same direction, a translational shift can be imposed on the finger pad interacting with the fabric, as it is shown in Fig. 11.8. This shift can be used to convey kinaesthetic, directional and vibrotactile information to users. Finally, since motors can be independently controlled, it is also possible to modify contact area geometry (eccentricity), thus simulating incipient slippage conditions or curvature perception.

Fig. 11.8

How the system works during the interaction with a human finger pad. K is the stiffness of the fabric, which depends on motor positions, $\theta _1$ and $\theta _2$

The FYD-2 is also endowed with a load cell (Micro Load Cell (0-780g) - CZL616C from Phidgets, Calgary, Alberta, Canada) placed at the base of the device, to record the normal force exerted by the user finger interacting with the fabric. Notice that the shear force is not considered. This is in agreement with the instructions given to users interacting with the device. Indeed they were recommended to not perform movements of the finger across the surface and to not apply lateral forces, in order to eliminate any anisotropic effect or distortion in softness but only focusing on normal indentation of the specimens (Lederman and Klatzky 1997a).

The system has also a web camera (Microsoft “LifeCam HD–3000” with a resolution of $640 \times 480$) and two high luminosity LEDs (whose luminosity can be regulated with a trimmer) just beneath the fabric ($30$ mm), for measuring the contact area in real-time (cfr. Fig. 11.9). Force and area information are then used to implement the force/area tracking algorithm, whose results are described in the following subsections.

Fig. 11.9

FYD-2, exploded drawing view

5.2 Experiments

As discussed in the Introduction, softness perception relies on both haptic channels—kinaesthesia and cutaneous information—although tactile sensing plays a predominant role. Based on these considerations, the fabric-based device exhibits two types of behaviour and hence of control:

• (I) the FYD-2 can be controlled to track $F(\delta )$, while acting as a contact area (real-time) display

• (II) the FYD-2 can be controlled to track $F(A)$, while actively using, together with the measured force, the real-time measured contact area as a feedback signal.

Notice that both $F(\delta )$ and $F(A)$ curves are not independent since they represent the cutaneous and kinaesthetic characterisation of the softness properties of a given object. Therefore they are determined by the fabric (object) characteristics. This represents a common limitation for all haptic devices: to properly decouple these curves it would be necessary to have two independent control variables, as proposed for example by Scilingo et al. (2010), where a conventional kinaesthetic haptic display is combined with a cutaneous softness one.

For case (I), the aim is to mimic a given stiffness. In this case, the display behaves like common kinaesthetic devices (Hannaford and Okamura 2008 which can be regarded as force displays), although cutaneous cues and the measurement of the contact area provide additional information. On the other side, case (II)—$F(A)$ tracking problem—represents a more challenging issue, given that the law that relates the growth of the contact area and the indenting force, i.e. the CASR paradigm, represents a large part of the tactile information used for softness discrimination.

To implement these controls, the first step is to characterise the device. The characterisation procedure is analogous to the one used for the first prototype. Different positions of the motors were considered, with same starting point and angular displacement used for both. The angular values $(\theta =\theta _1=-\theta _2)$ used for the characterisation range from $10^\circ$ to $80^\circ$, with an incremental step of $10^\circ$. The range of contact force is from $0$ to $20$ N. In this case, the force/displacement characteristics, interpolated at fixed motor positions, are quadratic ($R^2>0.94$), and the stiffness [$\sigma$ in (N/mm)] of the fabric can be computed directly deriving the contact force with respect to the displacement. Let the characteristic be $F=\lambda \delta ^2$, with $\lambda$ (N/mm$^2$) representing the quadratic coefficient of the parabolic curve at fixed motor positions. In this case the stiffness of the fabric depends on the displacement and it can be defined as  (Grioli and Bicchi 2010; Serio et al. 2011): $\sigma (\delta ) = {\partial F}/{\partial \delta } = \rho \delta$, where $\rho =2\lambda$ (N/mm$^2$) represents the stiffness coefficient.

5.2.1 Constant Stiffness Tracking

For this kind of experiment, we used the finger pad of a right-handed male participant (age $32$) as the indenter for probing the fabric approximately every second. Since the fabric stiffness is not constant but it depends on the indentation, we control motor positions using motor encoders to know $\theta$ value. From this value it is then possible to retrieve the angular coefficient [$\rho _a$ in (N/mm$^2$)] of the actual stiffness curve from the characterisation characteristics or interpolating between them. Using the information about the contact force measured by the load cell of the device, the actual indentation $\delta$ can be obtained as $\delta =\sqrt{{F}/{\lambda }}$. Finally, the actual stiffness [$\sigma _a$ in (N/mm)] of the fabric can be computed as $\sigma _a=\rho _a\delta$ (see also Fig. 11.10 for more details).

In Fig. 11.11, the results of tracking a constant stiffness of $\sigma _r~=~1~N/mm$ and the control scheme adopted are reported. A PID control (constants $P=1$, $I=0.01$, $D=0$, heuristically found) is then used to control motor positions, based on the error ($e$) between $\sigma _r$ (reference stiffness) and $\sigma _a$ (actual stiffness). In this case, after an initial transitory phase due to motor positioning, we get an RMSE of $0.18~N/mm$, less than $20\;\%$ of the reference value. The effect of the transitory phase on human perception will be investigated in future experiments.

Fig. 11.10

The control scheme used for constant stiffness tracking. A PID control is used to control motor positions, based on the error ($e$) between $\sigma _r$ (reference stiffness) and $\sigma _a$ (actual stiffness)

Fig. 11.11

a Stiffness control. b Force. Reference stiffness (dashed line) versus controlled stiffness (continuous line) (a). Force measurement (b)

5.2.2 Trajectory Area Tracking

Fig. 11.12

The block diagram of control for $F(A)$ curve tracking. Let be $F(A)=\xi _r A_r^2$, the quadratic curve to be tracked, with $\xi _r$ in (N/mm$^4$) the quadratic coefficient of the curve. In order to properly implement the control, we need to know the actual $\xi _{a}$ (N/mm$^4$) coefficient. This coefficient is obtained each time by dividing the indenting force measured by the load cell for the squared value of the measured area $A_m$. $F_m$ is the measured force. A PID controller is then used to control the quadratic coefficient

In order to reproduce common quadratic $F(A)$ characteristics (Bicchi et al. 2000), the position of the motors needs to be controlled and suitably rapidly changed, based on the actual contact area. This fact motivated the need for a fast actuation system. Let be $F(A)=\xi _r A_r^2$, the quadratic curve to be tracked, with $\xi _r$ in (N/mm$^4$) the quadratic coefficient of the curve. In order to properly implement the control, we need to know the actual $\xi _{a}$ (N/mm$^4$) coefficient. This coefficient is obtained each time by dividing the indenting force measured by the load cell for the squared value of the measured area $A_m$. A PID controller is then used, which is based on the error between $\xi _r$ and the actual $\xi _a$ (see Fig. 11.12), constants: $P=5$, $I=0.3$, $D=0$. The effectiveness of this control scheme was experimentally verified by tracking the characteristic curves of different silicone specimens, realised with different percentage of plasticiser, whose characteristic curves were experimentally obtained, as described below.

For the sake of space, in Fig. 11.13 we report only the results for the specimen at $0\;\%$ plasticiser percentage. In this case, we get an RMSE of $41.3$ mm$^2$, less than $14\;\%$ with respect to the reference value.

Fig. 11.13

a $F(A)$ Control. b Force. Reference area (dashed line) versus Controlled area (continuous line) (a). Force measurement (b)

5.3 Evaluation Experiments

We report a preliminary assessment of the FYD-2 performance by a comparative evaluation of the rendered softness and the objective compliance of real materials. Results of such experiments from one right-handed male participant (age 27) are reported. The participant had no physical limitation that would affect the experimental outcomes.

As in Fujita and Ohmori (2001), the goal was to test how effectively softness discrimination can be elicited by the system. To achieve this objective, we used three different silicone specimens, chosen as in Scilingo et al. (2010). The specimens, whose softness properties were reproduced by the FYD-2, were half–spheres of radius of 20 mm and they were made of material obtained by mixing a given quantity of a commercial bicomponent, room temperature-curing silicone (BJB TC-5005A/B), with a variable percentage of plasticiser (BJB TC-5005C), acting as a softener. The amount of softener in the mixture was chosen as 0, 10 and 20 %, referring to the specimen $SS1$, $SS2$ and $SS3$, respectively. To derive the relationship between the contact force and the contact area we used a custom made characterisation system reported in Fig. 11.14. The procedure was analogous to the one reported in Scilingo et al. (2007).

Fig. 11.14

The system used for the characterisation of the silicone specimens

The system consists of an indenter attached to a servo-controlled linear actuator by Firgelli (L-12-50-100-6-L). The specimens to be characterised were placed under the motor stroke and put in progressive contact with a transparent glass. The indenting velocity was 5 mm/s and the range of force was $0\div 20$ N.

A webcamera—Microsoft LifeCam HD-5000—was put under the glass. As the indenter pushed against the specimen the web cam captured a snapshot of the surface flattened against the plexiglass. In order to enhance contours of contact area a thin white paper behaving as optical filter was placed between the specimen and the plexiglass.

Following the previously described techniques, we obtained the contact area on the basis of heuristically found binarisation thresholds based on luminosity. Furthermore, for each contact area, the indentation force was also measured by means of a load cell placed at the base of the system. In this manner the $F(A)$ curves were obtained for $SS1$, $SS2$ and $SS3$, see Fig. 11.15.

Fig. 11.15

The force/area ($F(A)$) characteristics for the silicone specimens

Fig. 11.16

Confusion matrices showing how the objective compliance was subjectively perceived by one participant

Using the previously adopted terms, let be $SH1$, $SH2$ and $SH3$ the rendered stimuli corresponding to $SS1$, $SS2$ and $SS3$, respectively. The rendered stimuli were obtained by reproducing the $F(A)$ curves of the silicone specimens, using the control scheme described in the previous subsection. $SH1$, $SH2$ and $SH3$ were presented three times in a random order to the participant (right-handed, male, age $30$) and then he was asked to associate them to their physical counterparts. The participant did not have time limitations since he was allowed to touch the silicone specimens and the rendered stimulus as many times as he wanted. The experiment was performed in blind conditions.

Results are shown in Fig. 11.16, where the perception of artificial specimens was associated to the perception of real ones in a confusion matrix structure. In other words, the three stimuli are reported along the row dimension, while the ranked responses are reported along columns.

These results suggest a high degree of reliability in force/area tracking as well as in eliciting overall discriminable levels of softness. Although these results are preliminary and have no statistical significance, they are promising performance for this display. We intend to undertake further testing, with a larger number of participants and a greater number of rendered specimens, to give a more complete assessment of this device.

6 Conclusions

In this chapter we have presented fabric-based softness displays. Such displays, which allow real-time measurements of the contact area, are based on the CASR (Contact Area Spread Rate) paradigm which states that a large part of tactile sensing information for softness discrimination is retained in the relationship between the contact area growth over the finger pad and the increasing indenting force. This paradigm can be regarded as a haptic synergy, since it represents an approximation and reduction between the high dimensional space of the mechanics of touch and human perception. We have described different prototypes of the fabric-based softness display, showing their effectiveness in eliciting a compelling softness sensation, by properly mimicking force-area and stiffness properties of real materials.