Keywords

1 Introduction

Surface-haptic promises to restore the tangibility of virtual interfaces while users are interacting with flat and featureless touchscreens. Virtual bumps, texture, clicks and scrolling effects produce sensations that guide the users’ motion and provide them with feedback about their actions. One of the most promising approaches developed to date for generating rich perceptual sensations consists in modulating the friction between the user’s finger and the plate via either electrostatic adhesion [1] or ultrasonic squeeze-film levitation [2, 3]. The main advantage of these methods over vibrotactile stimulation is that they produce stimuli which not only include transient events –i.e. texture, impacts or vibrations– but affects continuous forces exerted by a sliding fingertip. Modulating these forces makes it possible to induce of tactile illusions of shape and large features, such as bumps and holes, similar to what could be achieved with robotic force feedback devices [4, 5].

In spite of their advantages, friction modulation methods tend to produce signals that are distorted and attenuated [6, 7]. The main factor responsible for the signal variance is the complexity of the frictional behavior of a finger sliding over a glass plate. The angle of the finger, the pressure force, and the moisture of the skin all affect the friction force [8]. Worst still, even if the exploration conditions are accounted for, sweating and the subsequent softening of the stratum corneum induce large changes in the friction with time [9].

In addition the fact that they both rely on friction to produce forces to the fingertip, each of the above two technologies has its own drawbacks. Despite the advantages of electro-adhesion methods, such as their large functional bandwidth and their compatibility with strong forces, the occurrence of non-linearities results in the distortion of the signal [1]. Squeeze-film levitation methods based on ultrasonic standing waves make it possible to modulate the friction force in a wider dynamic range, but have disadvantages such as poor frequency responses [10] and large static non-linearities [11]. It has been proposed to combine both methods in order to increase the dynamic range of possible signals [12, 13], but this does not solve the variability inherent to friction force modulation processes.

Feedforward filters can be used to counteract the attenuation inherent to ultrasonic friction modulation and provide users with fast and clear signals. Dynamics is improved by temporarily overdriving the actuators in order to quickly reach a given friction force [14]. Distortions due to static non-linearities can also be corrected using a look-up table [15]. Although these feedforward model-based approaches give better performances than those of open-loop systems, the inherent variability prevents to precisely regulate a given force. Ben Messaoud et al. recently proposed to tackle these drawbacks by introducing real-time measurements of the friction force that are fed to a closed-loop feedback control able to reject unknown disturbances [16]. The control strategy relies on a sliding mode strategy to robustly remove static error and improve the dynamic response.

However, like any closed-loop control system, noise from the force sensor was re-injected into the actuators, which resulted in noisy stimulus. In order to overcome the limitations of this approach, we developed a custom built capacitive sensor which is able to resolve the force with a dynamic range ratio of more than 1:50,000. This sensor is used as the backbone of a closed-loop force feedback scheme which is able to produce low noise signal, yet accurately control of the friction force by means of a state-of-the-art proportional-integral controller.

2 Simulations

2.1 Linearized Dynamic Model of Ultrasonic Friction Modulation

In order to account for the stochastic nature of ultrasonic friction modulation and design the most appropriate controller, the relationship between the amplitude set on the plate and the frictional force that is actually produced has to be modeled. We recorded the force and amplitude data of 26 participants while they were sliding there finger on a glass plate excited with a 30 kHz ultrasonic carrier of 2.5 \(\upmu \)m amplitude modulated by a 1 Hz sine wave. Participants were free to explore the surface which resulted in a dataset that contained various exploration speed, forces and finger posture. Datapoints showing force as a function of the vibration amplitude are presented in Fig. 1a along with statistics across trials on binned data. Although the individual trials often showed the existence of a clearly negative correlation between the vibration amplitude and the friction force, the relationships observed from trial to trial were not consistent. To account for this behavior in simulations, the non-linear Gaussian model described in [11] was extended by adding an unknown perturbation d, which was bounded so as to prevent the occurrence of negative friction force values. The modified friction force model was therefore:

$$\begin{aligned} f=f_0\, \exp \left( -\frac{a^2}{2\tau ^2}\right) + d = g(a) + d \end{aligned}$$
(1)

where \(f_0 = 0.8\) N is the nominal friction force, a is the vibration amplitude, and \(\tau = 1.32\,\upmu \mathrm {m}\) is the susceptibility to ultrasonic levitation. The deterministic behavior is denoted by the function g(a), and d is taken to be a 1 Hz sine wave with an amplitude of 0.8 N and an offset of 0.4 N.

Fig. 1.
figure 1

a. Friction force and amplitude data points recorded in 26 sliding trials are shown in gray. Means and standard deviations are presented in black. b. Block diagram of the control scheme.

Full size image

2.2 Control Strategy

The block diagram of the control strategy implemented in this study is shown in Fig. 1b. The friction modulation process, in which the amplitude is the input, delivers a friction force. This process is modeled by means of a linear time-invariant function serving as a first order low pass filter with a cutoff frequency of 100 Hz, in line with the attenuation reported in [10]. The filter is followed by the non-linear relationship described by Eq. 1 to complete the model for the friction modulation process.

A lookup table is implemented in the controller to compensate for the non-linearity of the ultrasonic friction modulation captured by Eq. 1. The lookup table and the friction modulation process combines into a single linear time-invariant transfer function called P(s) , to which a saturation function is added, where the amplitude of the ultrasonic wave is bounded between 0 and 2.5 \(\upmu \)m. The controller C(s) therefore has to compensate for the corrected process P(s), which acts as a first-order low-pass filter, and for a slowly evolving unknown disturbance, d.

A force sensor measures the friction force applied to the finger along with some undesirable noise n. This parameter typically fits a zero-mean random Gaussian process. The dynamics of the sensor are modeled by means of the transfer function S(s), which, to simplify the simulation, is taken to be a low-pass filter with a cutoff frequency of 1 kHz. The force measured \(f_m\) is then subtracted from the set-point force \(f_r\) in order to compute the error e, which is subsequently fed to the controller.

2.3 Precision and Accuracy Trade-Off

The ideal control scheme would have a fast, effective means of disturbance rejection without having to add any noise to the friction force measured by the system. In practice, these two objectives impose opposite constraints on the controller. In the current implementation, a proportional-integral controller was adopted so that \(C(s) = K_p+K_i/s\). A larger set of gains \(K_p\) and \(K_i\) ensures fast convergence, but results in the amplification of the noise introduced by the sensor. This noise is detrimental to the user’s tactile experience because it adds fluctuations that are not part of the original signal. This trade-off between the convergence to a specific value (the accuracy) and the noise injected (the precision) in the closed loop has to be evaluated, giving, the maximum noise that the sensor can generate.

From the block diagram described in Fig. 1, taking only the linear behavior into account, the expression for the force perceived by the finger in the Laplace domain can be expressed as follows:

$$\begin{aligned} f_{a}=d\,\frac{1}{1+G}+f_{r}\,\frac{CP}{1+G}-n\,\frac{G}{1+G} \end{aligned}$$
(2)

where \(G(s) = C(s)P(s)S(s)\). To study the effects of the noise n on the variance of the friction force \(\sigma ^2_f\), the variance of the setpoint and the disturbance can be taken to be null in the steady state. Error propagation analysis therefore gives the following equation:

$$\begin{aligned} \sigma ^2_f=\left| \frac{G}{1+G}\right| ^2 \, \sigma ^2_n \end{aligned}$$
(3)

This equation shows that the noise perceived by the user, \(\sigma ^2_f\) is affected by the gain of the controller. A lower gain attenuates the noise originating from the sensors. However, since the disturbance is affected by the sensitivity function \(\frac{1}{1+G}\), which favors high gains, an optimum can be found.

Fig. 2.
figure 2

a. The fidelity of the control given by the \(R^2\) parameter in comparison with \(f_a=f_r\) in the case of a set of \(K_ {i} \) and \(K_ {p} \) coefficients. b. \(K_ {i} =2\) and \(K_ {p} =40\). Over-corrected signal results in noisy signals. c. \(K_{i}=2\) and \(K_ {p} =10\). The right balance between a high gain and fast convergence. d. \(K_ {i} =2\) and \(K_ {p} =1\). Low gains give unsatisfactory disturbance rejection. e. Effects of greater sensor noise on the precision of the control. f. Greater noise results in significant saturation of the controller.

Full size image

Figure 2a shows the results of a set of simulations, where the controller’s parameters \(K_p\) and \(K_i\) were varied and the signal fidelity of the control was measured. The fidelity of the system is reflected in the goodness-of-fit \(R^2\) between the input and the output of the simulation in comparison with an ideal transfer function in which \(f_a = f_r\). Both the disturbance rejection and the noise attenuation have a positive impact on the fidelity. In this simulation, the setpoint signal was a logarithmically-swept sinusoid ranging from 1 Hz to 100 Hz with an amplitude of 1.2 N, the disturbance was a 1 Hz sine wave with an amplitude of 0.4 N and the noise was a white Gaussian noise with an amplitude of \(10^{-5}\)N. As shown by the graph in Fig. 2, proportional gains \(K_p\) ranging between 5 and 20 provide an efficient disturbance rejection while preventing any significant feedback noise. The integral term \(K_i\) is responsible for rejecting the static error, but as long as it is above the value of 2, its tuning has fairly little influence. The input-output relationship with three different proportional gains \(K_p = [40,10,1] \) is shown in Fig. 2b,c and d, respectively. A high gain, as in Fig. 2b, results in a noisy and saturated command, whereas a low gain, as in Fig. 2d, results in a low precision controller. The optimum efficiency was found to occur at values of \(K_p = 10\) and \(K_i = 2\), as shown in Fig. 2c.

The influence of the noise on the fidelity and the saturation of the actuation is shown in Fig. 2e,f. The saturation index was calculated from the ratio between the number of data points having a saturated input and the total number of data points. Noise levels above \(10^ {-4}\) N in the force sensor will significantly decrease the precision and increase the likelihood of saturating the output. We therefore adopted a proportional gain of \(K_p = 10\).

Based on the results of the simulation, it is clear that the effectiveness of the controller depends on the noise added by the sensor, and that low noise results in a sharp, untainted tactile stimulation. It has been estimated that the lowest force perceivable by the human somatosensory system is in the \(5.10^{-4}\) N range [14]. The output noise produced by a sensor with a noise level of \(5.10^{-5}\) N remains subliminal.

3 Friction Force Sensor Design

The output of the simulation reveals that the force sensor is the centerpiece of the control strategy, and proper care must be taken in the choice of technology and design to achieve high-fidelity tactile renderings on surface-haptic devices.

3.1 Performance Requirements

To ensure that the force perceived by the user will remain untainted by the noise, while being able to support forces as high as 2.5 N, the sensor noise must be less than \(5.10^{-5}\) N. These requirements lead to a dynamic range figure of 1:50,000 or about 95 dB. The stiffness and frequency bandwidth are also crucial factors, since they directly affect the sensor’s response via the transfer function S(s) . A softer sensor might have a high signal to noise ratio, but this will be at the expense of a low frequency bandwidth. The glass plate typically weights approximately \(m_p = 400\)g, and the sensor’s first resonance frequency is prescribed by \(f_0 = 1/2\pi \sqrt{{k}/{m_p}} = 250\,\mathrm {Hz}\) in the case of a sensor stiffness of 1N.\(\upmu \)m. This means that in order to resolve 2 N, the sensor will be displaced by 2 \(\upmu \)m. Since force sensors are based on the measured displacement of a known elastic structure, the underlying displacement sensor should have a minimum noise of 0.5 pm.

The sensitivity requirements rule out the use of metal and semiconductor strain-gauge sensors, which usually have signal-to-noise ratios of the order of 1:1,000 in rigid load-cells. Piezoelectric sensors, while having an exceptionally high signal-to-noise ratios and high stiffness, are not suitable for closed-loop feedback applications because of the low-frequency drift, which would require frequent resetting of the control loop. All these constraints are within the range of capacitive sensors, which are capable of a dynamic range within the present requirements while remaining impervious to drift [17].

Fig. 3.
figure 3

Mechanical and electrical schematics of the capacitive force sensor, illustrated on one axis

Full size image

3.2 Implementation

The capacitive force sensor is based on the architecture illustrated in Fig. 3. The ultrasonic plate is fixed to a frame that is suspended by four curved leaf springs grounded to the rest of the device. The curved leaf spring have a relatively high mobility in the plane of the surface, with a lateral stiffness of \(6.10^{6}\) N/m, while maintain rigidity three order of magnitude higher along the normal axis which keep it stiff. This structure constrains the motion of the inner frame to two translations collinear with the friction force. The deformation of the leaf spring is determined by four non-contact sensors, each of which measures the distance of each edge of the suspended frame to the grounded frame. The measurements are differential in each dimension, with one sensor experiencing a reduction in the gap, while the one on the opposite side detects an increase. This differential amplification method counteracts the thermal effects and unwanted electromagnetic perturbations. Each sensor is composed of one rectangular active electrode, surrounded by a single trace connected to ground that shields the active electrode from external perturbation. These electrodes are made out of 75 \(\upmu \)m-thick copper coated using an electroless nickel immersion gold process which provides a smooth finish and immunity to corrosion (pcb-pool, Aarbergen, Germany). The copper trace is mounted on a FR4 substrate for a total active area of \(40\times 4.5\mathrm {mm}^2\). An intermediate part maintains these electrodes parallel to the inner frame, which is conductive and excited by a 120 kHz sinusoidal signal V, polarizing the sensing electrode. Charges \(q_\pm \) are proportional to the voltage V and inversely proportional to the distance \(e_\pm \) such that \(q_\pm = V\, \epsilon _0 A/e_\pm \), where \(\epsilon _0\) is the permittivity of the air and A is the active area of the sensors.

The capacitive measurement circuit is adapted from the low-noise topology described in [17]. The charges from the sensing electrodes are transformed into a voltage via a charge amplifier and the signal is then passed through a selective filter with a center frequency matching the excitation of 120 kHz. After the filter, there is no artifacts left in the signal. At that stage, the envelope of the signal, which depends on the distance between the electrode and the outer frame, is recovered using synchronous demodulation followed by a low pass filter with 1 kHz cutoff frequency. The last operation takes the difference between the signal coming from two electrodes to recover the signed voltage \(V_s\) that reflects the displacement of the inner frame and therefore the force that is applied by the fingertip.

3.3 Sensor Characterization

The sensor was calibrated under quasi-static conditions, using a set of standard weights which applied to the frame a known gravitational force via a string and pulley system. The results are presented in Fig. 4a, and the linear regression showed a goodness of fit of \(R^2 = 0.95\). The frequency bandwidth measurements were performed using an impact hammer. The signal was then normalized and converted into the maximum sensing value. The frequency response showed the occurrence of an initial resonance at 145 Hz owing to the large glass plate. The noise of the sensor studied using a 10 s sample without any external perturbations show the existence of a floor at \(5.10^{-5}\) N in the low frequency and an attenuation above 300 Hz, due to a first order filter. The frequency response and noise spectrum are shown in Fig. 4b. The dynamic range of the sensor before its first resonance is 1:50,000, or about 93 dB.

Fig. 4.
figure 4

a. Linearity of the lateral force sensor b. Frequency spectrum of the noise and impulse response c. Picture of the device

Full size image

4 Friction Force Feedback

4.1 Hardware

A picture of the final device is shown in Fig. 4c. A glass plate measuring 255 \(\times \) 140 \(\times \) 3 mm\(^3\) vibrating at a frequency of 46 kHz in a 16 \(\times \) 0 mode provided the friction modulation. Four piezoelectric actuators are bonded to the plate and provide a maximum displacement of 2.5 \(\upmu \)m. The plate is clamped to the inner frame at the end of 3 nodal lines on each side. The inner and outer frame and curved leaf springs were milled out of a block of 2024-grade aluminum alloy.

In addition to the lateral force sensor, the system is equipped with an optical position sensor based on [14], not used in this study, and 4 normal force sensors at each corner to measure the user’s finger normal force. Finger position and normal force sensors can drive the production of complex virtual environments. The real-time control is ensured by a low-latency 5 kHz timer implemented on a micro-controller (Teensy 3.1, PRJC, Portland, or, USA). Every cycle the sensors values are acquired, the PI control and the look-up table are computed and the amplitude of the ultrasonic carrier is modulated. The amplitude modulated ultrasonic carrier is then filtered with a 10 to 50 kHz bandpass filter and amplified before being sent to the piezoelectric actuators. The controller parameters were set at \(K_p = 10\) and \(K_i = 2\) in accordance with Sect. 2.3.

4.2 Closed-Loop Performance

Comparisons were made between the open-loop and closed-loop modes of operation of the friction modulation using a commanded force \(f_r\) matching a 1 Hz sine wave between 0.3 and 0.9 N, which corresponds to a range of reliably achievable forces. This friction range was implemented using three 1 Hz sine waves 0.1 N in amplitude with a mean value of [0.4 0.6 0.8] N as the setpoint to minimize the variations in the friction force. One participant slid his finger along the interface at 20 mm.s\(^{-1}\) 5 times with each force signal and 50 periods of the modulation were recorded. As shown in Fig. 5a, the open-loop response poorly matched the commanded force since the linear regression showed a goodness of fit of only \(R^2=0. 22\). The closed loop condition gave satisfactory tracking of the setpoint, since the linear regression yielded a goodness of fit of \(R^2 = 0.98\). Low friction values caused most of the errors, partly because the large amplitudes required could not be reached with the present ultrasonic plate.

Fig. 5.
figure 5

Friction force feedback control. a. The closed loop limits the variability and compensate for the static non-linearity. b. The system response to successive unit steps shows a response time of around 30 ms. c. During a unit step, the commanded amplitude of the ultrasonic waves overshoots.

Full size image

The step responses recorded are shown in Fig. 5b. The frictional force was tracked with a response time of \(37\pm 20\) ms (SD) with a decreasing friction step and \(32\pm 16\) ms with an increasing friction step, with an exploration speed of approximately 20 mm.s\(^{-1}\) in both cases. The results shown in this figure can be improved with larger gains in the closed loop at the expense of greater noise. The traces in the figure show that the steady state friction was not as noise-free as one might have expected in view of the signal to noise ratio of the sensor. It seems likely that the additional fluctuations may have been due to friction noise and tremors of the user’s hand. Figure 5c gives a close-up view of the control strategy in a single step. In order to achieve fast responses, the controller overshoots in amplitude before converging back to a lower steady-state value to stabilize the output.

5 Discussion

The disturbance rejection of the closed-loop force feedback is on a par with the results obtained by [16], who used a strain gauge sensor and a sliding mode controller. The steady state friction is kept within 20 mN of the set point, which is a substantial improvement over the 0.25 N variations which occur when ultrasonic friction modulation is operating in the open-loop mode. The custom-built force sensor has a noise floor of less than \(10^{-4}\)N, and the gain of the controller is set so as to limit the noise fed back to the actuator, which results in the production of tactile stimuli that are sharp and devoid of artifacts. Yet the traces in Fig. 5b show that the actual friction force undergoes fluctuations in the order of 20 mN. These fluctuations are also present in open loop control [10] and in closed loop control that uses strain-gauge load cells [16]. The remaining fluctuations are probably due to physiological and frictional noise [18], which the feedback system struggles to cancel. A frequency synthesis of the control taking into account the friction noise might help to develop a higher order controller with which these fluctuations could be ironed out.

It is worth noting that the present simulation of the entire system revealed that in order to obtain a specific noise level in the output force, the sensor noise has to be one order of magnitude lower. This particular ratio, which was not predicted by the noise sensitivity function based on linear control theory, may have resulted from the non linearity present in the real-life system. The response time is also reduced from 50 ms to 20 ms with respect to open-loop condition, at slow sliding speeds. These results could be further improved by using a more powerful actuator, since the current dynamics were limited only by the maximum amplitude that the plate can reach.

6 Conclusion

In this paper, we report on the simulation and implementation of a surface-haptic device which can be used to deliver precise and accurate stimuli to a user’s sliding finger in the dynamic range of 0.3 to 0.9 N. In particular, it was established that the combination of a force sensor with a wide dynamic range (1:50,000) and a PI controller sufficed to considerably reduce the variability of the stimuli produced by the ultrasonic friction modulation.

The performances of the current device were limited by the power of the ultrasonic plate, and ongoing engineering studies are now focusing on building a more powerful actuator which is compatible with fast, large changes in the amplitude. Thanks to the online regulation of the friction force, frictional stimuli with great precision and accuracy are delivered, and this interface therefore provides a valuable tool for use in psychophysical experiments requiring finely controlled stimuli [19]. The improved sharpness and fidelity are also key assets for simulating unambiguous virtual environments on user interfaces.