Keywords

1 Introduction

Clicks and detents are valuable features used to signal the progression of scrolling down a list or tuning a value. One example is the tactile feedback provided by an indexed volume knob, where the periodic mechanical events are used to precisely control the volume even in the absence of visual feedback. A sense of progression can be created while scrolling on a touchscreen by using force feedback [1] or vibrotactile stimulation [2] or by varying the finger-to-surface friction [3, 4] while the user is exploring the surface.

Natural texture perception provides an interesting framework for creating virtual gratings that are intuitive, enjoyable and easily discriminable. Texture is often perceived via multiple physical quantities such as compliance, thermal conductivity and roughness [5]. The latter characteristic is thought to be one of the main factors contributing to the sensation of tactile texture [6]. Roughness is perceived partly via the changes in lateral force that a finger experiences while it is sliding over a surface [7]. Although the relationship between surface geometry and changes in the friction force is non-linear and stochastic [8, 9], recording and replaying the spectra of these friction force profiles suffices to produce realistic impressions of the roughness [10]. It has been previously reported that the spatial spectra and the magnitude of the lateral force vibrations both contributed to participants’ subjective assessment of texture, and low amplitude and high frequency textures were judged to be more pleasant [11].

Changes in the tangential forces can elicit realistic impressions of relief and texture [12] in subjects exploring a surface with a finger. In the case of virtual spatially-periodic gratings, roughness was found to be correlated with spatial period, modulation amplitude and the nominal friction level [7]. A study on the dimensionality of periodic texture has also shown that frequency is one of the key factors involved in the distinction of texture [13].

It has been found to be of interest to render virtual grating directly on a surface using variable friction approaches based on either ultrasonic levitation [14, 15] or electro-adhesion [16]. To create sharp sensations, square waves have often been used although the falling and rising edges of the steps are perceived differently [17]. A more suitable approach was based on the fact that the friction force elicited by natural textures is composed of a spectrum of frequencies that can vary with time and space [18]. Even purely sinusoidal gratings generate a complex pattern in which fundamentals, harmonics and noise are superimposed [8].

In music, harmonics play a central role in the perception of notes, as well as being responsible for the timbre, which we use to discriminate between one instrument and another. The perception of their haptic counterpart is not well understood. For example it has recently been found that gratings composed of two spatial frequency components (between 2 and 6 \(\mathrm {mm}^{-1}\)) can be perceptively matched to a single intermediate pure tone grating [19]. The aim of the present study was to determine how the fundamental frequency and subsequent harmonics contribute to our judgment and our ability to discriminate between artificial tactile textures simulated on an ultrasonic variable friction device in the same way as the harmonics affect our perception of musical notes. The results obtained here show that the overtone frequency and the strength of the signal are major factors involved in generating the impression of an artificial grating.

2 Experimental Procedure

2.1 Virtual Textures

The virtual gratings are rendered by varying the frictional force of the fingertip on a glass surface as a function of the coordinate of the finger using squeeze film levitation methods. Since changes in the friction force correspond to the user’s hand motion, the periodic fluctuations are perceived as a spatially-distributed corrugated pattern. Textures were encoded along the medial-ulnar axis and defined as a 2000-point vector, where each 12-bit value represented the amplitude of the ultrasonic vibration, and hence the decrease in the friction occurring at a specific point across the 50-mm width of the surface-haptic interface. The 0.25 mm spatial resolution of the gratings was sufficiently low to be unnoticeable.

Fig. 1.
figure 1

The 30 spatially modulated periodic signals used in these experiments and their respective spectra. Each spectrum encodes 3 gratings of different amplitudes.

Full size image

Each of the gratings tested was generated by superimposing two harmonic sinusoidal signals called a tone and an overtone. This waveform was selected for its relative simplicity, easily producing a sharp, uncluttered sensation while showing enough complexity to be able to create a large range of textures. Three factors were varied to create the 30 signals illustrated in Fig. 1. First the spatial period of the fundamental tone was made to vary between \(\lambda \) ={10, 5, 2.5, 1.25, and 0.625} mm in order to simulate a wide range of spatial frequencies. The overtone spatial period was either n = {2, 4 or 8} times smaller than the spatial period of the fundamental. Samples with an overtone spatial period smaller than 0.317 mm were too fine to be simulated faithfully and were therefore discarded. As can be seen from Fig. 1, the first five grating spectra were created with 2 successive frequencies (\(\times \)2 ratio), while only four gratings had a x4 ratio and one grating had a x8 ratio. The relative phase between the fundamental and the overtone was set at 0. Since friction reduction phenomena can only affect the magnitude of the lateral force that the fingertip experiences and not the direction, the baseline friction of each signal was set at 50% of the total range of friction forces that the device could produce. Lastly, the peak-to-peak amplitude of ac part of the textural waveform was made to vary between 3 levels: a = {25%, 56% and 87%} of the total dynamic frictional range.

2.2 Apparatus

The apparatus used here was based on squeeze-film levitation methods whereby the friction of the fingertip touching a glass plate is controlled in real time. The ultrasonic friction reduction device used in the present experiments was built on a rectangular plate (60\(\,\times \,\)50\(\,\times \,\)2 mm\(^3\)) actuated by 4 piezoelectric actuators bonded to the surface. The actuators excited a 5\(\,\times \,\)0 mode at a frequency of 39 kHz with a maximum amplitude of 4 \(\upmu \)m peak-to-peak at the anti-node. With this particular normal mode, users could explore the surface from left to right on an anti-node and therefore have a constant friction modulation power across the width of the interface. The friction coefficient could be varied from an initial value of 0.8 when the ultrasonic vibration was small to a minimum value of around 0.2 when the vibration was at its maximum strength.

The device was equipped with four semiconductor strain-gauge force sensors placed in each corner of the glass plate. These sensors measured the normal force and the position of the center of pressure exerted by the finger using the relative amplitude of the force measured by each sensor. A calibration procedure using a least-square regression ensures an accurate estimation of the position. Semiconductor strain-gauges and low-noise amplifier used in the design resulted to a position resolution of 0.25 mm at a rate of 2 kHz. The real-time loop measured the position of the finger, and sent the appropriate value stored in the signal vector to a digital-to-analog converter. The signal was then multiplied to the ultrasonic carrier, amplified and fed to the actuators. The amplitude-modulated signal thus created changes in the friction force when a fingertip was sliding over the plate.

2.3 Participants and Protocol

Thirty healthy volunteers, 6 women and 24 men, 4 left-handed and 26 right-handed, age ranging from 23 to 57 years (mean 35 years), participated in the study. They were naive to the aims of the study and had no previous experience with surface haptic devices. Participants sitting in front of the haptic surface were provided with a touchscreen interface for giving their answers. They could explore a single grating at a time on the haptic surface without any time or speed constraints. In order to ensure that they perceived a consistent signal, they were instructed to keep their finger movements to the median-ulnar axis, parallel to the nodal lines of the ultrasonic vibration pattern. They wore headphones to prevent them from using any auditory cues.

The experiments were conducted in two sessions. The first session involved a free-sorting procedure, in which participants were asked to place textures that felt similar in a group. Participants could select and move the textures on the touchscreen in front of them. Clicking on a texture would load it onto the haptic device. They were free to make as many groups as they wanted, but all the gratings had to be placed in one of the groups. In the second session, participants were presented with all the textures sequentially and asked to rate their pleasantness to the touch on a scale ranging from “not pleasant at all” to “very pleasant”. The textures were presented in Williams Latin Square Order to prevent the occurrence of any position and succession effects.

3 Multidimensional Scaling Analysis

Participants made from 2 to 9 groups of gratings, with the majority choosing 3 or 4 groups, as shown in Fig. 2a. A similarity matrix was constructed from the groups. Whenever two gratings were placed in the same group, they scored one similarity point. This matrix was then transformed into a dissimilarity matrix by normalizing it to unity and subtracting it from 1, and the result was used as an input in the non-metric Multidimensional Scaling analysis (MDS).

The aim of the MDS algorithm was to place each grating in an N-dimensional space so that the distances between gratings, regarded here as perceptual differences, would show up as clearly as possible. Coordinates were then assigned to each grating in each of the N dimensions. Figure 2b shows the evolution of the fitting stress as a function of the number of dimensions chosen for the analysis. A model with a high number of dimensions captures more of the variability of the data, but at the expense of a less parsimonious description. The stress curve shows a knee in the case of a 2 dimensional model. The rest of the study was therefore performed using a 2 dimensional space.

Fig. 2.
figure 2

a. Number of groups formed by the subjects during the free-sorting task. b. Multidimensional Scaling Stress depending on the number of dimensions. c. 2-dimensional perceptual space obtained with the MDS algorithm. Numbers refer to the name of the grating. Amplitude and overtone frequency are indicated by the size and color of each point respectively. The signal parameters \(f_{fund}\), \(f_{over}\) and a are projected as arrows, with their size showing the goodness of fit.

Full size image

Figure 2c shows the positions of the gratings in the 2-dimensional space. In this figure, the more perceptually similar two gratings are, the closer together they are placed. The first dimension is called the principal perceptual axis because it has a better discriminatory power.

In order to examine these dimensions more closely, we searched for models explaining the dimensions in terms of the 3 main construction parameters: the amplitude and the two defining frequencies (the fundamental and overtone frequencies) using polynomial regressions. We noted that the regressions were more accurate when the frequencies were expressed as a logarithmic ratio to the first harmonic frequency \(f_{0}=0.1~\mathrm {mm}^{-1}\) such as \(\log (f/f_{0})\). One possible reason why the logarithmic function performed better than the linear function is that spatial frequency judgments are made along a logarithmic scale [20].

3.1 First Dimension

The amplitude was almost invariant to change in the case of the first dimension (linear regression, \(R^{2}=0.004\)) but was affected by the frequency of the fundamental and that of the overtone. As shown in Fig. 2c, the amplitude a vector obtained was perpendicular to the first dimension, while the frequencies were almost colinear with the first dimension. Interestingly, linear regressions of the frequencies showed that the goodness of fit with the dimension of the frequency of the overtone (\(R^{2}=0.92\)) was greater than that of the frequency of the fundamental vibration (\(R^{2}=0.69\)). Higher polynomial orders did not significantly improve the fit.

Fig. 3.
figure 3

Linear regression of dimension 1 coordinates of each grating with the frequency of their (a) fundamental and (b) overtone.

Full size image

The data and the linear regression on the fundamental frequency and the overtone frequency are presented in Fig. 3. One of the reasons why a less than ideal regression was obtained with the frequency of the fundamental, is that three of the samples (Nos. 28, 29 and 30) were outliers. These samples were also those giving the highest ratio between the fundamental and the overtone, which suggests that the participants may have decided to classify the texture based on the height of the pitch rather than on the low frequency component. Since the principal perceptual axis is best explained by the overtone, it can be assumed that this axis mainly determined the perceptual quality of a texture, at least to a larger extent than the frequency of the fundamental and the amplitude of the signal Since the overtone had the higher frequency, this might also indicate that most efficient way of distinguishing between two friction-modulated haptic gratings consists in comparing the shortest spatial variations.

3.2 Second Dimension

Conversely to the first dimension, and in accordance with the Fig. 2c, the second dimension is invariant to both frequencies (linear regressions, both \(R^{2}<0.001\)). In contrast, it is well explained by the amplitude of the gratings (linear regression, \(R^{2}=0.63\)). The linear regression is shown in Fig. 4a. Figure 4b illustrates that even if the data are best explained by a linear relationship with the amplitude, it seems that the error between the regression and the data is weakly dependent on the frequency, without major differences between the frequency of the fundamental (quadratic regression, \(R^{2}=0.17\)) and frequency of the overtone (quadratic regression, \(R^{2}=0.17\)).

Fig. 4.
figure 4

Regressions of the second dimension coordinates of each grating depending on (a) the amplitude and (b) the overtone frequency. (c). Mixed model in which the amplitude is combined with the overtone frequency to predict the dimension 2 data.

Full size image

A multi-factor polynomial regression performed using a linear relationship on the amplitude and the square of the log of the overtone frequency, resulted in a better fit with the data (\(R^{2}=0.79\)) than the simple linear regression performed on the amplitude.

4 Subjective Ratings of Pleasantness

After completing the free sorting experiment, participants were asked to rate the pleasantness of each texture on a 7-points Likert-type scale ranging from “not at all pleasant” to “very pleasant”. Each grating was therefore awarded 30 pleasantness scores ranging from 1 to 7.

Subjects’ opinions as to which textures were more pleasant than the others were not always in agreement. A hierarchical clustering algorithm based on a principal component analysis was performed to place subjects in groups depending on their responses, as shown in the dendrogram presented Fig. 5a. The simplest method of segmentation consisted in separating 5 individuals giving unusual answers from the majority (25 participants). Based on these results, the group of 5 participants whose answers were significantly different from the others were classified as outliers.

Fig. 5.
figure 5

(a). Dendrogram resulting from the hierarchical cluster analysis on subjects’ clustering patterns. (b) Bilinear regression of pleasantness with respect to the overtone frequency and the amplitude in the case of each grating. (c). Results in terms of the pleasantness of textures in the 2 dimensional MDS space

Full size image

Pleasantness scores were averaged in order to determine the correlations with the constitutional parameters of the grating. Pleasantness was found to be linearly correlated with the amplitude of the modulation (linear regression, \(R^{2}=0.42\)) and the frequencies. In line with previous findings, the correlations were greater with the overtone frequency (linear regression, \(R^{2}=0.40\)) than with the fundamental frequency (linear regression, \(R^{2}=0.34\)). This difference supports the hypothesis that higher frequencies were the main factor on which the participants’ perception depended. Higher polynomials did not significantly improve the correlations. A bilinear model (bilinear regression, \(R^{2}=0.82\)) gave a satisfactory fit, see Fig. 5b. The fact that the most pleasant gratings in the opinion of the remaining 25 participants were those which had the lowest overtone frequencies and the lowest amplitudes suggests that subtle low-pitch textures are the most pleasant to the touch. The projection of the data onto the multi-dimensional space showed that the pleasantness rating is a vector that falls between the opposite frequencies and amplitude vectors, see Fig. 5c.

The same analysis performed on the 5 outliers yielded exactly the opposite results as far as the subjects’ pleasantness judgments were concerned. The outliers preferred high-amplitude and high-frequency gratings, possibly because of their distinctive quality.

5 Discussion

Overall, the results obtained in this study show that the perception of two-tone periodic friction-modulated gratings depends mainly on the spatial frequencies, in line with previous findings on the perceptual effects of haptic icons presented in terms of force feedback [13], and that the frequency is a strong predictor of texture saliency. The textures tested in this study were composed of two superimposed harmonics with a frequency ratio ranging from 2 to 8. The gratings with the highest ratio between the frequency of the fundamental and that of the overtone (28, 29, 30) had a negative impact on the correlation between the fundamental frequency and the first dimension, but strengthened the correlation with the overtone frequency. This indicates that when two frequencies differ conspicuously, the higher frequency of the two provides the clearest distinctive cues. The prevalence of the overtone frequency is probably due to subjects having a more acute sensitivity to the high frequencies involved in the fast undulations [21]. A larger set of frequency ratios should now be tested in order to confirm whether the higher frequencies have the greatest effects on the perceptual quality of periodic textures. Also, because the dynamic of the friction force modulation is highly variable, future experiments will take advantage of finer control of the lateral forces provided by friction force feedback [22].

The results obtained on the second dimension show that the amplitude of the modulation was also a major factor in the perception of textural pleasantness. The amplitude of the modulation can be thought as the strength of the sensation. The correlation between the amplitude and the second dimension was not as straightforward as that observed between the frequency and the first dimension: the frequency content also affects the second dimension. In particular, the medium spatial frequencies (corresponding to spatial periods of 0.63 and 1.25 mm) required less amplitude than the high and low frequencies to be perceived as strongly as the latter. Since the subjects’ typical speed of exploration was around 100 mm/s, this would correspond to temporal frequencies in the 80 to 160 Hz range, which are perceived at lower perceptual thresholds [23]. It is worth noting that in the present study, the finger velocity was unconstrained, so temporal signals produced could differ. However the perception of the textures is known to be invariant to exploration speed [24].

Mapping the signal parameters in two dimensions is a useful means of obtaining a set of textures which are easily distinguishable. However, the perceptual space alone does not tell us much about the subjective quality of each texture. The pleasantness ratings given by the participants certainly give an idea of how the frequency content and the amplitude contributed to the perceived quality of each texture. The majority of the participants (25 out of 30) preferred a low frequency content and a low amplitude, in line with the results obtained in previous studies in which participants were asked to judge the pleasantness of natural textures [11]. These findings should be taken into account in order to create enjoyable tactile interfaces. However, presenting subjects with an unpleasant grating (one with a high amplitude and a high spatial frequency) may also be a useful means of obtaining information such as the existence of an undesirable area or the limits of a setting.

6 Conclusion

In this study, participants were asked to form groups of similar textures, which were simulated by superimposing two sinusoidal waves with spatial frequencies differing by an integer multiple. The multidimensional analysis performed here showed that the frequency content, especially the overtone frequency, was the main descriptor used by the subjects, followed by the strength of the vibration. The participants’ pleasantness ratings showed that their preferred textures had a low frequency content and a low amplitude.

The results presented here should provide a useful basis for creating artificial sensations for mimicking the sensations which occur when tuning a indexed slider directly on a touchscreen. The spatial presentation generates salient distinguishable sensations that can be modulated on a pleasantness scale.