1 Introduction

Mechanoreceptors within human hands provide various tactile afferent information, which makes it possible for human to perceive and manipulate objects, and further interact with surrounding environments. There are mainly four types of mechanoreceptors including FA-I FA-II, SA-I and SA-II, which are responsible for the perception of different frequency bands of forces [1, 2]. The FA types enable human to feel dynamic forces, such as vibrations, while SAs help us sense static forces, e.g., grasping force. A combination of numerous mechanoreceptors not only tells us precise afferent information of different objects, but provides necessary feedback in dexterous manipulations, which plays a significant role in the development of prosthesis, manipulators, surgical robots, and so on [3,4,5].

So far, a variety of tactile sensors have been developed based on different working principles. As a major research field, electronic skin receives numerous attentions from worldwide researchers [6,7,8,9,10], which focuses on developing a type of very thin and light film that is sensitive to outer stimuli. MEMS (Micro-electromechanical Systems) technique is usually required to manufacture such e-skins while more attention in this field is mostly paid to novel material or microstructure designs [11,12,13]. Although extremely high sensitivity can be achieved when the film is attached to a robotic hand, expensive cost and low robustness limit its applications. Force measurement equipment that is designed for specialized functionalities falls into another category of tactile sensors, such as the wearable sensor for three-axis force measurement and the sensing bumps to measure normal and shear pressures [14, 15]. This type of sensors is usually used in special occasions and inapplicable to prosthesis or robotic hand directly.

To improve the robustness, various tactile fingers are designed inspired by human fingers. Piezo-resistive material is widely used to measure dynamic forces and the most commonly used type is PVDF film. Its responses to dynamic forces have been validated via sinusoidal stimuli [16]. A four-piezo-resistor-integrated sensor array was embedded into a fingertip to discriminate three roughness surfaces reaching an accuracy not less than 97.6% [17]. In plenty of designs, only pressure or torque sensors were attached to the finger surface to perceive contact forces [18,19,20,21]. Only static force sensors were enough to perceive some properties of objects, such as beef tenderness [22]. In addition to strain gages, pressure sensors, custom three-axis tactile sensors, and torque sensors were usually chosen as core sensing elements. Both strain gages and PVDFs were embedded into two layers of soft material in a random manner, which was aimed at mimicking the distribution of mechanoreceptors within human skin [23]. These two types of sensing elements were attached to the inner surface of the finger skin and then assembled with the finger part, which could detect and prevent slips [24]. A novel fingertip was designed with a curved surface cured as soft contact area, in which PVDF films and force sensing resistors were embedded into a rigid-soft-mixed finger [25]. Besides, new emerging sensing techniques promoted the invention of tactile sensors in different working principles, such as vision-based tactile sensors [26], measurement taking advantage of magnetic flux density [27], fiber Bragg grating structure [28], proprioceptive-sensing-based approach [29], and so on.

In our previous works, a low-cost and easy-to-fabricate tactile fingertip was designed with only two PVDF films contained [30,31,32]. High accuracy was achieved when it was applied to discriminate 8 surface roughness, which also proved that the spatial distribution of sensing elements was significant. In this paper, a novel tactile fingertip was designed with two perpendicular strain gages and two perpendicular PVDF films embedded into a soft material. In addition to the difference in sensing element design, platinum cured silicone was utilized to mimic the soft skin, which is softer than our previous used material and more similar to human skin. A preprocessing system is designed to convert the resistor changes into voltages and then filter and amplify the four-channel signals. Three different experiments are conducted to explore the responses of our developed tactile sensor to deformations induced by static forces, the performance in discriminating different contact status and the accuracy in recognizing six object surfaces. Excellent performance proves the effectiveness of our developed bio-inspired tactile fingertip in perceiving both static and dynamic forces.

2 Design of a Bio-Inspired Tactile Fingertip

2.1 Tactile Fingertip

The structure of our developed bio-inspired tactile fingertip is shown in Fig. 1, where the mechanoreceptors in hairless skin of human hands [1, 33], the schematic structure of the fingertip, and the actual tactile sensor are depicted respectively. Meissner and Pacinian endings belong to FA type which are replaced by PVDF films in our design, while SA type mechanoreceptors, i.e., Merkel and Ruffini endings, are substituted by strain gages. It should be noted that any PVDF film or strain gage is perpendicular to the other one to sense forces from different directions. PMMA bar, electric wires and platinum cured silicone play the roles of finger bone, nerve fiber and skin, respectively. A silicone cube is attached to the end of PMMA bar to support the four sensing elements.

Fig. 1
figure 1

Design of the bio-inspired tactile fingertip. a Mechanoreceptors in hairless skin of human hands [1, 33]. b Schematic structure of the designed fingertip. c Actual fingertip and its mold for fabrication

Full size image

The fabrication process of our designed tactile fingertip is shown in Fig. 2. At first, the PVDF films and strain gages need to be prepared with electrodes and wires fixed to them using conductive paste. After initial test of the electronic signal from the four sensing elements, they are attached to a silicone cube, which can be manufactured with a mold. Then the cube is fixed to the end of a PMMA bar. Prepare the silicone mixture with two different parts of the platinum cured silicone at a mass ratio of 1:1 and degas the mixed fluid to avoid possible gas cavities after curing. Insert the PMMA bar as well as other parts into the mold, and pour the silicone mixture into it. A tactile fingertip is obtained after an hour for curing.

Fig. 2
figure 2

The manufacture process of our designed tactile fingertip

Full size image

2.2 Electric Circuit for Analog Signal Preprocessing

Before the sensor data are collected, a preprocessing system is necessary, which is mainly responsible for amplification, filtering and conversion of the analog signals. An electric circuit is designed and a part of it is shown in Fig. 3. As PVDF is a self-powered piezoelectric sensor, a voltage difference will be produced between two sides of the film upon stimuli, which can be measured directly. Amplification and filtering are implemented to improve the quality of the outputted signal. The final output for PVDF can be expressed as Eq. (1):

$$ U_{output1} = \Delta U \cdot A_{1} $$
(1)

where \(\Delta U={PVDF1}_{+}-{PVDF1}_{-}\) is the voltage difference between the two sides of a PVDF film. \({A}_{1}\) is the amplification factor determined by the chosen operational amplifier. In this work, AD620 is chosen and \({\mathrm{A}}_{1}=49.4\mathrm{k}\Omega /{R}_{2}+1\). A low pass filter is designed to denoise the signal, where the cut-off frequency is \({f}_{C1}=1/(2\pi {R}_{1}{C}_{1})\).

Fig. 3
figure 3

Diagram for a part of the electric circuit for analog signal preprocessing. a Circuit for PVDF films. b Circuit for strain gages. The dashed frame indicates the conversion part that applies Wheatstone bridge

Full size image

Strain gage is used to measure the static force, whose resistor value will change when deformed. The resistor change needs to be converted to voltage for collection. As shown in Fig. 3b, Wheatstone bridge is applied since the change value is too tiny to be measured, after which further amplification and filtering are implemented. The final output for strain gages is given in Eq. (2):

$$ U_{output2} = \left( {\frac{{RSG1_{1} - RSG1_{2} }}{{RSG1_{1} - RSG1_{2} + R_{3} }} - \frac{{R_{6} }}{{R_{4} + R_{6} }}} \right) \cdot V_{cc} \cdot A_{2} . $$
(2)

where \({A}_{2}=49.4\mathrm{k}\Omega /{R}_{7}+1\) is the amplification factor for stain gages, RSG11 and RSG denote the resistor at the two ends of SG1, and the other symbols can be found in Fig. 3.

3 Experimental Setups

To verify the performance of our developed tactile fingertip, three experiments are conducted as follows:

3.1 Measurement of Static Force

Sensitivity of a tactile sensor is critical to the perception performance. To test the responses of the tactile fingertip to applied forces, a force measurement setup is established in Fig. 4. The fingertip is fixed vertically on the setup with a forcemeter (precision: 0.1N) connected. Position of the fingertip can be adjusted manually via a rotation handle and the real-time height values can be read from a displacement meter (precision: 0.01 mm). After preprocessing, four-channel signals outputted from the tactile fingertip will be digitalized and acquired by a data acquisition card (USB6346, National Instrument). The acquired data are recorded in a laptop. The sampling rate is set as 10 kHz in all experiments in this work.

Fig. 4
figure 4

Force measurement setup

Full size image

The initial displacement is set as zero when the fingertip is nearly in touch with the measurement probe and the force meter reading is zero. Then the applied force, acquired voltages from stain gage 1 and 2 are recorded with an interval of 0.2 mm in the displacement. Totally, 21 pairs of measured data are recorded.

3.2 Contact Experiment

A robotic sliding setup is built as shown in Fig. 5, where the tactile fingertip is installed at the end of a robotic arm (UR5, Universal Robot), which could provide a stable movement with a repeatability accuracy of ± 0.03 mm.

Fig. 5
figure 5

Robotic sliding setup. a The whole experimental setup. b Amplified view of the sliding process

Full size image

The tactile fingertip will be controlled to slide on object surface undergoing a series of processes including untouched, starting to touch, touched and staying still, sliding on the surface, starting to lift, untouched (sliding direction is shown in Fig. 5b). To verify the robustness on different surfaces, four surfaces are used in the contact experiment, as given in Table 1, including carpet, mouse pad, jeans and paper box.

Table 1 Surface samples in contact experiment
Full size table

3.3 Surface Recognition Experiment

One of the main functionalities of PVDFs is surface recognition, e.g., texture or roughness. In this work, sensing elements for both static and dynamic forces are included, which could provide more perception information for surface property discrimination. The experimental setup is completely the same to that in Fig. 4. Based on the discrimination result of contact status, only the sliding period of signals is selected and used for recognition. Every sliding process is repeated for 10 times and there are 6 surfaces in total as shown in Table 2, where the surfaces are different in material, roughness, hardness, texture, and so on.

Table 2 Surface samples in surface recognition experiment
Full size table

4 Results and Discussion

4.1 Responses to Deformation

When displacement of the fingertip varies from 0 to 4 mm in vertical direction, the applied force is measured via a standard force meter. The relationship between the force and displacement is illustrated in Fig. 6a, which shows that the stress increases with respect to the displacement change nonlinearly. In Fig. 6b, a very similar variation trend of the voltage with respect to displacement is given for strain gage 1 and 2, which also behaves nonlinearly. The fingertip is found to be sensitive to static force as an interval of 0.2 mm in the displacement will induce a relatively large change of voltage.

Fig. 6
figure 6

The relationship between the force and displacement (a); voltage and displacement of strain gage 1, 2 (b); correspondence between voltage and force (c)

Full size image

Obvious systematic offset exists for the two stain gages, which is induced by the strain gage resistor, the fabrication process of the fingertip, the imbalance of the post-processing circuit and so on. When they are applied to force measurement, signals during non-contact period, i.e., no deformation is produced on the tactile fingertip, are collected in advance, whose average value are calculated and subtracted for offset compensation. To show the correspondence between the produced voltage and the force applied to the fingertip clearly, the offset is compensated and the relationship between the voltage and force is depicted in Fig. 6c. Although the variation of the two strain gages is not identical, they behave very similarly and nearly linearly with respect to the force. It indicates a closely related relationship between the voltages and forces, and we can easily infer the static force based on the measured voltages on strain gages.

4.2 Discrimination of Contact Status

There are 6 time instant \({t}_{1}\sim {t}_{6}\) and 5 duration processes when the tactile finger contact with a surface as depicted in Table 3. Accordingly, signals acquired from the four channels are shown in Fig. 7.

Table 3 Contact status between the fingertip and a surface
Full size table
Fig. 7
figure 7

Four-channel signals when the fingertip slides across surface 1

Full size image

As a high sampling rate is set (10 kHz), no obvious difference can be found for PVDF film 1 and 2 between different contact statuses. From signals of the two strain gages, distinct characteristics are observed and there exists a little difference between strain gage 1 and 2 in the variation trend. However, it is still difficult to distinguish a contact status from the others as significant noise is contained in both static force signals.

To show the variation trend more clearly, a Savitzky–Golay filter is designed to smooth the signals [34], in which the polynomial order is 3 and the frame length is 101. It can be found from Fig. 8 that the signals become much smoother and the variation trend is easier to judge. To automatically discriminate different contact status, the gradient is obtained according to \(dV/dt=[V\left({t}_{i}+\Delta t\right)-V\left({t}_{i}\right)]/\Delta t\). Here, \(\Delta t=5ms\). The gradient that has the maximal absolute value is found in the lifting period (t5 ~ t6) while a series of continuous positive gradients occur in the contact process (t1 ~ t2). During the sliding process (t3 ~ t4), the gradient signal fluctuates induced by the slid surface.

Fig. 8
figure 8

Filtered signals from SG1 and SG2 and their gradients when the fingertip slides across surface 1

Full size image

In addition, to reveal the influence of the two critical parameters of Savitzky–Golay filter on the smooth effect, four different parameter configurations are applied to SG1 and SG2 signals during 10 s ~ 11 s period and the filtered results are given in Fig. 9. As for different configurations, the filtered signals exhibit various smoothing effects. For instance, in the second row, there still exist plenty of thorns, whereas, many details are kept along with the signal. When a much larger value of frame length is adopted, the signal looks much smoother, but, at the same time, nearly all the local details are abandoned. Therefore, appropriate parameters should be chosen when Savitzky–Golay filter is applied, which is a trade-off between the overall variation trend and inclusion of local details.

Fig. 9
figure 9

Smoothing effect of SG1 and SG2 signals during 10 – 11 s using Savitzky–Golay filter with different parameter configurations. Six rows from top to bottom: row1: original signal; row2: order = 1, frame length = 3; row3: order = 2, frame length = 15; row4: order = 3, frame length = 101; row5: order = 4, frame length = 201; row6: order = 5, frame length = 401

Full size image

According to the gradient of strain gages, different contact status can be discriminated automatically. With the pause time before and after sliding process varying, four experimental configurations are explored: 1) Pause for 4 s, sliding for 8 s and pause for 4 s; 2) Pause for 5 s, sliding for 8 s and pause for 3 s; 3) Pause for 6 s, sliding for 8 s and pause for 2 s; 4) Pause for 7 s, sliding for 8 s and pause for 1 s. Figure 10 shows the strain gage signals as well as the automatic discrimination result of contact status when the tactile fingertip slides across surface 1. Different contact status can be recognized correctly.

Fig. 10
figure 10

Strain gage signals in four different experimental configurations. \({V}_{SG1}, {V}_{SG2}\): Voltage of SG 1 and SG2

Full size image

When the fingertip slides across the four surfaces in Table 1, it can be found from Fig. 11 that differences exist nearly during all contact statuses, which will be useful in recognizing different surfaces. Whereas, the contact status can still be recognized automatically as they share the same variation trend in each period.

Fig. 11
figure 11

Two strain gage signals when the fingertip contacts and slides across four different surfaces in Table 1 and in experimental configuration 1

Full size image

4.3 Recognition of Surfaces

According to the discrimination method of contact status proposed in 4.2, signals in the four channels can be extracted conveniently. Robotic exploration on six object surfaces in Table 2 is implemented with the tactile fingertip.

Considering the zero-point shift that is induced by electronic noise, temperature variance and other uncertainty factors, both PVDF and strain gage signals will be subtracted by the initial values at the beginning, which are obtained during the untouched period. Similar to the enhanced discrimination method applied in our previous work [31], Wavelet Transformation (WT) is first utilized to decompose the two PVDF signals into four-level approximation and detail components respectively in time and frequency domain. Then the Standard Deviation (SD) and energy are extracted from the time domain WT components of PVDF signals while in frequency domain, the mean value is added additively. As signals from the strain gages belong to a measure of the static forces, neither WT nor Fast Fourier Transform (FFT) is applied. Thus, there are three features extracted from each strain gage: mean, SD and energy. In total, there are 86 features for the four-channel sensing elements.

When the Sequential Feature Selection (SFS) algorithm is applied, the most discriminant features are selected one by one. The first five features are adopted to input into the surface recognition model as listed in Table 4. It includes: (1) SD of strain gage 2; (2) SD of approximation component a1 from PVDF 2 in time domain; (3) ~ 5) SD of approximation components a1 ~ a3 from PVDF1 in time domain. SD is found a more discriminant feature for the recognition. To visualize the feature space constituted by the selected features, the first three features are shown in Fig. 12. It can be found that different surface samples occupy obviously different value ranges characterized by Feature 1, whereas overlaps exist between many samples, e.g., surface 1, 2 and 4. The difference between different surfaces becomes more obvious when feature 2 is added as shown in Fig. 12b. In a 3D feature space, this difference is further enhanced.

Table 4 The five selected features for surface recognition
Full size table
Fig. 12
figure 12

Sample distribution based on the first three selected features. a Feature 1. b Feature 1 & 2. c Feature 1, 2 & 3

Full size image

The five selected features are inputted into a recognition model established with Extreme Learning Machine (ELM), as for which the wavelet kernel is applied. 80% of the dataset is used for training while the left 20% for test. To avoid the possible errors caused by the small size of dataset, each training and test process is repeated for 500 times and the mean prediction accuracy is assessed. Finally, 100% training accuracy and 99.72% test accuracy are obtained. Standard deviation of the test result is 1.6 for the 500 times repeated predictions. Improvement is achieved in both the increase of prediction accuracy and the decrease of standard deviation compared with our previous work. Detailed recognition result is given in Fig. 13. The surfaces of Book cover, Paper box, Tower 1 and Tower 2 are recognized correctly and completely. Four samples of Carpet surface are recognized as Mouse pad while 13 out of 1000 surface 2 Mouse pad are misclassified as Carpet. It reveals that more similarity exists for the two surfaces, which can also be found from Fig. 12 as their sample distributions in feature space are too close to discriminate.

Fig. 13
figure 13

Confusion matrix of the test dataset for robotic surface recognition

Full size image

4.4 Discussion

As a typical application of tactile sensors, recognition of different surfaces is a synthetic result since their difference comes from lots of factors including roughness, hardness, texture, and patterns etc. By contrast, the discrimination of single property is more challenging. For example, in our previous work, surface roughness was discriminated with the roughness parameter Ra ranging from 0.4 µm to 50 µm [31]. Although the final recognition accuracy reached 97.88% with our proposed enhanced discrimination algorithm, the accuracy for the particular surface of Ra = 0.8 µm was only 83%. When it was applied to pattern discrimination of 15 aluminum bronze plates, the overall accuracy decreased to as low as 72.93% [32]. While the focus of this paper is the perception capability of daily objects with our improved tactile fingertip, the discrimination of surface pattern or roughness is worthy of further investigation, a promising application of which is machining quality assessment and surface defect inspection. A combination of machine vision and the tactile perception is expected to be an implementable, intelligent and high-accuracy approach.

5 Conclusion

In this paper, a bio-inspired tactile fingertip is designed, which is able to measure both static and dynamic forces simultaneously. The mechanoreceptors are mimicked with four sensing elements while the finger bone, soft skin, and nerve fiber within human hand are substituted with different materials and structures. After signal preprocessing with a custom design of electric circuit, the tactile signals are converted, filtered and amplified. The fingertip is proved to be sensitive to static deformations via the exploration of correspondences between the deformation of fingertip, signal voltage and applied forces in a force test experiment. Based on a robotic experimental setup, it is found that the sensing elements for static forces are critical in discriminating different contact status between the tactile fingertip and other objects, which is very helpful in extracting signals of different contact periods. Influence of different experimental configurations and surfaces is explored further. Finally, as high as 99.72% of accuracy is achieved in recognizing six object surfaces in daily life, which demonstrates the tactile sensing ability of our designed sensors.