Keywords

1 Introduction

Soft sensors have many potential benefits including biocompatibility, adaptability, and effective and safe interactions with humans and objects as compared with conventional sensors crafted from rigid materials which may not be adequate to measure significant deformations because of their inability to adapt to different surface textures [1,2,3]. In addition, rigid sensors are often bulky and may create user discomfort when in contact with the skin. Instead, soft sensors with stretchable capabilities along with flexibility can resolve these issues using simpler structures. Such sensors can be fabricated from relatively inexpensive materials and offer high reliability and accuracy. Generally, soft sensors can be made from various materials, including but not limited to elastomers, polymers, and hydrogels. Elastomers like silicone rubber are commonly used due to their elasticity and flexibility. Polymers, such as polyurethane and polydimethylsiloxane (PDMS), are also popular due to their ability to be easily molded into different shapes and biocompatibility. Hydrogels, such as polyacrylamide and alginate, are also frequently used due to their high-water content and biocompatibility [4].

Stretchable soft sensors can be employed to capture movements of various parts of the human body, such as the wrists, elbows, and knees, due to their ability to undergo stretchable deformations. Several types of soft wearable sensors have been reported in the literature, depending on the materials used in their construction [5, 6]. Theoretically, using soft materials offers an intriguing chance to develop new wearable systems that adapt and adjust to the user’s morphology and motions without impeding them, resulting in comfortable wearable devices. Nevertheless, soft sensors are still developing, as evidenced by the scarcity of widely used commercial products used for their fabrication. It is still unclear which soft sensing technique is the most suitable for strain or curvature measurement. So far, the proposed soft sensing technologies faced subpar intrinsic performance or scaling capabilities [2, 7].

Different types of soft sensors with varying levels of performance have been proposed. Soft piezoresistive strain sensors are made of soft polymers loaded with conductive particles but suffer from creep and hysteresis [8]. Although soft capacitive sensors typically have excellent linearity and minimal hysteresis, they generally have low sensitivity [2]. Soft, optical waveguides with high stretchability and low optical loss have also been developed [9, 10]. However, although highly promising, they can have poor linearity [11] and are also relatively heavy. A filmy stretchable strain sensor with three layers was developed using carbon conductive grease as the resistance electrode layer in the middle and soft elastomer as the two sealing layers [12]. Even though these sensors demonstrate high functionality, there is variability in the properties with repeated use. Approaches relying on graphene [13] and carbon nanotubes [14] have shown excellent sensitivity but require advanced fabrication methods. Recent advancements have demonstrated the capacity of these soft ultrasonic waveguides to perform decoupled measurement of strain and contact location by guiding ultrasonic acoustic waves within the soft polymer waveguide [7]. Due to their fabrication using an unmodified elastomer, these waveguides possess a softer nature and exhibit reduced hysteresis than comparable piezoresistive and capacitive particle-loaded composites. Moreover, these sensors offer the advantage of simplified manufacturing processes and eliminate the possibility of leakage.

Given that hands play a crucial role in human interaction with the surrounding environment by recognizing, grasping, and manipulating objects, it is of significant practical importance to develop soft sensors capable of detecting hand movements [15]. In this regard, there have been several noteworthy research publications related to sensing skins with soft sensors for hand motion detection and estimation [16,17,18,19,20]. A sensing skin that measures finger abduction/adduction was developed by Kyongkwan et al. [16]; however, it was relatively bulky. Commercial elastomer VHB and conductive bond were used to create a transparent, very thin artificial skin to make the sensing skin lighter and thinner [20]. However, since VHB elastomer exhibits excessive viscoelasticity, it could lead to user discomfort or self-adhesion. Other significant studies have utilized the elastomer Ecoflex as a substrate material owing to its exceptional stretchability [17,18,19]. However, the Ecoflex substrate is milky white in color after curing, which makes it challenging to attain the objective of being transparent. In [18], a wearable soft artificial skin was developed using the elastomer Ecoflex and liquid metal by incorporating microchannels into the elastomer base to inject conductive materials. However, a common drawback of these production methods is that they typically require a certain thickness of elastomer base to create channels, thereby restricting the reduction of overall sensor thickness. Therefore, reducing the overall weight of soft sensing gloves is challenging, and the manufacturing procedures are relatively intricate. Although previous studies have shown great potential for applying soft sensors for hand motion detection, developing ideal soft sensors with features such as being transparent, stretchable, lightweight, and easily worn on human hands remains challenging due to the limited availability of fabrication materials and complex manufacturing processes. An ideal artificial skin should be transparent, highly stretchable, lightweight, and thin.

This study presents a solution to the aforementioned challenges by introducing the design and fabrication of a soft, transparent, and stretchable artificial skin integrated with soft ultrasonic waveguides for finger joint movement detection. The artificial skin and the ultrasonic waveguides are designed and fabricated. For the system integration, electronic configuration and system assembly are performed. Finally, experimental validation is performed, and results are discussed. This work is the first to integrate soft ultrasonic waveguides with artificial skin to be used as stretchable sensing skins for hand motion detection. The primary contribution of this study involves the utilization of soft ultrasonic waveguides to create a sensing skin that is transparent, soft, lightweight, and possesses remarkable stretchability. These combined factors contribute significantly to enhancing the wearability of the developed system. Moreover, the artificial skin covers the dorsal part of the hand, while leaving the fingertips uncovered, thereby minimizing any impediment to the user’s dexterity. This novel sensing skin has the potential for effective hand motion detection.

2 Design and Fabrication

The system design and fabrication phase includes artificial skin manufacturing, hand frame manufacturing, and soft ultrasonic waveguide manufacturing.

2.1 Artificial Skin and Hand Frame

The preparation procedure for the artificial skin is as follows. The plate for the spin coater (KW-4C, Beijing Saidecase Electronics Co., Ltd, China) is designed on the user interface of a laser cutter and cut using the laser cutting machine (GD Han’s Yueming Laser Group Co., Ltd, China) (Fig. 1). The plate used is acrylic material with a thickness of 0.1 mm. The dimension of the plate design is chosen based on the average human hand size, which is 16 cm by 9 cm in length and breadth, respectively [21]. The length is shorter than the average hand length because the artificial skin does not cover the whole finger since only the metacarpophalangeal(MCP) joints of the fingers are the targets for this study. Next, the chosen elastomer is mixed, degassed then poured onto the plate, and the spin coater is used to spread the elastomer throughout the plate with specified parameters to get the desired skin thickness. The parameters used for the elastomer Ecoflex\(^\textrm{TM}\) 00–31 Near Clear\(^\textrm{TM}\) are Speed: 200 round per minute (rpm), Acceleration: 500, and Time: 30 s (s). Then it is degassed again and cured on a hot plate at 60 \(^{\circ }\)C for approximately 25 min. After curing, it is unmolded and removed from the plate.

Fig. 1.
figure 1

Design and fabrication process of the artificial skin manufacturing plate.

Full size image

The material for the artificial skin should be chosen carefully to make it soft, lightweight, transparent, and compatible with the human hand. Four elastomers from the Ecoflex series were tested for the artificial skin, including Ecoflex\(^\textrm{TM}\) 00–10, Dragon Skin\(^\textrm{TM}\) FX- Pro\(^\textrm{TM}\), Ecoflex\(^\textrm{TM}\) 00–31 Near Clear\(^\textrm{TM}\), and Ecoflex\(^\textrm{TM}\) 00–45 Near Clear\(^\textrm{TM}\). Ecoflex\(^\textrm{TM}\) 00–10 and Dragon Skin\(^\textrm{TM}\) FX Pro\(^\textrm{TM}\) have good softness and stretchability properties but are translucent and not as transparent as the Ecoflex\(^\textrm{TM}\) Near Clear\(^\textrm{TM}\) series. The Ecoflex\(^\textrm{TM}\) 00–45 Near Clear\(^\textrm{TM}\) (Table 1) exhibits transparency; however, upon curing, it tends to possess a slightly increased hardness and thickness compared to Ecoflex\(^\textrm{TM}\) 00–31 Near Clear\(^\textrm{TM}\). At last, the Ecoflex\(^\textrm{TM}\) 00–31 Near Clear\(^\textrm{TM}\) (Table 1) was chosen for the artificial skin manufacturing because it is transparent, soft, thin, and stretchy enough to make it comfortable to be worn directly on the hand and suitable for any hand size. The thickness of the fabricated artificial skin was measured to be around 200 \(\upmu \)m.

Fig. 2.
figure 2

Hand frame design.

Full size image

Since the artificial skin covers only the dorsal part of the hand, a hand frame is designed to fix the artificial skin with the hand (Fig. 2). The hand frame consists of finger rings and a wristband. The hand frame is 3D printed using a soft and transparent Thermoplastic Polyurethane (TPU) material (30\(^\circ \) soft glue) by Wenext company. The finger rings are highly stretchable, being easily adapted for different hand sizes. A Velcro material is attached to the opposite ends of the wristband to fit the artificial skin for different hand sizes.

2.2 Soft Ultrasonic Waveguide

The soft ultrasonic waveguides were designed to meet the following design requirements for this project. 1) The piezoelectric transducer (AM1.2 \(\times \) 1.2 \(\times \) 1.7D-1F, Tokin, Japan) should be fully embedded within the waveguide. 2) The soft ultrasonic waveguides should be long enough to cover the finger joint and, at the same time, short enough to guarantee tolerably low acoustic losses. 3) The soft ultrasonic waveguides should be soft, lightweight, and transparent.

The soft ultrasonic waveguide manufacturing process follows a previously proposed procedure [22]. Molds were fabricated using a stereolithographic 3D printer (Objet 30, Stratasys, USA). The waveguide manufacturing steps are as follows: 1) Spray a release agent (Mann, Ease Release 200, USA) onto the mold; 2) Mix and degas the chosen elastomer; 3) Manually place and align the transducer on the designated platform within the waveguide; 4) Pour the degassed elastomer into the mold and leave it to cure in the oven at 50\(^{\circ }\)C for approximately 40 min; 5) Demold and clean the manufactured ultrasonic waveguide.

Table 1. Properties description of the materials in Ecoflex\(^\textrm{TM}\) Near Clear\(^\textrm{TM}\) series.
Full size table

Choosing the appropriate material for the soft ultrasonic waveguide is crucial to ensure the waveguide’s high accuracy and sensitivity to minor joint movements. Since one of our design requirements is for the soft ultrasonic waveguides to be transparent, we only tested the two materials in Ecoflex\(^\textrm{TM}\) Near Clear\(^\textrm{TM}\) series (Table 1). The Ecoflex\(^\textrm{TM}\) Near Clear\(^\textrm{TM}\) series has a water-clear translucent color. When cured, it is transparent, which is in harmony with our design requirements. Furthermore, the Ecoflex\(^\textrm{TM}\) 00–45 Near Clear\(^\textrm{TM}\) has an excellent signal property with a stronger echo signal and less noise when compared to the Ecoflex\(^\textrm{TM}\) 00–31 Near Clear\(^\textrm{TM}\). Hence, the Ecoflex\(^\textrm{TM}\) 00–45 Near Clear\(^\textrm{TM}\) silicone elastomer is chosen for manufacturing the waveguide.

3 Electronic Configuration and System Assembly

The overall system is composed of five soft ultrasonic waveguides, one multiplexer (CD74HC4067, Texas Instruments, USA), one MAX14808 acoustic pulser evaluation board (Maxim Integrated Products, USA), an Analog Digilent 2 digital oscilloscope (AD2, Digilent, USA), power supplies, and a PC. Each ultrasonic waveguide is sequentially excited by the MAX14808 acoustic pulser evaluation board, with the excitation pulse timing controlled via the AD2 digital output channels. Once the transmitted acoustic wave is reflected back, the transducers act as receivers, receiving the echo waves, and the signal is captured using one of AD2’s oscilloscope channels. To enable the simultaneous collection of data from all five sensors using a single oscilloscope channel in AD2, we utilize a multiplexer to expand the number of available input channels. This is necessary because there are only two oscilloscope channels available in AD2. The multiplexer is also controlled by digital signals through the AD2 digital output channels.

The system assembly process comprises four main stages: artificial skin manufacturing, wire embedding, hand frame attachment, and waveguide integration (Fig. 3). The ultrasonic waveguides are positioned on the MCP joint, on top of the artificial skin covering the dorsal part of the hand. The free end of the waveguides is attached to each finger ring of the hand frame using a soft, thin, and translucent two-sided tape (YZ202 0316, TianTian Factory, China).

Different wiring methods were tested to connect the transducer wires on the other end of the waveguides to micro coaxial cables (IPEX 1st generation, Xinlisheng Hardware Electronics Co., China), which transmit electrical signals to and from the MAX14808 acoustic pulser evaluation board. Two ink materials were tested to print the wire on the artificial skin using the DB100 multifunctional electronics printer (Shanghai MiFang Electronic Technology Co., Ltd., China). First is BASE-CD01, a silver paste prepared by mixing silver powder and resin filler. This material is suitable for dispensing and screen-printing processes and is widely used in conductive lines, flexible switches, shielding circuits, and other scenarios. However, when printed on the soft artificial skin, the adhesion was not very good, and it fell off after curing, so this material could not be used for our application. The second one is BASE-CGI1which is a Ga/In Alloy liquid metal material. Compared to other conductive materials, it has high density and conductivity and is suitable for preparing flexible and stretchable wires on elastic substrates. This material also has good tensile properties than BASE-CD01; however, the manufacturing process is relatively complex.

In contrast to silver paste, liquid metal doesn’t cure and remains fluid after printing, so it requires to be encapsulated by another layer of polymer. Encapsulating the printed liquid metal wire with another layer of Ecoflex elastomer using an adjustable film applicator (KTQ-II1, Guangzhou Xinyi Laboratory Equipment Co., Ltd, China) can be challenging due to its fluidity. It may result in a disconnected line, causing an interruption in the electrical flow. Another challenge of working with liquid metals is achieving a specific wire thickness, as the material’s fluidity and peculiar properties can make it difficult to control the parameters accurately. As reported by other researchers [18, 23, 24], the best way to use liquid metals for wiring is to embed microchannels within the elastomer-based artificial skin and fill the microchannels with liquid metal. Nevertheless, this manufacturing process is complex relative to the purpose of serving simple wiring in our case, as it requires dexterity and extra care. Additionally, the manual manufacturing process may not be precise enough to prevent liquid metal from leaking. Therefore, this material was also not used for our application.

Fig. 3.
figure 3

The system assembly process.

Full size image

Finally, thin silver-plated copper wires (Kunshan Lvchuang Electronic Technology Co., Ltd., China) with a diameter of 0.1 mm were chosen for connecting the wires of the transducers on the waveguides to the micro coaxial cables. Once the artificial skin is fabricated, the thin silver-plated copper wires are placed on top of the artificial skin with the same double-sided tape mentioned above. Next, the Ecoflex\(^\textrm{TM}\) 00–31 Near Clear\(^\textrm{TM}\) elastomer is poured on top of the wired artificial skin and coated evenly using the adjustable film applicator to embed the wires, leaving a small portion out on both ends for electrical connection with the rest of the system. Finally, it is cured on a hot plate at 60 \(^{\circ }\)C for about 25 min. The overall thickness of the wire-embedded artificial skin is around 500 \(\upmu \)m. The ends of the embedded wires closer to the fingers are soldered to the transducers on the waveguides, and the other ends closer to the wrist are soldered to the micro coaxial cables. This choice of wiring reduced the manufacturing complexity while also keeping the thickness of the wire-embedded artificial skin relatively thin. Finally, the artificial skin is attached to the finger rings and wristband of the hand frame using super glue (deli, SUPER GLUE No.7146, China). The finger rings fix one side of the artificial skin to the user’s fingers, and the wristband with the Velcro attached is used to fix the other side of the artificial skin to the user’s wrist.

4 System Performance

4.1 Operating Principle of Soft Ultrasonic Waveguides

The transducer within the ultrasonic waveguide is excited using MAX14808 acoustic pulser evaluation board by an acoustic pulse train generated by a series of 4 consecutive square pulse waves with a frequency of 980 kHz and a voltage of 15 V [22]. This generates acoustic waves within the waveguide. Once the waves reach the other end, they get reflected back, and the transducer, which now acts as a receiver, receives the reflected echo signals and finally outputs a voltage. The imaging method used in this case is called “pulse-echo” because the same transducer is used to generate the acoustic pulse train and to measure the reflected acoustic wave [25]. The distance between the transducer and the waveguide’s end increases upon stretching the waveguide, changing the Time of Flight (TOF) of the acoustic waves. The TOF is the time interval between the generation of the acoustic pulse train and the echo signal’s main peak. The waveguide’s length can be estimated by measuring this time interval.

4.2 Signal Processing

The AD2 oscilloscope channel captures analog voltage signals data during a 1040 \(\upmu \)s long window, divided into five 208 \(\upmu \)s long windows for each waveguide on the MCP joints. The sampling frequency used is 1/15 * 10\(^8\) Hz. The data is collected on a PC by a Python program which filters the signal in real-time using the Chebyshev bandpass filter, 8th order, 500 KHz–3 MHz. The data is appended row-wise to a CSV file. The echo signal’s peak can be obtained through signal enveloping by performing Hilbert transform. Then the signal’s TOF can be obtained by subtracting the time of generation of the acoustic pulse train from the time of the echo signal’s main peak. Each of the five different colors on the sample filtered signal (Fig. 4) represents the signal from each waveguide on the five fingers’ MCP joints. The first wave packet in each window is noise from reflected signals at the transducer-elastomer interface, and the second wave packet is the first received echo which is our target.

Fig. 4.
figure 4

Filtered analog signal collected by the oscilloscope channel of the AD2.

Full size image

4.3 Application for Hand Motion Detection

The integrated sensing system has a total weight of about 12 g and an area of about 140 cm\(^2\), resulting in a pressure of approximately 8 Pa towards the dorsal part of the hand. Owing to the excellent stretchability of the soft artificial skin and ultrasonic waveguides, the developed system can be used to detect large deformations such as finger joint movements. A static hand gesture experiment was conducted to demonstrate the system’s capability of detecting static finger joint positions. The experiment involved a subject wearing the ultrasonic waveguide-integrated artificial skin and performing static hand gestures (Fig. 5).

Fig. 5.
figure 5

The ultrasonic waveguide-integrated artificial skin worn on a human hand.

Full size image

A series of gestures were designed and performed, first extending all five fingers, then flexing each finger from the thumb to the pinky finger one by one, and finally flexing all fingers to make a fist (Fig. 6a). The subject was requested to perform each gesture for 4 s, and we measured the MCP joints of the five fingers. The changes in the received echo signals and the corresponding TOF when subjected to different hand gestures can be real-time observed through a Python program (Fig. 6b and 6c). The experiment was repeated ten times for each gesture, and the performance of the integrated sensor system did not change significantly, implying excellent repeatability. All the signals were stable, and their distinct differences under different finger positions highlighted the system’s reliability and suitability for hand motion detection.

Fig. 6.
figure 6

Hand motion detection. (a): The hand gesture set designed for this study. (b): The corresponding output signals of the hand gestures in (a). (c): The TOF plot for each gesture while performing the gesture within 4 s with a total period of 24 s for performing the whole gesture set.

Full size image

When performing the first gesture, all the MCP joints on the five fingers are extended, keeping all the ultrasonic waveguides unstretched. Therefore, the echo signals received from all five ultrasonic waveguides are stronger and have higher amplitude. However, when performing the second gesture, the thumb MCP joint is flexed, stretching the ultrasonic waveguide attached to that joint. Hence, the signal from the ultrasonic waveguide attached to the thumb finger has increased TOF and reduced amplitude, while the signals received from the other waveguides attached to the rest of the fingers remained similar to the first gesture. The output signal for the remaining gestures can be explained in a similar manner (Fig. 6a and 6b). The outcomes observed during the execution of the last fist gesture reveal the system’s capability to detect motion across multiple finger joints, distinguishing it from the other gestures which solely involve single finger joint motion.

The increase in the TOF may not be easily seen from Fig. 6b but is clearly displayed in Fig. 6c. When there is no flexion of the MCP joint on any of the fingers, the TOF values of the ultrasonic waveguide remain at a low level, nearly 135–145 \(\upmu \)s. When the MCP joint of any finger flexes, the length of the ultrasonic waveguide extends, resulting in an increase in the TOF values up to nearly 155–160 \(\upmu \)s. If the MCP joint of any finger returns to the extension state, the TOF values drop back to the low level again (Fig. 6c). Throughout the experiment, each gesture’s highest and lowest TOF levels remain in a certain narrow range, indicating that the developed system has excellent repeatability, which is an essential characteristic for practical applications.

From the TOF plot (Fig. 6c), we saw a noticeable increase in TOF whenever joint flexion occurs in any of the fingers’ MCP joints. This implies that the TOF is a fundamental feature of the ultrasonic waveguides that can be used to detect different static finger joint positions. In future work, more advanced gestures can be recognized using this sensor system by applying machine learning algorithms.

5 Conclusion

This paper presents the design and fabrication of an artificial skin integrated with soft ultrasonic waveguides. The work involved designing and fabricating the artificial skin, hand frame, and soft ultrasonic waveguides and integrating them into a functional system. Hand gesture experiments were conducted to validate the system’s performance and suitability for hand motion detection. The experiment results indicate the excellent reliability and repeatability of the proposed system. The features of the soft ultrasonic waveguides are more to be explored, and it provides a promising prospect for hand gesture recognition applications.