1 Introduction

Autonomous robots have replaced many human tasks in factories, homes, automobiles, etc. The robots are currently attracting the most attention in the medical field. In particular, various robots for precise surgical operations have been developed such as AESOP™, ZEUS™, and the most famous Da Vinci™. Da Vinci, in particular, has advantages such as 3D visualization, better hand skill, revision of tremors, teleoperations, and precise anastomose [1, 2]. These surgical robots are used for robotization of laparoscopies. Laparoscopic surgeries are considerably suited for the applications of surgery robots because laparoscopy requires a smaller work area and precise control [3]. The robotization of surgical operations has several advantages, including less bleeding; smaller invasion, thus decreasing pain and shortening convalescence; and minimal organ exposure [4].

Recently, surgical robots are being developed for conventional surgeries such as vitrectomy, coronary artery bypass graft (CABG) surgery, neurosurgery, and angiography. The surgery is characterized to be operated in a very narrow space, and many researchers have been studying handheld surgical tools to help the surgeon. Uneri et al. developed a “Steady-Hand” system to help the surgeon perform vitrectomy. The system consists of a manipulator to correct the tip position of the surgical needle from the tremor. The manipulator holds the surgical tool to prevent vibration caused by tremor without obstruction of the surgery [5]. MacLachlan et al. [6] also reported a handheld robotic microsurgery system named “Micron.” It has attitude and position sensors to prevent the vibration of the tip of the needle with submillimeter displacements of piezoactuators. This equipment measures the vibration of the tip of the needle and cancels the vibration for ensuring stable surgery. Chang et al. [7] presented a handheld surgical tool that cancels tremor by using a combination of a linear delta manipulator and a voice coil motor. The tool employed a 3-DOF linear delta manipulator for ensuring maximum possible travel space in a direction, a large workspace, maximum stiffness of the z-axial motion, and good precision in the required axis. Berkelman et al. developed a retinal surgical tool that has a double-cross flexure beam sensor. The sensor was designed to measure force as small as applying retina operation [8]. Iordachita et al. presented a fiber-optic force-sensing tool for retinal microsurgery. They developed 1-cm-long, 160-\(\upmu \)m-diameter fiber Bragg grating (FBG) strain sensors [9]. Different from previous studies, Saxena et al. [10] developed a surgical tool that provides haptic stimuli measured by a force sensor located at the tip of the tool. The force sensor and a tactile display are actuated with solenoids. It transmits one-dimensional measured force to user’s fingertip with the tactile display.

In this paper, a smart handheld surgical tool with tactile feedback is presented to prevent tragic accidents, such as penetrating and scraping of retina (retinal tear). The tool consists of a sensor and a tactile display based on an electroactive polymer. The capacitive force-sensing scheme is applied, and the tactile display using a dielectric elastomer actuator (DEA) is used to deliver the stimulation to the user’s fingertip [11,12,13,14,15,16]. The strength of the stimuli is determined by the driving frequency of the dielectric actuator, and the frequency is interlocked by the magnitude of force as shown in Fig. 1. A prototype of the tool is fabricated, and the proposed design is validated experimentally.

Fig. 1
figure 1

Concept of the smart handheld surgical tool

Full size image

This paper is organized as follows. In Sect. 2, the principles of the DEA and the capacitive sensor are explained. In Sect. 3, the design of the force sensor and the tactile display as well as their integration is presented. In Sect. 4, we explain the fabrication process of the sensor and the tactile display. In Sect. 5, the results of the experiments are presented to validate the handheld surgical tool, and conclusions are drawn in Sect. 6.

2 Principle

2.1 Principle of sensor

The capacitive sensor consists of a dielectric layer sandwiched with conductive electrodes as shown in Fig. 2. In this sensor, the capacitance can be expressed as

$$\begin{aligned} {C_0} = {\varepsilon _0} {\varepsilon _r} {\frac{{{A_0}}}{{{t_0}}}}, \end{aligned}$$
(1)

where \({C_0}\) is the capacitance. \({\varepsilon _0}\) is the free-space dielectric permittivity (\({8.85\times 10^{-12}\,\hbox {F}/\hbox {m}}\)), and \({\varepsilon _r}\) is the relative permittivity of the dielectric material. \({A_0}\) represents the area under the electrodes, and \({t_0 }\) is the thickness between the electrodes. When the relative position between the electrodes is changed by an external force, the capacitance is increased or decreased because of the change in area and thickness. Until now, the most popular force sensor is the resistive-type sensor, typically strain gauges. However, the multi-axial resistive-type sensor is not suitable for surgical applications because of its weight and relatively high cost. On the other hand, the benefits of a capacitive force sensor include cost effectiveness, lightweight, and ease of fabrication [17,18,19].

Fig. 2
figure 2

Principle of the capacitive sensor

Full size image

2.2 Principle of the tactile display

In this study, the DEA, a type of electroactive polymer, is adopted for the tactile display. The basic principle of the dielectric actuator is as shown in Fig.  3 [20].

Fig. 3
figure 3

Principle of the dielectric elastomer actuator

Full size image

When a voltage is applied across the dielectric elastomer film coated with compliant electrodes on both sides, the elastomer is compressed in the thickness direction, while it is expanded in the lateral direction. Due to the contraction of electrical charges stored in the surfaces, its actuation is generated. The actuation principle of the DEA couples the domains of mechanical and electrical energy to generate energy transduction. The effective mechanical pressure along the thickness direction by a voltage input is given as follows:

$$\begin{aligned} p = \varepsilon \varepsilon _\circ \Big (\frac{V}{t}\Big )^2 = \varepsilon \varepsilon _\circ E^2, \end{aligned}$$
(2)

where p, called Maxwell stress, is the compressive pressure from charges stored on the surface. \(\varepsilon _\circ \) and \(\varepsilon \) are free-space dielectric permittivity (\(\varepsilon _\circ = 8.85 \times 10^{-12}\,\hbox {F}/\hbox {m}\)) and relative permittivity of the dielectric material, respectively. V and t represent the supplied voltage and the thickness between the electrodes, respectively. E denotes the applied electric field. When a voltage is applied to the electrode sandwiching the dielectric elastomer layer, Maxwell stress is generated along the perpendicular direction to the electrode due to electrostatic force, and stress \(\sigma \) is calculated as

$$\begin{aligned} {\sigma }= {\varepsilon _0} {\varepsilon _r} {\left( {\frac{V}{{t_0}}} \right) ^2,} \end{aligned}$$
(3)

where \({\sigma }\) is the Maxwell stress and V is the applied voltage. The DEA has many advantages such as flexibility, softness, small size, relatively high performance, and cost effectiveness [21,22,23,24,25].

Fig. 4
figure 4

Capacitive force sensor with air gap. a Initial state. b Under applied force

Full size image
Fig. 5
figure 5

Design of a deformable structure

Full size image
Fig. 6
figure 6

Exploded view of the sensor

Full size image

3 Design

3.1 Design of the sensor

The conceptual design of the proposed sensor is shown in Fig.  4a. The sensor consists of a printed circuit board (PCB) and a ground electrode attached to the structure located close to the PCB. When an external force is applied to the tip of the tool, the space between the ground electrode and the PCB electrodes is changed. Because of the change in distance, the capacitance between the ground electrode and PCB electrodes also changes, as shown in Fig. 4b. Consequently, the sensor can detect the magnitude of force with the capacitance change and the direction of force.

The design of the deformable sensor structure is shown in Fig.  5. The tip is supported by three bars shaped like a crown. Finally, each part designed is shown in Fig. 6.

Fig. 7
figure 7

Mathematical modeling of the sensor structure. a Initial mathematical modeling of the sensor structure. b Mathematical modeling of the sensor structure with torsion spring

Full size image

3.1.1 Mathematical modeling of the capacitive sensor

As shown in Fig. 7a, the capacitive sensor can be modeled. When a force is applied on the tip of the needle, the sensor structure rotates around the center of gravity of the T-shaped part. If the T-shaped part is stopped by force \(F_{2}\) caused by the stiffness of springs with spring constant k, the moment is explained as shown in Eq. 4.

$$\begin{aligned} {F_1} \times {l_1} = {F_2} \times {l_2} \end{aligned}$$
(4)

Point a is the connection point of the T-shaped part and the spring. Therefore, the vertical displacement between point a and its initial position b is explained as shown in Eq. 5.

$$\begin{aligned} b = \frac{{{F_2}}}{{2k}} \end{aligned}$$
(5)

Next, rotation angle \(\theta \) is determined as shown in Eq. 6.

$$\begin{aligned} \theta = \arcsin \left( {\frac{b}{{{l_2}}}} \right) \end{aligned}$$
(6)

Now this model can be modified with the addition of a torsion spring that has spring constant \(k_{\tau }\) as shown in Fig. 7, and its spring constant \(k_{\tau }\) is determined as shown in Eq. 7b. The torsion stiffness can be also determined by using the distance between point a and the center of gravity.

$$\begin{aligned} {k_\tau } = \frac{M}{\theta } = \frac{{{F_1} \times {l_1}}}{{\arcsin \left( {\frac{b}{{{l_2}}}} \right) }} \end{aligned}$$
(7)

The cross sections of copper ground electrode and the PCB electrode, \(c_{1}\) and \(c_{2}\), are calculated as shown in Eqs. 8 and 9, when constant.

$$\begin{aligned} {c_1} = \frac{{{d_1} - {d_2}}}{{\tan \theta }} \end{aligned}$$
(8)
$$\begin{aligned} {c_2} = \frac{{{d_3} - {d_4}}}{{\tan \theta }} \end{aligned}$$
(9)

As a result, the capacitances of cell 1 and cell 2, \(C_{1}\) and \(C_{2}\), are calculated as shown in Eqs. 10 and 11, when the depth of the cells is t [17].

(10)
(11)

3.2 Design of the tactile display

Although many tactile displays have been reported, they have too many parts with complicated structures and they are not suitable for cost-effective mass production. For this reason, an improved tactile display is employed in this study.

As shown in Fig.  8, the device consists of a frame with a rigid coupling and a film-type DEA. After assembling the frame and the actuator by sandwiching the rigid coupling, the device is built.

The movement of the DEA is transferred to the touch spot, and then, the touch spot produces displacement as shown in Fig.  9. Usually, the DEA contracts when a voltage is applied. However, the pre-strain or the initial force can determine the direction of force and deformation of the DEA. Thus, in the “voltage off” state, the actuator is under pretension because of the elasticity of the membrane around the touch spot and rigid coupling as shown in Fig.  9a. The DEA is deformed by the elasticity of the membrane around the touch spot and rigid coupling. In addition, the membrane around the touch spot is stretched and the deformation of the DEA and the membrane around the touch spot maintain their balance for force. When the input voltage is supplied to the actuator, that is in the “voltage on” state, the actuator contracts toward the thickness direction and expands as shown in Fig.  9b. Because of the expansion of the actuator, the pretension is reduced and the coupling moves down. And then, the membrane around the touch spot comes back to the initial state. It is an indirect actuation by using the rigid coupling. Hence, the possibility of electrical damage to the user due to the high-voltage input is avoided. In addition, the touch spot provides soft feels of touch from the intrinsic compliance of the actuator and the frame.

Fig. 8
figure 8

Design of enhanced rigid coupling tactile display

Full size image
Fig. 9
figure 9

Concept of enhanced rigid coupling tactile display. a Voltage off. b Voltage on

Full size image

Three tactile displays form part of the tactile display module as shown in Fig. 10. The displays are located \(120^\circ \) apart circumferentially around the cylindrical frame of the grip. Thus, assuming that the surgeon grasps the tool with fingers, each tactile display stimulates the point of the thumb, the index finger, and the medial side of the first joint of the middle finger. Each tactile display is rounded on the tool holder.

Fig. 10
figure 10

Design of the tactile display module

Full size image

4 Fabrication process

4.1 Fabrication of the sensor

Every part of the sensor was manufactured by a 3D printer (Objet24, Stratasys Co.). In the case of the capacitive sensor, considerable noise is generated by external objects such as hands and metals. The deformable structure was made by using the 3D printer. The 3D printer can be used to manufacture the structure shown in Fig. 11 immediately and relatively more cheaply. A copper tape was coated on the sensor casing for shielding. After these procedures, the fabrication of the sensor was completed as shown in Fig. 12.

4.2 Fabrication of the tactile display

In this study, the basic material of the actuator is acrylonitrile butadiene rubber (NBR). NBR has a good dielectric constant and is therefore mostly used for high-voltage power cable insulators. Moreover, it has controllable mechanical properties when mixed with additive materials such as a plasticizer and a cross-linking agent. NBR is a synthetic rubber copolymer of acrylonitrile (ACN) and butadiene. Many types of NBRs are used in engineering applications, and each model has a different ratio of acrylonitrile to butadiene. This ratio determines the properties of the type of NBR. With increased acrylonitrile, NBR becomes a better material, but the material becomes harder to handle. The optimized NBR is KNM-35H (Kumho Chemical Co.) and contains 35% acrylonitrile.

Fig. 11
figure 11

Capacitive force sensor structure

Full size image
Fig. 12
figure 12

Capacitive force sensor

Full size image

The NBR solution was made of a mixture of NBR, dioctyl phthalate (plasticizer), dicumyl peroxide (cross-linking agent), and toluene (solvent). The experimentally optimized ratio of the solvent was determined in previous research by Nguyen et al. [26]. The dielectric elastomer sheet was made of the NBR solution using a dispensing machine.

The sheet is initially in the gel phase and should be hardened to form an elastomer sheet. Previously, it was cured at \(170\,^{\circ }\mathrm {C}\) in an oven. Bubbles could not be perfectly eliminated in this step. Hence, to solve this problem, we used a hot press machine for curing the sheet. The hot press machine provides several advantages. Polymer materials consist of polymer chains. Therefore, curing under high temperature and pressure can make the polymer elastomer denser. This means that the elastomer sheet has a higher dielectric constant; the DEAs therefore perform better under the same voltage.

The electrodes were covered on the sheet by brushing. The electrode powder is a mixture of carbon black and graphite. Carbon black consists of sphere-shaped carbon particles. Graphite is an allotrope of carbon on the accumulated hexagonal layer. Carbon black contributes to conductivity, and graphite takes on the role of a connector. Graphite provides nanoscale stretchability during the working of the actuator.

We made the coupling part, which was molded by using silicone (Shinetsu, KE1606) with a mold. First, the mixture of the silicone and hardener was placed on the bottom part of the mold. Then, the air bubble was removed in a vacuum chamber, and the mixture was placed in the upper part to close the mold. Finally, the fabrication of the tactile display was completed by attaching the coupling part and the actuator, as shown in Fig. 13. The tactile display module was made as shown in Fig.  14.

Fig. 13
figure 13

Tactile display

Full size image
Fig. 14
figure 14

Tactile display module

Full size image
Fig. 15
figure 15

Experimental result of the capacitive force sensor. a Experimental result on X-axis of the capacitive force sensor. b Experimental result on Y-axis of the capacitive force sensor. c Experimental result on Z-axis of the capacitive force sensor

Full size image

5 Experiments

5.1 Experiments on the sensor

The sensor was held in a jig. A push–pull gauge was attached to the motorized stage with an applied force on the tip of the sensor. The motorized stage was controlled using LabView™. The capacitance of the sensor was shown in the LabView program. The capacitance–digital converter (CDC) chip soldered on the PCB sends the capacitance to a microcontroller (STM32F103 Cortex-M3). The microcontroller communicates with LabView using the CAN protocol. The resulting graph of the sensor is shown in Fig.  15. Each graph shows the capacitive response on each axis. As demonstrated by the graph, the sensor worked well. To make the sensor a force sensor, it should be calibrated. The relation between the force and the capacitance is measured with an experimental system shown in Eq.  12.

$$\begin{aligned} \left[ {\begin{array}{*{20}{c}} {{C_1}}\\ {{C_2}}\\ {{C_3}} \end{array}} \right] = \mathbf{C }\left[ {\begin{array}{*{20}{c}} {{F_x}}\\ {{F_y}}\\ {{F_z}} \end{array}} \right] = \left[ {\begin{array}{*{20}{c}} { 56.90}&{}\quad { - 29.46}&{}\quad {434.77}\\ { 51.78}&{}\quad {244.32}&{}\quad {343.83}\\ { 724.84}&{}\quad {71.15}&{}\quad {398.35} \end{array}} \right] \left[ {\begin{array}{*{20}{c}} {{F_x}}\\ {{F_y}}\\ {{F_z}} \end{array}} \right] \end{aligned}$$
(12)

Equation 12 is used to convert the capacitance to a force. Consequently, calibration matrix \(\mathbf {A}\) is the inverse of matrix \(\mathbf {C}\) as shown in Eq. 13.

$$\begin{aligned} \begin{aligned} \left[ {\begin{array}{*{20}{c}} {{F_x}} \\ {{F_y}} \\ {{F_z}} \end{array}} \right]&= {{\mathbf {C}}^{ - 1}}\left[ {\begin{array}{*{20}{c}} {{C_1}} \\ {{C_2}} \\ {{C_3}} \end{array}} \right] = {\mathbf {A}}\left[ {\begin{array}{*{20}{c}} {{C_1}} \\ {{C_2}} \\ {{C_3}} \end{array}} \right] \\&= \left[ {\begin{array}{*{20}{c}} { - 0.009}&{}\quad { - 0.0006}&{}\quad {0.0015} \\ { - 0.0032}&{}\quad {0.0041}&{}\quad {0.0000} \\ { - 0.0022}&{}\quad {0.0004}&{}\quad {- 0.0002} \end{array}} \right] \left[ {\begin{array}{*{20}{c}} {{C_1}} \\ {{C_2}} \\ {{C_3}} \end{array}} \right] \end{aligned} \end{aligned}$$
(13)

To evaluate performance of the sensor, condition number \(\hat{k}\) was calculated. First, calibration matrix \(\mathbf {A}\) was expressed with a singular value decomposition as shown in Eq.  14.

$$\begin{aligned} \mathbf{A } = \mathbf{UD }{\mathbf{V }^T}\ \end{aligned}$$
(14)

In Eq.  14, diagonal matrix \(\mathbf {D}\) is the matrix of the eigenvalues, and matrix \(\mathbf {D}\) is calculated as shown in Eq.  15.

$$\begin{aligned} \mathbf{D } = \left[ {\begin{array}{*{20}{c}} {0.0053}&{}\quad 0&{}\quad 0\\ 0&{}\quad {0.0024}&{}\quad 0\\ 0&{}\quad 0&{}\quad {0.0011} \end{array}} \right] \end{aligned}$$
(15)

Therefore, condition number \(\hat{k}\) is 4.86 as shown in Eq. 16

$$\begin{aligned} \hat{k} = \frac{{0.0053}}{{0.0011}} = 4.86 \end{aligned}$$
(16)

The condition number is considerably low. To be more specific, the sensor can measure only small forces due to the low Young’s modulus. As shown in Fig.  16, calibrated forces differ from applied force. The reason is that the capacitive sensor does not have a linear relationship between force and capacitance value; hence, the calibration matrix has a limit of accuracy. For the same reason, the accuracy of the sensor has limitation as shown in Fig.  17. A new numerical method is required to solve the problem of nonlinear calibration matrix.

Fig. 16
figure 16

Applied force and measured forces on each axis

Full size image
Fig. 17
figure 17

Accuracy of the sensor on each axis

Full size image

5.2 Experiments on tactile display

To test the tactile display, input voltages were applied to the tactile display. The voltage was produced by a function generator (Tektronix AFG3021) and amplified with a high-voltage amplifier (Trek high-voltage amplifier 10). In Fig.  18, the actuator could produce \(168\,\upmu \)m. In our case, we need to discuss the SA I and FA I receptors [27, 28]. SA I receptors respond to stimulations in the range of 0.4–3 Hz, and FA I receptors respond to stimulations in the range of 3–40 Hz. The threshold of the two cases is 100 \(\upmu \)m at 1 Hz, respectively [29]. Hence, the tactile displays show sufficient performance for our purpose.

Different from the existing tactile displays, the cylindrical structure of the new coupling part is light in weight. Thus, the convenience of handling is increased. Besides, the novel fabrication method enables cost-effective and enhanced performance of the tactile displays.

Fig. 18
figure 18

Displacement of the tactile display at 1 Hz

Full size image
Fig. 19
figure 19

Design of the entire system of the handheld surgical tool

Full size image

6 System integration

The surgical tool system was assembled as shown in Fig.  19. The sensor can measure the three-axis direction of force, and each force determines the operation of each tactile display. The change in the capacitance by the force on the needle tip is measured with a capacitance digital convertor (AD7147, Analog Device Inc.). The converted signal is transmitted to a microcontroller, and the LabView program on a PC takes the capacitance value via CAN communication.

The LabView program generates a signal of which the frequency is controlled by capacitance. It is because, according to physiology, the strength of the tactile sense is not determined by the magnitude of the neurosignal, but by its frequency. With a higher operation frequency of the tactile display, the stimuli felt by a human are stronger, as shown in Fig.  20. The signals are output via a data acquisition board (National Instrument, USB-6525). High-voltage sources are required for the operation of the tactile displays. A high-voltage DC–DC convertor (IMAGIS Co. Ltd) was adopted. It converts a 5 V signal to 3.5 kV. The signal generated with LabView is sent to the high-voltage DC–DC convertor using the data acquisition board. The high-voltage signal operates the actuators of the tactile displays. Finally, the tactile stimuli are transmitted to the finger of the user with the tactile displays. The developed system is shown in Fig.  21.

Fig. 20
figure 20

Tactile stimuli and human feeling

Full size image
Fig. 21
figure 21

Developed system

Full size image

7 Experiments on the prototype

Basically, the experiment setup combines the sensor and tactile display experimental system. The push–pull gauge measured the force applied to the tip of the sensor. The tactile display was operated with the integration system. The laser measurement system measured the movement of the tactile display.

As shown in Figs. 22 and 23, the operation of the tactile display controlled with the capacitive force sensor is normal. Therefore, the smart handheld surgical tool with tactile feedback works normally.

Additionally, user tests were performed. Three people used the surgical tool. The experimenters were one 28-year-old male, one 30-year-old male, and one 27-year-old female. Their tasks were two; stimuli of the tactile display are enough, and change in the stimuli can be distinguished by the applied force on the sensor. Although there were differences of strength of the tactile sense among the users, all the subjects could distinguish different stimuli by forces applied in the sensor. As a further study, individual optimization of frequency or voltage for applying on medical field will be needed.

Fig. 22
figure 22

Frequency of the tactile display versus force applied on the force sensor

Full size image
Fig. 23
figure 23

Displacement of the tactile display versus force applied on the force sensor

Full size image

8 Conclusions

In this paper, a smart surgical tool equipped with a tactile sensor and a tactile display was developed for microsurgery. The sensor is a capacitive one, and the tactile display is actuated with a dielectric elastomer. The control part takes the magnitude and the direction of force from the sensor and generates a signal to operate the actuators of the tactile displays. A prototype of the whole system was fabricated, and its performance was successfully tested. The surgical tool is expected to contribute to the improvement of surgical technologies.

In future works, the sensing performances of the sensor will be upgraded to conduct inside eye or outside force sensing during surgery, especially force resolution. Also, we are conducting the evaluation of the overall system which is a tactile feedback controlled in the complete human in the loop system.