Modeling and Analysis of Soft Bionic Fingers for Contact State Estimation

Abstract

Contact state estimation is significant for evaluating grasp stability of bionic hands, especially in unknown environments or without visual/tactile feedback. It still remains challenging, particularly for soft bionic hands without integrating complicated external sensors on soft fingers. Focusing on this issue, a proprioceptive-sensing-based systematic solution is proposed to estimate the contact state of soft bionic fingers in a single grasp. A kinematic model for soft fingers is first developed to capture the joint rotation angles and tendon displacement. A kinetostatic model is further built to estimate the contact force when soft fingers come in contact with objects. On this basis, a system stiffness model for soft fingers during preshaping and initial contact with objects is proposed to perceive the contact state. Moreover, an instantaneous stiffness model for soft fingers when initial contact occurs is developed for estimating the contact position on certain phalanges, especially the contact position along the distal phalange. The proposed proprioceptive-sensing-based approach is the first application in soft fingers without integrating complicated external sensors, which makes them concise and practical. Experiments are carried out to demonstrate the effectiveness and efficiency of our proposal.

1 Introduction

Robotic hands have become prevalent for decades. They can be employed in diverse fields ranging from industrial automation, and space exploration to medical assistance [1]. They can accomplish a large variety of tasks precisely and efficiently instead of humans, especially in hostile environments [2]. Traditional rigid robotic hands are usually fully actuated, which makes the mechanisms dexterous and delicate [3]. Motors independently drive each joint of the robotic hands through transmission mechanisms such as gear [4] or tendon [5]. But they also have several deficiencies, such as complex control systems and excessive mass. Underactuated mechanisms are increasingly adopted in the design of robotic hands to overcome the drawbacks mentioned above [6], which have fewer actuators than the degrees of freedom. These adaptive systems provide passive motions and are particularly capable of enveloping objects of varying shapes mechanically. There are mainly two different forms of underactuated robotic hands, i.e., the tendon-based mechanism [7] and the linkage-based mechanism [8]. These underactuated transmission mechanisms can reduce the number of actuators to only one in each finger. For the former, a motor is generally connected with the tendon transmission mechanism and drives the finger to bend in one direction. Torsion springs are usually used in the joints to extend the finger when the motor releases the driving mechanism. For the latter, a motor is employed to drive a revolute joint or a prismatic joint, which makes the finger realize bidirectional movement actively [9].

In recent years, more and more attention has been paid to soft robotic hands which are mainly made of flexible materials [10]. They are compliant with unstructured environments and can provide high levels of safety when interacting with humans. On the one hand, they inherit the tendon transmission mechanism of the underactuated robotic hands described above [11]. On the other hand, they also usually transmit motion through pneumatic structures or other driving forms, such as granular/layered jamming structures, low-melting-point alloys and shape memory polymers/alloys [10]. In this paper, the described soft bionic hand is developed based on the tendon transmission mechanism.

Grasping is the most common application for robotic hands. For a particular task, contact state estimation is significant for evaluating grasp stability, especially in unknown environments or without visual/tactile feedback [12, 13]. There are mainly two different approaches to deal with the contact state estimation for robotic hands. One is the conventional tactile sensors embedded on the phalanges of robotic fingers. The other is the proprioceptive sensors which basically refer to the ones at motors, such as optical encoders and current/voltage sensors. The first method is to collect data directly from exteroceptive sensors for tactile feedback. The second method is to detect data within robotic mechanisms for evaluating the grasping capability mediately.

For the first method, different types of sensors, such as optoelectric, piezoresistive, and capacitive tactile sensors, are usually adopted for various applications [14]. They are advisable for measuring the corresponding contact state accurately [15]. Naeini et al. [16] proposed a dynamic-vision-based measurement approach for estimating the contact force of a gripper in a single grasp. Fernandez et al. [17] built an inexpensive compliant tactile fingertip which was capable of detecting contact localization. Kim et al. [18] proposed a tactile sensing module and installed three air pressure sensors at the tip of each robotic finger. The sensing data was utilized to recognize the contact point of the finger against an object. Deng et al. [19] presented a grasping control method based on Gaussian Mixture Model to capture the contact information from the tactile data. Recently, research on sensor skins becomes a growing trend, especially in the field of soft robotic hands. The skins could cover large areas of the robotic hands, and provide contact information, such as spatial localization. Thuruthel et al. [20] embedded a soft resistive sensor into a soft actuator to estimate the coordinates of the fingertip when in contact. Shih et al. [21] integrated sensor skins on a soft gripper, which enables haptic visualization when twisting an object.

For the second method, proprioceptive sensors are seen as an alternative to conventional tactile sensors. In the early days, Kaneko et al. [22] proposed an active sensing method through joint compliance, and the method could estimate the contact point when grasping an unknown object. Belzile and Birglen [23] analyzed the stiffness of common underactuated fingers and established the contact locations without using any exteroceptive sensors. Santina et al. [24] proposed an approach to estimate interaction forces from postural measures. Compliance model and the geometric configuration of an underactuated robotic hand were considered. Abdeetedal and Kermani [13] designed a link-driven underactuated gripper with a trimmer potentiometer and an embedded load cell. Experiments demonstrated tactile capability by using proprioceptive sensors. In summary, proprioceptive sensing allows for a low cost, yet accurate grasping assessment. It also makes the robotic hands more concise and practical, without integrating complicated external sensors on the robotic fingers. In most cases, this method is used with traditional fully actuated and underactuated mechanisms. But its application in the field of soft robotic hands has not been seriously considered.

Our previous work [25] investigates a stable grasp method by planning the contact position and applied force for each robotic finger. It focuses on a traditional rigid robotic hand and fingertip grasping. In this paper, based on a soft bionic hand we designed, we move a step further by proposing a systematic solution to estimate the initial contact state of the soft fingers through proprioceptive sensors. The initial contact state has a close relationship with grasp stability. It could be an efficient alternative to conventional tactile sensors, which allows the soft fingers to have a low cost and concise mechanism. The main contributions are:

• 1. A proprioceptive-sensing-based systematic solution is proposed to estimate the contact state of soft robotic fingers in a single grasp. It is the first application in soft fingers without integrating complicated external sensors, which makes them concise and practical.

• 2. A kinematic model for soft fingers is developed to capture the joint rotation angles and tendon displacement. And a kinetostatic model is further built to estimate the contact force when soft fingers come in contact with objects.

• 3. On the basis above, a system stiffness model for soft fingers during preshaping and initial contacting with objects is proposed to perceive the contact state. Furthermore, an instantaneous stiffness model for soft fingers when initial contact occurs is developed for estimating the contact position on certain phalanges, especially the contact position along the distal phalange. Experiments are carried out to demonstrate the effectiveness and efficiency of our proposal.

The paper is organized as follows. Section 2 describes the systematic architecture of the proposed approach. Then the soft robotic hand used in this paper is described in Sect. 3. The kinematic modeling and kinetostatic modeling for soft fingers are carried out successively in Sect. 4. On the basis above, Sect. 5 presents the contact stiffness models of soft fingers. Section 6 reports the experimental details, results and discussion. Finally, the conclusion is given in Sect. 7.

2 Systematic Architecture

The paper aims to provide a systematic solution for estimating the contact state of soft robotic fingers in a single grasp based on proprioceptive sensing. Unlike traditional strategies that mainly pay attention to fully actuated and underactuated mechanisms, our method is the first application in soft fingers without integrating complicated external sensors. The overall architecture of the proposed systematic solution is shown in Fig. 1. It is mainly composed of two components: kinematic/kinetostatic modeling and contact stiffness modeling. The kinematic modeling represents the motion characteristics of the soft fingers and is used for obtaining desired grasp configurations. The kinetostatic modeling is built for estimating the contact force when the soft fingers come in contact with objects. Based on the above modeling information, the system stiffness model is created to perceive the initial contact state, i.e., whether the contact occurs or not. Furthermore, the instantaneous stiffness model is proposed to estimate the initial contact position, especially the contact position along the distal phalange. According to the result of contact state estimation, the grasp stability of the soft fingers can be evaluated. Note that the grasp stability evaluation is not the content of this paper.

Fig. 1

Overall architecture of the proposed systematic solution

In this paper, we aim to estimate the initial contact state when the soft fingers come in contact with objects by using proprioceptive sensors, which has a close relationship with grasp stability. The situation of enveloping grasp after initial contact is not the focus of this paper. In addition, the contact between the soft finger and the object is considered to be rigid and frictionless. The object is immovable with reference to the palm to ensure that the soft finger could envelop it instead of pushing it away. The dynamic effect could be ignored for simplifying the modeling and analysis because the kinetic energy of the soft finger is quite low when a grasp happens. The proprioceptive sensors, i.e., position and current, are integrated into the motor.

3 Soft Bionic Hand

The hand prototype, depicted in Fig. 2a, is a soft bionic hand developed by the authors [26]. It performs human-like grasp in activities of daily life and has high compliance when interacting with unstructured environments. The prototype has 16 degree-of-freedoms and 6 actuators. Each soft finger is a single-input that can achieve flexion/extension motion. The soft thumb is driven by two independent actuators for obtaining flexion/extension and abduction/adduction movements. The design principle and parameters of the soft fingers are illustrated in Fig. 2b, and so is the soft thumb. Each finger possesses three elastic joints, i.e., Metacarpophalangeal (MCP) joint, Proximal Interphalangeal (PIP) joint and Distal Interphalangeal (DIP) joint. The proportion for the three joint angles is chosen $1:1:\frac{2}{3}$ to mimic human-like grasping [27, 28]. The distinct feature is the incorporation of elastic joints and soft phalanges. The transmission system of each finger is based on the tendon-actuated mechanism, and the elastic joints are built by torsional springs with optimized stiffness. The soft phalanges are constructed through the laminated multi-material manufacturing approach. The elasticity modulus of Polymer I is much larger than that of Polymer II, so the Polymer I hardly deforms when the tendon goes through the channels in it. Teflon tubes are used in the channels to minimize the sliding friction. The soft finger is assembled reliably through adhesive technology. $r_{1}$, $r_{2}$ and $r_{3}$ represent the radius of pulleys. $k_{1}$, $k_{2}$, $k_{3}$ are the stiffness of torsional springs, and $\theta_{10}$, $\theta_{20}$, $\theta_{30}$ are the corresponding pre-tightening angles respectively. $T$ represents the tendon force of the soft finger.

Fig. 2

The Dr. hand, a soft bionic hand. a The prototype of the soft hand. b Design principle of the soft finger

4 Kinematic and Kinetostatic Modeling

In this paper, the initial contact state estimation is realized by detecting the variation of the contact stiffness as modeled from the proprioceptive sensing. Before conducting the contact stiffness analysis, it is necessary to perform the kinematic and kinetostatic modeling. The former is to capture the joint rotation angles and tendon displacement under a certain tendon force. The latter is to compute the contact force that is related to the transmission mechanism of the soft finger.

4.1 Kinematic Modeling

As shown in Fig. 3, the soft finger contains two basic movements including the rotation at each elastic joint and the bending deformation of each soft phalange. In our previous work [26], the independence of the two different movements has been proved. So they can be calculated separately.

Fig. 3

Parameters of a soft module under a certain tendon force

First, consider the bending deformation of the soft phalange. Since Polymer I is much stiffer than Polymer II, so the former part could be considered as a rigid link. According to the design principle of the soft finger, the deformation of the soft parts is small. So the soft joint is simplified to a rectangular cantilever beam. The maximum bending angle for the endpoint N can be achieved according to the Euler–Bernoulli equation:

$$\theta_{i} = \frac{{M_{f} d_{i} }}{{E_{f} I}}$$

(1)

where $M_{f} = {\text{Th}}$ represents the bending moment, $E_{f}$ is the elastic modulus of Polymer II, $I = \frac{{wf \cdot tf^{3} }}{12}$ is the second moment of area for the beam.

According to the geometric parameters of the soft phalange under a tendon force, the tendon displacement can be computed:

$$l_{i} = \left( {d_{i} { + 2}h_{i} \tan \beta } \right) - {\text{2cos}}\alpha_{i} \left[ {\left( {\frac{{d_{i} }}{{\theta_{i} }} - h_{i} } \right)\tan \frac{{\theta_{i} }}{2} + h_{i} \tan \beta } \right]$$

(2)

where $\alpha_{i} = \frac{{\theta_{i} }}{2}$.

Second, consider the elastic joint rotation. When a tendon force is applied, the moment equilibrium equation can be expressed as:

$${\text{Tr}}_{j} = k_{j} \left( {\theta_{j} + \theta_{j0} } \right)$$

(3)

where $\theta_{j}$ is the rotation angle at each elastic joint, and $\theta_{j0}$ is the corresponding pre-tightening angle. $\theta_{j} = 0$, if $\frac{{{\text{Tr}}_{j} }}{{k_{j} }} \le \theta_{j0}$, and $\theta_{j} { = }\frac{{{\text{Tr}}_{j} }}{{k_{j} }} - \theta_{j0}$, if $\frac{{{\text{Tr}}_{j} }}{{k_{j} }} > \theta_{j0}$. When the elastic joint rotates at an angle, the tendon displacement can be easily obtained: $l_{j} = \theta_{j} r_{j}$. Then the total tendon displacement can be achieved:

$$l = \sum\limits_{i = 1}^{4} {l_{i} } + \sum\limits_{j = 1}^{3} {l_{j} }$$

(4)

The result of the kinematic modeling is shown in Fig. 4. Figure 4a presents the relationship between the rotation angle of each joint and the tendon force. Figure 4b indicates the change of the tendon displacement with the tendon force. The inflection point indicates that the three elastic joints begin to rotate.

Fig. 4

Result of the kinematic modeling

4.2 Kinetostatic Modeling

In this section, we aim to construct a kinetostatic model for exploring the relationship between contact force and the driving moment generated by the actuator. Since the soft phalange behavior is burdensome to be considered directly in the kinetostatic modeling, the soft joint is converted to a revolute joint with a torsion spring based on the pseudo rigid body model [29, 30], as illustrated in Fig. 5. The simplification is effective because the physical dimension of the soft joint is small and its bending deformation is tiny.

Fig. 5

Kinetostatic parameters of the simplified soft finger model

Assuming one contact at each phalange. According to the principle of virtual work, the input virtual powers are equal to the output virtual powers in the underactuated system. So we can obtain:

$$t^{T} \omega_{a} = \sum\limits_{i = 1}^{7} {\xi_{i} \circ }\, \zeta_{i}$$

(5)

where, $t = \left[ {\begin{array}{*{20}c} {T_{a} } & {T_{1} } & \cdots & {T_{7} } \\ \end{array} } \right]^{T}$, $\omega_{a} = \left[ {\begin{array}{*{20}c} {\dot{\theta }_{a} } & {\dot{\theta }_{1} } & \cdots & {\dot{\theta }_{7} } \\ \end{array} } \right]^{T}$, $\xi_{i} = \left[ {\begin{array}{*{20}c} {w_{i} } & {v_{i}^{x} } & {v_{i}^{y} } \\ \end{array} } \right]^{T}$, $\zeta_{i} = \left[ {\begin{array}{*{20}c} {f_{ti} } & {f_{i} } & {\tau_{i} } \\ \end{array} } \right]^{T}$. $t$ is the input torque vector exerted by the actuator and torsional springs in each joint, which can be calculated through (1) and (3). $\omega_{a}$ represents the joint velocity vector. $\xi_{i}$ is the twist of the $i{\text{th}}$ contact point on the $i{\text{th}}$ phalange, and $\zeta_{i}$ is the corresponding wrench. The operator $\circ$ stands for the reciprocal product of screws in the plane. Assuming the actuator is applied at the $1{\text{th}}$ joint of the soft finger, $T_{a}$ is the actuation torque, $T_{i}$ is the torque generated by the torsional springs at the $i{\text{th}}$ joint of the soft finger, and $\dot{\theta }_{a}$ is the angular velocity of the actuator. $w_{i}$ is the angular velocity of the $i{\text{th}}$ phalange, and $v_{i}^{x}$, $v_{i}^{y}$ are, respectively, the x and y components of the velocity at the $i{\text{th}}$ contact point. $f_{ti}$, $f_{i}$, $\tau_{i}$ are, respectively, the tangential force, the normal force and the torque applied by the $i{\text{th}}$ phalange.

$\xi_{i}$ can be further defined as [31]:

$${{\varvec{\upxi}}}_{i} = \sum\limits_{k = 1}^{i} {\dot{\theta }_{k} } {{\varvec{\upxi}}}_{i}^{{o_{k} }}$$

(6)

where $\xi_{i}^{{o_{k} }} = \left[ \begin{gathered} 1 \\ {\text{Er}}_{ki} \\ \end{gathered} \right]$, $E = \left[ {\begin{array}{*{20}c} 0 & { - 1} \\ 1 & 0 \\ \end{array} } \right]$. $\xi_{i}^{{o_{k} }}$ is the joint twist associated to $O_{k}$ with respect to the $i{\text{th}}$ contact point. $r_{ki}$ is the vector from $O_{k}$ to the $i{\text{th}}$ contact point.

According to the principle of friction cone, the relationship of $f_{t}$ and $f$ can be established:

$$f_{t} = \mu f$$

(7)

where the coefficient of friction $\mu$ simply allows us to relate $f_{t}$ and $f$, $f = \left[ {\begin{array}{*{20}c} {f_{1} } & {f_{2} } & \cdots & {f_{7} } \\ \end{array} } \right]^{T}$, $f_{t} = \left[ {\begin{array}{*{20}c} {f_{t1} } & {f_{t2} } & \cdots & {f_{t7} } \\ \end{array} } \right]^{T}$, $\mu = {\text{diag}}\left( {\begin{array}{*{20}c} {\mu_{1} } & {\mu_{2} } & \cdots & {\mu_{7} } \\ \end{array} } \right)$.

Similarly, $\tau_{i}$ also can be written as:

$$\tau = \eta f$$

(8)

where the coefficient matrix $\eta$ simply allows us to relate $f$ and $\tau$, $\tau = \left[ {\begin{array}{*{20}c} {\tau_{1} } & {\tau_{2} } & \cdots & {\tau_{7} } \\ \end{array} } \right]^{T}$, $\eta = {\text{diag}}\left( {\begin{array}{*{20}c} {\eta_{1} } & {\eta_{2} } & \cdots & {\eta_{7} } \\ \end{array} } \right)$.

Hence $\zeta_{i}$ can be written as

$$\zeta_{i} = f_{i} \left( {\mu_{i} x_{i}^{ * } + y_{i}^{ * } + \eta_{i} z_{i}^{ * } } \right)$$

(9)

where, $x_{i}^{ * } = \left[ {\begin{array}{*{20}c} 1 & 0 & 0 \\ \end{array} } \right]^{T}$, $y_{i}^{ * } = \left[ {\begin{array}{*{20}c} 0 & 1 & 0 \\ \end{array} } \right]^{T}$, $z_{i}^{ * } = \left[ {\begin{array}{*{20}c} 0 & 0 & 1 \\ \end{array} } \right]^{T}$, $x_{i}^{ * }$ represents the unit wrench corresponding to a pure force along $x_{i} = \left[ {\begin{array}{*{20}c} 1 & 0 \\ \end{array} } \right]^{T}$. Similarly, $y_{i}^{ * }$ along $y_{i} = \left[ {\begin{array}{*{20}c} 0 & 1 \\ \end{array} } \right]^{T}$, and $z_{i}^{ * }$ along the out-of-plane axis $z$.

Substituting (6) and (9) into (5)

$$t^{T} \omega_{a} = f^{T} J\dot{\theta }$$

(10)

where $J = J_{1} - \mu J_{2} + \eta J_{3}$, $\dot{\theta } = T\omega_{a}$. $J$ is the Jacobian matrix which is dependent on the coefficients $\mu$ and $\eta$, the relative orientation of the soft phalanges, and the contact location on the soft phalanges. Note that the coefficients $\mu$ and $\eta$ are usually small and can be neglected. $\dot{\theta }$ represents the time derivative of the joint coordinates. $T$ is related to the transmission mechanism of the soft finger. And the actuation torque $T_{a}$ can be propagated to each phalange based on the mechanism.

According to (6) and (9), $J$ can be deduced as

$$J = \left[ {\begin{array}{*{20}c} {r_{11}^{T} x_{1} } & 0 & \cdots & 0 \\ {r_{12}^{T} x_{2} } & {r_{22}^{T} x_{2} } & \cdots & 0 \\ \cdots & \cdots & \cdots & \cdots \\ {r_{17}^{T} x_{7} } & {r_{27}^{T} x_{7} } & \cdots & {r_{77}^{T} x_{7} } \\ \end{array} } \right]$$

(11)

where $r_{ii}^{T} x_{i} = k_{i}$, $k_{i}$ is the distance between the $i{\text{th}}$ contact point and the $i{\text{th}}$ joint. $r_{ij}^{T} x_{j} = k_{j} + \sum\limits_{k = i}^{j - 1} {l_{k} \cos \left( {\sum\limits_{m = k + 1}^{j} {\theta_{m} } } \right)}$, $i < j$ and $l_{i}$ represents the length of the $i{\text{th}}$ phalange.

Therefore, (10) can be further simplified as

$$f = J^{ - T} T^{ * T} t$$

(12)

where $T^{ * }$ is the pseudo-inverse matrix of $T$. $T^{ * }$ can be calculated according to the transmission mechanism of the soft finger.

$$T^{ * } = \left[ {\begin{array}{*{20}c} X & E \\ \end{array} } \right]^{T}$$

(13)

where, $X$ is the transmission coefficient vector that depends on the transmission mechanism of the soft finger. ${\mathbf{X}}{ = }\left[ {\begin{array}{*{20}c} {\frac{1}{{X_{1} }}} & { - \frac{{X_{2} }}{{X_{1} }}} & \cdots & { - \frac{{X_{7} }}{{X_{1} }}} \\ \end{array} } \right]$, $X_{1} { = }1$, $X_{i} { = }{ - }\frac{{r_{i} }}{{r_{1} }}, \, i > 1$, $r_{1}$ and $r_{i}$ respectively represent the radius of the pulleys located at the corresponding joints. $E$ is the $7 \times 7$ identity matrix.

5 Contact Stiffness Modeling

In this section, the contact stiffness as seen from the proprioceptive sensor is defined. The system stiffness modeling during preshaping and initial contact with objects is conducted. It is used to perceive the initial contact state, i.e., whether the contact occurs or not. Furthermore, the instantaneous stiffness with a small increase of the actuation torque is modeled for estimating the initial contact position on certain phalanges, especially the contact position along the distal phalange.

5.1 System Stiffness Modeling During Preshaping and Initial Contact

The preshaping of the soft finger means that there is no contact during this stage. So the contact force that occurs between the object and the phalange is null, i.e., $f = 0$. In addition, $\tau = T^{ * T} t = \left[ {\begin{array}{*{20}c} {\tau_{1} } & {\tau_{2} } & \cdots & {\tau_{7} } \\ \end{array} } \right]$ can be obtained according to (12). They represent the equivalent torques at each joint, i.e., the sum of torques generated by the torsional springs and the actuator. Thus, each equivalent torque is also null:

$$T^{ * T} t = 0$$

(14)

Accordingly, the preshaping motion of the soft finger can be computed. The system stiffness of the soft finger is defined as $K_{a}$ that is seen from the single actuator:

$$K_{a} = \frac{{T_{a} }}{{\theta_{a} }}$$

(15)

where $T_{a}$ is the actuation torque and $\theta_{a}$ is the rotation angle of the actuator.

We assume that the actuator is applied at the MCP joint for simplification, and the transmission ratio between the corresponding pulley and the actuator is 1. For a preshaping motion sequence, $T_{a}$ and $\theta_{a}$ can be obtained according to Fig. 3 and (14). Figure 6 shows the change in the system stiffness during a preshaping motion sequence. In the first stage (preload-overcoming), $K_{a}$ remains constant and the preload generated by the torsional springs is gradually overcome with the increase of actuation torque $T_{a}$. Note that the preload exists only at the MCP, PIP and DIP joints. In the second stage, as the three joints begin to rotate, $K_{a}$ gradually decreases.

Fig. 6

The change of the system stiffness for the soft finger during a preshaping motion sequence

Note that $K_{a}$ will change when coming into contact with an object. Assuming only one contact occurs at the $i^{th}$ phalange. The joint angles from $\theta_{1}$ to $\theta_{i}$ will remain constant, and the other will continue to grow as the tendon force increases. Figure 7 shows that the contact occurs when the DIP, PIP and MCP joints rotate certain angles. At this time, the actuation torque $T_{a}$ are 15.75 Nmm (Fig. 7a) and 18Nmm (Fig. 7b), respectively. Dotted lines represent the system stiffness curves in a preshaping motion sequence. $K_{ai}$ represents the system stiffness when the contact point is on the $i{\text{th}}$ phalange. The result shows that $K_{a}$ changes significantly when a contact occurs after a certain amount of preshaping. It can be used to initially perceive the contact state and determine whether the contact occurs or not. Another indication is that the change of $K_{a}$ is firmly dependent upon the contact location on the phalanges. It is necessary to further consider the instantaneous stiffness when contact occurs to evaluate where the initial contact happens precisely because the computation is conducted under a quasi-static condition.

Fig. 7

The system stiffness of the soft finger increases significantly when initial contact with an object occurs

5.2 Instantaneous Stiffness Modeling When Initial Contact Occurs

To precisely evaluate the initial contact location on the soft finger when contact occurs, the instantaneous stiffness $K_{c}$ is defined, namely

$$K_{c} = \frac{{dT_{a} }}{{d\theta_{a} }}$$

(16)

The elements of $f$ are null excluding the $i{\text{th}}$ component because the only one contact point $i$ is on the $i{\text{th}}$ phalange. The Jacobian matrix ${\mathbf{J}}^{ - T}$ in (12) can be rewritten as:

$$J^{ - T} = \left[ {\begin{array}{*{20}c} {j_{1} } & {j_{2} } & {\begin{array}{*{20}c} {...} & {j_{i} } & {...} \\ \end{array} } & {j_{7} } \\ \end{array} } \right]^{T}$$

(17)

For ease of calculation, the $i{\text{th}}$ component of $f$ is first not considered. Thus, $J^{ - T}$ is reduced to (7–1) rows where corresponding elements in $f$ is equal to zero, namely ${\mathbf{J}}_{i}^{*}$.

$$J_{i}^{*} = \left[ {\begin{array}{*{20}c} {j_{1} } & {j_{2} } & {\begin{array}{*{20}c} {...} & {j_{i - 1} } & {j_{i + 1} } & {...} \\ \end{array} } & {j_{7} } \\ \end{array} } \right]^{T}$$

(18)

$J_{i}^{*}$ is further simplified based on the Gauss-Jordan elimination method. Then (12) can be rewritten as

$$J_{i}^{*} \tau = 0$$

(19)

Differentiating with respect to $T_{a}$ on both sides of (19), it becomes

$$\frac{{dJ_{i}^{*} }}{{dT_{a} }}\tau + J_{i}^{*} \frac{d\tau }{{dT_{a} }} = 0$$

(20)

Note that $\tau$ is the equivalent torque vector at joints and keeps null during preshaping. So (20) can be further rewritten as

$$J_{i}^{*} \frac{d\tau }{{dT_{a} }} = J_{i}^{*} \left( {X - K\frac{d\theta }{{dT_{a} }}} \right) = 0$$

(21)

where $K = {\text{diag}}\left( {\begin{array}{*{20}c} {K_{1} } & {K_{2} } & \cdots & {K_{7} } \\ \end{array} } \right)$ represents the stiffness matrix, and $K_{i}$ is the stiffness of the torsional spring on the $i{\text{th}}$ joint. Note that the stiffness of the equivalent torsional springs on the soft joints can be calculated by $K_{i} { = }\frac{{E_{f} I}}{{d_{i} }}$ approximatively.

To make the system fully determined, the $i{\text{th}}$ component of $f$ is reconsidered. Similarly, the contact equilibrium equation can be defined as

$$\Gamma_{i}^{T} \frac{d\theta }{{dT_{a} }} = 0$$

(22)

where $\Gamma_{i}^{T}$ represents the coefficient vector when contact occurs on the $i{\text{th}}$ phalange. It ensures the contact configuration of the soft finger and can be calculated by establishing geometric equilibrium equations. Defining that $k_{i}$ is the distance between the contact point and the corresponding joint. The contact position is relative to the MCP joint of the soft finger. The contact point coordinate at the instant of contact can be expressed as

$$\left[ {\begin{array}{*{20}c} {x_{i,c} } & {y_{i,c} } \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {k_{i,c} \sin \alpha_{i} + \sum\limits_{j = 1}^{i - 1} {l_{j} \sin \alpha_{j,c} \, } } & {k_{i,c} \cos \alpha_{i,c} + \sum\limits_{j = 1}^{i - 1} {l_{j} \cos \alpha_{j,c} } } \\ \end{array} } \right]$$

(23)

where $\alpha_{i} = \sum\limits_{j = 1}^{i} {\theta_{j} }$. The contact point coordinate after contact is:

$$\left[ {\begin{array}{*{20}c} {x_{i} } & {y_{i} } \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {k_{i} \sin \alpha_{i} + \sum\limits_{j = 1}^{i - 1} {l_{j} \sin \alpha_{j} \, } } & {k_{i} \cos \alpha_{i} + \sum\limits_{j = 1}^{i - 1} {l_{j} \cos \alpha_{j} } } \\ \end{array} } \right]$$

(24)

Let $\left[ {\begin{array}{*{20}c} {x_{i,c} } & {y_{i,c} } \\ \end{array} } \right]{ = }\left[ {\begin{array}{*{20}c} {x_{i} } & {y_{i} } \\ \end{array} } \right]$, we can obtain:

$$k_{i} {\text{sin(}}\alpha_{i,c} - \alpha_{i} ) = \sum\limits_{j = 1}^{i - 1} {l_{j} \left[ {{\text{sin(}}\alpha_{j} { - }\alpha_{i,c} {\text{) + sin(}}\alpha_{i,c} { - }\alpha_{j,c} {)}} \right]}$$

(25)

Differentiating with respect to $T_{a}$ on both sides of (25), we can obtain

$$\sum\limits_{j = 1}^{i - 1} {\frac{{l_{j} \cos (\alpha_{j} { - }\alpha_{i,c} )}}{{k_{i} }}} \frac{{d\alpha_{j} }}{{dT_{a} }} + \frac{{d\alpha_{i} }}{{dT_{a} }} = 0$$

(26)

Let $\lambda_{j}^{i} { = }\frac{{l_{j} \cos (\alpha_{j} { - }\alpha_{i,c} )}}{{k_{i} }}$, where $\lambda_{1}^{1} { = 1}$. (26) can be rewritten as

$$\sum\limits_{j = 1}^{i} {\lambda_{j}^{i} } \frac{{d\alpha_{j} }}{{dT_{a} }} = 0$$

(27)

By combining (22) and (27), the coefficient vector ${{\varvec{\Gamma}}}_{i}^{T}$ can be obtained

$${{\varvec{\Gamma}}}_{i}^{T} { = }\left[ {\begin{array}{*{20}c} {\sum\limits_{j = 1}^{i} {\lambda_{j}^{i} \, } } & {\sum\limits_{j = 2}^{i} {\lambda_{j}^{i} \, } } & \cdots & {\sum\limits_{j = i}^{i} {\lambda_{j}^{i} \, } } \\ \end{array} } \right]$$

(28)

In addition, based on the transmission mechanism of the soft finger, (16) can be rewritten as

$$K_{c}^{ - 1} = X^{T} \frac{d\theta }{{dT_{a} }}$$

(29)

Finally, $K_{c}^{ - 1}$ is established by combining (21), (22) and (29).

$$K_{c}^{ - 1} = X^{T} \left[ \begin{gathered} J_{i}^{*} K \hfill \\ - \Gamma_{i}^{T} \hfill \\ \end{gathered} \right]^{ - 1} \left[ \begin{gathered} J_{i}^{*} \hfill \\ 0^{T} \hfill \\ \end{gathered} \right]X$$

(30)

When the soft finger is under a certain tendon force, and the initial contact occurs at a certain position on a phalange, the corresponding $K_{c}$ can be computed from (30), as shown in Fig. 8. $K_{c}$ has a distinct change from MCP phalange, PIP phalange to DIP phalange. It contributes to validate at which phalange the initial contact takes place. Note that the change of $K_{c}$ is very small when the contact position occurs along the MCP phalange. One reason is that the variable $k_{1}$ does not appear in (30), so the contact location along the link 1 has no effect on $K_{c}$. The other reason is that the equivalent stiffness of the soft joints is much larger than that of the MCP joint, and the contact location along the link 2 ~ 4 has little effect on $K_{c}$. In addition, the $K_{c}$ curves under arbitrary tendon force are independent. For example, the two $K_{c}$ curves shown in Fig. 8 are generated under the two different actuation torques respectively. They do not affect each other. So the corresponding $K_{c}$ can be computed independently under a certain tendon force, and compared with the estimated one.

Fig. 8

$K_{c}$ is a function of the initial contact position on each phalange of the soft finger, under certain tendon forces. $k_{i} /l_{i}$ represents the initial contact position along the corresponding phalange

Another important feature is that the contact location along the DIP phalange has a great influence on the instantaneous stiffness. So $K_{c}$ can be further used for contact position estimation along the DIP phalange, as shown in Fig. 9. Two typical contact positions are considered because the length of the DIP phalange is small. The contact point near the position A contributes to grasping stability. The contact point near the position B is easy to cause the ejection of the soft finger when contact force increases.

Fig. 9

$K_{c}$ is a function of the initial contact position along the DIP phalange of the soft finger. (a) Two typical contact positions are chosen, position A ($k_{7} /l_{7} = 0.4$) is close to the DIP joint which contributes to grasping stability, position B ($k_{7} /l_{7} = 0.8$) is close to the fingertip which is easy to cause the ejection of the soft finger when contact force increases

In summary, the instantaneous stiffness analysis provides two steps for the initial contact state estimation of the soft finger. First, it perceives the contact state and determines which phalange is in contact. Thus the grasping mode, i.e., enveloping and pinching, can be adjusted. Second, for pinching mode, $K_{c}$ can be further used to adjust the initial contact position along the DIP phalange. Finally, the initial contact state of the soft finger is systematically estimated.

6 Experiment and Discussion

To evaluate the proposed method, different experiments were conducted. The soft finger is equipped with a brushless DC servomotor (FAULHABER 1226S006B, reduction ration 64:1) to drive the tendon transmission mechanism. A Hall sensor and current sensor are integrated into the motor. So the torque and position information can be obtained and used for the estimation of the initial contact state of the soft finger.

6.1 Evaluation of the System Stiffness During Preshaping and Initial Contact

To evaluate the system stiffness during preshaping and initial contact, and perceive the contact state, preshaping experiments and initial contact experiments were carried out, as shown in Fig. 10.

Fig. 10

Experimental setup for the system stiffness evaluation

In each experiment, the actuation current and rotation angle of the motor were recorded simultaneously. Then the actuation torque $T_{a}$ and the actuator angle $\theta_{a}$ were computed. Figure 11a, c and e show the $T_{a}$ curves with no contact and when contact occurs. The curves present an observable variation of $T_{a}$ when the soft finger is in contact with an object, which allows to compute $K_{a}$. Figure 11b, d and f show the $K_{a}$ curves when preshaping and contacting. In the first stage (preload-overcoming), for the system stiffness $K_{a}$, the experimental data is inconsistent with the analysis data. The main reason is that the tendon displacement is small, and the fluctuation is relatively large, especially in the beginning. In the second stage, the three elastic joints begin to rotate and the experimental data is basically consistent with the analysis data. When initial contact occurs, the variation of $K_{a}$ is obviously different from that during preshaping. We can see that it is valid to perceive the contact state and evaluate whether the contact occurs or not, by using the system stiffness analysis method. But these curves do not clearly reflect which phalange the contact occurs on. It will be demonstrated in next section.

Fig. 11

Evaluation of the system stiffness during preshaping. $T_{a}$ and $K_{a}$ increase dramatically when contact occurs. a, b refer to the contact that occurs on the link 5, c, d refer to the contact that occurs on the link 6, e, f refer to the contact that occurs on the link 7

6.2 Evaluation of the Instantaneous Stiffness When Initial Contact Occurs

To evaluate which phalange is in contact when initial contact occurs, a series of experiments were performed. The soft finger configuration is random when contact occurs. The actuation torque $T_{a}$ when initial contact occurs is related to the configuration. An experimental example for estimating the initial contact location is shown in Fig. 12. In the beginning, the soft finger was at rest. During the preshaping stage, $0 < T_{a} < T_{a,c}$, and $T_{a,c}$ represents the actuation torque at the moment of contact. After contact, the soft finger was actuated continuously for a small increase of the actuation torque $\Delta T_{a}$. Figure 13a shows an example for the instantaneous stiffness comparison at each link of the soft finger. For each link, the grasping configuration when initial contact occurs is random. The analytical $K_{c}$ under a certain actuation torque was computed according to (30). The experimental $K_{c}$ was calculated by using the variation actuation torque $\Delta T_{a}$ and the variation actuator angle $\Delta \theta_{a}$, namely, $K_{c} \approx \frac{{\Delta T_{a} }}{{\Delta \theta_{a} }}$. As predicted, a distinct increase of $K_{c}$ can be observed from MCP phalange, PIP phalange to DIP phalange. Note that the $K_{c}$ values of the link 1 ~ 4 were used for the MCP phalange, and the link 5 ~ 6 for the PIP phalange, and the link 7 for the DIP phalange. The result shows the validity of the contact position estimation method, though there are some deviations compared with the ones computed by the stiffness model.

Fig. 12

An example for estimating the initial contact position. a Initial configuration of the soft finger. b Preshaping motion. c Initial contact occurs. d Continue to actuate for a small increase of the actuation torque

Fig. 13

Experimental results of the instantaneous stiffness analysis for each phalange. a An example for $K_{c}$ calculation at each link of the soft finger. b Analytical $K_{c}$ and estimated values for contact position estimation, each experiment refers to seven contacts on each link of the soft finger

To evaluate the accuracy of the initial contact position estimation for each phalange, a total of 21 more contact experiments with different soft hand configurations were performed. Figure 13b shows the experimental result, and an ideal $K_{c}$ line is also illustrated. It shows that the estimated $K_{c}$ is distinct along with the MCP, PIP and DIP phalanges of the soft finger. Its fluctuation range along the ideal line is small with a mean absolute error of 0.37 and a standard deviation of 0.22.

To evaluate the accuracy of the initial contact position estimation along the DIP phalange, 30 more experiments were performed. The experimental conditions are the same as those described above. Due to the small length of the DIP phalange, two typical positions were mainly considered for $K_{c}$ estimation and computation, which can be used for preventing unstable grasp. The contact point near the position $k_{7} /l_{7} = 0.4$ contributes to grasping stability. The contact point near the position $k_{7} /l_{7} = 0.8$ is easy to cause grasping ejection after contacting, as shown in Fig. 14. Corresponding $K_{c}$ curves are shown in Fig. 15a. An ideal $K_{c}$ line is illustrated in Fig. 15b. The deviations of the experimental $K_{c}$ are within a reasonable range compared with the ones computed by the stiffness model. The mean absolute error is 0.53 and the standard deviation is 0.45. The result demonstrates that the method of initial contact position estimation along the DIP phalange is valid.

Fig. 14

Ejection happens when the contact force on the DIP phalange increases. a Initial configuration of the soft finger. b Initial contact occurs. c Contact force increases. d Ejection happens

Fig. 15

Experimental results of the instantaneous stiffness analysis for the DIP phalange. a $K_{c}$ curves at two different contact positions along the DIP phalange. b Analytical $K_{c}$ and estimated values for the two different contact positions along the DIP phalange

6.3 Application to Practical Grasp

The effectiveness of the proposed proprioceptive-sensing-based approach has been demonstrated in the experiments described above. In this section, we applied this method to the practical grasp of the soft bionic hand. Different daily objects were used for grasp. Three groups of experiments were carried out for contact state estimation when contact occurred. The first experiment refers to the evaluation of whether the contact occurs or not. The second experiment refers to the evaluation of the initial contact position estimation for each phalange. The third experiment refers to the evaluation of the initial contact position estimation for the DIP phalange. 40 grasps with different daily objects were performed in each experiment. Figure 16 shows part of the grasp sequences of the soft hand grasping different objects. Figure 17 shows the success rate of contact state estimation for each group of experiments. The proposed method is able to evaluate whether the contact occurs or not with a high success rate, and compute a fairly precise estimation of the contact position. It demonstrates the efficiency of our proposal.

Fig. 16

Part of grasp sequences of the soft hand grasping different objects

Fig. 17

Success rate of the three groups of experiments

Meanwhile, there are also some practical limitations that affect the accuracy of the contact position estimation. First, the friction of the tendon transmission mechanism is not considered in this paper, so a deviation from the theoretical model is inevitable. Second, the simplified soft finger for kinetostatic modeling may have some influence on the stiffness model.

6.4 Discussion

When the soft finger performs a full closing without contacting with an object, recorded actuation torque is generated by the friction of the mechanism, the efficiency of the motor, and the resistance of the torsional springs. Actuated efficiency is evaluated through the comparison of experimental and theoretical analysis, to minimize the impact of the mechanism and motor transmission. Hereafter, actuation torque is only opposed by the torsional springs. In practice, the soft finger is actuated to form a reasonable grasping configuration to ensure enough contact space with the object. The range of actuation torque $T_{a}$ is about 15 Nmm ~ 22.5 Nmm. Too small actuation torque makes the soft finger still in the stage of preload-overcoming. Excessive actuation torque makes the finger completely closed. In addition, the experiments were carried out at a low speed of the soft finger, to minimize the dynamic effects.

Although this paper has made progress in the initial contact state estimation based on proprioceptive sensing for soft fingers, some practical factors still limit the further application of this method. First, though the method can determine which phalange is in contact and estimate the typical contact positions on the DIP phalange accurately, it is better to carry out further research in this field to make the stiffness model of the soft finger more accurate along each phalange. Second, for irregular objects, especially those with sharp corners, they may come in contact with the bare tendon in practical grasping experiments. It will affect the accuracy of the initial contact stiffness calculation for the soft finger. The structure of the soft phalanges should be further optimized to improve the robustness of the method. Third, the proposed method can estimate the initial contact position of the soft finger in a single grasp. When the soft finger continues to rotate, subsequent contacts could also cause stiffness variations. So the situation of multi-point contact becomes another modeling problem, which could be our next research focus.

In addition, different grasping configurations of the soft bionic hand could affect the grasp stability, which mainly involves the contact state between fingers and objects. In this paper, our proposed method has achieved contact state estimation when grasping objects, especially in unknown environments or without visual/tactile feedback, which provides a prerequisite for achieving grasp stability. On this basis, the planning approach of grasping configurations for soft fingers should be further modeled and analyzed.

7 Conclusion

This paper aims to realize contact state estimation of soft robotic fingers based on proprioceptive sensing. The system stiffness model of the soft finger during preshaping and initial contact with an object is proposed. It can perceive the initial contact state and validate whether the contact occurs or not effectively. Moreover, the proposed instantaneous stiffness model of the soft finger when initial contact occurs can determine which phalange is in contact. Particularly, for pinching mode, it can further estimate the initial contact position along the DIP phalange, which is useful for stable grasp. The experimental results demonstrate that the proposed systematic solution is effective, without integrating complicated and fragile tactile sensors on the surface of the soft finger. It contributes to simplifying the soft finger mechanism and making it concise and practical.