Performance Evaluation of a Surgical Telerobotic System Using Kinematic Indices of the Master Hand-Controller

Abstract

This paper investigates how kinematics of the master hand-controller, in a teleoperated system, is related to performance of the entire system. Experimental validations are presented on a surgical robotic system by emulating a part of microsurgery procedure in a laboratory setting. Isotropy index is chosen as a quantitative kinematic tool. The performance of the system is evaluated using four measures: rates of slave and master actuators efforts as well as distances travelled by the slave end-effector and the haptic implement. Results indicate that when the haptic device moves within regions with higher isotropy index, the performance improves. In order to further enhance the performance, the master site is augmented by a clutch that is found helpful to increase the hand-controller dexterity.

An erratum to this chapter can be found at http://dx.doi.org/10.1007/978-3-662-44196-1_50

1 Introduction

In robot-assisted minimally invasive surgery (RMIS), enhanced surgeon dexterity may lead to improved surgical performance [1]. In a teleoperated RMIS, dexterity is a factor that limits the use of the master hand-controller (HC) [2]. A HC with high dexterity allows the surgeon to have better control of the master HC [3, 4]. The first step to improve the dexterity is to find a tool by which the surgeon can recognize movement within a reasonable region of dexterity. The tool, employed in this study, is manipulability of the HC. Manipulability describes the degree to which a manipulator can freely apply forces and torques, or move in arbitrary directions [5]. Manipulability is a quantitative tool to determine the ease of arbitrarily changing kinematic and/or dynamic properties of the HC implement. With the manipulability measure of HC, an intuitive visualization of movement directions or applying forces is available. Manipulability measures are considered as useful tools to analyze, design, and control a robotic manipulator [6]. The measure is also a useful tool to visualize how the HC can contribute to a task execution by considering its configuration. Some of manipulability measures, previously defined, are Yoshikawas measure of manipulability [6], condition number [6], isotropy index [7], and eccentricity measure of manipulability [8]. A manipulability measure does not give exact numerical information about velocity or force values, but rather, it suggests directions along which the HC can access or apply forces [9]. The manipulability measures are highly related to the structure and configuration of the HC [10]. For example, when the HC is located in a singular configuration, the surgeon’s hand movement, in some directions, becomes difficult; while in other directions is optimal. Conversely, the HC may be placed in a configuration such that the surgeon has an identical accessibility in all directions, i.e. the isotropy index, for example, approaches $1$.

A few studies have examined the enhancement of robotic surgery by increasing surgeon’s dexterity through evaluating the dexterity of surgical tools or the slave manipulator [2, 11, 12]. The authors are unaware of any published work on investigating the effect of the master HC manipulability on performance of the teleoperated system used for conducting robotic surgery. Therefore, employing the concept of manipulability, for the HC, should assist surgeons move the HC within the working space with better dexterity. Indeed, by measuring the HC manipulability of HC, the surgeon can change the configuration of the HC in order to locate it in a better position in terms of dexterity.

In order to investigate the correlation between the HC manipulability and the teleoperated system performance, the manipulability velocity ellipsoid (MVE) is first explained. The MVE is the ease of arbitrarily changing the position and orientation of the tool located at the HC implement [13]. In fact, MVE is the geometric interpretation of the eigenvectors of Jacobian matrix which are obtained by using the singular value decomposition (SVD) technique [14]. Using SVD, the isotropy index of the HC is calculated. At the same time, the performance of the system is evaluated using four performance measures: (i) rate of slave manipulator effort, (ii), rate of master HC effort, (iii) distance travelled by the slave end-effector, and (iv) distance travelled by the haptic implement. There are two trajectories along which each performance measure and the isotropy index are computed. These values are then used to investigate the correlation between the HC manipulability and the task performance. In order to further enhance the system performance, a clutch is added to the master site. When the operator realizes that the HC is moving within the region with poor manipulability, she/he disengages the clutch, moves the HC implement to a region with better manipulability measures, and engages the clutch again to continue the surgical task.

The rest of this paper is organized as follows. The concept of manipulability and the manipulability velocity ellipsoid are described in Sect. 2, followed by defining the isotropy index. The experimental test rig is described in Sect. 3. Experimental validations are presented in Sect. 4. Section 5 outlines the conclusion of this paper.

2 Kinematic Measures

Let ${P^i=[x^i y^i z^i \phi ^i_x \phi ^i_y \phi ^i_z]^T}$ denotes posture (position and orientation) of the HC implement with respect to the reference frame, ${\{x_R, y_R, z_R\}}$. In each point of the HC’s workspace, there are some limiting factors affecting the HC motion such as configuration and velocity/torque limitations. By increasing velocities or accelerations of the implement along a given trajectory or locating the manipulator in an improper configuration (e.g. close to singular configuration), effects of HC kinematics on dexterity increase, which may violate the workability of the HC. In this case, the quantitative tools can be used to design a control mechanism that can prevent the operator from moving along an improper trajectory.

Consider the HC’s degree of freedom (DOF) is denoted by $n$, the implement’s DOF in Cartesian space is shown by $s$. Assuming the HC is non-redundant, i.e. $n$=$s$, we have:

$$\begin{aligned} \dot{P^i}=J(q^i)\dot{Q^i} \end{aligned}$$

(1)

where $J(q^i)$ is the Jacobian matrix of the HC. $Q^i=[q^i_1 ... q^i_n]^T\in R^n$ and $\dot{Q^i}=[\dot{q}^i_1 ... \dot{q}^i_n]^T \in R^n$ represent the vector of joint angular/linear displacement and velocity, respectively. Note that $Q^i$ describes the relative displacement between two adjacent links. The HC implement linear/angular velocity is defined by $\dot{P}^i=(\dot{x}^i, \dot{y}^i, \dot{z}^i, \dot{\phi }^i_x, \dot{\phi }^i_y, \dot{\phi }^i_z)^T\in R^s$, when $s=6$. Note that if $|J(q^i)|\not =0$, the HC would be in non-singular condition. In this case, the system can kinematically be evaluated. Hereafter, $J(q^i)$ will be written as $J$ for notational compactness.

In order to correlate the performance of the teleoperated system to the manipulability concept, an index is required to quantitatively measure the dexterity of the HC. To define this index, we employ the concept of the manipulability velocity ellipsoid (MVE). The MVE helps us visualize directions of the velocity at the HC implement. Definition of MVE will later be used to calculate the isotropy index. Using MVE, we are able to define some tools such as determinant, eigenvalues, and eigenvectors in order to analyze properties of the ellipsoid. In this study, the generalizations are captured by the singular value decomposition (SVD) of the mapping matrix in the manipulability ellipsoid [14].

In Eq. (1), if we map the unit circle $\{\dot{Q}^i\in R^n~|~\Vert \dot{Q}^i\Vert =\sum \limits _{w=1}^n(\dot{q}^i_w)^2=1\}$ to the space of implement velocity, $\dot{P}^i$, the MVE is defined by $\{\dot{P}^i\in R^s:\dot{P}^i=J\dot{Q}^i~|~\Vert \dot{Q}^i\Vert =1\}$. As seen, the mapping matrix of velocity is the Jacobian matrix. Using SVD, three sub-matrices of $J$ are derived as follows:

$$\begin{aligned} J=U\varSigma V^T \end{aligned}$$

(2)

where ${U=[u_1 ... u_n] \in R^{n\times {n}}}$ and ${V=[v^T_1... v^T_n] \in R^{s \times s}}$ are orthogonal matrices. $\varSigma \in R^{n \times s}$ is a combined matrix (a diagonal matrix and a zero matrix) with non-negative real numbers, and is written as:

$$\begin{aligned} \varSigma =[diag(\sigma _1~\sigma _2~...~\sigma _n)~|~0~...~0] \end{aligned}$$

(3)

In (3), $\sigma _1>\sigma _2>\cdots >\sigma _n$ are the eigenvalues of $J J^T$. Note that the number of zeroes is equal to $n-s$. The length of each principal axis of MVE is given by eigenvectors of $J J^T$. The principal axes are also shown by $\sigma _n u_n$. The major axis of the ellipsoid, ${\sigma _1 u_1}$, corresponds to the direction of the implement that can move most easily while the least easily happens at ${\sigma _n u_n}$ direction. For example, when the ellipsoid becomes a circle (in 2D), the implement can move with uniform ease in both directions, such configuration is called isotropic configuration. Geometrically, isotropy index ($\varSigma _I$) is the ratio of the length of minor semiaxis to the length of major semiaxis of the MVE $(\varSigma _I=\frac{\sigma _n}{\sigma _1})$. This is well-behaved because it remains bounded between $0$ and $1$. The isotropy index, which is defined by the reciprocal of the condition number of $J$, expresses the uniformity of manipulability. Note that the HC implement is able to move with uniform ease in all directions when isotropy value is close to $1$ ($\varSigma _I \rightarrow 1$). For a 2-DOF planar robotic arm, for example, Fig. 1 shows how the Jacobian matrix maps the unit circle in joint space to the MVE in Cartesian space.

Fig. 1.

Mapping the joint space into Cartesian space using MVE concept.

3 Description of the System

Figure 2 illustrates the developed experimental setup which composed of a KR6 Kuka manipulator (slave) and a PHANToM Desktop haptic device (master). The master and slave sites are connected through a Local Area Network (LAN) in which the time delay and the packet loss are negligible. This teleoperated system, which is part of neuroArm project [4], is designed to investigate effects of employing different control strategies, hand-controllers, and surgical tools on task performance. As observed in Fig. 2b, a bipolar forceps is attached to the slave end-effector to conduct the experiment. A 6-DOF force/torque sensor is mounted between the bipolar forceps and the manipulator end-effector to measure interaction forces. In this setup, the interaction forces are then transferred to the operator’s hand using the master haptic device. In addition to the force feedback, the operator utilizes a real-time 3D visual feedback provided by two HD microscope cameras attached to a surgical microscope. Two sets of experiments are presented in which an operator was asked to pick up a piece of cotton strips ($20\,\mathrm{mm}\times 12\,\mathrm{mm}\times 2\,\mathrm{mm}$) and place it to another cube (see Fig. 2). In practice, cotton strips are often placed between brain pathology and adjacent brain as an aid to dissection.

Fig. 2.

Teleoperated surgical robotic system: (a) Kuka manipulator and microscope, and (b) roll and actuation mechanism for bipolar forceps. Master implement velocity is mapped into velocity of slave end-effector. Position of end-effector is then computed using integration techniques.

The PHANToM Desktop haptic device is a linkage-based haptic device. The coordinate systems of the Desktop haptic device is shown in Fig. 3. With reference to Fig. 3, the Jacobian matrix, which will be used to measure the isotropy index, is defined as:

$$\begin{aligned} J= \begin{bmatrix} l_1s_1c_2+l_2s_{23}&l_2s_2c_2+l_2c_{23}&l_2c_1c_{23}&0&0&0\\ l_1c_1c_2+l_2s_{23}&l_2s_1c_{23}+l_1s_1s_2&l_2s_1c_{23}&0&0&0\\ 0&l_1c_2+l_2s_{23}&l_2(s_2c_3+c_2s_3)&0&0&0\\ 0&s_1&s_1&c_1s_{23}&J_{45}&J_{46}\\ 0&-c_1&-c_1&s_1s_2c_3+c_2s_3&J_{55}&J_{56}\\ 1&0&0&-c_{23}&-s_4s_{23}&J_{66}\\ \end{bmatrix} \end{aligned}$$

(4)

Fig. 3.

Generalized coordinate systems attached to PHANToM Desktop haptic device.

where $J_{45}=c_1s_4c_2c_3+c_1s_4s_2s_3+s_1c_4$, $J_{46}=c_1c_4c_5c_{23}-c_1s_5s_{23}+c_5s_1s_4$, $J_{55}=s_1s_4s_2s_3-s_1s_4c_2c_3-c_1c_4$, $J_{56}=c_4c_5c_{23}-s_1s_5s_{23}-c_1c_4c_5$, and $J_{66}=c_4c_5s_{23}-s_5c_{23}$.

4 Experimental Results

There were a total number of $2$(paths) $\times$ $11$(trials) = $22$ trials collected. Each path represents particular mean values of isotropy indices (0.419 and 0.322). An operator was asked to hold the stylus of the haptic device in a pen holding fashion (similar to holding of a surgical tool) and complete the task along two paths (see Fig. 4a). Each path represented different value of isotropy index. The experiments were repeated 11 times by the operator along each trajectory.

Four performance indices were used to investigate performance of the system: slave actuators effort per second ($\hat{e}_s$), master actuators effort per second ($\hat{e}_m$), distance travelled by the slave end-effector ($d_s$), and distance travelled by the haptic implement ($d_m$). After comparing the performance along both Path I and Path II, a new measure was defined, HC measure: the ratio of HC isotropy index and rate of HC effort ($\varSigma _{HC}=\frac{\varSigma _I}{\hat{e}_m}$). As observed, the HC measure includes the properties of both operator’s effort and HC isotropy index. Figure 4b illustrates a typical trajectory of the HC implement that was randomly chosen from trials completed along first path. The angular displacements of the HC actuator, for this typical task, are also depicted in Fig. 4c. As observed, although the operator was asked to move along straight lines, some deviations from trajectories were unavoidable.

Fig. 4.

(a) Desired task trajectories; (b) a typical experimental result along Path I, and (c) corresponding angular displacements of HC’s joints. Both Path I and Path II have the same length

Fig. 5.

(a) Isotropy index ($\varSigma _I$) and (b) HC measure ($\varSigma _{HC}$) for typical test shown in Fig. 4. $\varSigma _{HC}$ is a function of $\varSigma _I$ and $\hat{e}_m$.

Figure 5 shows the isotropy index $(\varSigma _I)$ and the HC measure $(\varSigma _{HC})$ that were obtained for the trial depicted in Fig. 4b. Table 1 lists a summary of information obtained from all 22 trial runs. Each parameter, shown in Table 1 is obtained by averaging the mean value of that parameter in each trial. For example, in order to obtain $\varSigma _I$, the mean value of isotropy indices in each trial is calculated. Afterwards, the average of all 11 mean values is computed. Results, presented in Table 1, indicate that Path I exhibited higher values of isotropy index and HC measure as compared to Path II. Although the distances travelled by the HC implement ($10.21\,\mathrm{cm}$ and $10.11\,\mathrm{cm}$) and the slave end-effector ($13.21\,\mathrm{cm}$ and $14.09\,\mathrm{cm}$) were almost same, the slave and master efforts over time was less in Path I. Therefore, the operator could move the HC more easily along Path I than Path II. The values of the newly-defined HC measure confirm this result ($925\frac{sec}{rad}$ in Path I as opposed to $708\frac{sec}{rad}$ in Path II). The proposed HC measure has benefits of both isotropy index and operator’s effort. The HC measure not only investigates the smoothness of the operator’s motion, but also incorporates the role of the HC configuration. Therefore, using a single measure, effects of both operator and HC are investigated.

Table 1. Summary of quantitative performance measures, isotropy index, and HC measure.

One way to improve the surgeon’s dexterity, during robot-assisted surgery, is to find a technique by which the surgeon’s hand is always kept within an acceptable HC isotropy index. The technique, proposed in this study, was to augment the master workstation by a clutch. Using clutch, when the surgeon feels that the HC falls within low isotropy index, the clutch is disengaged. The clutch stops sending information to the slave site; therefore, the slave motors do not actuate and stays fixed. Then, the surgeon can move the HC towards regions within higher index, and reengage the clutch to continue surgery. In order to investigate effects of the clutch on HC manipulability, three experiments were conducted, along Path II. Table 2 shows how the operator’s performance is improved by employing the clutch. The operator engaged/disengaged the clutch for $2$, $5$, or $6$ times during each trial in with-clutch mode. Results showed that by using the clutch, mean values of the isotropy index and the HC measure increased as compared to without-clutch mode. However, no conclusion could be established on the correlation between the number of clutches pressed and the value of the isotropy index.

Table 2. Effect of adding clutch on isotropy index and HC measure.

5 Conclusions

This paper discussed the correlation between kinematics of the master HC and performance of the teleoperated system. The kinematic index was the isotropy index. Experimental validations were provided utilizing a setup comprising a Kuka manipulator, a PHANToM Desktop haptic device, and a test bed that allows us to emulate a microneurosurgical task. An operator was asked to move a piece of cotton strips along two paths. Each trajectory indicated a particular range of isotropy measure. For each trial, the isotropy index and four performance measures were calculated. Results indicated that when the HC moved along a trajectory with higher isotropy measure, the performance of the system was improved. This shows that the surgeon’s ability in moving the HC can effectively change the system performance. Finally, a new measure was proposed by which the isotropy index and the HC effort were evaluated by a single measure. The new measure is more useful when the aim is at comparing the performance of different HCs. The results suggest that the proposed measure can be a potential tool for evaluating the performance of teleoperated systems.