Keywords

1 Introduction

Minimally invasive surgery (MIS) provides patients with improved outcomes compared to open surgery [1]. Robot-assisted minimally invasive surgery (RMIS) improves on the benefits of MIS by providing surgeons with augmented dexterity [2], flatter learning curves [3], the ability to scale inputs (force, position, and/or velocity), and superior integration with pre- and intra-operative tools such as path-planning software and imaging techniques [4]. The primary criticism for RMIS is that it may decouple surgeons from their sense of touch, which they routinely rely on to assess tissue condition and/or properties during procedures [5]. Many groups hypothesize that robotic telesurgery with force feedback will address this concern by restoring the sense of touch [6], which could ease surgeon workload and limit complications. However, there is very little conclusive data in the literature to support or refute this belief. While the inclusion of force feedback provides supplemental information regarding the interaction between patient and tool-tip, it does not necessarily provide an accurate gauge as to the amount of force applied to patient structures. Previously, we investigated the relationship between force feedback and applied force during a simple tool interaction task. We further incorporated a novel system of notifications during telesurgery, and found that operators were better able to avoid unintentional tissue punctures via a telesurgical apparatus both with notifications and force feedback. However, the use of force feedback significantly increased task completion times [7]. This paper expands on these results; the goal of this work is to perform an initial factor screening to determine what affects maximum force application and task completion time for telesurgery. It is hypothesized that by isolating these key factors we will enable a quantification of operator force application awareness during teleoperated tasks.

During surgery, it is important to restrict the application of forces to safe levels, particularly with respect to non-target structures. Existing real-time controls that do so during RMIS are mostly limited to virtual fixtures and system-wide force/position/velocity clipping. In a microneurosurgical context, the former is inadequate for procedures that cannot physically isolate target anatomy from delicate structures such as cranial nerves. Notifications may solve this issue by allowing operators to interact with delicate structures while improving their awareness of safe force application levels.

The use of notifications to alert operators of impending negative events during focus-intensive tasks is not new. Several groups have explored the utility of single-channel and multimodal notifications – primarily auditory and vibrotactile – to warn drivers of imminent collisions [8]. Another study used a combination of force feedback, visual feedback, and auditory warnings to help construction robot teleoperators improve performance with regards to grasping force [9], which is particularly important for telesurgical tasks such as suturing. There is no consensus as to which notification modality is preferred during these tasks, nor whether single-channel or multimodal notifications produce superior operator performance. Furthermore, it is unclear whether or not results gathered from vehicle or machinery operation tasks could be generalized to the operation of a telesurgical robot by a highly specialized professional. For this reason, we analyze the effects of factors such as notification modality, force margin, force level, and force feedback on the force applied via a telesurgical apparatus.

2 Experimental Design

Experiments utilize a custom prototype 7 degree of freedom (DOF) telesurgical system whose movement is restricted solely to the Z-axis to penetrate layers of translucent plastic fitted to a custom mount. The synthetic membranes are not comparable to real tissue, but rather allow for repeatable tool-tip interactions due to their nearly-static contact geometry. An average puncture force \(F_{p.ave}\) for each thickness of synthetic membrane is established prior to trials, which is then used in conjunction with varied force margins for activation of notifications during teleoperation.

2.1 Apparatus

A SensAble Phantom Desktop 6 DOF haptic interface is used as the master, controlling a Kuka KR-6 slave robot for all experiments. A custom tool coupled with an ATI Gamma 6 DOF force/torque sensor attaches to the robot’s end-effector. The custom tool-tip is machined to a dull point from a 28 mm length of solid 4.5 mm diameter cylindrical aluminium that allows for repeatable interactions with the synthetic membrane due to its symmetrical tip profile. The Desktop reproduces forces from the Gamma scaled by a factor of 0.2 when its amplifiers are activated.

Fig. 1.
figure 1

System diagram

Full size image

An HP Compaq 6200 Pro with a 3.4 GHz Intel Core i7 processor running 64-bit Windows 7 Professional SP1 with 4 GB of RAM processes information for the master system, connecting the Desktop, an external speaker, and a 10 mm \(\times \) 3.4 mm Polulu Shaftless Vibration Motor controlled via a Quanser Q2-USB data acquisition board (DAQ). The slave system uses a custom PC with a 3.3 GHz Intel CORE i5 processor running 32-bit Windows 7 Professional SP1 with 4 GB of RAM. The slave PC connects to the Kuka workstation and another Quanser Q2-USB DAQ, which interfaces with the force sensor and its accompanying hardware (a National Instruments DAQ and signal conditioning box provided by ATI). Matlab/Simulink R2011a with Quanser QUARC 2.2 blocks provides a real-time interface between hardware and the master and slave PCs respectively. Master and slave PCs communicate over a LAN via the TCP/IP protocol.

Also, a Leica M525 OH4 surgical microscope coupled with two Ikegami HDL 20D microscope camera systems, a Sony LMD2451MD LCD monitor, and RealD 3D glasses provides a magnified real-time 3D video feed of the tool-tip’s interaction with the synthetic membrane. Figure 1 shows an overview of the experimental apparatus.

2.2 Design of Experiment

A simple 1DOF tool-tip interaction task is used for this experiment: participants (operators) are asked to apply a slow, steady, downwards force to a synthetic membrane via the telesurgical apparatus until they feel they’ve reached a maximum, then retract the tool to the starting position. Operators are informed that a puncture of the membrane is classified as a failed trial, whereupon the puncture conditions are stored and the trial is repeated. Experimenters instruct operators to observe the real-time 3D video feed from the surgical microscope closely for visual deformation cues to help gauge applied force. Operator performance is quantified based on the maximum force applied during each trial (\(F_{max}\)) and the task completion time (\(t_{c}\)), as bounded by the last zero crossing of force data before \(F_{max}\) and the first zero crossing after \(F_{max}\) (i.e. the task completion time is the period during which the tool-tip contacts the membrane). The task is performed with both single and double layers of the synthetic membrane (\(f_{Level.1}\) and \(f_{Level.2}\) respectively), both with and without force feedback (FF), and with a variety of notification modalities: auditory (audio, \(a\)), vibrotactile (haptic, \(h\)), auditory and vibrotactile (\(ah\)), or none at all (visual, \(v\)). When notifications are employed, they are triggered when an operator applies a force in excess of a predetermined force margin, defined as either 30 % or 70 % of the average puncture force \(F_{p.ave}\) (\(f_{Margin.1}\) and \(f_{Margin.2}\) respectively).

Thus the notification feedback modality (audio, haptic), the inclusion of force feedback, the percentage of average puncture force at which notifications are applied (force margin), and the amount of average puncture force required (force level) are screened as primary factors affecting two response variables, \(F_{max}\) and \(t_{c}\). These five factors were input into Minitab using a 2\(^{5-1}_{V}\) design with five replicates to produce the randomized trials for the experiment, which were then exported to Matlab via Excel and used to set experiment variables in Simulink for each individual trial. Table 1 summarizes the five main factors and the low/high levels defined for each.

Table 1. Main factors and associated levels
Full size table

2.3 Initialization, Calibration, and Training

The entire system undergoes a full initialization and recalibration process between operators to ensure the consistency of results. First, the slave robot initializes to the same pre-set home position. Second, an artificial membrane is fitted to a custom mount centred below the slave’s tool-tip and clamped in position such that it is perpendicular to the tool-tip’s linearly-restricted axis of motion. Third, the force sensor data at a negligible tool-tip velocity is zeroed to account for any bias. The tool-tip advances in the negative direction along the Z-axis until a maximum force is achieved or the membrane is punctured, and then the process is repeated for the next trial.

From 10–30 calibration trials are run by experimenters between each operator until \(F_{p.ave}\) can be determined with a variance less than or equal to 0.1 N for \(f_{Level.1}\) or 1 N for \(f_{Level.2}\). Simulink multiplies this \(F_{p.ave}\) for each operator by the force margin for each trial to set the threshold at which notifications are generated. When audio notifications are activated, the speaker emits a 60 dB, 0.05 sec ‘beep’ repeated at a frequency of 5 Hz. The vibrating motor, attached to the inside of the operator’s non-dominant wrist, reproduces the same signal at an amplitude of 0.75 g when activated. The physical locations of both notification sources remain constant throughout all experiments.

Each operator performs two sets of four training trials prior to beginning the experiment: one with no notifications, one with an audio notification, one with a haptic notification, one with both audio and haptic notifications, and then all four repeated with force feedback. Given the 2\(^{5-1}_{V}\) experiment design with five puncture-free replicates per individual combination of factors, each operator performs 80 successful trials in random order and response variables are stored for both successful and unsuccessful trials. Experimenters inform operators which feedback modalities to expect prior to the commencement of each trial. Nine operators with varied levels of teleoperation experience in neuroArm’s [4] Surgical Performance Laboratory perform the full experiment.

3 Results

The overall average puncture force from across all ten calibration trials was 11.4 N for \(f_{Level.1}\) and 21.8 N for \(f_{Level.2}\). Thus for a trial using a 30 % force margin with a single layer of the synthetic membrane, notifications are triggered when an operator applies 3.4 N to the membrane via the slave. As the average puncture force \(F_{p.ave}\) varies between operators, the \(F_{max}\) for each trial is transformed into a percentage of the given operator’s average puncture force, \(F_{max.n}\), to allow for inter-operator comparisons:

$$\begin{aligned} F_{max.n}= \frac{F_{max}}{F_{p.ave}} \end{aligned}$$
(1)

As task completion times also vary widely between operators, a normalized task completion time \(t_{n}\) is similarly calculated using the maximum task completion time across all trials for a given operator. The factor combinations, original response variables, and transformed response variables for all ten operators are imported back into Minitab for statistical analysis using a General Linear Model ANOVA that accounts for all main factor effects and two-factor interactions with \(\alpha = 0.05\). It is assumed that higher order interactions are negligible.

Relevant results from Minitab are reproduced in Table 2, where DF is the number of degrees of freedom, F is the F-distribution value, and P is the P-value. Here we see that force feedback, force margin, and operator are all factors that produce statistically significant effects on the normalized maximum applied force. It is difficult to ascertain exactly what their effects are, however, given that all three are also subject to interaction effects. It seems intuitive that the force margin would impact the applied force, as a heightened awareness of any boundary could logically alter an operator’s behaviour. That individual operators might apply different force magnitudes is also unsurprising, but will require follow-up experiments with a much larger number of participants in order to confirm. Similarly, the existence of effects from the way an operator handles force feedback or from the interplay between force feedback and the amount of puncture force required are interesting to note, but will require further investigation to clarify.

Table 2. Analysis of variance results
Full size table

Table 2 illustrates an even more complex array of factor interactions that produce statistically significant effects on the normalized task completion time. The force level is the only factor providing a main effect without interactions. If an operator is applying force slowly and smoothly, it seems logical that it would take more time to apply more force. Again it is unsurprising that different operators react differently to audio notifications, force feedback, and the force margin, though operator effects will likely diminish with a larger number of participants. It also seems intuitive that there is an interplay between notification modality and force margin, as one might expect an increased awareness of notifications to be accompanied by a heightened sensitivity to notification modality. What is somewhat unexpected given that most participants reported no perceived difference between responses at different notification modalities, is that there is a statistically significant interaction between audio and haptic notification modalities. It would be appropriate to perform follow-up experiments in a surgical setting where participants are constantly bombarded by an overload of sensory information in order to confirm this interaction.

Table 3. Overall operator reliance and puncture results
Full size table

As the experiment progressed, a noticeable reliance on either notifications or visual feedback emerged, as measured by the proximity of \(F_{max}\) to either the force threshold at which a notification occurred or the average puncture force. This is quantified using two new variables, \(R_N\) for a normalized numerical representation of an operator’s reliance on notifications and \(R_V\) for the visual equivalent. Note that in Eqs. 2 and 3, \(f_{M}\) is the force margin and \(F_{p.ave}\) is specific to the number of synthetic membranes for the given trial:

$$\begin{aligned} R_{N} = \frac{|F_{p.ave}\cdot f_{M}- F_{max}|}{F_{p.ave}\cdot f_{M}} \end{aligned}$$
(2)
$$\begin{aligned} R_{V} = \frac{|F_{p.ave}-F_{max}|}{F_{p.ave}} \end{aligned}$$
(3)

By calculating individual values of \(R_{N}\) and \(R_{V}\) for each operator’s 80 trials, we can determine a percentage representation of how much an operator relies on notifications or visual feedback overall. Table 3 shows this overall reliance per operator along with the total number of punctures they produced, listed according to the type of notification (visual - no notification, audio, haptic, audio and haptic) and feedback (no force feedback, force feedback) received. It is immediately apparent that operators 6, 7, and 8, who relied heavily on visual cues to judge force application, produced substantially more punctures than their notification-reliant counterparts.

This phenomenon is even more apparent in Table 4, which averages raw task completion times and column-wise puncture results within each of the two groups: notification-reliant operators (\(N\)) and visual feedback-reliant operators (\(V\)). Though larger sample sizes are required, these preliminary results imply that a reliance on visual feedback alone increases both an operator’s task completion time and the number of times they apply excessive force, regardless of notification modality or force feedback inclusion.

Table 4. Average task completion time and puncture results based on operator reliance
Full size table

4 Conclusion

Factor screening identifies multiple significant interaction effects. Preliminary results show that operators who rely on visual feedback more than on notifications apply higher forces more often on average during simple teleoperated tasks and take longer to complete them. This must be confirmed in a larger pool of participants, but may imply that properly optimized notifications could help surgeons consistently decrease both their task completion times and the frequency with which they apply excessive forces via RMIS. Future extensions of this work should focus on continued factor characterization, factor optimization, and the confirmation of results using natural tissues instead of synthetic membranes.