When using a minimally invasive robotic system, such as the Intuitive Surgical da Vinci [1, 2], a surgeon controls the movement of robotic, surgical instruments by manipulating the handles of a master console located across the room from a patient. The surgeon can visualize the procedure through a stereoscopic video feed, but he or she cannot feel what the robotic instruments touch. Any physical connections between the surgeon’s hands and the robotic instruments are removed, and haptic sensations such as the tension of a suture, the texture of tissue, and even collisions between the robotic arms are physically imperceptible with these robotic systems. Providing such haptic feedback is difficult because sensing and displaying physical interactions between the robotic tools and the patient are challenging technical problems [3]. Despite the important role that tactile information plays during direct manipulation [4] and laparoscopic surgery [5], the current commercially available surgical robot controllers do not provide significant haptic feedback.

Robotic minimally invasive surgery (RMIS) with the da Vinci robot has become a progressively more popular option for many types of surgery since its FDA approval in 2000 [6]. Compared to open procedures, minimally invasive surgery is generally believed to reduce tissue damage, patient discomfort, and hospital stay duration, enabling faster recovery, reduced morbidity, reduced pain, and improved cosmesis [7, 8]. Proponents of RMIS also laud the robot for providing capabilities that are not available in traditional laparoscopic surgery, such as a natural hand-tool motion mapping, tremor filtration, motion scaling, high-resolution stereoscopic vision, and greater instrument dexterity [9]. However, RMIS can be slower and more expensive than non-robotic surgery, leading some to believe it “offers no clear, significant advantage over standard laparoscopic techniques” [10]. The controversy over robotic surgery and its continuing expansion make the robot’s lack of haptic feedback an increasingly interesting topic for study and technology innovation.

Many experienced robotic surgeons say that they learn to feel with their eyes, discerning haptic information, such as forces, from visual information [11], but some researchers and some surgeons believe that RMIS is still limited by its lack of haptic feedback [9]. One recent study used video analysis of endoscopic procedures to show that the absence of haptic feedback caused tissue slippage and tissue damage in robotic surgery [7]. Another experiment showed that surgeons exerted significantly less consistent suture tensions when tying knots with a da Vinci robot than when tying knots by hand or with handheld instruments, indicating that the robot’s lack of haptic feedback may increase the incidence of excessive forces [12]. Such studies substantiate the hypothesis that RMIS is indeed limited by its lack of touch feedback.

In light of findings that haptic feedback may reduce operative times, tissue damage, and excessive forces [7], many researchers have tried to develop haptic feedback systems for robotic surgery. One team added force-sensing resistors to the internal surfaces of a Cadiere grasper and placed pneumatic balloon actuators on the da Vinci master handles as a way of relaying grasp force information [13]. Novices and experts using this system to do a peg transfer task applied significantly lower grasp forces when the tactile feedback was active [14], supporting the aforementioned hypothesis that haptic feedback encourages gentle handling of tissue. However, force sensors are technologically challenging to integrate with surgical instruments due to sterilization and robustness requirements [13].

Another team modified a bimanual Endovia Medical Laprotek system to provide force feedback by adding three-axis strain gage force sensors to the graspers and replacing the standard master devices with two augmented Sensable Phantom haptic interfaces [15]. The addition of three-axis kinesthetic force feedback significantly reduced the forces applied by both surgeons and non-surgeons performing a cannulation task with this system. Unfortunately, strain gages are difficult to integrate into functional surgical instruments and typically measure only lateral, not axial, tool forces [16].

Many other researchers have sought to provide haptic feedback by estimating interaction forces and torques from the robot’s position, using position sensors that are readily available in the robot. In one early example, Madhani, Niemeyer, and Salisbury implemented haptic feedback based on the joint angle readings of the Black Falcon [17], an early prototype of the da Vinci slave. While the Black Falcon was specifically designed to have low friction and low inertia, the authors found its position-based force feedback to be “more of an annoyance than a help during suturing” [17]. They attributed this dislike to the fact that “there were background forces which caused fatigue during fine motions,” indicating that the quality of the feedback greatly affected operator preference and performance [17].

Another research team subsequently developed a method of passively canceling some of the robot’s friction and successfully implemented their scheme on the tool insertion axis of a modified da Vinci robot [18]. The resulting force feedback enabled subjects to locate a stiff linear region in a heart model more accurately than without haptic feedback, but the feedback conferred no benefit for the task of locating a stiff nodule in a prostate model [19]. High levels of friction in the robot’s drive train (caused by taut cables winding around tiny pulleys) and the large inertia of the robot arm tend to obscure the small-magnitude forces that stem from instrument contact with tissue. As such, attempts to add force feedback to robotic surgery have largely been unsuccessful

Measuring and displaying forces has complications beyond the aforementioned challenges of sensor accuracy and reliability. Another major problem is the inherent trade-off between the robotic system’s stability and its transparency, which relates to the quality of the haptic feedback it provides. As explained by Okamura [3], relaying force feedback to the master handles of a surgical robot can lead to uncontrollable oscillations because the two devices form a closed-loop system; when a tool contact force is presented to the surgeon as a force on the master handle, the force causes the handle to move, which then causes the instrument to move and changes the force. The resulting oscillatory movements (instability) would be unacceptable in a surgical environment. Other challenges include the size, sterilization, biocompatibility, cost, and robustness required to integrate such sensors into fielded systems [3]. Although many studies have shown force feedback to be useful in the lab setting, fundamental technical challenges and broad-ranging implementation issues have prevented these approaches from being adopted in the operating room.

To bypass the problems of inertia and friction, several researchers have tested the idea of sensory substitution, where an estimate of the robot’s applied force is conveyed via visual and/or auditory indicators. In one specific example, experimenters used varying heights of a visually displayed bar to indicate the magnitude of applied force, and they used a single auditory tone to indicate when the force exceeded a predefined threshold [12]. Results showed that the auditory cues enabled surgeons to achieve a suture-tension consistency comparable to hand ties, and both visual cues and the combination of visual plus auditory cues yielded consistencies better than hand ties. The participating surgeons strongly preferred the visual feedback over the audio feedback due to the continuous real-time information it provided, as opposed to the audio’s discrete alerts [12].

Interestingly, another study by the same group demonstrated no improvements in palpation accuracy for a similar visual force-feedback indicator [19]. These contradictory results seem to indicate that the utility and appeal of haptic information is strongly affected by the details of implementation and/or the task performed. Some researchers also hypothesize that sensory substitution is less natural than direct haptic feedback and therefore may result in longer learning curves and increased cognitive load [20, 21]. These results suggest that sensory substitution might not be the most consistent or effective way of providing haptic information.

Given the many challenges of force feedback and the inconsistent effects of sensory substitution, a small group of researchers have explored the alternative approach of measuring and displaying vibrations as a form of haptic feedback in robotic teleoperation. Changes in contact between a robot’s end-effectors and other physical objects cause high-frequency, transient vibrations that can easily be measured. As first demonstrated in a bench-top experiment with a pair of two-fingered robotic hands [22], haptic feedback of these vibrations enabled subjects to distinguish between smooth and damaged ball bearings and perform a membrane puncturing task more gently than without vibration feedback [22]. These experiments substantiated the potential benefits of providing vibrotactile feedback to relay haptic information. After applying this idea to our own bench-top teleoperation system [23] and demonstrating its ability to transmit realistic textures [24], we adapted the approach for use on a da Vinci robot [25].

Our system, called VerroTouch, measures instrument vibrations using a high-bandwidth MEMS-based accelerometer attached to each patient-side manipulator (Fig. 1). By locating the sensors near the tool mounting point and underneath the sterile drape, we avoid the problems of sterilization, biocompatibility, and size, and we bypass the cost and complexity of modifying the tools themselves. Custom circuitry amplifies and filters the left and right vibration signals and presents them to the surgeon through both haptic and auditory channels. Haptic feedback is delivered via electromagnetic voice coil actuators (similar to audio speakers) mounted to the master control handles (Fig. 2), and auditory feedback is played through left and right speakers mounted to the sides of the console.

Fig. 1
figure 1

da Vinci robotic surgical system, with patient-side manipulators (left) and master console (right). The surgeon controls the instruments by moving the master handles. The white clips on the patient-side manipulators detect vibrations

Full size image
Fig. 2
figure 2

da Vinci master handle augmented with a custom actuator for haptic display of instrument vibrations

Full size image

In the first human-subject study of VerroTouch, eleven surgeons (six residents, two attending surgeons with no robotic experience, and three experienced robotic surgeons) used an augmented da Vinci S to perform three dry-lab tasks. They performed each of the tasks under four sensory feedback conditions of instrument vibration: auditory, haptic, auditory plus haptic, and none. While the feedback type was not found to affect the performance of subjects in this study, subjects expressed a significant positive preference for the inclusion of the feedback [25].

More recently, we demonstrated the feasibility of implementing both haptic and audio feedback of tool vibrations during live surgery: two surgeons successfully used the same augmented da Vinci robot to complete two transperitoneal nephrectomies and two mid-ureteral dissections with uretero-ureterostomy on a porcine model [26]. Retrospective video analysis revealed that 82 % of the surgeon’s actions caused a detectable tool vibration signal, with the magnitude of the vibration depending on the physical intensity of the surgeon’s action [26].

The difficulty of providing force feedback combined with the proven feasibility and indicated user preference for haptic feedback of instrument vibrations suggests that vibrotactile and audio feedback could be a successful way of restoring lost haptic information during robotic surgery. The studies reported in this article were designed to gain more information regarding user preferences and the potential benefits of this approach to haptic feedback for robotic surgery.

Materials and methods

We conducted two experiments to learn whether or not surgeons and non-surgeons prefer haptic and audio feedback of instrument vibrations during robotic surgery. Experiment 1 involved 20 people who performed dry-lab tasks with and without both types of feedback, while Experiment 2 involved approximately 100 people who briefly evaluated the system in a conference setting. All subjects in both studies gave informed consent, and the experiment protocols were approved by the Institutional Review Board of the University of Pennsylvania under protocol #816061.

Both studies used the same da Vinci Standard robot equipped with components that give the operator the option of receiving haptic and/or auditory feedback of high-frequency instrument vibrations. In both studies and both surveys, the term “haptic feedback” is used interchangeably with “vibrotactile feedback”. As described above, the VerroTouch system converts vibrations of the robotic tools to haptic vibrations on the left and right master handles and/or sounds played through speakers on the left and right sides of the console. The intensity of the haptic feedback is controlled by a dial mounted on the surgical console near the left keypad: a dial setting of 0 corresponds to no haptic feedback, and the maximum setting of 9 corresponds to very strong haptic feedback. A dial on the speakers controls the intensity of auditory feedback, which ranges from 0 % (off) to 100 % (maximum).

Experiment 1

Twenty one subjects were recruited to perform up to four dry-lab tasks using the augmented da Vinci Standard robot in the GRASP Laboratory at the University of Pennsylvania, as shown in Fig. 1.

The 21 subjects included five experienced da Vinci surgeons (having performed 20 or more RMIS procedures on human patients), six attending surgeons with no case experience in RMIS, and ten non-surgeons. One of the experienced da Vinci surgeons was female, one of the other attending surgeons was female, three of the general population subjects were female, and the rest were male. Subjects were required to have the following: 18 or more years of age, normal sensory and motor function of their arms and hands, normal or corrected-to-normal close-range vision, and normal or corrected-to-normal close-range hearing. After completing the study, one male attending surgeon with no RMIS experience was excluded from analysis for reporting a lack of normal sensory and motor function in his arms and hands, leaving a total of 20 subjects in this experiment. Each subject completed the experiment in a single session spanning from 30 to 90 min.

The subject began by watching a video that explained the four tasks that they could choose to perform with the robot. As shown in Fig. 3, the tasks included FLS peg transfer, needle pass, modified FLS circle cutting, and FLS suturing [26, 27]. The FLS peg transfer task [27] involves using two large needle drivers to pick up triangular blocks and transfer them between posts. Needle pass [26] consists of using two large needle drivers to pass a curved surgical needle with attached suture through holes in a piece of clear, rigid plastic. We modified FLS circle cutting [27] to use gauze that varies in thickness; the subject uses curved scissors and a needle driver to cut a circle out of the gauze. And lastly, FLS intracorporeal suturing [27] requires the subject to use a pair of needle drivers and a curved surgical needle with attached suture to perform various anastomoses in simulated tissue. Subjects were told they could choose the number of tasks they wanted to perform and the number of times they wanted to perform each one. Subjects were also told they could ask questions throughout the experiment and could watch the instructional video as many times as needed.

Fig. 3
figure 3

The four dry-lab tasks used in Experiment 1, as seen from the da Vinci camera

Full size image

After watching the video, the subject completed a short questionnaire documenting their age, occupation, gender, handedness, sensory motor function, and experience with a variety of activities including surgery, video games, and haptic interfaces. The subject was then instructed to read through (but not complete) the survey regarding their opinions of the haptic and auditory feedback to become familiar with the questions they would be answering later in the study. All surveys were presented electronically via a laptop or desktop computer located next to the da Vinci.

The subject then went through an introduction to the da Vinci Standard and the tool vibration feedback system. Participants unfamiliar with the da Vinci received a 3-min explanation of how to hold the master handles and operate the robot. They then took approximately 3 min to practice these basic skills. Subjects were then introduced to the haptic and audio feedback system. All subjects used the robot for approximately 5 min to interact with a wooden block covered with various textured surfaces and strings tied between rigid screws. At the start of the 5 min, both haptic and auditory feedback modalities were off. After 1 min of interacting with the block, the subject was instructed to turn on the haptic feedback and gradually increase the intensity in increments of 2, using the dial mounted near the master console’s left keypad. Once the haptic feedback reached an intensity level of 8 (one less than the highest setting of 9), the subject was instructed to adjust the volume of the audio feedback in a similar manner.

Subjects then choose to perform up to four of the available tasks, according to their personal interest. They performed these tasks in any order they preferred. To balance the presentation order of the feedback, each subject was assigned to start each task either with or without tool vibration feedback. Subjects were allowed to perform each task as many times as necessary to determine the levels of haptic and auditory feedback that they preferred for that task. For example, subjects would sometimes cut a gauze circle multiple times (cutting subsequently larger diameters) to experience the task again. During each task, subjects were allowed to change the haptic and audio levels as often as they liked; the only requirement was that they experienced the task at least once with haptic and audio feedback off and once with haptic and audio feedback on. After completing each task, the subject reported his or her preferred level of haptic feedback (0–9) and preferred level of audio feedback (0–100) for that task. Subjects repeated this process for up to four tasks.

Throughout this study, the surgeon’s camera view and the left and right instrument vibrations were recorded onto a DVD using the methods presented in [28]. After completing the selected tasks, the subject filled out a brief summative survey that contained the questions listed in Table 1. Most questions were answered using a sliding bar indicator from “strongly disagree” to “strongly agree”, corresponding to numerical values from 0 to 100.

Table 1 Questionnaire for Experiment 1
Full size table

Experiment 2

Ninety four subjects were recruited to briefly evaluate haptic and audio feedback of instrument vibrations on the da Vinci robotic surgery system. This experiment was conducted in the Learning Center at the 2013 Annual Meeting of the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) in Baltimore, MD. Prospective participants included any booth visitors who were willing to test the previously described da Vinci robot and fill out a questionnaire. Subjects consisted of 38 attending surgeons, 13 fellows, 17 residents, 8 medical students, and 18 non-medical personnel, yielding a total of 94 participants.

Study participation occurred during regular conference hours and lasted from 7 to 20 min per person. Conference attendees walking by the booth were verbally recruited to learn about and test our method for adding haptic and audio feedback of tool vibrations to a surgical robot. Interested participants were shown pre-recorded videos of urologic robotic surgery on a porcine model [26] and bariatric robotic surgery on a human patient [28]. During these videos, participants could feel the recorded tool vibrations through actuated wands held in their hands and could hear the recorded tool vibrations through stereo speakers. This video demonstration was accompanied by a verbal explanation of RMIS, the lack of haptic feedback in current systems, and how accelerometers, haptic actuators, and speakers can be used to add haptic and audio tool vibration feedback to such a system.

People who wanted to take part in the study by trying the robot were asked to choose either circle cutting, suturing, or peg transfer. If a person was unfamiliar with operating the robot, they were shown how to sit at the console, hold the master handles, and manipulate the robot tools. They would then complete their chosen task as the experimenter changed the intensity of the haptic and audio feedback or instructed them on how to change feedback intensity. Subjects were allowed to continue the task until they felt they had experienced enough or until the next subject was ready to test the system (approximately 5 min). Subjects filled out a brief survey at the end regarding their preferences and thoughts on the addition of tool vibration feedback to robotic surgery (Table 2).

Table 2 Questionnaire for Experiment 2
Full size table

Results

The results were analyzed using statistical and graphical methods to compare preferences between participant subgroups and to summarize overall findings of the experiments.

Experiment 1

The twenty subjects in the first experiment elected to do two, three, or four of the available tasks. All twenty subjects performed peg transfer; 19 performed needle pass; 19 performed circle cutting; and 13 performed suturing. The average time for completion of the respective tasks (rounded to the nearest minute) was 9 min for peg transfer, 12 min for needle pass, 12 min for circle cutting, and 11 min for suturing.

Question 14 of Experiment 1 asked subjects to rate the strength of their preference for having either audio and/or vibrotactile feedback of tool vibrations during surgery. They recorded their preferences on a sliding scale of 0–100, corresponding to “Strongly would NOT prefer” to “Strongly would prefer.” 100 % of subjects reported numbers higher than 50, indicating that all users would prefer feedback to some degree. Figure 4 shows a box plot of the surgeons’ responses. The median level of preference for surgeons was 74 out of 100, and the inter-quartile range was 18 (66–84), a relatively small value that indicates good agreement across subjects. The highest response was 100 (strongly prefer).

Fig. 4
figure 4

Surgeons’ response to level of preference in question 14 in Experiment 1: “Would you prefer to have either audio and/or vibrotactile feedback during live surgery?” Subjects adjusted a slider ranging from 0 to 100, with 0 corresponding to Strongly would NOT prefer and 100 corresponding to Strongly would prefer. The white line in the box shows the median response, and the circle shows the mean. The box shows the inter-quartile range, and the whiskers depict the range of the data

Full size image

Subjects were also asked to indicate which type of feedback they would prefer to have during surgery (audio, haptic, neither, or both). As shown in Fig. 5, 45 % of subjects (9/20) indicated that they would prefer only haptic feedback, and 50 % of subjects (10/20) stated that they would prefer both haptic and audio feedback. 0 % of subjects stated a preference for only audio feedback, and 5 % of subjects (1/20) reported that they would prefer neither haptic nor audio feedback. The one subject who preferred no feedback was a urologist with extensive experience operating with the da Vinci.

Fig. 5
figure 5

Distribution of responses to question 15 in Experiment 1: “If you were performing surgery, which of the following would you prefer? (If you are not a surgeon: as a patient, which would you want your surgeon to have this feedback during live surgery?): Just Audio, Just Vibrotactile, Neither, Both”

Full size image

To understand the reasons behind these preferences, we examined the responses to questions 2 through 9, which asked subjects to report their level of agreement with four statements regarding the effects of the audio feedback and the haptic feedback. Responses to these questions were recorded on a sliding scale of 0–100, corresponding to “Strongly Disagree” to “Strongly Agree.” The first two questions asked the subject how they thought the audio and haptic feedback affected their awareness of instrument contacts. As shown in Fig. 6, subjects strongly agreed that haptic feedback of instrument vibrations made them more aware of the instrument contacts, with a median agreement level of 86 out of 100. Audio feedback was also thought to increase awareness, but to a lesser extent, with a median agreement level of 61, and more widely varying responses among subjects. The difference between haptic and audio means was found to be significant (paired t test, p = 0.0023).

Fig. 6
figure 6

Level of agreement with questions 2 through 9 about the effects of haptic and audio feedback of instrument vibrations. Each subject adjusted a slider that ranged from 0 to 100 for each question. The line in the middle of the box shows the median response, and the circle shows the mean. The box shows the inter-quartile range, and the whiskers depict the range of the data

Full size image

Another pair of questions centered on whether the subject thought the feedback made them more careful with the robotic instruments. In a similar pattern, subjects agreed that both types of feedback made them more careful, with the haptic feedback exerting a stronger and slightly more consistent influence than the audio. The median level of agreement for haptic feedback inducing carefulness was 70 out of 100, while it was 50 for audio; this difference is significant (paired t test, p = 0.0311). Compared to the questions about awareness, subjects disagreed more about the feedback’s effects on carefulness, with the range of responses spanning or almost spanning the full range from 0 to 100.

The other two pairs of questions focused on possible negative effects of the provided sensory feedback. Questions 4 and 5 asked the subject the extent to which they thought the two forms of feedback interfered with their ability to use the surgical robot. Overall subjects disagreed with the statement that the haptic feedback interfered, reporting a median agreement level of 20, with a relatively small inter-quartile range of 17 (11–28). Subjects had more diverse opinions about the audio feedback, rating it as interfering somewhat more than the haptic feedback. 35 % of subjects reported levels of agreement higher than 50 for Question 5, indicating that audio feedback interfered. The median level of agreement was 28, and the inter-quartile range was 46 (13–59), also higher than that of the haptic feedback. However, the difference between the average level of agreement for haptic and audio feedback interfering with a subject’s ability to use the robot was significant (paired t test, p = 0.0313).

The final pair of questions asked subjects whether the two types of feedback caused peculiar or undesirable sensations. The subjects did not agree that the haptic feedback caused negative sensations such as numbness, tingling, or cramping in their hands, giving a median level of agreement of 28. They were somewhat more critical of the effects of the audio feedback, giving a median level of agreement of 46 (neither agree nor disagree) that the audio feedback caused sensations such as discomfort and disorientation.

Analysis using Kruskal–Wallis one-way non-parametric ANOVA showed no significant differences between surgeons and non-surgeons for questions 2 through 9 and question 14 (p values ranged from 0.17 to 0.97).

After assessing the effects of the audio and haptic feedback, subjects were asked to select the tasks for which they found either modality of instrument vibration feedback useful. From the subjects who tested each task, Fig. 7 shows the proportion of subjects who found the feedback useful for that task. The majority of subjects found the feedback useful for all four task-types. The percentage of respondents who found either haptic or audio feedback useful for a task ranged from 66 % for suturing to 93 % for needle pass.

Fig. 7
figure 7

Percent of participants who completed a task and agreed that tool vibration feedback was useful for each of the four tasks in Experiment 1

Full size image

We also recorded the feedback intensity levels that each subject preferred for each of the tasks they tested. Figure 8 shows box plots of these preferred intensity levels. Preferences were largely consistent across tasks. Interestingly, subjects tended to prefer a low intensity of audio feedback; the median preferred audio intensity was 0 % (off) for both the needle pass and suturing tasks. More than half the subjects did not want to receive audio feedback on those tasks. The preferred intensities for haptic feedback of instrument vibrations were much higher, with median preferred intensities ranging from 40 % for suturing to 69 % for circle cutting.

Fig. 8
figure 8

Subjects’ preferred feedback level for audio and vibrotactile feedback for each task. Values range from 0 (no audio or vibrotactile feedback) to 100 (maximum audio or vibrotactile feedback)

Full size image

Subjects who performed the gauze-cutting task were asked to indicate their level of agreement with question 13, which asked the extent to which the subject was able to notice the variations in the thickness of the gauze during the circle-cutting task. Subjects responded using a sliding scale (from 0 Strongly Disagree to 100 Strongly Agree). Range of response varied greatly from 13 to 100, with a median response slightly above neutral (66). Some subjects claimed that they could “hear the difference” when closing the scissors over different gauze thicknesses, while other subjects indicated that feedback did not provide information regarding gauze thickness.

Experiment 2

Ninety four participants in Experiment 2 spent 4–15 min operating the da Vinci under any or all of the following feedback conditions: just audio, just vibrotactile, both, or neither.

One hundred percent of the Experiment 2 subjects responded “yes” to Question 4, which asked participants whether they thought it would be useful for surgeons to have the option of using vibrotactile feedback.

Questions 3 and 7 asked whether the subject found audio or vibrotactile feedback useful when performing their respective tasks. Subjects who responded to only one of these questions were omitted from the data set. 83 of the 94 subjects who filled out the survey answered both questions 3 and 7. Subgroups included medical students, residents, fellows, attending surgeons, and others (non-medical professionals). 92–100 % of subjects from each subgroup indicated that they found some type of tool vibration feedback useful. Percentages finding both audio and vibrotactile feedback useful ranged from 41 % (7/17) for residents to 86 % (6/7) for medical students.

Figure 9 shows the responses for various groups of subjects: The upper left pie chart groups all subjects together, and the other five pie charts illustrate subgroups. 61 % of respondents (51/83) found both vibrotactile and audio feedback useful; 35 % (29/83) found only vibrotactile feedback useful; and only 1 % (1/83) found only audio useful. The one subject who preferred only audio feedback was an attending. Two subjects (a resident and a fellow) reported that they found neither audio nor vibrotactile feedback useful (2 %).

Fig. 9
figure 9

(Top left) Distribution of responses to questions 3 and 6 of Experiment 2: “Did you find the vibrotactile feedback useful? (Yes or No)” and “Did you find the audio feedback useful? (Yes or No)”. The upper left plot shows data for all subjects. The other plots illustrate response rates broken down by level of medical experience, as labeled

Full size image

Chi-squared statistical analysis was used to compare responses of attending surgeons to responses of non-attending surgeons for questions 3, 5, 6, and 7 regarding usefulness of haptic feedback, increased awareness, increased carefulness, and usefulness of audio. There were no significant differences between attending and non-attending surgeons (Chi-squared values ranged from 0.13 to 2.6.)

Question 4 was not statistically analyzed because 100 % of responders agreed that the option of haptic feedback was useful. There were no significant differences between attending surgeons and non-attending surgeons for all other Yes/No questions for Experiment 2 (p values ranged from p = 0.11 to p = 0.72).

To understand potential negative and positive consequences of adding tool vibration feedback, subjects were asked whether the addition of feedback made them more aware of instrument contacts and whether the addition of feedback interfered with their ability to use the robot. Figure 10 illustrates the percentage of people who responded “yes” to Question 7 (“Did the haptic feedback make you more aware of your instrument contacts?”) and Question 6 (“Did the haptic feedback interfere with your ability to use the robot?”). Two subjects did not respond to Question 6 and were omitted from analysis. All 94 of participants responded to Question 7. Of the 92 who responded, 12 % believed that haptic feedback interfered with their ability to use the robot, while 96 % of responders believed that feedback made them more aware of tool contacts. We use the terms vibrotactile and haptic interchangeable. However, due to the fact that a majority of people who responded yes for Question 7 (“Did haptic feedback interfere…”), later indicated in Question 8 (comments) that audio specifically interfered or was “distracting”, subjects may have misunderstood Question 6 as referring to both vibrotactile and audio feedback.

Fig. 10
figure 10

Percent of participants in Experiment 2 who thought feedback made them more aware of instrument contacts (left) or interfered with their ability to use the robot (right)

Full size image

Subjects in Experiment 2 illustrated similar preferences to subjects in Experiment 1, preferring some feedback to no feedback and preferring vibrotactile feedback more often than audio feedback. All participants believed that it would be useful to have the option of using vibrotactile feedback.

Discussion

Engineers and surgeons often cite the lack of haptic feedback as a limitation of current commercially available robotic surgery systems [9]. Previous studies have shown that VerroTouch provides a feasible way of restoring various touch sensations to the hands of the surgeon in a clinically relevant setting [26], but user opinions of this type of haptic feedback had not previously been scrutinized. The results of this study indicate that surgeons and non-surgeons alike find this technology useful, and that the majority would prefer to receive haptic feedback of instrument vibrations when performing surgery. Even individuals who would not personally want to use this type of feedback believe that the option of using this technology would be beneficial to other users.

A high percentage of subjects in both Experiment 1 and Experiment 2 value the addition of audio and/or haptic feedback of tool vibrations, demonstrating a strong preference for this technology among the tested populations. Of the subjects in Experiment 1, 95 % percent reported that they preferred one or both types of feedback, and 98 % of subjects in Experiment 2 reported they preferred one or both types of feedback. A surprising 100 % of subjects in Experiment 2 reported that they thought it was useful to have the option of receiving haptic or audio feedback, illustrating that all subjects in the SAGES conference study believe this method of providing haptic feedback has merit.

One aspect of the feedback that subjects did not wholly agree upon was the usefulness of the audio cues. Some subjects in both experiments reported that the audio was “unnatural,” “distracting,” or “jarring” and that they preferred haptic as opposed to audio feedback. This preference for haptic over audio feedback likely stems from the fact that the audio feedback is derived from sensory substitution of instrument vibrations and does not exactly replicate the audible sounds created by the tools. As such, it is not surprising that some subjects indicated a dislike for audio feedback, and at times indicated that they thought audio feedback “interfered” with their ability to use the robot. Subjects with a medical background may have been considering the often noisy environment of the operating room when deciding whether audio feedback would interfere with their ability to use the robot.

Other subjects, however, reported that audio was helpful for some specific tasks such as cutting. Two surgeons in Experiment 1 reported that they could “hear the different thicknesses” of material that they were cutting based on the sound of the closure of the scissors. This information could potentially be useful when cutting through different types of tissue. Another subject specifically mentioned that the haptic and audio feedback “would be helpful for doing mitral valves.” The subject, a cardiac surgeon, further stated that for his work, “more is better,” indicating that he greatly valued any restoration of lost sensations and also found the audio feedback useful. Another experienced surgeon hypothesized that audio or haptic feedback could be useful for certain types of surgery, but stated that it didn’t seem necessary for the type of surgery he practices (general surgery). Such responses highlighted that the different modalities of feedback may be more or less beneficial depending on the clinical task at hand.

The variations in results for preferred level of haptic or audio feedback also indicate that subjects may obtain different benefits from the feedback depending on the task. On average, subjects preferred a moderate level of haptic feedback and a rather low level of audio feedback for each task; these preferences are likely due to the fact that the highest available level of audio was much louder than would be used in practice, while the available range of haptic feedback was more modest. The results also show that for each task at least one subject preferred the highest level of haptic feedback available, indicating that an even higher maximum level of vibrotactile feedback might be appreciated. The large range of responses for preferred level of audio feedback also again reflects the fact that some subjects strongly disliked audio feedback, while others found it useful. It is unclear whether audio is useful in performing some or all of the tasks, but it is clear that some users want the option of being able to turn it off. Subjects’ differing preferences and the possibility of task-dependent benefits clearly indicate that RMIS systems that provide tool vibration feedback should allow users to adjust the strength of each type of feedback according to their specific preference and the stage of the procedure.

Vibrotactile feedback received an overall positive response regardless of level of surgical training. Reasons for disliking the feedback, however, seemed to differ between groups. A novice user with no surgical training in Experiment 1 consistently preferred low or no levels of feedback. While performing tasks, he said he performed “better” without audio or haptic feedback. Compared to expert surgeons, novices tend to perform dry-lab tasks with significantly larger instrument vibrations [29]. The absence of vibration feedback may give some novices the illusion that they are performing a task better than they are because there is nothing to tell them otherwise.

A few expert surgeons who preferred low or no levels of feedback reported that they did not feel the need for feedback of tool vibrations. A study conducted by Hagen et al. found that surgical experts consistently reported that they could perceive haptic feedback during robotic surgery, whereas novices or inexperienced subjects reported missing haptic feedback [11]. In conjunction with these findings, one subject in Experiment 1 reported that he did not need the feedback because he had learned to “feel with [his] eyes”. Other experienced surgeons who did not want to use the feedback indicated that they were “used to” performing surgery without it. Such preference is common and is often referred to as a psychological bias toward that which is familiar [30].

Although this study did not evaluate task performance, the differing preferences between experienced and novice surgeons suggest that haptic feedback plays different roles depending on the operator’s level of experience with the robot. Consequently, the addition of haptic feedback may affect the duration of training needed for novices to become expert robotic surgeons, or it may provide additional information as to how well a novice is performing (as mentioned above). Further studies need to be conducted to explore these possibilities and to discover precisely how haptic feedback of tool vibrations affects surgical task performance.

Even if a subject did not find tool vibration feedback necessary for completing a task, our results indicated that the feedback influenced how a person performed the task. Subjects in both experiments and across all groups agreed that haptic feedback made them more careful and more aware of tool contacts when performing each of the tasks. Even when a subject reported that the feedback was not useful for some of the tasks, they often reported that it did make them more careful or aware of instrument contacts. Subjects showed a wider range of opinions on the effects of the audio feedback, reporting that it does increase awareness and carefulness but not as much as the haptic feedback. Future studies on this technology should compare task performance with perceived performance, carefulness, and awareness.

The overwhelming response of preference for the feedback tested in this study may indicate that users not only want haptic feedback, but that the implementation of tool vibration feedback in this study provided users with satisfactory and useful information. Overall, these findings support our hypothesis that surgeons and non-surgeons would prefer to have haptic feedback of instrument vibrations when using a robotic surgery system. Further studies are currently being conducted to assess how the addition of haptic feedback affects other aspects of robotic surgery such as learning or overall surgical performance and the effects of removing this type of haptic feedback after one has habituated to it.