Despite the hefty price tag [1], robotic minimally invasive surgeries have gained significant popularity since their introduction in the late 1990s. In fact, in 2010, 67% of radical prostatectomies in the USA were performed robotically [2]. This growing popularity is largely due to the benefits that surgical robotic systems, such as Intuitive Surgical’s da Vinci, provide over traditional laparoscopic surgery. Advanced features, such as motion scaling, increased degrees of freedom, teleoperation, and improved ergonomics can make these expensive systems a better choice for abdominal, pelvic, and cardiothoracic surgical procedures [3–7]. Despite the aforementioned benefits, robotic minimally invasive surgeries introduce their own set of challenges, leading many to question their effectiveness compared to that of traditional laparoscopic surgeries [8]. In the past, the lack of haptic feedback has been identified as one major drawback of robotic systems [9–11]. While traditional laparoscopy provides some kinesthetic feedback through the instrument itself, most robotic systems suffer from a complete lack of kinesthetic and tactile feedback. In recent years, a variety of haptic feedback systems (HFS) have been introduced that attempt to address this limitation [12–14]. The absence of haptic feedback, however, may not be the sole factor affecting the performance of the surgeon in robotic procedures. By taking into consideration the vital role of proprioception in robotic surgical operations, it is possible to hypothesize that certain performance drawbacks may, in fact, stem from how the brain handles integration of visual sensory information and proprioceptive information.

Robotic systems generally consist of a control console and one or more robotic arms. The robot’s arm(s) are controlled using signals from the console where the surgeon uses a variation of joystick-like controls to direct movements of the robotic arms. This separation of the console and the robotic arms is what allows for many of the important features of robotic systems such as teleoperation and motion scaling. These features, however, can introduce a different problem. The movement of the controls in the console is limited by the available physical space in the console and the reach of the surgeon’s arms. Motion scaling further exaggerates this effect by reducing the ratio of movements of the arms relative to the movements of the robotic controls. For example, when the robotic arms are capable of moving beyond a certain extent on any one axis, the controls in the console may have reached the limits on that same axis. Robotic systems such as the da Vinci have attempted to solve this issue by introducing a “clutch” operation. When active, the clutch temporarily disconnects the robotic arm guidance from the console so that the controllers in the console can be repositioned to allow for additional movement along the previous axis.

While the clutch operation is often used extensively during a procedure, its effects on the performance of the surgeon have not been largely studied. Evidently, the utilization of the clutch control can potentially introduce what we refer to as visual–perceptual mismatch. The human brain at all times maintains a perception of the location of one’s limbs in space, defined as the sense of Kinesthesia [15, 16]. This knowledge, together with tactile sensory information, allows an individual to perform many tasks without visual aid. For example, one may pick up an object and pass it from one hand to another without a visual cue (i.e., with both eyes closed). During robotic surgery, employing the clutch can create a mismatch between the brain’s awareness of the location of one’s hands in the console and what the brain’s visual system reports about the locations of the robotic arms (as seen through the robot’s laparoscopic camera). In surgical tasks that require coordination between the two arms, this mismatch, coupled with the lack of tactile sensation, can potentially introduce an additional cognitive load, thereby affecting the surgeon’s overall performance. In this paper, we investigate the effect of visual–perceptual mismatch on the performance of da Vinci surgical robot operators. Additionally, we evaluate the effectiveness of adding tactile sensory information as a way of reducing the cognitive load caused by this mismatch.

Methods and procedures

Research environment and apparatus

In this study, a series of tests were used to evaluate whether visual–perceptual feedback has a significant effect on the performance of individuals during surgical tasks. The protocol used was approved by the UCLA Institutional Review Board. We evaluated the performance of novice subjects in performing a peg transfer task through quantitative evaluation of the grip force, number of faults (defined as the number of times the subject dropped a peg), and time to completion.

Task

The task was adapted from the standard peg transfer test in the Fundamentals of Laparoscopic Surgery (FLS) education module developed by the Society of American Gastrointestinal Endoscopic Surgeons (SAGES) [17]. The task requires that the subject uses Intuitive Surgical’s da Vinci IS 1200 system to pick up a peg with one arm, rotate it along z axis (one side of the pegs is marked), pass it to the other arm without dropping the peg, and then place it back down on the opposite side of the field (Fig. 1A). In order to control for the peg dropping in different positions and the difficulty of picking the peg back up again, the proctor resets the peg to its original position if it was dropped during the transfer. For these studies, both da Vinci arms were fitted with fenestrated bipolar forceps (Fig. 1A, B).

Fig. 1
figure 1

A Two pegs placed on opposite sides of the field of view prior to start of trial, B da Vinci fenestrated bipolar forceps and C da Vinci fenestrated bipolar forceps fitted with piezoresistive normal force sensors

Full size image

Participants

A total of 45 subjects (age range 16–30 years) were recruited from the general population through email notifications that went out to UCLA students, faculty and staff. All subjects who participated in the study had no prior experience with robotic surgery.

Procedure

Each novice subject was given a 2-min training period prior to the first test in order to provide familiarity with basic da Vinci controls. Each subject performed the task five times with 30-s break intervals between each trial. During the break, the subject was not allowed to operate the robot or get up from the seat. Subjects were not allowed to utilize the clutch or the camera during the task. The camera position and zoom were adjusted to be identical for all trials. Before the experiment, the proctor validated the positioning of the controls and the camera by performing the full task; this ensured that the task could be completed without device error, such as hindered movement of both the robotic arms and the subjects’ arms in the console.

Subjects were instructed to perform the task with three goals in mind. The first priority was to perform the task without dropping the peg. The second was to reduce the time to completion. The third and lowest priority was to minimize the force applied to the peg without sacrificing the first two goals. Subjects were asked to grab at least half of the peg and also to grab the pegs by the side, thereby allowing deformations in the peg to be easily visible.

The 45 subjects enrolled in the study were split into three groups. Three axes of the robotic control were defined relative to the subject, where x-axis represented movements of the controls to the left/right, y-axis represented movements toward/away from the subject, and z axis represented movements of the controls up/down. For the first group (aligned group), the proctor configured the robotic arms and controls using the clutch such that no mismatch would exist between visual and proprioceptive information in any of the axis. In the second group (misaligned group), a mismatch was created by positioning the robotic arms directly in front of each other and then using the clutch to create a control misalignment in the z axis (Fig. 2). In other words, while the robotic grasper tips were positioned in the same exact height, the console controls were shifted apart by pulling the right control up toward the maximum height limit and pushing the left control down toward the lower height limit (total of 10 cm z axis misalignment). The third group (haptics-misaligned group) was used to evaluate the benefit of introducing haptic feedback. This would allow us to determine the extent to which tactile sensory information assists in compensating for visual–perceptual mismatch (if at all). Therefore, the same configuration as the misaligned group was used, but with the addition of a normal force tactile feedback system we had developed for the da Vinci [13]. For providing tactile feedback, we installed piezoresistive force sensors on the graspers (Fig. 1C) and utilized a feedback system described by Franco et al. [13] which relies on pneumatic actuators installed on the fingertips to convey normal forces.

Fig. 2
figure 2

A Aligned group with no visual–perceptual mismatch. There is no misalignment in z axis and B misaligned group with visual–perceptual mismatch from a 10 cm z axis misalignment

Full size image

The utilization of three different groups of subjects helps in reducing training bias that can occur if the same subject performs all three different tasks. Subjects were blinded to which the concept of the study and the group assignment.

Measurements

The grip force, time of completion, and number of faults for each subject were collected by a custom application that receives grip force data directly from grasper-mounted Flexi force sensors [13]. The proctor marks the beginning of each trial and the number of times the participant drops the peg directly in the program. The end of trial is marked as soon as the participant correctly places the second peg down. For data analysis, we looked at the average grip force during the entire trial, the total time to completion, the number of faults/drops, and the grip force score. The average grip force from both hands was calculated using data from two grasper-mounted FlexiForce sensors (at a rate of 50 Hz). The grip force score was calculated as the integral of the force–time graph. Force score can often be a more accurate way to identify the overall confidence of the subject when performing the task. For all grip force data, only the gripped time, that is, the times when the subject is holding on to the peg, is analyzed. Gripped time is identified by only considering the time when the force values are above 20% of the minimum force. Using “gripped” force values eliminate the time when the subject is not holding on to the peg and is useful in determining how much force the subject applies when actually holding on to the peg. Additionally, while time to completion is a good measure of overall performance of a participant, this measure is dependent on the number of times the subject drops the peg. For this reason, we also look at the time for completion of a single transfer. The last successful peg transfer in each trial was used for measuring this time value.

Statistics

For each subject, we used the mean of all five trials for data analysis. All statistical analysis was performed using Math.Net and R statistics packages. We used ANOVA for comparison between groups followed by pairwise post hoc analysis if the null hypothesis was rejected (p < 0.05). In cases where the normality or equal variance assumptions were not met (i.e., time to completion and force score), log 2 transformations were used prior to analysis using one-way ANOVA. For analysis of the number of faults, square root transformation was used to improve homogeneity of variance and was followed by a one-way ANOVA test. For all post hoc analysis, the p value was adjusted using the Holm method.

Results

As shown in Fig. 3, the misalignment in the z axis resulting in visual–perceptual mismatch has a significant effect on the number of times the subjects of different test groups dropped the peg (p = 0.012, one-way ANOVA). The subjects in the misaligned group (no HFS) drop the peg significantly more often than the aligned group (p = 0.011, Holm–Bonferroni correction). When haptic feedback is enabled (haptics-misaligned group), a reduction in number of faults/drops is observed compared to the misaligned group; however, while a trend is present, this reduction is not statistically significant (p = 0.088, Holm–Bonferroni correction).

Fig. 3
figure 3

Average number of faults (peg drops) and SE of mean bars for each group

Full size image

As shown in Fig. 4, there also exists a significant difference between groups with respect to the time to completion of the tasks (p < 0.001, one-way ANOVA). The visual–perceptual mismatch resulted in the participants of the misaligned group taking significantly longer to complete the task (p < 0.001, Holm–Bonferroni correction) compared to that of the subjects in the aligned group. In this scenario, however, enabling haptic feedback appears to shorten the time to completion and compensate for the existence of visual–perceptual mismatch. The subjects in the haptics-misaligned group performed significantly better than those in the misaligned group (p = 0.008, Holm–Bonferroni correction) while no significant differences were observed between the aligned and haptics-misaligned groups (p = 0.309, Holm–Bonferroni correction).

Fig. 4
figure 4

A Comparison of the total time to completion for all groups. B Comparison of the time to completion for a single, successful transfer. Data shown as mean of each group with SE of mean bars

Full size image

One of the important limitations of using time to completion as a metric for evaluating subject performance is that it is dependent on the number of faults; thus, Fig. 4B shows the time of completion for a single, successful transfer. No statistically significant difference seems to exist between the peg transfer time for a successful peg transfer between the different group (p = 0.136).

No significant differences were observed between the groups for the average grip force while holding the peg (p = 0.865, one-way ANOVA). Similarly, looking at the average grip force, purely during a successful peg transfer, also shows no significant difference among the groups (p = 0.072). It is worth noting that a significantly lower, left-hand grip force (p = 0.024) was observed during a successful peg transfer. It is possible that a higher number of subjects would have allowed the improvements in grip force due to HFS to be more clearly visible.

The force score is a measure of how much force the subject applied in a specific period of time and is calculated using a numerical integration of the force–time graph. This metric provides another way to look at the total amount of force the operator applies to the peg and may better correspond to the amount of tissue damage that could arise during surgical operation.

A statistically significant difference was observed between the force score for the different groups (p = 0.049, one-way ANOVA). The aligned group had a better force score (lower score is better) compared to the misaligned group (p = 0.046, two-tailed t test; p = 0.096, Holm–Bonferroni correction). Similarly, the haptics-misaligned group showed a better score compared to the misaligned group (p = 0.034, two-tailed t test; p = 0.096, Holm–Bonferroni correction), indicating that the addition of tactile feedback helped compensate for mismatch caused by the z axis offset (Fig. 5).

Fig. 5
figure 5

Comparison of force score (N * t) between all three groups. Data shown as mean of each group with SE of mean bars. B Comparison of force score for the right hand (p = 0.095). C Comparison of force score for left hand (p = 0.053)

Full size image

Discussions

Robotic surgical systems have provided many valuable features that arguably produce better ergonomics for surgeons and improved outcomes for patients. Despite this fact, they continue to suffer from some limitations. Significant research has been dedicated to identification these shortcomings, in the hope that a better understanding of these limitations can allow engineers to introduce improvements and design changes needed to remedy outstanding problems. Some of the most commonly discussed limitation of robotic systems have included the lack of appropriate tools for various cardiac and abdominal surgeries [18–20], lack of accessibility in large area surgeries [21], limited function and arm collision in non-standard positions such as Trendelenburg [22–25], and, most obviously, lack of haptic feedback [5, 13, 20, 22, 26]. Research has recently been focused on developing solutions for the aforementioned problems, from single-access robotic systems such as Titan Medical SPORT Surgical System to haptics feedback devices that can act as accessories for existing surgical systems [27].

Despite extensive research, the effect of relative coordinate systems on surgeon performance, utilized by nearly all robotic surgical systems, has not been previously studied in detail. Furthermore, studies of the benefits of haptic feedback in robotic operations have previously been primarily focused on reduction in tissue damage and improvement in surgical errors during tasks such as knot tying [10]. The effects of HFS on visual–perceptual mismatch and more specifically, robotic tasks that involve coordination between the two arms had not been previously studied.

The existence of a clutch operation as a solution for movement of robotic controls in a limited 3D space creates a relative coordinate system where the benefits of human perception are significantly reduced. In essence, every time the operator utilizes the clutch, changes are made to the coordinate system and any learned mapping that exists in the brain about the relative location of the individual’s arm and the robot’s arms/graspers becomes invalidated. This suggests that the surgeon may become almost completely dependent on visual cues when operating the robot. We studied the possibility of mismatch-induced performance reduction using a da Vinci surgical system. The peg transfer task which was used during these experiments involved passing a peg from one arm to another. This is an important feature which targets coordination between the arms, an important part of most surgical tasks. The passing of the peg from one arm to another requires that the two robotic arms be aligned perfectly so the peg can be properly passed from one grasper to another. The brain’s mapping of the location of the limbs in space can be crucial to this type of task. Under normal conditions, this type of perception, coupled with tactile sensory information, is what allows passing items from one hand to another in a visually impaired individual without much difficulty.

The evaluation of the number of peg drops (number of faults) provided excellent information about the effect of visual–perceptual mismatch on this ability. As expected, we saw a clear disadvantage in the z axis misaligned group compared to the group where the controls and the robotic arms were aligned in the z axis. A possible underlying reason for the reduced performance is that the users have difficulty in passing the peg from one hand to another. When we introduced the z axis misalignment, subjects in the mismatch/misaligned groups consistently had difficulty aligning the arms of the robots during the transfer process. The existence of the mismatch creates an additional cognitive load to the limited depth perception provided through the laparoscopic cameras. In the aligned group, the ability of the brain to locate and coordinate one’s limbs in space can help compensate for this limitation, allowing the peg transfer to take place more easily. It is also worth noting that despite the existence of a z axis misalignment in the haptics-misaligned group, there is a clear trend (p = 0.088) indicating that enabling tactile feedback can assist in reducing the effects of visual–perceptual mismatch on the subject’s ability to pass the peg. This is expected since adding this type of sensory information helps in allowing the subject to detect when sufficient contact does not exist with the receiving grasper, thereby reducing the probability of the peg being dropped.

Utilizing overall time to completion for the tasks as a measure of overall performance, the subjects that operate the da Vinci with the controls misaligned in the z axis have a clear disadvantage compared to the aligned group. Most interesting, however, is the extent to which enabling tactile feedback helps in compensating for this (Fig. 4A). Of course, it is important to note that the total time to completion is highly dependant on how many times the subject dropped the peg during the transfer. Still, these data help confirm the trend (p = 0.088) we observed when comparing the number of drops between the misaligned and the haptics-misaligned groups. The total time to completion helps more clearly define the significant benefits of the tactile feedback in compensating for some of the handicap that is introduced by the visual–perceptual mismatch.

In order to determine whether the number of faults is the only factor affecting the differences in the overall time to completion, we also looked at the time to successfully complete a single peg transfer. It is worth noting that while the data do not show a significant difference (p = 0.136) among the different groups (Fig. 4B), a trend still appears to be present. Considering the cross-subject design of this study, there is strong indication that a higher number of subjects, a more complex transfer task or even different metrics (ex. path of arm movements) can help in determining whether any other factors are affecting time to completion. Furthermore, an important observation worth discussion is the small discrepancy that exists between the significance of the total time to completion and that of the total time to completion for a successful transfer. While there is a clear significant difference between haptics-misaligned and the misaligned groups in the total time to completion, this significance is more difficult to observe for the number of faults and the time to complete a single successful peg transfer. In order to identify where this discrepancy comes from, one must consider how we measured the time to completion of successful transfer. Because time to completion of single successful transfer is measured from the moment the subject picks up the peg, it does not take into account the time that a subject spends attempting to position the grasper correctly to get sufficient contact with the peg to properly hold it. Considering that the existence of tactile feedback, even as a purely binary measure, allows the user to easily recognize when there is sufficient contact with the peg, it is therefore easy to understand why the total time to completion is so significantly improved for subject that have the HFS enabled.

Finally, the installation of force sensors on the graspers also allowed us to evaluate the grip force during the peg transfer. Surgical graspers are capable of producing large magnitude forces at the robotic graspers and potentially damage the tissue during surgery. Visual cues can, to some extent, help surgeons reduce the amount of force applied to the tissue. Haptics feedback systems have been shown to help reduce excessive grip forces, especially in non-experts, by providing tactile sensory information at the surgeon’s fingertips [26]. We looked at the average grip force between the different groups both during the entire trial and during a single successful transfer. While a trend appears to be present (p = 0.072) for the applied grip force during a single successful peg transfer, indicating a potential benefit to having the tactile feedback, it is difficult to draw a definitive conclusion. One reason may be that the existing visual–perceptual mismatch adds another level of cognitive load which weakens the subject’s ability to maintain the proper grip force during the entirety of the task.

It is worth noting that the peg transfer task, which was adopted from FLS educational module, was representative of some of the simplest tasks which a surgeon may need to perform. Naturally, the simplicity of this task may reduce the extent to which visual–perceptual mismatch affects the subject’s performance. On the other hand, the simplicity of the task is what allows us to more easily correlate the results of this study with significant performance reductions in more complex, real-world, surgical tasks.

The data presented from this study support our original hypothesis that visual–perceptual mismatch has a negative effect on the performance of the operators of surgical robots such as the Intuitive’s da Vinci. This performance handicap is not limited to the speed and accuracy by which a task is performed but may also affect a surgeon’s ability to focus on other aspects of the surgery (i.e., reduction in excessive grip force). It is possible that extensive training with robotic surgical systems can eventually train the operator’s neural pathways to more quickly create new mappings of the coordinate system following a clutch operation, thereby reducing overall performance impact. However, to what extent, and how fast the brain can adjust for these changes, as well as how experience plays a role in this adaptation, will require additional study. Since most modern surgical robotic systems consist of some type of clutch operation, it may be necessary for the effects of visual–perceptual feedback to be studied further so that solutions and/or alternatives can be engineered.

The results of this study also provide some hints about discrepancies discussed in the literature with regard to the benefits of haptic feedback in robotic surgical operations [10]. If the extent of handicap created by the visual–perceptual mismatch can vary depending on the extent of multi-axis misalignment that is created from utilization of the clutch as well as the experience of the surgeon with robotic systems, then it is possible to conclude that so will the benefits of HFS in compensating for this handicap. Without controlling for the effects of visual–perceptual mismatch, studies that attempt to evaluate haptic feedback systems may observe variations between subjects that may be difficult to predict. On a larger scale, this observation may not be limited to haptic feedback but rather apply to evaluation of other sensory augmentation mechanisms that attempt to improve the functionality of robotic surgical systems.