Simulation in minimally invasive surgery training currently is required as a part of general surgery residency training. Curricula would ideally link psychomotor and cognitive educational activities to procedural performance on actual patients [1].

Many tools have been developed to help surgical educators more efficiently and safely train surgeons. The curricula for the use of these tools should be developed in a methodical, evidence-based manner. Unfortunately, simulator development and validation studies have progressed more slowly than the demand to adopt these technologies. Much research still is needed to determine which tools should be used, which need further refinement, and which should be abandoned.

Findings have shown low-fidelity simulators to be useful in helping trainees acquire psychomotor skills. The Fundamentals of Laparoscopic Surgery (FLS) curriculum and test are based on a low-fidelity benchtop model of laparoscopic surgery [2]. The FLS program must be completed for certification by the American Board of Surgery [3].

High-fidelity virtual reality (VR) platforms offer several advantages over low-fidelity models [4]. One aspect offered by low-fidelity models that many VR simulators lack is tactile sensation. Minimally invasive VR simulators have been developed with simulated force feedback, adding approximately $20,000 to $30,000 to the price [5, 6]. Despite the high additional cost, few studies have addressed the added value of this expensive feature.

Haptics is defined as the science of applying tactile sensation and control to interaction with the environment [7]. Haptic systems have been developed for robotic surgery applications and for minimally invasive surgical simulators. In minimally invasive surgery, surgeons derive most of their sensory input from visual cues [8]. The tactile sensation felt by the operator in laparoscopic surgery is not an accurate representation of the instrument interacting with tissue [9, 10]. Despite this inaccurate representation, many believe that haptic-enabled simulators offer training and performance benefits [8, 11].

Two recent studies evaluated trainee performance on VR simulated tasks with and without haptics engaged [5, 6]. The results were mixed, showing a benefit in only one task out of the combined six studied. The two studies both involved performance tests with untrained novices.

Ultimately, novice performance on a simulated task is not as critical to a simulator as the acquisition of skills through training. Therefore, we set out to determine the added benefit of haptics in the acquisition of laparoscopic skills. We performed a randomized, controlled trial to determine the effect of haptics on training and performance with a VR simulator.

Materials and methods

In our investigation of haptics in virtual laparoscopic cholecystectomy training, the LapMentor II (Simbionix, Cleveland, OH, USA) VR simulator was used exclusively. The LapMentor II is a second-generation laparoscopic VR simulator with simulated, computer-controlled, motorized force feedback. It has training modules for basic tasks, procedural tasks, and ability to perform full virtual procedures.

Expert proficiency data for the LapMentor II were published by Stefanidis et al. [12] in 2008. These criteria were established by averaging 10 repetitions of expert fellows and attending surgeons who had each performed more than 100 laparoscopic procedures (personal communication).

The current research was approved by the University of Cincinnati Institutional Review Board and by the United States Army Human Research Protection Office. The study enrolled 33 novice laparoscopic undergraduate, medical, and graduate students. Many of the participants could not finish the study due to repeated machine failures.

Each participant was randomly assigned with equal probability to one of three study groups: control, haptics-trained, or non-haptics-trained group. All the participants were given an educational presentation consisting of an introduction to laparoscopic surgery, laparoscopic cholecystectomy, and the LapMentor II. In addition, all the students completed a prequestionnaire before data collection. The prequestionnaire included questions pertinent to hand–eye coordination (musical instrument skill level), psychomotor skills (video gaming experience), and demographic information.

Students assigned to the haptics- and non-haptics-trained groups were required to achieve expert proficiency in a series of basic laparoscopic tasks. The proficiency levels used were taken from previously published expert values (Table 1).

Table 1 Proficiency targets and number of repetitions required to achieve proficiencya
Full size table

The basic tasks were nine laparoscopic tasks given by the Basic Task Module of the LapMentor II: camera navigation 0º, camera navigation 30º, hand–eye coordination, clip application, grasp and clip, ball drop, cutting, cautery application, and object translocation. The haptics group performed all nine basic tasks to proficiency with the haptics feature of the LapMentor II engaged, whereas the nonhaptics group reached proficiency with haptics disengaged. Students assigned to the control group performed the nine basic tasks in the series with haptics engaged but completed each task only one time, with no proficiency level required.

Completion of the basic tasks was organized into one or two 4-h sessions (Fig. 1). Each session with the simulator was proctored by nonclinical research team members with knowledge of the interface and modules. For the haptics and nonhaptics groups, the proctors confirmed that the subject had reached proficiency in all tasks before allowing the subject to proceed to procedural training and assessment.

Fig. 1
figure 1

Study design showing training with basic tasks, training with laparoscopic cholecystectomy procedural tasks, and assessment with virtual reality cholecystectomies

Full size image

After completing the nine basic tasks, all three groups were required to complete four tasks of the LapMentor II Procedural Task Module one time. These four tasks broke the full laparoscopic cholecystectomy into four simple tasks to help the students learn the anatomy of the full procedure and how to apply the skills practiced during the basic tasks. These four procedural tasks were (1) clipping and cutting the cystic duct and artery of the retracted gallbladder, (2) retracting the gallbladder with the left hand while clipping and cutting with the right hand, (3) dissecting the cystic duct and artery to achieve the “critical view,” and (4) separating the gallbladder from the liver bed.

The nonhaptics group performed all these tasks with haptics disengaged to maintain the trainees as naive for testing the potential benefit of haptics in performance. The control and haptics groups performed these tasks with haptics engaged.

To test performance between groups, all the participants completed 10 VR laparoscopic cholecystectomies. The haptics feature of the LapMentor II was toggled on and off, alternating after each completed full procedure, with haptics engaged for attempts 1, 3, 5, 7, and 9. The LapMentor II automatically collected all the data used in the analysis.

We chose to compare instrument movement and efficiency parameters because the cognitive safety components of training were not stressed in the study. The parameters compared were total time, efficiency of cautery, number of movements, total path length, and speed of instruments. All the participants who completed the study were given a questionnaire to assess their attitudes toward the simulator.

The haptics and nonhaptics groups were compared on the number of trials needed to reach the criteria (Table 1) for the basic skills tasks using Wilcoxon rank sum tests. To test performance, each procedural parameter was modeled individually. The p values were derived from mixed models treating subjects as random within fixed groups. Repeated measures effects were modeled for haptics engaged or disengaged, and to adjust for learning, we used the log of the attempt number (1 to 10). An additional set of models added the interaction between groups and the engagement or disengagement of haptics. All reported p values are two-tailed, and alpha for all tests was a two-tailed 0.05 unadjusted for multiple tests.

Results

The control, haptics, and nonhaptics groups did not differ significantly in terms of sex, education level, hand dominance, video game experience, surgical experience, and nonsurgical simulator experience. All the groups were predominantly right-handed. All 33 participants were included in this analysis.

More attrition occurred in the haptics and nonhaptics groups than in the control group. In the haptics group, 12 trainees started the study. Six trainees completed the proficiency training, and four completed the procedural assessment. The two who did not compete the procedural training were available, but the simulator failed.

In the nonhaptics group, 10 started the study, with 4 completing the proficiency training and procedural assessment. In the control group, 11 started the study, and 10 completed the procedural assessment. According to the proctors, it took more sessions and total time commitment for the students to complete proficiency-based training in the haptics and nonhaptics groups than in the control group. Simulator failures were common, contributing to scheduling difficulties with the participants. Only one participant explicitly quit because he thought he would not reach the task proficiency level (task 3 after 112 attempts).

With regard to training, the haptics and nonhaptics groups differed significantly in the number of attempts required to reach proficiency with any of the nine basic tasks (Table 1). Regarding performance, we compared the means across all 10 trials to determine differences between the groups (Table 2), with p values derived from the mixed models described earlier (without interactions). The haptics and nonhaptics groups showed no statistically significant differences. The haptics and control groups differed significantly in mean number of left instrument movements (haptics, 98 ± 39 vs. control, 174 ± 92; p = 0.03) and mean path length of the left instrument (haptics, 179 ± 76 vs. control, 353 ± 218; p = 0.04).

Table 2 Mean performance of control, haptics-trained, and non-haptics-trained groups on 10 virtual reality cholecystectomies
Full size table

There was a trend toward significance in total time (haptics, 440 ± 114 vs. control, 553 ± 233; p = 0.1). There was a statistically significant difference between the performance of the nonhaptics and the control groups in total time (nonhaptics, 376 ± 116 s vs. control, 553 ± 233 s; p = 0.02), number of left instrument movements (nonhaptics, 95 ± 39 vs. control, 174 ± 92; p = 0.02), and path length of the left instrument (nonhaptics, 177 ± 99 cm vs. control, 353 ± 218 cm; p = 0.04). There was a trend toward significance in efficiency of cautery (nonhaptics, 72 ± 8% vs. control, 59 ± 15%; p = 0.06).

Using the same statistical model, we compared the performance of haptic cholecystectomies and nonhaptic cholecystectomy trials (Table 3). In combining all groups, we found a negative effect of haptics, increasing the number of right instrument movements (estimate of effect, 29; p = 0.008) while having a positive effect on the speed of the right instrument (estimate of effect, 0.2 cm/s; p = 0.0004). Engaged haptics also improved (decreased) the number of left instrument movements (estimate of effect, −14; p = 0.02).

Table 3 Effect of haptics feature being engaged during virtual reality cholecystectomies
Full size table

In the models that included an interaction for group by haptics condition, the only parameter showing such an interaction was efficiency of cautery (p = 0.04), on which the haptics group performed better with haptics engaged (haptics engaged, 70 ± 7 vs. haptics disengaged, 65 ± 8), and the nonhaptics group performed better without haptics (haptics engaged, 69 ± 6 vs. haptics disengaged, 74 ± 9).

The learning curves of the parameters are shown in Fig. 2. We compared the performance over the attempts with all the groups combined and found that the log of the attempt number in our model was a statistically significant factor in total time (p < 0.0001), number of right instrument movements (p < 0.0001), number of left instrument movements (p < 0.0001), path length of the right instrument (p < 0.0001), path length of the left instrument (p < 0.0001), and speed of the right instrument (p < 0.0001). Efficiency of cautery and speed of the left instrument were not metrics that demonstrated statistically significant learning in our model.

Fig. 2
figure 2

Learning curves for all the parameters evaluated, with means for the control, haptics, and nonhaptics groups represented. A Total time. B Efficiency of cautery. C Number of right instrument movements. D Number of left instrument movements. E Path length of the right instrument. F Path length of the left instrument. G Speed of the right instrument. H Speed of the left instrument

Full size image

On the poststudy questionnaire, three of four participants in the training groups could tell when the haptics was engaged, and 9 of 10 control participants could tell when haptics was enabled.

Discussion

Haptics has been incorporated into VR simulators without compelling evidence of its value in training. This is an important issue because this feature adds to both the acquisition and the maintenance costs of the simulator. There are several reasons to question the utility of haptics. First, many of the forces felt at the instrument handle are “interference” forces due to deformation of the abdominal wall and friction forces of the trocar, not “useful” forces of the instrument interacting with tissue [9]. Second, in the landmark VR-to-operating room study by Seymour et al. [13], which used the nonhaptic Minimally Invasive Surgery Trainer–Virtual Reality (MIST-VR) simulator, the findings showed improved performance in the live operative setting after nonhaptic VR training. Third, no clear benefit of haptics was shown in two recent studies evaluating the performance of novices on haptics tasks [5, 6].

In our study, the training efficiency of the haptics and nonhaptics groups did not differ. We found no differences between the groups in the number of attempts necessary to reach expert proficiency in any of the nine basic skills tasks. We trained the study groups to proficiency in the basic tasks only, making this a study of psychomotor performance rather than cognitive skill performance. All groups performed the four laparoscopic cholecystectomy procedural tasks one time. This was to introduce them to the relatively unfamiliar complex task of a full VR cholecystectomy. For this reason, we did not include safety metrics in our assessment.

Proficiency-based training in the basic tasks of the LapMentor II did produce performance gains in both the haptics and nonhaptics groups. This provides evidence that proficiency-based training on VR simulators is beneficial regardless of haptics. We did find that more attrition occurred in the haptics (n = 12 reduced to 6) and nonhaptics (n = 10 reduced to 4) groups than in the control group (n = 11 reduced to 10). We did not quantify total time nor survey those who did not complete the study because many of them could not be reached.

Many students did struggle to complete the proficiency training. We previously reported that the metrics we used may have been too difficult [14]. It was not infrequent for participants to spend two to three 4-h sessions to complete the study. Other reasons why participants did not complete the study were frequent machine failures and scheduling difficulty. These factors often overlapped and produced delays in training.

In this study, we aimed to discover the effect of haptics on the training and performance of laparoscopic novices. No statistically significant differences were observed between the haptics and nonhaptics groups in terms of training or performance, but the nonhaptics group outperformed the control group on more parameters than the haptics group. This finding suggests that the haptics feature included in the LapMentor II does not improve training.

Learning occurred over the course of the procedural performance assessment in all the parameters assessed except efficiency of cautery and speed of the left instrument. The learning curves (Fig. 2) show that the learning effect for the control group was much more pronounced than for the training groups. Total time, in particular, shows a pattern similar to the total time learning curve comparing control subjects with intermediate level and expert level groups in the Gallagher and Satava [15] assessment of MIST-VR tasks. In our study, the haptics group seemed to have more learning than the nonhaptics group just as did the intermediate group compared with the expert group in their study. This pattern was not apparent for the other parameters. From this we conclude that the learning experienced during the 10 virtual cholecystectomies was less pronounced in both training groups than in the control group, which is the same effect that operative experience has had in similar simulator trials. The learning curves show that the skill acquired in the basic task proficiency training transferred to the new task of cholecystectomy. Haptics training did not appear to show a benefit over nonhaptics training with regard to improved learning during the course of the VR cholecystectomy procedures.

We evaluated the performance of the groups with haptics engaged and disengaged during the VR cholecystectomies. To minimize mistakes in data collection, we chose to have all subjects perform their first trial with haptics engaged, alternating the haptics condition within each subject thereafter. Thus, trials without haptics (trials 2, 4, 6, 8, and 10) were conducted, on the average, with a bit more experience than the trials with haptics (trials 1, 3, 5, 7, and 9). To adjust for this difference (and to remove error variance due to a strong effect in which we were not primarily interested), we accounted for learning by including the trial number (1 to 10) in the statistical model as a one degree of freedom parametric predictor. Furthermore, we expected (and observed) a learning effect that was not linear with the trial number. Trial-to-trial improvement was most pronounced in the early trials, shrinking as more learning had taken place. To fit this process, we used the log of the trial number in the statistical model.

We are unable to draw any conclusions as to the performance benefit of haptics in our study. The performance of all the groups with haptics enabled seemed to increase the number of right instrument movements, to decrease the number of left instrument movements, and to increase the speed of the right instrument. In separating out the haptics and nonhaptics cholecystectomies in the individual groups, the only parameter that reached statistical significance was efficiency of cautery. The haptics group had improved efficiency with haptics engaged, whereas the nonhaptics group had worse efficiency of cautery with haptics engaged. We cannot clearly conclude that haptics has a positive or negative effect on performance.

We chose to assess the performance of our groups with the simulator’s VR laparoscopic cholecystectomy. We opted to do this because the haptics feature could be turned on and off, and the subjects were already familiar with the simulator. This module has shown early evidence of concurrent validity using Objective Structured Assessment of Technical Skills. The VR laparoscopic cholecystectomy procedure was shown to transfer skills to the live pig model [16].

Our study had several limitations. First, less than half of the subjects in the haptics and nonhaptics groups finished the study. Only the top performers or those with the most interest may have been self-selected. Second, the haptics feature of the LapMentor II has an experts’ rating of only 6/10 for the realism of the force feedback [17]. Thus, we were potentially testing an underdeveloped haptics system. The simulator also had frequent failures and needed multiple repairs during the data collection phase, placing delays in the training of some participants and causing others to drop out of the study. Finally, our study design partially confounded learning with the haptics on/off condition, making our estimate of the haptics effect somewhat dependent on our parameterization of the learning effect.

Despite these shortcomings, the reported data call into question the utility of haptics in minimally invasive VR training. Our data suggest that lower-cost nonhaptic VR simulators are likely to deliver better value than expensive haptic VR simulators.