Abstract
Background
Haptics is an expensive addition to virtual reality (VR) simulators, and the added value to training has not been proven. This study evaluated the benefit of haptics in VR laparoscopic surgery training for novices.
Methods
The Simbionix LapMentor II haptic VR simulator was used in the study. Randomly, 33 laparoscopic novice students were placed in one of three groups: control, haptics-trained, or nonhaptics-trained group. The control group performed nine basic laparoscopy tasks and four cholecystectomy procedural tasks one time with haptics engaged at the default setting. The haptics group was trained to proficiency in the basic tasks and then performed each of the procedural tasks one time with haptics engaged. The nonhaptics group used the same training protocol except that haptics was disengaged. The proficiency values used were previously published expert values. Each group was assessed in the performance of 10 laparoscopic cholecystectomies (alternating with and without haptics). Performance was measured via automatically collected simulator data.
Results
The three groups exhibited no differences in terms of sex, education level, hand dominance, video game experience, surgical experience, and nonsurgical simulator experience. The number of attempts required to reach proficiency did not differ between the haptics- and nonhaptics-training groups. The haptics and nonhaptics groups exhibited no difference in performance. Both training groups outperformed the control group in number of movements as well as path length of the left instrument. In addition, the nonhaptics group outperformed the control group in total time.
Conclusion
Haptics does not improve the efficiency or effectiveness of LapMentor II VR laparoscopic surgery training. The limited benefit and the significant cost of haptics suggest that haptics should not be included routinely in VR laparoscopic surgery training.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Simulation in minimally invasive surgery training currently is required as a part of general surgery residency training. Curricula would ideally link psychomotor and cognitive educational activities to procedural performance on actual patients [1].
Many tools have been developed to help surgical educators more efficiently and safely train surgeons. The curricula for the use of these tools should be developed in a methodical, evidence-based manner. Unfortunately, simulator development and validation studies have progressed more slowly than the demand to adopt these technologies. Much research still is needed to determine which tools should be used, which need further refinement, and which should be abandoned.
Findings have shown low-fidelity simulators to be useful in helping trainees acquire psychomotor skills. The Fundamentals of Laparoscopic Surgery (FLS) curriculum and test are based on a low-fidelity benchtop model of laparoscopic surgery [2]. The FLS program must be completed for certification by the American Board of Surgery [3].
High-fidelity virtual reality (VR) platforms offer several advantages over low-fidelity models [4]. One aspect offered by low-fidelity models that many VR simulators lack is tactile sensation. Minimally invasive VR simulators have been developed with simulated force feedback, adding approximately $20,000 to $30,000 to the price [5, 6]. Despite the high additional cost, few studies have addressed the added value of this expensive feature.
Haptics is defined as the science of applying tactile sensation and control to interaction with the environment [7]. Haptic systems have been developed for robotic surgery applications and for minimally invasive surgical simulators. In minimally invasive surgery, surgeons derive most of their sensory input from visual cues [8]. The tactile sensation felt by the operator in laparoscopic surgery is not an accurate representation of the instrument interacting with tissue [9, 10]. Despite this inaccurate representation, many believe that haptic-enabled simulators offer training and performance benefits [8, 11].
Two recent studies evaluated trainee performance on VR simulated tasks with and without haptics engaged [5, 6]. The results were mixed, showing a benefit in only one task out of the combined six studied. The two studies both involved performance tests with untrained novices.
Ultimately, novice performance on a simulated task is not as critical to a simulator as the acquisition of skills through training. Therefore, we set out to determine the added benefit of haptics in the acquisition of laparoscopic skills. We performed a randomized, controlled trial to determine the effect of haptics on training and performance with a VR simulator.
Materials and methods
In our investigation of haptics in virtual laparoscopic cholecystectomy training, the LapMentor II (Simbionix, Cleveland, OH, USA) VR simulator was used exclusively. The LapMentor II is a second-generation laparoscopic VR simulator with simulated, computer-controlled, motorized force feedback. It has training modules for basic tasks, procedural tasks, and ability to perform full virtual procedures.
Expert proficiency data for the LapMentor II were published by Stefanidis et al. [12] in 2008. These criteria were established by averaging 10 repetitions of expert fellows and attending surgeons who had each performed more than 100 laparoscopic procedures (personal communication).
The current research was approved by the University of Cincinnati Institutional Review Board and by the United States Army Human Research Protection Office. The study enrolled 33 novice laparoscopic undergraduate, medical, and graduate students. Many of the participants could not finish the study due to repeated machine failures.
Each participant was randomly assigned with equal probability to one of three study groups: control, haptics-trained, or non-haptics-trained group. All the participants were given an educational presentation consisting of an introduction to laparoscopic surgery, laparoscopic cholecystectomy, and the LapMentor II. In addition, all the students completed a prequestionnaire before data collection. The prequestionnaire included questions pertinent to hand–eye coordination (musical instrument skill level), psychomotor skills (video gaming experience), and demographic information.
Students assigned to the haptics- and non-haptics-trained groups were required to achieve expert proficiency in a series of basic laparoscopic tasks. The proficiency levels used were taken from previously published expert values (Table 1).
The basic tasks were nine laparoscopic tasks given by the Basic Task Module of the LapMentor II: camera navigation 0º, camera navigation 30º, hand–eye coordination, clip application, grasp and clip, ball drop, cutting, cautery application, and object translocation. The haptics group performed all nine basic tasks to proficiency with the haptics feature of the LapMentor II engaged, whereas the nonhaptics group reached proficiency with haptics disengaged. Students assigned to the control group performed the nine basic tasks in the series with haptics engaged but completed each task only one time, with no proficiency level required.
Completion of the basic tasks was organized into one or two 4-h sessions (Fig. 1). Each session with the simulator was proctored by nonclinical research team members with knowledge of the interface and modules. For the haptics and nonhaptics groups, the proctors confirmed that the subject had reached proficiency in all tasks before allowing the subject to proceed to procedural training and assessment.
After completing the nine basic tasks, all three groups were required to complete four tasks of the LapMentor II Procedural Task Module one time. These four tasks broke the full laparoscopic cholecystectomy into four simple tasks to help the students learn the anatomy of the full procedure and how to apply the skills practiced during the basic tasks. These four procedural tasks were (1) clipping and cutting the cystic duct and artery of the retracted gallbladder, (2) retracting the gallbladder with the left hand while clipping and cutting with the right hand, (3) dissecting the cystic duct and artery to achieve the “critical view,” and (4) separating the gallbladder from the liver bed.
The nonhaptics group performed all these tasks with haptics disengaged to maintain the trainees as naive for testing the potential benefit of haptics in performance. The control and haptics groups performed these tasks with haptics engaged.
To test performance between groups, all the participants completed 10 VR laparoscopic cholecystectomies. The haptics feature of the LapMentor II was toggled on and off, alternating after each completed full procedure, with haptics engaged for attempts 1, 3, 5, 7, and 9. The LapMentor II automatically collected all the data used in the analysis.
We chose to compare instrument movement and efficiency parameters because the cognitive safety components of training were not stressed in the study. The parameters compared were total time, efficiency of cautery, number of movements, total path length, and speed of instruments. All the participants who completed the study were given a questionnaire to assess their attitudes toward the simulator.
The haptics and nonhaptics groups were compared on the number of trials needed to reach the criteria (Table 1) for the basic skills tasks using Wilcoxon rank sum tests. To test performance, each procedural parameter was modeled individually. The p values were derived from mixed models treating subjects as random within fixed groups. Repeated measures effects were modeled for haptics engaged or disengaged, and to adjust for learning, we used the log of the attempt number (1 to 10). An additional set of models added the interaction between groups and the engagement or disengagement of haptics. All reported p values are two-tailed, and alpha for all tests was a two-tailed 0.05 unadjusted for multiple tests.
Results
The control, haptics, and nonhaptics groups did not differ significantly in terms of sex, education level, hand dominance, video game experience, surgical experience, and nonsurgical simulator experience. All the groups were predominantly right-handed. All 33 participants were included in this analysis.
More attrition occurred in the haptics and nonhaptics groups than in the control group. In the haptics group, 12 trainees started the study. Six trainees completed the proficiency training, and four completed the procedural assessment. The two who did not compete the procedural training were available, but the simulator failed.
In the nonhaptics group, 10 started the study, with 4 completing the proficiency training and procedural assessment. In the control group, 11 started the study, and 10 completed the procedural assessment. According to the proctors, it took more sessions and total time commitment for the students to complete proficiency-based training in the haptics and nonhaptics groups than in the control group. Simulator failures were common, contributing to scheduling difficulties with the participants. Only one participant explicitly quit because he thought he would not reach the task proficiency level (task 3 after 112 attempts).
With regard to training, the haptics and nonhaptics groups differed significantly in the number of attempts required to reach proficiency with any of the nine basic tasks (Table 1). Regarding performance, we compared the means across all 10 trials to determine differences between the groups (Table 2), with p values derived from the mixed models described earlier (without interactions). The haptics and nonhaptics groups showed no statistically significant differences. The haptics and control groups differed significantly in mean number of left instrument movements (haptics, 98 ± 39 vs. control, 174 ± 92; p = 0.03) and mean path length of the left instrument (haptics, 179 ± 76 vs. control, 353 ± 218; p = 0.04).
There was a trend toward significance in total time (haptics, 440 ± 114 vs. control, 553 ± 233; p = 0.1). There was a statistically significant difference between the performance of the nonhaptics and the control groups in total time (nonhaptics, 376 ± 116 s vs. control, 553 ± 233 s; p = 0.02), number of left instrument movements (nonhaptics, 95 ± 39 vs. control, 174 ± 92; p = 0.02), and path length of the left instrument (nonhaptics, 177 ± 99 cm vs. control, 353 ± 218 cm; p = 0.04). There was a trend toward significance in efficiency of cautery (nonhaptics, 72 ± 8% vs. control, 59 ± 15%; p = 0.06).
Using the same statistical model, we compared the performance of haptic cholecystectomies and nonhaptic cholecystectomy trials (Table 3). In combining all groups, we found a negative effect of haptics, increasing the number of right instrument movements (estimate of effect, 29; p = 0.008) while having a positive effect on the speed of the right instrument (estimate of effect, 0.2 cm/s; p = 0.0004). Engaged haptics also improved (decreased) the number of left instrument movements (estimate of effect, −14; p = 0.02).
In the models that included an interaction for group by haptics condition, the only parameter showing such an interaction was efficiency of cautery (p = 0.04), on which the haptics group performed better with haptics engaged (haptics engaged, 70 ± 7 vs. haptics disengaged, 65 ± 8), and the nonhaptics group performed better without haptics (haptics engaged, 69 ± 6 vs. haptics disengaged, 74 ± 9).
The learning curves of the parameters are shown in Fig. 2. We compared the performance over the attempts with all the groups combined and found that the log of the attempt number in our model was a statistically significant factor in total time (p < 0.0001), number of right instrument movements (p < 0.0001), number of left instrument movements (p < 0.0001), path length of the right instrument (p < 0.0001), path length of the left instrument (p < 0.0001), and speed of the right instrument (p < 0.0001). Efficiency of cautery and speed of the left instrument were not metrics that demonstrated statistically significant learning in our model.
On the poststudy questionnaire, three of four participants in the training groups could tell when the haptics was engaged, and 9 of 10 control participants could tell when haptics was enabled.
Discussion
Haptics has been incorporated into VR simulators without compelling evidence of its value in training. This is an important issue because this feature adds to both the acquisition and the maintenance costs of the simulator. There are several reasons to question the utility of haptics. First, many of the forces felt at the instrument handle are “interference” forces due to deformation of the abdominal wall and friction forces of the trocar, not “useful” forces of the instrument interacting with tissue [9]. Second, in the landmark VR-to-operating room study by Seymour et al. [13], which used the nonhaptic Minimally Invasive Surgery Trainer–Virtual Reality (MIST-VR) simulator, the findings showed improved performance in the live operative setting after nonhaptic VR training. Third, no clear benefit of haptics was shown in two recent studies evaluating the performance of novices on haptics tasks [5, 6].
In our study, the training efficiency of the haptics and nonhaptics groups did not differ. We found no differences between the groups in the number of attempts necessary to reach expert proficiency in any of the nine basic skills tasks. We trained the study groups to proficiency in the basic tasks only, making this a study of psychomotor performance rather than cognitive skill performance. All groups performed the four laparoscopic cholecystectomy procedural tasks one time. This was to introduce them to the relatively unfamiliar complex task of a full VR cholecystectomy. For this reason, we did not include safety metrics in our assessment.
Proficiency-based training in the basic tasks of the LapMentor II did produce performance gains in both the haptics and nonhaptics groups. This provides evidence that proficiency-based training on VR simulators is beneficial regardless of haptics. We did find that more attrition occurred in the haptics (n = 12 reduced to 6) and nonhaptics (n = 10 reduced to 4) groups than in the control group (n = 11 reduced to 10). We did not quantify total time nor survey those who did not complete the study because many of them could not be reached.
Many students did struggle to complete the proficiency training. We previously reported that the metrics we used may have been too difficult [14]. It was not infrequent for participants to spend two to three 4-h sessions to complete the study. Other reasons why participants did not complete the study were frequent machine failures and scheduling difficulty. These factors often overlapped and produced delays in training.
In this study, we aimed to discover the effect of haptics on the training and performance of laparoscopic novices. No statistically significant differences were observed between the haptics and nonhaptics groups in terms of training or performance, but the nonhaptics group outperformed the control group on more parameters than the haptics group. This finding suggests that the haptics feature included in the LapMentor II does not improve training.
Learning occurred over the course of the procedural performance assessment in all the parameters assessed except efficiency of cautery and speed of the left instrument. The learning curves (Fig. 2) show that the learning effect for the control group was much more pronounced than for the training groups. Total time, in particular, shows a pattern similar to the total time learning curve comparing control subjects with intermediate level and expert level groups in the Gallagher and Satava [15] assessment of MIST-VR tasks. In our study, the haptics group seemed to have more learning than the nonhaptics group just as did the intermediate group compared with the expert group in their study. This pattern was not apparent for the other parameters. From this we conclude that the learning experienced during the 10 virtual cholecystectomies was less pronounced in both training groups than in the control group, which is the same effect that operative experience has had in similar simulator trials. The learning curves show that the skill acquired in the basic task proficiency training transferred to the new task of cholecystectomy. Haptics training did not appear to show a benefit over nonhaptics training with regard to improved learning during the course of the VR cholecystectomy procedures.
We evaluated the performance of the groups with haptics engaged and disengaged during the VR cholecystectomies. To minimize mistakes in data collection, we chose to have all subjects perform their first trial with haptics engaged, alternating the haptics condition within each subject thereafter. Thus, trials without haptics (trials 2, 4, 6, 8, and 10) were conducted, on the average, with a bit more experience than the trials with haptics (trials 1, 3, 5, 7, and 9). To adjust for this difference (and to remove error variance due to a strong effect in which we were not primarily interested), we accounted for learning by including the trial number (1 to 10) in the statistical model as a one degree of freedom parametric predictor. Furthermore, we expected (and observed) a learning effect that was not linear with the trial number. Trial-to-trial improvement was most pronounced in the early trials, shrinking as more learning had taken place. To fit this process, we used the log of the trial number in the statistical model.
We are unable to draw any conclusions as to the performance benefit of haptics in our study. The performance of all the groups with haptics enabled seemed to increase the number of right instrument movements, to decrease the number of left instrument movements, and to increase the speed of the right instrument. In separating out the haptics and nonhaptics cholecystectomies in the individual groups, the only parameter that reached statistical significance was efficiency of cautery. The haptics group had improved efficiency with haptics engaged, whereas the nonhaptics group had worse efficiency of cautery with haptics engaged. We cannot clearly conclude that haptics has a positive or negative effect on performance.
We chose to assess the performance of our groups with the simulator’s VR laparoscopic cholecystectomy. We opted to do this because the haptics feature could be turned on and off, and the subjects were already familiar with the simulator. This module has shown early evidence of concurrent validity using Objective Structured Assessment of Technical Skills. The VR laparoscopic cholecystectomy procedure was shown to transfer skills to the live pig model [16].
Our study had several limitations. First, less than half of the subjects in the haptics and nonhaptics groups finished the study. Only the top performers or those with the most interest may have been self-selected. Second, the haptics feature of the LapMentor II has an experts’ rating of only 6/10 for the realism of the force feedback [17]. Thus, we were potentially testing an underdeveloped haptics system. The simulator also had frequent failures and needed multiple repairs during the data collection phase, placing delays in the training of some participants and causing others to drop out of the study. Finally, our study design partially confounded learning with the haptics on/off condition, making our estimate of the haptics effect somewhat dependent on our parameterization of the learning effect.
Despite these shortcomings, the reported data call into question the utility of haptics in minimally invasive VR training. Our data suggest that lower-cost nonhaptic VR simulators are likely to deliver better value than expensive haptic VR simulators.
References
Sutherland LM, Middleton PF, Anthony A, Hamdorf J, Cregan P, Scott D, Maddern GJ (2006) Surgical simulation: a systematic review. Ann Surg 243:291–300
Society of American Gastrointestinal and Endoscopic Surgeons FLS Trainer Box (2008). http://www.flsprogram.org/trainerbox.php. Retrieved 7 Feb 2010
The American Board of Surgery (2008) ABS to require ACLS, ATLS and FLS for general surgery certification. http://home.absurgery.org/default.jsp?news_newreqs. Retrieved 7 Feb 2010
Aggarwal R, Balasundaram I, Darzi A (2008) Training opportunities and the role of virtual reality simulation in acquisition of basic laparoscopic skills. J Surg Res 145:80–86
Panait L, Akkary E, Bell RL, Roberts KE, Dudrick SJ, Duffy AJ (2009) The role of haptic feedback in laparoscopic simulation training. J Surg Res 156:312–316
Salkini MW, Doarn CR, Kiehl N, Broderick TJ, Donovan JF, Gaitonde K (2010) The role of haptic feedback in laparoscopic training using the LapMentor II. J Endourol 24:99–102
Bholat OS, Haluck RS, Murray WB, Gorman PJ, Krummel TM (1999) Tactile feedback is present during minimally invasive surgery. J Am Coll Surg 189:349–355
Westebring-van der Putten EP, Goossens RH, Jakimowicz JJ, Dankelman J (2008) Haptics in minimally invasive surgery: a review. Minim Invasive Ther Allied Technol 17:3–16
Picod G, Jambon AC, Vinatier D, Dubois P (2005) What can the operator actually feel when performing a laparoscopy? Surg Endosc 19:95–100
van den Dobbelsteen JJ, Schooleman A, Dankelman J (2007) Friction dynamics of trocars. Surg Endosc 21:1338–1343
Schijven M, Jakimowicz J (2003) Virtual reality surgical laparoscopic simulators. Surg Endosc 17:1943–1950
Stefanidis D, Acker CE, Swiderski D, Heniford BT, Greene FL (2008) Challenges during the implementation of a laparoscopic skills curriculum in a busy general surgery residency program. J Surg Educ 65:4–7
Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, Satava RM (2002) Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 236:458–463 discussion 463–464
Thompson J, Broderick T, Doarn C, Roesch M, Henry B, Stephan Z, Davis J, Weber J, Boyden M (2010) Refined task and proficiency criteria is required for optimal virtual reality minimally invasive surgery training. J Surg Res 158:250–251
Gallagher AG, Satava RM (2002) Virtual reality as a metric for the assessment of laparoscopic psychomotor skills: learning curves and reliability measures. Surg Endosc 16:1746–1752
Lucas SM, Zeltser IS, Bensalah K, Tuncel A, Jenkins A, Pearle MS, Cadeddu JA (2008) Training on a virtual reality laparoscopic simulator improves performance of an unfamiliar live laparoscopic procedure. J Urol 180:2588–2591 discussion 2591
Ayodeji ID, Schijven M, Jakimowicz J, Greve JW (2007) Face validation of the Simbionix LAP Mentor virtual reality training module and its applicability in the surgical curriculum. Surg Endosc 21:1641–1649
Acknowledgments
We sincerely thank the data collection team including Brian Henry, MD, Mr. Zack Stephan, Ms. Janna Davis, Ms. Maria Boyden, and Mr. John Weber for their determination. Salary support for Mr. Charles Doarn and Dr. Timothy Broderick and the purchase of the Simbionix Lap Mentor II VR simulator were provided by grant W81XWH-07-2-0035 from the United States Army Medical Research and Materiel Command and the Telemedicine and Advanced Technology Research Center.
Disclosures
Timothy J. Broderick has a consulting relationship with Ethicon Endo-Surgery. Jonathan R. Thompson, Anthony C. Leonard, Charles R. Doarn, and Matt J. Roesch have no conflicts of interest or financial ties to disclose.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Thompson, J.R., Leonard, A.C., Doarn, C.R. et al. Limited value of haptics in virtual reality laparoscopic cholecystectomy training. Surg Endosc 25, 1107–1114 (2011). https://doi.org/10.1007/s00464-010-1325-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00464-010-1325-2