Abstract
Objective
Evaluation of the benefits of a virtual reality (VR) environment with a head-mounted display (HMD) for decision-making in liver surgery.
Background
Training in liver surgery involves appraising radiologic images and considering the patient’s clinical information. Accurate assessment of 2D-tomography images is complex and requires considerable experience, and often the images are divorced from the clinical information. We present a comprehensive and interactive tool for visualizing operation planning data in a VR environment using a head-mounted-display and compare it to 3D visualization and 2D-tomography.
Methods
Ninety medical students were randomized into three groups (1:1:1 ratio). All participants analyzed three liver surgery patient cases with increasing difficulty. The cases were analyzed using 2D-tomography data (group “2D”), a 3D visualization on a 2D display (group “3D”) or within a VR environment (group “VR”). The VR environment was displayed using the “Oculus Rift ™” HMD technology. Participants answered 11 questions on anatomy, tumor involvement and surgical decision-making and 18 evaluative questions (Likert scale).
Results
Sum of correct answers were significantly higher in the 3D (7.1 ± 1.4, p < 0.001) and VR (7.1 ± 1.4, p < 0.001) groups than the 2D group (5.4 ± 1.4) while there was no difference between 3D and VR (p = 0.987). Times to answer in the 3D (6:44 ± 02:22 min, p < 0.001) and VR (6:24 ± 02:43 min, p < 0.001) groups were significantly faster than the 2D group (09:13 ± 03:10 min) while there was no difference between 3D and VR (p = 0.419). The VR environment was evaluated as most useful for identification of anatomic anomalies, risk and target structures and for the transfer of anatomical and pathological information to the intraoperative situation in the questionnaire.
Conclusions
A VR environment with 3D visualization using a HMD is useful as a surgical training tool to accurately and quickly determine liver anatomy and tumor involvement in surgery.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Hepatic resections are often the only curative treatment options for malignant hepatic lesions [1]. They can be complex surgical procedures with considerable morbidity and mortality rates [2,3,4,5,6]. Thorough planning is required in a multidisciplinary team weighing radiological findings, surgical options and medical possibilities for deciding on the extent of individual resection [4, 6,7,8,9]. Training in liver surgery takes many years until an adequate level of competence is achieved [10, 11]. Decision-making in liver surgery requires detailed knowledge of the liver and vessel anatomy and its variations, as well as tumor biology and comorbidities. To determine the most beneficial approach for each patient, heterogeneous data and information from a wide range of medical disciplines must be considered [5, 6, 8, 9, 12,13,14,15]. Due to this complexity, it can be difficult for surgical novices to comprehend decision-making in hepatic surgery, which is further complicated when considering that the traditional way of determining the surgical strategy using tomography imaging data on a 2D monitor does not provide the optimal framework for decision-making in high-risk procedures and complex cases. 3D operation planning has been proven to facilitate surgical decision-making in liver surgery as it aids in identifying the unique anatomy and tumor involvement and can also assist in choosing the most adequate procedure [16,17,18,19]. In addition, many studies have shown that 3D display is superior to 2D display of tomography images in learning surgical liver anatomy [20,21,22,23,24] and can help correctly locate hepatic tumors and decide on the optimal extent of the hepatic resection [25]. The feasibility and benefit of 3D surgical planning has already been described and a number of commercial solutions are available [16, 26, 27].
Virtual reality (VR) head-mounted displays (HMD) revolutionize the way we interact with data, allowing for an immersive and intuitive way of gathering experience in surgery [19, 28]. As previously stated, decision of adequate therapy, in this case on the example of hepatic surgery, requires a synthesis of multiple heterogeneous datapoints from different medical disciplines, often presented in different forms of media, such as pictures, video, data tables, and free text among other modalities, often distributed across multiple data platforms [5, 6, 8, 9, 12,13,14,15]. This heterogeneity demands more intuitive ways of data presentation for the surgeon to make timely and correct decisions [6, 12, 29, 30]. VR may provide a new framework to combine surgery data and relevant clinical information. Surgical education for medical students and residents could also be improved by VR and 3D imaging technologies and training tools [31,32,33,34,35,36,37]. VR with HMD provides an immersive and interactive solution for individual and grouped interaction and integrated presentation of imaging and necessary clinical information [19, 29, 30, 36,37,38,39,40,41]. The necessity for tools that allow for remote interaction with patient data and with medical specialists has been increasing with the rise of specialized “expertise centres”, and the COVID-19 pandemic has only increased this demand [41,42,43,44].
The aim of this study is to explore the benefits and problems of a VR environment using a HMD as an immersive and interactive tool for training surgical novices for liver surgery and to compare it to on-screen 3D visualization and 2D-tomography data.
Material and methods
Patient cases
Four representative patient cases from the Department of General, Visceral, and Transplantation Surgery at Heidelberg University Hospital who underwent liver surgery were chosen for this study, of which one case was used as a training dataset in order to familiarize patients with their respective visualization method. Table 1 shows the patient vignette information and important radiological findings for the three patient cases used as test datasets. Additionally, participants had access to the patient’s most recent lab results. All patient data was anonymized before it was included in this study.
Imaging data and segmentation
Radiological images were anonymized and then retrieved from the Pictures Archiving and Communication System (PACS) of the Department of General, Visceral, and Transplantation Surgery at Heidelberg University Hospital in an anonymized fashion. The images complied with the Digital Imaging and Communications in Medicine (DICOM). Several open-source software applications were used for the segmentation and post-processing of the original DICOM-images. Organ surfaces were segmented semi-automatically using the Medical Imaging and Interaction Toolkit (MITK, German Cancer Research Center, Heidelberg, Germany, www.mitk.org). Vessels and bile ducts were segmented semi-automatically using ITK-snap (www.itksnap.org). Segmentations were performed using images in the portal venous phase. The segmentation of the arteries was performed using images in the arterial phase. To align the arterial with the portal venous images, they were registered using 3D Slicer (www.slicer.org). The post-processing of the mesh models was performed using MeshMixer (Autodesk, San Rafael, California, U.S.A., www.meshmixer.com). The final models were reviewed by a board-certified radiologist and by a general surgeon specialized in liver surgery (see Figs. 1, 2, 3).
Virtual reality environment
For the presentation of patient information, associated anonymized computed tomography images, and 3D-models, the developed IMHOTEP-software was used (Karlsruhe Institute for Technology, Karlsruhe, Germany, www.imhotep-medical.org) (Fig. 4) [41, 45]. The software was installed on a XMG U505 computer (Schenker Technologies GmbH, Leipzig, Germany) with Intel® Core™ i7-4790S CPU with 3.20 GHz, 16 GB Rapid Access Memory and NVIDIA® GeForce™ GTX 980 M graphic card. The immersive aspect of the operation planning was realized using the virtual reality head-mounted display Oculus Rift™ (Oculus VR LLC, Menlo Park, California, USA). The Oculus Rift™ created a stereoscopic 3D perspective through its two LCD displays. When the user changed his head position and orientation, the view on the virtual scene was changed accordingly. IMHOTEP allowed the user to view the segmented 3D data, patient information and computer tomography images in this virtual reality environment.
Interaction with the VR environment
Five Surgical views (all, tumor + vessels, tumor + arteries, tumor + veins, tumor + bile ducts) were created with predefined viewing angles, zoom and different transparency of the organs and vessels. Using these views, the users could quickly navigate to the view which was most relevant to the question at hand, or highlight vessels of interest in order to better comprehend and prepare for the individual layout of the given patient’s liver anatomy and pathology. The users could also individually use the mouse to turn and zoom the liver in free space and visualize these objects from various individual angles as well as adjust the transparency of the organs and vessels.
Study design
This was a registered prospective, single-center, three-arm, parallel-group randomized controlled study (DRKS00011000). The study was carried out in the Department of General, Visceral, and Transplantation Surgery at Heidelberg University Hospital. Medical students at Heidelberg University Medical School during their clinical years were included. All participants received a standardized introduction covering the following topics: segmental liver anatomy, standard anatomy and anomalies of the arterial, venous and portal venous system, anatomy of the bile ducts and possible resection lines for liver surgery. Participants were randomly assigned by the investigator to one of the visualization methods (2D vs. 3D vs. VR) in a 1:1:1 ratio using the closed envelope technique with a computer-generated list for randomization. The study protocol can be viewed under the following URL: https://drks.de/search/en/trial/DRKS00011000).
All participants had unlimited time to familiarize themselves with the visualization method they were randomized to. The technical aspect of familiarization was done with a fourth patient dataset that was not used otherwise. After the introduction and familiarization, each participant then evaluated imaging data and patient information for three consecutive liver cases with increasing difficulty. In the “2D”-group, participants evaluated the imaging data in sectional views on a flatscreen monitor, patient information and labs were available on a printed sheet. In the “3D”-group, participants evaluated the imaging data as a 3D-model on a flatscreen monitor, patient information and labs were available on a printed sheet. In the “VR”-group, participants evaluated the imaging data in the VR environment, patient information and labs were integrated into this environment. Participants had to answer an 11-item-questionnaire assessing liver anatomy, tumor involvement and proposed liver resection (see Online Appendix 1). Time to answer the questions was also measured. The questionnaire was developed by board-certified surgeons with a specialization in liver surgery at the Department of General, Visceral, and Transplantation Surgery at Heidelberg University Hospital. The correct answers were defined by a board-certified radiologist and by a general surgeon specialized in liver surgery from the same institution. The correct answers for each case can be found in Table 1. After the liver cases were evaluated, participants were asked to fill out an 18-item evaluation form (see Online Appendix 2) using Likert-scales, multiple choice items and free answer options to assess the satisfaction, usefulness and potential of this system. Google™ Forms (Google Inc., Mountain View, California, USA) was used for data acquisition.
The primary outcome measure was the difference in the score (sum of correct answers) as measured by the 11-item anatomy and surgical indication evaluation questionnaire. The secondary endpoints were the time it took to answer the above-mentioned questionnaire, as well as the perceived satisfaction, usefulness and potential of the evaluated visualization method as per the 18-item evaluation form.
Continuous data was assessed using descriptive parameters (mean, standard deviation, minimum, median and maximum). Categorical data was assessed using relative and absolute frequencies. A three-group analysis using a Kruskal–Wallis-Test was carried out to compare the scores between the three groups (2D vs. 3D, 2D vs. VR, 3D vs. VR). Then pairwise comparison between the groups was performed using a Mann–Whitney U-Test (2D vs. 3D, 2D vs. VR, 3D vs. VR). The same comparison was used to determine statistical differences between the liver cases. A Chi-square test was be used in the case of comparing categorical data. Graphical representations of the statistical data were added whenever appropriate. The level of significance was set to 5%. Those evaluating the outcome and assessing the statistical outcome were blinded regarding to the groups. All statistical calculations were carried out using SPSS (IBM Corporation, New York, USA) software.
Ethical approval
The study was approved by the local ethical committee in Heidelberg (S-349/2016). The trial was registered with the German Clinical Trails Register (DRKS00011000) prior to the beginning of the study. All procedures were conducted in accordance with the ethical standards of the Helsinki Declaration of 2013.
Patient data anonymization
All patient data was retrieved in an anonymized fashion before it was integrated in the IMHOTEP-software and thus included in the study.
Results
Between May and August 2016 90 medical students were recruited and participated in this study. Figure 5 shows the trail recruitment flowchart. Table 2 shows the statistical baseline data of the randomized study groups.
Results by visualization method
The VR-group and 3D-group had significantly more correct answers across all cases than the 2D group (p < 0.001). The difference in results between 3D- and VR-groups were not statistically significant (p = 0.987). The VR-group and 3D-group had significantly lower time to answer across all cases than the 2D group (p < 0.001). The difference in time to answer between 3D- and VR-groups were not statistically significant (p = 0.419). Results are summarized in Fig. 6.
Analyzed by individual patient cases, the VR-group and 3D-group had significantly more correct answers than the 2D group (p < 0.001 for each patient). The difference in results between 3D- and VR-groups were not statistically significant in any patient case (p = 0.994 for patient 1, p = 0.827 for patient 2, p = 0.908 for patient 3). The VR-group and 3D-group had significantly lower time to answer per patient case than the 2D group (p < 0.001 for each patient). The differences in time to answer between 3D- and VR-groups were not statistically significant in any patient case (p = 0.823 for patient 1, p = 0.600 for patient 2, p = 0.315 for patient 3). Results are summarized in Fig. 6.
Learning curve analysis by intragroup comparison of correct answers
In addition to comparison across groups, the patient cases were evaluated regarding differences in correct answers and answer times for each group by consecutive patient case.
In the 2D group, the sum of correct answers was significantly higher for patient 3 compared to patient 1 (p = 0.001) and patient 2 (p = 0.008). The difference between patient 1 and 2 was not statistically significant (p = 0.619). The decision time was significantly lower for patient 3 compared to patient 1 (p < 0.001) and patient 2 (p < 0.001). The difference between patient 1 and 2 was not statistically significant (p = 0.395).
In the 3D-group, the sum of correct answers was significantly higher for patient 3 compared to patient 1 (p < 0.001) and patient 2 (p < 0.001). The difference between patient 1 and 2 was not significant (p = 0.809). The decision time was significantly lower for patient 3 compared to patient 2 (p < 0.001) and patient 1 (p < 0.001). The decision time was significantly lower for patient 2 compared to patient 1 (p = 0.006).
In the VR-group, the sum of correct answers was significantly higher for patient 3 compared to patient 1 (p < 0.001) and patient 2 (p = 0.001). The difference between patient 1 and 2 was not significant (p = 0.940). The decision time was significantly lower for patient 3 compared to patient 2 (p = 0.001) and patient 1 (p < 0.001). The decision time was significantly lower for patient 2 compared to patient 1 (p = 0.019). Results are summarized in Fig. 7.
Subjective evaluation questionnaire
A Mann–Whitney-U analysis of the subjective evaluation results showed that for all questions except “the visualization method was realistic”, the 2D group rated their visualization method significantly worse than both the 3D and VR-groups. For the question “the visualization method was realistic”, there were no statistically significant differences amongst all groups. In all evaluation questions, there were no statistically significant differences in the answers between the 3D and VR-group. The results for the subjective evaluation are presented in Fig. 8.
Discussion
A VR environment with HMD can be used by surgical novices to accurately and quickly determine surgical liver anatomy and tumor involvement in complex liver cases in order to decide on an operative strategy. In the present study, the sum of correct answers in the test was significantly higher and decision time was significantly shorter with VR and 3D compared to 2D while there no differences between 3D and VR. These results were consistent in all three patient cases. In all three study groups, average performance increased per patient case, with case three having the highest number of correct answers and lowest time to answer. In the subjective evaluation, 3D and VR were preferred over 2D in all aspects. VR was evaluated as superior for the identification of anatomic anomalies, risk and target structures and for the transfer of anatomical and pathological information to the intraoperative situation compared to 3D and 2D. While 3D and VR were evaluated as being superior to 2D in overall pleasantness, for planning standard liver resections, and in medical education and training, the overall most favored visualization method was VR.
Other studies have shown 3D visualization to be advantageous in learning surgical liver anatomy with participants in 3D groups consistently answering anatomical questions more correctly and faster than 2D-groups [20, 22,23,24,25, 37]. While the cited studies mainly asked generic questions about liver anatomy with some questions regarding resections, in the present study participants were asked to answer all relevant information needed to determine an operative strategy. Jurgaitis et al. showed that 3D visualizations improved medical students’ ability to localize hepatic tumors and correctly determine the extent of the hepatic resection [25]. The present study has shown that with the help of 3D-models, surgical novices could additionally differentiate between physiological anatomy and the pathology of a patient and make a surgical decision. The 3D visualization system may facilitate the teaching of liver anatomy and pathology and could help medical students to understand the steps in deciding on the type and extent of the hepatic resection. The positive results from previous studies [20, 22,23,24,25, 37] suggest that 3D visualization and virtual reality compare favorably or may be superior to current surgical visualization teaching methods. Continued studies, e.g. focusing on other organ systems or surgical operations, may aid in establishing VR as a valid and modern surgical teaching tool [34]. Furthermore, surgical guidance systems relying on a combination of CT imaging, 3D segmentation and augmented and virtual reality systems are being rapidly developed and becoming increasingly robust [27, 39, 46,47,48,49,50]. This should give additional weight to the argument that medical students should receive training in, and interact with these systems, particularly in surgical fields [33, 51,52,53]. Surgeons may have different opinions on resectability of liver tumors depending on their expertise and experience, but also depending on their understanding of the patients’ imaging data combined with other relevant information [54]. The IMHOTEP tool may help surgical novices better understand the differences in decision-making between different surgeons. It may also help residents more quickly acquire competency in anatomical and pathological assessment of patient data [30, 55]. All three groups in the present study scored better in consecutive cases and significantly reduced the time to answer. The improvement in both speed and accuracy in all three modalities suggests a learning curve for all visualization methods. This is emphasized by the fact that even though patient 3 was deemed the most difficult patient case by the specialists, all three groups had improved their correctness score by patient 3. This adds to the argument that students should receive frequent training in such visualization systems, and demonstrates the need for continuous re-exposure to simulated or real patient cases as one of the most effective methods of learning in the medical/surgical environment.
Another argument for the marked improvement in the 3D and VR-groups may be not only because of the 3D modality, but also due to the fact that interacting with this medium is more intuitive and thus more enjoyable [56, 57]. Participants were most satisfied with VR and the VR-group was least likely to prefer a different visualization method, which again points to VR being an effective medium for keeping the subject engaged with the material, thus facilitating the learning process [40]. This satisfaction with VR training has been noted in a previous publication by Nickel et al. [58]. Pleasantness in both the 3D- and VR-group in the present study was rated very highly. However, for some users, motion sickness can be a problem arising with the use of the VR-glasses [59]. In studies examining this phenomenon, the incidence of motion sickness varies greatly and is dependent on a variety of factors, including individual susceptibility to motion sickness, duration of exposure, postural variation (standing/sitting), actions being performed, and visual motion stimulus, i.e. simulated displacement or simulated motion of the “own” virtual body [60]. The configuration least likely to cause motion sickness appears to be a short exposure time, sitting in place, and no visual motion stimuli, i.e. the virtual avatar remaining in place in concurrence with the user. As the IMHOTEP tool lacks any reason for the user to be exposed to postural variation or visual motion stimulus, the risk of motion sickness can be kept at a minimum. Variability between HMD systems has also been reported [61]. For wearers of glasses the use of VR-glasses may be awkward and make the experience less pleasant. This will likely be corrected by future improved designs of VR-glasses that accommodate for wearers of glasses.
One can argue that with better training of surgical liver anatomy and surgical decision-making, 2D radiological images might be evaluated better after a 3D/VR training period. Surgical topography can be difficult to present and visualize for novice surgeons and medical students, and the addition of 3D visualization of complex anatomy has been shown to improve learning speed and retention when compared to traditional 2D methods [20, 22, 24, 25, 62]. Surgical novices might, for example, improve their reading of CT-images if a correlation between the 3D-model and the 2D images is implemented in the software. Metzler et al. showed that training purely with 3D does not directly transfer to enhance the understanding of 2D CT-images in students [23]. However, the combination of conventional 2D images with simultaneous 3D or VR models may enable a better transfer of understanding. Further integration of conventional 2D-imaging into the IMHOTEP VR software, in order to tie it more closely to the 3D-model, is being planned and its effects will be evaluated in a future study.
The process of image segmentation is a bottleneck for 3D visualizations. In the present study, only open-source segmentation tools with semi-automatic algorithms were used for segmentation. The tools provided accurate results but the process of segmentation was time-consuming, amounting to several hours per patient case, and required board-certified surgeons and radiologists to verify the results. There are commercial segmentation services that can be used to create 3D-models from radiological images [63]. As these services continue to improve and become more commercially viable, VR integration of preoperative planning may find increasing relevance in the clinical setting.
The presented study only used standard clinical imaging modalities, freely downloadable open-source software and commercially available hardware thus enabling a cost effective and easy reproducibility. The imaging data used for this study was computed tomography images, which are generally available for surgical oncologic resection planning. The software used for creating and post-processing the segmentations and 3D-models is open-source software and can be downloaded freely (https://www.mitk.org, www.slicer.org, www.meshmixer.com). The source code for the virtual reality visualization software IMHOTEP can be downloaded freely (http://imhotep-medical.org/) and further developed as needed. A final consideration to be given to the VR environment is the aspect of telemedicine and telecommunication. As the years since the onset of the COVID-19 pandemic has demonstrated, there is an increasing demand for viable methods of digital communication at a distance, both at work and for social reasons [42, 64]. In light of the shift in many fields to remote work, and the evidence showing that this has not resulted in a loss of productivity, it appears likely that telework will remain attractive for many employers and employees even after cessation of current pandemic restrictions. Aside from the COVID-19 pandemic [44], the need for accurate telecommunication has been increasing in the medical field over the past decade [43, 65], as expert consultations e.g. in multicenter tumor boards become increasingly common, and VR solutions have been proposed for many medical applications, including remote bedside consultation, tumor board discussion and surgical planning and intraoperative guidance systems [66,67,68,69]. Especially in complex cases such as surgical liver planning, the medium should allow for accurate communication and interpretation of information, even in a remote setting. In such cases, an interactive VR platform could allow for precise discussions, for example regarding tumor location and resection possibilities, with reduced risk for error.
Limitations
The aim of the presented study was to evaluate and explore the benefit and problems of a VR environment for training of surgical novices for liver surgery. We interpret the results that with the correct way of visualizing the clinical data even surgical novices can determined the correct liver anatomy and thus have the basis to making a decision on the right liver resection. The findings of this study do not serve to validate this tool as a surgical planning tool and cannot be generalised to surgical practitioners. The creation of the three-dimensional images by segmentation of the underlying computed tomography images is still a time-consuming process. Especially creating surgically and radiologically accurate 3D-models remains a process based on the surgical and radiological expertise and needs the validation of these experts.
Conclusion
The findings in the present study demonstrate that three-dimensional VR visualization is a valid and viable tool for teaching surgical liver anatomy. The VR environment was preferred over the other methods by the participants and it added more enjoyment to the learning process and may thus help create a better learning effect. VR and 3D display of patient anatomy is useful for training of liver surgery for surgical novices, enabling quicker and more accurate assessment of unique patient cases and allowing for improved surgical decision-making compared to 2D display.
References
Shin DS, Ingraham CR, Dighe MK, Wang C, Vaidya S, Moshiri M et al (2014) Surgical resection of a malignant liver lesion: what the surgeon wants the radiologist to know. Am J Roentgenol 203(1):W21–W33
Bramhall SR, Gur U, Coldham C, Gunson BK, Mayer AD, McMaster P et al (2003) Liver resection for colorectal metastases. Ann R Coll Surg Engl 85(5):334–339
Jaeck D, Bachellier P, Guiguet M, Boudjema K, Vaillant JC, Balladur P et al (1997) Long-term survival following resection of colorectal hepatic metastases. Association Francaise de Chirurgie. Br J Surg 84(7):977–980
Gobel T, Graf D, Hosnowsky C, Blondin D, Knoefel WT, Haussinger D et al (2017) Multimodal and sequential treatment improves survival in patients with hepatocellular carcinoma. Z Gastroenterol 55(3):251–259
Morise Z (2019) Developments and perspectives of laparoscopic liver resection in the treatment of hepatocellular carcinoma. Surg Today 49(8):649–655
Gobel T, Blondin D, Kolligs F, Bolke E, Erhardt A (2013) Current therapy of hepatocellular carcinoma with special consideration of new and multimodal treatment concepts. Dtsch Med Wochenschr 138(27):1425–1430
Fan MH, Chang AE (2002) Resection of liver tumors: technical aspects. Surg Oncol 10(4):139–152
Simo KA, Halpin LE, McBrier NM, Hessey JA, Baker E, Ross S et al (2016) Multimodality treatment of intrahepatic cholangiocarcinoma: a review. J Surg Oncol 13(1):62–83
Weiss MJ, Cosgrove D, Herman JM, Rastegar N, Kamel I, Pawlik TM (2014) Multimodal treatment strategies for advanced hilar cholangiocarcinoma. Langenbecks Arch Surg 399(6):679–692
Sheikh MR, Osman H, Butt MU, Jeyarajah DR (2016) Perception of training in hepatopancreatobiliary surgery among general surgery residents in the Americas. HPB (Oxford) 18(12):1039–1045
Friedell ML, VanderMeer TJ, Cheatham ML, Fuhrman GM, Schenarts PJ, Mellinger JD et al (2014) Perceptions of graduating general surgery chief residents: are they confident in their training? J Am Coll Surg 218(4):695–703
Marz K, Hafezi M, Weller T, Saffari A, Nolden M, Fard N et al (2015) Toward knowledge-based liver surgery: holistic information processing for surgical decision support. Int J Comput Assist Radiol Surg 10(6):749–759
Rampone B, Schiavone B, Confuorto G (2010) Current management of hepatocellular cancer. Curr Oncol Rep 12(3):186–192
Morise Z, Kawabe N, Tomishige H, Nagata H, Kawase J, Arakawa S et al (2014) Recent advances in liver resection for hepatocellular carcinoma. Front Surg 1:21
Soares KC, Kamel I, Cosgrove DP, Herman JM, Pawlik TM (2014) Hilar cholangiocarcinoma: diagnosis, treatment options, and management. Hepatobiliary Surg Nutr 3(1):18–34
Radtke A, Sotiropoulos GC, Molmenti EP, Schroeder T, Peitgen HO, Frilling A et al (2010) Computer-assisted surgery planning for complex liver resections: when is it helpful? A single-center experience over an 8-year period. Ann Surg 252(5):876–883
Oldhafer KJ, Stavrou GA, Prause G, Peitgen HO, Lueth TC, Weber S (2009) How to operate a liver tumor you cannot see. Langenbeck’s Arch Surg 394(3):489–494
Hallet J, Gayet B, Tsung A, Wakabayashi G, Pessaux P, International Consensus Conference on Laparoscopic Liver Resection Group (2015) Systematic review of the use of pre-operative simulation and navigation for hepatectomy: current status and future perspectives. J Hepatobiliary Pancreat Sci 22(5):353–362
Mise Y, Tani K, Aoki T, Sakamoto Y, Hasegawa K, Sugawara Y et al (2013) Virtual liver resection: computer-assisted operation planning using a three-dimensional liver representation. J Hepatobiliary Pancreat Sci 20(2):157–164
Nickel F, Hendrie JD, Bruckner T, Kowalewski KF, Kenngott HG, Muller-Stich BP et al (2016) Successful learning of surgical liver anatomy in a computer-based teaching module. Int J Comput Assist Radiol Surg 11(12):2295–2301
Monch J, Muhler K, Hansen C, Oldhafer KJ, Stavrou G, Hillert C et al (2013) The LiverSurgeryTrainer: training of computer-based planning in liver resection surgery. Int J Comput Assist Radiol Surg 8(5):809–818
Muller-Stich BP, Lob N, Wald D, Bruckner T, Meinzer HP, Kadmon M et al (2013) Regular three-dimensional presentations improve in the identification of surgical liver anatomy—a randomized study. BMC Med Educ. https://doi.org/10.1186/1472-6920-13-131
Metzler R, Stein D, Tetzlaff R, Bruckner T, Meinzer HP, Buchler MW et al (2012) Teaching on three-dimensional presentation does not improve the understanding of according CT images: a randomized controlled study. Teach Learn Med 24(2):140–148
Beermann J, Tetzlaff R, Bruckner T, Schoebinger M, Muller-Stich BP, Gutt CN et al (2010) Three-dimensional visualisation improves understanding of surgical liver anatomy. Med Educ 44(9):936–940
Jurgaitis J, Paskonis M, Pivoriunas J, Martinaityte I, Juska A, Jurgaitiene R et al (2008) The comparison of 2-dimensional with 3-dimensional hepatic visualization in the clinical hepatic anatomy education. Medicina (Kaunas) 44(6):428–438
Radtke A, Bockhorn M, Schroeder T, Lang H, Paul A, Nadalin S et al (2006) Computer-assisted planning in living donor liver operation. Zentralbl Chir 131(1):69–74
Kenngott HG, Wagner M, Gondan M, Nickel F, Nolden M, Fetzer A et al (2014) Real-time image guidance in laparoscopic liver surgery: first clinical experience with a guidance system based on intraoperative CT imaging. Surg Endosc 28(3):933–940
Fondation MOVEO (2016) Fondation MOVEO—La réalité virtuelle au service du savoir des chirurgiens. http://www.fondation-moveo.fr/projets/realite-virtuelle/. Accessed 7 Aug 2020
Condino S, Carbone M, Ferrari V, Faggioni L, Peri A, Ferrari M et al (2011) How to build patient-specific synthetic abdominal anatomies. An innovative approach from physical toward hybrid surgical simulators. Int J Med Robot 7(2):202–213
Soler L, Marescaux J (2008) Patient-specific surgical simulation. World J Surg 32(2):208–212
Kowalewski KF, Garrow CR, Schmidt MW, Benner L, Muller-Stich BP, Nickel F (2019) Sensor-based machine learning for workflow detection and as key to detect expert level in laparoscopic suturing and knot-tying. Surg Endosc 33:3732
Kowalewski KF, Hendrie JD, Schmidt MW, Garrow CR, Bruckner T, Proctor T et al (2017) Development and validation of a sensor- and expert model-based training system for laparoscopic surgery: the iSurgeon. Surg Endosc 31(5):2155–2165
Kenngott HG, Wunscher JJ, Wagner M, Preukschas A, Wekerle AL, Neher P et al (2015) OpenHELP (Heidelberg laparoscopy phantom): development of an open-source surgical evaluation and training tool. Surg Endosc 29(11):3338–3347
Nickel F, Brzoska JA, Gondan M, Rangnick HM, Chu J, Kenngott HG et al (2015) Virtual reality training versus blended learning of laparoscopic cholecystectomy: a randomized controlled trial with laparoscopic novices. Medicine (Baltimore) 94(20):e764
Schmidt MW, Kowalewski KF, Schmidt ML, Wennberg E, Garrow CR, Paik S et al (2019) The Heidelberg VR Score: development and validation of a composite score for laparoscopic virtual reality training. Surg Endosc 33(7):2093–2103
Viglialoro R, Esposito N, Condino S, Cutolo F, Guadagni S, Gesi M et al (2018) Augmented reality to improve surgical simulation. Lessons learned towards the design of a hybrid laparoscopic simulator for cholecystectomy. IEEE Trans Biomed Eng 7:2091
Barsom EZ, Graafland M, Schijven MP (2016) Systematic review on the effectiveness of augmented reality applications in medical training. Surg Endosc 30(10):4174–4183
Mersmann S, Seitel A, Erz M, Jahne B, Nickel F, Mieth M et al (2013) Calibration of time-of-flight cameras for accurate intraoperative surface reconstruction. Med Phys 40(8):082701
Kenngott HG, Wagner M, Nickel F, Wekerle AL, Preukschas A, Apitz M et al (2015) Computer-assisted abdominal surgery: new technologies. Langenbecks Arch Surg 400(3):273–281
Friedrich M, Bergdolt C, Haubruck P, Bruckner T, Kowalewski KF, Muller-Stich BP et al (2017) App-based serious gaming for training of chest tube insertion: study protocol for a randomized controlled trial. Trials 18(1):56
Kenngott HG, Pfeiffer M, Preukschas AA, Bettscheider L, Wise PA, Wagner M et al (2022) IMHOTEP: cross-professional evaluation of a three-dimensional virtual reality system for interactive surgical operation planning, tumor board discussion and immersive training for complex liver surgery in a head-mounted display. Surg Endosc 36(1):126–134
Bartik A, Cullen Z, Glaeser E, Luca M, Stanton C (2020) What jobs are being done at home during the COVID-19 crisis? Evidence from firm-level surveys. Natl Bureau Econ Res. https://doi.org/10.3386/w27422
Hollander JE, Carr BG (2020) Virtually perfect? Telemedicine for Covid-19. New Engl J Med 382(18):1679–1681
Patel SY, Mehrotra A, Huskamp HA, Uscher-Pines L, Ganguli I, Barnett ML (2020) Trends in outpatient care delivery and telemedicine during the COVID-19 pandemic in the US. JAMA Intern Med. https://doi.org/10.1001/jamainternmed.2020.5928
Pfeiffer M, Kenngott H, Preukschas A, Huber M, Bettscheider L, Muller-Stich B et al (2018) IMHOTEP: virtual reality framework for surgical applications. Int J Comput Assist Radiol Surg 13(5):741–748
Vadala G, De Salvatore S, Ambrosio L, Russo F, Papalia R, Denaro V (2020) Robotic spine surgery and augmented reality systems: a state of the art. Neurospine 17(1):88–100
Lee D, Yu HW, Kim S, Yoon J, Lee K, Chai YJ et al (2020) Vision-based tracking system for augmented reality to localize recurrent laryngeal nerve during robotic thyroid surgery. Sci Rep 10(1):8437
Lave A, Meling TR, Schaller K, Corniola MV (2020) Augmented reality in intracranial meningioma surgery: a case report and systematic review. J Neurosurg Sci 64:369
Hussain R, Lalande A, Guigou C, Bozorg-Grayeli A (2020) Contribution of augmented reality to minimally invasive computer-assisted cranial base surgery. IEEE J Biomed Health Inform 24(7):2093–2106
Lang H, Huber T (2020) Virtual and augmented reality in liver surgery. Ann Surg 271(1):e8
Rizzetto F, Bernareggi A, Rantas S, Vanzulli A, Vertemati M (2020) Immersive virtual reality in surgery and medical education: diving into the future. Am J Surg 4:856
Barber SR, Jain S, Mooney MA, Almefty KK, Lawton MT, Son YJ et al (2020) Combining stereoscopic video and virtual reality simulation to maximize education in lateral skull base surgery. Otolaryngol Head Neck Surg 162(6):922–925
Graafland M, Schraagen JM, Schijven MP (2012) Systematic review of serious games for medical education and surgical skills training. Br J Surg 99(10):1322–1330
Mohammad WM, Martel G, Mimeault R, Fairfull-Smith RJ, Auer RC, Balaa FK (2012) Evaluating agreement regarding the resectability of colorectal liver metastases: a national case-based survey of hepatic surgeons. HPB (Oxford) 14(5):291–297
Mylopoulos M, Regehr G, Ginsburg S (2011) Exploring residents’ perceptions of expertise and expert development. Acad Med 86(10 Suppl):S46–S49
Haubruck P, Nickel F, Ober J, Walker T, Bergdolt C, Friedrich M et al (2018) Evaluation of app-based serious gaming as a training method in teaching chest tube insertion to medical students: randomized controlled trial. J Med Internet Res 20(5):e195
Kowalewski KF, Hendrie JD, Schmidt MW, Proctor T, Paul S, Garrow CR et al (2017) Validation of the mobile serious game application Touch Surgery for cognitive training and assessment of laparoscopic cholecystectomy. Surg Endosc 31(10):4058–4066
Nickel F, Bintintan VV, Gehrig T, Kenngott HG, Fischer L, Gutt CN et al (2013) Virtual reality does not meet expectations in a pilot study on multimodal laparoscopic surgery training. World J Surg 37(5):965–973
Munafo J, Diedrick M, Stoffregen TA (2016) The virtual reality head-mounted display Oculus Rift induces motion sickness and is sexist in its effects. Exp Brain Res. https://doi.org/10.1007/s00221-016-4846-7
Saredakis D, Szpak A, Birckhead B, Keage HAD, Rizzo A, Loetscher T (2020) Factors associated with virtual reality sickness in head-mounted displays: a systematic review and meta-analysis. Front Hum Neurosci 14:96
Guna J, Geršak G, Humar I, Krebl M, Orel M, Lu H et al (2019) Virtual reality sickness and challenges behind different technology and content settings. Mobile Netw Appl. https://doi.org/10.1007/s11036-019-01373-w
Torjesen I (2015) New 3D virtual human model aims to transform surgical training. BMJ. https://doi.org/10.1136/bmj.h6071
Newe A, Becker L, Schenk A (2014) Application and evaluation of interactive 3D PDF for presenting and sharing planning results for liver surgery in clinical routine. PLoS ONE 9(12):e115697
Gabbiadini A, Baldissarri C, Durante F, Valtorta RR, De Rosa M, Gallucci M (2020) Together apart: the mitigating role of digital communication technologies on negative affect during the COVID-19 outbreak in Italy. Front Psychol. https://doi.org/10.3389/fpsyg.2020.554678
Dorsey ER, Topol EJ (2020) Telemedicine 2020 and the next decade. Lancet (London, England) 395(10227):859
Zhang J, Gao F, Ye Z (2020) Remote consultation based on mixed reality technology. Global Health J 4(1):31–32
Li J, Chen G, Ridder HD, Cesar P (2020) Designing a social VR clinic for medical consultations. Extended abstracts of the 2020 CHI conference on human factors in computing systems. Association for Computing Machinery, Honolulu, pp 1–9
Moglia A, Ferrari V, Morelli L, Ferrari M, Mosca F, Cuschieri A (2016) A systematic review of virtual reality simulators for robot-assisted surgery. Eur Urol 69:1065
Kim H-J, Jo Y-J, Choi J-S, Kim H-J, Park I-S, You J-S et al (2020) Virtual reality simulation and augmented reality-guided surgery for total maxillectomy: a case report. Appl Sci 10(18):6288
Acknowledgements
The present study was conducted within the setting of the Collaborative Research Center 125: Cognition Guided Surgery funded by the German Research Foundation (DFG).
Funding
Open Access funding enabled and organized by Projekt DEAL. This work was funded by Deutsche Forschungsgemeinschaft (Grant No. SFB 125).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Disclosures
Anas Amin Preukschas, Lisa Bettscheider, Martin Wagner, Matthias Huber, Arianeb Mehrabi, Stefanie Speidel, Thilo Hackert, Beat Peter Müller-Stich, Felix Nickel, Hannes Götz Kenngott received funding from the German Research Foundation (DFG) within the setting of the Collaborative Research Center 125: Cognition Guided Surgery. Philipp Anthony Wise, Micha Pfeiffer, Mohammad Golriz, Lars Fischer and Fabian Rössler have no conflicts of interest or financial ties to disclose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Preukschas, A.A., Wise, P.A., Bettscheider, L. et al. Comparing a virtual reality head-mounted display to on-screen three-dimensional visualization and two-dimensional computed tomography data for training in decision making in hepatic surgery: a randomized controlled study. Surg Endosc 38, 2483–2496 (2024). https://doi.org/10.1007/s00464-023-10615-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00464-023-10615-8