Extensive practice is of major importance to becoming a skilled laparoscopic surgeon. Traditionally, surgeons have gained their skills hands-on in the operating room under the supervision of expert surgeons. The skills needed for laparoscopic surgery are unique and differ from those used during open surgery and therefore necessitate a different training approach. Preclinical practice using inanimate models such as box trainers or virtual reality (VR) simulators increases psychomotor skills and translates into improved performance in the operating room [1]. For this reason, the use of simulation in surgical training curricula is becoming more widely accepted. Objective assessment of performance is fundamental to provide formative feedback during training, allowing for continuous skill refinement. Traditional box trainers have been criticized for being unrealistic in presented exercises and for the lack of any form of objective assessment [2]. However, most VR simulators do provide objective assessment and feedback, but lack haptic feedback.

A number of studies [3–5] have shown that haptic feedback is of importance for adequate laparoscopic training, in particular for laparoscopic suturing. The term ‘haptic feedback’ refers to the combination of tactile feedback through sensory skin receptors, and kinesthetic feedback through muscle, tendons, and joint sensory receptors. Especially for laparoscopic suturing skills it is important that the trainees experience haptic feedback when performing the task. Several key aspects of learning these skills involve refined interaction with tissue and suturing materials, which most likely demands a more sophisticated level of feedback than can be obtained by visual feedback alone. Haptic feedback results in significantly improved skills transfer to the trainee, compared with training without haptic feedback [2]. In general, it is assumed that high-fidelity simulations with haptic feedback imply better training effects and a better transfer of skills to the clinical setting [6]. However, realistic haptic feedback during laparoscopic training is currently lacking in VR simulators.

The degree of realism of a simulation varies depending on the hardware and software capabilities of the simulator [7]. Because of limited computing power, most VR simulator systems generally only represent a part of the physical environment. This means that certain limitations have to be accepted for any simulation (e.g., suturing and knot tying). If accepted, this results in simplified representations of organs, which do not behave like they do in the clinical setting (e.g., inadequate haptic feedback, limited visual details, etc.) [7].

Various studies [1, 7–11] have been performed comparing traditional box trainers with VR simulators for laparoscopic training, most of which focused on basic laparoscopic skills. Only the study by Avgerinos et al. concluded that for the intracorporeal knot-tying task there was no statistically significant difference in the score measures between the two simulator systems [8]. They all additionally conclude that further studies are needed to establish which simulation technique is more effective for training [1, 7, 9–11].

At this point, the available studies do not provide similar results, and also have not shown significant differences between the two laparoscopic training approaches [7, 10, 11]. This study therefore focuses on the importance of haptic feedback and the additive value of virtual reality simulation in laparoscopic suturing training versus traditional box trainers.

Materials and methods

This study took place during several laparoscopic skills courses in the Catharina Hospital Eindhoven, the Netherlands, and the Academic Hospital of Al Ain, United Arab Emirates, from November 2006 till January 2007. The study has two arms: one investigating the opinions of 45 persons attending five laparoscopic skills courses, and another testing the performance of trainees (N = 20) on laparoscopic suturing after different types of training. For the study we used traditional box trainers, VR simulators, and questionnaires.

Equipment

Two types of laparoscopy simulators were used in this study: VR simulators and traditional box trainers.

Traditional box trainer

The traditional box trainers we used (Fig. 1) are composed of a box inside which the laparoscopic suturing task are performed on a foam suturing pad (Limbs & Things Ltd., Bristol, UK), a Telepac system (Karl-Storz Endoscope, Tuttlingen, Germany), and an additional video monitor (Neovo X17a). The endoscope, camera, and laparoscopic needle holders used were from Karl Storz.

Fig. 1
figure 1

Traditional box trainer, containing a foam suturing pad (Limbs & Things Limited, Bristol, United Kingdom), a Telepac system with endoscope (Karl-Storz Endoscope, Tuttlingen, Germany), video monitor (Neovo X17a), and needle holders (Karl-Storz Tuttlingen, Germany)

Full size image

VR simulators

Two SimSurgery VR simulator systems were used in this study. Both VR simulator systems ran the SimPort software (SEP 1.04.3 SimSurgery, Oslo, Norway). However, their hardware platform differed: one VR simulator system incorporated the SimPack platform (SimSurgery, Oslo, Norway) (Fig. 2), while the other system made use of two Xitact HTP instrument ports (Xitact/Mentice SA, Morges, Switzerland) (Fig. 3).

Fig. 2
figure 2

SimSurgery (SEP 1.04.3 SimSurgery, Oslo, Norway) virtual reality laparoscopic simulator with the SurgicalSim education platform (SEP) (SimSurgery, Oslo, Norway)

Full size image
Fig. 3
figure 3

Xitact HTP instrument ports (Xitact/Mentice SA, Morges, Switzerland)

Full size image

The SimPort software package used in this study contains various training modules, including exercises related to specific laparoscopic skills, techniques, or procedures. Current training modules include camera navigation, instrument manipulation, dissection, suturing, and haemostasis. For this study we only used a selection of tasks from the suturing module: ‘Two-handed stitch with traction’, ‘Realistic surgeon’s knot’, ‘Realistic interrupted suture’, and ‘Realistic free knot’ (Fig. 4). Even though the Xitact HTP instrument ports in one of the simulator systems is capable of providing haptic feedback, the SimSurgery software does not support these haptic features. The settings of the force sensors were adjusted to compensate for the forces of inserting the instruments, the tasks performed on both VR simulator systems selected for this study are therefore all without haptics.

Fig. 4
figure 4

Modules used for the suturing training during the session on the SimSurgery VR simulator in this study

Full size image

Subjects

In total 45 participants were involved in the study regarding the opinion on both simulator systems, of which 20 people participated in the Netherlands and 25 in the United Arab Emirates. Some laparoscopic experience was mandatory prior to participation to ensure a reference point to the clinical setting. Laparoscopic suturing experience was an exclusion criterion for the participation in the arm involving the assessment of the suturing and knot tying skills. The participants in the Netherlands were surgical and gynaecology residents, who participated in laparoscopic skills courses as part of their residency. The participants from the United Arab Emirates also participated in a course for laparoscopic skills.

Protocol

Before starting the training sessions on either of the simulators, all participants (N = 45) filled out the first part of the questionnaire containing questions on demographics and laparoscopic experience. A general introduction of the simulators was given, followed by a demonstration and explanation of the laparoscopic ‘surgeon’s knot’ by an expert. Next, all participants were randomly and blinded divided into two equally sized groups: group A started with a training session on the traditional box trainer for half an hour followed by a session on the SimSurgery VR simulator for half an hour; group B started with the same session on the SimSurgery VR simulator, followed by the session on the traditional box trainer (Fig. 5)

Fig. 5
figure 5

Overview of the protocol of the study of all participants (N = 45)

Full size image

Performance of suturing skills

The assessment of the performance was only tested in the Netherlands (Fig. 5); this subgroup is represented in the table as group Anl (N = 10) and group Bnl (N = 10). These participants (N = 20) accomplished the full training course, after which one ‘suturing and knot-tying task’ was assessed by an expert laparoscopist, on the traditional box trainer. The assessment of the skills was done using a standard evaluation form, which consisted of eight items scored on a five-point-Likert scale. Table 1 presents a summary of the scoring of the assessment criteria examined by the expert observer. The criteria ‘Quality (strength) of knot’ was tested by the objective observer by taking out the suturing pad with the tied knot and pulling the thread, to observe if the knot was tight and secured. The expert observers had done more than 100 clinical laparoscopic procedures and had extensive laparoscopic suturing experience. The participants of both group Anl (N = 10) and Bnl (N = 10) were equally divided between two objective expert observers, to avoid inter-examiner differences.

Table 1 Scoring of the final laparoscopic knot
Full size table

The control group was gathered from the data of the assessment of group Anl (N = 10), after the initial half-hour training session on the box trainer (Fig. 5).

Opinion of the participants

After finishing the training sessions, all participants (N = 45) were asked to fill out the questionnaire regarding their opinion on the simulators used in the study and their role in laparoscopic suturing training. The questionnaire used in this study consisted of three parts. The first part was about the demographics and prior laparoscopic and simulator experience of the trainees. In the second part, questions were asked regarding the realism and haptic feedback of both simulators. These questions were answered on a five-point Likert scale. The last part consisted of questions concerning the preferences of the trainees regarding laparoscopic suturing training.

Statistics

All data was processed and analyzed using SPSS 13.0. Significant differences in opinion regarding the two simulators were calculated with the paired t-test. The significant differences between the examined final knots were calculated with the independent-sample t-test.

Results

Demographics

All 45 participants who attended the laparoscopic skills courses completed the training sessions on both simulators and filled out the questionnaire. This overall group (N = 45) consisted of 48.9% residents of surgical specialties, 46.7% surgical specialists, and 4.4% surgical interns (in their last year of training). The participants from the Netherlands (N = 20) were surgical residents and gynaecology residents. The male:female ratio was equally divided between both groups (2:1). None of these participants had any clinical laparoscopic suturing experience. Of the 25 participants from the United Arab Emirates (N = 25), 23.3% had limited clinical experience with laparoscopic suturing, and 13.9% had more experience. All participants (N = 45) had some experience with laparoscopic procedures (32.6% had done more than 50 clinical laparoscopic procedures). Only 11.6% had prior experience (more than twice) on laparoscopic simulators such as VR systems and box trainers.

Performance of the suturing skills

Table 1 presents the scoring on the assessment criteria examined by the expert observer of the participants of the Netherlands (groups Anl, N = 10 and Bnl, N = 10). The criterion ‘Quality (strength) of knot’ for both groups Anl (mean 3.90) and Bnl (mean 3.80) was scored slightly better than the control (mean 3.60), but these differences were not significant (p = 0.563 and p = 0.641, respectively). When the scores of the eight criteria were summed, group Anl (which started on the traditional box trainer) had a mean of 30.80, group Bnl (which started on the VR simulator) of 27.60, and the control group of 28.20. Group Anl scored higher on the summer value and on most other criteria, but these differences were not significant. As shown in Table 1, only the scores on ‘Taking proper bites of the suturing pad, during suturing’, have a tendency towards a significant difference between groups Anl and Bnl (means: group Anl 4.10, group Bnl 3.60; p = 0.054).

Opinion of the participants

As presented in Table 2, there was a significant difference in opinion of the participants with regard to realism and haptic feedback between the two simulator systems used for this study. Overall, the traditional box trainer scored higher on all aspects than the SimSurgery VR laparoscopic simulator (p < 0.001).

Table 2 Opinion on laparoscopic suturing simulators
Full size table

The highest mean score for the VR simulator is given for ‘global impression’ (3.00), but this is a neutral opinion on the five-point Likert scale. This feature scored a mean 3.95 for the box trainer, which is low compared to the other ratings for this simulator (Table 2). For ‘realism of needle and thread’, more than 64% of the participants rated this feature as ‘excellent’ on the traditional box trainer. However, the VR simulator has a mean of 2.75 on this feature, with a standard deviation of 1.056. For the VR simulator all standard deviations are between 0.894 and 1.107, which implies that the opinion on the features of this simulator is divided amongst the participants. The standard deviation of the opinion of the box trainer is lower, except for ‘haptic sensation of the tissue’, for which the opinion is also more controversial. For the VR simulator, the features ‘haptic sensations of the tissue’ and ‘resistance of needle and thread’ were regarded as ‘absolutely unrealistic’ by 41.5% and 48.8% of the participants, respectively. On these same features the traditional box trainer was rated as ‘good’ by 29.8% and 42.2%, respectively, and ‘excellent’ by 33.3% and 35.6% respectively. There were no significant differences in the opinions between the participants of the Netherlands and the participants of the United Arab Emirates.

Of all the participants, 53.3% preferred to practice on the VR simulator first, followed by the traditional box trainer, while 37.8% preferred to start on the box trainer (Fig. 6). On the question of whether it is necessary to use both simulator systems for laparoscopic suturing training (Fig. 7) 53.3% of the participants believed that they were both necessary for proper training whereas 46.7% believed that the traditional box trainer alone would suffice for the training. The answers on these questions were equally divided between groups A and B. The opinion regarding these questions was not influenced by the order in which the participants followed the suturing training.

Fig. 6
figure 6

Preference of practice order: 53.3% of the participants preferred to practice on the VR (SimSurgery) simulator first and then on the traditional box trainer, while 37.8% preferred to start their training on the traditional box trainer; 8.9% did not have a preference for either order

Full size image
Fig. 7
figure 7

Questionnaire results on whether both the box training and VR module were required during the suturing training programme. The possible answers on this question were: ‘yes’, ‘only box trainer’, ‘only VR suturing module (no preference)’ or ‘only SimSurgery simulator’. The results indicated that 53.3% were of the opinion that both simulator systems were necessary, while 46.7% thought the box trainer would be sufficient on its own

Full size image

Discussion

Importance of objective assessment

To be an effective educational tool, the metrics provided by a VR simulator must provide meaningful information to the trainee [12]. Time, tip trajectory, and smoothness are often used in VR simulation and can provide a much more precise and comprehensive evaluation of basic laparoscopic skills than can be measured by timing drills in box trainers. The benefits include an objective analysis of errors and economy of motion; two parameters that cannot be assessed accurately by an observer [13]. However these metrics are not necessarily the most valuable proxies to assess the performance of the trainee. The SimSurgery VR simulator uses various measurements to assess each performance, such as time and tip trajectory (motion analysis); it additionally records various types of errors made during the performance. When the same task is performed more than once, a performance curve will be visualized for each task repetition. This can be used by the trainees and faculty to monitor their performance and progression and aim remedial training to correct specific deficiencies [13]. An essential tenet of educational theory is that learning should be accompanied by evaluation for both formative (feedback) and summative (final assessment) purposes. This feedback can be used to create dedicated learning programs to enhance skills specifically in the areas that are deficient, and to verify that a required skill level has been attained [12].

The feedback after each performance provided by most VR simulators could motivate trainees to practise their skills more extensively, until they have reached their goal. Providing feedback and setting goals tends to motivate trainees more, compared with a self-directed group [14]. With a traditional box trainer, feedback is subjective and an expert needs to be on hand to assess performance, making it more difficult to set specific goals [15].

One downfall of the assessment method used by VR simulators is that they often oversimplify and only provide feedback on the performance of the whole task. For example, when repetitions within a repetitive task are in a random order, it is possible that there is a slight difference in difficulty within the task. This variation in difficulty can make the performance scores appear lower (or higher) than the previous, because this difference in complexity is not compensated in the presented feedback. In our study, for example, we found that when a participant did not immediately grab the needle properly at the start of the procedure, it became almost impossible to complete the task. The overall performance turned out much lower, while performing in their normal performance curve, during subsequent subtasks within the exercise. A general remark of the participants was that the feedback given by the VR simulator was not representative of their skills; the participants felt that they were better evaluated by an expert observer.

The participants of this study did not appreciate the benefit of the objective feedback of the VR simulator over the benefits of the traditional box trainer. They preferred the box trainer for laparoscopic suturing training. Similar results were also found in the study of Stefanidis et al. [9], who compared a VR simulator with a box trainer. They did not find significant differences in the ratings regarding the ability to provide performance feedback to the trainee, despite the ability of VR simulator to provide additional performance metrics and read outs.

Importance of haptic feedback

A surgeon must be able to identify tissue properties and handle tissue in a safe manner [6]. Challenges facing the laparoscopic surgeon include the loss of haptic (tactile) feedback due to the interposition of long instruments between the surgeon’s hand and the tissue. To compensate for the compromised haptics, the surgeon has to rely on visual input from the operating field [9]. However, with laparoscopic surgery, there is a loss of important depth cues due to the use of a two-dimensional display monitor. Another disadvantage of laparoscopic surgery is the fulcrum effect created by the insertion of the instruments through the abdominal wall, which causes the instrument tips to move in the opposite direction to the surgeon’s hand [13]. Because of these additional mental translations and counterintuitive interactions, it is important that laparoscopic skills are practised extensively before application in the clinical setting. During laparoscopic procedures, the major part of the haptic feedback is lost. However the little haptic feedback that remains is valuable and it is therefore important that the simulator system provides haptic feedback, preferably as optimal as possible. A disadvantage of the majority of VR simulators is the absence of haptic feedback to the surgeon [13].

Shortly after its introduction, laparoscopic surgery was associated with a high complication rate. The term ‘performance curve’ was introduced to refer to the number of operations a surgeon has to perform to reach an expertise level with an acceptable complication rate [7]. Further studies showed that even experienced laparoscopic surgeons had to go through a performance curve again when they had to learn a new laparoscopic procedure or technique [7], such as laparoscopic suturing. Data from previous studies suggests that training up to a predetermined level on a box trainer suture model provides trainees with skills that translate into improved clinical performance [13, 16]. It has been reported that haptic feedback is very important in laparoscopic simulation and can shorten the first part of the performance curve [6], increasing the effectiveness of laparoscopic training [17–19]. Currently though, this is also one of the most controversial issues in VR laparoscopic simulator design, as it is very complicated to incorporate realistic haptic feedback into VR systems [17]. In particular for laparoscopic suturing training, because of the interactions between the instruments, needle, thread, and tissue, it is important that haptic sensations during the simulation is optimal, but this has not yet been achieved in VR simulation.

One advantage of the traditional box trainer over most VR simulators is that it provides the haptic feedback that is absent in most VR systems. Additionally, it allows the trainee to use whatever instruments he or she prefers to use in the operating room, and they are less expensive [12]. While VR trainers may have some advantages, most participants of studies on this topic feel that traditional box trainers help them more, are more interesting, and are preferred over VR trainers if only one trainer is allowed [20]. This opinion is supported by the outcome of our study, as shown in Fig. 7. The participants favored the box trainers, which involved the use of real needle drivers and interaction with real needle, thread, and suturing pad, over the suturing module of the VR simulator. As shown in Table 2, the traditional box trainer is rated higher than the SimSurgery VR simulator on both realism and haptic features.

It is important to note that our results do not imply that VR simulator systems are not suitable for training in laparoscopy in general, or for basic skills or component tasks of procedural training in laparoscopic training. Previous studies have proven the value of VR simulation without haptics for several types of laparoscopy training [2, 4, 21, 22]. This study does show however that, for laparoscopic suturing, haptic feedback is considered a necessity, and that training on box trainers is preferable over VR simulator systems, until these are also capable to provide sufficient haptic feedback.

There were no significant differences between the ratings of the assessment after laparoscopic suturing training on only a traditional box trainer (control) or the combination with a VR simulator (groups Anl and Bnl); neither could we find an optimal order in which the training should be followed to master the laparoscopic suturing skills. Group Anl, which started on the box trainer followed by the VR simulator, had the highest summed score, but it was still not significantly higher than either group Bnl or the control group. From these results we can state that VR simulation in the current form does not have an additional value to traditional box trainers in laparoscopic suturing training.

Conclusions

The outcome of this study allows us to conclude that there was no significant difference in outcome between laparoscopic suturing training on a traditional box trainer alone or in combination with a virtual reality laparoscopic simulator. While an important advantage of VR is objective assessment of the performance, the participants of the current study preferred the traditional box trainer over the VR simulator. Therefore we can state that we did not find an additional value of virtual reality simulation to traditional box trainers in laparoscopic suturing training. One should consider that future development in VR simulation should focus on basic skills and component tasks of procedural training in laparoscopic surgery, rather than on laparoscopic suturing.