Laparoscopic surgery is a widely employed surgical technique in the operating room. The benefits of laparoscopic surgery include less postoperative pain, quicker recovery, and minimal scarring [1, 2]. Novice surgeons looking to adopt laparoscopic techniques encounter a significant learning curve due to challenges like lack of depth perception, lack of haptic feedback, and limited range of motion [3]. To overcome this obstacle, there is an increasing emphasis on simulation based training to develop the fundamental skills specific to laparoscopic surgery [4, 5].

The Fundamentals of Laparoscopic Surgery (FLS) program was developed to help surgical residents and surgeons develop the psychomotor skills and dexterity required during basic laparoscopic surgery. The FLS program utilizes a Fundamentals of Laparoscopic Surgery Training Simulator (FLS Trainer Box) to teach basic technical skills pertinent to laparoscopic surgery. The FLS Trainer Box was developed based on the McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS) program [6, 7] and contains five of the original seven tasks; peg transfer, precision cutting, placement and securing of ligating loop, simple suture with extracorporeal knot, and simple suture with intracorporeal knot. Among the five tasks, intracorporeal suturing is the most difficult technical skill to acquire. Even for experienced surgeons, intracorporeal suturing and knot tying are considered a significant barrier to performing laparoscopic surgery [8].

Practicing on a FLS trainer box is effective but requires large number of consumables (a Penrose drain and suture per practice trial, costing ~ $4.50 per trial), and the scoring is somewhat subjective (despite extensive proctor training) and not immediate. In contrast, virtual reality simulation provides unlimited practice without using consumables. Moreover, they can provide instant and objective feedback of the trainee’s performance. Studies have shown that existing virtual reality simulators demonstrated face validity and construct validity [1, 9, 10]. In addition, virtual reality simulator training has also been shown to be effective for acquisition of laparoscopic surgery skills [5, 11,12,13,14,15,16,17].

A virtual basic laparoscopic skill trainer (VBLaST©) was developed to simulate the five tasks of the FLS Trainer Box. The purpose of this study is to evaluate the face and content validity of the VBLaST suturing simulator (VBLaST-SS©) which was developed by Qi et al. [18].

Methods

The participants were asked to perform the standard FLS intracorporeal suturing task on each of the VBLaST-SS© (Fig. 1) and the FLS Trainer Box. In the FLS Trainer Box, to keep consistency with the VBLaST-SS©, the needle was placed in the needle driver and placed in the box trainer before the task started. Timing began when the participant started to operate the needle drivers. The task included piercing the needle through two black dots on a Penrose drain, then tying three knots intracorporeally to close the slit in the Penrose drain. The first knot must be a double throw, and the following two knots must be single throws. Participants were asked to transfer the needle to the other hand between each throw. After three knots were finished, participants were asked to cut both sides of the suture inside the box trainer.

Fig. 1
figure 1

The intracorporeal suturing task in VBLaST-SS©

Full size image

At the beginning of the session, participants were asked to complete a questionnaire regarding their demographic information and experience on surgery, simulator, and videogames. After a video demonstration, participants had 15 min or up to three times to practice on each system (Fig. 2). The presentation order of the systems was balanced. The trial session was 10 min on each system, starting with the system used first in the practice session. The trial session was video recorded for future analysis. After the participants completed the session, the VBLaST-SS© automatically calculated the participant’s performance, and the Penrose drain they used in the FLS Trainer Box was collected to be measured. The performance scores on each system were calculated based on the time taken (s), deviations to the black dots (mm)—the intended piercing points for the needle, and the incision gap (mm) post-suturing.

Fig. 2
figure 2

Participant performing suturing task on VBLaST-SS©

Full size image

After performing the tasks, the participants were then asked to finish a 13-item questionnaire regarding the face and content validity of the simulator. Participants answered questions with ratings from 1 (not realistic/useful) to 5 (very realistic/useful). In addition, they finished a NASA-TLX questionnaire regarding their perceived workload.

Three participants’ performance data, apart from their completion time, on the VBLaST-SS© were excluded in the final analysis due to issues with their data. In these cases, the participants did not penetrate both sides of the Penrose drain or the software did not recognize their performance. The software then provided the maximum value of the deviations (i.e., 40.840706 mm) and the maximum value of the incision gap (i.e., 3 mm). A Wilcoxon signed ranks test was performed to identify differences in performance on the VBLaST-SS© compared to the FLS Trainer Box.

Results

Demographics

Twenty medical students (MS, n = 20), six residents (n = 6), and a research assistant (n = 1) voluntarily enrolled in this study. One participant withdrew from the study for scheduling reason. All 25 remaining participants completed the study. The demographic data are shown in Table 1.

Table 1 Demographic data
Full size table

Face validity

Post-task questionnaire results are shown in Table 2. Questions 10–12 regarding VBLaST-SS© comparing to animal model were excluded from the analysis due to limited experience of the participants on animal models. Responses to 8 of the 10 questions (80%) averaged above 3.0 out of 5. Those questions that rated the highest were the degree of realism of the target objects in the VBLaST-SS© compared to the FLS (3.88) and the usefulness of the VBLaST-SS© simulation in learning hand-eye coordination skills compared to the FLS (3.84). Those rated lowest were the quality of the force feedback in the VBLaST-SS© compared to the FLS (2.88) and the degree of realism of the instrument handling in the VBLaST-SS© compared to that in actual laparoscopic surgery (2.93).

Table 2 Post-task questionnaire
Full size table

Performance

Performance scores are shown in Table 3. Average intracorporeal suturing completion time on the VBLaST-SS© was 421 (SD = 168) seconds compared to 406 (175) seconds on the box trainer (p = 0.620). There was a significant difference between systems for the incision gap (p = 0.048). Deviation in needle insertion from the black dot was smaller for the box trainer than the virtual simulator (1.68 vs. 7.12, p < 0.001).

Table 3 Comparison of the performance scores between VBLaST-SS© and Real FLS
Full size table

Discussion

This study compared the performance on the intracorporeal suturing task for a newly developed virtual reality laparoscopic surgery simulator and on the traditional FLS box trainer. Participants showed comparable performance time on the VBLaST-SS© and traditional box trainer. No significant difference was found in task completion time. However, there were significant differences in performance quality for remaining incision gap and needle insertion deviation. The differences in remaining incision gap may result from the difference of calculation method and the difficulty to distinguish knot tightness in the virtual environment. Although the ability to tighten the knot in the VBLaST-SS© was mentioned during the practice session, participants tended to continue the task without doing so. The differences in needle insertion deviation may result from limited depth perception and the limited mobility of the Penrose drain target in the virtual environment. Some participants mentioned in their comments that the Penrose drain in the VBLaST-SS© made the task harder to perform because of this characteristic.

Based on feedback from participants, the VBLaST-SS© was considered moderately realistic compared to the traditional box trainer (mean rating 3.84 out of 5). However, the quality of force feedback and the realism of the instrument handling received low ratings. Several participants mentioned in their comments that when handling the needle with needle drivers, the force feedback was helpful but too weak. They also suggested that there should be more force feedback on the Penrose drain. This suggests that the force feedback might need to be improved in future iterations of the system, not only for needle handling but also for Penrose drain. Sankaranarayanan et al. [4] evaluated the face and construct validity of peg transfer, pattern cutting, and endoloop on the same simulator system and found that quality of force feedback had the lowest rating. This supports the importance of force feedback in the design of virtual reality simulator trainers.

The main limitation of this study was a small number of participants who were more experienced laparoscopic surgeons. While this limited the ability to evaluate the system in terms of skill differentiation, the individuals who tested the system are those who are currently training on physical suturing trainers and are the target user group for a virtual reality-based skills trainer. Future work is needed to determine whether the VBLaST-SS© intracorporeal task can be used to distinguish between expert and novice surgeons.

Overall, the VBLaST-SS© system showed face validity and has the potential to support training on suturing skills. Due to the limitation of force feedback, as mentioned by the participants, the current system can supplement rather than replace current training methods. However, with further improvement and validation, the VBLaST-SS© can supplant the FLS box trainer as a cost-effective means of practice. The FLS box trainer and instruments have a one-time cost of $2770, plus $4.50 for consumables per practice trials. By comparison, the VBLaST-SS© system costs about $6400 for all parts and the computer system. Therefore, if the VBLaST-SS© system is used for more than 820 practice trials (~ 10 trainees to proficiency), the simulator becomes more cost-effective than the box trainer. In the next stage of the study, the training effectiveness of the VBLaST-SS© will be evaluated to compare participants’ performance after training on different systems and their ability to transfer their skill between systems and to retain the skill over time. Future work will also consider skill transfer from practice on the VBLaST-SS© to the operating room and the addition of advanced suturing tasks for the simulator.