Minimally invasive surgery has developed dramatically during the past decade. Laparoscopic surgery is now applied to cholecystectomy and antireflux surgery as the gold-standard procedure, and is even beginning to demonstrate advantages compared with conventional open surgery in various other procedures. The benefits of laparoscopic surgery are less postoperative pain, better cosmetic results, faster recovery and a shorter hospital stay. Despite these advantages, laparoscopic surgery is technically demanding and requires new psychomotor skills that differ from those needed in conventional open surgery. These skills include altered tactile feedback, different eye-hand coordination, translation of a two-dimensional video image into a three-dimensional working area and the fulcrum effect [3, 6, 7]. Thus, the educational activities for laparoscopic surgery should be intensified to ensure that good quality laparoscopic surgery is performed.

Inanimate box trainers are often used for laparoscopic simulator training, and the performance on these trainers was found to be well correlated with intraoperative assessments of residents performing a laparoscopic procedure [24]. Intense training on inanimate box trainers has also been shown to improve operative performance [16, 23]. Despite these advantages, objective assessment of performance requires human supervision and scoring, and the metrics are usually limited to task completion time and/or accuracy rate based on subjectively monitored human evaluation.

Virtual-reality (VR) laparoscopic surgical simulators are considered to represent educational tools with great potential. Recently, evidence that VR simulators can translate into improved outcomes in the operating room has been reported in prospective randomized trials [8, 13, 15, 20, 25, 30]. These simulators can also offer objective performance assessment without the need for monitored human supervision, and directly measure multiple aspects of a subject’s psychomotor performance on specific laparoscopic skills. Therefore, VR simulators are expected to be useful as assessment tools for laparoscopic technical skills. However, for such VR simulators to be widely accepted as assessment tools, they must be proven to show validity for measuring laparoscopic skills. A basic component of this validity is the ability of a simulator to detect differences among the performances of individuals with increasing levels of experience. This form of validity is referred to as construct validity.

In the present study, we investigated whether eye–hand coordination skill, one of the important psychomotor skills for laparoscopic surgery, measured using the LAP Mentor VR laparoscopic surgical simulator, was able to differentiate among subjects with different laparoscopic experience and thus demonstrate its construct validity.

Materials and Methods

LAP Mentor

The LAP Mentor (Simbionix USA Corp., Cleveland, OH), a VR laparoscopic surgical simulator, allows for practice of basic laparoscopic skills using the basic task module (BTM), and also more-complex skills mimicking surgical laparoscopic procedures [22] (Fig. 1). The BTM includes 0° camera manipulation, 30° camera manipulation, eye–hand coordination, clip application, grasping and clip application, two-handed maneuvers, cutting, diathermy and object translocation. The procedural tasks include step-by-step instructive procedural tasks and full procedural tasks of laparoscopic cholecystectomy. In this study, the eye–hand coordination task in the BTM was selected for use.

Figure 1.
figure 1

The LAP Mentor laparoscopic surgical simulator (Simbionix USA Corp., Cleveland, OH).

Full size image

Eye–hand coordination task

In this task, two instruments, one for each hand, are available for use. One of the instruments is blue, while the other is red. They become visible on the screen as soon as the instruments are inserted. During the task, flashing balls of each color must be touched on the top of the stick with the same color instrument. After one ball is touched, the next ball starts flashing and must be touched etc. (Fig. 2).

Figure 2.
figure 2

Screen appearance of the eye-hand coordination task in the basic task module of the LAP Mentor.

Full size image

Subjects

A total of 31 surgeons, who were all right-handed, were enrolled as the study subjects (29 males, two females; mean age, 37.5 years; range 26–54 years). The participants were divided into two groups according to their experience as an operator in laparoscopic surgery. The novice surgeons group consisted of 15 surgeons who had performed fewer than 10 laparoscopic procedures (13 males, two females; mean age, 33.8 years; range 26–51 years; mean number of procedures, 1.4; range 0–5 procedures), while the experienced surgeons group consisted of 16 surgeons who had performed more than 50 laparoscopic procedures (16 males, 0 females; mean age, 40.9 years; range 31–54 years; mean number of procedures, 117.6; range, 53–350 procedures). None of the participants had any prior experience with the LAP Mentor.

Procedure

This study was carried out at the Kyushu University Training Center for Minimally Invasive Surgery. The Lap Mentor was used as a pre-evaluation tool in the laparoscopic training curriculum. After receiving instructions, all subjects performed a single trial of the eye–hand coordination task in the BTM of the LAP Mentor. In this task, the LAP Mentor provides 15 parameters: task completion time, number of correctly hit balls, total number of balls, accuracy rate of correct hits, number of movements (NOM) of each instrument, total path length of each instrument, shortest path length of each instrument after the ball starts flashing, actual path length of each instrument after the ball starts flashing, economy of movement (EOM) of each instrument and average speed of each instrument. Among these parameters, the task completion time, NOM of each instrument, EOM of each instrument and average speed of each instrument were analyzed. NOM represents roughness. Continuous movement for more than 3 mm or changing the movement direction by 90° counts as a movement. EOM represents efficiency, which is calculated according to the following formula: EOM (%) = shortest path length after the ball starts flashing / actual path length after the ball starts flashing × 100.

Statistical analysis

All the results are expressed as the mean ± standard error on the mean (SEM). Since the subjects’ scores were normally distributed, Student’s t-test was used to investigate the differences between the two groups. Values of p < 0.05 were considered to indicate statistical significance.

Results

The experienced surgeons completed the tasks significantly faster than the novice surgeons (novice vs. experienced: 73.4 ± 5.4 vs. 52.6 ± 2.9 s, p = 0.0018) (Fig. 3). The experienced surgeons also achieved a significantly lower NOM of the left instrument (novice vs. experienced: 31.9 ± 2.5 vs. 24.6 ± 1.2, p = 0.013), better EOM of the left instrument (novice vs. experienced: 51.2 ± 2.6% vs. 60.8 ± 3.3%, p = 0.031) and faster average speed of the left instrument (novice vs. experienced: 2.5 ± 0.2 vs. 3.1 ± 0.2 cm/s, p = 0.025) than the novice surgeons, whereas there were no significant differences between the two groups for the NOM of the right instrument (novice vs. experienced: 36.8 ± 2.8 vs. 31.7 ± 1.7, p = 0.12), EOM of the right instrument (novice vs. experienced: 43.4 ± 1.9% vs. 47.3 ± 3.9%, p = 0.38) and average speed of the right instrument (novice vs. experienced: 2.9 ± 0.2 vs. 3.4 ± 0.2 cm/s, p = 0.071) (Figs. 4–6).

Figure 3.
figure 3

Time required to complete the task and accuracy rate of touched balls measured for novice and experienced surgeons.

Full size image
Figure 4.
figure 4

Number of movements (NOM) of each instrument during the task measured for novice and experienced surgeons.

Full size image
Figure 5.
figure 5

Economy of movement (EOM) of each instrument during the task measured for novice and experienced surgeons.

Full size image
Figure 6.
figure 6

Average speed of movement of each instrument during the task measured for novice and experienced surgeons.

Full size image

Discussion

With the rapid development in minimally invasive surgery, acquiring the specific skills necessary to perform laparoscopic surgery is now inevitable for all surgeons. The practice of laparoscopic surgery requires new psychomotor skills that are counterintuitive to open surgery, and thus not easy to learn. Such observations have created the need for operative training with laparoscopic surgical simulators.

VR laparoscopic surgical simulators show great promise in several areas of surgical training, and practice with some VR simulators has been shown to improve performance in the operating room in prospective randomized studies [8, 13, 15, 20, 25]. A recent study further demonstrated that the LapSim laparoscopic surgical simulator, one of the VR simulators, is more useful as a training tool than an inanimate box trainer [30]. Therefore, evidence that skills acquired on inanimate box trainers or VR simulators can be translated into actual operating room situations has been clearly established.

VR laparoscopic surgical simulators can also exert benefits when they are used as assessment tools for laparoscopic skills, since they can provide objective performance assessment without monitored human supervision and directly measure multiple aspects of a subject’s psychomotor performance. However, if the simulator metrics are unable to reflect the performances of different subjects, thus lacking construct validity, the simulators will have limited usefulness as assessment tools. If certain task parameters can be shown to differentiate among different levels of experience of the subjects, they can be used to determine the baseline skills and monitor their progress over time.

In our study, the eye–hand coordination task of the LAP Mentor was selected to demonstrate its validity. Although the task is very easy, it can simply evaluate eye–hand coordination skill, which is considered to be one of the most important psychomotor skills for laparoscopic surgery. In the present study, the task completion time was able to differentiate between experienced and novice surgeons. This result demonstrates that this task of the LAP Mentor is able to distinguish between different levels of laparoscopic experience, thereby providing evidence for construct validity.

To investigate which factors affected the differences in task performance between the two groups, the NOM, EOM and average speed of each instrument were analyzed. NOM is an indicator of roughness, while EOM is used to assess whether the subject is able to make purposeful movements with the instruments during the task. There were significant differences between the novice and experienced surgeons in the NOM, EOM and average speed of the left instrument, since the experienced surgeons achieved a significantly lower NOM, better EOM and faster average speed than the novice surgeons. Interestingly, however, there were no significant differences in the NOM, EOM and average speed of the right instrument between the two groups. These results indicate that eye–hand coordination skill of the nondominant hand has construct validity and is a more efficient assessment tool than that of the dominant hand.

To date, several other VR laparoscopic surgical simulators have been reported to possess construct validity. The minimally invasive surgical trainer-virtual reality (MIST-VR) [27] has been validated in a number of trials showing that the system is able to differentiate between experienced and inexperienced surgeons [1, 2, 9–12, 14, 17, 19, 28]. Construct validity has also been shown for the LapSim laparoscopic surgical simulator [4, 5, 21, 26, 29]. In the present study, construct validity has been shown to apply to the eye–hand coordination task of the LAP Mentor.

On the other hand, Woodrum et al. [29] reported that only certain parameters were sufficiently sensitive to show variation in performance among subjects with different laparoscopic skills using the LapSim laparoscopic surgical simulator. A study by Sherman et al. [26] demonstrated that time error scores were a valid measure of performance for distinguishing between expert and naive surgeons, whereas there were no significant differences in the motion score for the dominant hand. Similarly, in our study, the task completion time, NOM, EOM and average speed of the nondominant hand were able to differentiate between subjects with varying laparoscopic experience, but there were no significant differences between the two groups for these measurements of the dominant hand. One possible reason why we may not be able to detect any differences for the dominant hand is that the task was very easy. However, at least, the results indicate that assessment of the nondominant hand is more sensitive than that of the dominant hand when evaluating different laparoscopic skills according to eye-hand coordination skill. Such a difference in construct validity between the dominant hand and nondominant hand has never previously been reported. Furthermore, our results suggest that the skill of the nondominant hand could be a key factor for improving laparoscopic skills.

Recently, McDougall et al. [18] reported that all the tasks of the LAP Mentor, except for camera manipulation, had construct validity to differentiate between medical students and experienced surgeons. However, the evaluation in their study was carried out according to an automatic scoring system, of which the detailed parameters are unknown. Actually, in their study, the eye–hand coordination task could differentiate between medical students and experienced surgeons, but not between residents and much more experienced surgeons. On the other hand, in our study, some parameters in the same task were able to differentiate between novice and experienced surgeons. These differences in the obtained results may depend on the parameters analyzed in each experiment. Additional studies will be needed to evaluate construct validity for other tasks of the LAP Mentor using various parameters. When the parameters related to construct validity are taken into consideration, this VR simulator may be incorporated into a training program as an assessment tool. Therefore, the next step in the evaluation of the LAP Mentor is to demonstrate its face, concurrent and predictive validities as an assessment tool.

In conclusion, our study has demonstrated that eye–hand coordination skill of the nondominant hand only on the LAP Mentor can successfully distinguish between novice and experienced surgeons, suggesting construct validity of the LAP Mentor.