Since the da Vinci® Surgical System (Intuitive Surgical, Sunnyvale, CA) received approval from the Food and Drug Administration for use in General Surgery in the year 2000, the clinical application of robotic-assisted surgery has increased dramatically. In 2010, there were a total of almost 1,800 systems installed worldwide. The number and variety of procedures performed per installed system continue to rise [1]. The market share of robotic prostatectomy compared to laparoscopic and open prostatectomy is estimated to exceed 80 % in the United States according to the manufacturer [2]. In gynecologic surgery, the growth of robotic procedures has been exponential. Robotic surgical skills are not derivative from those attained in open or laparoscopic surgery. The interface between the surgeon and the patient is dramatically different in robotic surgery. Surgeons are removed from physical contact with the patient during surgery. A lack of force feedback, the need to clutch and reposition the master controls, endowrists, camera navigation, and 3D imaging are just a few of the unique aspects of robotic surgery that require acclimation and deliberate practice to master.

Early in the era of laparoscopic surgery, many surgeons adopted laparoscopic techniques with little training or experience and negative consequences for some patients [3]. In the current era, the acquisition of basic laparoscopic surgical skills has been moved out of the operating room and into the skills lab. Extensive research has been conducted on simulation-based training as a mechanism for the acquisition of laparoscopic surgical skill. In the Fundamentals of Laparoscopic Surgery [FLS, American College of Surgeons (ACS) and the Society of Gastrointestinal Endoscopic Surgeons (SAGES)] curriculum has been validated as a high-stakes simulation based curriculum in laparoscopic surgery that can be used to develop and assess laparoscopic surgical skill [4, 5]. In 2009, the American Board of Surgery began to require that surgeons seeking board certification successfully complete the FLS program.

There is a need for a reliable, valid, high-stakes training, and assessment tool in robotic surgery. While several high fidelity, virtual-reality (VR) robotic surgical simulators exist—few have been validated in order to demonstrate that the acquisition of skills attained in a simulated robotic environment translate into skills integral to performance in the operating room. We sought to determine if novice medical students could train to proficiency levels established by experienced robotic surgeons on the VR robotic simulator, and if this led to an improved performance in simulated tasks on the actual da Vinci system.

Materials and methods

We previously developed and validated a robotic surgical skills curriculum which composed of five selected exercises of the more than 30 available pre-programmed tasks in the daVinci™ Skills Simulator, a computer-based virtual reality (VR) training tool [6]. The simulator docks to the daVinci console, and simulated tasks are performed using the master controls. In our preliminary work, we demonstrated that experienced robotic surgeons (>20 robotic cases as primary faculty) significantly outperformed novices (first year medical students) on all tasks providing evidence of construct validity for our curriculum. Construct validity is the ability of a test to measure the traits it purports to measure, in this case experience and by inference robotic surgical skill [7].

Proficiency standards for our curriculum were established based on the trimmed mean performance levels of five experienced robotic surgeons. In our preliminary work, the metrics of time to complete each task and “overall score” (a computer-derived score) were found to correlate best with experience and were, therefore, used as training targets. For the current study, subjects were required to meet proficiency levels for both time and overall score on all five tasks that comprise our curriculum. These five tasks are available on the DaVinci Skills Simulator and are called Pick and Place, Camera Targeting (Level 2), Peg Board (Level 2), Matchboard (Level 2), and Suture Sponge (Level 3). The proficiency targets we used for this study are listed in Table 1.

Table 1 Proficiency targets as defined by the trimmed mean performance of experienced robotic surgeons on a 5-task VR curriculum
Full size table

For the current study, we recruited first year medical students (no prior robotic or other surgical experience) and experienced robotic surgeons to participate in our study upon IRB approval. We sought to determine if medical students could train on the daVinci Skills Simulator until they were able to achieve proficiency scores in our curriculum on three consecutive attempts for each task. To determine if practice on the simulator results in increased skill on the daVinci platform, we had subjects perform two FLS tasks in a video trainer using the daVinci robot (not the simulator). The two selected FLS tasks were pegboard transfer and intracorporeal knot tying. Subject’s performance was assessed on the FLS tasks prior to any exposure to the simulator trainer and again upon successful attainment of proficiency goals on all five VR simulator tasks. Subjects were given a brief orientation to the daVinci system and each simulated task. A brief period of orientation and practice was allowed in the daVinci Skills Simulator and in the daVinci docked to the video trainer prior to testing. Experienced robotic surgeons did not undergo proficiency training but were asked to perform the FLS tasks in the daVinci video trainer model to allow comparisons between VR trained novices and experienced robotic surgeon’s skill levels on the FLS robot model.

The FLS tasks were scored using a modification of a formula which was previously published [8]. For the peg transfer task, a cutoff time of 300 s was used. Peg transfer score was calculated according to the formula:

$$300{-}\left( {\text{Seconds to transfer all 6 pegs back and forth}} \right){-}({\text{Penalty}} = \% {\text{ of dropped pegs unable to be transferred}}).$$

The intracorporeal knot-tying task was scored according to the formula:

$$300{-}\left( {\text{Seconds to complete knot}} \right){-}\left( {\text{Penalty for a loose or slip knot}} \right).$$

We chose to use a cutoff score of 300 s rather than 600 as utilized by Fraser et al. [7] This was due to time constraints on daVinci system (which is heavily utilized clinically). Subjects were asked to perform three repetitions of each FLS task prior to and following proficiency training. At each pre- and post-training assessment on both the VR simulator and the daVinci surgical system in the video trainer, subjects were asked to complete the NASA Task Load Index (NASA TLX) to assess workload.

Statistical analysis was conducted using MStat V5 software (Mcardle Laboratory, Madison, WI). Wilcoxon rank sum scores were used to compare continuous variables, and Fisher exact test was utilized for categorical variables.

Results

A total of 25 medical students were recruited to participate in this proficiency training study. One subject completed initial assessment and began the process of proficiency training before dropping out of the study due to time constraints, bringing the novice study group to 24. Three experienced robotic surgeons performed the FLS tasks in the robotic video trainer. Data were collected during the training interval between the pre-test and post-test for the final 14 novice subjects only due to a data collection error in the first 10. Workload was assessed only for the same 14 subjects for the same reason. All 24 novices to complete the study attained proficiency levels and underwent pre- and post-VR training testing on the daVinci robot in the video trainer FLS tasks.

Subjects were able to attain proficiency levels on all 5 VR tasks in a mean 71 total trials and 164 min on the training console (Table 2). There was a significant improvement in performance on the FLS tasks for novice subjects following proficiency training. There were a greater number of successful knot-tying attempts on the robotic FLS tasks following training (Table 3). Subject-rated workload of FLS tasks as assessed with the NASA-TLX instrument decreased for all assessments (Table 4).

Table 2 Attempts and time to achieve proficiency levels on 3 consecutive attempts for all tasks for novice subjects
Full size table
Table 3 Peg transfer and intracorporeal (IC) knot tie score pre- and post-proficiency training in novice subjects
Full size table
Table 4 Subject rated difficulty using NASA task load index (NASA TLX) sum workload scores (higher score = higher workload)
Full size table

Following completion of the VR training curriculum, there was no significant difference between the performance of novice and experienced robotic subjects on the FLS Peg Transfer task (220.9 novice vs. 229 experienced; p = 0.36), however, experienced robotic surgeons significantly outperformed trained novice subjects on the IC knot-tying task (104.7 novice vs. 213.5 experienced; p < 0.001).

Discussion

We have demonstrated that novice surgeons can train on a virtual reality robotic surgical simulator to performance levels on par with those of experienced robotic surgeons. Novice surgeons can successfully complete the curriculum at a high rate and in a reasonable time period. Furthermore, we have provided evidence that the skills developed on the simulator carry over to the daVinci surgical platform. This has potential implications for the future of robotic training programs and perhaps ultimately for credentialing and granting of robotic surgical privileges for surgeons.

Current robotic surgical training for practicing surgeons looking to adopt robotic technology typically involves the completion of an online industry sponsored training module, attendance at a 2-day industry sponsored training course (utilizing an animal model for simulated procedures and practice on a robotic simulator), and a variable period of proctoring defined by each individual hospital’s credentialing committee. Residents and fellows at programs with experienced robotic surgeons may gain exposure to robotic surgery as the bedside surgeon with occasional opportunities for limited console time. The very nature of robotic surgery often prevents significant exposure and experience for trainees.

The value of moving the venue for the acquisition of basic surgical skills from the operating room to the simulation lab has been proven. The current paradigm for developing and assessing laparoscopic surgical skills—the FLS curriculum—has been widely adopted. Surgical societies including MIRA (Minimally Invasive Robotic Association) and SAGES have encouraged the rapid implementation of such a curriculum in robotic surgery, but no equivalent to FLS currently exists in robotic surgery [9]. Working with the actual robot on anatomical samples, animal models, or inanimate models is costly in terms of equipment and mobilizing the robot. Robotic surgery simulators may offer a more economical training alternative. Perrenot and colleagues developed a 5 task robotic curriculum on the dV-trainer (Mimic dV-Trainer™, Mimic Technologies Inc., Seattle, WA) [10]. The dV-trainer is a stand-alone simulator on which the daVinci Skills Simulator that docks to the daVinci console is based. Seventy-five subjects of varying skill and experience completed a curriculum that was very similar to ours. Face and content validity of the curriculum on the dV-trainer were high. Reliability of scoring was high, and the global scores on the simulator were strongly correlated with previous experience in robotic surgery (construct validity). The authors concluded that the dV-trainer is a valid tool to assess the skills of robotic surgery. Other investigators have demonstrated the face, content, and construct validity of the dV-trainer and daVinci Skills Simulator using different combinations of the tasks available [11–14].

The content, face, and construct validity of the inanimate robotically performed FLS tasks employed in our study have previously been documented. Investigators at the University of Texas-Southwestern developed a curriculum using the daVinci surgical system, and nine inanimate training exercises [15, 16]. The FLS peg transfer task and the intracorporeal knot-tying task performed robotically in a video trainer as in our study were among the simulated tasks assessed. Each task was scored for time and accuracy using modified FLS metrics. All nine inanimate tasks demonstrated construct validity. Furthermore, the inter-rater reliability, test–retest reliability, and internal consistency of the two FLS tasks common to the UT-Southwestern studies and the current study were acceptable for high-stakes evaluations. Our findings are similar to those of Lerner et al. [17], who demonstrated that practice on the dV-trainer resulted in improved performance on simulated inanimate tasks using the daVinci robot. Given the fact that, in our study, no training occurred with the inanimate tasks on the daVinci platform between assessment periods, we believe that the improved performance observed in the post-proficiency training evaluations reflects an acquisition of robotic surgical skills, which has occurred in the daVinci Skills Simulator.

Our study is unique in that we established proficiency levels based upon the performance of experienced robotic surgeons in the VR simulator. Most of the referenced published research on robotic simulators describes the impact of a specified number of training sessions on performance. Compared with unstructured training, or training based on repetition without declared end points, training to proficiency targets meets the educational needs of both the trainee and the trainer. Surgical trainees desire clear goals and expectations and prefer to master technical skills in an effective and efficient manner that can adapt to their individual learning styles and capabilities. Surgical educators also desire effective training that produces a consistent and measurable high level of performance for all trainees. Proficiency-based training has been shown to result in excellent skill acquisition and long-term retention while maximizing the efficiency of the learning process [18, 19]. Gauger et al. [20] demonstrated that the ability of novice surgeons to reach high levels of proficiency in simulated tasks and operative performance is improved by use of specific performance targets. These investigators also discovered that if learners are left to determine their own proficiency targets and practice needs, less practice occurs, and the level of accomplishment is lower.

Mental workload is a reflection of the amount of attention that an operator can direct to a task at a given moment [21]. In general, the mental workload associated with an easy task is low, and difficult tasks produce higher workloads. Higher workload tasks have been demonstrated to be associated with an increased likelihood of performance errors [22]. The NASA-TLX questionnaire has been demonstrated to accurately reflect workload changes during simulator training [23]. Workload reduction during training reflects increasing comfort with a task and, consequently, its mastery. It has been suggested that higher workloads during surgery are potentially linked to intraoperative stress [24]. Stress is postulated to impair surgeon judgment, decision-making, and communication—all factors that may negatively affect performance [25]. The impact of simulator training on workload should not be overlooked.

Recently, a number of lawsuits have been brought forward naming Intuitive Surgical as a defendant and alleging that surgeons were inadequately trained to perform a particular robotic procedure [26]. It has also been suggested that robotic surgery complications may be under-reported [27]. Considering the above, it is important that surgeons and hospitals pay attention to their credentialing and training requirements for new robotic surgeons. Basic robotic surgical skills should ideally be developed in a safe and reproducible platform such as the daVinci Skills Simulator before surgeons are allowed to perform clinical cases on the system. While the value of requiring novice robotic surgeons to practice their skills before operating on patients is apparent, surgical expertise is complex, and simulator training is not the entire solution to the concerns listed above. Research largely has focused on the development and assessment of technical skills. Cognitive performance has received much less attention. However, skills such as judgment and decision-making are central components of surgical expertise. Appropriate case selection, hands-on courses, case observation, proctoring, and periodic quality reviews are all play a role in ensuring a safe initial robotic surgical experience for new robotic surgeons.

Goal directed training in a VR robotic surgery simulator results in significant improvement in performance of selected FLS tasks. This suggests that practice on a robotic surgery simulator translates to improved performance (and by inference enhanced skill) on the daVinci surgical platform. The results of this study position the daVinci Skills Simulator as a good candidate for a large-scale skills certification program similar to the FLS program. With an increased focus on patient safety in healthcare, it will be important to ensure that surgeons possess the necessary skills to safely perform robotic surgery.