The da Vinci Surgical System developed by Intuitive Surgical (Sunnyvale, CA, USA) is currently the only surgical robotic system with approval from the US Food and Drug Administration (FDA) for use in general laparoscopic surgery [1]. The system consists of several key components, including an ergonomically designed console where the surgeon sits while operating, a patient-side cart where the patient is situated during surgery, four interactive robotic arms, a high-definition 3D vision system, and proprietary EndoWrist instruments [2]. Since its introduction in 2009, the da Vinci Si model has incorporated the virtual reality (VR) “da Vinci Skills Simulator” (dVSS) using the da Vinci surgeons’ console as the user interface (Fig. 1). One of the main advantages of this integrated high fidelity VR simulator is the ability to function without the need of the robotic arms, allowing surgeons from any specialty to improve their robotic skills and adjust to the master surgeon’s console controls in a safe, controlled environment [3].

Fig. 1
figure 1

da Vinci Si surgeon’s console and Skills Simulator (dVSS) backpack

Full size image

In the case of laparoscopic surgery, the use of VR simulators and “trainer boxes” has shown to be valuable tools for laparoscopic skills certification based on proficiency scores derived from experts’ performance [4–7]. However, a similar type of evaluation is limited or minimal in the case of robotic surgery. Different skills such as robotic camera control, clutching, EndoWrist manipulation, and three-dimensional visualization of the operative field are highly specialized features unique to this type of technology and certainly different from the techniques used in laparoscopic or open surgery. The use of robot-assisted laparoscopic surgery platforms is certainly increasing and expected to expand significantly in the near future, focusing attention toward the exploration and development of surgical simulators to train surgeons, fellows, and residents.

Prior studies reported that VR robotic platforms such as the dVSS have shown evidence for “face,” content, and construct validity [8–11], despite the fact that these types of validity refer to an older validity framework that has been abandoned for more than a decade [12]. Recently, Perrenot et al. [13] compared the performance of 5 groups with varying levels of robotic surgery expertise by evaluating a series of exercises between a VR robotic platform (dV-Trainer, Mimic technologies, Seattle, WA, USA) and a matching physical model in a dry lab. In their experiment, the simulator was not only able to clearly differentiate performance among different levels of robotic expertise, but also showed that this VR platform can be used as a tool to assess basic robotic surgical skills among trainees. Similarly, Hung et al. [14] evaluated the impact of simulator training and transfer of skills to an animal model among 24 robotic surgery trainees. Each subject’s performance was pretested in an ex-vivo animal model followed by randomization in either a simulator (dVSS) or no simulator training group. A final post-training assessment was conducted using the same ex vivo animal models demonstrating a higher benefit of simulator training among trainees with low baseline robotic skills.

Although widespread curriculum and certification implementation exist for laparoscopic surgery via the “Fundamentals of Laparoscopic Surgery” program, development of robotic skills training remains limited. Despite the availability of multiple robotic simulators on the market, residency skills laboratories often find it prohibitive to make an investment of this magnitude when acquiring these platforms. Additionally, if the teaching hospital owns a robotic surgical device patient utilization will take priority over educational activities with this type of technology. Residency training programs often lack institutional guidelines for resident participation in robotic surgical cases and despite efforts to develop proficiency-based robotic surgery assessments [15–18], their implementation remains limited. The purpose of this study was to develop an institutional robotic simulator (dVSS) training curriculum among general surgery residents and evaluate its feasibility in a University-based setting with only one robotic surgical system available.

Materials and methods

The dVSS platform was selected to conduct this IRB approved study at the University of Texas Health Science Center at San Antonio (UTHSCSA). Despite lacking a robotic simulator in our skills development laboratory (Johnson Center for Surgical Education), our proximity to the affiliated resident training Hospital facility (University Health System, San Antonio, Texas) operating rooms allowed us to easily transport both the da Vinci surgeons’ console and simulator backpack one to 2 days per week when available. Before the first OR case of the day, one surgical research resident, previously proctored by an Intuitive Surgery representative, transported the surgeons’ console and simulator backpack from its usual location in the operating room to the simulation center. After the last OR case, the equipment was returned, thus avoiding any interference with usual OR routine. Each transport took approximately 15 min. In addition to the simulator, an external monitor and a laptop were required for the research coordinator to record performance metrics and orient the trainees regarding the use of the dVSS. Both of these devices were already available at the simulation center and inferred no extra costs.

General surgery residents and faculty were invited via email to take part in the study. Twenty-two participants including four faculty, four senior (two PGY3 and two PGY4), and 14 junior (PGY1) residents were enrolled in a 4-week VR robotic training curriculum developed at our institution. None of the participants had prior significant robotic surgical experience. All were required to complete the four modules (system overview, docking, surgeon console, and safety features) of the Intuitive Surgery online training program (available at www.davincisurgerycommunity.com/) for the da Vinci Si console. Participants were asked to take the computer-generated test upon completion of the online modules and obtain a score equal to or greater than 80 % in order to access the dVSS.

We analyzed the description and characteristics of the various simulated modules available in the dVSS and reviewed in detail the provided manufacture’s instruction manual (Mimic Technologies) selecting a set of seven robotic skills (camera control, energy control, EndoWrist manipulation, basic needle driving, advanced needle driving, needle control, and fourth arm control) based on the exercise’s primary endpoint. For each primary robotic skill, two exercises were selected, one for the pre- and post-testing session, while the other was selected to be used during the training phase of our experiment (Table 1). The seven robotic skills were selected trying to address common challenges faced by novice robotic surgeons, including adjusting to a three-dimensional field of vision, manipulation of the system controllers with both hands, and utilization of feet pedals for camera, energy, and fourth arm control. The corresponding fourteen exercises have been previously reported to have “face,” content, and construct validity [8, 10, 13, 14, 19] and allow surgeons in training to develop the necessary skills to master the robotic console ranging from basic to more advanced features.

Table 1 Core robotic skills and exercises selected to assess them
Full size table

During the pretest session (session one), we asked participants to complete a pre-training questionnaire which gathered demographic data, level of training, interest in robotic surgery, and surgical expertise (recorded as the number of cases in three categories: open, laparoscopic, and robotic). Subsequently, the research coordinator gave a 5-min overview on how to operate the dVSS and allowed each participant to “warm up” for 5 min using the “playground” game available in the simulator. During the baseline assessment (pretest), participants were asked to complete one single trial of each of the seven selected robotic exercises. No verbal instructions or feedback were given during the assessment portion of session one. Each participant’s overall performance and other exercise-unique metrics were recorded into a personalized user account database generated by the dVSS.

A week later, participants were asked to return to the simulation center for the training phase of the experiment. The training sessions consisted of two individual 1- to 2-h blocks, each separated by 1 week where trainees practiced a different set of exercises that evaluated the same core robotic skills assessed during pretest. Participants were informed that the goal of each task was to achieve an overall score of 80 % or higher on two consecutive attempts in order to move to the next exercise. If this preset manufacturer’s proficiency score was not achieved, then a maximum of six trials per exercise was allowed before advancing to the next simulated skill. This threshold was determined based on the findings described by Perrenot et al. [13], which reported a learning plateau upon completion of a series of six exercises in five different robotic modules. Additionally, this allowed each training session to be one to 2 h. It is important to mention that during the training phase, participants were able to ask questions and receive hints on how to better perform in each of the exercises. Trainees were not allowed to participate in real robotic cases or practice in the simulator while enrolled in the study.

One week after concluding the second block of the training phase, participants were asked to complete a post-testing assessment that used the same set of exercises used during pretest. Again, overall performance and unique-exercise metrics were recorded. Finally, a post-testing survey using a 5-point Likert scale was administered to collect the self-perceived relevance of each of the selected exercises.

Robotic skills and exercises description

Skill 1: camera control

The first core robotic skill assessed the ability of a trainee to control the camera in a large, three-dimensional workspace. In the pre- and post-testing sessions, we selected the “Camera Targeting 2” exercise (Fig. 2). For the training session, we chose the “Camera Targeting 1” exercise (Fig. 3). In these exercises, the trainee learns how to manipulate the camera by positioning a light blue sphere within the center of the camera target. Once accomplished, the trainee can move to a different sphere and repeat the process until completing the task. The main difference between both of these exercises is that in “Camera Targeting 2” the trainee must pick-up and carry stones of different sizes from target to target and place them in designated baskets, adding more complexity to the exercise.

Fig. 2
figure 2

Screenshots of the seven different robotic exercises evaluated during pre- and post-testing

Full size image
Fig. 3
figure 3

Screenshots of the seven different robotic exercises evaluated during training

Full size image

Skill 2: energy control

“Energy Dissection 2” and “Energy Dissection 1” were selected to assess the second core robotic skill (energy control) during the testing and training phases, respectively. In these exercises, the trainee must cauterize and cut six small branching blood vessels that anchor to a larger vessel with the highest precision and lowest possible blood loss. The difference between these exercises is that in “Energy Dissection 2” the cauterized blood vessels tend to re-bleed, requiring the trainee to be attentive of the surgical field.

Skill 3: EndoWrist manipulation

To assess the use and precision of the EndoWrist manipulation core robotic skill, we selected the “Peg Board 2” and “Peg Board 1” exercises. These tasks require the trainee to pick-up and transfer color rings sequentially from a peg board wall to a single peg on the floor with various levels of difficulty between exercises.

Skill 4: basic needle driving

“Suture Sponge 2” and “Suture Sponge 1” were selected to assess the basic needle driving skills. Both exercises consist of inserting and extracting a curved needle through several pairs of targets on the edge of a sponge with random variations in their positions.

Skill 5: advance needle driving

The first advanced core robotic skills (i.e., advanced needle driving) consist of a set of two exercises. For the pre- and post-test sessions, the “Suture Sponge 3” exercise was selected. The trainee is required to insert and extract the needle through several pairs of targets on the edge of a sponge with more complex variations in their positions and bite sizes than in prior similar exercises. During the training phase, the “Tubes” exercise was selected based on the premise that the same core robotic skill primary endpoint is assessed. In this task, the trainee is required to use both instruments to manipulate and facilitate needle driving while recreating a virtual end to end anastomosis.

Skill 6: needle control

During testing sessions, the “Thread the Rings” exercise was selected to evaluate the ability for the trainee to control the needle. This task consists in passing a needle attached to a short suture through a number of eyelets distributed in the floor of a virtual workspace. For the training session, the “Needle Targeting” exercise was selected given its ability to assess the primary endpoint of the robotic skill to be assessed. In this task, the trainee was asked to remove needles from a rack and accurately insert them through a pair of matching colored targets.

Skill 7: fourth arm control

The final core robotic skill consisted of two alternate exercises. For the pre- and post-testing sessions, the “Ring Walk 3” skill was selected. In this exercise, the trainee is asked to alternate between walking a ring along a tortuous vessel and moving the camera with precision while retracting obstacles along the way with the fourth arm that impedes movement of the ring. The training counterpart (“Match Board 3”) consisted in a lifting a set of horizontal and vertical panel doors, picking up objects (numbers and letters), and placing them in their corresponding spaces while using an extra robotic arm.

Results

Demographics

A total of 22 participants were recruited (14 junior, four senior residents and four faculty). Most were male, right handed, and under the age of 40. Regarding prior operative experience, junior residents had fewer open and laparoscopic cases compared to their senior counterparts and faculty. All faculties have been in clinical practice for 10 years or less, and 50 % completed a minimally invasive and laparoscopic surgery fellowship. Two of them had prior robotic surgery experience; however, each had performed fewer than 50 robotic cases as primary surgeons (Table 2).

Table 2 Demographic data
Full size table

Metrics

The overall score (percentage displayed upon completion of each simulated exercise) is obtained by computing the individual weighted scores of each of the components that constitute each robotic skill. Some of these components are shared among the majority of exercises including economy of motion, use of excessive force, instrument collision, instruments out of view, master workspace range, and time to complete the exercise. Additionally, other components were unique to individual robotic skills such as the case of the “energy control” exercises (energy dissection 1 and energy dissection 2), in which blood loss, misapplied energy time, and number of broken vessels are taken into consideration for the final score calculation. The same was observed for exercises requiring hitting a target as it occurs for the basic and advanced needle driving skills where a higher number of missed targets could detrimentally affect the overall score.

Overall performance

A significant improvement in the seven core robotic skills was observed from pre- to post-test regardless of participant’s level of training (two-tailed paired t test, p < .001). Interestingly, participants were only able to achieve an overall score of 80 % or higher in only five of the seven exercises during post-test. This finding seems to have a direct relation with the median number of trials attempted to complete each simulated exercise during the training sessions, where advanced needle driving and fourth arm control reached the maximum number of possible attempts (six) as stipulated by our protocol (Table 3). When overall performance was compared among groups, no difference was found independent of the level of training or surgical expertise (ANCOVA, all p > .05).

Table 3 Overall score during pre- vs. post-test and median number of trials per robotic skill
Full size table

Skill 1: camera control

A statistically significant improvement in the overall score was appreciated from pre- to post-test (two-tailed paired t test, p < .001) in all participants. A median number of three trials were required to reach proficiency during the training session. Component analysis also revealed a significant decrease in the number of drops and instrument collisions (χ 2, p = .001 and p < .001, respectively) during post-test. Also, we found significant improvement in economy of motion, time to complete the exercise (paired sample t test, both p = .001), and time with instruments out of view (p = .046) upon completion of our training curriculum.

Skill 2: energy control

Similar to skill 1, we found a statistically significant improvement in the economy of motion, instrument collision, excessive force, and time to complete components (Table 4). Interestingly, we observed that trainees had fewer broken vessels (χ 2, p = .007), less blood loss volume, and less misapplied energy time (paired sample t test, p = .007 and p = .023, respectively) during post-testing, showing a significant gain of skills in these skill-unique component scores.

Table 4 Core robotic skills and detailed sub-metric analysis
Full size table

Skill 3: EndoWrist manipulation

The preset proficiency threshold of 80 % was achieved after a median number of two trials by our participants. Component analysis showed a significant decrease in the number of drops and instrument collisions (χ 2, p < .001, both), more efficient economy of motion, less time to complete the exercise, and with fewer instruments out of view (paired sample t test, all p = .001) during post-test.

Skill 4: basic needle driving

A significant improvement in the overall score, drops, economy of motion, and time to complete the exercise was observed upon completion of their final session (paired t test, all p < .001). No statistical improvement was observed in the number of missed targets from pre- to post-test (χ 2, p = .947). Participants were able to reach proficiency during their training sessions after a median of three trials.

Skill 5: advance needle driving

Significant improvements were observed in overall score, drops, economy of motion, and time to complete the exercise (paired t test, all p < .001). Nonetheless, participants during the training blocks required a median of six trials to complete the task, reaching the upper limit of attempts allowed for successful exercise completion.

Skill 6: needle control

Improvements in economy of motion, time to complete the exercise, needle drop, and instrument collision count were found during post-test (all p < .05). A median number of two trials were required to reach proficiency during training.

Skill 7: fourth arm control

Significant gain of skills regarding the economy of motion, use of excessive instrument force, and time to complete the exercise was observed (paired t test, p = .001, .023, and <.001 respectively). As observed with the advanced needle driving core skill, a median number of 6 trials were required by our participants in the training phase reaching the maximum number of attempts to complete the exercise.

Survey

In our post-testing-survey, participants were asked to evaluate using a 5-point Likert scale the usefulness of the curriculum by scoring the selected exercises that represented each of the seven core robotic skills. Despite lacking any real experience with actual robotic surgery cases, the purpose of this survey was for participants to evaluate if the selected exercises truly assessed the skill they were meant to measure. In this regard, each task obtained a median score of 4 (relevant) or 5 (very relevant).

Additionally, we saw an increase in the interest for robotic surgery among senior residents and junior faculty upon completion of the curriculum; however, this finding failed to reach statistical significance (Wilcoxon Signed-Ranked test p = .059). Interestingly, after our study concluded, seven additional participants (one medical student, one PGY3 and five PGY1 residents) completed our curriculum having heard overwhelmingly positive feedback from prior participants. Their scores and performances were not added to our final analysis.

Discussion

Recent changes in the work-hour regulations have challenged program directors and surgical educators into developing new pedagogical approaches to provide residents with educational tools to maximize their learning process while in training. The addition of surgical skills laboratories to general surgery programs has partially filled these voids. Given the prohibitive cost of robotic simulators, novel training strategies need to be considered. Our study showed that it is possible to collaborate with a hosting institution (in our case University Health System—San Antonio, Texas) without affecting regular patient care. Additionally, significant learning gains were obtained with a minimal time commitment (i.e., less than 1.5 h per week, over a 4-week timeframe).

Until recently, it was unclear which of the many provided virtual reality modules were useful for robotic skills evaluation or how many training trials were necessary to reach the manufacturer’s preset proficiency goal of 80 %. Lyons et al. [18] systematically identified the modules available in the dVSS, which were clinically relevant to develop proficiency-based training programs. Eight core skills were selected based on task deconstruction by expert robotic surgeons. Simulator performance metrics showed a statistically significant difference when expert robotic surgeons outperformed novices in all eight tasks. Our study similarly identified a group of seven core robotic skills based on a detailed review of the manufacturer’s manual and task descriptions, obtaining a comparable set of skills. Only one of the tasks (clutching) was not considered as an isolated core robotic skill in our study since this function is not the primary focus of any of the available exercises, but instead is required for the proper manipulation and successful completion of many of the exercises. Another significant difference is the fact that we were not able to find statistical differences in non-robotic expert surgical faculty performance compared to junior and senior residents. This suggests that prior operative experience for open and laparoscopic cases might not transfer into newer technologies such as robotic surgery.

All of our study participants had a statistically significant gain of skills from pre- to post-test in regards to the overall score for the seven core robotic skills. Participants achieved an overall score of 80 % or more during post-testing in five out of seven exercises. More complex robotic skills such as advanced needle driving and fourth arm control required a median of six trials during the training phase, which was determined in our protocol as the maximum number of attempts each exercise could be attempted. This seems to go against prior evidence suggesting a learning plateau is reached after six attempts [13] and implies that more complex robotic skills require longer practice sessions to reach proficiency.

Other metrics such as economy of motion and time to complete the exercise showed significant improvement across the seven core robotic tasks showing the selected exercises are appropriate methods of training to enhance robotic skills precision and efficiency. Additionally, task-unique categories such as blood loss volume, number of broken vessels, and misapplied energy time were found to significantly improve in the exercises selected to assess the core skill of energy control. Upon completion of the training curriculum, participants determined the chosen modules were relevant for the development of a basic robotic skills curriculum. Moreover, an increased interest in robotic surgery was observed among senior residents and junior faculty and subsequently, a few of the participants in the study have completed more advanced robotic training with ex vivo models.

In this study, we attempted to develop a training curriculum based on preset proficiency metrics suggested by the manufacturer and available upon completion of each exercise. It is unclear whether these metrics are truly derived from expert performance. Patel et al. [20] recently reported the number of trials necessary to reach an 80 % overall score in six different robotic exercises. For “Camera Targeting 1” five trials were necessary to reach this preset proficiency threshold, almost twice the number of attempts required by our participants for the same task. It is possible that one explanation for this difference is that participants in the Patel et al. study trained under the traditional “repetition leads to improvement” paradigm, whereas in our study, participants were informed of the goals and maximum number of allowed repetitions. Our approach was based on the construct of “deliberate practice” [21] where learners engage in repetitive performance, receive detailed assessments and are provided with informative feedback. Research shows that deliberate practice leads to better results than traditional repetition- and time-based approaches [22]. In our study, participants were constantly given feedback and were instructed to practice until the proficiency goal was achieved, which is consistent with the deliberate practice methodology. However, due to time constrains and logistical issues, we established a maximum number of trials that participants had to achieve proficiency. The maximum number of trials was set at six for each robotic task following the research by Perrenot et al. [13]. Thus, our training paradigm deviated from the true deliberate practice paradigm.”

One of the major limitations of our study is the inability to predict to what degree the skills acquired in the simulator will transfer to the real world. Despite not finding statistical differences among residents and junior faculty, we presume that experienced surgeons would have an advantage with regards to better familiarization with the anatomy and necessary surgical techniques to perform a given case in clinical practice. Additionally, if trainees are not exposed to an educational curriculum with clear task-oriented goals and expert-derived proficiency benchmarks, one falls into the risk of wasting considerable time in exercises that might not be fundamental for the acquisition of core robotic skills.

In summary, utilization of the dVSS package available for the da Vinci Si (2009) and Xi (2014) models allows trainees to enhance core robotic surgical skills in the same surgeon’s console utilized for real robotic cases in a low stakes environment. Additional advantages of training using the dVSS include saving significant costs in supplies, annual service contract fees, and materials which can become prohibitive when physical or organic models are required. Thus, the use of an exclusive VR platform is a feasible training option for surgical residents and fully trained surgeons interested in learning robotic skills in a safe, controlled setting. A future multi-institutional study with the dVSS, a larger sample size and unlimited trial attempts, is needed to further assess the effectiveness of this training model compared to the traditional repetition based educational approaches.

Conclusion

A dedicated virtual reality robotic training curriculum using the dVSS significantly improves the seven basic robotic surgical skills necessary to operate the da Vinci Si surgical console. Nonetheless, more than six training trials appear to be necessary to reach proficiency levels on more advanced skills.