The use of minimally invasive surgical techniques has become the standard of care in gynecologic and other surgical specialties. It is important for trainees to acquire laparoscopic and robotic surgical skills efficiently and without compromising patient safety. This is particularly relevant in light of resident work hour restrictions and subsequent time-limited training pathways. Surgical simulation provides an alternative paradigm to supplement the traditional apprenticeship model of training. Simulation has been shown to promote laparoscopic and robotic skills acquisition, improve surgical performance, and decrease operative times [1–3]. Between the two minimally invasive platforms, several studies demonstrate that new learners achieve proficiency in robotic skills more easily than in laparoscopic skills, and learners can perform robotic surgery without compromising patient outcomes during the learning period [4, 5].

Ideally, proficiency attained in either robotic or laparoscopic surgery would translate to the other technique with low rates of skills deterioration. However, current research into the transferability of surgical skills between laparoscopic and robotic simulation platforms is limited. Some studies indicate that there is unlikely to be any major advantage garnered by prior laparoscopic experience when attempting to learn robotic surgical methods [6], while others demonstrate significant transferability of skills between modalities [7, 8]. Moreover, there is a lack of data concerning medium- and long-term retention of skills on both platforms, though it does appear that proficiency acquired through simulation in a single modality is maintained [9, 10].

In order to assess the retention of minimally invasive surgical skills, we present the second stage of a randomized single-blinded controlled trial. In Stage 1, skills-naïve participants were evaluated on both laparoscopic and robotic simulators before and after training on one simulation modality. During Stage 2, we sought to evaluate retention of skills using both objective parameters and a modified previously validated global rating scale.

Materials and methods

Study design

We previously completed the first stage of a randomized single-blinded controlled trial in which 40 Johns Hopkins University School of Medicine students without previous simulation or operative experience were randomly assigned to perform ten repetitions of peg transfer tasks on either the laparoscopic box trainer (N = 20, Fundamentals of Laparoscopic Surgery, Venture Technologies Inc., Waltham, MA) or robotic computer-based simulation (N = 20, dV-Trainer, Mimic, Seattle, WA) platforms. These tasks were chosen because they were analogous between training platforms with similar required motions and instrument techniques. Previous research has demonstrated significant improvement in time to task completion for peg transfer exercises with successive practice repetitions, with the most notable improvement demonstrated in the first nine repetitions [11]. The initial training sessions occurred between November 26 and December 17, 2014. Two expert minimally invasive surgeons evaluated participants on both simulation platforms before and immediately after practice sessions using a modified version of previously validated global rating scales [12–14]. Evaluators were blinded to students’ training platform. Detailed information about the initial study including descriptions of randomization methods, training protocols, and performance metrics has been described elsewhere [15].

The second phase of this randomized controlled trial occurred between February 24 and April 18, 2015. All 40 students who participated in Stage 1 were contacted via e-mail and asked to return for Stage 2. Stage 2 involved participants performing the laparoscopic and robotic peg transfer task once each without a practice session in the minimally invasive training suite at our institution. During Stage 1, randomization of which task participants performed first was determined using Microsoft Excel random number generator. Participants in Stage 2 followed the same task order as in Stage 1. Blinding protocols for procedures, evaluators, and performance metrics were identical to the first stage of the trial. This study received approval from the Johns Hopkins Institutional Review Board (IRB00031436).

Outcome measures

During the initial study, participants self-reported demographic information including age, gender, medical school class year, handedness, and experience playing musical instruments, organized sports, and video games. Outcome measures for Stage 2 were the same as those for Stage 1 and consisted of both objective and subjective metrics for the laparoscopic and robotic peg transfer tasks. Primary objective parameters included time to task completion (manually recorded in seconds using a stopwatch for the laparoscopic task and automatically recorded by the dV-Trainer for the robotic task), error rate (number of peg drops, manually recorded in the laparoscopic task and automatically recorded in the robotic task), and motion metrics captured by the Mimic dV-Trainer. These motion metrics included economy of motion (cm, where higher numbers indicated less economy of motion), number of instrument collisions, time spent using excessive force (seconds), time instruments which were out of view (seconds), and workspace range (cm).

Subjective parameters consisted of the component and total scores of a modified global rating scale of laparoscopic and robotic operative performance. The scale was derived from the combination of three previously validated rating scales for open (OSATS, Objective Structured Assessment of Technical Skills) [14], laparoscopic (GOALS, Global Operative Assessment of Laparoscopic Skills) [13], and robotic (GEARS, Global Evaluative Assessment of Robotic Skills) [12] surgery. The scale was modified to apply to both the laparoscopic and robotic simulation settings while avoiding redundancy. The resulting modified global rating scale of laparoscopic and robotic operative performance included seven components: (1) depth perception, (2) bimanual dexterity, (3) efficiency, (4) tissue handling, (5) time and motion, (6) instrument handling, and (7) flow of operation. Each component was graded on a five-point anchored Likert scale with higher scores indicating higher proficiency. Components from the aforementioned previously validated global rating scales that were not considered relevant to our experimental setting included “Knowledge of Instruments” [14], “Use of Assistants” [14], “Knowledge of Specific Procedure” [14], “Autonomy” [12, 13], and “Robotic Control” [12]. “Force Sensitivity” [12] and “Respect for Tissue” [14] were considered redundant with the inclusion of “Tissue Handling.”

Statistical analysis

Participants were grouped by minimally invasive training platform (laparoscopic or robotic), and their performances on both simulators in Stage 2 were compared using Student’s t tests for continuous variables to assess for transferability of skills. Additionally, participants’ time to task completion, error rates, and modified global rating scale scores from Stage 1 were recorded for comparison. We assessed participants’ Stage 1 and Stage 2 performances on the laparoscopic and robotic tasks, respectively, to evaluate retention of skills. Data analysis was performed using Stata/IC software, version 13.1 (Stata Corporation, College Station, TX). A p value <.05 was considered statistically significant.

Results

Of the 40 students who participated in the initial phase (Stage 1), 23 participants (11 laparoscopic-trained and 12 robotic-trained) underwent repeat evaluation in the retention phase (Stage 2, Fig. 1). Mean retention time was 15.6 weeks (range 11.4–19.6 weeks). There were no significant differences in demographic characteristics between Stage 2 participants in the laparoscopic and robotic training groups (Table 1).

Fig. 1
figure 1

Methodology for Stages 1 and 2 of this randomized controlled trial

Full size image
Table 1 Demographic characteristics for participants who underwent repeat evaluation 16 weeks (range 11.4–19.6) after simulation, by training platform (N = 23)
Full size table

Evaluation of the laparoscopic peg transfer task approximately 16 weeks after simulation training showed no difference in performances between participants randomized to laparoscopic or robotic practice in Stage 1 (Table 2). There were no significant differences between groups in objective metrics (time to task completion and peg drop count) or modified global rating scale measures.

Table 2 Repeat evaluation of laparoscopic peg transfer task 16 weeks (range 11.4–19.6) after simulation training, by training platform (N = 23)
Full size table

Evaluation on the robotic peg transfer task demonstrated fewer peg drops (p = .018) and improved economy of motion (p = .017) among robotic practice group participants compared to the laparoscopic practice group (Table 3). Furthermore, participants randomized to robotic practice demonstrated a trend toward improved performance on all modified global rating scale components and the composite measure, though the only significant difference was seen in Tissue Handling (p = .020).

Table 3 Repeat evaluation of robotic peg transfer task 16 weeks (range 11.4–19.6) after simulation training, by training platform (N = 23)
Full size table

To investigate retention of skills, we compared laparoscopic-trained participants’ performances on the laparoscopic task and robotic-trained participants’ performances on the robotic task at three time points: (1) prior to training (Stage 1: Baseline), (2) immediately after training (Stage 1: Post-training), and (3) at repeat evaluation 16 weeks later on average (Stage 2). Comparison of results from Stage 1: Baseline and Stage 2 indicates skills acquisition from baseline, whereas comparison of results from Stage 1: Post-training and Stage 2 indicates how well skills were retained after they were acquired.

Laparoscopic-trained participants’ performances on the laparoscopic task were significantly better 16 weeks after initial practice (Stage 2) compared with baseline (Stage 1: Baseline) as measured by time to task completion, but not by the modified global rating scale composite or component measures. Table 4 outlines the performances at various time points of laparoscopic-trained participants who participated in both stages of the trial. Furthermore, there was skills deterioration as evidenced by significantly worse objective (time to task completion) and subjective performances (modified global rating scale total score and the following component measures: depth perception, efficiency, tissue handling, time and motion, and flow of operation) in Stage 2 compared with Stage 1: Post-training (Table 4).

Table 4 Comparison of laparoscopic-trained participants’ performances on the laparoscopic task 16 weeks (range 11.4–19.6 weeks) after training (Stage 2) with their performances prior to training (Stage 1: Baseline) and immediately after training (Stage 1: Post-training) (N = 11)
Full size table

Robotic-trained participants’ performances on the robotic task were significantly better 16 weeks after initial practice (Stage 2) compared with baseline (Stage 1: Baseline) as measured by time to task completion, peg drop count, economy of motion, and the modified global rating scale composite score and all component measures. Table 5 details the performances of robotic-trained participants who participated in both stages of the trial. There was skills deterioration evidenced by time to task completion and economy of motion results. However, global rating scale composite scores, all component scores, and peg drop counts were not significantly different in Stage 2 compared with Stage 1: Post-training.

Table 5 Comparison of robotic-trained participants’ performances on the robotic task 16 weeks (range 11.4–19.6 weeks) after training (Stage 2) with their performances prior to training (Stage 1: Baseline) and immediately after training (Stage 1: Post-training) (N = 12)
Full size table

Comment

Our findings reveal no differences between laparoscopic and robotic training groups during the laparoscopic simulation task. However, there was a trend toward better performance on the robotic platform among participants who trained on the robotic simulator. Robotic skills acquired through simulation appear to be better maintained than laparoscopic skills.

Prior research into laparoscopic simulator training indicates that skills retention is impacted by the initial training intervention and simulation task complexity. In a 2010 study by Rosenthal and colleagues, 100% of fifteen skills-naïve medical students who completed the Fundamentals of Laparoscopic Surgery program retained sufficient knowledge and abilities at 6 months and 12 months to pass the certification exam [16]. Conversely, some studies demonstrate substantial decreases in laparoscopic simulator performance metrics over 6–12 months [17, 18], while others report deterioration for only more complex laparoscopic tasks [9, 18]. Our trial involved a 1-day training intervention and is therefore not entirely analogous to studies that use more extensive minimally invasive surgical curricula.

As each of our participants practiced one exercise for ten repetitions during training, we did not address the optimal duration and interval of training, which is an important area for future research. Repeated learning and practice with simulation trainers appears to be more beneficial for long-term skills retention. Spruit et al. [19] examined the optimal schedule for laparoscopic skills training and determined that spacing training over 3 weeks compared to massed training on a single day led to better performance at 2 weeks and at 1 year. Another group of researchers found that “Optional Deliberate Practice” between training sessions is effective at reducing deterioration of laparoscopic skills [20].

Some researchers suggest that skills deterioration occurs over an even shorter period of time without training. Varley et al. [21] found that laparoscopic simulation training allowed for acquisition and retention of skills for at least 4 weeks, though there was significant performance deterioration by 12 weeks. These results are similar to our findings. When compared with testing immediately after practice, laparoscopic-trained participants took longer to complete the simulation task and had lower global rating scale scores during repeat evaluation 16 weeks later. However, laparoscopic practice did confer improved speed 16 weeks after training compared to baseline performance. Therefore, although participants retain some improvements in speed after a one-day training intervention without increased error, the improvement in quality of performance was not retained over 16 weeks.

Previous research by Teishima et al. [10] using the Mimic dV-Trainer (Mimic, Seattle, WA) to teach robotic skills to urologic surgeons with limited experience demonstrated that proficiency gained through simulation training was maintained 1 year after the intervention without additional practice. Our results correspond with this study and support the use of simulation in robotic skills training. We found that improved quality of performance measured by the composite global rating scale was retained over 16 weeks among the robotic practice group. Additionally, robotic practice was associated with improved speed and economy of motion compared to baseline, though participants’ performances on these objective parameters were worse than on immediately after training.

An important strength of this study is the participation of skills-naïve medical students rather than physicians with varying skill levels. This naïve cohort allowed for an accurate assessment of the impact of initial simulation training on retention of skills. One limitation was that participants were scheduled for repeat evaluation (Stage 2) based on convenience. Although times from training to re-testing during Stage 2 were not uniform, all participants were tested at least 11 weeks after initial training, which is comparable to other studies in the literature. Additionally, participants were not specifically queried about their exposure to minimally invasive surgical instruments or activity during the intervening time between Stages 1 and 2. However, 21 of the 23 students who completed Stage 2 of the study were in their first two (preclinical) years of medical school. There is no formal minimally invasive training built into the curriculum at this stage. The remaining two participants were medical students in their third year. Although it is possible that these students completed surgical clinical clerkships during the period between Stages 1 and 2, it is unlikely that they played active roles in the minimally invasive portion of surgical cases given the emphasis placed on training residents at our institution. Only 58% of Stage 1 participants were available for participation in Stage 2. Although this participant retention rate is similar to other published studies [9], it limits overall study power.

In conclusion, the second stage of this randomized controlled trial revealed that robotic skills acquired through simulation are better maintained than laparoscopic skills. To our knowledge, this is one of the few studies to examine the impact of minimally invasive simulation training modality on the transferability of skills between laparoscopic and robotic simulation platforms and the retention of those skills. Further research is needed to determine optimal duration and interval of training, and how simulation skills relate to performance in the operating room and patient care outcomes.