Introduction

The development of surgical robotics in the 1990s was an important step for minimally invasive surgery. Approved in 2000 by the US Food and Drug Administration (FDA), the da Vinci Surgical System® provides a three-dimensional visualization, seven degrees of freedom mimicking the movements made during standard open surgery, tremor-filtered instrument control, and an ergonomically friendly operative environment. These advantages contribute to the annual increase in surgeons’ adoption of this technology, by overcoming several limitations associated with conventional laparoscopic surgery (CLS) such as poor camera control, depth perception, inverted hand-instrument movements, and limited degrees of freedom of the instrument tips. Though these technical features have been invaluable for some types of minimally invasive surgeries, few studies have objectively evaluated the performance advantages of robotic-assisted laparoscopic surgery (RALS) over CLS [1–4].

It is well recognized that a surgeon’s performance improves with experience and training. The “learning curve” consists of an initial steep (active learning) phase in which learning occurs and performance improved rapidly. This is followed by a plateau phase in which there is little further improvement in performance. During rigorous years of training, surgeons observe, assist, and perform many surgical procedures under supervision to improve their abilities and to achieve this plateau in their learning curve. The highly technical nature of CLS requires operative skills distinct from those of open surgery and is usually associated with a significant learning curve for the novice surgeons [5]. It has been suggested that RALS in comparison with CLS is associated with a quicker learning curve, enabling novice surgeons to perform complex laparoscopic tasks such as knot tying and sewing [6–8]. In addition, it is suggested anecdotally that surgeons especially those in training experience less frustration in performing RALS compared to CLS. The goals of this study were (1) to objectively evaluate the difference in the learning curves by novice and expert surgeons in performing fundamental laparoscopic skills (e.g., peg transfer, precision cutting, and suturing with incorporeal tying) using CLS and RALS and (2) to evaluate the surgeons’ level of frustration in performing these tasks when using these two surgical techniques.

Materials and methods

Forty-three medical students and surgeons were prospectively evaluated in performing three standardized laparoscopic tasks. After consent was obtained, the participants were stratified into two groups: (1) the experienced participants (EPs), composed of residents and surgeons who had at least 6 months of CLS experience including FLS training, training on virtual reality or real environment simulators, and/or participating in patient cases and (2) the novice participants (NPs), composed of medical students and surgeons who has not had any CLS training. None of the participants had any prior RALS training. Both groups attended weekly CLS and RALS training session for five consecutive weeks.

The CLS platform consisted of a training box with three preset ports (FLS Trainer System, VTI Medical, Waltham, MA). A standard color camera with 4.3-mm focus lens provided the 2D optics for the CLS platform. Standard laparoscopic instruments were provided and used at the discretion of the participants. The RALS platform consisted of a da Vinci Standard Surgical System, 30° 12 mm laparoscope with a 3D camera, and standard 8-mm endo-wrist instruments.

At each training session, three laparoscopic tasks (peg transfer, precision cutting, and simple suturing with intracorporeal knot tying) were performed using CLS and RALS. The three CLS tasks were those used in the Fundamentals of Laparoscopic Surgery (FLS) training program. The corresponding RALS tasks were selected from those described by Dulan et al. [9] to replicate the same surgical skill training validated by FLS. The fourth skill in the FLS program (suturing with extracorporeal knot tying) was not performed because there was no robotic equivalent. The duration of the training was selected based on previous literature, which suggested that the plateau phase of the learning curve for the tasks used in FLS could be achieved after 3–5 repetitions [8, 10].

Detailed description of surgical skill tasks

  1. 1.

    Peg transfer: For the CLS version of this task, the participants were asked to move six pegs from one set of posts to another set using two Maryland graspers/dissectors, while transferring the peg from one hand to another before placement. The process was repeated transferring all pegs back to the original set of pegs in the same manner. A notable departure from the FLS program, we did not apply any penalty to the subjects who had dropped a peg, since there are no established standards for assigning penalties with the robotic version of training. In the RALS version of this task, four pegs were moved, one at a time, from the center of the peg board to respective towers placed at 0°, 90°, 180°, and 270° from the center using a large needle driver and a De Bakey forceps. As with the CLS version, the pegs were transferred from one hand to the other prior to placement onto the towers and back to their original positions. The towers were of different heights, requiring the subject to clutch and move the camera accordingly. For both the CLS and RALS version, the time recorded for this task began when the first peg was grasped and ended when the last peg was released.

  2. 2.

    Precision cutting: For both the CLS and RALS version of this task, the participants were asked to cut a marked circle off the surface of a square piece of gauze suspended between clips using a curved scissor and a grasper/dissector (a Maryland for CLS and a De Bakey for RALS). We applied the same penalty provided in the FLS program based in any deviance from the line demarcating the circle in either task. The recorded time started when the gauze was grasped and ended upon the completion of cutting the marked circle.

  3. 3.

    Simple suturing with intracorporeal knot tying: For the CLS version of this task, a penrose drain with an incision along its dorsal aspect and two marking sites on each side were used. The participants were asked to place the suture precisely through two marks using two needle drivers and to tie three square knots (the first one being a surgeon’s knot). If the task was not performed in this precise manner, the participants were asked to repeat the procedure. For the RALS version, the exact same task was performed with the exception that a skin suturing practice pad was used instead of the penrose drain. The recorded time began when the instruments contacted the suture and ended when the last knot was tightened.

The time limit for each task was set to 1200 s. If the time expired before the task was completed, the task was stopped and the maximum time of 1200 s was recorded. The recorded time for each task was collected and plotted for each participant, generating a learning curve.

Measurement of frustration

Each of the participants completed a validated survey on frustration level (the Imperial Stress Assessment Tool (ISAT)) before the first session (Survey Form 1) and subsequently after each session (Survey Form 2) [11]. Survey Form 1 consisted of three sections. In “Introduction” section, basic demographic information was assessed. In “Materials and methods” section, the level of surgical experience was ascertained. In “Results” section, specific baseline attitudes toward laparoscopic and open surgery were determined. Survey Form 2 consisted of two sections. In “Introduction” section, the level of frustration experienced during the session was evaluated, and in “Materials and methods” section, the impact of the frustration experienced was assessed (i.e., the resultant mood and impact on the rest of the day). The subject’s mood was also assessed using a scale from 0 to 10.

Statistical analysis

We estimated a 20 % difference in time as being significant. Based on finding this difference in one of three skills and in one of three parameters evaluated, a power of 0.8, and alpha of 0.05, the sample size was estimated at 10 for each group. Time penalties for inaccuracy were added to task completion times. Summary measures (mean and standard errors) of the recorded time for the completion of each task were calculated according to the surgical approach (CLS or RALS), the participants’ level of experience, and the session number (#). A learning curve was generated for each variable by plotting the mean completion time versus session #. We defined the active learning phase of learning curve as when the slope is greater than 10 s/session and the plateau phase when it is less than 10 s/session. The learning curve was then further characterized by determining: (1) the magnitude of improvement (time @ session #1 − time @ plateau session/time @ plateau session × 100 %); (2) the rate of improvement (slope of the active learning phase of the learning curve), and (3) the quickness of improvement (session # at which plateau was achieved). Statistical models were created to compare the completion times of each task according to the variables above, using generalized estimating equations (GEE) with Poisson’s marginal distribution and link logarithmic function [12, 13]. The interactions between the variables were assessed using Bonferroni multiple or post hoc comparison.

With regard to the frustration surveys, the results of binary questions were described with absolute and relative frequencies and were compared only among the main variables without testing for interactions due to the small number of samples. The results of the binary questions were compared using GEE with marginal binomial distribution and logic link function. For the remaining questions in the survey in which a scale of 0–10 was used, the results were described using means and standard deviations according to each main variable. Adjusted models were used with the same methodology, but with marginal Poisson’s or Gaussian distribution and logarithmic link function. All statistical analyses were performed using SPSS 15.0 software with a significance level set at 5 %.

Results

Of the 43 participants enrolled in this study, 12 had previous experience in CLS (EPs), and 31 were novice (NPs). Thirty-seven participants completed all five sessions, while six subjects completed only four of the five sessions. The times for completion were plotted for each task performed and separated according to the experience level and the surgical approach.

Peg transfer

When this task was performed with CLS, both the EPs and NPs achieved a plateau of the learning curve after the second session (Fig. 1). The EPs had a lower mean completion time for this task at every corresponding session compared to their novice counterparts (p < 0.01). Presumably, because the EPs already had experience with this task, they had less magnitude (45 vs. 65 %) and lower rate of performance improvement (30 vs. 90 s/session) compared to their novice counterparts (Table 1).

Fig. 1
figure 1

Learning curves of peg transfer skill according to the surgical technique (conventional laparoscopy vs. robotic-assisted) and the participants’ experience level

Full size image
Table 1 Characterization of the improvement in performance based on surgical technique, experience level, and session number
Full size table

When this task was performed with RALS, both the EPs and NPs achieved a plateau of the learning curve after the third session (Fig. 1). Like with CLS, the EPs had a lower mean completion time for this task at every corresponding session compared to their novice counterparts (p < 0.01). While neither the EPs nor the NPs had any prior RALS training, they had similar magnitude (67 vs. 68 %, respectively) and rate of performance improvement (105 vs. 86 s/session, respectively), indicating that prior laparoscopic training was not required for RALS skill acquisition (Table 1). With this task, the absolute completion times could not be directly compared between the CLS and RALS approach, since the exercises were not completely congruent (i.e., the RALS exercise took longer to perform because there were more camera and instrument movements involved).

On univariate analysis, experience level, RALS, and latter session # were all statistically significant predictors of completing the peg transfer skill more quickly (Table 2). However, on multivariate analysis, only the latter session # in combination with each of the other two variables was statistically significant. This finding suggests that with the peg transfer skill, repetition was more important than the surgical technique (CLS vs. RALS) for both the EPs and NPs.

Table 2 Multivariate analysis of surgical skills time performance based on experience level, surgical technique, and session number
Full size table

Precision cutting

When this task was performed with CLS, the EPs did not demonstrate any significant change in completion time from the first to the fifth session (p > 0.1), though they had a lower mean completion time for this task at every corresponding session compared to their novice counterparts (p < 0.01) (Fig. 2). Again these findings are presumably due to the fact the EPs had prior experience with this or similar surgical skill. In contrast, the NPs demonstrated continued improvement in the completion time with a plateau in the learning curve after the fourth session. They achieved 68 % improvement in performance at a rate of 50 s/session (Table 1).

Fig. 2
figure 2

Learning curves of precision cutting skill according to the surgical technique (conventional laparoscopy vs. robotic-assisted) and the participants’ experience level

Full size image

When this task was performed with RALS, the EPs achieved a plateau of the learning curve after the second session, while the NPs after the fourth session (Fig. 2). The EPs had a lower mean completion time for this task at every corresponding session (p < 0.01), had less magnitude of improvement in performance (36 vs. 66 %), and had comparable rate of improvement in performance (52 vs. 30 s/session) compared to their novice counterparts (Table 1).

With this specific task, the completion time between CLS and RALS can be directly compared. The EPs completed the cutting task in comparable time to that with using CLS on the first session (Fig. 2). However, from the second to the last session, the EPs tended to complete the task more rapidly with RALS; however, this difference did not reach statistical significance (p = 0.07). In contrast, the NPs required less time to complete the task using RALS compared to CLS from the first to the fifth session (p = 0.01).

On univariate analysis, experience level, RALS, and latter session # were all statistically significant predictors of completing the cutting task more quickly (Table 2). However, on multivariate analysis, none of the variables in combination were statistically significant. This finding suggests that a large sample size is required to further delineate the correlation between these variables in predicting performance in the precise cutting task.

Simple suturing with intracorporeal knot tying

When this task was performed with CLS, the EPs did not demonstrate any significant change in completion time from the first to the fifth session (p > 0.1), though they had a lower mean completion time at every corresponding session compared to their novice counterparts (p < 0.01) (Fig. 3). Again these findings are presumably due to the fact the EPs had prior experience with this or similar surgical skill. In contrast, the NPs demonstrated continued improvement in the completion time with a plateau in the learning curve after the fourth session. They achieved 118 % improvement in performance at a rate of 101 s/session (Table 1).

Fig. 3
figure 3

Learning curves of simple suturing with intracorporeal knot tying skill according to the surgical technique (conventional laparoscopy vs. robotic-assisted) and the participants’ experience level

Full size image

When this task was performed with RALS, the EPs had a similar result to that of performing the task with CLS, with no improvement in their completion time from the first to the fifth session (p = 0.50) but were able to complete the skill in faster times compared to the NPs at every corresponding session (Fig. 3). In contrast, the NPs demonstrated continued improvement in the completion time with a plateau in the learning curve after the fourth session. They achieved 80 % improvement in performance at a rate of 53 s/session (Table 1).

With this specific task, the completion times using CLS or RALS can be directly compared. For the EPs, there was no difference in the completion time between CLS and RALS (p = 0.20) (Fig. 3). However, for the NPs, the RALS times were shorter (p < 0.01). By the last session, they were able to complete the task not only faster than when using CLS but also had times comparable to the times of their experienced counterparts.

On univariate analysis, experience level, RALS, and latter session # were all statistically significant predictors of completing the suturing/tying more quickly (Table 2). However, on multivariate analysis, only the combination of RALS and latter session # was statistically significant (p < 0.01). This finding suggests that the use of RALS in repetition will help improve performance of this task. Whether or not this is needed for both the EPs and NPs remains unclear, since the experience level trended toward significance when combined with the other two variables (p = 0.07–0.08).

Frustration

There were significant differences in the level of frustration based on the participant’s level of surgical experience, the use of CLS vs. RALS, the surgical task being performed, and the session number (Table 3). As expected, the EPs had fewer number of sessions (%) in which they felt frustrated (18 vs. 45 %, respectively, p < 0.01), had fewer episodes of frustration during each session (mean = 0.3 vs. 1.09, p < 0.01), and had lower degree of frustration (mean = 0.52 vs. 2.13, p < 0.01) compared to their novice counterparts regardless of the task being performed or the use of CLS or RALS. Moreover, the EPs reported being in better mood (mean = 8.25 vs. 7.02, p < 0.01) and less frustrated (mean = 0.27 vs. 2.14, p < 0.01) after performing the surgical tasks than their novice counterparts.

Table 3 Assessment of frustration and mood during and following a training session based on experience level, surgical technique, session number, and the surgical skill performed
Full size table

In general, the use of RALS was associated with less frustration. With both EPs and NPs, frustration was observed in 30 % of the RALS tasks compared to 47 % of the CLS tasks (p < 0.01). The mean episodes (0.47 vs. 1.31, p < 0.01) and the degree of frustration during each task (mean = 2.31 vs. 1.11, p < 0.01) were also lower when performing the tasks with RALS compared to CLS. Similarly, the participants reported being a better mood (mean = 7.61 vs. 7.07, p < 0.01) and less frustration (mean = 1.32 vs. 1.99, p < 0.01) after using RALS than CLS.

As expected, the participants were more likely to feel frustrated during the first initial sessions compared to the latter ones (59 % on the first vs. 29 % on the fifth, p < 0.01, Table 2). In addition, the first sessions elicited a higher number of frustrating events while performing the task (mean = 1.64 on the first vs. 0.64 on the fifth, p < 0.01), as well as more intense level of frustration (mean = 2.52 on the first vs. 1.24 on the fifth, p < 0.01) compared to the last session. When comparing the initial sessions compared to the latter sessions, the participants’ mood was similarly significantly improved (mean = 7.05 on the first vs. 7.71 on the fifth, p < 0.01), and they had less frustration afterward (mean = 2.32 on the first vs. 1.29 on the fifth, p < 0.01).

Finally, performing certain skills elicited greater frustration than others (Table 2). For both the EPs and NPs, the peg transfer skill was associated with significantly less mean number of sessions in which the participants felt frustration, mean frustration score during and after, compared to the cutting (p = 0.01) and the suturing/tying task (p < 0.01). However, there was no difference in the above-mentioned aspects of frustration when comparing the cutting with the suturing/tying task (p > 0.20). Mean number of times felt frustrated during a session was most significant when performing the suturing/tying skill (p < 0.01), while mood was not statistically different between the three skills.

Discussion

A significant amount of practice in performing surgery is necessary to achieve consistency and to improve accuracy and dexterity. This process can be represented by an individual’s learning curve for a specific surgical task or skill. In this study, we characterized the learning curve associated with performing three laparoscopic tasks with variable degree of difficulty using RALS and CLS by experienced and novice surgeons. In general, for the novice surgeons, RALS allowed for better performance as measured by time to completion and accuracy (since surgical errors were assigned specific time penalties). For complex tasks such as precise cutting or suturing/tying, it required only four sessions before achieving a plateau in performance. The novice surgeons were able to acquire RALS surgical skills in a similar manner to the experienced surgeons, suggesting that prior laparoscopic experience is not an absolute requirement for RALS learning. Importantly, significant and rapid improvement in performance as measured by magnitude, rate, and quickness was also seen with CLS. For the experienced surgeons, the use of RALS provided only marginal improvement in performance with cutting and in suturing/tying task (as the peg transfer task cannot be directly compared between RALS and CLS with the methodology used in this study). For both the experienced and novice surgeons, the use of RALS was associated with less number of sessions in which they felt frustrated, less number of frustration episodes during a session, lower frustration score during and after the session, and higher good mood score.

Our findings are similar to those in prior studies in which the use of RALS resulted in improved performance of complex surgical skills especially by less experienced surgeons and that prior laparoscopic skills were not required in the learning of RALS. Stefanidis et al. [14] compared the performance of inexperienced surgeons in performing laparoscopic intracorporeal suturing in a live, porcine, Nissen fundoplication models using CLS and RALS. The authors observed that RALS enabled the participants to perform the task faster, more accurately and securely, and with fewer errors. In addition, they noted that robotic assistance significantly shortened the learning curve associated with performing laparoscopic suturing. Blavier et al. [10] compared the performance of inexperienced surgeons passing a needle through a series of rings using CLS and RLS with 2D and 3D view. The authors observed that RALS improved surgical performance and learning, particularly by the 3D view advantage.

Chandra et al. [15] compared the performance of experienced and novice surgeons in performing suturing in a virtual reality surgical simulator using CLS and RALS. The authors observed that for novice surgeons, RALS provided improved performance as measured by total task time, more economic instrument path length, and smoothness. However, for the experienced surgeons, RALS provided only improvement in the economy of movement as measured by instrument path length. Similarly, Kim et al. [4] observed that in their study, RALS had better performance than CLS in all tasks during the two trials. However, these results were more noticeable for the novice participants. Finally, in a review of 89 articles, Kenngott et al. [16] observed that in experimental studies, current robotic systems have proven their superior suturing capabilities compared to conventional laparoscopic techniques, mainly attributed to 3D visualization and full seven degrees of freedom. However, in clinical studies, these benefits have not yet been sufficiently reproduced.

In this study, we provided a novel method of characterizing the learning curve based on the magnitude, rate, and quickness of improvement in performance. Based on these parameters, we observed that the learning curves associated with CLS for the three laparoscopic skills tested were similar to those of RALS for the novice surgeons. Heemskerk et al. evaluated eight novice surgeons performing 176 laparoscopic tasks with RALS and CLS. The authors similarly observed that though RALS allowed for faster and more accurate completion of the skills, CLS showed faster skill acquisition [3]. Taken together, these findings suggest that practice and repetition would allow for learning even of more complex laparoscopic tasks such as suturing/tying, regardless of whether a surgeon is using RALS or CLS technique.

In addition, we observed that the use of RALS resulted in less frustration and improved mood for both experienced and novice surgeons. Although a certain degree of stress can help task performance, excessive levels can result in deterioration of abilities and can clearly have an impact on patient safety [5, 8]. In addition, less frustration experienced during surgical performance may encourage an individual to adopt a technique more readily and to be more apt to use and practice it. Stefanidis et al. [14] similarly observed in their study that RALS was associated with less frustration and less mental and physical demands. In our study, we observed a statistically significance but not a vastly large magnitude of difference in frustration parameters between participants using RALS and CLS. However, it should be noted that this was the amount of frustration experienced while performing this task only once during the session. Consequently, this finding still may have clinical relevance since during a procedure, a task such as suturing/tying is repeated numerous times, allowing the level of frustration to become additive over time.

There are several limitations to our study. First, the basic fundamental skills for CLS and RALS are inherently different. It is not feasible to use the same test to assess CLS and RALS due to inherent technological difference between CLS and RALS. For example, in CLS when the surgeon wants to move and manipulate objects at different heights, the surgeon’s hands and eyes automatically make the adjustments. For this to occur with RALS, the surgeon has to initiate clutching in addition to moving his eyes and hands, so that the robotic arms can reach the desired height. The surgeon could still perform the action without clutching, but this would be outside of the optimal functional parameters of the robot. Consequently, in performing the peg transfer, RALS evaluation has to incorporate clutching to test for movement in order to have a fair comparison when the instruments are optimally functioning. The analogy would be likely to provide the surgeon with a laparoscopic holder in which one of the jaws does not close. Consequently, we are not directly comparing time, but rather we are comparing the learning curve (which is reference back to each individual’s time and performance at the first session and with each technique). That is, all the time using CLS are compared to the first session using CLS only. Similarly, with regard to frustration, it is measured with respect to each technique as a function of number of sessions. We then compare the CLS and RALS’ learning curves based on specific parameters that characterize the learning curves such as magnitude, rate, and quickness. While there is a difference in the actual skills themselves, we believe comparing learning curves to be a fair in evaluating CLS and RALS skill acquisition. Second, we did not assess other measurements of performance other than time and accuracy. Scoring systems such as the Robotic Skills Assessment Score (RSA Score) [17] and the Global Operative Assessment of Laparoscopic Skills (GOALS) [18] have been utilized to further characterize performance with parameters such as depth perception, tissue handling, dexterity, efficiency/economy of movement, and autonomy. However, we were unable to identify a scoring system that has been validated for both RALS and CLS in order to allow for a comparison. For the same reason, we were not able to utilize more complex measurement of performance such computerized instrument path and motion assessment [19] in this study. Third, our analysis of the plateau phase of the learning curve is just one interpretation of when learning has been achieved and may not be applicable to all surgeons. It has been suggested that there are no objective measures that mark the achievement of learning [20]. This depends on an individual’s comfort, experiences, and results with alternative approaches. Surgeons whose sole or predominant experience is with CLS or RALS may have a different perception of the learning curve compared with an experienced open surgeon. Fourth, we did not evaluate numerous surgical skills/tasks. We limited our evaluation to the above-mentioned three surgical skills, because they have been thoroughly evaluated and validated in FLS. Though it would have been ideal to perform the peg transfer skill in the same exact manner for both CLS and RALS assessment, we felt that the peg transfer skill utilized for FLS was too simplistic for RALS assessment and did not adequately assess camera and instrument clutching which is inherent in RALS. Fifth, the number of participants in our study is smaller in comparison with some study in the literature. Nevertheless, we were able to observe statistically significant findings. Finally, it is unknown how these findings would extrapolate to the actual operating environment. To answer this, additional evaluation has to be done when the participants are performing surgery on patients or an animal model, which is beyond of the scope and capacity of this study.

Conclusion

It has been suggested that CLS can be rather demanding due to several technical drawbacks (such as 2D vision and limited degree of freedom in the instruments) which has limited the application of minimally invasive surgery to a variety of cases [6, 7, 21, 22]. The advent of RALS has addressed and provided a solution to these technical limitation of CLS, and hence, is thought to be superior to CLS. From this study, we concluded that though RALS does provide for more rapid completion of certain surgical tasks with less frustration, the advantages of RALS may be of most benefit when doing more complex tasks and by less experienced surgeons. We suggest that repetition and practice are as equally important as the choice of surgical technique in the acquisition of laparoscopic skills.