Introduction

Minimally-invasive robotic surgery is increasingly used during major uro-oncological interventions [1, 2]. Its implementation strengthened the concept of “precision surgery” [3] which represents the next step towards the transition to the most effective treatment with minimal invasiveness for the patient [4]. Current robotic surgery makes use of technologically advanced platforms that adhere to the “master–slave” principle, wherein the surgeon—seated behind a sophisticated console—remotely controls the telemetry of the robot. The resulting synergism between man and machine provides the ability to accurately monitor the robot’s every move and to use this information to refine the feedback to the surgeon by applying artificial intelligence (AI).

Increasing the surgical accuracy via robotics comes with an increased technical complexity. This means that achievement of surgical proficiency requires the development of more advanced training models [5, 6]. In this context, modern, automatized and objective approaches for assisting this learning process such as AI could play an important role. Conversely, the same technologies may aid proficient surgeons by enhancing their interpretation of the surgical field (e.g., enhancing surgical guidance via augmented reality) [3].

AI systems entail computers that are trained to solve problems by mimicking human cognition. Machine learning (ML) and deep learning (DL) models are subfields of artificial intelligence that allow the computer to make predictions based on underlying data patterns. Evidence suggest that the interaction between medical professionals and ML algorithms improve the decision-making process by decreasing the error rate [7]. For instance, a ML approach using diagnostic and therapeutic information was able to improve lung cancer staging compared to a method based solely on clinical guidelines (accuracy of 93% for ML vs. 72% for the clinical approach) [8]. By allowing for the processing of multiple data sources and clinically valuable feedback, ML has shown higher predictive accuracy than classic statistical methods [9]. Understandably, AI induced reduction of the surgical error rate would increase the surgeon’s performance [10]. By actively stimulating the implementation of AI, it may be possible in the short term to reshape the surgical workflow in terms of precision, efficiency, and personalized approach for each specific patient profile.

The aim of the current narrative review was to summarize the available literature on AI methods used in robotic surgery, both during the training process and applied in clinical care.

Methods

A literature search of the MEDLINE/PubMed and Scopus database was performed using the following keywords: artificial intelligence, autonomous surgery, machine learning, robotic surgery, and surgical navigation. All relevant English-language original and review studies published between January 2015 and June 2019 were analyzed by three authors (IA, EM, PD) and summarized after an interactive peer-review process of the panel.

Literature review

Although ML algorithms have been successfully implemented in a large number of domains (e. g., economics, engineering, telecommunication, and medical diagnosis) their use in the surgical field is still scarce [11]. Moreover, only few studies specifically focused on applying ML in robotic surgery [12,13,14,15,16,17,18]. These analyses reported promising results in terms of development of an objective and automated tool for skills assessment, increasing efficiency of operating room use and predicting surgical outcomes (Table 1).

Table 1 Artificial intelligence models implemented in robotic surgery
Full size table

AI and surgical skill assessment

The growing public concern regarding surgeon proficiency [12], as well as the widespread implementation of robotic surgery have led to an increased need for adequate structured training models and objective evaluation tools of the clinical competence. Several methods were developed to improve the traditional training modality, as the novel surgical techniques require adapted training for both technical and non-technical skills [19]. Combined use of dry-lab, wet-lab, mentored training, and clinical practice has proven to be effective [20, 21]. These modular training methods allow the acquisition of adequate robotic surgical skills under the supervision of expert surgeons. Despite the crucial importance of assessing trainees’ performances and the presence of structured programs for trainees [6, 22,23,24], subjective assessment, lack of consistency and limited interobserver reproducibility represent important biases [25,26,27]. Therefore, there is an impending need for developing objective metrics that provide a completely unbiased evaluation [28].

Using ML algorithms that integrate motion analysis, energy, and force usage, it is possible to obtain an automated, accurate and quantitative skills assessment [12,13,14]. The key step in the ML based skills assessment is to extract and quantify meaningful features of the surgical gestures that can be representative of surgeon dexterity. These tools may be used to provide feedback during the surgeon’s learning curve as well as to ensure periodic surgeon re-evaluation and credentialing [29].

The first experience with ML methods to evaluate robotic surgery skills level was reported by Fard et al. [12] (Table 1). The authors recorded metrics of eight robotic surgeons with varying levels of experience (novice and experts) while each of them performed for five times surgical tasks such as suturing and knot-tying. Kinematic data of the robotic arms was extracted and quantified using the following global-movement metrics: task completion time, path length, depth perception, speed, motion smoothness, curvature, turning angle, and tortuosity. Based on these data, the authors demonstrated with 90% accuracy that the automated ML algorithm allowed differentiation between a novice and expert surgeon just a few seconds after tasks completion. However, given the small number of participants, future studies are needed to validate this model on a larger cohort.

Similarly, Wang et al. [13] (Table 1) developed a DL model based on an artificial neural network, which represents a more complex ML approach. This model was inspired by the inter-neuronal connections in biological nervous systems, combining different ML algorithms to process data and learn from input information. They considered pattern recognition to be the most important application of DL algorithms. As such, their model was used to recognize motion characteristics during robotic surgery training. Eight surgeons were assessed while performing five trials of different tasks, such as suturing, knot-tying and needle passing. The artificial neural network model showed an accuracy of > 90% for robotic surgical skills assessment within 1–3 s. This feedback was provided without the need for gesture segmentation or entire trial observation.

Ershad et al. [14] (Table 1; Fig. 1) proposed an alternative ML skills assessment, namely the analysis of the surgeon’s “movement style”. The rationale of this approach stems from the fact that an expert surgeon may execute maneuvers with ease and efficacy, in a calm, relaxed, and coordinated fashion compared to a novice surgeon. They obtained kinematic data from 14 surgeons with different experience level in robotic surgery, each of whom performed two tasks (ring and rail and suture) for three times. Movement tracking was realized by applying three-dimensional electromagnetic tracking of the surgeons shoulder, wrist and hand positions during virtual simulator training [14]. Training videos were categorized by crowdsourcing using a number of adjectives that describe the style of behavior. This was subsequently used to train a classifier model. The authors reported a 68.5% increase in skills level classification accuracy compared to raw kinematic data acquisition. One potential advantage of this approach is that it eliminates the need for adequate surgical expertise for interpreting surgical skill level, since it takes into account qualitative motion features (e.g., smooth, calm, and coordinated) that are surgeon-dependent, rather than task-dependent.

Fig. 1
figure 1

Sensor positioning: human subject conducted simulated tasks using the da Vinci Standalone simulator while wearing an electromagnet tracker. Reproduced with permission

Full size image

AI to overcome the lack of haptic feedback

Although robotic surgery routinely implements 3D visualization of the surgical field and improves the surgeon’s dexterity during procedures, the dissection movements, applied force and assessment of tissue responses are based only on visual cues. The lack of haptic feedback might impair the surgical outcomes by excessive force applied to fragile structures, disruption of anatomical planes and suture breakage. For instance, excessive traction applied on the neuro-vascular bundles during robotic radical prostatectomy (RARP) leads to neuropraxia and delay in the recovery of the sexual function [3]. On the contrary, application of insufficient force during knot-tying can lead to suture failure. In this regard, Dai et al. [16] (Table 1) developed a suture breakage warning system that uses sensors mounted on the robotic instruments and provides vibrotactile feedback when the tension is approaching the suture’s failure point. The authors showed that the use of the haptic feedback system significantly reduced suture breakage by 59%, ensured 3.8% less knot slippage and showed higher consistency in repeated tasks in novice surgeons. Furthermore, by practicing with the tactile feedback system, the surgeons showed lower rates of suture breakage (17% for the first 4 knots vs. 2% for the following 6 knots, p < 0.05) [16]. Conversely, this was not observed when surgeons practiced without haptic feedback, where a high rate of suture breakage was reported (20%) [16].

AI and its impact on surgical logistics

Although there is a significant number of robotic systems available worldwide, healthcare systems consider the costs of surgery to be high [2]. Various strategies have been discussed to reduce costs, such as minimizing the instrumentation [30, 31], increasing the case-load or starting multi-specialty robotic programs [32]. All these strategies remain at surgeon or hospital discretion. One of the most important factors affecting surgical costs is the operative time (OT) [33]. A more accurate prediction of OT could help improve logistics, something that is already routinely applied in industry [17, 33].

Zhao et al. [17] (Table 1) showed that a ML approach can also significantly refine the time estimation of robotic procedures. The authors developed a model to predict case duration for robotic surgery including 28 factors related to the patient (e.g., age, obesity, malignancy, tumor location, and comorbidities), to the procedure type, as well as to the robotic system model and the expertise of the table-side assistant (e.g., resident, fellow, assistant, and attending). A number of 424 robotic urological, gynecological, and general surgery procedures were analyzed and used to construct the ML model. The application of this predictive model led to an increase of 16.8% in the accuracy of prediction of OT, showing that its implementation may improve the daily operative time schedule with a significant impact on the utilization of the operating room resources.

Furthermore, the automatic assessment of surgical skills and kinematic data obtained from the surgeon was shown to have a predictive role for the surgical outcomes [34, 35], as hypothesized by Goldenberg et al. [36] for RARP. Hung et al. [15] (Table 1) performed an analysis of automated performance metrics derived from the data of the “dVLogger” (recording device attached to the robotic system that captures both video and movement data, Intuitive Surgical) of 78 full-length RARP. The analysis included 25 features regarding overall/dominant and non-dominant instrument kinematic data (e.g., travel time, path length, movement, and velocity) and system events (e.g., frequency of clutch use, camera movement, third arm, and energy use) with the purpose to evaluate their accuracy for the prediction of surgical outcomes. A ML model was able to predict length of stay based on automated surgical performance metrics with an accuracy of 87.2%. Furthermore, predicted patient outcomes such as OT and Foley catheter duration were significantly correlated with the actual outcome (r = 0.73, p < 0.001 for OT; r = 0.45, p < 0.001 for Foley catheter duration). The most important predictor among surgical performance metrics was camera manipulation (e.g., idle time, adjustment frequency and position adjustment) which may act as an indicator of robotic surgical expertise. This study represents a step towards the optimization of logistics, as it may lead, for example, to a cost-effective individualized catheter removal time [37].

AI and prediction of postoperative outcomes

In a recent review, Chen et al. [38] showed that several studies support the superiority of AI systems trained with clinical, pathologic, imaging, and genomic data over D’Amico risk stratification for the prediction of treatment outcome, representing a step towards individualized patient care. But apart from patient-related factors, there are also surgeon-related factors that can impact postoperative outcome of the patients. Hung et al. [18] (Table 1) assessed the role of automated performance metrics and DL models for the prediction of urinary continence (UC) in a group of 100 patients who underwent RARP. Urinary continence was defined as the use of no pads during 2 years follow-up. The association of kinematic data with clinical patient features showed the highest accuracy in prediction of UC recovery at a median of 4 months after RARP compared to clinical features only (concordance index 59.9% vs. 57.7% for sole automated performance metrics vs. 56.2% for clinical features only). Furthermore, the automated performance metrics measured during apical dissection and vesicourethral anastomosis ranked highest in this dataset, suggesting that surgical technique is more important than patient features to achieve early UC, as previously confirmed by Goldenberg et al. [36]. In their report, Hung et al. [18] showed that the patients operated by surgeons with more efficient automated performance metrics had a 10.8% and 9.1% higher continence rate at 3 months and 6 months, respectively, compared to patients operated by surgeons with less efficient metrics.

AI and surgical guidance

Apart from the applications in surgical skills assessment and prediction of outcomes, AI systems can provide help to already proficient surgeons by improving the visualization of the intraoperative anatomy, using augmented reality [39]. When augmented reality overlay such as 3D reconstruction of the surgical target are accurately positioned over the patient anatomy, it enhances the interpretation of surgical field [40]. Although the preliminary advantages of these systems have been proven, their application in real cases is still experimental due to the lack of registration accuracy in real-life setting [41, 42]. In this context, ML automatized registration models can be used to adapt for organs movements, but more advancements need to be made in video based recognition of the surgical field. Furthermore, the automatic identification of the robotic instruments from the background of the visual field can avoid their occlusion by the image overlay and enhance surgical guidance [43]. When ML can compensate for the motility of the organs during surgery and can provide tissue and instrument recognition, a more accurate connection can be made between preoperative 3D imaging data-sets and the patient. This will also support the implementation of “GPS-like” surgical navigation strategies [44].

Future prospective: AI for development of autonomous robotic surgery

Recent development in AI systems and ML models have paved the path for the next generation of surgical robots that can learn and autonomously perform different tasks under human supervision [45]. Autonomous robots have been very successful in industry, but there is still much to be done before such systems can handle the complexity of soft tissues and unforeseen events. Potential applications of autonomous robotic surgery could be procedures that require precise dissection in restricted spaces, thus preventing iatrogenic surgical injuries. For example, procedures in otorhinolaryngology [46], ophthalmology [47], gynaecology [48], orthopaedics [49] and, ideally, in urology can be enhanced by these robots.

The first use of autonomous robots for performing surgical tasks in humans was described on cadavers [50]. Cadaver models are ideal for robot training, as they allow the use of different contrast agents or tracers to guide autonomous dissection without having the issue of toxicity. In real-time surgery, these tracers can help the identification of individual variation of organs and blood vessels, leading to the tailoring of the surgical technique. Furthermore, the ML process can be enhanced by the use of sensors to measure the tool-tissue interaction, potentially leading to tissue recognition [51].

Before entering the clinical practice, performance standards for evaluating autonomous robotic surgery have to be defined. The most important issues to be assessed are adaptation to unforeseen events, accuracy of surgical gestures and repeatability [50]. In this regard, the automation of surgical gestures can overcome the inherent differences between surgeons and can lead to more consistent and predictable results.

Still, there are some ethical and safety issues that arise regarding autonomous robotic surgery. First of all, the use of a surgical robot cannot be done without patient consent. Second, it is important to assess in what situations the robot has been trained and what it has learned during the training process to prevent inappropriate information storing and impair patient safety. Also, the possibility to adapt to unforeseen situation or complications is one of the main challenges facing autonomous surgery, as robotic systems will usually choose a random solution to a situation it has not been trained in [50]. As such, the use of more diverse training models can improve the performance of autonomous surgical robots in various situations.

Conclusion

Robotic surgery represents a good setting to refine the application of AI in surgical practice. The preliminary results of its clinical implementation are encouraging. Machine learning approaches enable the possibility to assess and clinically interpret large volumes of data, providing important feedback regarding surgical skills acquisition, efficiency of surgical procedure planning and prediction of postoperative outcomes. However, standardization of data collection and interpretation and validation on a larger number of subjects is needed for ensuring a widespread implementation.

The use of AI in robotic surgery could change the surgical training, target definition and surgical guidance, surgical logistics, outcomes and may eventually result in the standardization and consideration of autonomous surgery. To allow this, a multidisciplinary collaboration between surgeons, engineers and software developers is mandatory. Ultimately AI should help advance precision surgery and improve both functional and disease related outcomes without compromising patient safety.