The Institute for Surgical Excellence: Its Role in Standardization of Training and Credentialing in Robotic Surgery

Abstract

The Institute for Surgical Excellence is a 501(c)(3) nonprofit organization dedicated to improving surgical care and patient outcomes for emerging technologies. It is the only cross-specialty organization that has a strategic plan to create and manage a Full-Cycle Model for Education, Training, Assessment, and Surveillance for robotic-assisted surgical (RAS) procedures. The components of the model include the development and validation of expert consensus-driven outcome measures and metrics, comprehensive curricula with supporting simulations, train-the-trainer standards, credentialing standards, and long-term surveillance through a RAS registry.

Each component of the full-cycle model was addressed through a series of consensus conferences to develop new standards for RAS. The eleven consensus conferences included the following:

• 1. Fundamentals of Robotic Surgery (FRS) Outcome Measures Consensus Conference

• 2. FRS Curriculum Planning Consensus Conference

• 3. FRS Curriculum and Simulation Development Consensus Conference

• 4. Consensus Conference for the FRS Curriculum Effectiveness Evaluation Study Design

• 5. Robotic Training Network (RTN) Consensus Conference

• 6. Specialty Specific Curricula Consensus Conferences—Gynecology

• 7. Specialty Specific Curricula Consensus Conferences—Thoracic Surgery

• 8. Train-the-Trainer Consensus Conference—Curriculum Development

• 9. Train-the-Trainer Consensus Conference—Teaching to Train and Assess Regarding Errors

• 10. Credentialing Consensus Conference

• 11. Registry Consensus Conference

Each consensus conference will be described in this chapter.

1 Introduction

The Institute for Surgical Excellence (ISE) is a 501(c)(3) public nonprofit organization (www.surgicalexcellence.org) formed in 2014. Its mission is to create lasting solutions for complex healthcare problems related to emerging surgical technologies, with the ultimate goal to improve patient outcomes and safety. ISE utilizes a consensus-driven approach by bringing together key stakeholders including surgeons, educators, researchers, hospital leadership, government, and industry innovators to create new surgical standards. ISE facilitates the process of identifying issues with the use of new technology, setting clearly defined goals to address them, promoting collaboration among stakeholders, determining and filling gaps in knowledge, and promoting information sharing with healthcare consumers.

ISE is the only cross-specialty organization that has a strategic plan to create and manage a Full-Cycle Model for Education, Training, Assessment and Surveillance for robotic-assisted surgical (RAS) procedures. The Full-Cycle Model is depicted in Fig. 11.1, and each area will be described in this chapter.

Fig. 11.1

Full-Cycle Model for Education, Training, Assessment and Surveillance for robotic-assisted surgical (RAS) procedures

Each component of the full-cycle model was addressed through a series of consensus conferences to develop new standards for RAS. ISE organized, facilitated, and managed most of these 11 consensus conferences that will be described in this chapter and members of the ISE leadership team participated in all of consensus conferences.

The Delphi process was conducted for each of the consensus conferences to drive consensus of the subject matter experts. The Delphi method is a structured process to effectively drive consensus of a group of individual experts when addressing complex issues, especially where there are not evidence-based standards.

A modified Delphi method was used to generate key questions for the Delphi survey based on facilitated discussions during live meetings. The final Delphi survey was then distributed anonymously in a classic Delphi format via the Internet (Google forms) to participants in three successive rounds. After each Delphi round, participants received feedback in the form of a statistical analysis of the group response. Questions in which there was ≥80% concurrence were removed from the next round of the survey. Repeated iterations of anonymous voting continued over three rounds, where an individual’s vote in the next round was informed by knowledge of the entire group’s results in the previous round. After the three rounds, the consensus views that represented at least 80% of the expert panel were distributed to all stakeholders and in many cases published in peer-reviewed journals.

2 Fundamentals of Robotic Surgery (FRS) Consensus Conferences

With leadership from Drs. Richard Satava, Roger Smith, and Vipul Patel, the backbone of RAS standardization is the four consensus conferences of the FRS initiative. FRS was funded through a Department of Defense grant and an unrestricted educational grant from Intuitive Surgical. FRS is a multispecialty, proficiency-based curriculum of basic technical skills to train and assess surgeons to safely and efficiently perform robotic-assisted surgery. It was developed by over 80 national and international robotic surgery experts, behavioral psychologists, medical educators, statisticians, and psychometricians. The clinical robotic surgery subject-matter experts represented all of the major surgical specialties in the United States that currently perform robotic-assisted surgical procedures, the Department of Defense, and the Veterans Administration (VA) [1].

2.1 FRS Outcome Measures Consensus Conference

In the first FRS consensus conference, called “Outcomes Measures ,” subject matter experts from multiple surgical societies, educational societies, surgical boards, and other governing organizations agreed upon the critical skills and tasks that needed to be included in a comprehensive basic curriculum. All stakeholders, including the accrediting bodies, were involved from the very beginning of the process to ensure the final curriculum and assessment methods would meet the rigorous requirements of determining proficiency, meeting standards, and possibly even fulfilling certification criteria.

A task deconstruction was performed to identify the specific skills and potential errors that need to be measured. A modified Delphi methodology was used to create a matrix of specific robotic surgical skills that are matched to their common errors, a description of the desired outcome and the quantitative metrics that support those outcomes. Following the conference, a classic Delphi survey was conducted through anonymous rating to ensure concurrence, to prioritize the ranking of the tasks, and to eliminate low-scoring tasks. As a result of this process, 25 key outcome measures were identified and prioritized to include in the FRS curriculum listed below [1]:

• Situation awareness

• Eye-hand instrument coordination

• Needle driving

• Atraumatic handling

• Safety of operative field

• Camera

• Clutching

• Dissection — fine & blunt

• Closed loop communication

• Docking

• Knot tying

• Instrument exchange

• Suture handling

• Energy sources

• Cutting

• Foreign body management

• Ergonomic position

• Wrist articulation

• Robotic trocars

• System setting

• Multi-arm control

• Operating room setup

• Respond to robot system error

• Undocking

• Transition to bedside assistant

These identified outcome measures were the basis of future curriculum development.

2.2 FRS Curriculum Planning Consensus Conference

The second FRS consensus conference, called “Curriculum Planning, ” began the process of curriculum development and determined the methods for training and assessing the full range of technical skills (cognitive, psychomotor, team training, and communication) that are necessary to safely use a robotic surgery system. Several goals were formulated including the following [1]:

• Reviewing the consensus-driven outcome measures developed in the first consensus conference

• Breaking the 25 outcome measures into seven basic tasks that must be mastered by robotic surgeons. Each task should include a description, skills being assessed, objective measures, and potential errors that can occur

• Reviewing and adapting the curriculum template from the Alliance of Surgical Simulation for Education and Training (ASSET), which developed and published a curriculum template with wide consensus from surgical societies [2]

• Creating a curriculum outline and beginning the actual curriculum development process

2.3 FRS Curriculum and Simulation Development Consensus Conference

In the third FRS consensus conference, development of the comprehensive multispecialty curriculum and the simulation training were initiated. The four modules that were created are described below [1] (Fig. 11.2):

• Introduction to Surgical Robotic Systems—A technological revolution occurred with the introduction of laparoscopy and other minimally invasive surgeries. This first module of the curriculum provides a primer for surgeons who choose to pursue robotic surgery. It includes robotic components and instrumentation, as well as the advantages and disadvantages of robotic surgery.

• Didactic Instructions for Surgical Robotic Systems —This module contains the cognitive skills required to conduct safe and successful basic RAS procedures in the preoperative, intraoperative, and postoperative phases.

• Psychomotor Skills Curriculum —The goals and objectives of the basic skills and tasks for robotic surgery are to train and assess the proficiency of the psychomotor robotic skills of the surgeon. This will ensure that only the surgeons who are skilled and well trained in robotic surgery perform such complex procedures, making the patient the ultimate benefactor.

• Team Training and Communication Skills —The use of a robotic system creates unique demands, which are beyond those demands in open and laparoscopic surgery. The principle difference is the physical separation of the primary surgeon from the patient, the operative team members, and operative site. The result is that the surgeon must rely even more upon team participation and clear, unambiguous communication with team members.

Fig. 11.2

(a) Module 1: Introduction to robotic surgical systems. (b) Module 2: Didactic instructions. (c) Module 3: Psychomotor skills curriculum. (d) Module 4: Team training and communication skills

After each module, there is a summative assessment and report of the learner’s performance.

From the beginning, FRS has been developed as a cross-specialty and device-agnostic curriculum. Therefore, significant attention was paid to basic robotic principles that will be applicable for future robotic surgery systems and simulators.

The psychomotor skills curriculum was developed to train and assess surgeons interested in performing basic robotic surgery. A proficiency-based progression (PBP) model was utilized that trains a surgeon to perform a specific task to a defined “expert benchmark” (Fig. 11.3).

Fig. 11.3

Setting the expert benchmark

In PBP, the benchmark must be consistently met before allowing the surgeon to progress to the next task usually at a higher level of difficulty. It has been demonstrated that training to quantitative, evidence-based, proficiency standards result in expedited skills acquisition, less errors, improved efficiency, and greater patient safety [3,4,5,6].

The 25 key outcome measures that were identified and prioritized in the first consensus conference were incorporated into seven tasks that together train and test the basic skills needed by all robotic surgeons. In the online curriculum, each one of these tasks included a description of task to be performed, skills being assessed, metrics used for each task (both an objective numeric psychomotor metric test and a subjective Global Evaluative Assessment of Robotic Skills rating scale), and potential errors that can occur. These assessment tools were the basis of the FRS validation trial that will be described later in this chapter.

The seven tasks that were developed include (and are depicted in the image below) [1]:

• Task 1: Docking/Instrument Insertion

• Task 2: Ring Tower Transfer

• Task 3: Knot Tying

• Task 4: Railroad Track

• Task 5: Fourth Arm Cutting

• Task 6: Puzzle Piece Dissection

• Task 7: Vessel Dissection/Division.

These tasks were delivered in identical physical (FRS Dome) and VR simulation models that were developed and tested to provide the most effective and efficient psychomotor training. Several iterations of the physical model were developed and tested until a final model was agreed upon by subject-matter experts. Experts are seen in Fig. 11.4 discussing the first physical training model prototype.

Fig. 11.4

Experts discussing the first physical training model prototype

The FRS physical dome prototyping stages are depicted in Fig. 11.5.

Fig. 11.5

(a) Prototype concept, (b) prototype 1, (c) prototype 2, (d) prototype 3, (e) prototype 4, (f) prototype 5, (g) CAD design, (h) 1st 3-D printed model

The final FRS physical dome is shown in Fig. 11.6 with the seven psychomotor tasks that were developed.

Fig. 11.6

(a) Final physical model, (b) abdominal shell, (c) instrument insertion, (d) ring tower transfer, (e) knot tying, (f) railroad track, (g) 4th arm cutting, (h) puzzle piece dissection, (i) vessel dissection/division

2.4 Consensus Conference for the FRS Curriculum Effectiveness Evaluation Study Design

In the fourth FRS consensus conference, the design of the FRS effectiveness study was discussed by clinicians, psychologists, researchers, and psychometricians. This study was designed to meet the most rigorous evaluation that would satisfy criteria for high stakes testing and evaluation [1].

Topics discussed at the consensus conference included defining the research questions; developing hypotheses within construct validity; defining what constitutes an “expert” for benchmarking purposes; defining novice criteria, determining criteria for international institution participation.

In addition, there were extensive discussions regarding what types of validity and reliability would be measured along with usability and acceptability.

Study design phases were identified and included the following:

• Phase 1: Pilot testing to determine logistics and refinements to study model

• Phase 2: Obtaining feedback from the society leadership and boards

• Phase 3: Obtaining validity evidence at test sites

• Phase 4a: Demonstrating concurrent validity with video correlations

• Phase 4b: Demonstrating predictive validity through the full research study at twelve research sites.

The study design is depicted in Fig. 11.7.

Fig. 11.7

FRS Curriculum Effectiveness Evaluation Study Design

Following this consensus conference, a multi-institutional, multispecialty, single-blinded, parallel group, randomized control trial was conducted and managed by the ISE. The abbreviation was established previously in the paper. Participating institutions were selected based on a competitive process and had to be an American College of Surgeons’ Accredited Education Institute (ACS-AEI), have a minimum of three separate surgical specialties that were performing robotic surgery, availability of participants with variable experience in robotic surgery, and easy access to a robotic surgical system both in a simulated and clinical environment for training and testing. The institutions that participated in the study included the following [1]:

• Madigan Army Medical Center, Tacoma, WA

• Carolinas Medical Center, Charlotte, NC

• Hartford Hospital, Hartford, CT

• University of Pisa, Pisa, Italy

• Duke University Medical Center, Durham, NC

• Imperial College London, London, UK

• Lahey Hospital & Medical Center, Burlington, MA

• Lehigh Valley Health Network, Allentown, PA

• Methodist Institute for Technology, Innovation, & Education, Houston, TX

• University of Athens Medical School, Athens, Greece

• University of Pennsylvania, Philadelphia, PA

• University of South Florida Center for Advanced Medical Learning and Simulation, Tampa, FL

A PBP training model was utilized in the study. Participating experts setting expert benchmarks had to have a minimum of 50 robotic cases performed as a primary surgeon and be actively performing at least two robotic cases per month. Randomized subjects in the trial were novices defined as surgical residents, fellows, and faculty who had participated in less than five robotic cases. The ISE study coordination center assigned participants by simple randomization to the four study groups at the beginning of the study: the FRS physical Dome, the da Vinci Simulation System (DVSS), and the dV-Trainer training groups, and a control group.

Each novice participant had to successfully complete the online cognitive component of the curriculum before proceeding to the psychomotor tasks. A pretest was then performed on an avian tissue model that was identical to the posttest to determine the baseline psychomotor skills of each participant. The novice was then randomized to a control group or to one of the three experimental groups where training was conducted on the FRS physical dome, the DVSS, or the dV-Trainer, then required to reach the expert benchmark set for each task before going onto the next task. After reaching proficiency in each task, the novice underwent a final test on an avian tissue model that was video recorded. After study completion, each participant video was reviewed by two blinded raters who recorded task duration in seconds and task errors using a 32-criteria task-specific checklist (numeric psychomotor metric test). Raters also completed the Global Evaluative Assessment of Robotic Skills (GEARS) rating scale [7].

In Fig. 11.8, the FRS VR Dome (Fig. 11.8a), 3D-printed FRS physical Dome (Fig. 11.8b), and avian tissue model (Fig. 11.8c and d) are depicted.

Fig. 11.8

(a) FRS VR dome, (b) 3D-printed FRS physical dome, (c) and (d) avian tissue model

In this international multi-institutional, noninferiority blinded, randomized control trial, evidence was provided for the effectiveness of the FRS cognitive curriculum and the psychomotor skills training utilizing proficiency-based progression methodology in a physical model and virtual reality simulation platforms for robotic skill acquisition for basic robotic surgery. In addition, validity evidence for the use of the avian tissue model in performance assessment, which was able to discriminate between more experienced and less experienced surgeons, was obtained. In this study, the attending surgeons out-performed residents and fellows at baseline and on posttest. Lastly, by demonstrating better performance of those trained using FRS compared with controls, the authors argued for the wide adoption and implementation of FRS across training programs [1].

ISE manages the FRS website and online curricula, which can be found at https://surgicalexcellence.org/programs/fundamentals-of-robotic-surgery/ [8].

2.5 RTN Consensus Conference

Another example of RAS curriculum development that followed a similar path to the FRS curricular design and testing includes the Robotic Training Network (RTN) curriculum. The vision of RTN is to standardize the robotic surgical curriculum and education for residents and fellows-in-training for all surgical specialties utilizing robotic-assisted surgery. It is a collaborative network of nine institutions with a common vision to design a standardized approach to teach basic robotic surgical skills in a stepwise fashion to trainees throughout Graduate Medical Education training programs. The founding group included the following:

• Beth Israel Deaconess Medical Center

• Celebration Hospital Florida

• Cleveland Clinic

• Duke University

• Johns Hopkins University

• Lehigh Valley Health Network

• Newark Beth Israel Medical Center

• University of North Carolina

• Wright State University

Over the last several years, over 60 ACGME approved residency/fellowship training programs have been collaborating as part of the Robotic Training Network with a mission to develop an educational curriculum to teach the basic principles of robotic surgery.

The RTN Education and Training Course contains nine online modules with interactive content and questions.

• Module 1: Overview

• Module 2: Pretest

• Module 3: Background Knowledge

• Module 4: Posttest

• Module 5: Introduction to the Robotic System

• Module 6: Bedside Assistant

• Module 7: Console Surgeon

• Module 8: Team Training and Communication

• Module 9: Specialty-specific Education and Training

Once all nine modules are completed, the trainee receives a certificate and can proceed to the psychomotor skills training and assessment. There are five skill drills in the RTN curriculum.

• Tower transfer—Transfer of rubber band from the inner small towers to the outer graduated height towers.

• Roller coaster—Manipulate the rubber band around wire loop.

• Big dipper—Place the needle into sponge in various arcs through prespecified dot patterns.

• Train tracks—Place a running suture with the needle entering and exiting through the dots.

• Figure of eight—Place a figure of eight stitches with the needle entering and exiting through dots followed by a square knot.

The skills tasks are depicted in the images of the physical model in Figs. 11.9 and 11.10.

Fig. 11.9

Outside and inside views of the RTN physical training model components

Fig. 11.10

RTN Virtual Reality Trainer (developed by Mimic Technologies, Inc)

Similar to FRS, identical skills can be performed on VR simulators, as well.

In addition, RTN designed and provided validity evidence for an assessment tool to complement their educational robotic surgical training curriculum called the Robotic—Objective Skills Assessment Test (R-OSATS) [9] (Fig. 11.11).

Fig. 11.11

Robotic—Objective Skills Assessment Test (R-OSATS)

ISE manages the RTN website and online curricula, which can be found at https://surgicalexcellence.org/programs/robotic-training-network/ [8].

2.6 Specialty-Specific Curricula Consensus Conferences—Gynecology

The FRS curriculum and task trainers are designed to be basic education and training for robotic surgeons across all specialties performing robotic surgery. More advanced specialty specific robotic education and training are also needed. In response to this need, ISE conducted two specialty-specific consensus conferences described below and is currently developing others.

The first specialty specific consensus conference was for gynecologic robotic surgery. It included representatives from all major gynecologic society stakeholders and others including:

• American College of Obstetricians and Gynecologists (ACOG)

• American Association of Gynecologic Laparoscopists (AAGL)

• Council on Resident Education in Obstetrics and Gynecology (CREOG)

• American Board of Obstetrics & Gynecology (ABOG)

• Society of Gynecologic Oncology (SGO)

• American Urogynecologic Society (AUGS)

• American Society for Reproductive Medicine (ASRM)

• Society of Gynecologic Surgeons (SGS)

• American Medical Association (AMA)

• Joint Commission on Accreditation of Healthcare Organizations (JCAHO)

Based on the groundwork set by the original four FRS consensus conferences, all elements of outcome measures and metrics, curriculum design and early development, simulation design, and team training and communication were accomplished during a three-day specialty-specific consensus conference. Gynecologic RAS-specific issues were addressed in the new Fundamentals of Robotic Gynecologic Surgery (FRGS) curriculum, including instrumentation used, unique patient positioning requirements, trocar placement, and robotic and team positioning, to mention a few. Examples of screenshots from the curriculum are provided in Fig. 11.12.

Fig. 11.12

Examples of screenshots from the Fundamentals of Robotic Gynecologic Surgery (FRGS) curriculum

A task deconstruction was completed for a hysterectomy, which is the “signature” procedure for gynecology. The 25 basic skills identified in the FRS consensus conferences were reviewed as were the seven basic FRS tasks. These served as a prerequisite to the more advanced gynecologic tasks. Four new advanced tasks were identified that are related to the successful completion of a hysterectomy. They included the following:

• Dissection of the bladder flap

• Ureter dissection/exposure

• Anterior and posterior colpotomy incisions

• Vaginal cuff closure

Some of the participating gynecologic robotic experts also worked with simulation companies to develop and refine the new gynecologic specific tasks. Examples of the VR simulations (developed by 3D Systems) are shown in Fig. 11.13.

Fig. 11.13

Examples of the VR simulations (developed by 3D Systems). (a) Bladder flap, (b) colpotomy, (c) vaginal cuff, (d) ureter dissection

In addition, new team training scenarios were developed that correlate to additional communication skills that are unique to gynecologic surgical procedures.

2.7 Specialty-Specific Curricula Consensus Conferences—Thoracic Surgery

The second specialty -specific consensus conference was for thoracic RAS. It included 18 thoracic robotic surgeons from the US and one from Europe representing a European thoracic robotic curriculum development consortium. The Society of Thoracic Surgeons (STS) and the American Association for Thoracic Surgery (AATS) were also represented at the consensus conference.

Similar to the gynecology consensus conference, based on the groundwork set by the FRS, all elements for the Fundamentals of Thoracic Robotic Surgery (FTRS) curriculum were determined during a 3-day specialty-specific consensus conference. A task deconstruction was completed for a thoracic “signature” procedure, lobectomy, which included the following:

• Takedown of inferior pulmonary ligament, division of pleura

• Dissect subcarinal posterior/paratracheal/ hilar nodes

• Dissect/divide superior vein

• Dissect superior hilar/peribronchial nodes

• Dissect/divide pulmonary arteries

• Dissect/divide bronchus

• Complete posterior/horizontal fissure

• Management of PA Injury

For each of these steps, training items and potential errors were identified. New medical illustrations were developed by ISE to support the curriculum, as well as more than two dozen video examples. These were all included in the curriculum (Fig. 11.14).

Fig. 11.14

Medical illustrations developed for the FTRS curriculum

Thoracic robotic surgery experts have worked with a simulation company to make the VR lobectomy procedure more realistic and anatomically accurate. An example of the VR lobectomy simulation (developed by 3D Systems) is shown in Fig. 11.15.

Fig. 11.15

VR lobectomy simulation (developed by 3D Systems)

2.8 Train-the-Trainer Consensus Conference —Curriculum Development

ISE hosted two international Train-the-Trainer (TTT) Consensus Conferences and brought together experts and stakeholders from the United States and Europe to determine the key elements of a core TTT RAS curriculum through an expert consensus process.

In the first TTT conference, the Delphi methodology was used to develop consensus-driven guidelines for selecting and verifying trainers in robotic surgery. A TTT curriculum for RAS training was initiated with the goal to improve cognitive education, psychomotor training, training around errors, team communication, expert feedback, assessment tools, scoring systems, and remediation. After three rounds of Delphi surveys, consensus was obtained in more than 60 elements in six different categories that are described below. This laid the foundation for the development of proficiency-based progression models for robotic trainers.

2.8.1 Category 1: Consensus on Terminology

Uniform communication language is important for understanding roles in surgical training, because if there is ambiguity in the “surgical training” terminology, it may have negative implications in various clinical settings. Thirty terms were defined and agreed upon [10], including Master Trainer, Delegate, Trainer, and Trainee (Fig. 11.16).

Fig. 11.16

Train-the-trainer terminology

2.8.2 Category 2: Prerequisites for TTT Course Selection and TTT Qualifications

The panel reached consensus view that TTT delegates should be experts in their field and that there should be defined selection criteria for getting a place on a TTT course. The panel agreed that while delegates should be experts in their field, all good surgeons are not necessarily good trainers. The panel also concluded that important individual qualities for a surgical trainer include being knowledgeable, interested, enthusiastic, supportive, and a good communicator. The surgical trainer should also enjoy training, have time to train, and have the restraint and wisdom to know when it is appropriate to “take over” in simulation vs. clinical setting [10]. The best trainers are those who know when to give progressive responsibility and autonomy to trainees [11].

2.8.3 Category 3: Objectives and Focus of a TTT Course

The panel agreed there should be clearly defined objectives for the TTT course and that it should focus on both educating the delegate to become a verified trainer and how to set up a “training program.” Identified by the panel as a key focus point and essential for the TTT course included instruction on how to optimize guidance on defined technical skills and training in providing feedback and debriefing, following technical skill assessment. The panel also recognized the need for training on technical skills rating and calibration exercises and the importance of psychometric robustness of these technical skills assessment tools. In addition, the TTT course should provide opportunities to practice rating training skills in the operating room and laboratory/simulation setting [10].

2.8.4 Category 4: Pre-course Considerations

The group considered what needs to be included in a “checklist” of basic requirements for setting up a TTT course. The group’s guidance for pre-course e-learning modules was that it should include the following [10]:

• Details of the TTT course content and clearly defined objectives of the TTT course.

• List of skills to be taught.

• Definitions for terminology.

• Defined role-play tasks and aims of role-play.

• Educational theory information related to the course.

• Relevant subject-matter details related to future training courses (e.g., FRS TTT course should describe the FRS).

• Procedural-based TTT courses should describe important standardized content to be given by the trainers (e.g., the important anatomy, port placement, and surgical steps).

• Pre-course evaluation should include an assessment of the delegate’s knowledge of the course subject matter and/or technical procedure aspects to be given in the training course.

There was consensus agreement within the panel that completion of e-learning related to the TTT course should be a basic requirement before attending the TTT course. Furthermore, it is important to identify the participant’s gaps in knowledge of the proposed training program they will run and address them before the TTT course commences.

2.8.5 Category 5: Theory and Course Content

The panel reached agreement on multiple areas of educational theory and course content for a standardized TTT course. Areas of agreement that related to subject matter include the following [10]:

• Highlight the importance of team contribution to training and describing the behaviors of “good” team members

• How to deal with the difficult trainee

• Guidance on how to avoid “taking over” as the trainer. Explanation of the “Six Steps” of safe mentoring[12].

• Cognitive task analysis

• Describe and explain “deliberate practice”

• Description of proficiency-based progression

• How to reflect and the importance of reflection

• How to debrief

• How to give informative feedback

• Task deconstruction

• Describe and explain “Performance enhancing feedback”

• Take-home messages need to be identified

Areas of agreement related to exercises, feedback, and assessment tools include the following:

• Practical demonstrations/role-play/tasks/group participation

• Task repetition to demonstrate deliberate practice

• Example of task deconstruction

• Explanation of the Six steps: 1. Stop; 2. Identify; 3. Explain; 4. Structured Teaching; 5. Elicit Check of Understanding; 6. Proceed if Safe.[12]

• Proficiency-based progression exercises

• Role-play exercise that describes and explains the effect of cognitive load [12].

• Informative feedback exercise [12].

• Practical training role-play: role-play scenarios played out with delegate interaction and assessment of trainer’s performance with open discussion and feedback

2.8.6 Category 6: Measuring Outcomes

The panel identified that RAS assessment includes technical skills, cognitive assessment, and nontechnical skills. The robustness of technical skills assessment tools is important for the continuum of training [10]. Technical skills are currently commonly assessed with Likert scale measures such as GEARS. Whereas proficiency-based progression is based on objective metrics that often relate to the completion of tasks and the avoidance of errors. For proficiency-based progression, delegates should have the opportunity to repeat the scenario until they achieve proficiency.

At the end of the course a post-course evaluation test of the delegates should be compared with the pre-course test, and informative feedback should be given to the delegates on their performance. Delegates should also have the opportunity to comment on the various aspects of the course with a written questionnaire to evaluate both the course and the master trainer.

Finally, the panel identified that there is poor standardization around the definition of errors, causes and consequences of errors, classification of errors, outcome measures and metrics needed to evaluate errors, and training to prevent errors. In response to this need, a second train-the-trainer consensus conference was convened.

2.9 Train-the-Trainer Consensus Conference—Teaching to Train and Assess Regarding Errors

The second train-the-trainer consensus conference focused on defining errors and to standardize how to train trainers to properly educate the avoidance, recognition, and treatment of errors. There is very little standardization worldwide regarding errors, which are of course a key element in determining patient safety. ISE implemented a consensus-driven approach by inviting three dozen master trainers from around the world to ensure a scholarly and practical method to develop the highest quality of training and assessment.

The first step accomplished in the consensus conference was to agree on a taxonomy for errors (Fig. 11.17).

Fig. 11.17

Consensus-driven standardization of taxonomy for errors

The second task was to develop a standardized algorithm for errors (Fig. 11.18).

Fig. 11.18

Consensus-driven standardization of algorithm for errors

The third task was to have expert robotic surgeons do task deconstructions of the signature robotic procedures from each specialty. Any associated potential errors were included for each step of the procedure. The potential errors were then prioritized regarding which ones would be most important to teach with consideration of the teaching methods.

The ability to effectively handle adverse events is essential to safe surgery. The relative rarity of these events in clinical practice makes simulation an essential component in training the surgeon and surgical team how to handle adverse events and in confirming proficiency [13]. At the present time, there are very few models that teach about errors in a standardized way.

It was determined that simulations should start with generic adverse events applicable to all RAS such as controlling bleeding from blood vessel injury. More advanced training would include procedure-specific adverse events such as ureteral injury during a hysterectomy or sigmoid resection. Emphasis should be placed on properly identifying the problem and the most appropriate next response.

Expert presentations were provided at the consensus conference describing some of the most effective real tissue simulators for teaching full procedures and associated errors like the KindHeart, Inc., Chapel Hill, NC (Fig. 11.19).

Fig. 11.19

KindHeart training model (developed by KindHeart, Inc.)

Full procedure simulation can also be accomplished with sophisticated synthetic 3-D printed tissue models that are very realistic, like those developed by the University of Rochester, Rochester, New York. The models were fabricated at their Simulation Innovation Laboratory where they developed a number of organ models for use as minimally invasive surgical simulation platforms (e.g., RAS) using a patented process of 3-D printing and hydrogel [Polyvinyl Alcohol (PVA)]. Examples of models are shown in Fig. 11.20.

Fig. 11.20

(a) Perfused hydrogel kidney in its corresponding 3-D printed mold, (b) RAPN simulation platform after completion of the simulation and removal of the tumor (inferior vena cava, aorta, colon, and spleen seen in the background), (c) prostatectomy hydrogel model with bladder, urethra, and dorsal venous complex with incorporated stretch sensors, (d) RARP simulation platform after completion of the simulation and removal of the prostate (bilateral neurovascular bundles left intact after full nerve sparing)

ISE and its advisory board are presently working with VR simulation companies to develop new simulators that incorporate the avoidance, recognition, and management of adverse events to help standardize them across all simulation platforms.

During the second TTT consensus conference, there was universal agreement that comprehensive training in handling adverse events cannot and should not occur in the clinical setting. The real tissue and synthetic full procedure simulations have the clear advantages of allowing the complete orchestration and control of the event (the degree of bleeding, for example), the application of a predefined curriculum, and an unlimited number of repetitions allowing for deliberate practice while putting no patients at harm’s way.

As a result of consensus achieved at the second TTT consensus conference, an Adverse Event Curriculum including simulations is presently under development for intraoperative bleeding. Unexpected intraoperative bleeding has been broken down into minimal, moderate, and massive bleeding. Videos of each level of bleeding are used to train surgeons to correctly identify the level of bleeding occurring (Fig. 11.21).

Fig. 11.21

(a) Mild bleeding from pulmonary vein. (b) Moderate bleeding. (c) Severe bleeding

For each level of emergency, an action plan has been developed and the component technical tasks identified. For minimal bleeding, the first line of management is pressure control with possible cautery/energy application, or topical application of clotting materials. In moderate bleeding, pressure control is utilized to prepare for vessel ligation with suture, clips, or energy application, or possible vessel repair. With massive bleeding, it is imperative to stabilize the situation with pressure control until an emergent conversion to an open procedure can be accomplished. Appropriate communication should accompany the simulation exercise. With massive bleeding, the surgeon is expected to request appropriate blood products, notify the anesthesiologist and surgical team of the emergency situation, and call for surgical backup/assistance. Once proficiency in these component tasks and communication skills has been achieved, the trainee performs a full specialty-specific procedure with adverse event management.

Full validation of this Adverse Event Bleeding Curriculum will be needed before any expansion of the curriculum into other adverse events such as intraoperative organ injury (spleen, bowel, liver, ureter, and bladder), difficulty with abdominal distention, stapler misfiring, robotic malfunction, and others.

2.10 Credentialing Consensus Conference

As with any complex new surgical procedure, technique, or technology, there is generally a steep learning curve. There is no exception to surgeons developing skills to perform RAS. The rapid growth of RAS has presented significant challenges since there are no accepted standards for credentialing and privileging to demonstrate the surgeon, and the entire team are proficient in performing safe robotic-assisted procedures. Unlike the aviation industry where pilots go through standardized intense training and simulation and are tested every 6 months to determine maintenance of skills, once surgeons become board certified, there are minimal or no requirements to maintain surgical skills or prove proficiency.

In response to these serious gaps, ISE organized and hosted a Robotic Surgery Credentialing Consensus Conference that brought together 36 representatives from institutions with extensive robotic surgery credentialing experience, surgical societies, medical associations, government, and industry. The goal of the meeting was to help develop standards regarding how hospital systems evaluate the qualifications of applicants who wish to perform robotic-assisted procedures or renew privileges in their facilities to ensure the highest quality and safety of surgical care.

The first part of the consensus conference was information gathering from seven top robotic institutions that described their present credentialing process (i.e., how they assess the qualifications of physicians or other healthcare professionals) and privileging criteria (i.e., determination of specific surgical conditions and procedures that a surgeon will be allowed to perform at a healthcare institution). In addition, seven major surgical societies presented their robotic credentialing recommendations. The participating institutions and societies included the following:

Institutions

• AdventHealth

• Emory

• Northwell Health

• Lehigh Valley Health Network

• Memorial Sloan Kettering

• Kaiser Permanente

• SUNY Downstate

Societies

• American Urology Association (AUA)

• American College of Obstetricians and Gynecologists (ACOG)

• Society of Robotic Surgeons (SRS)

• Society of Thoracic Surgeons (STS)

• Society of American Gastrointestinal and Endoscopic Surgeons (SAGES)

• American Association of Gynecologic Laparoscopists (AAGL)

• European Urology Society (EUS)

In addition, Dr. Dimitrios Stefanidis led the efforts prior to the conference to review robotic credentialing policies from 39 institutions, including both academic institutions (56%) and community programs (44%). He summarized the policies and presented them at the consensus conference. He found that there was tremendous variability between the details of the various policies, but many included general criteria in the following areas:

• Prerequisites

• Competency assessment

• Proctoring of initial cases

• Delineation of basic vs. advanced procedures

• Surgery outcome assessment

• Team training criteria

• Maintenance certification

With this background information, three working groups were formed to discuss various aspects of the credentialing/privileging process including the following:

• Prerequisite Education and Training Qualifications

• Assessing the Surgeon’s Performance—Quantitative Metrics

• Ongoing Monitoring and Surveillance

Following several hours of discussion each group presented their conclusions. These conclusions were converted into a questions format and submitted for the Delphi process. Three rounds of the Delphi process achieved greater than 80% consensus on 76/91 (83.5%) questions in the survey. A few of the important consensus items include a need for [14]:

• A common credentialing pathway for basic robotic surgery skills created across all specialties that use robotic surgery

• A separate but common pathway for credentialing of advanced robotic surgery procedures

• Documenting cognitive, technical, and non-technical training in specialty specific robotic procedures for which there is intereste in obtaining privileges

• A proficiency-based training paradigm with objective metrics

• Proficiency obtained as a first assistant before serving as a primary surgeon

• Monitoring the surgeon’s initial cases through random audit of operative videos by independent experts and supplemented by chart review as needed.

• Specific parameters to be monitored for maintenance of privileges.

2.11 Registry Consensus Conference

The new era of information science has resulted in immediate availability, analysis and sharing of real-world data (RWD) that is available at the time of the occurrence—at the pace of innovation and change. However, the potential benefit of emerging technologies and innovations are slowed by the continued use of prospective clinical trials, peer-review evaluations, and the submission of research publications, which require rigorous and careful evaluation and prolonged completion time.

One solution that has emerged is the development of “registries”—databases which are created in near-real time and which reflect data that are available at the time of occurrence, as opposed to the traditional practice of stored data that are awaiting review and possible publication. Implementing this solution, healthcare professional communities of individual physicians, hospitals, medical governing bodies and societies, industry, and federal agencies can work together using information before it has become obsolete, allowing for real-time analysis and decisions that reflect the current status in the process of dynamic change.

Due to the rapid innovation and transformation of RAS devices, it was determined that an RWD robotic surgery registry was needed to serve a diverse group of stakeholders including:

• Physicians to evaluate their operative performance for self-improvement

• Educators to develop standardized training programs and certification processes for ongoing education, remediation, and privileging

• Hospitals to develop quality measures, effectiveness, and risk assessment for quality improvement

• Industry to assess the performance of their devices to promote more rapid iterations toward improved functionality and safety

• Government to maintain minimal safety and effectiveness standards and stay informed of new developments that could influence policies

• Patients to participate in quality initiatives to continuously improve surgical outcomes

In an effort to design, develop, and successfully implement the RWD RAS registry, ISE organized and hosted a Robotic Registry Consensus Conference that brought together 44 robotic surgery experts, registry experts, the FDA, MDEpiNet, society representatives, and industry representatives from eight present and future robotic manufacturers.

Through a diligent Delphi process, the participants of the Robotic Registry Consensus Conference developed a consensus-driven core minimal data set that included the following:

• Patient demographics and patient history

• Procedure information

• Robotic device and instruments information

• Information about intraoperative issues/events

• Postoperative information and claims data

• Surgeon and OR staff experience/training

Each element of the data set was then analyzed to determine how it would be most efficiently and effectively collected (e.g., electronic health record, OR staff, and robotic information system). A pilot registry is being planned to collect in near real-time device-related and process-related data from the core minimal data set and test the system and assumptions. It will be interoperable with clinical databases and utilizes those data to improve device safety, surgeon/team performance, and public health.

A technology company, Medstreaming/M2S, was selected to develop and manage the registry. It has a long history in developing and launching large registries in partnership with societies in several procedural/surgical areas. At the time of writing this chapter, ISE is in final stages of recruiting institutions that will participate in the pilot registry.

In an effort to be inclusive of all stakeholders, ISE has provided updates to the FDA through the Q-Sub process, presented at several national society meetings, and hosted a meeting between the FDA and industry to discuss anticipated uses and benefits of the robotic registry data.

Once the pilot is completed and the results analyzed, lessons learned from the pilot will be implemented into a national robotic-assisted surgery registry that will include all surgical specialties that are performing robotic procedures. ISE is partnering with the Society of Robotic Surgery (SRS) and other surgical societies for the successful implementation and dissemination of the robotic registry. ISE will also work closely with MDEpiNet to ensure there is interoperability with other existing registries in the United States and internationally.

3 Closing Remarks

ISE’s Full-Cycle Model for Education, Training, Assessment, and Surveillance for RAS procedures is providing a roadmap for areas where consensus is needed and standardization can be achieved. Through ISE’s development and participation in these 11 consensus conferences, significant progress toward standardization has been made. As Martin Luther King, Jr., so eloquently said, “If you can’t fly then run, if you can’t run then walk, if you can’t walk then crawl, but whatever you do you have to keep moving forward.” ISE plans to continue to move forward with these consensus-driven efforts. There is still so much to do. To maximize results, this work must be done in partnership with hundreds of robotic surgical experts from around the world, researchers, surgical societies, government, industry, and with input from patients. Together, we will fly! (Fig. 11.22).

Fig. 11.22

Consensus conferences hosted and facilitated by ISE