Simulation in Vascular Surgery

Abstract

Increasingly stringent restrictions of duty hours for residents have led to decreasing experience with complex open vascular interventions; simulation is a promising avenue for both skills training and skill assessment. Various different forms of simulation are now available including benchtop models, animal models, cadavers, and high-fidelity simulators. This chapter reviews the available modalities, the evidence supporting their use in both open and endovascular surgical training, components of a successful simulation training program, and the application of simulation training in the context of two academic centers – LSU and OHSU.

Introduction

The current trend in surgical training is a move away from the traditional Halstedian apprentice model of graded responsibility to a more structured curriculum-based approach requiring documentation of proficiency [1]. Traditional resident educational paradigms have shifted as a result of changes in healthcare over the past decade. Mandated restrictions on resident work hours, shorter hospital inpatient length of stay, and the development of outpatient surgery have led to a striking reduction in training opportunities for surgical residents. In the setting of quality-assurance targets, increased public scrutiny and concern for healthcare quality and safety, and ethical concerns of “practicing” on patients, it is no longer acceptable, or appropriate, for residents at any level of training to practice new skills on patients, even if they have a patient’s explicit consent [2,3,4,5,6].

Concurrent with these trends and their impact on general resident training, dramatic technologic advances have transformed the field of vascular surgery. Advances in technology, devices, and techniques have pushed the specialty from a subfield of general surgery into an entirely new area of expertise with its own independent board certification and training programs. Vascular surgery faces the additional challenges of a rapidly changing field in which technologies have drastically impacted the practice of vascular surgery. The scope of pathology once relegated entirely to open surgical management is shifting increasingly and exponentially toward endovascular interventions as endovascular therapies are increasingly utilized to treat patients with peripheral vascular disease, abdominal aortic aneurysms, and carotid artery disease [7,8,9,10,11,12]. The result is a field in which open operations are less often encountered by trainees, and those circumstances requiring an open procedure involve highly complex and challenging cases unsuitable as training material even for senior residents. This shift away from open vascular operations has resulted in both fewer open operations for training in traditional open vascular techniques and a need to introduce catheter-based techniques to novice vascular surgical trainees [13,14,15,16]. Nevertheless excellence in open surgical techniques is still required of surgical residents, and incorporation of endovascular training into the curriculum of vascular training is now considered essential [17, 18]. This has all occurred in the setting of paradigm shifts in vascular surgery training. Residents can now enter a vascular surgical training program directly out of medical school. This training model is becoming more popular, and the number of programs offering the 0–5 training curriculum continues to increase. These integrated 0–5 vascular residencies pose new educational challenges as residents entering the specialty have very limited or basic surgical skills and little to no endovascular experience.

In response to these external constraints, surgical skill and simulation centers have emerged at academic institutions across the USA. A consensus statement from the Society for Vascular Surgery (SVS) , the American College of Cardiology (ACC), and the Society for Vascular Medicine and Biology (SVMB) published in 2005 encouraged simulation, stating that “In an effort to assist physicians with differing backgrounds and skills to reach a common benchmark of proficiency, metric-based simulation should be incorporated into training. This will provide skills acquisition in an objective manner, based on real-world situational experience” [19]. Surgical skills laboratories and simulation training allow for motor skill acquisition in a structured, stress-free environment free of adverse consequences to actual patients. Basic surgical skills are learned and practiced on models and simulators, with the aim of better preparing trainees for the operating room experience [20, 21]. Simulators offer the ability to perform multiple procedures while avoiding the real life time challenges of anesthesia induction, room turnover, and paperwork. Additionally, simulation can allow novices to perform repeated attempts at the same intervention without risk to a human patient. Simulation also provides an excellent opportunity for error analysis and simulated management of procedural complications [22]. Used properly, simulation offers an economic use of training time which is perhaps the most valuable resource to a residency program. In a recent survey of current trainees, 86% of respondents report that they believe there is educational value in simulation. Fifty-six percent of programs currently offer simulation training, most commonly in the form of peripheral endovascular simulators (70%), anastomotic models (58%), or endovascular aortic aneurysm repair simulation (53%), and more than a third of current fellows and senior residents (37%) have attended outside simulation courses [23].

This chapter will provide an overview of the results of available studies utilizing simulation to teach vascular techniques and discuss the potential benefits of using simulation in vascular surgery training.

Simulators Used in Vascular Surgery

Numerous simulation devices exist for vascular surgical training each with its own benefits and shortcomings. These models can be broadly characterized into five categories: low-fidelity synthetic, high-fidelity synthetic, animal, cadaveric, and virtual reality. Endovascular procedures lend themselves to simulation technologies much in the same way that laparoscopy does as 2D imaging leads to an ease of developing high-fidelity simulation.

The earliest versions of synthetic models for vascular trainees came in the form of benchtop anastomotic models. These required only a stable platform, graft material, suture, and basic instruments. With these mock-ups, structured, low-risk practice could be performed and was shown to be useful (primarily for junior residents) in improving skill [24]. Newer synthetic models range from simple to extremely high fidelity. Blood vessel anatomy can be synthetically simulated using devices as simple as a plastic tube to complex multi-material sculptures (Fig. 1). Synthetic models can be inexpensive and are the most broadly available tools for vascular simulation. Synthetic models can also be created for endovascular use. Systems of pressurized tubes can be used to simulate stent placement and catheter manipulation.

Fig. 1

Example of high-fidelity simulated synthetic abdominal aorta

We know from the aviation literature that level of simulator fidelity needs to be matched to stage of skill acquisition. Low-fidelity simulators (e.g., synthetic models) are appropriate for early training (cognitive stage) of novice learners, whereas high-fidelity simulators are more appropriate for advanced and more experienced learners.

Low-Fidelity Models

Low- fidelity simulations use materials and equipment that are different from those of the actual task considered. Partial-task trainers have always been applied to open surgery in the most basic forms. These models consist of 3D representations of body parts or body regions and provide functional anatomical landmarks useful for learning a particular skill. For example, plastic arms can be used for practicing venipuncture, or blue phantom models can be used for practicing ultrasound-guided percutaneous access skills. These basic models allow novice learners to practice the individual tasks of a procedure. The downside of using these models is that the interface with the user is passive and procedures are performed with no response from the simulator [25].

Whereas partial-task trainers allow for simulation of a specific or individual skill, procedure-specific trainers allow for simulation of a group of tasks in chronological order of an operation or part of an operation. These models are usually made from silicon or rubber and contain various levels of realism. Examples include the models manufactured by Limbs & Things (Bristol, Avon, UK). These inanimate models are used to practice open surgical skills including saphenofemoral junction dissection and ligation, carotid endarterectomy, and aortoiliac aneurysm repair (Fig. 2). These models are currently utilized in our own vascular skills lab to teach junior-level residents. These models are portable and easy to set up but tend to be relatively expensive requiring replacement of their main components. They will be discussed further below.

Fig. 2

Procedure-specific trainers allow for simulation of a group of tasks in chronological order of an operation or part of an operation. These models are manufactured by Limbs & Things (Bristol, Avon, UK) and are used to practice open surgical skills including saphenofemoral junction dissection and ligation, carotid endarterectomy, and aortoiliac aneurysm repair. Commercial companies (Limbs & Things, Bristol, Avon, UK) manufacture inanimate organ parts for saphenofemoral junction dissection and ligation, carotid endarterectomy, and aneurysm repair

Low-fidelity model partial-task trainers are also available for endovascular skills training. These models are relatively inexpensive (~$3000/unit) compared with the higher-fidelity models. These models are effective for learning basic endovascular skills and allow for tactile force feedback to be experienced by the learner while using real wires, catheters, balloons, and stents. Unfortunately, one-time use of the equipment can be costly, adding to the expense of training, and these models lack realism and face validity.

A low-fidelity endovascular model, the Simulator for Testing and Rating Endovascular Skills (STRESS) , has recently been described [26]. This simple low-tech model consists of a light source covered by a container which holds a dry glass model of the abdominal aorta and renal and iliac arteries with various stenotic lesions, elongations, and tortuosities. The model does not require fluoroscopy, contrast, running water, balloons, or stents. A camera mounted above the glass model provides a view of the entire “abdomen” on a monitor (Fig. 3). Using computer software, a plain abdominal radiographical image is merged with the live-camera feed, replicating a plain fluoro-mode while blurring the few visible edges of the glass model. Real catheters and guide wires can be introduced into the introducer sheath prepositioned in the external iliac arteries. Wire and catheter skills can be practiced while looking at the computer screen, giving the impression of using fluoroscopy. Contrast angiography can be simulated in the live view, replicating a non-subtracted, single-shot, contrast injection, which disappears after a few seconds.

Fig. 3

Schematic drawing of the STRESS machine which consists of a light source covered by a container that holds a dry glass model of the abdominal aorta and renal and iliac arteries with different tortuosities and stenoses. A camera placed above the glass model provides a view of the entire “abdomen” on a monitor. Using computer software, a plane abdominal radiographical image is merged with the live-camera feed, replicating a plain fluoro-mode. Introducer sheaths are prepositioned

High-Fidelity Models

An example of a higher -fidelity synthetic system is the pulsatile flow aortic model developed by Vascular International Foundation and School (VI). A group in Europe dedicated to providing supplementary training for vascular surgeons through short, intensive courses with hands-on skills training (for both open and endovascular procedures), VI has been offering training for over 20 years. This model has been widely embraced in Europe, where work hours are limited to 48-h weeks, as a means for trainees with insufficient operative exposure to gain experience. Furthermore, as training models vary widely throughout the European Union, these standardized teaching methods can ensure some measure of homogeneity in training. These techniques of standardized training have proven to be superior in a randomized study compared to traditional techniques, with the standardized group demonstrating improved technical scores (95% vs. 75%) and global rating scores (84% vs. 67%) [27]. These short training courses have also been shown to improve technical performance and quality on both carotid patch angioplasty and open aortic repair [28, 29]. Their open aortic model (Figs. 4 and 5) features a pulsatile flow system and a mock-up of abdominal contents. Using a replaceable aorta, trainees are able to experience a realistic feel of the vessel wall when performing an anastomosis. The synthetic abdominal contents allow for the rehearsal of retractor and clamp placement. Another benefit of this system is the pulsatile flow which allows for identification of defects in the anastomosis. The main drawback of the model is price and limited portability which preclude daily use by residents.

Fig. 4

Vascular International Foundation open aortic model

Fig. 5

Vascular International Foundation open aortic model

Animal Models

Animal models offer a high degree of realism and animal labs are still used for open and endovascular training. We have used animal models in our institution to teach senior-level residents and endovascular fellows techniques for iliofemoral angioplasty and stenting as well as open aortoiliac artery replacement. Animal models have also been used as test models for endovascular devices. Arterial stenosis and aneurysmal disease can be artificially induced through endothelial injury or sutured patches, respectively [30,31,32]. Use of animal models is limited. The animals can be expensive, especially if used for only one or two procedures, require special facilities and instruments, and have anatomical and size differences compared with humans. Furthermore, there are ethical and legal constraints associated with using animal models. Despite these limitations, large animal models do offer a highly realistic training opportunity for advanced interventions that cannot be simulated by a computer model.

Human Cadavers

Human cadaver models provide realistic conditions for both open and endovascular training. Human cadavers remain a mainstay in medical school education and are making a resurgence in surgical training as well. The American College of Surgeons (ACS) and Association of Program Directors in Surgery (APDS) have recently mandated incorporation of phase II modules into the surgical curriculum. A large majority of the modules include human cadaver dissection and practice of sentinel procedures. Cadavers, like animal models, have also been utilized to test endovascular devices. Garrett and colleagues describe how pulsatile antegrade arterial flow can be established in the arterial system of a fresh human cadaver following a thrombolytic process [33]. Endovascular procedures with standard arterial punctures and closures have been performed using this model. While this cadaver model provides the most realistic model to practice open and endovascular skills, use of this model is limited by restricted availability and cost associated with preservation and storage of the bodies.

Virtual Reality Simulation

Virtual reality (VR) is defined as computer technology that allows for a user to perform an operation or procedure in real time using a simulated three-dimensional system. VR simulation has been used extensively in high-stakes industries such as the airline, nuclear, and oil industries. In the aviation industry, it has been effective in providing pilots a means of training without actually flying an airplane [34,35,36]. Successful utilization of simulation in aviation ultimately led to the development of simulation programs applicable to minimally invasive surgery with Satava first proposing the use of the surgical simulator in 1993 [21].

Endovascular therapy poses technical challenges similar to those experienced in minimally invasive surgery, including reduced tactile sensation, and the need to overcome the proprioceptive-visual issues of working in a three-dimensional field displayed on a two-dimensional fluoroscopy screen. Several endovascular VR simulators are commercially available and include the Procedicus Vascular Intervention System Training (VIST™) simulator (Mentice AB, Götenborg, Sweden), the ANGIO Mentor™ (Simbionix, Cleveland, Ohio), and the SimSuite® (Medical Simulation Corporation, Denver, Colorado). These high-fidelity simulators include haptic, visual, and aural interfaces that provide near-realistic representations of the real procedure. These simulators provide a variety of training applications and include modules for angioplasty and stenting of the carotid, renal, coronary, superficial femoral, and iliac arteries. More recent technology has allowed for simulated aortic aneurysm repair, neuro-interventions, closure of patent foramen ovale, deployment of a caval filter, and implantation of cardiac pacemaker leads.

The Procedicus Vascular Intervention System Training (VIST™) simulator comprises a mechanical unit housed within a plastic mannequin cover, a high-performance desktop computer, and two display screens (Fig. 6). Modified instruments are inserted through the access port using a haptic interface device. Commercially available simulation modules can mimic arterial occlusive disease in the coronary, carotid, renal, and iliofemoral regions and over the wire lead placement for biventricular pacing. The learner selects appropriate instruments to perform virtual interventional procedures using the simulated fluoroscopic screen. Performance is measured using metrics such as total procedure time, fluoroscopy time, and markers of quality of performance such as stent placement accuracy (Table 1).

Fig. 6

The Procedicus Vascular Intervention System Training (VIST™) simulator comprises a mechanical unit housed within a plastic mannequin cover, a high-performance desktop computer, and two display screens. Modified instruments are inserted through the access port using a haptic interface device. Performance is measured using metrics such as volume of contrast used, fluoroscopy time, and markers of stent placement accuracy

Table 1 Comparison of VR endovascular trainers

The ANGIO Mentor™ Ultimate endovascular trainer has a similar range of arterial procedures as the VIST™. It differs from the VIST™ in that there is greater emphasis on patient monitoring, drug administration, and response to physiologic disturbance. For example, atropine can be administered to correct for bradycardia related to simulated carotid sinus stimulation. Appropriate therapies can also be provided for hypoxia and hypertension. This device allows for simulated complications to occur so that management of the complication can be practiced in a virtual environment. Two more affordable and portable versions of the simulator are now available, the ANGIO Mentor™ Express and ANGIO Mentor™ Mini (Fig. 7). These devices have a similar simulation package with less peripheral attachments such that the Mini can be transported in a handheld case.

Fig. 7

The ANGIO Mentor™ Ultimate endovascular trainer has a similar range of arterial procedures as the VIST™ with more advanced haptic technology. The ANGIO Mentor™ Express and ANGIO Mentor™ Mini have similar simulation packages but with less peripheral attachments such that the Mini can be transported in a handheld case

The SimSuite® is a larger simulator system with up to six interactive screens to facilitate multidisciplinary team training (Fig. 8). This system provides multispecialty training packages and personnel to support the training program. These simulators allow for pre-procedure briefing, patient intervention, and post-procedure analysis. Similar to the ANGIO Mentor™ system, response to patient physiology is a feature of this simulator.

Fig. 8

The SimSuite simulator provides tactile “haptic” feedback and displays real-time imaging and physiologic information

Virtual reality simulators have an advantage over low-fidelity simulators in that they have software capable of providing metric feedback. Learner’s skill can be objectively assessed, and output metrics can be used for objective evaluation and feedback of trainee progress. This provides an avenue for both self-directed learning and curriculum development. Some endovascular simulators also allow for surgical planning. Specific anatomical details of the patient from radiologic images can be installed into the simulator computer and the planned procedure can be rehearsed on the simulator prior to performing the same procedure on the patient. VR simulators have the added advantage of reuse ad infinitum and have no associated ethical issues related to their use.

There are, however, several limitations to using VR simulators for endovascular training. The most obvious limitation is the exorbitant cost of the simulator. Most of these devices cost more than $100,000 for a single unit and many additional thousands for maintenance over time. Endovascular simulators require regular maintenance and housing space. The need for constant software updates and calibration often necessitates a full-time technician to manage technical failures, regular calibration, and maintenance and updating of required software. In the current setting of vascular surgical training, with one or two fellows training at a given institution at one time, it is hard to justify the expense of a simulator when there is currently little data to support their validity and transferability. One proposed low-cost alternative is to set up regional centers where fellows could travel periodically for short training sessions [37]. Current training on the simulator is also limited by challenges in unrealistic tactile feedback and graphical interfaces. Significant improvement in haptic response and realism of the virtual environment are needed. Until the realism and the high cost of these simulators improve, it will be difficult to transition these devices out of the research labs into the training labs. Finally, it must be recognized that these devices are still partial-task simulators as they cannot teach some of the important skills associated with endovascular cases such as arterial puncture and closure.

In addition to use in training, vascular surgery simulation must be considered for its potential to revolutionize testing and assessment of vascular surgical skills.

Methods of Assessment

Traditionally competency in surgery has been defined as completion of a defined length of training or number of cases. In fact, this still holds true for endovascular procedures [38]. Other than some skills lab incorporation, there are currently no clear guidelines from the ACGME with regard to simulation training in vascular surgery. Additionally, at this time no US specialty board accepts simulation experience as a proxy for patient case logs. Operative log data lack content validity as they only indicate the volume of operations performed and do not capture procedural understanding, participation, or performance level. As such they are recognized to be an unreliable and indirect measure of technical skill [39, 40]. And it has been demonstrated that no correlation exists between the individuals’ operative experience as reported by case logs and their technical performance [41]. There is now increasing recognition that the number of procedures performed and time in training does not equate to expertise. As a result, the trend in medical skills training is to move toward using objective assessment tools to demonstrate technical competence.

Formal testing of surgical dexterity is not a modern concept. Fellowship in the Royal College of Surgeons required a technical skill exam through the early 1940s. In the USA, the American Board of Surgery conducted intraoperative assessments on prospective candidates through 1952. Both practices were halted due to logistical problems such as time, cost, and standardization.

In a prospective randomized trial, residents’ scores on the multiple-choice American Board of Surgery In-Training Exam (ABSITE) did not correlate with their technical ability measured by either skill testing or intraoperative assessment [42]. This supports the findings of a pilot technical skill assessment conducted with the European Board of Surgery Qualification in Vascular Surgery exam in 2002. European candidates performed a saphenofemoral junction ligation and a tibial artery anastomosis on open models. Additionally, dexterity was assessed with a knot-tying test . Internal consistency was demonstrated among the three technical exams, but the study found no correlation between technical ability and the candidates’ scores on an oral knowledge examination. Currently, the multiple choice Vascular Surgery In-Training Exam (VSITE) is the only standardized test given during vascular surgery residency, and no standardized method exists for surgical skill evaluation. Written and oral examinations, the established markers of surgical competence, only assess knowledge base and clinical reasoning and do not evaluate technical performance or nontechnical skills critical to managing an operation or crisis scenario. In most programs, direct observation has been the only assessment tool utilized for the appraisal of technical ability. Simulation-directed surgical skill testing offers a potential solution to these issues.

There is some evidence supporting simulation as a valid means of skill testing in vascular surgery. Two studies have shown that performance of a carotid endarterectomy on a benchtop model can discriminate senior from junior trainees, but not more advanced levels [43]. Bench models may not properly evaluate complex decision-making and crisis resolution. Technical competence on a bench model may not translate into an independent environment. Therefore evaluating technical competence during crisis may help delineate these advanced trainees. Simulated procedures in high-fidelity operating room theaters have been used successfully in this regard [44].

The Imperial College Surgical Assessment Device uses electromagnetic sensors to track hand movements. Economy of motion during simple tasks such as knot tying has been shown well to correlate with dexterity in complex procedures. No correlation with endovascular skill has yet been demonstrated. Virtual reality systems can often offer direct feedback metrics such as procedure time, fluoroscopy time, handling errors, and contrast volume.

Observer assessments can be performed with checklists, global assessments, or some hybrid method.

Methods of assessing performance and improvement in performance in a surgical skill are essential to the development and implementation of a vascular surgical skills lab. Objective measures of skills performance utilized in skills training will be discussed below.

Time-Action Analysis

At its simplest, a scoring system for skills training may include time and errors. Time-action analysis has been used extensively as a method of objective assessment of performance in open and minimally invasive surgery [45,46,47]. The method can be applied to real life or simulator performance and involves breaking the procedure down into a series of steps with performance analyzed by how long the learner takes to complete the task [48, 49]. This procedure is very personnel and resource intensive because of required setup and video analysis. Decreased time to perform the task may indicate progression of skill, but the amount of time taken to complete the individual procedural steps does not in and of itself offer any measure of the quality of the performance. Therefore, time-action analysis may require supplemental markers to fully assess progression of skills.

Error Analysis

The 1999 National Academy of Science Institute of Medicine report, “To Err is Human,” raised awareness of patient safety issues [50]. “Error in the performance of an operation” was cited as one of the leading causes of patient deaths in hospitals. The uncontrolled introduction of laparoscopic cholecystectomy made the public and surgical community more aware of the implications that surgical training could have on patient safety [51]. Cost issues related to surgical complications have made third-party payers keenly aware of training and surgeon competency; as a result, human reliability and error analysis is now an evolving field in healthcare.

Error scores have been proposed as discriminators of technical skill though inherent difficulties exist in defining surgical or medical error as there is no standardized taxonomy [52]. It is, however, possible to differentiate technical skill by examining both the frequency and type of error committed during laparoscopic cholecystectomy and pyloromyotomy [53,54,55]. To date error analysis in endovascular training and assessment is at an early stage with no reported studies examining this question in vivo. Modern simulator technology allows reporting of catheter and device handling errors. Patel et al. reported a reduction in the composite catheter handling error scores of interventional cardiologists performing a virtual carotid angiogram following simulator training [56].

Motion Analysis

Motion analysis may offer a less time-consuming option. Efficient and purposeful hand movements are a discriminator of technical skills in surgery [57]. The technology is already available, and indeed surgical dexterity is currently assessed using this modality for the open surgery portion of the European Board of Surgery Qualifications in Vascular Surgery (ESBQ VASC) examination. The Imperial College Surgical Assessment Device (ICSAD) is used to track hand movement in three dimensions using electromagnetic sensors with a composite score based on economy of motion and qualitative analysis [58]. Clearly this technology is associated with significant cost. Nonetheless, this is a potentially exciting area for future research with no published studies to date examining hand motion analysis in the open vascular and endovascular arena.

Objective Structured Assessment of Technical Skills (OSATS)

Beyond simple metrics, rating of technical performance by expert observers remains an important assessment tool. In 1996 at the University of Toronto, Faulkner, and colleagues, under the direction of Richard Reznick, introduced the Objective Structured Assessment of Technical Skills, or OSATS. A global rating scale (GRS) is a quantitative assessment tool based on appraisal of seven aspects of quality in operative performance. Each component is evaluated on a 5-point grading scale. The items included respect for tissue, time and motion, instrument handling, knowledge of instruments, use of assistants, flow of operation/forward planning, and knowledge of the specific procedure [24]. This method has been demonstrated to differentiate between experience levels in both open and minimally invasive surgery [59,60,61].

A modified GRS has been shown to differentiate endovascular experience and training using a VR simulator. Hislop et al. have proven the construct validity of an OSATS-derived Modified Reznick Scale (MRS) for post hoc video-based rating by two blinded observers during a virtual selective carotid angiography [62]. The first two studies examining VR transfer of training to the catheterization lab both used the modified rating scales [63, 64]. Tedesco et al. have demonstrated that a single-blinded expert observer was able to discern differences in endovascular experience during a virtual renal artery stent procedure using a structured global rating scale [65]. Although the EVEREST study included only experienced interventionalists, interventionalists who scored high on the OSATS-derived generic rating scale were more likely to be experienced in CAS [66].

Procedure-based assessments possess high inter-rater reliability (G > 0.8 using three assessors for the same index procedure), excellent construct validity, and positive user satisfaction and acceptability (trainees and reviewers). The tool, however, is very procedure-specific and long (checklist of up to 62 items) which limits its practicality for use in evaluating common but increasing complex hybrid open and endovascular procedures.

Procedure-specific checklists used in conjunction with GRS have been shown to be effective and reliable assessment tools of surgical dexterity using synthetic and cadaveric models as well as in live operating [67, 68]. Post hoc video analysis, though not mandatory, does reduce the potential for bias. The main disadvantage of this mode of assessment is that a large amount of time is required from expert assessors. Full-length video viewing is required as edited video assessment appears to reduce the reliability of assessment [69]. Based on a systematic review of methods of assessment, checklists and global rating scales presently appear to be most accepted as the “gold standard” for objective technical skill assessment. Their use in the OR, however, has been limited partly due to the variability of operative procedures (i.e., they do not all conform to a standardized checklist), the time required for completion of these tools, and faculty familiarity with these tools and their application. Furthermore, benchmark levels of performance for these assessments have not been defined. While these shortcomings should not prevent their use for formative assessment (assessment for learning, i.e., feedback and discussion), they may prohibit use for high-stakes examinations (summative assessment).

VR Simulators

The major advantages of VR simulation are the ability to automatically and instantly provide an objective performance report based on quantitative and qualitative assessment parameters. Error scores and rating scales can be used in combination [62, 63, 70]. Used in a standardized setting, it is possible to distinguish between subjects of different levels of experience [71,72,73]. Assessment of nontechnical skills, such as appropriate drug administration and physiologic monitoring, is also possible with most of the current generation of simulators.

The validity of this method of assessment is under evaluation as discussed below. Currently, performance reporting remains unsatisfactory, quantitative measures of performance related to procedure time and use of the c-arm are well reported, but further work is necessary for developing more subtle indicators of performance and judgment such as clinical outcome and technical error. Though further work is required, simulation-based assessment is potentially a mechanism for selecting candidates for surgical or interventional training programs and may be a requirement for recertification or gaining credentials to perform procedures [74].

Relationship Between Nontechnical Skills and OR Performance

While not actual measures of surgical performance, self-reported operative competence and stress levels appear to be important markers of coping ability. The evidence suggests that effectively coping with stressful events in the OR has a beneficial impact on technical skills performance [75]. Similarly, the relationship between nontechnical (communication, decision-making, situational awareness, and leadership skills) and teamwork skills and technical performance in the OR is strong, and it is now widely reported that deficiencies in teamwork, rather than simply poor technical ability, contribute more commonly to adverse events in the OR [75].

The Evidence for Simulation in Open Vascular Skills Training

Sidhu and colleagues from the University of Toronto have demonstrated that laboratory training does improve basic vascular skills [76]. Acquisition of skill was significantly affected by model fidelity and level of training as measured by checklist and final product analysis. Practice on high-fidelity models (cadaver brachial arteries) improved skill acquisition for both junior and senior residents learning vascular anastomosis techniques, as compared with low-fidelity models (plastic tubing). This was the first study to address the combination of the effects of level of training and model fidelity on skill acquisition. These findings conflicted with previous studies performed at the same institution that demonstrated equivalency of low- and high-fidelity models for plastic surgery and urology procedures [77, 78]. This work suggests that there is more benefit in using higher-fidelity models for more experienced learners. In other words, for optimal motor learning, the level of difficulty during the skill acquisition must be adjusted to the learner’s current expertise level.

A saphenofemoral junction model was used by Wolfe and Darzi to assess the surgical competence of learners of all levels of experience, from senior house officers to experienced consultants, by using the procedure-specific Imperial College Evaluation of Procedure-Specific Skill (ICEPS) rating scale in conjunction with the Objective Structured Assessment of Technical Skill (OSATS) global rating scale [43]. The saphenofemoral junction groin model (Limbs & Things, Bristol, UK) depicted the human saphenofemoral junction and its tributaries. This model allows for incision of the skin and dissection through the superficial fatty and deeper fascial layers. The fluid-filled long (greater) saphenous vein and its four groin tributaries can be identified and ligated and the saphenofemoral junction disconnected. This study showed that surgical performance continues to improve significantly beyond consultancy. Importantly this study demonstrated the construct validity and high interobserver reliability of the ICEPS rating scale supporting its use in formative and summative assessment.

Carotid endarterectomy (CEA) is an operation that is associated with substantial risk should the operation not be performed appropriately. A synthetic benchtop model (Limbs & Things, Bristol, UK) has been developed in conjunction with St Mary’s Hospital, London, UK. This model consists of a plastic box and supporting structures with a replaceable latex carotid artery containing adherent plaque. John Wolfe and colleagues demonstrated that this bench model is a valid tool for the evaluation of basic technical skills in the performance of CEA . Use of the model in a simple, easily reproducible benchtop environment discriminated between junior and senior vascular trainees by both evaluations of video performance and end-product scores [44]. The model, however, failed to discriminate between senior trainees and consultant surgeons, with these two groups performing at the same level in all assessments. This demonstrates the inherent weakness of using these benchtop models for training more advanced learners. Decision-making, judgment, situation awareness, and leadership skills cannot be evaluated in this situation. More complex simulations, i.e., simulated operating rooms, may be needed to discriminate between more senior trainees and consultants. That being said, these simple models allow the basic steps of a procedure to be taught to trainees in a non-pressurized environment where the patient is not at risk. Use of this model allows for acquisition of the basic skill components of a CEA (order of the clamp placement and removal, site of the arteriotomy incision, and basic considerations of the endarterectomy) before moving to the operating room. Also, video review has the advantage of identifying errors in performance that can be demonstrated to trainees, providing valuable feedback [79].

Because of the previously noted trend toward preferential endovascular repair, the incidence of open aortic surgery is significantly decreasing. Records from Medicare beneficiaries from 1995 to 2008 and ACGME records from 1999 to 2008 demonstrated that the average annual number of open AAA repairs performed by vascular fellows decreased from 44.1 to 21.6 in this time period. Also noted was a concomitant increase in endovascular repair of AAA with approximately 78% of AAA repairs in 2008 done by EVAR [80]. The efficacy of simulation training for open AAA repair was investigated by Robinson et al. [81]. They randomized a group of senior residents to one of two simulation training sessions. The first was performed with vascular attending oversight, and the second session was an identical course conducted with a skills lab coordinator. The authors reported that the less experienced residents demonstrated greater improvement after simulation training and that those mentored by a vascular attending had a significant improvement in overall operative competence, but those overseen by a skills lab coordinator did not. Their primary conclusion was that simulation training efficacy was dependent on vascular staff involvement. The study was not, however, able to demonstrate that improvement in the simulation lab correlated with improvement in the OR.

Another study noted that 24 senior general surgery residents participating in 5 structured 4-hour cadaver skill sessions where they performed 5 different vascular exposures, including the supraceliac aorta, demonstrated significant improvement in both the mean pre- and post-oral examination scores (P < 0.001) and the mean operative confidence scores (P < 0.001) for each individual exposure [82].

The Evidence for Simulation in Endovascular Skills Training

Driven by the need to validate endovascular VR training, three specialties involved in the endovascular treatment of vascular diseases in Europe have joined forces as the European Virtual reality Endovascular RESearch Team (EVEREST) . The goal of this group is to improve training of the present and future endovascular therapists through combined research and curriculum development. It is understood that before endovascular simulators can be universally applied to vascular training programs, demonstration of reliability, feasibility, and validity is necessary. It is incorrect to assume that a realistic simulation equates to an effective training or assessment model [83].

Perhaps more than in any other vascular bed, simulation can play a vital role in instructing interventions in the cervical carotid circulation and therefore deserves special attention here. Since carotid interventions provide a small absolute risk reduction, even a rare technical error can override a surgeon’s margin of efficacy. Additionally, small missteps during a carotid stent placement can result in severe morbidity and even mortality. Clearly these procedures must be assiduously learned prior to attempting independent performance. There are few true high-volume centers, however, and a paucity of experts to train novices. In the USA, multispecialty consensus statements issued by the American College of Cardiology, American College of Physicians, Society for Cardiovascular Angiography and Interventions, Society for Vascular Medicine and Biology, and Society for Vascular Surgery provide recommendations on the training and credentialing for CAS and other catheter-based interventions [19, 84]. This statement reflects a recent worldwide shift in focus toward outcome-based education throughout the healthcare professions. This paradigm change derives in part from attempts by academic institutions and professional organizations to self-regulate and set quality benchmarks, but chiefly it represents a response to public demand for assurance that doctors are competent [33].

This stance was adopted by the Food and Drug Administration (FDA) with the approval of a CAS system in August 2004 [85]. The FDA supports the use of simulation training as a component of physician training for CAS. Another requirement of the FDA approval for CAS was the initiation of a post-marketing surveillance study to assess the safety of the new device in everyday use and to assess its safety in the hands of operators with varying levels of experience. Two such post-marketing surveillance studies provide promising results [86, 87]. These studies evaluated the performances of experienced endovascular physicians who sought to learn a new procedure by using short training courses.

Validity

An overview of the published papers that have sought to support the validity of various modules of computer-based simulators is provided in Table 2 [62,63,64,65, 70,71,72,73, 88,89,90,91,92,93,94,95,96,97,98,99]. Most research has been conducted using the Vascular Interventional Surgical Trainer (VIST, Mentice, Gothenburg, Sweden).

Table 2 Overview of VR endovascular assessment and training studies

Patel et al. revealed that participants of the Guidant CAS 2-day regional training course using the VIST simulator had improved performance across five test trials as assessed by the metrics (catheter handling errors, procedure time, fluoroscopy time, and contrast volume) [70]. This study represents the largest collection of such data to date in carotid VR simulation and is the first report to establish the internal consistency of the VIST simulator and its test-retest reliability across several metrics. These metrics are fundamental benchmarks in the validation of any measurement device. Composite catheter handling errors represent measurable dynamic metrics with high test-retest reliability that is required for the assessment of high-stakes procedural skills.

A supervised 2-day virtual CAS training course for experienced endovascular physicians on the ANGIO Mentor™ simulator provided similar results. Post-course interventions were performed faster, with less radiation, and with fewer catheter handling errors. Spasm of the internal carotid artery occurred less frequently. Post hoc ratings by two experienced CAS physicians showed excellent inter-rater reliability, reduction in number of observed errors, and an increase in quality of performances when comparing the group’s pre- and post-course performances.

Dayal et al. evaluated the use of simulation to train novice and advanced interventionalists in carotid angioplasty and stenting (CAS) [72]. After didactic instruction, each participant performed CAS followed by training on the VIST simulator and performance of a second graded CAS. Participants had reduced procedural and fluoroscopic time and improved wire and catheter techniques. These results were consistently better for experts than novices. This supported the construct validity of the simulator that it can accurately reflect the skill of the individual.

Hsu et al. conducted a similar randomized trial comparing performance of CAS by skilled and untrained interventionalists [88]. After a pretest, participants were randomized to receive supervised practice on the Procedicus VIST simulator or no practice. Procedural time and successful completion improved significantly and correlated with previous experience, thereby supporting construct validity of the simulator. The majority of the participants rated the simulator as realistic with good force feedback supporting face validity. These participants also agreed that training on endovascular simulators should be mandatory prior to performing CAS in actual patients.

Studies carried out by the EVEREST group differed from these two studies. Only physicians with the basic endovascular skills and appropriate medical background to treat carotid artery stenoses were included [71, 94,95,96]. Experienced interventionalists were found to have shorter procedural and fluoroscopic times and improved wire and catheter techniques for CAS . These findings confirm the ability of the simulator to accurately reflect the skill of an individual, again supporting its construct validity [62, 71,72,73, 94, 96,97,98].

Learning Curve

The term “learning curve” used in the context of skills training refers to the time taken and/or the number of procedures an average practitioner needs to be able to perform a procedure independently with an acceptable outcome [100]. Learning curve can be measured in terms of patient outcomes (morbidity or mortality) or as measures of surgical procedure (blood loss and operative time) [101]. Mastery of the clinical tasks of an endovascular procedure often follows a steep learning curve; this has obvious implications for patient safety, particularly when novices are performing invasive procedures on real patients.

Lin et al. analyzed the outcomes of sequential groups of patients undergoing CAS and demonstrated decreased procedure-related complications, fluoroscopic time, and contrast volume used with increased experience [102, 103]. Simulation-based training may allow the early part of this learning curve to take place without exposing the patient to unnecessary risk. Other studies examining the potential for using VR systems in endovascular skills training have analyzed the learning curves of both novice and expert subjects. Results are mixed. Dayal et al. demonstrated improved simulated performance of CAS procedure by novice subjects. Expert performance was not improved following training [72]. Hsu et al. showed significant improvement in both novice and expert subjects [88]. Aggarwal et al. analyzed the learning curves of experienced open vascular surgeons and demonstrated improved performance (procedure time and contrast used) following VR simulator training using a renal artery stenting model [71]. A second study from this unit showed that while there is an expected learning curve in performing endovascular tasks, endovascular skills were widely applicable, and once learned these skills could be readily transferred between different simulated procedures [95]. Similar improvements in simulator training have been reported for iliac and renal angioplasty [92, 93].

These training studies suggest that repetitive practice on the endovascular simulator benefits the novice learners more than the expert subjects. Learning curves are shortened as the novice becomes more familiar with the simulator. Psychomotor skills gained with simulator practice can become automated by the time the procedures are performed in real patients [104].

Transfer of Skills

Skills transfer, i.e., significant improvement in operative performance following a period of dedicated skills training, has been demonstrated following VR training in laparoscopy [105, 106]. Recent evidence of skills transfer using VR simulation for endovascular skills training is encouraging. Berry et al. demonstrated improvements in both combined global rating scale and task-specific checklist after repetitive practice in both the porcine and VR groups. The improvement was shown to transfer from the VR simulator to the porcine model [63]. Only one randomized trial in the endovascular field has examined skills transfer from the VR to the OR [64]. Surgical residents with no prior endovascular experience were enrolled. All participants received the same didactic introduction and were randomized to receive either mentored simulation training (max 2 h) on a standardized iliofemoral angioplasty/stenting model or no simulation training. The simulator-trained group received significantly higher ratings on a supervised real iliofemoral procedure compared with the control group. Large randomized controlled trials need to determine whether simulated training in other endovascular procedures also translates into improved skills and if these skills are maintained over time.

Performance Benchmarks

Simulator- derived performance reporting allows the learning curve of an individual trainee to be tracked. Practice can continue until a predetermined benchmark level of skill (based on the median performances of highly experienced physicians in the field) can be demonstrated. Further work is required to define appropriate benchmark levels of skill both within VR simulation and in vivo. Personalized training such as this may be a more effective way of training than undertaking a set number of repetitions [106]. This style of training is known as proficiency-based training – please see the chapter on this topic for more details.

Design and Implementation of a Stepwise Proficiency-Based Vascular Training Curriculum

Successful incorporation of simulation into residency programs is dependent on the effectiveness of the curriculum. Although a particular simulator may be associated with numerous facets of validity, it is the curriculum that dictates how rapidly trainees will learn [25]. The curriculum ultimately dictates how effective a particular simulator will be in providing clinically relevant and useful skills. An effective skills curriculum should encompass goal-oriented training; a cognitive component; deliberate, distributed, and variable practice with appropriate methods for instruction and feedback; an amount of overtraining and maintenance training; and sensitive and objective metrics for measuring skill proficiency [51].

Examples of Comprehensive Vascular Skills Training Programs

The OHSU Program

Most of the emphasis in teaching vascular skills have focused on advanced endovascular techniques. With the introduction of the 0/5 training programs in the USA where residents enter vascular training directly from medical school, there is a need to teach vascular skills early in training. To this end, we have developed a vascular skills lab with a basic curriculum appropriate for novice surgical residents. Skills taught in our skills lab include performance of an ankle-brachial index (ABI) and vascular-specific physical examination, interpretation of noninvasive vascular laboratory studies, ultrasound-guided percutaneous vascular access, ultrasound assessment of venous conduits for bypass grafting and dialysis access, techniques of performing basic vascular anastomoses, interpretation of imaging studies pertinent to vascular surgery (angiography, CTA, MRA), radiation safety, fluoroscopy, and basic catheter skills (Table 3).

Table 3 Basic vascular surgery skills

We have utilized a rather broad definition of vascular “surgical skills” and have incorporated features beyond just technical skills into the laboratory. We feel that these nontechnical skills are essential components of vascular surgical training for novice trainees. These nontechnical skills have been identified as skills that residents have the least confidence in because of the variable opportunities on the differing services to patient exposure. We therefore incorporated these “surgical skills” into the curriculum to eliminate learning opportunities based on random exposure to the skill set and because our current training format does not allow for specific time commitment to the learning of these skills.

Description of Laboratory Modules

Vascular- specific skills are grouped into four modules in our laboratory with each module organized around a specific theme. Modules are 2 h in length and each module is initially covered in one session. The modules incorporate (1) a didactic portion which includes a group lecture with handouts of the lecture for self-learning and if appropriate a video demonstration of the skill, (2) hands-on exposure of the different skills, (3) practice of the individual skill, and (4) post-module questionnaires evaluating course content and teaching techniques. Pre-module and post-module cognitive and skill tests are administered.

Ultrasound Basics

A didactic lecture format familiarizes trainees to the basic principles of ultrasound physics and to the principles of central venous catheterization. Participants are taught anatomic landmarks to safely place arterial and central venous catheters and are provided an algorithm to maximize safety in placement of arterial and central venous catheters. Complications that can occur with percutaneous access are discussed and treatment algorithms to manage these complications addressed.

Residents are also given a brief orientation on the SonoSite™ portable ultrasound machine with instruction on transducer selection, anatomy, and orientation, as well as how to optimize the ultrasound image through changes in gain and depth. These skills are practiced on simple synthetic models. Ultrasound-guided percutaneous access techniques are then practiced. Simple synthetic models are used initially, using a standard Cook™ micropuncture introducer set ; these skills are then applied to more lifelike models. Percutaneous access techniques on a prosthetic internal jugular vein, subclavian vein, and common femoral artery are practiced and ultrasound-guided placement of a central venous catheter performed. Performance of the skill is measured using a task-specific checklist and a global rating scale.

Finally, trainees are instructed on how to use the ultrasound for visualization of venous conduits. The greater and lesser saphenous, basilic, and cephalic veins are identified on a model. The trainees then use each other as models to learn ultrasound visualization of these conduits. Learners are required to measure and record the diameter of individual venous segments.

Vascular Laboratory Interpretation

The key components of vascular anatomy and the vascular physical examination are reviewed. The use of a continuous wave Doppler to perform an ABI is described as is interpretation and clinical significance of audible monophasic, biphasic, and triphasic continuous wave Doppler signals. Residents practice performance of ABIs on each other.

A PowerPoint lecture introduces residents to the noninvasive studies available for evaluating upper and lower extremity arterial disease. Residents are instructed on interpreting normal and abnormal Doppler-derived waveforms, segmental pressures, duplex studies of native arteries and bypass grafts, toe pressures and toe/brachial indices (TBI), and laser Doppler examinations. Trainees also learn to interpret vein mapping studies of the greater and short saphenous veins as well as cephalic and basilic veins. They are introduced to vascular laboratory studies for detection of deep venous thrombosis and valvular reflux. Carotid, renal, and mesenteric duplex examinations are described and interpretation criteria for carotid, renal, and mesenteric artery stenosis presented.

Residents are given handouts with a succinct summary of the material covered in the didactic session. Interpretation of standardized vascular lab work sheets using vascular lab cases is practiced. Answers to the “unknowns” are reviewed and feedback provided in a group discussion.

Vascular Instruments and Anastomotic Techniques

Trainees are introduced to instruments , sutures, and basic techniques required to perform a vascular anastomosis. Participants are first taught the names and characteristics of the instruments used for vascular isolation, clamping, and suturing. They are also familiarized with sutures and needle types used in constructing a vascular anastomosis and are introduced to prosthetic grafts utilized for dialysis access, arterial bypass, and open aneurysm repair. A handout including basic vascular techniques and a picture of commonly used vascular instruments is provided.

A video clip demonstrating the proper technique of an end-side vascular anastomosis is reviewed and discussed. Using benchtop models and grafts, the trainees are then taught to create transverse, longitudinal, and circumferential arteriotomies. Trainees perform basic vascular anastomoses, including patch angioplasty, end-to-end, and end-to-side closures. Participants then practice these skills. Concurrent and summary feedback is provided to each resident. Performance of the skill is measured using a task-specific checklist and global rating scale.

Vascular Radiology

This module is designed to prepare the trainees to pass the required OHSU Hospitals and Clinics non-radiologist fluoroscopy physician test: Trainees are instructed to study the OHSU fluoroscopy training manual prior to beginning the module. Participants are taught basic radiation physics and instructed on the biological effects of radiation, how radiation exposure is monitored, and in the use of lead protective clothing, i.e., lead glasses, shields, and gloves. Techniques used to obtain the sharpest fluoroscopic images while limiting X-ray dose rate to the patient and operator are described. The learners are also introduced to the control panel of a C-arm and instructed on how to acquire, view, and store intraoperative images on the hospital digital imaging system.

Differing contrast agents, drug interactions, and complications related to contrast administration are described, and residents learn to identify and have a working knowledge of the sheaths and catheters most commonly used for intraoperative angiography and venography and to understand the steps in performing an intraoperative angiogram or venogram.

Finally, residents are taught to interpret basic normal and abnormal imaging studies pertinent to vascular surgery including CT angiograms and digital subtraction angiography. A collection of unknown normal and abnormal studies is interpreted by the residents. These studies are reviewed and discussed as a group.

Review of Our Data

Preliminary data clearly indicates the vascular skills laboratory is well received by the learners. Residents believe that all of the lab modules meet their educational objectives and that the content is appropriate and applicable to their training needs. Technical skills improved, and post-module cognitive test scores were significantly higher than pre-module tests for all modules tested. Interestingly we found that senior residents scored no differently than junior residents on cognitive testing suggesting that the skills lab should be introduced early in the surgical training program. We recognize that this curriculum has imperfections. Early in the course, it became clear that we had too much material and too many tasks in each module. It also became obvious that we did not provide enough time for deliberate practice and scheduled reinforcement of the technical skill. Our current lab curriculum has made provisions for these findings. We have also expanded our curriculum to include cadaver and porcine models for our more senior residents.

The LSU Program

At LSU, the Fundamentals of Vascular Surgery Symposium is held annually for integrated vascular surgery residents from around the USA. The pilot program for the open skill testing (FVS) occurred in October 2012. Twenty surgical trainees completed three vascular skill assessment models, each under the observation of two experienced assessors blinded to their training level. Two models were designed to simulate an end-to-side anastomosis (ES) and a patch angioplasty (Patch). A third model required suturing around a clockface design printed on patch material (Figs. 9 and 10) to emulate radial suturing as would be performed on a proximal aortic anastomosis. The model is placed in a clear plastic tube to simulate the depth of the abdominal cavity (Fig. 11). Trainees are given 5 min to perform the task of suturing around the entire “clock” with a 3-0 SH suture. Residents’ scores on this simulation correlate strongly with their operative experience (Spearman’s rho = 0.789, P < 0.001). Benefits of the clockface model include its relatively low cost and ease of transport, allowing trainees to practice away from the hospital.

Fig. 9

(a–c) LSU clockface model instructions

Fig. 10

Clockface model demonstration

Fig. 11

Plastic tube in which the clockface model is secured to simulate the depth of the abdominal cavity

ACGME log experience was recorded. Secondary evaluations of all three finished models were then performed by four blinded assessors. Inter-rater reliability among the seven assessors was high (Cronbach’s α = 0.93). Evaluations acquired by direct observation correlated well with participants’ training level/experience for all three models (ES r = 0.85, Patch r = 0.71, CF r = 0.82). Highest correlation with training level/experience was obtained with a combined score for each participant incorporating all observed ratings on each model (r = 0.93). Evidence for construct validity was collected by demonstrating each model’s ability to discern junior (Pre-MD to PGY2) from senior (PGY 3–5) trainees (ES P < 0.005, Patch P < 0.05, CF P < 0.001). Internal consistency was confirmed for each participant on all three models (Cronbach’s α = 0.89). Finished product evaluation demonstrated fair to poor correlation with training level/experience (ES r = 0.51, Patch r = 0.53, CF r = 0.24). These results supported construct validity for three vascular skill assessment models. Our data also demonstrates that the most accurate assessments are obtained by direct observation with trained evaluators.

The goals of the course are twofold:

• 1. For vascular surgery residents: With a faculty to resident ratio approaching 1:1, attendees spend 3 days receiving hands-on instruction in vascular techniques. Special emphasis is placed on procedures less commonly performed during residency such as open thoracoabdominal aortic approaches, subclavian/tibial vessel exposures, and complex endovascular procedures.

• 2. For program directors: Using vascular skill assessment models, the course faculty spend hours observing and grading each attendee. This feedback is provided directly back to the program director. These outside assessments of residents’ skill are a unique and valuable resource for portfolio building, milestone development, and individualized simulation curriculum design.

The course curriculum includes instruction (fresh cadaver lab, endovascular skill stations, open skill stations, didactics) and assessment (FVS, FEVS). There are also opportunities for simultaneous teaching and assessment (“Suturing with the Experts,” planning stations for EVAR measurement). Course content has been adjusted based on attendee feedback, most notably increasing the cadaver content, shortening the didactics, and providing more hands-on instruction and immediate feedback.

As a measure of educational effectiveness, all residents complete a pre- and post-self-assessment of confidence in nine vascular skills. Pooled results from the first 3 years demonstrate a statistically significant improvement in each proficiency including performance of carotid stent (p < 0.05), thoracoabdominal aorta exposure (p < 0.001), and EVAR planning based upon CTA measurement (p < 0.01). All attendees (100%, 48/48) reported being either “Very” or “Extremely” satisfied with the education experience. Questionnaires were sent to each program director after the course and a 93% response (28/30) was achieved. All reported being either “Very” or “Extremely” satisfied with the skill assessments generated by the course, and 96% of the responders (27/28) felt the reports would be useful in helping the residency program address the attendees’ strengths and weaknesses.

From this experience, we feel the following components are useful in creating a valuable vascular surgery simulation course:

• 1. High faculty to attendee ratio (minimum 1:2)

• 2. Low attendee to simulation station ratio (maximum 2:1)

• 3. Large fresh cadaver component

• 4. Emphasis on procedures rarely performed during residency

• 5. Limited didactics

• 6. Focused individual skills training and feedback to attendees

• 7. Focused individual skill assessment and feedback to program directors

The Fundamentals of Endovascular Surgery (FEVS) model was developed in both silicon and virtual reality versions. Twenty individuals (with a range of experience) performed four tasks on each model in three separate sessions. Tasks on the silicon model were performed under fluoroscopic guidance, and electromagnetic tracking captured motion metrics for catheter tip position. Image processing captured tool tip position and motion on the virtual model. Performance was evaluated using a global rating scale, blinded video assessment of error metrics, and catheter tip movement and position. Motion analysis was based on derivations of speed and position that define proficiency of movement (spectral arc length, duration of submovement, and number of submovements).

Performance was significantly different between competent and noncompetent interventionalists for all three performance measures: motion metrics, error metrics, and global rating scale. The mean error metric score was 6.83 for noncompetent individuals and 2.51 for the more experienced group (P < 0.0001). Median global rating scores were 2.25 for the noncompetent group and 4.75 for the competent users (P < 0.0001). The FEVS model successfully differentiated competent and noncompetent performance of fundamental endovascular skills based on a series of objective performance measures. Furthermore, we were able to successfully demonstrate that performance on an exact replica VR model correlated to performance on the physical model, further lending support to the validity of this platform. This model is now being proposed to serve as a platform for skill testing for all trainees, and multi-institution trials of both models were planned for launch in 2018.

Future Studies

The intent of simulation training is to shorten and flatten the learning curve for real procedures. To date no studies have objectively investigated the degree to which VR endovascular simulators satisfy this demand. Research needs to be conducted, similar to that performed in the airline industry and laparoscopic field, to calculate the transfer-effectiveness ratio (TER) for vascular simulator-based training curricula [107, 108]. Transfer-effectiveness ratio is calculated as the difference in number of trials or time taken to achieve performance criterion (in the air) between untrained and simulator-trained pilots divided by total training time received by the simulator-trained group. This ratio allows you to calculate how time-effective the addition of a simulator would be in a training program in relation to initial outlay costs. Ratios >0.5–1.0 are achieved by training programs containing modern flight simulators and 2.28 by proficiency-based training curricula including laparoscopic simulators [109].

Credentialing and certification of surgeons as part of continuing education is not a new concept. Currently, the American Board of Surgery utilizes the six core competencies established by the ACGME for their Maintenance of Certification (MOC) program. This program insists on continual learning over time. To ensure MOC , physicians need to demonstrate (1) evidence of professional standing through maintenance of an unrestricted license, hospital privileges, and satisfactory references, (2) evidence of commitment to lifelong learning through continued education and periodic self-assessment, (3) evidence of cognitive expertise based on performance on a secure examination, and (4) evidence of evaluation of performance in practice, using tools such as outcome measures and quality improvement programs, and evaluation of behaviors such as communication and professionalism [51]. Although technical skills training and simulation are not part of the ABS MOC program, future studies in this area would be important. Research conducted on more senior learners with limited endovascular skill is needed. Simulation could potentially play an important role in the reentry of these physicians into mainstream practice and maintenance of technical skills for “certification.” Physicians who have completed training may benefit from continuing education and simulator-based training to support their continued learning and improvement of cognitive and technical skills. Repetition, self-assessment, and the opportunity for feedback are the cornerstones for deliberate practice as defined by Ericsson [85, 86].

Similar to athletes and musicians, physicians may benefit from “warming-up” on a simulator before an elective procedure. The opportunity for endovascular therapists to practice complex endovascular procedures before performing them in the actual patient is currently being evaluated. Imagine a patient with a symptomatic CAS, challenging anatomy, and high anesthetic risk. The software “PROcedure Rehearsal Studio™” (Simbionix, Cleveland, Ohio, USA) rapidly loads the patient’s CT scan data from a CD onto the simulator and generates a digital three-dimensional model of the patient’s clinically relevant anatomy from the scan data; subsequently a simulated interventional environment is created. This enables interventionalists to try different approaches with a range of endovascular tools prior to treating the actual patient [110]. This technology is indeed exciting and may have an impact on health economics (reduction in operating and fluoroscopy time, number of tools reduced, cost of the procedure) and eventual outcomes for the patient.

The importance of teamwork in preventing medical error is well recognized [111, 112]. Future research aims to enhance nontechnical skills of both physicians and teams by simulator-based training. A virtual interventional suite allows the endovascular therapist and the interventional team (anesthetist, radiographers, theater nurses, and angiography suite nurses) to work in a realistic environment on simulated tissues. They can be exposed to complex or rare life-threatening events and learn how to manage crisis situations in a simulated interventional suite allowing feedback by knowledgeable instructors without exposing patients to risk [113, 114].

Conclusion

In the era of rapidly expanding technology, shorter vascular training paradigms, and ever-increasingly public scrutiny of surgical outcomes, the vascular and endovascular skills training and simulation center has been embraced for the training of the next generation of vascular specialists. Simulators are an exciting and necessary development in the training of vascular surgeons. Their use in training should be accompanied by a structured curriculum with competency assessment.