Graduate Medical Education

Abstract

The story of simulation-based training in medicine cannot be told without foundational chapters on the role of simulation training in graduate medical education in anesthesiology, the test kitchen for many simulation training pioneers. The ACGME Anesthesiology RRC formalized this role in 2011 by creating a mandate for simulation-based training in anesthesiology residency training. This mandate comes at the same time that the ACGME has charged anesthesiology residency training programs with developing methods to ensure that trainees are achieving attainment of specific competency-based milestones prior to completion of their training. For educators in graduate medical education in anesthesiology, this dual mandate provides an opportunity for curricular evolution as each mandate can mutually facilitate the other. The benefits of simulation-based medical education, particularly to the adult learner, can facilitate the goals of training for and formative and summative assessment of competency-based milestones. Furthermore, the milestone project provides educators with a practical outline in the development of content for a prospective simulation-based education program. The goals of a simulation-based curriculum can utilize these milestones to create a progressive curriculum modeled on Miller’s taxonomy of competency attainment, coupling targeted milestones and skills with appropriate simulation-based tools for graduate medical education and assessment. While each institution will face unique challenges in the integration of simulation-based training into their overall curriculum, multiple strategies can be employed leveraging the variety of simulation modalities (partial task trainers, high-fidelity simulation, OSCEs) and a variety of curricular formats. In developing the components of a simulation-based curriculum, educators must maintain focus on the purpose of each component, introductory training, formative assessment, or summative assessment, and tailor a curriculum that meets these goals taking into account the considerations and challenges unique to each of these goals.

Introduction

In 2011 the Anesthesiology Residency Review Committee (RRC) for the Accreditation Council for Graduate Medical Education (ACGME) announced a revision to the program requirements for anesthesiology residency training that represented a milestone in the history of role simulation based for anesthesiology education:

IV.A.6 Residents must participate in at least one simulated clinical experience each year [1].

The RRC further described requirements for incorporation of the six core competencies as defined by the ACGME along with a description by programs of the formal debriefing mechanisms utilized and the extent to which ancillary personnel are incorporated into the experience. However, the import of this announcement lies, to a large extent, in the factors leading to this announcement not enunciated by the Anesthesiology RRC or ACGME. The year preceding this announcement, the American Board of Anesthesiology (ABA) instituted a required simulation educational activity to satisfy one component of Maintenance of Certification in Anesthesiology (MOCA®) Part IV (Improvement in Medical Practice Component) [2]. These mandates represent a recognition by the accrediting and certifying bodies of the value simulation-based education and assessment can bring to the profession of anesthesiology.

The utilization of simulation-based technology in the profession of anesthesiology, and in anesthesiology residency training in particular, can be traced back to the utilization of “Sim One” at the University of Southern California in the 1960s [3]. Investigators, through the incorporation of the Sim One mannequin, sought to demonstrate an acceleration and enhancement of residents’ clinical performance through repetitive and deliberate practice in the simulated environment. In the 1980s, Gaba and colleagues utilized the CASE (Comprehensive Anesthesia Simulation Environment) to investigate anesthesiology resident trainee performance in the simulated environment, elucidating the utility of simulation technology for formative assessment in addition to education [4, 5, 6]. Soon thereafter, in the early 1990s, a team at the University of Florida developed the GAS (Gainesville Anesthesia Simulator) out of a goal to teach resident trainees basic skills in anesthesia practice [7, 8]. This work led to the development of a mannequin-based simulation technology incorporating physiologic modeling driven and drug recognition software which led to an advance in operator-driven manual control of physiologic output data in response to trainee actions.

Looking back on the development of high-fidelity mannequin-based simulation, it would be difficult to argue that the goal of training residents in the clinical practice of anesthesiology was not a critical impetus. In fact, for the last half century, one would not be far-off in characterizing simulation-based medical education (SBME) as the handmaiden of anesthesiology resident training. In recent years, educators have found an array of challenges effectively addressed through simulation-based education, assessment, and training. Simulation has provided a mechanism for the transfer of valuable skills from other high-stakes industries such as the airline industry such as Crew Resource Management (CRM) to anesthesia trainees. In fact, since the 1990s, there has been a great deal of literature in the instruction of anesthesiology residents in anesthesia crisis resource management (ACRM) skills [9] or other non-technical skill sets such as the Anaesthetists’ Non-Technical Skills (ANTS) system (task management, teamworking, situation awareness, and decision-making) [10]. Educators have utilized the simulated clinical environment in order to provide exposure to rare critical events for which clinical exposure during the standard period of training is unlikely [11], a technique which has become a mainstay of simulation education for resident trainees. Simulation training has even been used as a mechanism for the reintroduction of the impaired anesthesia trainee to clinical training and eventual practice [12].

This chapter is intended to provide a practical guide to the development of simulation curricula for a general graduate anesthesiology training program. While a curriculum for basic anesthesia skills and rare and critical scenarios aimed at the introduction of subspecialty techniques and clinical management and can be found in the chapters on undergraduate medical education (Chap. 13) and the subspecialties of anesthesiology (Chaps. 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, and 27), respectively, here focus will be placed on the development of simulation curricula designed to provide a mechanism for the progressive acquisition of skills in clinical anesthesiology practice that reflects the competency-based milestones approach to anesthesiology training as mandated by the ACGME. Toward that end, this chapter will provide a review of simulation-based curriculum development for anesthesiology training followed by a description of practical considerations for the department seeking to integrate simulation as a component of their graduate anesthesiology training program. Three general categories of curricula will be covered to reflect the progressive nature of graduate anesthesiology education: simulation-based education for an introduction to anesthesiology curriculum, simulation-based formative assessment for training-level specific competency-based milestone attainment, and simulation-based summative assessment for advancement based on competency-based milestone attainment.

General Considerations for Simulation-Based Curriculum Development

Simulation can serve as a valuable component of the broader residency curriculum, and the specific role it will serve is best elucidated a posteriori in the process of global residency curriculum design. This reflects the historical development of SBME in tandem with an attempt to meet the needs of educating anesthesiology resident trainees. The educator developing a simulation curriculum must maintain a focus on the unique benefits of SBME compared to more traditional methods of education or assessment to identify the content best served by this tool and to most effectively apply SBME to the content identified. This focus will ensure the creation of the most appropriate goals and objectives for the simulation-based curriculum as the first step in an effective curriculum design, construction, and execution. Many leaders in residency education and educators in general have looked upon the promise of this tool with a sense of great opportunity only to be humbled when encountering the resource-intensive nature of SBME. A recognition of these limitations and departmental resources must also be accounted for when contemplating the integration of a simulation-based curriculum, and multiple strategies have been described to aid in those creating curricula de novo. However, the first step in the process of curriculum development is the incorporation of the benefits of SBME to the adult learner in anesthesia.

Learning Theory and Curricular Content Selection

The Adult Learner

While an extensive review of the role adult learning theory plays in SBME is beyond the scope of this chapter, a brief recapitulation is necessary to highlight the relevant concepts to the topic of graduate anesthesiology education for the educator developing a simulation-based curriculum for education or assessment. The development of the CASE and GAS simulation technologies developed for anesthesiology residency education coincided with the description by Knowles of a necessary shift from pedagogy to andragogy or the art and science of adult learning [13]. This is fortuitous given Knowles’ recognition of the need for educators to address four principles he had identified to effective adult learning: an understanding of the rationale behind the learning, experiential learning allowing for mistakes, relevance of the material to their professional or personal life, and problem-centeredness, as opposed to content-centeredness. These authors propose inscribing these four principles into residency simulation curriculum development through the following four principles for simulation curriculum development:

• 1. Provide learners with the goals and objectives for the simulation-based learning exercise or module in relation to anesthesiology residency competency-based milestones.

• 2. Maximize the experiential and active components of SBME and minimize the potential for each exercise to devolve into didactics. Debriefing should also learner reflection and minimize educator instruction.

• 3. Ensure that learners are confronted with simulation-based exercises or modules relevant to their training level and/or subspecialty rotation.

• 4. Focus on competencies over “keywords” in developing curricular content, or preferentially utilize SBME to develop competencies as opposed to medical knowledge.

Learning Styles

Additional concepts have been studied in an effort to focus on learner styles and optimize curricular development, VARK typology being one of the more relevant to SBME [14, 15]. This system categorizes learning styles into four discrete categories: visual, auditory, reading, and kinesthetic or learning through carrying out physical categories [16]. Work has been done to identify the predominant learning style identified by adult students in medical education, and while a mixed style predominates, the kinesthetic style represents the most common unimodal preference, and teaching strategies employing a kinesthetic approach were the most preferred [17]. The obvious application of this insight to the development of a simulation curriculum for graduate anesthesiology training would be the maximization of “hot seat” exposure for each trainee when utilizing high-fidelity mannequin-based scenarios (serving the role of primary anesthesia care provider in the scenario). This principle is reflected in the requirement for MOCA Part 4 course ensuring that each participant is placed in “hot seat” at least once during each course [18]. Utilization of partial task trainers as part of high-fidelity scenarios or at discrete “stations” would be an additional strategy for maximizing kinesthetic learning .

Competency Attainment

In determining the content for a learning objective met through a simulation-based curriculum, the educator must identify the function this component will serve within a broader curricular context. In matching design to goals, be they educational, formative assessment, or summative assessment, Miller’s taxonomy of clinical competence can provide guidance in matching content and structure to goals [19]. This concept envisions a pyramid of clinical competence describing progression from “novice” to “expert,” the base of which is described as “knows,” or knowledge, underlying “knows how” or competence, followed by “shows how” or performance, and capped by “does” or action. Simulation can serve as an ideal format for facilitating progression in competency attainment, and the design of any curricular component would ideally reflect the trainee’s dynamic position in this progression. This chapter will reflect this principle in its presentation of differing modes of simulation-based curricula for graduate residency education as presented in Table 14.1.

Table 14.1 Application of Miller’s taxonomy to specific components of a simulation-based graduate anesthesiology curriculum

Kneebone described the inherent benefits of SBME in four parts: (1) the ability to tailor the training to the needs of the learner, (2) the provision of a safe environment in which the learner is permitted to fail, (3) the ability to provide objective evidence of performance, and (4) the capacity to provide immediate feedback [20]. The educator will emphasize certain of these SBME benefits over others in recognition of the learners’ expected progression through Miller’s taxonomy, and this emphasis should manifest in a very specific manner through curriculum design. For example, during an introduction to anesthesia curriculum for junior residents at the start of the academic year, scenarios may be less structured to allow learners to experience the consequence of certain actions, or inaction, as a scenario progresses . Debriefing may be interposed at intervals throughout a single scenario to highlight relevant concepts. Partial task trainers may be utilized in isolation to introduce learners to new technical skills. A curriculum designed to complement progressive milestone attainment for the non-novice may utilize more tightly scripted scenarios to allow for more objective measure of trainee performance with an emphasis on reflection during the debriefing process. Partial task trainers may be incorporated into large-scale mannequin-based scenarios to incorporate technical skills into the simulated clinical experience. Finally, when employing simulation for summative assessment, great emphasis must be placed on standardization of content as well as scoring tools to ensure a valid tool for potentially high-stakes assessment .

Clinical Competency in Anesthesiology Training

Miller’s taxonomy is reflected in the recent development of competency-based milestones which represent the result of an ongoing effort on the part of the ACGME to ensure the ability of graduating residents to provide patient care and work effectively in the healthcare system. A brief background on the genesis of this framework can help to identify the role SBME can play in competency attainment and assessment to help anesthesiology residency programs fulfill the mandate set for them by the ACGME.

In 1998, the ACGME initiated the Outcome Project to provide graduate medical training programs with guidance in the competencies expected of trainees by the conclusion of their graduate medical training [21]. This initiative also made accreditation contingent on programs’ trainees’ ability to demonstrate the educational outcomes identified through this project: the six core competencies are patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice. While the ACGME provided residency training programs with more specific components within each competency domain, however, these sub-competencies were general or not specific to any specialty training.

In 2009, the ACGME reached the culmination of the Outcomes Project through the introduction of a mandate for specialty-specific competency-based milestones [22]. Milestones are ultimately described within the broader context of the previously defined six core competencies. Within each competency domain, specialty-specific sub-competencies are described, and five progressive levels of competency demonstration utilized as the framework in which specific milestones provide specific performance descriptions for progressive sub-competency attainment within each core competency domain (Fig. 14.1). An example of this framework is presented in Fig. 14.1. Milestones for anesthesiology were introduced in 2014 and like milestones for other subspecialties are intended to provide benefits on three levels [23, 24]:

• Accreditation : Milestones serve as an objective metric for program evaluation, provide accountability to the public of training standards, and provide a context for further research and improvement of training standards.

• Graduate Medical Training Programs : Milestones provide a framework for clinical competency committees, guide curriculum development and assessment, and provide a mechanism for early identification of trainee struggle.

• Graduate Medical Trainees : Milestones provide explicit expectations, assist in self-directed learning, and provide a context for effective feedback.

Fig. 14.1

Example of milestones stratified within levels in a single sub-competency. (From the Anesthesia Milestone Project, December 2013. Copyright (c) 2013 The Accreditation Council for Graduate Medical Education and The American Board of Anesthesiology; reproduced with permission)

The benefits intended by the ACGME overlap to a large extent with those provided by SBME. An effective simulation curriculum for graduate anesthesiology education would ideally identify those sub-competencies and milestones best addressed through the experiential and kinesthetic aspects of simulation-based training while tailoring each session to training level and associated expectations for relevant milestone attainment. Furthermore, an SBME curriculum can be effectively guided by the milestones framework while also providing a tool for assessment, both formative and summative, through milestones performance within the simulated environment.

Challenges in SBME Curriculum Implementation

The clinical educator seeking to integrate SBME into the overall graduate anesthesiology training program quickly encounters the challenges inherent in this mode of instruction. In recent surveys of medical schools and anesthesiology training programs, just over half of medical student anesthesiology rotations incorporated SBME, while 55% of anesthesiology residencies incorporated SBME into intern training, 83% as part of clinical anesthesia year 1 (CA-1) training, and 96% as part of one or more rotations during residency [25, 26]. The main barriers to universal implementation cited in these surveys were time, financial, and human resources. To ensure the success of a curriculum for graduate anesthesiology education, these limitations must be taken into account and institutional resources must be realistically assessed and incorporated into the specific design of SBME implementation. While some strategies to overcome these obstacles will be presented here and in other section of this text, there are specific issues which must be addressed by each department seeking to integrate an SBME curriculum into their residency.

Simulation Facilities and Staffing

Those simulation centers endorsed by the ASA for ABA MOCA Part 4 simulation courses represent the spectrum of strategies through which anesthesiologist/educators have developed and implemented high-quality SBME curricula. On one end of this spectrum lie simulation centers housed within anesthesia departments fully staffed by faculty who provide the manpower for curriculum design, scenario development and scripting, hardware and software operation, confederate actors in immersive scenarios, debriefing, and even administrative tasks such as scheduling. This arrangement, while providing priority to the SBME initiatives of the anesthesiology department, depends a great deal on anesthesia faculty with heavy clinical workloads who demand relatively more financial resources for their time than non-physicians. Alternatively, many anesthesiology simulation programs interface with an institutional simulation center operated independently by their associated medical school or hospital. These centers often provide simulation technicians, administrative staffing, and a variety of other resources not available to anesthesia department-based simulation centers. This can drastically reduce the demands placed on the anesthesiologist/educator, but the SBME priorities of an anesthesiology residency program will be balanced against those of other departments and potentially an associated medical school. Many of the curricula presented in this chapter have been successfully implemented in both settings by making necessary modifications to staffing, scheduling, and format. A curriculum will be presented that was developed as a solution for those programs for whom no intra-institutional simulation center was available as well.

Simulation Hardware and Software

A variety of mannequin-based simulators , operating software, and partial task trainers exist representing a good deal of variability in cost and capability. The two most widely used models for mannequin-based simulation are the SimMan® manufactured by Laerdal Medical and the Human Patient Simulator (HPS®) manufactured by CAE Healthcare. SimMan® provides operator-dependent vital signs output coupled with a number of physiologically accurate cardiopulmonary findings on physical examination and allows basic airway management and effective fidelity for advanced cardiac life support coupled with a lower cost (around $60,000 for the popular SimMan® 3G model) and a low barrier or entry for educators intending to operate the software. The HPS® provides similar opportunities to the trainee for airway management and ACLS while incorporating physiologic modeling and direct interface between the mannequin and the anesthesiology workstation providing greater fidelity for the anesthesiology trainee. The HPS® couples these large advantages in fidelity and physiology with a much higher barrier to entry for the instructor intending to operate the software and a higher cost (over $200,000). Both Laerdal Medical and CAE Healthcare provide a variety of other models to represent the pediatric, neonatal, parturient, and other patients.

Educators looking to utilize scenarios and scripts created at one institution must consider that major revisions in design or implementation may be necessary if transferring from one model to another. Furthermore, certain goals of a simulation-based exercise for anesthesiology residents may be constrained based on the model utilized, as the practice of anesthesiology arguably places greater demands on a high degree of fidelity in SBME than the majority of medical practice.

Partial task trainers can greatly enhance a simulation curriculum through freestanding use for acquisition, practice, and assessment of technical skills or through incorporation into a mannequin-based immersive scenario. These can range greatly in cost and fidelity and would also be necessary for the educator looking to incorporate many essential skills in anesthesia training into an SBME such as neuraxial techniques, advanced airway techniques such as realistic lung isolation or bronchoscopy, central venous or arterial access, regional anesthetic techniques, ultrasound-guided needle manipulation, and echocardiography. More details on these products can be found in the chapters describing subspecialties of anesthesia in this text (Chaps. 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, and 27) as well as in the chapters on mannequin-based simulators and part-task trainers (Chap. 11).

A variety of other tools exist for SBME that can provide solutions for departments with more limited resources and include options such as standardized patient encounters and screen-based simulation products such as the Virtual Anesthesia Machine, a web-based interactive application run on the Adobe Shockwave Player [27]. Serious gaming may provide a future tool for educators, but this technology is currently in its infancy [28].

Simulation Faculty Expertise

More critical than human resource considerations such as simulator operation, administration of a simulation center, and confederate roles in scenarios, all of which were mentioned previously in consideration of simulation facilities and staffing, effective simulation educators require a great deal of training and experience. A successful SBME program for graduate anesthesiology training is not possible without trained simulation educators on faculty within the department providing the instruction. A number of simulation fellowship training programs exist to train simulation faculty in a variety of subspecialties [29]. Furthermore, a large number of programs for the training of current anesthesiology faculty are offered. The existence of these training programs reflects the complexity and skill involved in SBME curricular development and effective debriefing and the necessity of these skills for the success of a simulation program.

Time

The most widely reported challenge educators face in realizing their aspirations for a simulation curriculum is time. For an anesthesia residency training program in which the simulation educators also serve as clinical anesthesiologists and the simulation students also serve as trainee anesthesiologists capable of providing patient care under supervision, time spent in the simulation lab is time spent away from the provision of anesthetic care the department is tasked with. Given the revenue producing function of healthcare provision in modern medicine, this time can also be quantified monetarily, or in other words, time spent in the simulation lab costs the department an equal measure of time which those faculty or residents could be performing clinical duties and producing income for the department. To account for this additional demand on faculty and trainee time, a department must be sufficiently staffed to meet the educational mission in addition to the clinical functions of running an academic department. The challenge in provisioning time for simulation education not as commonly met with a traditional didactic session stems as the result of a feature of simulation commonly perceived as a pedagogical benefit. The small group format standardly utilized to facilitate kinesthetic learning, “hands-on” experience, and “hot seat” or primary management of the simulated clinical event is extremely inefficient from the point of view of one attempting to provide some targeted content to the largest possible audience over the shortest possible duration. The potential efficiency of didactic instruction is flexible and undeniable. Morning or afternoon sessions prior to or following scheduled anesthetic care provision are an extremely common strategy across graduate anesthesiology training programs for this reason.

While no formal systematic review has been conducted to assess strategies for scheduling simulation activities in graduate anesthesiology training, it is important to recognize that as one increases the efficiency of exposure (increasing the number of learners within a single session) one necessarily loses some of the benefits of this tool. A short, non-exhaustive list of some commonly observed strategies are listed in Table 14.2.

Table 14.2 Commonly utilized strategies for SBME scheduling in a graduate anesthesiology training program with relevant considerations and disadvantages

Introduction to Anesthesiology

Simulation is utilized across many specialties as a tool for the transition from medical school to residency training with many curricula described [30,31,32]. Despite the prevalence of simulation training in anesthesiology, a relative deficit of literature exists describing specific curricula designed for this purpose. A recent survey of anesthesia residency training programs revealed that 83% utilized SBME as part of orientation for junior anesthesiology residents [26], revealing high degree of penetrance. Therefore, this section will focus on considerations for providing the most effective introduction to anesthesiology curriculum and provide a sample curriculum.

Goals for the Junior Learner

The integration of SBME into a graduate anesthesiology training program’s introductory curriculum must reflect a recognition of learners’ novice level of clinical competency attainment, potential unfamiliarity with the anesthetic and perioperative environment, and a relative deficit of the medical knowledge relevant to anesthesia practice. In Miller’s model, these learners find themselves at the starting point of the transition from “knows” to “knows how.” Goals of an introductory simulation-based curriculum would focus on those novice competency domains or introductory principles best addressed through SBME’s experiential, kinesthetic approach compared to other methods of pedagogy. General goals and learning objectives effectively addressed as part of an SBME introduction to anesthesiology curriculum include:

• Gain familiarity with the operating room environment and anesthesia equipment.

• Display an understanding of the workflow of a general anesthetic.

• Apply relevant medical knowledge to basic anesthesia care.

• Perform basic airway management of the anesthetized patient.

• Participate in institutional safety initiatives such as screening relevant patients for pregnancy, universal time-out, and postoperative debriefing.

Multiple level 1 milestones of patient care can be introduced and practiced effectively in the simulated environment as a component of the curriculum, some examples of which are presented in Table 14.3 [23]. In some instance, level 2 milestones may be relevant to the novice learner as well.

Table 14.3 Level 1 milestones appropriate for integration into an introduction to anesthesiology simulation curriculum

General Strategies for an Introduction to Anesthesiology Curriculum

The characteristics of the novice learner and goals consisting of an introduction to multiple aspects of clinical anesthesia care, institutional practices, and applied medical knowledge impose a structure and form on SBME for an introductory curriculum that will differ from a standard SBME curriculum for anesthesiology residency. While no ideal format has been identified, these authors suggest integrating the following strategies for bringing learner characteristics in line with educational goals as part of the overall introductory curriculum:

• Open-ended scenarios : Allowing for a multitude of interval or final outcomes serves multiple purposes for the novice learner. Given lack of familiarity and experience, learner actions will be extremely unpredictable and difficult to script. This provides the educator with an opportunity to allow learners to experience the potential consequences of certain actions or omissions as they occur in a safe environment. Examples include patient desaturation in response to omission of manual or controlled ventilation following intubation, evidence of awareness with omission of anesthesia administration following intubation, or even evidence of anaphylaxis following antibiotic administration if patient allergies were not assessed during the preoperative assessment. These techniques may be more appropriate for scenarios involving an introduction to basic induction or emergence.

• Deliberate practice through scenario rewind: In the event of learner error, following manifestation of clinical consequences (e.g., unrecognized esophageal intubation leading to patient hypoxia and instability), return to the point in the scenario directly preceding the error allowing appropriate management and the ability to proceed through additional learning objectives.

• Integrated debriefing or scenario pauses: The novice anesthesiology resident is tasked with acquiring the requisite knowledge in parallel with its application, and the simulated clinical environment can provide an ideal forum for this process. Some examples of this practice include pausing the scenario: while patient monitors are placed to assess trainee knowledge of their utility and function, during preoxygenation to assess trainee knowledge of the principles of de-nitrogenation and patient positioning for ventilation and intubation, or after intubation to assess trainee knowledge of evidence successful intubation and adequate ventilation. During sessions on management of common complications such as hypoxia or hypotension, an in-depth discussion of cardiopulmonary concepts can be introduced at clinically relevant portions of the scenario, integrating traditional didactic instruction with experiential simulation-based learning.

• Focus less on specific pathology and more on general principles: While the educator may envision a specific pathophysiology to achieve the desired clinical presentation, the purpose of the clinical scenario highlights a general diagnostic and management approach as opposed to identification and management of a rare and critical event. For example, a perioperative myocardial infarction should be utilized as a strategy to provide an example of hypotension due to decreased cardiac output as part of a discussion basic cardiovascular physiology and should not be used for formative assessment of clinical management.

As with any group, an introduction to learning in the simulated environment at the outset of simulation-based instruction will ensure maximal utility of SBME for the novice learner. Given the role simulation will play throughout their training, an effective introduction is essential and should include an introduction to the simulation hardware and environment, a discussion of the safe learning space, and a general discussion of the general goals of SBME in residency training.

An explanation of the importance of “hot seat” learning, a delineation of the boundaries of the simulated and real world, and instructions for calling for help should also be provided.

A Sample “Introduction to Anesthesiology” Simulation-Based Curriculum

The following curriculum is adapted from the “Introduction to Clinical Anesthesia” simulation-based course first developed at the Human Emulation, Education, and Evaluation Lab for Patient Safety and Professional Study at the Icahn School of Medicine at Mount Sinai Hospital [33]. An adapted form of this course is currently being utilized by multiple anesthesiology training programs including the Department of Anesthesiology at the Wexner Medical Center at the Ohio State University as well as the Department of Anesthesiology, Perioperative, and Pain Medicine at Mount Sinai St. Luke’s and West Hospitals. This curriculum incorporates many of the principles and strategies described above based on decades of experience in instruction and multiple revisions to format and content. The complete course consists of five sessions over the first month of training. Each session is 2 hours in duration, consists of three scenarios, and is intended to involve 3–4 CA-1 resident learners with 2–3 faculty or senior resident educators. All scenarios will have roles including the patient, circulating nurse, and surgeon. This curriculum is intended to provide a progression from basic to more advanced topics relevant to the junior anesthesia resident.

Sessions 14.1 and 14.2 are an introduction to the basic practice of clinical anesthesia. Scenario pauses allow the learner to review knowledge to apply to recent or upcoming actions, simulation rewinds following learner errors allow a demonstration of consequences, and open-ended scenarios allow for wider variation in learner performance to take advantage of unanticipated learning opportunities. Each scenario will vary subtly in order that each participant can meet the learning objectives, and the instructor should draw upon the list of relevant content for assessment of relevant knowledge during pauses in the scenario.

Session 14.1. Preoperative Evaluation and Induction of General Anesthesia

Learning objectives	Perform a preoperative evaluation and physical examination. Obtain informed consent. Obtain appropriate preoperative testing. Perform a preoperative anesthesia machine check and room preparation. Properly apply external patient monitors. Participate in universal protocol or “time-out” Provide appropriate premedication. Perform airway management including preoxygenation, manual ventilation, intubation, and controlled ventilation. Administer appropriate medications for induction and paralysis. Initiate a balanced anesthetic.
Relevant knowledge	Components of a basic patient history Mallampati classification Indications for preoperative testing Interpretation of data from external patient monitors and how each monitor functions Reasoning behind preoxygenation and proper performance Doses, concentration, indication, and mechanism of action for medications administered Appropriate position for ventilation/intubation Types of laryngoscopes and relation to airway anatomy Laryngoscopy: Anatomy, views, and grading Modes of ventilation and ventilator settings Characteristics of volatile anesthetics
Scenario 1	Healthy young male requires general anesthesia for scheduled laparoscopic cholecystectomy.
Case stem:	Our patient is a 30-year-old male undergoing scheduled laparoscopic cholecystectomy. His past medical history is significant for biliary colic. He reports no other past medical history. He takes no medications. He has experienced throat swelling from cephalexin. He has had no prior surgeries or anesthetic care.
Physical examination:	Healthy, non-obese male, normal physical examination, Mallampati 1 airway. 20-gauge antecubital intravenous line in place.
Scenario details:	Learners are expected to perform learning objectives described above. Learner actions will determine scenario progression (learner errors will lead to appropriate consequence), and a scenario rewind can take place following an error to allow learners a chance to make appropriate interventions and proceed to scenario completion when appropriate (scenario is terminated following mechanical ventilation of the patient and initiation of a balanced anesthetic).
Scenario 2	Healthy young woman requires general anesthesia for scheduled breast augmentation.
Case stem:	Our patient is a 25-year-old female undergoing a breast augmentation. Her past medical history is negative. She takes no medications. She has no allergies to medications. She reports an appendectomy as a child with no complications from anesthesia.
Physical examination:	Healthy, non-obese female, normal physical examination, Mallampati 3 airway. 20-gauge antecubital intravenous line in place.
Scenario details:	Learners are expected to perform learning objectives described above. Manual ventilation will require insertion of an oral airway. Highlight the requirement of a preoperative pregnancy screen (if omitted, have circulating nurse discover omission following intubation). Learner actions will determine scenario progression (learner errors will lead to appropriate consequence), and a scenario rewind can take place following an error to allow learners a chance to make appropriate interventions and proceed to scenario completion when appropriate (scenario is terminated following mechanical ventilation of the patient and initiation of a balanced anesthetic).
Scenario 3	Healthy young male requires general anesthesia for emergent exploratory laparotomy .
Case stem:	Our patient is a 31-year-old male undergoing an exploratory laparotomy. His past medical history is negative. He takes no medications. He has no allergies to medications. He reports no prior surgeries. He was stabbed in the abdomen during an argument while enjoying a pizza dinner. He is currently hemodynamically stable, but CT shows evidence of bowel perforation.
Physical examination:	Healthy, non-obese male, clear abdominal wound, rebound tenderness, and guarding. Normal cardiopulmonary physical examination findings. Mallampati 1 airway. 16-gauge antecubital intravenous line in place.
Scenario details:	Learners are expected to perform learning objectives described above. Rapid sequence intubation is required; omission will lead to regurgitation and aspiration. Learner actions will determine scenario progression (learner errors will lead to appropriate consequence), and a scenario rewind can take place following an error to allow learners a chance to make appropriate interventions and proceed to scenario completion when appropriate (scenario is terminated following mechanical ventilation of the patient and initiation of a balanced anesthetic).

Session 14.2. Emergence from General Anesthesia

Learning objectives	Determine extent of neuromuscular blockade. Perform reversal of neuromuscular blockade. Provide for adequate postoperative analgesia. Initiate and accelerate elimination of volatile anesthetic. Provide nausea and emesis prophylaxis. Identify and review extubation criteria. Identify stage 2 anesthetic plane during emergence. Perform oropharyngeal suction prior to extubation. Identify an appropriate patient postoperative disposition.
Relevant knowledge	Dosing and duration action of opioid agents Opioid alternatives for postoperative analgesia Analysis of train-of-four neuromuscular stimulation Relative effects of neuromuscular blockade on different muscle groups Characteristics of phase 1 vs. phase 2 blockade Sites for peripheral nerve stimulation Dosage, concentration, and mechanism of action of neuromuscular blockade reversal agents Factors that accelerate clearance and elimination of volatile anesthetic Risks of hypercarbia and hypoventilation Stages of anesthesia Antiemetic agent dosing and mechanisms of action Strategies for aspiration prevention Strategies for prevention of negative pressure pulmonary edema Causes of delayed emergence
Scenario 1	Healthy young female requires general anesthesia for scheduled laparoscopic cholecystectomy .
Case stem:	Our patient is a 40-year-old female undergoing scheduled laparoscopic cholecystectomy. Her past medical history is significant for cholelithiasis. She reports no other past medical history. She takes no medications. She has experienced throat swelling from cephalexin. She has had no prior surgeries or anesthetic care. The surgeon reports 10 minutes are remaining for skin closure.
Physical examination:	Healthy, non-obese female, normal physical examination, intubated under general anesthesia. Airway. 20-gauge antecubital intravenous line in place.
Scenario details:	Learner actions will determine scenario progression (learner errors will lead to appropriate consequence), and a scenario rewind can take place following an error to allow learners a chance to make appropriate interventions and proceed to scenario completion when appropriate (scenario is terminated following patient extubation and stabilization).
Scenario 2	Obese woman undergoing gastric bypass for treatment of obesity
Case stem:	Our patient is a 38-year-old female undergoing gastric bypass surgery. Her past medical history is positive for morbid obesity, OSA, DM type II. She takes metformin, glargine insulin, and tramadol. She is supposed to wear a CPAP at night, but is noncompliant. She has no allergies to medications. She had an uncomplicated cholecystectomy 5 years ago and a C-sect. 2 years ago. The surgeon reports 10 minutes are remaining for skin closure.
Physical examination:	Morbidly obese female, intubated, under general anesthesia with clear bilateral breath sounds. The patient has an 18-gauge antecubital intravenous line in place.
Scenario details:	Learners are expected to perform learning objectives described above. Emphasis will be placed on issues of hypoventilation and obesity, the impact of obesity on volatile anesthetic elimination. The patient will undergo a significant period of stage 2 emergence, and extubation criteria should be emphasized. Learner actions will determine scenario progression (learner errors will lead to appropriate consequence), and a scenario rewind can take place following an error to allow learners a chance to make appropriate interventions and proceed to scenario completion when appropriate (scenario is terminated following successful patient extubation and stabilization).
Scenario 3	Delayed emergence in the setting of multiple risk factors
Case stem:	Our patient is a 73-year-old female undergoing cholecystectomy. Her past medical history is positive for type I diabetes, fibromyalgia, generalized anxiety disorder, and CRPS of the right lower extremity related to previous trauma. She takes gabapentin as well as valium and percocet on an as-needed basis in addition to insulin and atorvastatin. She states she’s allergic to morphine. She had an ORIF for a broken ankle 5 years ago complicated by severe postoperative nausea requiring an overnight stay. The patient has received a larger than normal dose of opioids in anticipation of higher requirements due to chronic pain therapy and a larger dose of benzodiazepine premedication due to anxiety. The surgeon reports 10 minutes are remaining for skin closure.
Physical examination:	Thin female, intubated, and under general anesthesia. Normal cardiopulmonary physical examination findings. 18-gauge antecubital intravenous line in place.
Scenario details:	Learners are expected to perform learning objectives described above. Delayed emergence will occur and following elimination of volatile anesthetic and reversal of neuromuscular blockade, learners are expected to contemplate reversal of opioids and benzodiazepines. Instructor may choose to allow for hypoglycemia discovered on laboratory analysis or an intracranial event to be the culprit. A full review of a delayed emergence management algorithm will be explored (scenario is terminated following either successful extubation or once the decision is made to obtain intracranial imaging).

Sessions 14.3 and 14.4 serve to provide an early conceptual framework for understanding pulmonary and cardiac physiology through the management of hypoxia and hypotension, respectively. These sessions utilize a “hybrid” format of immersive simulation in which planned “pauses” are utilized to reflect upon the clinical condition of the simulated patient and the learner’s interpretation and management to clinical findings. These interval pauses are exploited to incrementally introduce a conceptual framework through application of the relevant principles of physiology to the clinical scenario. This curriculum integrates accessory materials such as displays of equations and diagrams during these interval scenario pauses. The session descriptions that follow will make note of these pauses and the content applicable to each one.

Session 14.3. Hypoxia

Learning objectives	Perform a differential diagnosis for intraoperative hypoxemia. Identify and correct a mainstem endotracheal intubation. Identify and treat intraoperative hypoxemia due to pulmonary alveolar atelectasis. Appropriately communicate the presence of patient hypoxemia with operating room staff. Identify an oxygen supply failure and utilize an alternative method of patient oxygenation. Provide a differential diagnosis for hypoxemia with a concomitant reduction in expired carbon dioxide concentration. Manage intraoperative pulmonary air embolism.
Medical knowledge objectives	Define oxygenation, ventilation, and respiration. Identify all lung volumes and capacities. Understand the determinants of functional residual capacity (FRC). Calculate the rate of oxygen consumption for a 70 kg patient. Describe the alveolar gas equation. Develop and apply an algorithm for management of intraoperative hypoxemia. Describe shunt and dead space physiology at the alveolar level. Understand pulmonary perfusion and ventilation. Describe the pulmonary west zones according to arterial, venous, and alveolar pressure relationships. Describe the relationship between FRC and closing capacity. Recognize the utility of continuous or bi-level positive airway pressure. Calculate and estimate an alveolar to arterial oxygen tension gradient.
Scenario 1	Hypoxia during robotic laparoscopic hysterectomy
Case stem:	Our patient is a 45-year-old female undergoing a robotic hysterectomy. Her past medical history is significant for obesity, hypertension, uterine fibroids, and OSA (home CPAP). She reports no other past medical history. She takes hydrochlorothiazide. She has no known drug allergies. She has had no prior surgery or anesthetic.
Physical examination:	Obese female, intubated, under general anesthesia currently supine prior to incision. The patient has an 18-gauge antecubital intravenous line in place.
Scenario details:	Learner encounters the patient under general anesthesia and receives a report from the prior anesthesia care provider. An uncomplicated induction and intubation is reported, antibiotics have been administered, and neuromuscular blocking agents and loading dose of opioid analgesics have been administered. Pre-surgical “time-out” completed just prior to learner arrival. Prior care anesthesia provider leaves the operating room. Endotracheal tube depth should be surreptitiously deeper than appropriate.
Event 1:	Surgical port placement, insufflation, and placement of patient into steep Trendelenberg positioning lead to rapid desaturation (SpO2 mid-80s).
*Pause*	Define oxygenation, ventilation, respiration. Review lung volumes with an emphasis on FRC and its determinants. Calculate alveolar oxygen tension. Introduce an algorithm for hypoxia management  1. 100 FiO2  2. √ other vitals (ETCO2)  3. ETCO2 (+):   1. √ breath sounds/tube depth   2. √ inspired volume vs. expired volume   3. Diagnose cause of shunting  ETCO2 (−):   1. Switch off vent and attempt to hand bag    1. Can bag: Problem is in the ventilator    2. Cannot bag: Problem is proximal to the ventilator     Inspect ETT, elbow, circuit, etc.   2. If problem cannot be identified, utilize a self-inflating manual resuscitating bag with external oxygen or room air.
Event resolution	Learner should identify unilateral right-sided breath sounds, and resolution of hypoxia occurs with withdrawal of endotracheal tube to an appropriate depth.
Event 2:	The operator informs the learner that a fair deal of time has passed and lowers the oxygen saturation incrementally over the passage of time such that SpO2 settles to the mid-80s following a simulated passage of 1 hour of time.
*Pause*	Define shunt and dead space at the alveolar level. Identify the west zones and the relationship of arterial and venous pressures to alveolar pressure. Define anatomic and physiologic dead space. Discuss the relationship of closing capacity and FRC and the impact on arterial oxygenation.
Event resolution	Learner should identify the role of alveolar atelectasis and shunt in clinical hypoxemia, initiate an alveolar recruitment maneuver, and utilize positive end-expiratory pressure (PEEP) to increase patient FRC.
*Pause*	Discuss the role of CPAP and BIPAP in oxygenation and ventilation and their use in the perioperative setting. This will serve as the conclusion of the scenario.
Scenario 2	Loss of central oxygen supply intraoperatively
Case stem:	We continue with our patient, a 45-year-old female undergoing a robotic hysterectomy. Her past medical history is significant for obesity, hypertension, uterine fibroids, and OSA (home CPAP). She reports no other past medical history. She takes hydrochlorothiazide. She has no known drug allergies. She has had no prior surgery or anesthetic.
Physical examination:	Obese female, intubated, under general anesthesia currently supine prior to incision. The patient has an 18-gauge antecubital intravenous line in place. Trendelenberg position.
Scenario details:	Between scenarios, advise learners to take a brief “break” during which time the central oxygen supply will be disconnected and the rear oxygen tank will be removed from the operating room/simulation theater. Upon return from their break, have the scenario paused and the anesthesia machine shut down, and have the first participant provide “sign-out” to the incoming learner. Following sign-out, the anesthesia machine can be powered on and the scenario un-paused.
Event 1	Progressive hypoxemia will occur in the setting of an inability to ventilate or oxygenate due to loss of central oxygen supply.
*Pause*	Reintroduce the previously discussed hypoxia algorithm.
Event resolution	Using the hypoxia algorithm, the participant should utilize a self-inflating manual resuscitating bag and potentially an external (tank) source of oxygen.
*Pause*	Review the volume and pressure relationship of an “E” cylinder of oxygen. Review the role of pressure from central gas supply to drive positive pressure ventilation with many models of anesthesia workstations. Discuss the impact of an anesthesia machine mounted oxygen “E” cylinder. Scenario terminates following this discussion
Scenario 3	Intraoperative pulmonary embolism
Case stem:	We continue with our patient, a 45-year-old female undergoing a robotic hysterectomy. Her past medical history is significant for obesity, hypertension, uterine fibroids, and OSA (home CPAP). She reports no other past medical history. She takes hydrochlorothiazide. She has no known drug allergies. She has had no prior surgery or anesthetic.
Physical examination:	Obese female, intubated, under general anesthesia currently supine prior to incision. The patient has an 18-gauge antecubital intravenous line in place. Trendelenberg position.
Scenario details:	This scenario will be a continuation of the prior two, now with operating oxygen supply restored. Learner 2 will provide a patient sign-out to learner 3. The surgeon informs the team of an hour or more remaining.
Event 1	Following a few minutes, the circulating nurse announces that the sequential compression devices had not been running and turns them on. Over the next few minutes, the patient will become progressively hypoxic, hypotensive, and the expired carbon dioxide tension will decrease.
*Pause*	Utilize the hypoxia algorithm. Calculate an alveolar to arterial oxygen gradient. Estimate arterial oxygen tension based on SpO2 and a normal oxyhemoglobin dissociation curve. Discuss dead space physiology and its impact on ventilation. Briefly discuss the management options for pulmonary embolism. Scenario is concluded after this discussion.

Session 14.4. Hypotension

Learning objectives	Perform a differential diagnosis for intraoperative hypotension. Identify the pathophysiology of hypotension due to myocardial ischemia. Treat demand ischemia in the perioperative setting. Identify and treat intraoperative hypotension due to hemorrhage. Identify and treat intraoperative hypotension due to distributive shock from sepsis.
Medical knowledge objectives	Relate Ohm’s law to the systemic and pulmonary circulation. Identify the components of stroke volume (preload, afterload, contractility). Describe the determinants of myocardial oxygen supply and demand. Review the role of the frank-Starling curve on stroke volume. Understand the role of decreased cardiac output on dead space ventilation and expired carbon dioxide tension.
Scenario 1	Postoperative demand ischemia
Case stem:	Our patient is a 65-year-old male status post open partial colectomy due to adenocarcinoma of the colon. His past medical history is significant for obesity, hypertension, and coronary artery disease with stents 2 years prior, obstructive sleep apnea on home CPAP, hyperlipidemia, and non-insulin-dependent diabetes mellitus. He has been off of clopidogrel for 8 days and takes aspirin, metoprolol, atorvastatin, metformin, and hydrochlorothiazide. He has no known drug allergies.
Physical examination:	Obese male, in obvious distress, tachycardic, hypertensive, diaphoretic, and dyspneic. He has an 18-gauge antecubital intravenous line in place. All other access has been removed.
Scenario details:	The learner is called to the postoperative care unit to evaluate a patient. The nurse informs our participant that the patient had received a colectomy and was ready for discharge to the floor. Monitors and all intravenous and arterial access lines have been removed but for a single peripheral venous line. The nurse informs the learner that the patient is complaining of unbearable pain at the surgical incision site. Patient information available from bundled chart if requested by participant (intraoperative management includes conservative use of analgesics out of concern for postoperative respiratory complications)
Event 1:	ECG (if connected) reveals sinus tachycardia with ST depressions in the anterolateral distribution. Patient initially hypertensive and complaining of severe pain at the surgical site and difficulty breathing. If a 12-lead ECG is available, have available an example of anterolateral ischemia and sinus tachycardia.
*Pause*	Discuss management of ischemia encouraging the participant to independently describe MONA-B management. Discuss the likely etiologies of perioperative ischemia and infarctions in the context of the intraoperative, immediate postoperative, and general postoperative time periods. Discuss the concept of myocardial supply and demand mismatch and the components of both myocardial oxygen supply and demand.
Event 2:	As the learner requests administration of medical and other interventions for cardiac ischemia, begin to lower the patient blood pressure to a low normal level (systolic pressure in the low 90’s) while the nurse confederate attempts to retrieve beta-blocker treatment (if requested) with the goal of forcing the participant to evaluate the etiology of decreased cardiac output in the setting of worsening demand ischemia.
*Pause*	Introduce Ohm’s law (V = IR) and relate the variables to those in the systemic cardiovascular circuit generally (MAP-CVP = CO x SVR). Have participants describe how to calculate cardiac output (SV x HR) and define the components of stroke volume (preload, afterload, and contractility). Identify the etiology of hypotension in the context of demand ischemia, and interrogate the rationale for the specific interventions represented in MONA-B treatment as they serve to optimize myocardial supply and demand and how this improves cardiac output utilizing the framework of Ohm’s law and the relevant components in the systemic cardiovascular circuit .
Event resolution	The learner should recognize the utility of medical rate control with vasopressor support while also considering the need for coronary intervention.
Scenario 2	Hypotension due to acute urosepsis
Case stem:	Our patient is a 37-year-old woman with a medical history relevant for multiple episodes of nephrolithiasis and ureteral obstruction requiring cystoscopy and stone removal. She has no other past medical or surgical history. She takes no medications and has no allergies.
Physical examination:	Thin female, tachypneic, tachycardic, febrile, with low normal blood pressure. Favorable airway examination.
Scenario details:	The learner is called to the emergency room to conduct a pre-anesthetic evaluation for an urgent cystoscopy and ureteral stone removal. Emergency room staff confederate will inform the learner that the patient has received two liters of crystalloid via an 18 G peripheral venous catheter, as well as large doses of parenteral hydromorphone for pain. Antibiotics ordered by the urology service have been administered as well. Upon request, laboratory findings revealing leukocytosis and a negative pregnancy test will be provided.
Event 1	Learner is provided time to meet and evaluate patient in the emergency room. Attending anesthesiologist confederate will “call” the learner to receive a report on the patient and request the learner’s anesthetic plan.
*Pause*	Utilize the conceptual framework of Ohm’s law and the systemic cardiac circuit, and evaluate the etiology of hypotension in the context of this patient’s history. Reserve an in-depth discussion of the role of tachycardia in compensating for intravascular depletion for later in this scenario.
Event 2	Arrival to operating room and induction of general anesthesia. Patient hemodynamic assessment reveals tachycardia and systolic pressure in the low 90s. The learner should be provided with the intravenous access, invasive or external blood pressuring monitoring they request. Learner should be allowed to perform the induction technique of their choosing. The goal of this portion of the scenario is to illustrate the effect of iatrogenic insult to the patient’s compensatory response to evolving sepsis. If a non-judicious induction plan is utilized, severe hypotension will result. If a judicious induction plan is utilized, the attending anesthesiologist confederate may choose to (inappropriately) administer an agent such as esmolol in response to increased tachycardia to induce severe hypotension to facilitate the forthcoming event pause discussion.
*Pause*	Revisit the patient condition in the setting of the framework of Ohm’s law, the role of preload, afterload, and contractility on stroke volume, and place emphasis on the role of heart rate in compensating for preload reduction and drop in systemic vascular resistance. This should serve to reinforce a systematic approach to perioperative hypotension performed by identifying within the concepts and formulas previously introduced. Learners should be encouraged to identify the “source” of derangement within the equations being utilized, i.e., SVR and/or cardiac output, if cardiac output, HR and/or stroke volume, and if stroke volume, preload, afterload, and/or contractility. Further discussion should involve the appropriate interventions to address the derangements.
Event 3 (optional)	Allow participant to employ interventions discussed during the previous event pause to stabilize hemodynamics to pre-induction conditions and continue to cystoscopy. Immediately following renal stone extraction, severe hypotension resistant to vasopressor support will ensue. This can allow for a post-scenario discussion of the dangers of urosepsis and the importance of preoperative and intraoperative resuscitation .
Scenario 3	Hypotension secondary to hemorrhage
Case stem:	Our patient is a 28-year-old female with no past medical or surgical history presenting for emergency laparoscopy for ruptured ectopic pregnancy. She takes no medications and has no allergies.
Physical examination:	Thin female, tachypneic, tachycardic, hypotensive, and afebrile with a favorable airway.
Scenario details:	Learner will encounter this patient just as they are brought to the operating room by the obstetrics and gynecology team. Two large-bore peripheral venous catheters are in place. Blood products are available if requested. Laboratory data obtained in the emergency room reveals a hematocrit of 36.
Event 1	Induction of anesthesia. Learner is allowed to develop a plan for induction and Maintenance of anesthesia.
*Pause*	Returning to the framework of Ohm’s law presented earlier, discuss the cause of hypotension, existing compensatory changes, the impact of induction and maintenance of general anesthesia, and relevant considerations to preserve tissue perfusion.
Event resolution	Stabilization of vital signs following utilization of vasopressors, fluid administration, and potentially blood products after induction of general anesthesia.
Event 2	After the start of the procedure, large hemoperitoneum is discovered via laparoscopy. Deterioration of hemodynamics occurs upon initiation of pneumoperitoneum. This will manifest as severe hypotension, tachycardia, and a pronounced drop in expired carbon dioxide.
*Pause*	Revisit the cause of hypotension within the conceptual framework utilized in this exercise. Bring attention to the reduction of expired carbon dioxide tension. Introduce the application of Ohm’s law as a model for the pulmonary circulatory circuit and correlate the relevant variables to those in the systemic circuit. Reintroduce the lung zones discussed during the preceding “hypoxia” session (ideally show diagrams previously utilized). Show the role played by cardiac output in the increase or decrease of zone 1 physiology and the impact on dead space ventilation that occurs as a result. Encourage the learner to connect this to the change in expired carbon dioxide tension in order to bring attention to the utility of capnography in detecting changes in cardiac output.
Event resolution	Administration of blood products (massive transfusion protocol) and aggressive vasopressor administration .

Sessions 14.5, “Arrhythmias,” serves to reinforce the principles from the “Hypotension” sessions while incorporating advanced cardiac life support algorithms for managing unstable tachycardias, bradycardia, and pulseless rhythms. This session will continue to utilize a hybrid model of immersive simulation with intermittent “pauses” for the introduction of concepts and algorithms. This allows learners to gain an introduction to management principles and algorithms within the context of the relevant pathology while also allowing for deliberate practice through a “pause, discuss, reset, re-attempt” approach to experiential learning.

Session 14.5. Arrhythmias

Learning objectives	Evaluate hemodynamic instability in the context of a cardiac dysrhythmia. Identify and manage dysrhythmias including unstable tachycardias, bradycardias, and pulseless rhythms. Apply knowledge of treatable causes of pulseless electrical activity. Appropriately communicate the presence of an unstable arrhythmia with operating room staff. Understand the interaction between arrhythmias and cardiac output as it impairs coronary and other tissue perfusion.
Medical knowledge objectives	Describe the American Heart Association (AHA) algorithm for management of unstable tachycardias. Describe the AHA algorithm for management of unstable bradycardia. Describe the AHA algorithm for management of pulseless rhythms.
Scenario 1	Management of symptomatic bradycardia
Case stem:	Our patient is a 35-year-old male presenting for surgical treatment of a ruptured globe. He has no past medical or surgical history, takes no medications, and has no allergies. He reports that his injury was the result of a mishap in his workshop occurring just after lunch.
Physical examination:	Healthy male with an obvious foreign body and injured left eye. Mildly tachypneic, tachycardic, hypertensive with an 18 G peripheral venous catheter in his right hand.
Scenario details:	Learner encounters the patient in the operating room in preparation for urgent repair of a ruptured globe. Additional assessment can be made at this time. The learner will develop and conduct an anesthetic plan independently. While there are many salient issues raised by this case including the utilization of a rapid sequence induction and the choice of paralytic agent, these issues are secondary to the main learning objectives.
Event 1:	Following sterile preparation, draping, and surgical “time-out,” surgical manipulation leads to mild bradycardia (high 40s) without resultant hypotension.
Event resolution	Prior to learner treatment (if intended), with cessation of surgical stimulation bradycardia resolves.
Event 2:	The surgeon confederate announces that further traction will be necessary to proceed with the procedure. This will result in severe bradycardia to 28 beats per minute with resultant hypotension. If the learner requests cessation of surgical traction, our confederate will comply (with resolution of bradycardia), however stating that further similar traction will be necessary to achieve their surgical goals.
*Pause*	Reintroduce the concept of Ohm’s law and the systemic cardiac circuit to explain the role of bradycardia in this instance hemodynamic instability. A diagram displaying the frank-Starling curve and the effects of increasing preload on stroke volume should be discussed as well.
Event 3:	With ongoing surgical manipulation, heart rate decreases to 21 beats per minute with severe hypotension. Requests by the learner to cease surgical manipulation are met with a response by the surgical confederate to “treat it or something, I have to retrieve this foreign body!”
*Pause*	Display the AHA bradycardia algorithm. Discuss the current scenario insofar as it is addressed by the bradycardia algorithm. Discuss alternative scenarios in which symptomatic bradycardia may occur and the rationale behind the guidelines.
Event resolution	Learner administers 0.5 milligrams of atropine .
Scenario 2	Management of unstable tachycardia
Case stem:	Our patient is a 34-year-old female presenting for scheduled liposuction and breast augmentation. Her past medical history is significant for anxiety. She takes alprazolam daily and reports no drug allergies.
Physical examination:	Mildly overweight, anxious female with a favorable airway and a 20 G peripheral venous catheter in the left arm.
Scenario details:	The learner will meet the patient in the preoperative area and conduct a preoperative evaluation and develop an anesthetic plan. Anesthetic induction should proceed without incident as will the initiation of the surgical procedure.
Event 1	“Fast forward” the scenario into the midst of liposuction (45 minutes into the procedure). The patient’s cardiac rhythm will progress from mild tachycardia to a rapid supraventricular rhythm with concomitant hypotension not responsive to phenylephrine.
*Pause*	Revisit the concept of systemic cardiac circuit and the role of heart rate and stroke volume on maintenance of cardiac output and mean arterial blood pressure.
Event 2	Allow the learner to attempt multiple interventions. The surgical team should initially discourage suggestions of cardioversion given then current surgical field preparations (however this would be the sole intervention leading to resolution).
*Pause*	Provide the AHA tachycardia (with pulse) algorithm and discuss the current patient condition in the context of the algorithm. Discuss alternative scenarios in which an unstable tachycardia may occur and the rationale behind the guidelines.
Event resolution	Learner utilizes synchronized cardioversion to treat the unstable tachycardia.
Scenario 3	Management of pulseless rhythms
Case stem:	Our patient is a 65-year-old male undergoing a laparoscopic right partial nephrectomy. His past medical history is significant for coronary artery disease with stent placement 5 years ago, hypertension, non-insulin-dependent diabetes, obesity, nephrolithiasis, and a right renal mass. He takes metoprolol, clopidogrel, aspirin, furosemide, metformin, and glyburide and has no known drug allergies. He had a left knee replacement 10 years ago without complications.
Physical examination:	Obese male, intubated, under general anesthesia currently in left lateral decubitus position. He has a left radial arterial line in place, and two 18 G peripheral venous lines (one in each hand).
Scenario details:	This scenario will begin intraoperatively with a hand-off from an anesthesia care provider confederate requiring relief for the end of their shift. The learner will receive a full report for transition of care including information that antiplatelet therapy has been discontinued for more than 1 week, but metoprolol was taken the morning of surgery. Preoperative hematocrit and glucose were 43 and 143, respectively, and a recent arterial sample revealed new values of 39 and 123. The procedure has been underway for an hour and has been without incident thus far.
Event 1	Following a few minutes, the patient monitor will reveal frequent premature ventricular contractions. If the learner inquires about the status of surgery, the surgical confederates will state that the procedure has progressed well and without complication.
*Pause*	Discuss the clinical significance, meaning, causes, and treatment of premature ventricular contractions. This is a good opportunity to define terms such as couplet, triplet, bigeminy, and trigeminy.
Event 2	Patient deteriorates over a few minutes into pulseless electrical activity (PEA). This will be unresponsive to vasopressors as it progresses.
*Pause*	Briefly discusses the definition of PEA drawing a comparison between electromechanical dissociation (EMD) PEA and non-EMD PEA and the clinical presentation.
Event 3	Rhythm progresses to ventricular tachycardia.
*Pause*	Present the AHA algorithm for pulseless cardiac arrest. Allow learner to review management in the context of patient status.
Event 4	Progress to ventricular fibrillation despite management.
Event resolution	Application of defibrillation. This will convert the rhythm to PEA.
*Pause*	Discuss the causes of PEA and potential management. Discuss the likely causes of arrest in this patient (cardiogenic shock secondary to myocardial infarction, embolism, etc.)

Simulation Training for Formative Assessment

SBME has already achieved a high degree of penetrance in graduate anesthesia education with a majority of ACGME-accredited programs utilizing this technology [26]. While debate exists regarding the wisdom and feasibility of utilizing SBME as a tool for assessing resident competency [34], survey data reveals that 79% of training programs have developed or plan to develop this tool as a means of assessing milestone attainment. While concerns about performance data from simulation-based training being utilized by clinical competency committees in resident assessment are well-founded and deserve further consideration in the training community, this does not preclude the use of milestones as means for development of an SBME curriculum for formative assessment. The simulated environment provides an ideal setting for allowing residents to “show” (using Miller’s taxonomy) attainment of specific milestones which have been incorporated into SBME exercises in a manner that provides clear performance metrics. These authors prefer to utilize the phrase “facilitated self-assessment” to best describe the goals of such a curriculum.

General Strategies for a Simulation-Based Curriculum for Graduate Anesthesia Training

The simulated learning environment offers educators a tool for attaining goals in resident training less easily met through didactic or clinical means:

• Tightly scripted scenarios with clear performance metrics : If relevant information of performance is to be provided to the learner, simulated clinical scenarios must be standardized and structured in a way that clear outcomes metrics can be assessed. SBME is an ideal vehicle for providing opportunities for residents to receive feedback for practice improvement based on real-time performance assessment.

• Deliberate practice through scenario repeat: Toward the goal of learner improvement, following a simulated clinical exercise and debriefing, learners can revisit each scenario. This allows learners to apply lessons learned from feedback in debriefing and “better” manage each scenario. This will encourage active self-reflection and deliberate practice to reinforce the content and competencies assessed.

• Exposure and practice with rare and critical events: This is both a benefit and potential pitfall for the educator developing a simulation-based curriculum. The advice of these authors is to not develop a series of scenarios representing a wide swath of rare and critical events, but do start with general learning goals and objectives (ideally based on training-level specific milestones) and utilize a simulated clinical scenario that incorporates those goals and objectives. A wide variety of rare scenarios can easily serve a dual purpose of clinical exposure and provide residents the opportunity to practice general skills such as advanced resuscitation and crisis resource management.

• Integration of competency-based milestones : While an exhaustive list of the milestones well-suited to simulation-based assessment would be too large to reprint in this space, within multiple competency domains (patient care, communication, professionalism, practice-based learning), educators can use the milestones as a guiding superstructure in the development of a simulation-based curriculum for graduate anesthesia training. The structure of the competency-based milestones as developed by the ACGME also provides the advantage of developing a milestones-driven curriculum tailored to resident training level. The description of milestones progression provided within many sub-competencies (Table 14.4) can easily be reflected in a deliberately constructed curriculum consisting of training-level relevant scenarios.

Table 14.4 Non-exhaustive list of sub-competencies appropriate for formative assessment of milestone attainment through simulation for graduate anesthesiology trainees

A Sample Training-Level Specific Simulation-Based Curriculum for Formative Assessment

A simulation-based curriculum for formative assessment in graduate anesthesiology education is dependent to a large extent on the institutional and departmental simulation resources available and the level of departmental faculty expertise. The curriculum described here was developed in order to meet the needs of multiple training programs clustered within a single region, most of which had no access to a simulation center when the inclusion of a simulated clinical experience was included in the revised program requirements by the Anesthesiology Residency Review Committee in 2011. This curriculum consists of a 1-day workshop for each level of clinical anesthesia training. Each workshop is divided into “modules” consisting of three separate scenarios/stations through which small groups of trainees rotate to maximize the opportunity for exposure within a limited time period alternating in the role of the “hot seat.” Between modules, brief didactic or discussion sessions were utilized to reinforce relevant topics such as the ASA difficult airway algorithm, the AHA algorithm for pulseless cardiac rhythms, or team training and communication. The details for the scenarios utilized (scripts, supporting documents) are less important than the extent to which fidelity to milestones and competency-based objectives is served by the scenarios utilized or developed.

Workshop for Formative Assessment of Clinical Anesthesia Year 1 (CA-1) Trainees

Our group has developed a series of modules (Modules 17.1) designed to address level 1 and 2 milestones within sub-competencies in the domains of patient care, interpersonal and communication skills, professionalism, and systems-based practice. Specific anesthetic issues will focus on core topics such as airway management, ACLS, and common intraoperative issues such as hypotension and hypoxia.

Module 17.1. A Simulation-Based Workshop for Formative Assessment of Clinical Anesthesia Year 1 (CA-1) Trainees

Learning objectives	Perform a standard preoperative anesthetic assessment. Perform a standard anesthetic induction and emergence. Evaluate and treat common causes of intraoperative hypoxia. Evaluate and treat common causes of intraoperative hypotension. Evaluate and treat intraoperative arrhythmias. Display the ability to apply supraglottic airway techniques. Display the ability to apply videolaryngoscopic techniques.
Medical knowledge objectives	Describe the American Heart Association (AHA) algorithm for pulseless rhythms. Describe the ASA difficult airway algorithm.
Module 1	Partial task trainer stations Placement and management of supraglottic airways Using a variety of head or airway mannequins, trainees can display appropriate placement, securement, and use of a supraglottic airway device. Flexible and rigid video-assisted laryngoscopy Using a variety of head or airway mannequins, trainees can display endotracheal tube placement with the assistance of videolaryngoscopy and flexible fiberoptic bronchoscopy. Neuraxial anesthetics and sterile technique With the use of low or high-fidelity mannequin-based partial task trainer, trainees can demonstrate spinal and epidural anesthetic technique as well as sterile technique.
Module 2	High-fidelity simulated airway management Non-urgent difficult airway management This scenario should allow trainees an opportunity to perform a standard induction of anesthesia, work through multiple steps of the difficult airway algorithm (at the discretion of the instructors) and conduct a safe emergence from anesthesia. Elements of communication and professionalism can also easily be integrated into this scenario, and many of the following scenarios. Emergent critical airway management This scenario should allow provide trainees with an opportunity to manage a difficult airway working through all steps of the difficult airway algorithm concluding with cricothyrotomy. Our group utilized a scenario involving a patient in the surgical intensive care unit following a complex cervical spine procedure requiring transfusion of multiple blood products. The trainee encounters the patient in response to a “STAT” anesthesia page. The patient’s cervical spine is immobilized in a cervical collar and has self-extubated. Emergent intubation in a non-clinical location This scenario should provide the trainee an opportunity to encounter a patient requiring intubation away from the standard operating room or critical care unit environment in order that the trainee can display the ability to obtain the appropriate tools necessary and coordinate assistance from those present. This can be achieved involving a scenario in which a colleague is found unconscious in a call room with vomitus present in the airway.
Module 3	Management of hypoxia, hypotension, and pulseless cardiac arrest Hypoxia In this scenario, trainees will diagnose and treat multiple causes of hypoxia within a single scenario. Our group utilizes a scenario involving a laparoscopic hysterectomy in an obese patient. Following initiation of pneumoperitoneum and Trendelenberg positioning, a right mainstem bronchus intubation occurs. After diagnosis and treatment of mainstem intubation, the patient experiences insidious and slow developing hypoxia due to progressive atelectasis from habitus, pneumoperitoneum, and positioning effects of functional residual capacity. Hypotension This scenario should provide trainees with an opportunity to distinguish between causes of hypotension. Ideally, the scenario will provide an example of hypotension due to a decrease in vascular tone as well as hypotension due to a decrease in cardiac output. Our group utilizes a scenario involving an elderly hypertensive woman undergoing a partial colectomy. Following induction, an anesthesia-mediated decrease in systemic vascular resistance leads to hypotension (perhaps from overzealous dosing by a senior). Later, unrecognized bleeding leads to a decrease in cardiac output. Pulseless rhythms This scenario provides trainees with an opportunity to manage pulseless rhythms in the operating room. The specific scenario details are of less importance than simulating rhythms that can be effectively defibrillated as well as those that cannot. Our group utilizes a scenario involving a patient with severe coronary artery disease with recent drug-eluting stent placement undergoing emergent surgery for gastric bleeding. The electrocardiogram devolves from evidence of ischemia with ectopy to pulseless electrical activity, ventricular tachycardia, and ventricular fibrillation. The instructor can cycle through rhythms as is deemed appropriate.

Workshop for Formative Assessment of Clinical Anesthesia Year 2 (CA-2) Trainees

For trainees in the second year of training, simulation-based formative assessment should reflect the progression of clinical training house staff have experienced. Therefore, subspecialty simulations can play a role to expand the breadth of competencies assessed. While core concepts such as airway management and ACLS are readdressed, milestones assessed should be those between levels 2 and 3 of the relevant sub-competencies (patient care, interpersonal and communication skills, professionalism, and systems-based practice). Some technical skills are readdressed through the use of part-task trainers to allow for a demonstration of progression of technical skills (see Module 17.2).

Module 17.2. A Simulation-Based Workshop for Formative Assessment of Clinical Anesthesia Year 2 (CA-2) Trainees

Learning objectives	Perform a complex preoperative anesthetic assessment relevant to subspecialty anesthetic care. Perform a subspecialty anesthetic induction and emergence Evaluate and treat uncommon complications that arise within subspecialty anesthetic care. Evaluate and treat common causes of intraoperative hypotension Lead a team in the management of pulseless cardiac arrest. Display the ability to apply advanced airway techniques. Display the ability to perform cricothyrotomy and transtracheal jet ventilation.
Medical knowledge objectives	Describe the American Heart Association (AHA) algorithm for pulseless rhythms. Describe the ASA difficult airway algorithm.
Module 1	Partial task trainer stations Placement and management of supraglottic airways Using a variety of head or airway mannequins, trainees can display appropriate placement, securement, and use of a supraglottic airway device. Flexible and rigid video-assisted laryngoscopy Using a variety of head or airway mannequins, trainees can display endotracheal tube placement with the assistance of videolaryngoscopy and flexible fiberoptic bronchoscopy. Cricothyrotomy and transtracheal jet ventilation Through the use of a head or airway mannequin, trainees can perform cricothyrotomy with an angiocatheter or surgical cricothyrotomy kit in order to provide oxygenation through transtracheal jet ventilation or a more secure tracheostomy.
Module 2	High-fidelity simulated complications in anesthesia subspecialties High spinal anesthesia in an obstetric patient with a difficult airway This scenario should allow trainees an opportunity to recognize the development of a high spinal anesthetic through characteristic changes in hemodynamics and patient symptoms. This will require communication with the obstetric team regarding prioritization of fetal delivery or maternal support as a difficult airway is encountered requiring application of the difficult airway algorithm. Venous air embolism during a craniotomy This scenario will provide trainees with an opportunity to recognize the development of a high spinal anesthetic through characteristic changes in hemodynamics and patient symptoms. This scenario can easily integrate issues such as surgical complaints regarding the interventions required for treatment and the impact on the surgical field and surgical progress. Postoperative respiratory arrest due to narcotic overdose In this scenario, trainees encounter an obtunded patient in the post-anesthesia care unit. This patient has a complex history of chronic pain, and perioperative analgesic management has been conducted under the guidance of the pain management service and includes long-acting opioids and a battery of non-opioid analgesics.
Module 3	Management of advanced life-threatening anesthetic complications Malignant hyperthermia in a pediatric patient In this scenario, trainees will conduct a standard induction of anesthesia for a pediatric patient, be expected to diagnose and treat malignant hyperthermia, and direct the postoperative management and disposition. Particular attention should be paid to directing the acquisition and administration of therapeutic agents. Local anesthetic systemic toxicity (LAST) following a regional anesthetic block This scenario should focus on the identification and management of LAST resulting in cardiovascular collapse. In order to integrate milestone representative of the competency of systems-based practice, debriefing should focus on locating and obtaining lipid therapy within the perioperative environment. ACLS leadership Our group utilizes a scenario in which the trainee is expected to lead ACLS in the postoperative care unit. This scenario should focus on resource allocation, leadership, and communication in addition to command of the AHA algorithms.

Workshop for Formative Assessment of Clinical Anesthesia Year 3 (CA-3) Trainees

For trainees in the final year of training, simulation-based formative assessment should reflect a transition to supervision of anesthetic care management of multiple patients as may occur in the postoperative care unit. Simulated scenarios will be designed to allow for practice and end performance of these skills. Given the critical role of airway management and ACLS, these topics are revisited through partial task trainers and scenarios with an emphasis on supervision and team leadership. Transthoracic and transesophageal echocardiography are introduced through partial task training as well. Milestones assessed should be those between levels 3 and 4 of the relevant sub-competencies (patient care, interpersonal and communication skills, professionalism, and systems-based practice) (see Module 17.3).

Module 17.3. A Simulation-Based Workshop for Formative Assessment of Clinical Anesthesia Year 3 (CA-3) Trainees

Learning objectives	Perform oversight if the administration of an anesthetic with a junior practitioner. Display the ability to prioritize and delegate care of multiple critical patients. Recognize and effectively manage an operating room fire. Evaluate and treat postoperative complications that may occur in the post-anesthesia care unit. Lead a team in the management of pulseless cardiac arrest or other unstable cardiac arrhythmias. Display the ability to supervise or perform advanced airway techniques.
Medical knowledge objectives	Understand and acquire key transesophageal and transthoracic echocardiographic views of cardiac anatomy and function. Describe advanced airway techniques as part of the ASA difficult airway algorithm.
Module 1	Partial task trainer stations Transesophageal and transthoracic echocardiography Through the use of a partial task trainer, trainees will obtain and evaluate key views and describe function of cardiac anatomy. Flexible fiberoptic bronchoscopy Using a variety of head or airway mannequins, trainees can perform endotracheal tube placement with the assistance of flexible fiberoptic bronchoscopy. Participants will also utilize a virtual bronchoscopy partial task trainer to obtain views of glottic and tracheobronchial anatomy. Cricothyrotomy and transtracheal jet ventilation Through the use of a head or airway mannequin, trainees can supervise and instruct another in the performance of cricothyrotomy with an angiocatheter or surgical cricothyrotomy kit in order to provide oxygenation through transtracheal jet ventilation or a more secure tracheostomy.
Module 2	High-fidelity simulated situations in anesthetic supervision and administration Supervision of a difficult junior resident This scenario should allow trainees an opportunity to supervise a junior resident lacking proficiency in diagnosis, management, and procedural competence during a difficult ventilation and intubation upon induction of anesthesia. This will require the trainee to strike a balance between education and patient safety. Postoperative bleeding in the postoperative care unit In this scenario, trainees encounter a hemodynamically unstable patient status post renal transplantation in the postoperative care unit. The trainee will have to diagnose hemorrhage and direct the surgical team to return to the operating room. This scenario will transition to the hemorrhage scenario in the operating room. This scenario will focus on communication in addition to management of an unstable patient. Hemorrhage In this scenario, the trainee will transition the patient from the prior PACU scenario into the operating room where they will be required to oversee the anesthetic with a junior resident while receiving requests from the PACU nursing regarding other postoperative patients. This scenario emphasizes team leadership, OR management, teamwork, communication, task delegation, and resource allocation.
Module 3	Management of rare or complex life-threatening perioperative complications Intraoperative airway fire during tracheostomy placement in the operating room In this scenario, poor communication will occur between the surgical team and the trainee regarding the use of electrocautery during in the setting of a high fraction of inspired oxygen. The trainee can be led to provide high FiO2 through the simulation of a patient with tenuous arterial oxygenation due to ARDS. This scenario will focus on both the management of an airway fire and communication. Hematoma and respiratory distress following tracheostomy in the PACU In this scenario, the trainee will be called to evaluate a patient experiencing respiratory distress following thyroidectomy. They will discover an expanding hematoma compromising oxygenation and ventilation. Focus should be placed on communication with the surgical team, marshalling airway supplies and support, and difficult airway management. ACLS leadership In this scenario, the trainee(s) will be asked to assume alternating roles during ACLS. Trainees should practice leadership, closed-loop communication, speaking up, mutual support, and other crisis resource management skills.

Simulation for Summative Assessment

Beyond its role in instruction for graduate medical education, simulation technology offers an opportunity for summative assessment of clinical competency. Unlike formative assessment, in which trainees are given feedback to prompt self-reflection and promote further learning, summative assessment is utilized to evaluate trainees prior to attainment of a certain status or matriculation. This can vary from attainment of specific milestones or competencies to the more critical question of whether the trainee is ready to graduate and to independently practice.

The high-fidelity simulated environment allows the opportunity to evaluate residents in ACGME defined competency-based milestone domains difficult to assess using typical written exams. These milestones represent the highest layer of Miller’s pyramid, in which the student “does.” [19] Communication, teamwork, technical skills, complex diagnostic and therapeutic abilities, and performance in crisis situations are areas that are challenging to objectively measure [35]. Typically, training programs rely on clinical performance evaluations or input from faculty and the Clinical Competency Committee to measure performance in these competencies. However, these methods can be subjective among other shortcomings.

Although simulation has been seen as a tool in anesthesiology education since the 1950s, interest in using simulation as an evaluative tool has arisen more recently. This has resulted partly from the response of academic institutions and professional organizations (i.e., ASA, Anesthesia Patient Safety Foundation) to the public demand for assurance practitioner competency. The shift in the graduate medical education from a time-based to an outcomes-based model has included a shift toward recognition that specific milestones must be met and competencies demonstrated before advancement and certification [36, 37]. Medical licensing and professional accrediting bodies have accepted the role of performance assessments as a component in professional licensure and accreditation. In addition to the introduction of a mandatory, simulation-based component to Maintenance of Certification (MOC) by the American Board of Anesthesiology (ABA), summative assessments using simulation or OSCE have been adopted by the National Board of Medical Examiners (NBME), The Medical Council of Canada, The Fellow Royal College of Anesthesiologists, and The Israeli National Board Examination in Anesthesiology [38]. Furthermore, an Objective Structured Clinical Examination (OSCE) component has been added to the ABA APPLIED Board Examination as an adjunct to the oral exam. Through the development of these standardized, patient-based simulations, a wealth of experience and expertise has been gained regarding examination design, test administration, logistics, quality assurance, and psychometrics [39]. This has provided a foundation for the development of summative assessment through the use of simulation. For these high-stakes assessments in which decisions are made regarding the competency and/or the advancement status of a trainee, quality assurance measures and precision in scoring are essential.

Since the ACGME finalized the transition to a competency-based education model with the “milestone project,” residency programs have each been charged with developing their own approach to assessment competency-based milestone attainment. In response to this charge, many anesthesiology residency training programs have found simulation to be a valuable tool in assessment of competency attainment [40]. A survey of 132 US academic anesthesiology residency programs revealed that of those institutions that responded (66%), 40% utilize simulation for resident assessment and remediation. A far greater share of those responding (89%) utilize standard simulation for resident education. Many of the programs that have begun to use simulation for summative assessment are explicitly assessing milestone attainment to trigger transition between training levels and remediation. Some of these programs have focused on the competency domains of communication and professionalism, while others have incorporated checklists of complex clinical tasks into their evaluations. The biggest challenges reported by training programs in instituting and maintaining simulation into summative evaluations are insufficient time, simulation-trained personnel, and money. Others have expressed concern that using simulation for any reason than training or formative assessment is in conflict with the principle that the simulation environment should provide a “safe space for learning.” [26]

Key Steps in Developing a Simulation-Based Program for Summative Assessment

There are several important issues to consider when developing a curriculum for simulation-based assessment. One must define the purpose of the assessment (advancement in training level, data collection for the clinical competency committee), choose the appropriate clinical competencies to measure, create scenarios that elicit performances reflecting the targeted competencies, and develop measurement tools that can provide reliable and valid assessment scores.

Identifying the Purpose of the Assessment

The first step in developing a simulation-based assessment program is identifying the specific purpose of the assessment. For example, in resident education, a critical question is whether trainees have acquired the necessary competencies to independently provide safe anesthesia care following completion of their training. Once the aim of the assessment is clearly defined, one can develop the components of the test that will provide date to meet this goal (i.e., which specific clinical competencies will be assessed). A conventional way to choose competency-based milestones or competency domains for summative assessment is to utilize those already detailed in a training program’s curriculum (i.e., goals and objectives) based on those enumerated by the ACGME Anesthesiology Milestone Project. In addition, The Joint Council on In-Training Examinations, a committee of the American Board of Anesthesiology, publishes a content outline comprised of a detailed description of the basic and clinical topics in which an anesthesiology consultant should be competent. When creating objectives for a simulation-based assessment program, it is imperative to select only those milestones or skill domains that can be reasonably assessed in the simulated environment to collect accurate information regarding trainee competence. Several methods for development of simulation-based assessment programs and the relevant skill domains have been described in the literature. One method is the creation of a list of perioperative events that a resident should be able to effectively manage following the completion of their training and cross-referencing this list with topics in the American Board of Anesthesiologists content outline to assure that they conform to the attributes of a consultant as determined by the ABA. Using this method, Murray et al. [41] developed a set of six clinical cases for assessment: (1) postoperative anaphylaxis, (2) intraoperative myocardial ischemia, (3) intraoperative atelectasis, (4) intraoperative ventricular tachycardia, (5) postoperative stroke with intracranial hypertension, and (6) postoperative respiratory failure (Table 14.5). Another method described by Blum et al. involves the creation of a list of behavioral domains [42]. In this method, a panel of experts from one’s institution composed of clinically experienced board-certified anesthesiologists involved in residency education are asked, “What traits characterize residents who, upon graduation, have not achieved a minimum level of competency?”. Then, using a modified Delphi process,¹ these responses can be reduced to a list of key behaviors that are lacking in an underperforming senior resident. Blum et al. identified five key behaviors through this method: (1) synthesizes information to formulate a clear anesthetic plan, (2) implements a plan based on changing conditions, (3) demonstrates effective interpersonal and communication skills with patients and staff, (4) identifies ways to improve performance, and (5) recognizes own limits. Seven scenarios were then designed based on these five behavioral domains with clinical material incorporated using the ABA examination content outline and the ACGME core competencies . The resulting scenarios were (1) preoperative assessment of a patient scheduled for urgent exploratory laparotomy, (2) operative management of a patient with perforated ulcer and hemorrhage, (3) monitored anesthesia care for a patient with discomfort during basal cell carcinoma surgery, (4) post-anesthesia care for a patient with aspiration after basal cell carcinoma surgery, (5) management of anaphylaxis in a patient with transurethral resection of the prostate and bladder biopsy, (6) care for a patient with delayed awakening in the operating room after transurethral resection of the bladder, and (7) identification and management of mainstem intubation secondary to coughing in a patient undergoing total thyroidectomy [43].

Table 14.5 Scoring items

Scenario Development

Once competency domains and scenario topics have been identified, it is important to develop simulation scenarios that best target the specific skills or behaviors one intends to observe or measure. To minimize simulation artifact and optimize the precision of the assessment, scenarios should maintain fidelity to the clinical environment by providing those usual supplies, equipment, and patient characteristics with which the trainee is familiar. Scenarios that work well within the technical limitations of the simulation equipment while targeting the expected training level of the examinees are most successful. Summative assessment curricula are often composed of scenarios that focus on critical clinical situations for many valid reasons. First, the field of anesthesiology, by its nature, deals with rare yet catastrophic events that can lead to severely adverse patient outcomes if the practitioner is ineffective in their management. Secondly, rare and critical scenarios may provide an opportunity to identify residents that are struggling earlier in training given the inability to reveal some deficits through the performance of routine anesthetic care. Additionally, acute care simulation scenarios typically test difficult to evaluate skill sets that are frequently lacking in “borderline residents” such as prioritization, generating of a differential diagnosis, processing knowledge, assigning probabilities, isolating essential from non-essential information, integrating competing issues, acknowledging limits, and knowing when to call for help [44].

Given existing evidence of the effectiveness of the OSCE format [45] coupled with the recent integration an OSCE component into part II of the ABA APPLIED Examination, it is reasonable to utilize this format as a component in the design of a curriculum for summative assessment. OSCEs may be brief simulation scenarios in which a trainee would be required to quickly diagnose and perform key therapeutic actions, brief interactions with standardized patients or care team members, or partial task trainers integrated into a clinical scenario. Compared to high-fidelity simulated clinical scenarios with multiple learning objectives, there is evidence that a series of short OSCE-based scenarios with clearly defined objectives more accurately reflect a trainee’s abilities [2]. Moreover, many of the psychometric concerns associated with high-fidelity, mannequin-based assessments can be overcome with this style of testing [46].

Metrics

One of the most critical steps in constructing a high-stakes simulation-based curriculum for summative assessment is the creation of appropriate metrics. One must be certain that the scoring tools that are used accurately reflect the trainee’s abilities.

Two scoring methods have been most widely described anesthesia simulation literature – checklists and global assessments. Generally, checklists composed of key actions are considered a more objective measure of performance such as clinical diagnosis and management, while global assessments tend toward greater subjectivity while allowing more utility in assessing complex behavioral traits such as judgment, teamwork, and communication [47]. To create a valid scoring tool, it is important to understand the advantages and limitations of each system. Generally, a valid and comprehensive summative evaluation may use various scoring methods in combination.

Checklist Scoring

In this scoring method , a comprehensive list of actions is created comprising the essential steps for successful management of a scenario. For example, in a case of anaphylactic shock, steps in the diagnosis of anaphylaxis (i.e., examine the airway, auscultate the chest, elicit wheezing, etc.) and critical steps in the treatment (i.e., administer IV fluids, administer epinephrine, treat hypotension, administer diphenhydramine, administer steroids, etc.) would constitute the anaphylaxis checklist. While checklists are considered an objective scoring system, they suffer from subjectivity in their construction [48]. Typically, faculty convene and determine which actions are essential to successful trainee performance based in the scenario. To work toward a valid checklist, experts frequently integrate patient care guidelines and practice standards, but considerable debate may remain regarding those actions which are essential or inessential in a scenario. One may utilize a Delphi technique to achieve a consensus among experts to produce an objective and valid scoring tool [49].

Though checklist scoring can provide objective data in summative assessments, it is imperative to recognize the shortcomings of this scoring method. With checklist scoring, it is difficult to account for timing and sequence. In many scenarios, as in clinical practice, it is not only important what actions are performed but also the order and the timing of the actions. In the example of anaphylactic shock, for example, a resident may perform all the items in the checklist correctly, but if the time to give treatment is delayed or if epinephrine is given prior to diagnosis of anaphylaxis, clinical performance would be considered a substandard. To address the issue of timing, one may incorporate a time limit or a “time to action” component into the scoring rubric [50]. Checklist items that use a time limit for various actions are particularly useful in discriminating between more and less experienced trainees. An additional drawback to checklist scoring is a tendency to promote rote behaviors and reward “completeness” as opposed to the prioritization and performance of more critical steps over less critical steps. In clinical situations, a competent physician identifies the most critical steps to be performed in order to effectively manage the critical situation. Using a simple checklist would penalize residents who rapidly assess a patient and effectively manage the condition with the most critical steps but do not perform certain ancillary actions placed on the scoring sheet. On the other hand, a resident could perform the majority of actions on a checklist but fail to perform those most critical to effectively diagnosing and treating the patient. Creating a weighted checklist, in which more value is added to the most important checklist items, helps to address this concern. In addition, shortening of checklists to essential “key actions” ensures the examinees are primarily evaluated on the ability to perform the most critical steps in diagnosis and management. For example, in a case of anaphylactic shock, the checklist may be reduced to three essential steps: (a) diagnosis of anaphylaxis, (b) any treatment regime for suspected anaphylaxis, and (c) any dose of epinephrine given. In addition, a “time to key action” can be beneficial in long scenarios where accounting for time plays an important role.

Global Rating Scoring

A global assessment is an evaluation of the entire performance , as a whole. For example, the rater will decide on a Likert scale of 0 to10 the level at which a resident has performed. Typically, there are descriptors assigned to the values (i.e., level 0 = unsatisfactory, level 7 = performance of a competent consultant, level 10 = outstanding). Global assessments are a more subjective scoring tool than the checklist method, appropriate for measuring non-technical skills such as communication and teamwork that are, by their nature, less objective in nature. Given this subjective character, the main concern practitioners have in using global rating scales is the potential for inter-rater variability. This potential shortcoming can be minimized through proper scenario construction and effective rater training. Despite the criticism that global ratings are generally more subjective than analytical checklists, there is literature suggesting that holistic ratings have similar reliability and reproducibility [2]. Global ratings can be an effective and reliable tool for assessing complex and multidimensional skills such as teamwork and communication [51]. In some instances, such as when a competency is assessed across multiple scenario assessments, it may be acceptable to sacrifice a degree of score precision for a greater score validity in assessing non-technical skills such as judgment, communication, planning, situational awareness, and teamwork. Holistic ratings are particularly advantageous over analytical checklists in allowing evaluators to assess the sequence of actions taken by the trainee while accounting for inappropriate actions and unnecessary patient management. This approach can also liberate the rater from a long list of checklist items that can be taxing to complete and provide for uninterrupted observation.

The most challenging aspect of integrating global ratings into a summative evaluation is ensuring that raters are qualified, proficient, and reliable evaluators. Although faculty physicians involved in clinical education typically serve as raters, they are not necessarily qualified. Establishing an effective rater training program is crucial, especially when the appropriateness of behaviors that are scored is open to interpretation. Videotaping the performance of the examinee and later allowing raters to evaluate the session is beneficial as it permits time for a careful review and documentation of debatable performance.

While here is significant disagreement regarding the most effective scoring rubric, various studies have looked at the relationship between scoring modalities with some concluding a similarity in ranking of examinee performances regardless of the holistic or analytical method [52]. The importance of utilizing a scoring modality appropriate to the competencies being assessed is without debate. A combination of checklists, key actions, and global ratings can be used for a comprehensive assessment of resident performance of technical skills, acute care scenarios, communication, and teamwork skills, all of which are important elements of anesthesia practice.

Reliability and Validity

As opposed to formative assessments which aim to assist a trainee in his or her development, summative assessments serve a role in assuring the public that the trainee has met competency standards and is fit to practice independently. It is, therefore, imperative that summative assessments are valid and reproducible. In developing a simulation-based summative assessment, one need be certain of reliability or precision, repeatability, and reproducibility of the results. Reliability imparts confidence that the scores obtained fairly serve as a true reflection of the examinees’ abilities. However, examiner variability, candidate performance variability, or errors in measurement associated with the content of the assessment can negatively affect reliability through the introduction of error. Although inter-rater variability can be a significant source of error, there are several tactics that can help to mitigate this effect: selecting the appropriate skills or competencies for assessment, the method of assessment (i.e., live vs. video review), the scoring methods (i.e., checklists vs. global), use of rater training protocols, and utilizing multiple raters for any given examinee [49]. As one increases the quantity of assessment for each examinee, one better ensures that the performance measured reflects the examinees’ true clinical abilities. This can be achieved by incorporating multiple scenarios or tasks. In general, for multi-scenario performance-based assessments, issues regarding inadequate score reliability can be best addressed by increasing the number of simulated tasks rather than increasing the number of raters per simulated scenario [51]. Furthermore, with proper rater training and well-specified scoring rubrics, the impact of inter-rater variability on reliability can be minimized.

An assessment is valid if it measures what it sets out to measure. The ideal benchmark for validity of a simulation-based assessment is the ability to correlate performance with patient outcomes. Establishing a causal link between simulation performance and patient outcomes is challenging, and although strong evidence in the literature is lacking, numerous studies have concluded that the skills acquired in simulation can transfer to actual clinical practice. Based on this growing evidence, simulation-based training courses for faculty have been used to reduce malpractice insurance premiums [53, 54, 55]. If we want to make inferences (ability to practice independently) based on assessments of performance, validity is critical. Scientific literature regarding simulation-based performance assessments has argued that these assessments have validity and therefore inferences about clinician competency can be reliably based on the resultant scoring [56, 57]. To ensure the validity of a simulation-based assessment, scenarios should be designed to resemble clinical practice as closely as possible. This involves scripting cases and modeling simulated patients to mimic the clinical setting to which the examinee is accustomed. In addition, the specific tasks to be measured should be carefully chosen to reflect practice-based guidelines or expert opinion. One would expect a valid test to corroborate with other assessment tools such as in clinical evaluations and that examinees who perform better in a simulation-based assessment would outperform lower-performing examinees in the clinical setting as assessed through clinical evaluations. In fact, most studies investigating simulation-based summative assessments show that if the scoring systems are appropriate and the scenarios incorporate the appropriate content, individuals with more clinical training and advanced experience perform better in these simulation-based assessments. If the scores obtained from a simulation assessment do not reflect this clinical reality, the validity of the assessment results should be questioned. An additional factor that could lead to such a disparity includes examinee unfamiliarity with the simulated clinical environment. Most programs will introduce simulation-based summative evaluations only after trainees have had requisite exposure to the simulation environment through prior training or formative assessments.

By ensuring fidelity to the clinical environment and trainee familiarity with the simulated setting, a simulation-based summative assessment incorporating the appropriate targeted competencies and assessment strategies can play a role in aiding anesthesiology residency training programs in making critical decisions regarding trainee advancement.