Development of Scenario and Training Materials: Fundamentals, Interprofessional and Hybrid Scenarios, and Inclusion of Patients in Standardized Patient Methodology (in Implementing Best Practices in Standardized Patient Methodology)

Abstract

Development of scenarios and training materials for Standardized Patient (SP) based activities is a process in which multiple components need to be considered including educational objectives, level of the learner, healthcare content, and SP methodology expertise. The process for developing the content for human simulation activities is simplified by the use of a template including administrative details for recruiting SPs, defining the needs for the planned activity and materials for training the SPs. This chapter incorporates the Association of Standardized Patient Educators (ASPE) template and provides tips when developing cases and associated training materials. Suggestions when creating cases for interprofessional team assessment, and simulated families (and other groups) will be discussed.

Introduction

The purpose of this chapter is to outline a step-by-step development process that will support you in creating, adapting and revising standardized patient (SP) scenarios for your program. We interviewed members of consortia who work together to develop cases to give readers an idea for going outside of a single institution for additional benefits.

Scenario Development

The scenario development process starts with the curriculum learning objectives, upon which the patients’ story is based. All the scenario materials an SP Educator (SPE) needs to train the SP to authentically portray that story should include (but are not limited to); goals and learning objectives, level of the learner, patient story and content, supplemental learning resources obtained, researched and/or prepared by the SPE (e.g. video, illness story, timeline, moulage, props, setting, door instructions for learner), feedback or assessment tools and training guides (e.g. checklist, communication rubric or scale), and information about the administrative aspects of delivering the learning or assessment activity. Developing case materials and supporting documents is made easier with the use of a standardized template. Using a template helps to organize the details and standardize your approach. It helps the SPs learn a case quickly through using a familiar format.

With any educational endeavor, the first step is defining learning objectives for the activity specific to the level of the learner [3]. This aligns with the ASPE SOBPs 2.2.1 Clear goals and objectives that can be assessed and 2.2.2 Goals and objectives that specify the intended level of learners.

The Liaison Committee on Medical Education (LCME) , the body who accredits MD programs in the US and Canada, requires as part of the accreditation standards:

…the medical school ensures that the learning objectives for each required learning experience (e.g., course, clerkship) are made known to all medical learners and those faculty, residents, and others with teaching and assessment responsibilities in those required experiences [4].

The LCME defines learning objectives as: A statement of the specific, observable, and measurable expected outcomes (i.e., what the medical learners will be able to do) of each specific component (e.g., course, module, clinical clerkship, rotation) of a medical education program that defines the content of the component and the assessment [4].

Each measurable learning objective describes the desired outcome (what the learner should do; e.g. take a medical history), the conditions under which the skill is performed (e.g. with a patient in a primary care setting), and, if the examination is a summative assessment, the level of competence required to pass (e.g. obtain 85% of the history items on a checklist). The level of competence may be adjusted depending on the level of learner, for example, expecting higher performance for more advanced learners. Well-written objectives ensure the scenarios being developed are relevant to the learner/assessment activity. Often, clinical faculty want to develop cases based on interesting patients they’ve seen in practice, without regard for the educational objectives. So, it is useful to encourage clinical faculty to continually bring the SP case focus back to the educational objectives to ensure the case complements and speaks to the intended curriculum. Typically, cases written for SP-based activities include objectives for history-taking, physical examination, patient education, communication skills, and clinical reasoning. More specifically, objectives may include practicing team-based care (working interprofessionally), or successful performance of technical procedures.

Tip

SP availability is not a basis for developing cases; the educational objectives always guide case development.

Formative and Summative

Formative experiences foster knowledge acquisition, skills development, focus on providing feedback to the learner, and are designed to support behavior change in keeping with educational objectives and improve performance. While case development is not dramatically different between formative and summative activities, cases developed for summative assessment—where the outcome of the evaluation determine pass/fail status, promotion to the next level, or licensure decisions—must contain the highest level of detail. Case materials will support the SPE in training the SPs to provide standardized performances and accurately score the instruments designed to collect performance data. If SPs provide verbal or written feedback, they require additional training to meet formative or summative objectives. (See Chaps. 7, 8 on Training SPs and 9 – Communication Training).

For the purposes of this chapter we will continue with traditional definitions and application of formative and summative activities when preparing scenarios. However, after reading Chap. 5 which introduces the Human Simulation Continuum Model, you may find yourself thinking more broadly of human simulation (HS) and when to apply them. Selecting the appropriate HS application will influence how you prepare SPs, modify accompanying materials and refine the case template.

Blueprinting

Like an architect’s blueprint, developing the blueprint for formative experiences in a longitudinal curriculum or multi-station SP encounters in a clinical skills summative assessment is the next important step to guide case development. A blueprint ensures a balanced sampling of cases across task domains (e.g. history-taking, physical examination, and communicating with patients) and criteria such as patient demographics, acuteness of the problem, and organ systems [5]. Using a blueprint prevents learners from seeing only one type of patient (e.g. multiple acute respiratory, geriatric patients, etc.). Because health professions trainees must learn to recognize and appropriately provide culturally competent health care, an important part of the blueprint is diversity in the patient population. Ideally, the blueprint should sample the kind of population the learners are expected to see in clinical practice. This means case materials and SPs should demonstrate diversity in age, ethnicity, gender identity, sexual orientation, race, religion, cognitive and physical abilities, and socioeconomic status [4]. (For more on cultural diversity see Chap. 10).

Formative cases can be designed in a longitudinal, scaffolded fashion throughout a course or clerkship, incorporating the learner’s deeper understanding as they progress, and offer greater challenges once earlier challenges are mastered. The blueprint for formative activities skills progression takes into account this progression as seen in Fig. 6.1.

Fig. 6.1

Sample station for skills progression

For formative simulation activities, the number of cases used should be sufficient to allow the learner exposure to the concepts to later be measured with the summative assessments, so that adequate feedback about areas to improve is provided in advance of the summative assessment. In a summative clinical skills assessment, the number of stations impacts reliability of learner scores; generally, the higher the number of stations, the greater reliability of the examination [5,6,7,8]. The United States Medical Licensing Examination (USMLE), for example, consists of 12 stations. Health professions schools may be limited by the resources available such as fewer rooms or fewer available SPs and staff to run a larger exam. SPEs must balance the resources available with the realization that more stations usually provide a more reliable exam and resulting data set.

For the teaching and assessment of clinical reasoning in medical education, begin with two or three plausible diagnoses for the chief complaint, and build the case from there, aligning with the educational objectives. Avoiding single diagnosis cases allows the learner to develop and demonstrate their clinical reasoning skills by offering a differential diagnoses list after the encounter. For example, right lower quadrant pain in a young female could be appendicitis, ovarian torsion or ectopic pregnancy. The patient’s story will need to be consistent with the diagnoses. (See Chap. 13 Expanding the Field of SP Methodology).

Process for Developing Content

The process for developing case content is similar for formative and summative experiences. Depending on your context, case development for an SP program for which there is a single full-time educator may necessitate a different approach to resources, versus a program in which there are multiple SPEs. Years of experience show that many viewpoints contribute to better cases. Many academic institutions use a group approach for SP case development; however, context and resources will inform various approaches. These various approaches may include a committee approach and/or establishing a consortia and establishing a development team. However, one consistent requirement is collaboration with clinical faculty to develop the case content with the SPE.

One of this book’s editors, Lou Clark, PhD, MFA, has worked as a consultant and for a variety of academic institutions, so we asked her about her approach in these various contexts:

Authors:

Lou, what has worked well for you in the past in terms of developing SP scenarios?

Lou:

I’ve found that beginning with a template in mind is ideal, but then I actually start by interviewing the subject matter expert (SME) or course director who is interested in working with our team.

Authors:

Why is interviewing helpful at this stage?

Lou:

First of all, sometimes we will chat for a few minutes and I will realize that an SP activity is not the best match for the learning goals and objectives. If this is the case, I recommend other options and we do not proceed which saves everyone time and money. So, rather than jumping right into a canned event template noting learner demographics, event details, etc., I find it more useful to begin by asking some open-ended questions such as: what are your educational objectives, what are you hoping your learners will get out of this event? And, how will working with SPs benefit your learners versus other educational methods? If it seems SPs are a good match, then I generally continue the chat and complete an event template in a more organic style—not necessarily from top to bottom but fill it in as details come up. Using this approach enables me to listen for what is most important to the SME, and then to ask follow-up questions to check details when needed. This approach also feels more natural to me, and I actually find it models the communication skills we are encouraging in our learners including asking open-ended questions, listening, and using closed-ended questions to check details—letting the conversation unfold in an improvisational way rather than as a rote checklist.

Authors:

Makes sense, but this approach may not be comfortable for all SPEs. What are your thoughts on that?

Lou:

I agree and as with many aspects of SPE work, there is not one accepted practice. I think each SPE should explore all available resources, and then decide for themselves.

Authors:

What do you do next?

Lou:

This depends on the available resources. If I am the only SPE on the project, I will continue to develop the case with the initiating faculty and encourage him or her to involve at least one other clinician to gain multiple perspectives. If I am a part of, or leading a team, I will look to the expertise within the team and ask an SPE who is best suited to training this case. We will ask the SME who else on their team could contribute to the scenario, and then all work together to write the necessary medical information into the case. Then, we will generally complete the case anticipating the types of questions SPs would ask in training about things like medical jargon and who this patient is as a person, (e.g. home life, educational background, hobbies, etc.). If an SPE on the team has been an SP we will often not involve an SP at this stage. If not, we will try to recruit an SP to get their perspectives. If the case is designed to highlight health concerns related to an underrepresented group, it is of the utmost important to involve a member of that group in your case writing.

Authors:

Can you give us an example?

Lou:

If you are writing a case in which the learning objectives are about health concerns specific to transgender people, it is best to involve a person who identifies as transgender in the case development process.

Authors:

What next?

Lou:

When you are working with the same course director to develop multiple cases and if they are the one always providing information on who the patient is as a person, there is a risk that all of this background information will be similar because it is informed by a single person with a single background. In order to promote diversity within your SP program and the courses and programs it serves, it is crucial to have multiple perspectives—including SP input—into case development. Due to this I will not simply email the SP case template to the SME and ask them to complete it and email it back. If possible, I always encourage a 1-hour meeting in which I or another SPE sits at the keyboard and types into the template while we continue the initial interview with the SME and other clinicians. This way, the conversation becomes a shared creation of the case which we will then go on to pilot. This hour meeting generally saves time by anticipating questions that would have come up in SP training, ensures diversity is built in from the beginning rather than as an afterthought, and enables us to maximize SP training time in other ways.

Authors:

Thanks Lou, We appreciate you sharing your experience and expertise.

A committee approach for developing case content for licensure is used by the USMLE , National Board of Osteopathic Medical Examiners (NBOME) and the Medical Council of Canada (MCC) [6]. For the USMLE, a case development group consists of 3–4 physicians, 2 SPEs, and a case developer who is charged with capturing the discussion and completing the case materials after the meeting, as well as experts in the scoring instruments used (patient notes and communication skills scales) and an SP. To avoid one person’s medical opinion with regard to the best approach to a case, it is optimal that multiple clinicians should give input, preferably from varied specialties. In addition to the clinicians, as noted above, the SPEs and SP contribute. SPs may also be partners during the development process to briefly demonstrate aspects of the case for the clinicians. A first year post graduate physician is available to role play with the SP immediately after the case is developed while the committee observes, in order to make changes before the case goes further into development. While such a large group may seem beyond the resources of most schools, at the very least, more than one clinician, an SP, and an SPE should be involved in the process. The case writing group at the institution should mark out regularly scheduled time to work on developing cases to add to the case bank for formative and summative uses. This will help the group hone skills needed to continue to work as a team. There is also the practical benefit that cases may be alternated by class of learners so learners may not pass down curricular information from class to class.

Higher level learners can be used to rehearse or test out the case during development and for piloting. Piloting involves running the case with someone similar to the learners who will be using it (e.g. 4th year learners for cases intended for 3rd year learners; first year residents for cases intended for 4th year learners). Ideally every case should be piloted, but sometimes it is not feasible. If the case cannot be piloted separately, insert the case into the live exam as an unscored station for data collection purposes (learners should not be aware it is not scored until after the examination). Piloting cases uncovers flaws that can be corrected such as unforeseen diagnoses identified by the learners, missing information from the case materials, or SP portrayal challenges.

Tip

Revisions can be made based on the data collected by the pilot. Case development is an iterative process.

Another approach to maximizing resources is forming consortia with other programs to develop and share scenario materials. We interviewed Win May, MD, PhD, Professor in the Division of Medical Education, Department of Pediatrics, and the Director of the Standardized Patient Program in the Keck School of Medicine. The Keck School of Medicine is a member of the California Consortium.

Authors:

Could you please describe the organization of the consortium. Which schools belong and when did it start?

Dr. May:

The Consortium consists of the eight allopathic medical schools in California. The Consortium began in the 1990s, when the Macy Foundation awarded grants to support six consortia of US medical schools to develop and implement a SP Clinical Performance Examination. This Consortium was one of them and is the “lone survivor.” It started as the Southern California Consortium for the Assessment of Clinical Competence, with the five southern California medical schools. The Northern California schools joined later in the early 2000s. The Consortium has four Committees: Executive Committee, Research Committee, Finance Committee and the Trainers Committee.

Authors:

What resources are shared?

Dr. May:

All cases, training materials and checklists are shared as well as training videos.

Authors:

How does scenario development work for the consortium?

Dr. May:

There is a primary school and a secondary school for each case that is developed. The primary school is responsible for the case development. Once that is done, the case will be sent to the secondary school, for comments and suggestions. The completed case is presented at a Consortium meeting, where there is always lively discussion of the training materials and checklists.

Authors:

What are some of the strengths of taking part in a consortium?

Dr. May:

Strengths: (a) the ability to share and discuss training materials, checklists and videos is huge. We can bounce our ideas off one another, and get feedback from both clinician and trainer (SPE) perspectives; (b) All school data are collated and analyzed by a psychometrician, so we can see how our learners are performing, as well as how the case itself is doing; (c) There is a SPE meeting every year, where we can discuss issues and problems encountered when training the cases, and how to deal with them.

Authors:

What should folks thinking about starting a consortium need to know?

Dr. May:

It is such a worthwhile endeavor, and you will need to have members who are committed to the success of the Consortium.

The Mid Atlantic Consortium (MAC) is comprised of the SPEs at Clinical Skills and Simulation Centers representing several medical schools in the Mid-Atlantic region of the US. Collaboration arose in the mid-1990s from the Macy Foundation’s support of several early consortia. Originally called the Baltimore/Washington Consortium, the group consisted of George Washington , Georgetown, Howard, Johns Hopkins, Uniformed Services University of the Health Sciences, and the University of Maryland. In 2009, the consortium was renamed the Mid Atlantic Consortium.

The purpose of the MAC is to foster collaboration among centers and universities in the consortium. Karen Lewis, PhD, CHSE, Administrative Director of the CLASS Center and SP Program Director at the George Washington School of Medicine and Health Sciences provided information about the MAC , summarized in Table 6.1.

Table 6.1 Mid Atlantic consortium

Using a Template for Scenario Development

Now that you have developed your objectives and blueprint and we have learned about sharing resources versus single institution scenario development, we will focus on developing your case within a template. To illustrate how a template works, the ASPE development template will be used to explain the various parts of a scenario [2]. Developing scenario materials and supporting documents is made easier with the use of a standardized template.

Using a template helps to organize the details and standardize your approach to the activity. Once introduced, it also helps all the stakeholders in the educational activity to fully understand what is needed for an SP activity and to help them consistently prepare quality cases. Another benefit in using a standardized template is SPs learn a case quickly through using a familiar format. Please note that there is not a single accepted template SPEs must use, but in this chapter, we are intentionally highlighting the ASPE template because it is available through the professional association website and used by many SPEs. Also, of note, the sample case provided is for a summative assessment event. Other case examples for formative activities are provided throughout this book, (see Chaps. 9 Communication Training and 13 Expanding the Field). Below are the 10 parts of the ASPE case template used to develop a case:

• Part 1 – Administrative Details

• Part 2 – Door Chart/Note & Learner Instruction

• Part 3 – Content for SPs

• Part 4 – SP Checklist

• Part 5 – Checklist Guidelines

• Part 6 – Additional Materials

• Part 7 – Post-Encounter Activities

• Part 8 – Note Rubric or Answer Key for Post-Encounter Activities

• Part 9 – Briefing/Learner Orientation

• Part 10 - Debriefing

Many versions of comprehensive templates for case development exist in the literature and on the web. The organization of the content and the number of details may vary, but for learning and assessment activities, the template should be as complete as possible so that all relevant material needed for developing, training for portrayal, scoring and delivering feedback, piloting, and implementing a case are included. Use of a standardized template will make subsequent cases easier to develop and to include all necessary information, and will help to fulfill ASPE SOBPs:

• 2.2.3 Simulation design that meets the purpose.

• 2.2.4 Simulation design that is repeatable.

• 2.2.5 Information for SPs (e.g., situation and backstory, history, affect and demeanor, signs and symptoms to simulate, cues).

• 2.2.6 Training resources (e.g., props, moulage, videos, task trainer).

• 2.2.11 Data for managing the documents and recruiting SPs (e.g., author information, date of development, patient demographics, body type criteria).

You are encouraged to think of this template (and others) as a guide that will require modification to suit the resources and constraints found at individual institutions and the HS application selected. Additionally, templates may be reorganized to suit the flow of the activity planned.

Part 1: Administrative Details

It is critical to document administrative/logistical details necessary for the smooth implementation of learning or assessment activities as part of the case. Such “behind the scenes” information includes the objectives and purpose for the activity, the level of the learner being assessed (including any prerequisites for the activity such as a completion of a clerkship or other classes), recruiting demographics for the SPs (age range, gender, BMI, etc.), and other considerations such as props needed for realism or other necessary simulation equipment. Providing detailed documentation as part of each case will ensure that SPEs understand how to interpret the case and create an event that accurately conveys and addresses the educational objectives being taught or measured.

Administrative details in the ASPE template include the case SP name (avoid the use of humor or distracting information), reason for the visit to the health care provider, and chief complaint (which may be the same as the reason for the visit). It also includes the differential diagnoses as well as the actual diagnosis, (if there is one). If formative feedback is part of the scenario, feedback prompts specified by faculty and/or SPEs should be included (see Chap. 9 – Communication Training). If the case is used to assess clinical reasoning, it should be written so that more than one diagnosis is possible so that learners demonstrate reasons for listing the various diagnoses. The faculty must consider what information from the case would support each of diagnoses on the differential.

Logistical details that impact curriculum including a single faculty contact, assessment instruments, training agendas, and further instructions for additional staff are also critical to include with the case. Although we recommend a committee approach to developing cases for the best outcome, an individual clinician should be assigned to each case so that SPEs can contact one person (rather than the entire committee) regarding questions that arise during training, piloting the case, or on the day of the activity. A list of assessment instruments required (e.g. SP checklist, communication rubric or scale, post-encounter note, quiz) is helpful to use for staff setting up the simulation event so that all required parts of the activity are in place. Without a list, SPEs and staff must guess what faculty intended, which can lead to confusion for all involved including the learners. A recommended training agenda will be covered in another chapter; however, a summary of the amount of training needed (usually between 1 and 2 training sessions for formative activities and 2–4 training sessions for a summative assessment activity if resources permit) and the times and dates for those sessions should be documented for the SPE and SP. [9] If additional materials for training are needed, these should also be listed. Examples include physical diagnosis videotapes for training SPs to score accepted PE maneuvers; textbooks demonstrating anatomy for training responses to the physical examination, or example videos from SPs who played the case previously. These materials are also helpful for training SPs on scenario components including affect, communication skills assessment, and providing verbal feedback to learners. It is also important that SPs are open to and incorporate any case changes and feedback changes since last usage, which should be highlighted when utilizing video from previous events for training purposes. Instructions for additional staff: (e.g. simulation technician, proctor, simulation educator) clarify the role expectations and make explicit the tasks assigned to each staff member.

Part 2: Door Chart/Note and Learner Instruction

Expectations for SP-based activities should not be a guessing game for the learner. Clear, unambiguous instructions are needed for each case, so the learner knows what to expect, and what is expected. Enough information about the patient should be provided so the learner can be prepared before entering the room. Doorway information should include the setting (place and time), the patient’s name, age, gender, and chief complaint. Frequently, vital signs are provided (including pulse oximetry, if desired). Depending on the case, lab or imaging results may be included. In addition to information about the patient and his/her condition, instructions about the expectations of the learner should include the tasks to be completed (e.g. elicit an appropriate history, conduct a focused physical exam) and the length of the encounter (10 minutes, 20 minutes, 30 minutes, etc.). Also indicate if a case task involves a series of complex communication skills such as counseling or motivational interviewing.

Part 3: Content for SPs

The content for the SPs is the essential training material needed to standardize portrayal across your program, and, if applicable, multiple centers and learners. Also, in the case of formative activities, content for the SPs may provide direction on providing standardized feedback to learners on their performance (see Chap. 9 Communication Training). In addition to the expected medical information including the history of present illness, past medical history, family history, and social history, review of symptoms and other symptoms, aggravating and alleviating factors, and responses to physical examination maneuvers, the SPE and the SP need to know about the background and personality of the patient being portrayed and how to respond to questions about psychosocial aspects [11]. The material must contain details about how the SP should respond to jargon and how to disclose information (i.e. what is volunteered or specifically asked for and responses to multiple questions). Specific verbiage that must be memorized such as the opening statement/line and standard challenges that will be presented to each learner must be stated clearly. Since teaching about communication skills includes the frequent use of open-ended questions, SPs need responses to these in order to demonstrate that SPs are portraying fully developed patient characters that will share more if learners are curious and show interest in their lives. Ideally, SPEs should include SPs in crafting responses to open-ended questions during training that are appropriate to the medical details and background information provided in the case (see Chap. 8 Ten Step Training framework). SPs will often think of natural and creative responses to open-ended questions from the portrayal or performer perspective because they are preparing to bring the case to life. In response to “tell me more,” the SP should not be left to say, “What do you want to know?” The unintended consequence of this will be learners using a “shotgun” approach to history-taking, asking lots of closed, focused questions.

The case content for the SP should be written from the SP’s point of view to help the SP to learn the role (e.g. “I felt the headache come on about an hour ago” vs “your headache started an hour ago”). The language used should be comparable to the vocabulary the patient would use, (e.g. free of medical jargon and in keeping with their education level, personality, and communication preferences). This is an area where SPs and SPEs can make a big impact while working with clinicians, acting as “translators” from “medical-speak” and in providing suggestions as to how patient characters would sound and what they might say.

The content for the SPs should be geared toward providing stimuli for the objectives being assessed. In other words, if part of the construct being evaluated is empathy, the case should provide the SP with emotional material to facilitate providing learners with opportunities to demonstrate empathy toward the patient. This should be trained so that each SP provides each learner with the opportunity to demonstrate empathy— in a standardized fashion during each encounter. It is important to note that standardizing emotional responses from SPs does not imply generating the same exact expressions or tone of voice for each learner, but rather providing authentic responses rooted in who the patient is (see Chap. 9 Communication Training).

An important consideration is the ability of the SP to maintain the affect and behavior required for the period of time needed for the assessment. For example, playing a very anxious person or a depressed person for the course of the day can be extremely taxing physically and psychologically, as would simulating some conditions like shortness of breath. (See Chaps. 7, 8 Training SPs and 9 Communication Training). However, SPEs must appreciate why it is important to consider SP training, the impact on the SP, and the educational objectives when creating case content.

Tips for Scenario Development

Some helpful hints when developing scenario materials include:

• Matching the learning objectives to the case. For example, a case involving acute onset abdominal pain may not provide the material for assessing the examinee’s ability to perform a neurological examination.

• Consider the time required for the examinee to complete the expected tasks. A complete, thorough neurological examination and history cannot be performed by a novice health professions learner in 10 or 15 minutes. Enough time must be allotted for the tasks being assessed.

• SP responses and portrayal must be realistic. This means that cases must be written to be believable to aid the learner in more fully experiencing the SP encounter as they would a clinic or hospital patient encounter. Write the case from the patient’s perspective and how the patient would use medical terms. The use of formal language from pediatric or adolescent patients, or use of the term “fatigue” to describe tiredness or “radiate” to describe pain moving to another body part in a patient with low health literacy detracts from the realism of the experience, and may result in a negative impact to the learner’s performance.

• Include some diversity in the SP’s story. Not every patient drinks only a glass of wine on special occasions or is in a 25-year monogamous marriage. Likewise, avoid stereotypes like unmarried elderly librarians with cats.

• Balance the psychosocial information to provide enough detail to answer most common questions, without overwhelming the SP with too much minutiae that may never come out in the encounter. The cognitive load on the SP should be considered, particularly if they are expected to memorize a checklist for scoring purposes. Also, consider that SPs enjoy engaging in the training process as creative and may, if appropriate, help co-write their own patient character details outside of the realm of the medical information or any informative relevant to the checklist(s).

• Adjust the timeline for various times of day. Cases such as abdominal pain that comes on suddenly after a meal will need details for various timing throughout the day. The information provided to the SP should not hinge on a single meal that might be 12 hours removed. Learners should not be asked to “pretend” that it’s a different time of day than it really is; learners may struggle with pretending to adhere to case details such as time of day that are variable, and cases should be made as realistic as possible to address this problem.

Part 4: SP Assessment Instruments (Checklists)

The following applies to ASPE SOBP 2.2.9 Evaluation instruments and performance measures (e.g., checklists and rating scales, participant and facilitator evaluations).

The key element to SP-based assessment is checklist design, which should be based on educational objectives. Checklist items must be clear and unambiguous, stated in terms easy for raters to understand and to recognize, especially if SPs are completing the checklists. The tasks should be listed in a logical sequence from start to finish. Too many, too few, or nonessential items may produce scores which are meaningless in assessing clinical competence on a particular case. A classic article by Vu et al. [12] showed that increasing the number of checklist items adversely affects the accuracy of SP recordings and suggests limiting the checklist to between 15 and 20 items [10].

As with developing the case itself, it’s imperative that checklists be developed and reviewed by other faculty. Checklist items must be evidence-based to avoid items based on “tradition” or personal favorites of individual faculty. For example, standing on the right side of the bed or percussing the heart borders may be something some clinicians still do, but have little evidence to support the appearance on a checklist. Texts such as Magee’s Evidence Based Physical Diagnosis or JAMA’s Rational Clinical Examination are useful references for developing checklists [13, 14].

Checklists may be dichotomously scored, such as “yes/no” or “done/not done” or “asked/not asked.” For more detailed feedback, particularly with physical examination maneuvers, letting the learner know when they did attempt a maneuver but failed to execute it effectively provides important information for remediation more than “done/attempted/not done”.

Many physical examination items will be used repeatedly for developing future cases. In order to standardize checklist items across cases, it’s helpful to develop a database of physical examination maneuver items. These items can then be reused and distributed to all teaching faculty and learners so there is transparency about what will be assessed. Another approach is to standardize checklists across types of cases, so there is congruency with what is expected by the learners.

With regard to how the raters complete the checklist, we often think of SPs memorizing what happens in the encounter and completing the checklist afterward, but also consider the idea of a “real time” observer (e.g. faculty member, preceptor, or subject matter expert), or someone who will watch a videotape later on to complete the checklist.

Tips for better checklists include:

• Items should be behaviorally focused (describe the examinee’s actions). If items aren’t behaviorally focused, they are not observable for the SP or rater, and therefore make the rater’s job impossible. If you can’t make an item observable, the checklist becomes more about the rater’s opinion than the learner’s performance. The construct “empathy” is a good example. What does empathy look like? An item like “displayed empathy” leaves it up to the rater’s judgement; however, with training, a rater could recognize the item “made an empathic statement.” Professionalism is another construct that means different things to different raters, for example, was the learner’s white coat clean, did they wear sneakers, did they introduce themselves as a learner? Each behavioral aspect to be assessed needs to be defined in order to minimize bias and subjectivity from raters.

• Items should contain only one task. Lumping two or more tasks into one item makes scoring difficult for any rater who is completing the checklist. For example, if the item is “checked my pulse and my blood pressure” but the learner only does one of these; how is this scored? It also makes it difficult to analyze performance data (did the learner forget to check the pulse or did they forget the blood pressure?).

• Items should only include physical examination maneuvers raters can observe.

  For example, checking respiration can be done discreetly by a learner, so raters may miss this. Unless the learner is doing something concrete that the rater can witness, it should not be a checklist item. Also, it is vital SPEs ensure camera placement in their simulation center is optimal to capture all observable checklist behaviors. For this reason, is important to check the cameras prior to each simulation activity.

• Items in the scoring instrument should not be answered or addressed in the pre-brief, orientation, or any information provided to learners prior to starting the simulation. Learners are less likely to verify or demonstrate content that has been provided. For example, if you want the learner to take the blood pressure, do not put the blood pressure reading on the doorway instructions. If you want the learner to find out about the onset of the problem, do not put that the pain started yesterday on the doorway instructions.

Part 5: Assessment Instruments Guidelines

The following applies to APE SOBP 2.2.10 Training protocols for raters (SP or other).

Assessment instruments are an essential component of scenario development, and SPs and other raters need to be thoroughly trained to complete all assessment instruments. Approaches to training will be covered in other chapters, but the need for rater training cannot be over emphasized. Some have the false belief that because the faculty member is a healthcare professional, no preparation or training for scoring is needed, as these individuals can rely on “expert judgement.” Healthcare professionals and SPs are first human, with all the unconscious bias and personal preference any person has. Those who expect faculty members to arrive to an assessment without prior preparation for accurate scoring will not be able to rely on the resulting scores. Everyone rating the performance of learners (whether a faculty member or an SP) need thorough training in the use of the assessment instruments including familiarity with the guide to these instruments.

The purpose of assessment guidelines is to make explicit what actions on the part of the learner will receive credit by the SP or other rater. In doing so, one hopes to address any potential problems affecting interrater reliability, including personal bias. In other words, two raters looking at the same learner behavior need to be trained to rate the behavior in the same way. Therefore, creating clear assessment instruments (e.g. checklists or other rubrics) is an essential part of scenario development.

One approach to creating checklist guidelines is to approach each item thinking of as many ways as possible a learner could receive credit for the item. For example, the educational objective is “The learner will demonstrate the ability to ask a patient about sexual activity.” The checklist item, if scored by the SP, is written in lay language, and the guide to the checklist gives examples of questions learners could ask that would receive credit for the checklist item.

• 1. I have sex with my husband. (Checklist item written in lay language)

  • Are you sexually active?

  • Do you have sex with men, women or both?

  • How many partners do you have?

The three examples above are intended as just that; examples, not as an all-inclusive list. It is impossible to think of all the novel ways learners come up with questions but listing a few gives the rater an idea of what should receive credit.

For physical examination checklists, each item needs a thorough explanation as to what would constitute an observed behavior. Palpation of the liver may have many approaches, and each need to be listed in the guide to avoid penalizing learners who may use a different yet acceptable maneuver. Although time-consuming up front, the development of a physical examination database containing all acceptable approaches to each part of a complete physical examination can be used for teaching, self-study, and assessment. Commercially available videotapes or institutionally made ones can be used to supplement the text for each maneuver.

Part 6: Additional Materials

Standardized patient cases can be supplemented by additional materials such as previous health records, X-rays, lab results, lists of medication, or photographs. Some institutions use cards to list abnormal findings such as high or low blood pressure (these cards can be handed to the examinee after the blood pressure is taken). These materials should be listed on the case template so that the SPE responsible for setting up the assessment knows to include them.

Part 7: Post-encounter Activities

It is important to include any post-encounter learner activities in the scenario development process. These activities may include writing a patient note, ordering labs or imaging, or answering multiple-choice questions. In addition to supplementing the assessment of the encounter, these additional learner activities enable SPs to complete assessment instruments during the same timeframe. For formative activities, the post-encounter learner activity may be a written reflection, or a debriefing or feedback session conducted by the SP, SPE, or faculty member.

Currently, the USMLE Step 2 examination requires a typed patient note after each standardized patient encounter. The note consists of data gathering (listing history and physical findings from the encounter) and data interpretation (listing the differential diagnoses in order of likelihood, with support from the history and physical findings and listing any tests for follow-up, if indicated). Examinees have 10 minutes to complete the note. Examples can be found at the USMLE website [15].

Part 8: Note Rubric or Answer Key for Post-encounter Activities

Similar to the guide to the checklist, any post-encounter activity requires a rubric, or answer key, for those responsible for scoring. The committee drafting the scenario materials will need to spend time ensuring the key for scoring is clearly written and unambiguous. For example, using the sample format provided by the USMLE on the website, the committee would complete the rubric for each case by including the answers expected for each section of the post-encounter note (e.g. listing the important aspects of the history, physical examination, diagnoses with supporting evidence from the history and physical and diagnostic tests, if any). The importance of this step is to standardize the scoring across raters.

Part 9: Briefing/Learner Orientation

Briefing can be done before any simulation activity. Briefing follows the ASPE SOBP 2.2.8 Briefing instructions, time frames, instructions to learners.

Often overlooked, the orientation for the learners is vital so that they know exactly what is expected. Participating in simulation activities may be stressful and may interfere with the learner’s ability to perform. As a result, consider giving written (or electronic) instructions at the beginning of a course, a few days or a week prior to the event, and again an oral (or videotaped) presentation immediately prior to the start of the activity. This information should include the format and timing, session objectives, and any special instructions (e.g. bring a stethoscope, what to wear, etc.). Like the doorway instructions, the purpose of the briefing orientation is to make explicit what is expected so there are no surprises for the learers.

Part 10: Debriefing

Debriefing can be done after any simulation activity. Debriefing follows two Domains from the ASPE SOBPs: 1.1.8 Structure time and create a process for de-roling and/or debriefing; 2.2.7 Case-specific feedback or debriefing guidelines.

A variety of approaches can be utilized for group debriefing or individual debriefing. The term “feedback,” for the purposes of this chapter, is used when the SPE or SP provides specific information about performance with the intent to modify thinking and/or behavior to improve future performance. SPs may be trained using a template or a rubric to give constructive feedback to the learner after the encounter according to the objectives being assessed. Frequent monitoring to ensure SPs are providing constructive feedback consistently to each learner is recommended. Monitoring is ideally done by SPEs but may be done by other SPs trained on the same case (see Chaps. 7, 8 on Training and 9 on Communication Training).

A carefully planned approach including the time allotted for the activity should be used to standardize the format for all learners.

Scenario Development: Interprofessional Scenarios

All of the general considerations noted above apply to interprofessional scenarios. However, any template for interprofessional simulations (sometimes called “interprofessional simulation-based education”, or IPSE) [16] should address two sets of criteria: (1) How the scenario relates to the most widely accepted definition of interprofessional education; and (2) How the scenario relates to the Interprofessional Education Collaborative (IPEC) competencies (particularly in the US) [17, 18].

Interprofessional education is most commonly defined as “two or more professions learning about, with and from one another” [17]. While health science education may involve one of these activities (for instance, sharing the scope of practice or ethical standards of a licensed pharmacist with medical learners), they seldom involve all levels of learning about, with and from one another. Simulation plays a critical role in interprofessional education precisely because it is difficult to meet these criteria without it. Learners from two different professions taking the same pharmacology course does not meet the criteria for interprofessional education if they are not learning how each profession might use the same content. However, learners who learn how to apply didactic content, using their knowledge of pharmacology in the shared care of one or more patients or clients in a real or simulated context, are engaging in interprofessional education. Learners who participate in high-quality IPSE are better prepared for clinical education, maximizing their ability to participate actively in interprofessional healthcare teams and accelerating their time-to-competence [19, 20].

As with all simulations, successful Interprofessional education (IPE) should be scaled to the learners’ general level of capability. A typical model is the University of British Columbia’s “exposure, immersion, mastery” model of skill development. Simulation education can be wrapped around this educational model to create a comprehensive approach to interprofessional development (Fig. 6.2).

Fig. 6.2

IPE simulation model at the University of Minnesota. (Reproduced with permission of the University of Minnesota)

Similarly, IPSE should consider how the content of clinical scenarios meets IPEC competencies. The IPE Collaborative was created by several health science education accreditors in 2009 to achieve consensus on how to assess interprofessional education in their respective health science professional school curricula [18]. Specifically, the IPEC Competencies address four domains:

• I. Values/Ethics for Interprofessional Practice

• II. Roles/Responsibilities

• III. Interprofessional Communication

• IV. Teams and Teamwork

Each domain includes multiple competencies that should be taken into account when creating SP scenarios and assessment tools. Looking at Domain III Interprofessional Communication, a performance assessment may include: (Table 6.2) [18].

Table 6.2 Domain III Interprofessional Communication. General Competency Statement-CC Communicate with patients, families, communities, and professionals in health and other fields in a responsive and responsible manner that supports a team approach to the promotion and maintenance of health and the prevention and treatment of disease

Since different models of interprofessional practice may be unfamiliar (or not transparent) to SPs, they may require additional training or education to successfully assess individual or team performance. For instance, what would Competency 1, “Choose effective communication tools and techniques” look like in a labor and delivery scenario with nurse-midwives and OB-GYNs? With the development of successful, inclusive clinical models, such as Team Birth, SPs are challenged to evaluate performance that goes beyond traditional models of team care. Using Team Birth objectives, successful completion of this item might include a birth plan on a whiteboard.

Even though standardized patients may not have had an experience of a delivery that included the laboring mother and birth partner, a nurse-midwife, and an OB-GYN as a team, with training, they would still be able to determine if a birth plan was shared and communicated to all team members (who are likely going in and out of the patient’s room at different times) “using effective communication tools and techniques.”

One team care case for social work and family nurse practitioner learners provides a good example (see Table 6.3). Shared patient interviews and team care benefit patients (who only have to tell their story once) and providers (who hear the same history and can ask follow-up questions in real time). This model of team care is particularly appropriate for complex patient cases that involve both mental health and physical health findings. However, shared patient interviews can create challenges. Providers from different professions can find it difficult to negotiate competing and overlapping goals in patient interactions and coordinate their individual interactions with patients/clients. Patients who are unfamiliar with team interviews can be confused about the role of each provider and without coordination, can feel that they are being interrogated rather than cared for. Sharing the expected competencies with simulated patients/clients helps them to better understand the educational goals of the simulation, the authentic challenges of real patients in these kinds of interactions, and how best to give feedback to the learners in the simulated encounter.

Table 6.3 Case for social work and family nurse practitioner

Finally, training materials – especially guidelines for SPs giving feedback – should take into account best practices in IPSE and interprofessional facilitation [21, 22]. The value of interprofessional simulation is not only in exposing learners to different professional perspectives, but in providing them with an opportunity to experience tension and productive conflict in the practice of patient and client care. While SPs are not content experts, training them in the same facilitation techniques as clinical faculty adds an important element to the quality of learner experience. In debriefs, learners often defer to feedback they receive from patients, and find it at least as credible as feedback from faculty [23].

Scenario Development: Simulated Families and Groups

Developing scenarios for individuals requires careful attention to the medical history, condition, and affect of the SP or client; developing scenarios for simulated families adds a new layer of complexity. Simulating families means simulating relationships under stress, depending on the scenario. For instance, end-of-life scenarios demand a range of emotional responses, including deep sadness, regret, guilt, fear, anxiety, and anger [24]. Developing the specific strategies and scripting to express these emotions, realistically simulate the deep intimacy and historicity of family relationships, and replicate those multiple times in the course of a simulation requires time, rehearsal, careful preparation, and recalibration of performances.

Other types of groups (such as victims of a mass casualty incident) also require careful coordination and choreography. While they do not necessarily require a simulated familiarity, they do require realistic responses (such as dissociation, terror, or voyeurism) to crisis situations. Structuring those responses – and coordinating them with any medical or psychological challenges built into the scenario – necessitate specialized training techniques [25].

Scenario Development: Hybrid Simulations

SPs and SPs as embedded participants (EP) (individuals who portray a role in the scenario in order to ensure successful execution) add substantially to the realism of hybrid scenarios but face special challenges in working with partial task trainers and mannequins. Standardized patients need to be trained in the operation of partial task trainers, mannequins, and other equipment necessary to the implementation of the scenario. A standardized patient practicing with wearable birthing trainer prior to simulating an obstetric emergency (with another SP as her embedded participant birth partner) in a hybrid simulation (see Fig. 6.3).

Fig. 6.3

A standardized patient practicing with wearable birthing trainer in a hybrid simulation

This training should include troubleshooting with devices so they can assist with problem-solving if malfunctions should occur during implementation (see Fig. 6.3). While this sometimes requires the intervention of a technician, an EP or SP can improve the experience of learners even in a sub-par simulation encounter.

Additionally, EPs and SPs also require coordination in their performances. Like group scenarios, this may include close, familial relationships, but it may also include more distant, even adversarial relationships when SPs are portraying healthcare professionals or an intrusive element.

Scenario Development: Patient-Centered and Patient-Driven Simulation

The development and implementation of patient-centered and patient-driven simulations offers enormous benefits to learners, practitioners and, of course, patients and patient’s families. However, these simulations pose unique challenges for the SPE, crossing boundaries between fiction and fact, and potentially, education and therapeutic intervention.

Part of the challenge lies in understanding the definitions of “patient-centered” and “patient-driven” simulation [7]. As noted in Arnold et al., in patient-centered simulation, “patients’ views, needs, and goals for education are the focus of the simulation as opposed to the needs of a program of study or a healthcare professional group” [7 p.S51]. SPEs are uniquely positioned to serve as translators, ambassadors, and advocates, bridging the gap between providers and patients, systems and families. SPEs day-to-day work – moving fluidly between the learners learning needs, faculty objectives, and the realities of patient experience – offers a perspective that has no analog in healthcare education.

Just as we cannot assume that a “team of experts” will make an expert team, we cannot assume that content expertise will lead to an “expert case” [26]. Yet, we have no criteria for optimal simulation cases or optimal case development. Just as scholars have established how much “realism” is required for effective simulation, so we have the opportunity to try to establish how much specificity (and what kind) is required for an authentic case designed for meaningful learning. Patient-centered and patient-directed simulation offers intriguing opportunities for research in this arena. SP/EP scenario development can become more of a “co-production” by patients/clients and educators [27] and less of a “mirror for the teachers’ preconceptions” [28].

Summary

Scenario development and training techniques must be responsive to the specific needs of clinical context experts and learners. As healthcare systems around the world change, our case development and training techniques need to change with them. The fundamental principles of scenario development and training noted above should apply to all SP-based simulations, but there are special concerns and considerations (consistent with the ASPE Standards of Best Practice) in simulations that apply to newer models of care, innovative medical technologies, and/or emerging simulation techniques. These include: patient-centered/patient driven simulation; hybrid simulations combining live role players with wearable task trainers; simulated families or groups; and most importantly, interprofessional simulation. As more healthcare professions require interprofessional education and refine their standards for program accreditation, these recommendations will take on added importance.

Scenario development is a team process, beginning with the educational objectives for the activity. The teaching faculty—in partnership with other clinicians, SPEs, SPs, and community members from underrepresented groups—should be the ones involved in designing the formative and summative activities for the skills taught. This provides a feedback loop on teaching effectiveness while avoiding pitfalls surrounding one person’s opinion or personal preference. Good scenario development takes time, is an iterative process, and should be done on a routine basis in order to develop the skills of the participants working as a team and to increase the size of the case bank for security reasons.

The use of a template will provide a standardized approach to providing everything the SPE needs to implement a summative assessment of clinical skills. Pilot testing of cases must be done using volunteer learners, residents, or faculty members unfamiliar with the case, but with similar experience to the learners with whom it will be used, to see how the case will perform prior to use. Then, the case writing committee should make any needed adjustments to best meet the educational objectives.