Can Virtual Humans Teach Empathy?

Abstract

Millennial and generation X learners seek and accept technology-based teaching. In healthcare, such technology includes virtual standardized patients. Virtual human technology was created and tested to teach communication skills, including empathy, in patient encounters. With the help of learners as content experts (dental, medical, speech pathology, and pharmacy students and faculty), our team developed virtual standardized patient scenarios related to neurology, surgery, mental health, and other medical specialties, to offer practice opportunities for history taking, critical thinking, and empathic communication. We will describe the virtual human technology, including its properties suited to provide immediate empathy feedback in real time, to teach and reinforce empathic communication longitudinally, and to reach a large number of learners who can practice and reinforce the skill repetitively and cost effectively.

1 Virtual Patients Overview

1.1 What Are Virtual Patients?

As health educators search for new ways to assist learners in developing empathy skills, virtual patients are arising as a potentially transformative educational tool. The term virtual patient has been used to describe a variety of systems, which can potentially lead to confusion [1]. In this chapter, we focus on virtual standardized patients, which Kononowicz et al. [1] describe as “a virtual representation of a human being using artificial intelligence technologies and natural language processing to train communication skills.” Figure 9.1 shows an example virtual human patient, named Vinny. Vinny is a 3D computer simulation modeled from a traditional standardized patient and presents a patient that has difficulty swallowing (dysphagia). Vinny was created for speech pathology learners to practice patient communication. Vinny can be accessed using any internet-connected computer. The learner can speak or type to Vinny about topics related to his dysphagia. Vinny responds through speech and gestures. The learner gets real-time and post-experience feedback on their interaction, including how they did on opportunities to empathize with Vinny. Virtual standardized patients involve an input component composed of sensors to capture user behavior, a cognition component composed of algorithms to understand and respond to the user’s inputs, and an output component that includes hardware, displays, audio, and haptics that present the system’s response [2]. In this chapter, we will discuss research and advances in using virtual standardized patients, such as Vinny, to teach empathy to healthcare learners.

Fig. 9.1

Virtual patient Vinny DeVito

1.2 Trends in Development of Virtual Standardized Patients

Improving communication skills using virtual standardized patients is a relatively recent research area, with most work occurring since 2005 [1]. From 2005 to 2011, isolated trial studies aimed to understand the capabilities and limitations of such a platform. Short (usually less than 30 min) virtual standardized patient interactions were created to mimic a standardized patient encounter around a single disease. The use of virtual standardized patients was limited to collaborations between researchers and educators [1]. Since 2005, the research into virtual standardized patients has focused on increasing the realism of the interactions, developing effective feedback, and studying the educational benefits. Research has explored how people interact with virtual human patients, noting similarities and differences when compared to interacting with standardized patients [3, 4]. Researchers have studied how the physical system (e.g., display size) used to present the virtual human impacts the interaction, with a general understanding that increased levels of immersion (e.g., larger displays, immersive displays, higher quality audio) would create a more believable interpersonal interaction [5, 6]. While earlier work into virtual patients examined how they should be presented, including benefits and costs of using different display modalities, today’s work attempts to understand the optimal placement and the learning impact of virtual patients in a curriculum [7].

1.3 Current Manifestations of Virtual Standardized Patients

Most virtual standardized patients in large-scale educational use are presented on personal computing devices, such as tablets, laptops, and desktops. Virtual standardized patients are accessed via the internet, usually with a standard web-browser. Laptops or desktops and web-browsers are used due to the goal of widespread dissemination with limited hardware and software requirements. Repositories of virtual standardized patients are available for educators to augment course instruction. Neurological Examination Rehearsal Virtual Environment (http://nervesim.com) is an example web-based platform that presents virtual standardized patients presenting with double vision due to cranial nerve conditions. The American Association of Medical Colleges’ peer-reviewed open journal, MedEdPORTAL (www.mededportal.org), hosts a number of virtual patients that can be downloaded for free for personal or institutional use [8, 9]. Commercial systems such as Shadow Health, Inc. https://www.shadowhealth.com/ (co-founded by Benjamin Lok), vSim https://www.laerdal.com/us/vSim, and iHuman http://www.i-human.com/ are similar platforms currently in use in thousands of nursing and medical school curricula worldwide.

Virtual People Factory (VPF) [2] is a virtual human platform that enables medical educators to author and disseminate virtual standardized patient scenarios. In VPF, progress through the clinical scenarios is based exclusively on the active interaction between the user and the virtual human [2, 10]. VPF supports multiple input and output modalities and stores de-identified log files of every interaction in a database. Users can use any combination of speaking, typing, writing, gesturing, eye gazing, and body movement interacting with the virtual character. VPF allows for rapid authoring, multiple interactive input and output modalities, ability to leverage commercial artificial intelligence and machine learning platforms, and scalability. VPF provides a web-based portal that enables scenario authors (i.e., clinicians, educators, or healthcare trainees) to create a clinically accurate dialog with a virtual patient and provides feedback to the authors on overall usage, user pathways through the script, and opportunities for scenario improvement. The active interaction is rendered in the Unity3D game engine, which presents realistic animated virtual humans that can speak and gesture to the user on a variety of platforms including web-browser, mobile phone, tablet, virtual reality head-mounted displays, and large projection screens. VPF uses an un-annotated corpus retrieval approach (i.e., uses natural language approaches to find a list of corpus stimuli that are most similar to the input stimulus) [11]. VPF can pass the user’s input into artificial intelligence (i.e., natural language processing, and machine learning platforms) as to determine the best-authored response to present back to the user. VPF is hosted on Amazon Web Service, a secure, scalable cloud platform that allows thousands of simultaneous users, while ensuring HIPAA and FERPA-compliant level of security.

Depending on the virtual patient system, the response is captured as words, audio, and/or the user’s body language (e.g., using a web camera or Microsoft Kinect). Further, researchers investigated touch from human to a virtual human. Kotranza et al. [12] enhanced the communication of a mixed reality human (a mannequin instrumented with a co-located virtual human) with sensors that detect the user’s touch and elicit speech, gestures, and facial expressions from the mixed reality human. The authors found that the virtual human-to-human touch was used for the same communication purposes as human-to-human touch and enhanced the learners’ communication with the virtual human.

As presented in Fig. 9.2, once the learner observes the virtual standardized patient, asks questions, and is provided with responses by the virtual patient, a transcript of each interaction is generated. Each learner can read the transcript immediately after the end of each interaction. Furthermore, an instructor can retrieve the collective class transcripts and offer feedback to the class. Virtual standardized patients deliver content by means of a typical clinical case (e.g., chief complaint, history of present illness, medical, surgical, psychiatric, social, developmental, legal history, current medications, allergies, physical examination results, and test results). In addition, virtual standardized patients offer feedback by means of presenting the elements of history elicited by the learner during the interaction, in real-time, on the interaction screen. This way, the learner can monitor the progress of the interview (see “discoveries” illustrated in Fig. 9.3), as well as the full transcript of the interaction, available immediately after the interaction is completed [13]. The post-interaction feedback can be customized with a formal case presentation, and in some cases immediate feedback on the interviewer’s clinical or communication skills [14,15,16].

Fig. 9.2

Virtual standardized patient flow

Fig. 9.3

Example of VP interaction and immediate feedback on interview progress and symptoms elicited during the interview (discoveries)

In medicine, virtual standardized patients are described as multimedia interactive scenarios that allow safe practice and repetition and immediate feedback, help develop clinical skills, and can simulate rare but critical scenarios [7, 9, 10, 13, 17]. While classroom-based and online courses can effectively deliver course content, they cannot provide personalized learning for each trainee. Objective, individualized feedback given to each trainee is crucial for learning [9, 18, 19].

Virtual standardized patient development has expanded alongside the development of standardized patient simulation. A simulated patient encounter includes any medical encounter conducted for purely educational purposes that may or may not utilize the simulator’s personal medical history. The standardization occurs when the simulated patient offers consistent response content independent of the learner [20]. The virtual standardized patients and standardized patient actors both offer content standardization and share many other characteristics, while they differ in some areas, as illustrated in Table 9.1.

Table 9.1 Characteristics of virtual standardized patients and standardized patient actors

2 Why Virtual Standardized Patients?

2.1 Motivation for Virtual Patients in Medical Education

Virtual standardized patient use has grown due to concerns of safety and expense related to standardized patients, emphasis on self-learning, and expansion of the distributed education model, calling for standardization of instruction for learners enrolled in the same educational program on distant campuses.

In medicine, virtual standardized patients are described as multimedia interactive scenarios that allow safe practice and repetition and immediate feedback, help develop clinical skills, and can simulate rare but critical scenarios [7, 9, 10, 13, 17]. While classroom-based and online courses can effectively deliver course content, they cannot provide personalized learning for each trainee. Objective, individualized feedback given to each trainee is crucial for learning [17,18,19].

2.2 Where Do Virtual Standardized Patients Fit into Medical Education Curricula?

In their early evaluation, Cook and Triola [22] proposed that virtual patients are best utilized to develop clinical reasoning skills, due to their versatility in depicting a mix of cases for deliberate practice and ease of standardization. Berman and colleagues [10] defined virtual patients as “an interactive computer simulation of real-life clinical scenarios for the purpose of healthcare and medical training, education, or assessment”. Berman and colleagues [10] saw additional advantages of virtual patients for learners: (1) expansion of medical knowledge through interactive learning, (2) mobilizing learner’s intrinsic motivation to learn, (3) applying foundational knowledge, and (4) ability to focus on specific competencies. For educators, virtual patients offer unique opportunities to analyze educational data [10]. We successfully used virtual standardized patients to teach history taking, diagnostic reasoning, empathic communication [4, 5, 8, 14, 23], as well as suicide risk assessment [9, 17]. For example, a virtual patient which exposes learners to an actively suicidal patient ensures that a large number of learners (e.g., hundreds, in most medical schools) acquire basic suicide risk assessment skills before interviewing real patients [9].

2.3 Strengths and Limitations of Virtual Standardized Patient Technology

2.3.1 Strengths of Virtual Standardized Patients

More recently, commercial virtual standardized patient systems are available and in widespread use within some healthcare domains. For example, over one third of graduate nursing schools use virtual standardized patients in their courses. Educators have published broadly on their experiences with virtual standardized patients, learner perceptions, improvements in learner skills, and approaches to curricular integration [24,25,26]. Among areas covered by virtual patient simulation are on-demand, repeatable practice of clinical skills, clinical reasoning, critical thinking, communication, and decision-making in a broad psychosocial context [27]. As illustrated in Fig. 9.4, virtual standardized patients are customizable and can be created to present different races, ethnicities, gender, backgrounds, beliefs, personalities, and behaviors. Virtual standardized patient systems can provide individualized real-time and post-experience feedback on the learner’s performance. Individualized feedback is critical to improving communication skills, and virtual standardized patients enable educators to provide such feedback to large classes. Since deliberate practice and feedback are thought to be essential in developing expertise [28], including repeated opportunities for users to practice certain skills (e.g., history taking, suicide risk assessment or empathy), with no risk for standardized or real patients is an important advantage offered by virtual patients. Finally, virtual standardized patients are inexpensive relative to providing communication skills training through standardized patients or clinical hours at hospitals and clinics. The costs of commercial virtual patient systems are aligned with the cost of textbooks. Thus virtual patients are widely utilized in undergraduate and graduate nursing programs [7]. Widespread adoption into medical student training has been slower. Possible reasons for this disconnect include limited breadth of virtual patients compared to the wide needs of the educators and relatively limited evidence of the virtual patients creating effective learning. Further, the lack of a uniform system for financial sustainability of virtual patients interferes with their adoption [10].

Fig. 9.4

Examples of virtual patient appearance

2.3.2 Limitations of Virtual Patients

Virtual standardized patients have limitations that affect their efficacy to teach communication skills. The primary (and obvious) limitation is that communicating with a virtual human is different from communicating with a human. A virtual standardized patient interaction involves both technical and psychological factors that are different from a human interaction. In conversations, such as a nurse or doctor talking with a patient, all the participants use both verbal and nonverbal cues. Subtle cues, such as tone of voice, touch, body posture, and eye gaze are difficult to reliably track and process by currently available computing systems. Research in this area is currently underway. Reeves and Nass [29] showed that people are polite to computers, react differently to computers with female voices than those with male voices and feel that large faces on a computer screen invade their personal space. Overall, however, computer-generated signals elicit social reactions that are similar to reactions people have to other humans [29]. These properties translate to the virtual human domain and offer a foundation for using virtual humans to elicit responses from users and teach empathy [23, 30]. Previous work has shown that learners react with virtual standardized patients similarly to standardized patients, discuss similar topics, and demonstrate similar levels of communication ability [3]. While research into systems that can capture and track the learner’s verbal and non-verbal cues is progressing [31], many virtual standardized patient systems have the learner choose a question or statement to the virtual patient from a predefined list, as opposed to having the learner type questions, or convert the learner’s speech to text that can be processed. Thus, although interactive, some virtual patients solely target medical content recognition as opposed to active communication. This approximation of interpersonal interaction can result in some learners’ rationalizing that their performance was due to “talking to a computer” and not a reflection of one’s true communication abilities. Having educators frame, the virtual standardized patient experience within the learner’s educational experience and clearly delineating performance expectations will help mitigate some of the limitations of virtual standardized patients.

2.4 The Future of Virtual Standardized Patients in Healthcare Curricula

As allied health educators contend with increasing enrollment and clinical hour requirements, often with limited personnel and access to hospitals and clinics, the curricular need and market for virtual standardized patients will increase. Licensing boards have also established guidelines for replacing clinical hours with simulators (ranging from high-fidelity manikins to screen-based systems). Modernizing healthcare curricula includes a focus on self-learning. In this regard, virtual patients can have a role in “flipping the classroom” and teaching important clinical concepts from the comfort of the learner’s laptop [32]. Finally, virtual patient simulations have made their way into high-stakes licensing examinations in various healthcare professions [33].

3 Teaching Empathy with Virtual Standardized Patients

3.1 Can a Virtual Patient Be Used for Empathy Training?

Empathy is a complex phenomenon with affective, cognitive, and behavioral components [34, 35]. Healthcare providers’ support and empathy allows patients to express medical concerns, decreases anxiety, increases treatment adherence, and improves treatment outcomes [36,37,38,39]. Empathy is a fundamental communication skill for healthcare providers that includes understanding the patient’s perspective, communicating that understanding verbally and non-verbally, and acting therapeutically on that understanding [40]. Empathy is taught primarily in live communication skills workshops and using patient shadowing, narrative medicine, and wellness programs [41, 42]. With respect to empathy in particular, virtual standardized patients have been demonstrated to elicit learners’ verbal empathic responses [4, 14, 16, 23]. Accordingly, there have been attempts to correlate patient-rated empathy with clinician’s non-verbal empathy cues, including facial affective mirroring of the patient. Deladisma [4] coded learner interactions including eye gaze, head nod, body lean, and empathy towards virtual standardized patients and real standardized patients with a 4-point anchored scale. While the head nod and body lean were significantly more pronounced towards the standardized patients, learners displayed empathy towards the virtual patients and learners’ verbal empathy correlated with non-verbal communication. Challenges remain, however, when the learners try to express empathy to the virtual human, which may not be completely equipped to detect the verbal empathic responses and appropriately validate the learner’s attempt to relate.

3.2 Integrating Empathic Opportunities in Virtual Patient Technology: State of the Art

At the core of the virtual human empathic communication training are virtual patient scenarios with built-in empathic opportunities, which require the trainee to recognize and respond to the virtual human’s concern [14, 43]. Currently, most systems are able to provide opportunities for the learner to respond with empathy, such as having the virtual standardized patient speak to the user about a sensitive topic. For example, the virtual standardized patient could say, “I’m scared this could be cancer. What if this is cancer?” [12]. The learner has an opportunity to first identify that this is an opportunity to express empathy, and then he/she can respond. Algorithmic interpretation and evaluation of empathy is still in its nascent forms. Most systems simply provide a simplistic response to the user (e.g., “Thanks for saying that”), regardless of actual content. While not fully realistic, such an approach can cue the user that the system recorded their response as empathic or supportive and evaluated it positively.

3.3 Can Virtual Standardized Patients Evaluate Learner’s Empathy?

The initial approach to quantify empathy in virtual patient interactions included human evaluation of learner’s responses in real time, to provide immediate feedback on the quality of empathy. This approach leverages existing scales of empathy coding. This approach of human evaluation of empathy occurring in the background of the interaction, unbeknownst to the learner is termed a “Wizard-of-Oz” approach, as illustrated in Fig. 9.5. The “Wizard-of-Oz” approach is feasible in instances where the necessity of a system to support natural conversation outweighs the costs of human interaction and possible subjectivity and variance in virtual standardized patient responses [16].

Fig. 9.5

Rater interface for empathy coding in virtual patient interactions, using Empathic Communication Coding System [44]. From Borish, M., Cordar, A., Foster, A., Kim, T., Murphy, J., Chaudhary, N., & Lok, B. (2014). Utilizing real-time human-assisted virtual humans to increase real-world interaction empathy: Proceedings of the 5th Kanesi Engineering and Emotion Research International Conference (No. 100, pp. 441–455). Linköping; Sweden: Linköping University Electronic Press http://www.ep.liu.se/ecp/article.asp?issue=100&article=35

VPF [2] allows automation of the empathy feedback given to users, based on ECCS, such that users can receive immediate feedback on their responses to the empathic opportunities presented by the virtual patients. To validate the virtual patient feedback, users’ responses to the predetermined empathic opportunities found in the transcripts of the learners’ interactions with virtual patients could be expertly coded with ECCS by reliable empathy raters. Finally, upon completion of the interaction and receiving empathy feedback, the learner can provide an assessment and plan, complete surveys, and conduct a review of their virtual patient interaction. Thus, the VPF platform is a well-suited and scalable method to teach empathy.

To evaluate learners’ empathy, we built empathic opportunities into virtual patient scenarios [16]. These opportunities require the trainee to recognize and respond to the virtual human’s concern. To help standardize the introduction of “empathic moments” into VP scenarios and to uniformly code the user responses to these “moments”, we sought a reliable empathy-coding instrument [14]. Empathic Communication Coding System (ECCS) is a validated, expert-rated scale developed to code empathic opportunities, defined as explicit, clear, and direct statements of emotion, progress, or challenge by the patient. The ECCS also codes clinicians’ verbal responses to these opportunities ranging from level six (shared feeling/experience) to level zero, denial of the patient’s perspective. For example, an emotion expressed by a patient (i.e., “My sister died 3 months ago. I cry every time I think of her”) could elicit a level-5 clinician response (i.e., “It is hard to go through a death of a family member”) or a level-0 response, clinicians ignoring the empathic opportunity, (i.e., “Do you have any allergies?”) [14]. ECCS was a sensitive measure of increased empathy in live physician–patient interactions 6 months after live empathy training and can discriminate a persisting effect of clinicians’ empathy training on patients’ satisfaction and health outcomes after 6–12 months [45].

3.4 Studies and Findings

3.4.1 Virtual Patient Interactions with Empathy Feedback Increase Learners’ Empathy

Virtual standardized patients can enhance verbal empathy by giving immediate feedback and teaching empathic communication [16, 23]. In our initial attempt to systematically evaluate and teach empathy with virtual patients, we adapted the virtual patient Cynthia Young [13]. Cynthia is a 21-year-old college student, referred by her campus counselor, who presents with symptoms of a major depressive episode. Her function declined, and she became depressed, hopeless and poorly motivated, following a personal loss. The scenario already contains predetermined empathic opportunities (e.g., “My cousin and I were like sisters. I cry every time I think of her”). Cynthia Young elicits users’ empathic communication that is similar with learners’ performance in interactions with standardized patients [14, 23, 46]. Using one virtual patient encounter with Cynthia, followed by immediate empathy feedback, we demonstrated a significant improvement in medical students’ (n = 70) ability to offer encouraging, supportive, and empathic statements (p < 0.0001), as rated by SPs, in an interaction immediately following the virtual patient encounter, in comparison with learners who interacted with Cynthia Young but did not receive empathy feedback. The same virtual patient training significantly increased the number of empathic opportunities elicited by medical students in SP interactions (p = 0.0005). Further, post-virtual patient training with feedback on empathic responses, medical students’ verbal communication of empathy in SP interactions, assessed by expert raters using ECCS increased significantly (p = 0.027), compared with virtual patient interactions without empathy feedback [14].

3.4.2 Learners Express Higher Levels of Empathy Towards Virtual Patients Compared to Standardized Patients

One natural question that arises is how empathy with a virtual standardized patient is similar to, and different from, empathy with a standardized patient. Kleinsmith et al. [23] conducted research aimed at building this understanding by determining if learners can respond to a virtual patient’s statement of concern with an empathic response. A study was conducted at the University of Florida College of Medicine in which third-year medical students (n = 110) interacted with virtual patients in one session and with human standardized patients in a separate session a week apart [23]. During the separate interactions, the virtual and standardized patients presented the learners with empathic opportunities. Reliable expert raters later rated students’ responses to these opportunities. The virtual patient interactions occurred on learner’s laptop or desktop computer while the standardized patient interactions took place in patient exam rooms. The virtual and standardized patient interactions were counterbalanced to avoid order effects: in one condition students interviewed virtual patients before standardized patients and the order was reversed in the other condition. The results of pairwise comparisons indicate that empathic responses made to virtual patients were rated as significantly more empathic than responses made to standardized patients (p = 0.000). In summary, the empathy expressed towards virtual standardized patients was higher than empathy towards standardized patients and the empathy level correlated with the length (number of words) of the students’ responses. Even though virtual patients may be perceived as artificial, the educational benefit of employing them for training medical learners’ empathic communications skills is that virtual patients offer a low pressure interaction, with less time restrictions, which allows learners to reflect on their responses [23].

Patient Shadowing as an Empathy Teaching Tool in Virtual Patient Interactions

Cordar et al. [15] introduced patient shadowing in virtual standardized patient interactions. Patient-centered care promotes the physician knowing the patient in the entirety of his/her social and cultural context rather than focusing solely on an illness or injury. Patient shadowing was described as “having a committed and empathic observer follow a patient and family through their care experience” [47]. Patient shadowing has been integrated in medical school curriculum and in patient safety and quality improvement initiatives in direct patient care. Patient shadowing interventions involve learners acting as “patient navigators” for patients during clinic visits or learner volunteers portraying physical symptoms and being cared for by residents who were unaware of the experimental nature of the hospitalization [48]. These interventions were successful, with the learners becoming intensely aware of the importance of empathy in patient care [48, 49]. Cordar [15] used virtual standardized patients to simulate patient shadowing by introducing cut scenes from the videogame The Sims 3 (https://www.ea.com/games/the-sims/the-sims-3) into pre-existing virtual patient technology, in order to enhance medical learners’ empathy. The premise of this technological enhancement was that cut scenes could illustrate moments from the virtual patient’s daily life to help convey the patient’s struggle with their medical condition. Furthermore, by understanding the experience and perspective of the patient, the healthcare provider could show increased empathy towards the patient. The cut scenes introduced in the virtual standardized patient scenario with depression illustrated virtual patient Cynthia Young’s [13] experience of low energy by showing her getting out of bed, starting an activity, and returning to bed soon thereafter, eating ice cream and taking naps, watching TV, and crying [15]. Eighteen medical students were randomized to interaction with the same virtual patient with depression that included cut scenes and 17 to a virtual patient without cut scenes, which they completed on a desktop computer. After interacting with the virtual patients, each student completed an encounter with a standardized patient representing a major depression scenario that took place in a patient exam room. In the virtual patient interactions with cut scenes, the learner asking about a symptom of depression (e.g., low energy, anhedonia, sleep, appetite, depressed mood, and crying) triggered each VP cut scene (see Fig. 9.6).

Fig. 9.6

Cut scenes reflecting patient’s daily life. Reprinted by permission from Cordar, A., Borish, M., Foster, A. and Lok, B., building virtual humans with back-stories: training interpersonal communication skills in medical learners. In Bickmore T., Marsella S., Sidner C. (Eds) Intelligent Virtual Agents. IVA 2014. Lecture Notes in Computer Science, vol 8637. Springer, Cham (copyright) 2014 [50]

For example, if the learner asked about Cynthia’s appetite or weight gain, the cut scene where she eats ice cream and returns to bed would play, after Cynthia’s answer “All I do is eat and sleep”. The standardized patients rated the learners exposed to the virtual patient with cut scenes significantly higher than the learners who interacted with the virtual patient without cut scenes on the following communication checklist items: (1) “The examinee offered encouraging, supportive, and/or empathic statements” (p < 0.05) and (2) “The examinee appeared warm and caring” (p < 0.01). The difference between groups approached significance for the item “The examinee developed a good rapport with me.”

Empathy and Perspective-Taking in Virtual Patient Interactions

Empathy and perspective taking are closely connected [51]. Thus, Halan et al. [46] sought to explore if taking the perspective of a virtual patient could be used to teach empathy to healthcare learners. Taking the perspective of the patient is essential for healthcare learners to learn critical interpersonal skills including empathy. To study perspective taking and empathy, researchers conducted a semester-long user study with 24 healthcare students exploring the effects of having them create virtual patient agents, on subsequent virtual patient interviews. The authors hypothesized that learners who create and interview virtual patients of the same race will be significantly more empathic than learners who create virtual patients with a race discordant to their self-identified race. Early in the semester, speech and language pathology students in a Dysphagia Management course each created virtual patients of a particular race. The learners had to create the appearance and the conversational corpus. A virtual patient’s conversational corpus includes the questions the virtual patient could respond to, and most importantly, the virtual patient’s responses to questions. For example, learners had to author virtual patient responses to common questions such as “what worries you about your difficulty swallowing?” and “how is your swallowing problem impacting your family?” This creation exercise provided learners the opportunity to take the perspective of the patient, including patients of a race that could be similar or different than the learner’s self-identified race. Later in the semester, learners interviewed virtual patients of the same or of different race as the virtual patient they created. The learners’ level of empathy with the virtual patient was measured through specific empathic opportunities where the virtual patient brought up an issue that required the learner to deliver both information and therapeutic communication. For example, a virtual patient named Marty Graw asked the learner several minutes into the interaction, “Doctor, imagine you being sick all the time. How would you feel about being sick and coughing while talking to your patients? My condition is the same. I am a chef but cannot even taste any of the food I’m cooking.” Results indicate that healthcare learners who created and interviewed virtual patients of the same race were significantly more empathic than learners who created virtual patients with a race discordant to the one they interacted with in the experiment. These results could help the virtual patient creator to actively address any misconceptions or stereotypes that they may hold about a certain ethnicity or culture, as uncovered in the process of creation and interaction with the virtual patient.

4 Discussion

4.1 What Have We Learned?

Virtual patient training elicits empathy levels that approach or match those achieved with live empathy training. After virtual patient empathy training, language/speech pathology learners showed a mean empathy score of 3.7, similar to medical learners’ response to challenges faced by virtual humans (3.4) and similar with internists’ live responses to real patients (3.3), after live empathy training, all measured with ECCS (0–6 points scale where 0 = denial and 6 = sharing of patient’s perspective). As shown in Fig. 9.7, these results illustrate reliability, as well as consequences and test–retest validity for the virtual patient empathy training [14, 44, 46, 52, 53].

Fig. 9.7

Comparison of empathic responses after live or virtual patient empathy training interventions, followed by real patient, SP or virtual patient interactions. SP standardized patient, VP virtual patient, ECCS Empathic Communication Coding System

4.2 Future Directions in Teaching Empathy with Virtual Standardized Patients

Virtual standardized patients are a powerful tool for learners to practice, refine, reinforce, and receive feedback on empathy. Prior research has demonstrated that virtual patients can elicit empathy, help learners acquire best practices in expressing empathy, and provide opportunities for perspective taking and reflection [4, 5, 14, 23, 46]. Ericcson [51] established that deliberate practice leading to significant improvements in a certain skill occurs when “individuals are (1) given a task with a well-defined goal, (2) motivated to improve, (3) provided with feedback, and (4) provided with ample opportunities for repetition and gradual refinements of their performance”. Virtual standardized patients are well-suited tools for deliberate practice in learning empathy.

However, empathy is a complex construct. Although the virtual patient platforms allow integration and immediate delivery of feedback, much work remains in providing more nuanced, complex educational experiences. For example, concepts such as breaking bad news, end-of-life discussions, inter-cultural communication, and patient conflicts would require teaching empathy at a deeper level than current virtual patient approaches.

Teaching empathy at a deeper level will likely require a system capable of capturing, processing, and responding to the user in subtle and nuanced ways. Future work in sensing systems that can recognize a wide range of conversational cues, including tone of voice, prosody, eye gaze, facial expressions, emotions, and body language, will enable the system to understand the user’s intent, beyond simply the words being spoken. Such a system would also be able to present a virtual patient that could interpret the learner’s affect, such as where the learner was looking, or how she/he held their arms. Further, such a system would also be able to provide feedback on the learner’s empathic abilities, including crowdsourced real patients’ perception of learner’s empathy and expert-rated empathy feedback. Integrating this complex, multi-level feedback into a learning experience that can be assimilated by the learner is the subject of ongoing work. Lastly, the virtual patient potential for longitudinal reinforcement offers promise in helping develop empathic communication expertise. Such potential advances in deliberate practice of empathy, would revolutionize teaching and milestone acquisition in domains like delivering bad news, suicide risk assessment, and exploration of substance use or other self-harm behaviors.

The latest advances in artificial intelligence and machine learning will drive the understanding of affect cues and contextualized feedback. Artificial intelligence and machine learning will enable systems to effectively classify emotions by capturing the user’s facial expressions, understand nuance in communication such as concern and sympathy, and enable the virtual patient to respond accordingly to the user with speech, gestures, and emotion. As the technology advances, these innovations will allow educators to meet and expand their educational learning objectives for empathy by using virtual standardized patients.