Keywords

FormalPara Learning Objectives

After reading this chapter, you should know the answers to these questions:

  • How can computers improve the delivery of in-class and self-learning, as well as in-practice learning?

  • How can different approaches to learning be implemented using computers?

  • How can simulations supplement students’ exposure to clinical practice?

  • What are the issues to be considered when developing computer-based educational programs?

  • What are the significant barriers to widespread integration of computer-aided instruction into the medical curriculum?

1 Introduction

The application of digital technology to health science education is a sub-field of biomedical informatics. It includes the application of all aspects of information and computer technology to the content and delivery of education, as well as to research on the improvement and efficacy of education. Healthcare requires constant learning, with its practice in a multi-disciplinary team environment in an information-rich world. Digital technology offers new approaches to learning that:

  • increase engagement and retention of knowledge,

  • allow personalization of knowledge delivery,

  • enhance collaboration through connectivity,

  • support learning any time and anywhere,

  • make available the increasing volume of knowledge,

  • support learning of evidence-based clinical practice, and

  • enhance research through collection and analysis of large volumes of learner data.

In this chapter, we first discuss approaches to teaching and learning with digital technology. That section includes material about theories of learning, digital technologies for learning environments, and an overview of learner audiences. We then transition to digital learning systems, which includes learning management systems, learning content creation systems, just-in-time learning systems and performance support, usability and accessibility, interoperability standards, digital content and assessment of learning. Finally, we discuss future directions and challenges for the application of digital technology in health science education.

2 Approaches to Teaching and Learning with Digital Technology

The continual rapid increase in health sciences knowledge requires a shift in learning methods both by the health sciences student as well as the health professional. Decades earlier, memorization and recall of facts were a primary, and sufficient, learning goal. Current learning approaches require learning the basic concepts and methods of a discipline but, in addition, emphasize the ability to integrate knowledge and to solve problems in the context of everyday healthcare.

2.1 Theories of Learning

Understanding how digital technology can support learning in the health sciences begins with an appreciation of how people learn. This understanding provides a foundation for thinking about the learning process and how it is shaped by context, purpose, goals, complexity, and the diversity of learners.

At its most basic, learning involves a change in how a learner perceives and understands some part of their world. The term schema is commonly used, in cognitive science, to describe the cognitive frameworks the people use to organize the information they have and their beliefs about a particular concept, activity, or experience. Schemas help us quickly assess a situation and act appropriately. When individuals encounter new information, they try to incorporate it into their existing schema to enhance their understanding.

If new information contradicts the learner’s existing knowledge or beliefs, the learner adjusts in one of two ways. They may accept the new information as valid, and modify their schema accordingly. Alternatively, they judge the new information as invalid, unimportant, or irrelevant, and they do not adapt their schema to account for this new information. When strong existing schemas keep individuals from accounting for new, valid information it can be very difficult for them to make changes such as altering unsafe procedures or adopting new safety protocols. To encourage such knowledge and behavioral change, it is important to explicitly address existing misconceptions and support learner motivation to change.

Changes in schema presume that individuals actively construct their own meaning through the interactions they have with information, other people, and the environment around them. This constructivist approach argues that individuals are not blank slates and bring their own history and experience to every learning situation. It is also important to recognize that this construction is an inherently social and situated process. Often learners are learning together in classes, groups, or teams. The conversations and shared experiences with others provide interpersonal cognitive and affective context for the learning process, and can shape the direction and scope of knowledge construction.

Digital technology can support a variety of approaches to learning. A common approach is didactic teaching, a one-way transfer of information through lectures, presented online via technologies such as recorded digital video. This approach has the advantage that new, as well as remedial, material can be made available, with additional links to in-depth content. A considerable amount of the available digital content is didactic, though it is usually enhanced with various activities for active learning. Active learning approaches, on the other hand, focus on engaging learners in the learning process by having them interact with the content, with each other, and in reflection on what and how they are learning. Research has shown that instructional approaches that promote active learning consistently outperform transmission-only approaches at a statistically significant level (Freeman et al. 2014).

Flipping classes is a relatively recent strategy for encouraging active learning. Instructors flip the class when they provide the instruction, traditionally delivered through in-class lecture, online and the class time is devoted to active learning which replaces the majority of traditional homework. The homework becomes doing what needs to be done to prepare for the in-person class. Flipped classes are similar to hybrid or blended classes where the seat time that would be used for lecture is focused on active learning instead. Faculty engage with learners through case discussions, problem solving, and deep dives to further understand the content learned outside the classroom.

2.2 Digital Technologies for Learning Environments

Much, if not most, of today’s learning content is delivered digitally. This is evident in the tools that are used within and outside of the classroom, for on-the-job training, in specialized learning facilities and elsewhere. In the classroom, learning content such as PowerPoint slides, Prezi presentations, websites, simulations, games and other digital media are often projected using digital projectors or delivered directly to the devices of learners, such as smartphones, tablets and laptops. Audience interaction methods, such as Web-based surveys, shared digital whiteboards or student/audience response systems, can help learners interact with the learning content, the instructor(s) or fellow learners.

The classroom technologies and device ecology accessible to teachers and learners can extend the learning experience seamlessly beyond the classroom. For instance, videoconferencing allows individuals to attend lectures remotely, raise their hand and ask a question. Depending on the teaching style, this remote participation may approximate face-to-face attendance fairly closely or fall far short of it. Collaborative technologies, such as instant messaging, group chats or collaborative editing tools, can help bridge physical separation, and enable efficient and effective virtual group work.

Such collaborative work can also happen asynchronously and provide flexibility (within limits) to learner schedules. Discussion lists, messaging boards and collaboration sites enable episodic, time-independent contributions from and interactions with learners.

Some time ago, using advanced technologies, such as simulations, virtual reality and augmented reality, required a trip to a specialized facility, such as a simulation center or “virtual reality cave.” However, with devices such as Oculus or HTC Vive, such experiences are now available almost anyplace.

Last, social media such as Facebook, Google Hangouts, Twitter, Snapchat and Instagram have found important uses in education, ranging from real-time updates on projects to sharing experiences on a field-trip.

3 Overview of Learner Audiences

As discussed above, education in medicine, nursing, pharmacy, dentistry and other health professions is shifting from a focus on knowledge acquisition to competency-based education (Englander et al. 2013). Educational technologies are well positioned to support this transition, but only if they take the specific context of learners and their goals into account. We therefore discuss learner audiences and their particular needs in the following sections.

3.1 Undergraduate and Graduate Health Care Professions Students

Basic science programs in medical schools were among the first to implement technology-supported learning. Visually rich content in anatomy, neuroanatomy and pathology was much more accessible on the computer than through the microscope or via the cadaver dissection room (◘ Fig. 25.1). Excellent 3D learning programs for anatomy are available, such as Netter 3D Anatomy, Primal Pictures, VH Dissector, Anatomage Table,Footnote 1 and other products, providing ever more accurate visualization of the human in three dimensions. The use of microscopes in fields such as histology and pathology education has virtually disappeared. Interestingly, in many schools, the use of cadavers has seen a resurgence, both as an important learning tool and as a rite of passage into the health care profession.

Fig. 25.1
figure 1

A life-size reconstruction of a digital human is viewed in a horizontal computer screen or “digital table”. With finger gestures, learners can identify structures, remove layers of tissue to make vessels, nerves and bones visible, or rotate the body. Additional functions at the edges of the table allow further viewing and measurement functions. Clinical cases, with anatomy reconstructed from radiologic images, such as CT and MRI, allow study of actual cases. (Courtesy of Anatomage Inc., with permission)

Nursing schools have moved quickly to expand their use of technology in education. For instance, digitally-enhanced physical mannequins for simulation of realistic nursing scenarios are widely-used learning tools. Many nursing schools that used to share a simulation center with a medical school have found their own demand high enough to require building their own simulation centers.

Dental schools often share a part of their curriculum with medical schools, and as a result use the same or similar learning content. However, they also need specialized anatomical and simulation content for dental and craniofacial topics. 3D software for dental anatomy is used widely in pre-doctoral dentistry. Historically, simulation of dental procedures was practiced using physical objects such as chalk or plastic teeth, a practice that is still widespread. More recently, high fidelity digital simulators have been developed.

For many years, teaching hospitals had patients with interesting diagnostic problems such as “unexplained weight loss” or “fever of unknown origin”. This environment allowed for thoughtful “visit rounds,” at which the attending physician could tutor the students and house staff, who could then go to the library to research the subject. A patient stayed in the hospital for weeks as testing was pursued and the illness evolved. In the modern era of restricted insurance payments, managed care, and reduced length-of-stay, this opportunity for learning in the hospital environment, has vanished for most junior students. The typical patient in today’s teaching hospital is multi-morbid, usually elderly, and commonly acutely ill. The emphasis is on short stays, with diagnostic problems handled on an outpatient basis, and diseases evolving at home or in chronic care facilities. Thus, the medical student is faced with fewer diagnostic challenges suited to their level of knowledge, and has little opportunity to see the evolution of a patient’s illness over time.

One response of medical educators has been to try to move teaching to the outpatient setting; another has been to use problem-based learning and computer-simulated virtual patients. Problem-based learning (PBL) is a pedagogical approach where each small group of students is given a clinical problem and they engage in discussion to develop an understanding of the problem, identify relevant knowledge, seek required knowledge using online and library research, discuss and challenge each other’s interpretations, and settle on a solution to the problem. PBL is widely used in undergraduate clinical learning, teaching students self-directed learning, reflection, and teamwork. An interesting analysis of PBL by a student is available at Chang (2016).

Computer-simulated patients allow a full range of diseases to be presented and allow the learner to follow the course of an illness over any appropriate time period. Faculty can decide what clinical material must be seen and can use the computer to ensure that this core curriculum is delivered. Moreover, with the use of an “indestructible patient,” the learner can take full responsibility for decision making, without concern over harming an actual patient by making mistakes. These simulated patients may be fully virtual, or may be computer-enhanced physical mannequins.

3.2 Practicing Health Care Providers

Health Sciences education does not stop after the completion of formal training. The science of medicine advances at such a rapid rate that much of what is taught rapidly becomes obsolete, and it has become obligatory for health professionals to be lifelong learners, both for their own satisfaction and, increasingly, as a formal requirement to maintain their professional certification. Therefore, online courses and online certification examinations have become increasingly common for maintenance of certification. An additional advantage of online certification is the automatic tracking of learner performance, and the accompanying automatic generation of certificates and institutional compliance reports.

Health professionals are also required to demonstrate clinical competence through their performance in simulated clinical scenarios. Advanced Trauma Life Support and Advanced Cardiac Life Support are some of the areas in which clinical competence is demonstrated through actual participation in on-site scenarios. Online simulation of these and other scenarios can be used as preparation for testing in a live, crisis scenario. Some specialties, such as anesthesiology, have developed sufficiently rich online simulations in their specialty, with real time assessment, that they can reduce the requirement for use of live scenarios (◘ Fig. 25.2).

Fig. 25.2
figure 2

A screenshot of the SimSTAT simulation used for Maintenance of Certification by the American Society of Anesthesiologists. In a simulation of an operating room, the anesthesiologist ventilates the patient who is lying on the operating table. The simulation is viewed by the learner on a computer screen while the learner plays the role of the anesthesiologist. The learner guides the on-screen anesthesiologist to care for the unconscious patient by clicking on desired interactions, such as the equipment in the room or the icons at the bottom of the screen. Through these interactions, the learner can control the level of sedation, give medications, fluids, and gases, and monitor the patient’s physiologic status. The inset screen on the top left is a monitor for the patient’s vital signs. The inset screen on the top right is the display from the anesthesia machine. (Courtesy of CAE Healthcare, Inc., with permission)

3.3 Patients, Caregivers, and the Public

For those outside the health science professions, there is no systematic way to learn how to be a knowledgeable patient or home-based caregiver, or how to effectively communicate with a health care provider. For the provider interacting with the patient, the need to understand the patient accompanies the need to problem-solve. These changes in health care delivery are occurring slowly, and technology, particularly simulation and role playing, will be part of this change (Zaharias 2018).

Meanwhile, healthcare information is widely available to the general public, with the most reliable information being on web sites affiliated with federal library resources such as Medline Plus, academic organizations, professional societies or federal health agencies. Online courses, both free and paid, are also available, many from traditional universities or other online education organizations. Interactive learning applications, however, are not widely available to the public though they are able to provide more engaging learning.Footnote 2

Improving the patient’s health literacy has become an important approach to providing higher quality health care. Failure to comply with medication regimes, exercise plans, or hospital discharge instructions are a major cause of return visits to the hospital or clinic. Patient and family caregiver education, leading to better understanding of clinical instructions, could result in a more effective partnership between the patient and the healthcare provider (Nelson 2016) (see ► Chap. 11). Online learning resources are one approach to alleviating the compliance problem.

In the next section, we take a close look at how digital learning content is created and delivered.

4 Digital Learning Systems

4.1 Learning Content Management Systems

A Learning Content Management System (LCMS) is a software platform that allows learning content creators, such as faculty, to create, manage, host and track changes in digital learning content. Prior to the development of LCMSs, educational content creators had to assemble several separate and disparate items to create rich, engaging learning content. LCMSs, on the other hand, are one-stop platforms that integrate a wide variety of tools for content creation. They overlap with Learning Management Systems (LMS, described below) in that both support content hosting and delivery. However, LCMSs specialize in tools to create, manage and update content.

Personally created course content may be a web site or a blog, created by a faculty member on any of a range of web site or blog creation tools. Other personal content creation tools include tools for quiz item development, capturing video lectures or demonstrations, and creation of interactive animations or games. The only requirement is that the content files be compatible with the LMS, so that the content can be uploaded to the LMS and deployed to all learners without any need for special integration programming.

Collaborative content creation can be another powerful approach to learning. As opposed to structured content that is created by faculty or a similar creator, collaborative, learner-generated content is informal and created on the fly, for instance in a discussion forum. Structured discussion groups which encourage students to provide their thinking on the discussion questions, and to comment on content from other learners, are useful tools to support learning through active participation, argument, and reflection. Video conferencing tools, such as Zoom and Skype, support real-time discussion and collaboration, and can be content creation tools if there is a repository of the content.

When the content to be created is sufficiently complicated, requires strict adherence to organizational policy, or requires a range of skill sets and significant expense, it becomes necessary to approach it at the level of the enterprise. For example, the American Heart Association has developed courses for cardiac life support that are required training in the United States and many other countries.Footnote 3 These programs are created by teams, with each person providing a different skill, such as graphic design, programming, or content knowledge, rather than by an informal collaboration of learners with similar skills.

4.2 Learning Management Systems

A Learning Management System (LMS) is a repository of learning content, an interface for delivering courses and content to learners, and a platform for the course creator or administrator to track learner engagement and performance. From the learner’s viewpoint, the LMS provides a single login access to all courses that they may need. Once within a course, the learner can access content, such as text, videos, quizzes, games, handouts and assignments. The LMS may include administration features for the learner to select courses, register or join a wait list, pay for each course, and track their grades. It may also include resource sharing and collaboration between learners. From the faculty’s point of view, the LMS allows uploading and modification of course content, as well as a dashboard for viewing the performance of learners, groups and classes as a whole. LMS features may include various statistics, such as usage of course components or the performance of the class on individual test items.

Higher education institutions provide a relatively structured curriculum to well-defined learner populations. Their needs typically are served by educational LMS applications such as Blackboard Learn, D2L Brightspace, Moodle, or Canvas (Dahlstrom et al. 2014). Medical centers and corporations often use LMSs that are more suited to corporate needs. Typically, the most important requirement of such an LMS is tracking learner compliance with required training, and export of reports for regulatory and accreditation purposes. There are numerous enterprise-oriented LMSs. Some common ones are Captivate Prime, TalentLMS, and Totara, but market leadership of LMSs changes continually.

4.3 Just-in-Time Learning Systems and Performance Support

Learning also occurs outside of formal learning contexts. This “just-in-time learning” happens on demand, for instance when learners need instant information at a critical moment, or something goes wrong and they need to know what to do next. Instant information can provide immediate help, or performance support, but can also be considered a learning opportunity. A particularly powerful approach is to make these tools accessible when and where they are likely needed, for instance in online help areas of electronic medical records.

Examples of performance support or just-in-time learning tools include:

  • job aids, such as check lists, quick reference cards, and handouts;

  • protocols and templates, such as the SBAR (Situation, Background, Assessment, Recommendation and Request) technique for communicating critical information;

  • resource or policy documents;

  • video and audio recordings, such as brief demonstrations of care procedures, particularly helpful for home-based care providers; and

  • animations, simulations, and learning modules that include brief instructions, demonstrations, or explanations.

4.4 Interoperability Standards

The education enterprise process includes many parts, such as content, curriculum, LMSs, learner profiles, assessment, certification, and others. These parts are often supported by different tools and platforms that need to work together seamlessly. To enable this seamless environment, tools and platforms need to be interoperable, without the need for custom programming for integration. In this chapter, we discuss some of the interoperability standards in education (see ► Chap. 7).

Historically, the commonly used standard for such learning object interoperability was the Shareable Content Object Reference Model (SCORM).Footnote 4 It defines how a content object (a course) should be packaged and described, how it should be launched, and how data should be communicated between the LMS and the content package. A SCORM-compatible content object can be published to and played back from any SCORM-compatible LMS. SCORM can report on course completion and time spent. SCORM was last updated in 2009 and, as a standard, has not kept up with changing technology. However, it is still one of the most widely used standards for learning object interoperability.

The newer xAPI standard (aka TinCan API)Footnote 5 is much more robust in terms of analytics and mobile deployment. The drawback to xAPI is that it does not integrate with older LMSs and tends to be costly to deploy, often requiring professional development assistance.

With the rapid growth in the use of learning management systems in higher and continuing education and workplace learning, it has become critical to have interoperability standards that provide integration of web-based learning objects and applications. Learning Tools Interoperability (LTI)Footnote 6 is a standard developed by the IMS Global Learning Consortium to provide a means for seamless and secure pass-through of student credentials and grades between the LMS and the external application. LTI tools are usually web-based applications written in a server-side language which can serve a variety of purposes. These include, but are not limited to, hosting and serving video with quizzing, providing access to interactive learning materials from textbook publishers, allowing learners to create media, use specialized programs, and collaborate in integrated development environments (IDE), whiteboarding and mind mapping applications, or videoconferencing.

Other interoperability standards address various education services. For example, the MedbiquitousFootnote 7 organization has developed a Curriculum Inventory Data Exchange Standard that is being used by the Association of American Medical Colleges to collect and collate curriculum data from all its medical schools and map these curricula to competency requirements. Other Medbiquitous standards include the Educational Achievement Standard to document learner competency used by numerous medical certification organizations, and the Virtual Patient Standard to enable exchange of virtual patient simulations across institutions.

There has also been an interest in the exchange of education components, specifically learning content modules, between institutions. A number of content collections have been developed, with the most well-known being MERLOT. Standardized descriptions of learning objects, known as Learning Object Metadata, were developed, but exchanging and repurposing individual learning objects did not become commonplace. However, an interesting by-product was a standardized way to describe items in a content collection, to manage a library of learning objects and to track learner use of those objects. Another example of a repository of shared learning resources is the AAMC’s MedEdPORTALFootnote 8, which is peer-reviewed and contains both patient cases and clinical scenarios.

4.5 Usability and Access

Usability and access are important considerations in developing educational software (see ► Chap. 5). About 1 in 50 adults have some form of vision or hearing disability, and need alternate or augmented access to digital learning content. Two standards, Section 508 of federal law (508)Footnote 9 and the World Wide Web’s Web Content Accessibility Guidelines (WCAG),Footnote 10 address use of digital content by people with disabilities.

Section 508 specifies that digital information provided by or to the government must be accessible if there is “no undue burden”. In practice, designing accessible online content requires use of techniques available in current web design, such as the “Alt Text” tag for graphics, and indications to make the user interface more visible or audible. Adherence to Section 508 becomes more difficult in more complicated applications, such as 3D immersive environments and dynamic simulations, requiring creative solutions to presentation and interface requirements.

WCAG is a set of formal guidelines on how to develop accessible web content. It does not address non-web digital content.

5 Digital Content

Digital content, unlike a typical textbook or lecture, can be interactive. Three levels of interactivity are typically possible:

  • Level 1: The content includes text, graphics and video but interaction is primarily through clicking to move to the next chunk of content. This level may include simple quizzes such as multiple choice or true/false questions. Much of digital learning content consists of web pages or applications that incorporate this style of expository material.

  • Level 2: The content includes multimedia such as audio, video and animations. The interactivity supports simple puzzles and games, like sorting and matching. The cost of development is higher than for Level 1, but the content is more engaging.

  • Level 3: The content presented is very rich, including realistic three-dimensional environments and characters, or even immersive virtual reality (◘ Fig. 25.3). Interaction happens through games or simulations, with the content evolving based on the choices and decisions made by the learner.

Fig. 25.3
figure 3

A screenshot of a Level 3 interactive application, BattleCare. The learner can select tools from the medical kit on the right, and drag them onto the simulated patient to clean and compress the wound or to listen to heart and lung sounds. (Courtesy of Innovation in Learning, Inc., with permission)

5.1 Text/Image/Video Content

Much of digital learning content consists of web pages or applications that incorporate expository material, using text, graphics and video, and Level 1 interaction. Although much of the focus of computer-based teaching is on the more innovative uses of technology to expand the range of available teaching modalities, computers can be employed usefully to deliver didactic material, with the advantage of the removal of time and space limitations. For example, a professor can choose to record a lecture and to store the digitized video of the lecture as well as related slides and other teaching material, and upload this content to the institution’s learning management system (see Section on LMSs.) This approach has the advantage that relevant background or remedial material can also be made available through links at specific points in the lecture. The ease of creating online video lectures has led numerous universities and corporations to provide libraries of recorded lectures for study by learners at their own convenience.

Many refinements have been developed that use technology to optimize the delivery of didactic or expository content. Microlearning is the presentation of brief segments of content, typically ranging from 5 to 15 minutes in duration. Spaced repetition is the repeated presentation of select content to optimize its retention. Mastery learning is a process of testing the learner for competence in a segment of content before they are allowed to progress to the next. The Khan Academy,Footnote 11 which includes healthcare content in its catalog, uses brief videos to teach single or small groups of concepts. In this micro-learning approach, the learner can select and complete topics with a limited investment of time, and can demonstrate mastery.

Massively Online Open Courses (MOOCs) bring free-to-view, world-class university courses to a global audience. The first major MOOC, Introduction to Artificial Intelligence, launched with an astonishing 160,000 subscribers. The structure of the first courses was similar to a typical university course, with lectures released at the same time as they would be taught to an in-person class on campus, along with assignments and final examinations that needed to be turned in on time. Course support was provided by peer support through student discussion groups. Some MOOCs now require fees for certification of completion of courses. Private companies have sprung up providing support to students around selected MOOC, an indication of the ecosystems that develop around interesting technologies. EdX is a MOOC delivery platform launched by the Massachusetts Institute of Technology and Stanford University. Coursera, Udacity and FutureLearn are major private MOOC platforms.Footnote 12

5.2 Interactive Content

Teaching programs differ in the degree to which they impose structure on a teaching session. In general, drill-and-practice systems are highly structured. The system’s responses to students’ choices are specified in advance; students cannot control the course of an interaction directly. In contrast, other programs create an exploratory environment in which students can experiment without guidance or interference. For example, a neuroanatomy teaching program may provide a student with a fixed series of images and lessons on the brainstem, or it may allow a student to select a brain structure of interest, such as a tract, and to follow the structure up and down the brainstem, moving from image to image, observing how the location and size of the structure changes.

Each of these approaches has advantages and disadvantages. Fixed path learning programs ensure that no important fact or concept is missed, but they do not allow students to deviate from the prescribed course or to explore areas of special interest. Conversely, programs that provide an exploratory environment and allow students to choose any actions in any order encourage experimentation and self-discovery. Without structure or guidance, however, students may waste time following unproductive paths and may fail to learn important material, resulting in inefficient learning.

An example is the Tooth Atlas, used in dentistry. Understanding the three-dimensional structure of teeth is important for clinical dentistry. The key instructional objective of the program is to help students learn the complex external and internal anatomy of the variety of teeth in three dimensions. The rich, interactive 3D visualizations show teeth as they would be visually perceived as well as through very high resolution computed tomography scans, radiographs and physical cross-sections. The learners can rotate and section the computed models, and can control the transparency of each structure so as to study inter-relationships. While the visualization is highly exploratory, the embedded pedagogy is very structured, consisting of detailed textual quizzes with multiple-choice answers.

5.3 Games

A learning game places learning content within a digital video game. The game play experience engages and entertains the learner while certain steps in the game instill desired content knowledge. In a learning game, the learning content is embedded in the game. Gamification, on the other hand, has elements such as a score, achievement badges, or a “leader board”, to add excitement to an otherwise routine learning experience.Footnote 13

A game has the following components: a goal, such as finding the best treatment for a patient; a setting, such as a three-dimensional rendition of an emergency department; game play, such as the information, tests, procedures, and medications available; and game mechanics, such as accessing game play elements by drawing up medication in a syringe or selecting a medication dosage from a menu. Successful resolution of a clinical problem can give the same satisfaction as an enjoyable game. However, to be considered a game, there need to be challenges, such as conquering “enemies” or accomplishing “quests”, evolving clinical problems, or a restricted availability of supplies and personnel, that must be overcome, as well as a clear criterion of success. To be a learning game, actions during game play should result in learning, either by exposure of a nugget of information, by feedback from a mentor embedded in the game, or by trying alternative medical approaches to find an effective treatment.

Numerous learning games have been developed for all aspects of healthcare education but the evidence for their efficacy is not clear (Gorbanev et al. 2018). Funding for efficacy research is limited, and is one reason for the paucity of rigorous studies (Reed et al. 2007). Game development that has a clear learning goal, and has been informed by research during the design stage as well as during development of the game play, has been shown to be both engaging and an effective learning tool (Kato et al. 2008) (◘ Box 25.1 and ◘ Fig. 25.4a and b).

5.4 Cases, Scenarios and Problem-Based Learning

The learner is presented with a story that includes a clinical problem. The presentation may be only in text, with text and graphics, in a near-realistic three-dimensional environment, or even in an immersive virtual environment, with correspondingly varying levels of interactivity. The learner’s role may be constrained such that the learner knows who they represent, what resources are available, and what problem must be solved. Alternatively, the learner may be required to investigate the situation (examine the patient), define the problem, find any supporting resources (what imaging and laboratory tests are available or what learning resources are at hand) and guide the scenario to an end goal. As the learner proceeds, the scenario evolves on the computer, based on the actions taken and the progress of time.

Prognosis is a case-based program with over 500 cases covering most specialties (◘ Fig. 25.5). Each case begins with a brief story of the clinical presentation. The learner must choose among available tests, diagnoses and treatments, and then receives feedback on the choices made, as well as the preferred or optimal choices. The presentation and interactivity are very simple, and the cases brief, but the engagement provided by the clinical puzzle has made this a popular program among medical students and residents.

Fig. 25.5
figure 5

Screenshot from the case-based program, Prognosis. The learner is presented with a summary of the case. The simple graphic presents physical examination information in a similar format for each case. The following screen offers options for laboratory, imaging and other investigation options. The learner selects the management plan and receives summary feedback on the success or failure of the plan. The case ends with a review of the disease background and optimal management, including relevant references. (Courtesy of Medical Joyworks, LLC, with permission)

An approach that combines the benefits of exploration with the constraint of a linear path through the material is one that breaks the evolving scenario into a series of short vignettes. A situation is presented, information and action options are available, and a decision must be made. Each decision triggers the presentation of the next vignette. This could lead to a branching story line but, usually, the next vignette presents the result of the best actions from the previous vignette. A scenario about a virtual patient could have vignettes that lead the learner through the steps of diagnosis, tests, and the course of treatment. This approach is commonly used in computer-based testing of clinical knowledge where assessment of learner performance would be extremely difficult if the interactions were completely unconstrained.

The ability of the computer to track and store the learner’s actions allows post-processing and analysis of the tracked data. An interesting analytic capability is one that compares the performance of novice learners and experts to detect features that define expert information gathering or action sequences. Stevens et al. (1996) compared the information gathering and the conclusions of novices and experts on a set of immunological cases. Using neural nets to process the tracking data, they detected consistent differences in the problem solving approach of novices compared to experts. In particular, novices exhibited considerably more searching and lack of recognition of relevant information, while experts converged rapidly on a common set of information items. The potential of using such expert patterns of performance to educate novice learners has not been widely explored in the health sciences.

5.5 Simulations – Virtual Patients

Clinical training has been shown to benefit from the use of simulations to engage the learner (Gaba 2004; Aebersold 2018; Jeffries 2005). Learning is most effective when the learner is engaged and actively involved in decision making. The use of a simulated patient presented by the computer can approximate the real-world experience of patient care and focuses the learner’s attention on the subject being presented (Huang et al. 2007). The Association of American Medical Colleges has prepared an informational summary of the value of and the issues around issues of using simulation for education.Footnote 14

Talbot et al. (2012) present an analysis of the range of presentation and interactivity available in Virtual Patients. These simulations may be either static or dynamic. Under the static simulation model, each case presents a patient who has a predefined problem and set of characteristics. At any point in the interaction, the student can interrupt data collection to ask the computer-consultant to display the differential diagnosis (given the information that has been collected so far) or to recommend a data collection strategy. The underlying case, however, remains static. Dynamic simulation programs, in contrast, simulate changes in patient state over time in response to students’ diagnostic or therapeutic decisions. Thus, unlike in static simulations, the clinical manifestations of the dynamic simulation can be programmed to evolve as the student works through them. These programs help students to understand the relationships between actions (or inactions), and patients’ clinical outcomes. To simulate a patient’s response to intervention, the programs may explicitly model underlying physiological processes and use mathematical models. An example of a dynamic simulation of a virtual patient is SimSTAT (see ◘ Fig. 25.2), an operating room simulation that is used by the American Society of Anesthesiologists to train practicing anesthesiologists in the diagnosis and management of crises in the operating room.

Virtual Patients can be as simple as Prognosis, described above, or can be richly complex, simulating a complete encounter with a patient in a clinic or hospital room. Simulation of a conversational interaction with the patient or another character can be an important aspect of learning using a virtual patient (◘ Fig. 25.6).

Fig. 25.6
figure 6

Screenshot from Timeout Training, a mobile training application. This illustrates a time-out dialog between the learner (a resident) and the nurse, prior to initiating a thoracocentesis intervention. The learner selects from one of the presented dialog options. Careful design of dialog options can help in learning nuances of dialog possibilities as well in proper sequencing of a dialog. (Courtesy of Innovation in Learning, Inc., with permission)

5.6 Simulations – Procedures and Surgery

Procedure trainers or part task trainers have emerged as a new method of teaching, particularly in the teaching of surgical skills. This technology is still under development, and it is extremely demanding of computer and graphic performance. Early examples have focused on endoscopic surgery and laparoscopic surgery in which the surgeon manipulates tools and a camera inserted into the patient through a small incision. In the simulated environment, the surgeon manipulates the same tool controls, but these tools control simulated instruments that act on computer-graphic renderings of the operative field. Feedback systems inside the tools return pressure and other haptic sensations to the surgeon’s hands, further increasing the realism of the surgical experience.

Commercially available trainers are now in use for many surgical procedures. For example, the 3D Systems company provides a line of Simbionix simulators for training in laparoscopy, endoscopy, and hysteroscopy.Footnote 15 Other simulators are now available for all levels of surgery, beginning with training in the basic operations of incision and suturing, and going all the way to robotic surgery.

5.7 Simulations – Mannequins

Physical simulations of a patient, in an authentic environment such as an operating room, have evolved into sophisticated learning environments. The patient is simulated by an artificial mannequin with internal mechanisms that produce the effect of a breathing human with a pulse, respiration, and other vital signs (◘ Fig. 25.7). In high-end simulators, the mannequin can be given blood transfusions or medication, and its physiological response changes based on these treatments. These human patient simulators are now used around the world both for skills training and for cognitive training such as crisis management or leadership in a team environment (◘ Fig. 25.8). The environment can represent an operating room, a neonatal intensive care unit, a trauma center, or a physician’s office. Teams of learners play roles such as surgeon, anesthetist, or nurse, and practice teamwork, crisis management, leadership, and other cognitive exercises.

Fig. 25.7
figure 7

This plastic mannequin simulates many of the functions of a living patient, including eye opening and closing, breathing, heart rate and other vital signs. Gases, medications, and fluids can be administered to this mannequin, with resulting changes to its simulated vital signs. (Courtesy of Parvati Dev, with permission)

Fig. 25.8
figure 8

Three-dimensional computer-generated virtual medical environments are used to present clinical scenarios to a team of learners. Each learner controls a character in the scenario and, through it, interacts with devices, the patient, and the other characters. Learning goals may include medical goals such as stabilization of the patient, communication goals such as learning to point out a problem to senior personnel, or team goals of leadership and delegation. (Courtesy of Innovation in Learning, Inc. with permission)

A seminal study by Hayden et al. (2014) showed that 50% of nurse clinical training, in the Bachelor’s program, could be replaced by training on mannequin simulators. This is particularly important both because of the range of cases that can be presented on the simulator, and because of the difficulty in obtaining clinical training time in hospitals.

5.8 Virtual Worlds

An extension of the physical human patient simulator is the virtual world simulation, with a virtual patient in a virtual operating room or emergency room. Learners are also present virtually, logging in from remote sites, to form a team to manage the virtual patient. Products such as 3DiTeams and Health TeamSpaces are being used to construct and deliver team training in such virtual medical environments.Footnote 16

5.9 Virtual Reality

The use of virtual reality glasses, along with spatially accurate sound and virtual hands, creates an immersive experience that surpasses the experience of a three-dimensional world as seen on a computer screen on the learner’s desk. The learner feels truly “inside” the experience. The resulting immediacy is so real that it manifests itself through physiologic changes such as a speeding of the learner’s pulse and a total lack of awareness of the actual physical surroundings (◘ Fig. 25.9).

Fig. 25.9
figure 9

A combined image depicting a learner wearing virtual reality glasses, and the scene visible to the learner. The learner feels she is inside the operating room, viewing the procedure. (Courtesy of SimTabs, LLC., with permission)

There are two types of virtual reality in use at present. One is reality as represented by a completely synthetic three-dimensional environment, within which the learner navigates and acts. The other is represented as a 360° video of a real environment within which the clinical action has been recorded. The video virtual reality is useful for didactic training about procedures, such as new surgical methods, where the learner has a front row view as though they were actually present in the operating room.

Examples of simulations using virtual reality have been demonstrated by many universities and organizations. VR simulations of surgery (https://ossovr.com) and patient examination (https://oxfordmedicalsimulation.com) are in use in medical and nursing curricula. There are a few studies examining the learning efficacy of VR simulations (Kyaw et al. 2019). It is likely that the realism of virtual reality, its “face validity”, will result in its use in education even if rigorous efficacy studies are not available.

5.10 Augmented Reality

Augmented reality (AR) differs from virtual reality in that a real world is seen through the AR glasses while other information is overlaid on the world by the glasses. Information can be textual, such as heart rate and blood pressure data when looking at a physical mannequin. It can be graphic, such as an open wound seen overlaid on a person on a bed, simulating an injured patient. The AR overlay information changes depending on what is being viewed, creating a world of information on top of the real visible world.

Learning possibilities using AR are endless. A new nurse can walk into an empty operating room and “see” the contents of closets and drawers, thus being trained on the location of OR supplies. A nursing student can see a “pressure sore” evolve on the heel of a real person because of pressure on the heel in the bed. A medical student can “scroll” through the electronic medical record as they talk with a simulated patient.

AR in medical education is in its infancy but its applications are expected to be wide-ranging.

5.11 3D Printed Physical Models

Three dimensional (3D) printing is a novel application of printing. Slice data from an object, such as a CT image of a bone, is used to print a layer of solid material, such as plastic. Subsequent image slices are used for printing a cumulative stack of plastic slices until the entire object has been printed. The advantage of sequential printing of slices, instead of carving the shape from a solid block of plastic, is that hollow portions of the original object can be printed as holes in the slice data. A second advantage is that very complex objects can be printed using this technology.

3D models are beginning to be used for learning. For undergraduate students, cadaver dissection, plastinated specimens, and dried bone have provided the physical specimens. 3D models add a new option. For healthcare practitioners, patient-specific 3D models help in planning procedures, but they also help in educating the patient about their upcoming surgery. In a recent systematic review on surgical planning for congenital heart disease, the authors found that 25% of the studies showed 3D printed models were useful in medical education for healthcare professionals, patients, caregivers, and medical students (Lau and Sun 2018).

6 Assessment of Learning

Assessment of student learning compares educational performance with educational goals. Ideally, the content used for assessment resides within a curriculum and an educational program that has a clear set of educational goals. Therefore, student learning is measured (assessed) against these overall goals as well as against the goals of the specific learning module. Without these goals, any assessment is without direction and its purpose may be a mystery to the learner.Footnote 17

Assessment maybe formative (guiding future learning and promoting reflection) or summative (making a judgment about competence or qualification before being allowed to advance to the next level of study). Assessment can also be used by instructors and education program designers to identify whether the learning content can be improved, and brought closer to the identified learning goals (Epstein 2007).

Digital technology supports rich assessment both because of its ability to present many types of assessment tools, and because of its ability to track learner actions in great detail. Selected assessment methods are presented below.

6.1 Quizzes, Multiple Choice Questions, Flash Cards, Polls

Quizzes test the learner’s knowledge and, depending on the quiz format, the learner’s ability to solve problems. Quizzes can be presented as questions with single or multiple correct answers, or may require sorting and matching two sets of items. Digital technology simplifies the process of preparing, presenting and scoring quizzes, and can make them engaging and fun by adding imagery, animations and game-like success states.

Flash cards that present the question on one side of the card and the answer on the other can also be simulated using technology. The learner types their answer. Through simple word or phrase matching, the learner’s answer is matched to the expected answer, and scored based on the level of match achieved. For more complex answers, some level of natural language processing is required.

Polls are particularly useful for an instructor to assess, in real time, the current status of learner understanding in the classroom. The poll question, and multiple answers, are displayed on the classroom screen. Each learner selects an answer on a smartphone or on a polling device. The poll responses are immediately collated and presented as a bar graph. If all or most of the learners select the correct answer, the instructor can assume that the topic has been understood. If the responses are distributed over two or more answers, then the instructor can pause to review the topic and clear up learner misconceptions or lack of understanding.

6.2 Branching Scenarios

A branching scenario is a structured approach to assessment using simulation. A mini-scenario or a choice of data resources (such as laboratory tests) is presented at each branch point, and the learner chooses one out of a set of available decisions or responses. One or more of the decisions may be correct. Based on each consecutive decision, the learner moves through a branched scenario and achieves a final outcome to the scenario. The learner can be assessed on each decision or on the final outcome. If the same material is presented in a learning mode, the learner may receive feedback about each decision.

6.3 Simulations

Simulations for assessment may use standardized patients (actors trained to represent patients), realistic interactive mannequins, on-screen simulations, or simulations presented in virtual reality. In all cases, technology can be used to track learner actions and to assess their performance (Ryall et al. 2016). In all except standardized patients, this tracking is built into the simulation, and can be extracted and analyzed for performance reporting. These more complex, scenario-based simulations, differ from branching simulations in that a large number of decision choices are available to the learner at every moment. Thus the simulation is a more realistic representation of a clinical situation but is also correspondingly more difficult to score for assessment (Dillon et al. 2002).

6.4 Intelligent Tutoring, Guidance, Feedback

Intelligent Tutoring Systems (ITS) differ from other technology-based learning systems in that they offer individualization of the learning experience based on the learner’s performance while using the system (VanLehn 2011). Because of their architecture, continuous assessment of the learner is essential to the operation of ITSs, with guidance provided as needed, placing ITSs in the domain of formative assessment. Typically, an ITS is built to replicate one-on-one, personalized tutoring.

Modern ITSs use natural language for dialog between the learner and the tutor (◘ Fig. 25.10). Conversational dialog is more likely to uncover learner misconceptions or gaps in knowledge. As the student responds to the tutor’s questions, the response is compared to the expected response using statistical methods that compare the conceptual similarity of the two pieces of text. An example of a conversational tutoring system is Autotutor (Graesser et al. 2004), which has been used for learning domains ranging from physics and mathematics to training nurses for mass casualty triage (Shubeck et al. 2016).

Fig. 25.10
figure 10

Screenshot of a conversation with an intelligent simulated tutor. The learner, a first responder, converses with a tutor who uses natural language to guide a student to give detailed answers using their own words. (Courtesy of Innovation in Learning, Inc., with permission)

6.5 Analytics

Understanding and improving the process and outcome of education requires applying metrics into many facets of the educational process. With digital technology, measurement and resulting data availability has increased steadily. At the same time, educational institutions and businesses are beginning to develop methods to unlock potential uses of this vast amount of data.

A particularly desirable outcome is personalizing learning to each learner’s needs. The many assessment methods described above can be applied to generate a profile of the learner’s current knowledge state and to create a detailed list of topics to be learned. To implement such an adaptive system, the content itself must itemized and tagged so that the learner’s state can be mapped to the desired learning goal state, and content items can be delivered in an appropriate sequence to achieve optimal learning.

Data analytics can also be applied to individual courses, to identify topics most sought by students, and areas in which testing shows that students consistently fail. Such failure may indicate students’ lack of knowledge, but it could also provide a clue to areas in which teaching could be improved.

At the institutional level, analytics is used extensively to match community and business needs to the design of degree and certificate programs by universities. Businesses also use similar analytic approaches to discover knowledge gaps among their workers, and to design programs to develop or upgrade worker skills as changes occur in their industries.

7 Future Directions and Challenges

As this chapter has shown, computers have played, and will continue to play, an increasingly important role in health sciences education. How will the rapid change and fluid nature of innovation influence how we use technology in education in the future? As we increasingly “digitize” almost all aspects of our lives, we can expect information technology to continue to weave itself more and more into the essential fabric of how we teach and learn.

How can digital technology help advance teaching and learning? Most faculty have embraced, or at least accepted, technology’s growing role in education. Students often have higher expectations of technology use than most health sciences schools can fulfill. How computers can help improve education is a key question of interest to faculty and students alike. Faculty members are keenly interested in finding out how technology can help them become better teachers. Students want to know how computers can help them learn more efficiently and effectively. Current trends in digital learning indicate how some of these questions will be answered (Adams Becker 2017).Footnote 18 The following are examples of some of the challenges that we can expect to encounter.

  • Digital content production and verification remains an ongoing challenge. Digital learning content can range from inexpensive recording and streaming the video of a single lecture to very expensive and time-consuming creation of a rich and dynamic simulation of a disease process. Effective curation and distribution of high-quality content remains a challenge, with some healthcare faculty being reluctant to use content developed at other universities. An emerging trend that may increase use of existing content is to apply the methods of the “flipped classroom” to MOOC-based online courses. In this method, selected online content, from MOOCs or other sources, is assigned for study at home, and group time is used for instructor-led content discussion and problem solving. Such approaches can combine the best of online content with the strengths of classroom teaching by faculty, and it is possible that such hybrid or blended classes will become increasingly common.

  • Learning analytics is a direct outcome of digital learning content and learning management systems. An immediate challenge is to utilize the available data to improve the healthcare education process at the level of the individual, the course, the curricula, and the institution, and to match this education process to the needs of today’s healthcare. A more far-reaching challenge is to use data as evidence to understand what works and why. In particular, we need to understand the best approaches to blended online and face-to-face learning, the uses of collaborative and project-based learning, and the role of simulations and experiential learning.

  • Real-time feedback. Significant portions of pre-clinical training in healthcare require use of simulators. With embedded sensing and compute capability, and internet access, these simulators will become capable of real-time monitoring of learner performance. Display of this data on a performance dashboard will allow both learner and teacher to observe flaws in performance and for the teacher to provide appropriate guidance at the time it is needed. With the addition of intelligent tutors that are built into the simulator, the learner can receive needed feedback by using the simulator at any time of the day. Similarly, we can challenge ourselves to understand and implement intelligent, real-time feedback into all aspects of healthcare learning.

  • Artificial intelligence and adaptive learning. Understanding and engaging with each student’s success at the course level is the domain of the individual faculty, and remains a challenge for the application of appropriate digital technology. Implementation of adaptive learning, that is, adapting the presentation of learning content in response to continuous assessment of learner performance, will be an essential next step. We can expect student performance to be tracked, and personalized exercises and assessments presented, so that they can understand their strengths and weaknesses, and can request digital or in-person help they need for success.

  • Learning Management Systems will see significant evolution. Currently they are narrowly focused on the administration aspects of learning, ensuring that learners are aware of courses needed for their program, delivering course material with the appropriate sequence and timing, and checking when these courses have been completed. In the future, the challenge will be for LMSs to go beyond administration, and to support student learning. In particular, for healthcare education, LMSs will be required to support mastery- and competency-based education, with detailed tracking of concept and skill acquisition.

The topics presented above are only a small selection of the interesting challenges in future healthcare education. Journals such as “Academic Medicine” and “Computers and Education”, and websites such as Educause.edu, periodically discuss these and other challenges in more depth.

8 Conclusion

Digital learning is widespread in healthcare education and has proven to be both effective and engaging. Digital content ranges from basic web pages to highly immersive interactive 3D virtual spaces. Digital support of learning uses learner tracking to assess performance and to advise and guide the learner towards optimal learning outcomes. Artificial intelligence and adaptive learning methods have the potential to personalize learning, and to provide the institution with detailed understanding of how to support each learner as well as how to align educational approaches with institutional goals. Simulators, for hands-on procedures and for diagnosis and communication, will provide a learning environment that parallels the student’s progress through the real clinical environment, providing safe, realistic practice before learners must use that knowledge on real patients. Virtual and augmented reality will make these simulated environments and tools appear and feel realistic, while providing the content scaffolding and mentoring that may not be available in the real clinical environment. Next generation learning management systems will provide the administrative infrastructure to support the student as they progress through their educational program, deliver personalized learning to each student, and offer detailed dashboard information to both faculty and institutional administration.

To realize the full potential of digital learning, there must be significant investment in further development of digital learning technology and content. There must also be effort to develop faculty and staff so that they move beyond simply using technology to understanding how to make each technology elicit the desired learning outcome.

This is an exciting time in digital learning capabilities. It is an even more exciting time to solve the many challenges ahead so as to move towards high performance learning systems.

Suggested Readings

Bligh, D. A. (2000). What’s the use of lectures? San Francisco: Jossey-Bass. In this book, the author analyzes the best use of the lecture as a teaching method, and what lectures fail to teach.

Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How people learn: Brain, mind experience and school. Washington, DC: The National Academies Press. This National Research Council book synthesizes many findings on the science of learning, and explains how these insights can be applied to actual practice in teaching and learning.

NMC Horizon Report: 2017 Higher Education Edition. Available at: https://library.educause.edu/resources/2018/8/2018-nmc-horizon-report. This annual report highlights issues, trends and technologies in education.

Talbot, T. B., Sagae, K., John, B., & Rizzo, A. A. (2012). Sorting out the virtual patient: How to exploit artificial intelligence, game technology and sound educational practices to create engaging role-playing simulations. International Journal of Gaming and Computer-Mediated Simulations, 4(3), 1–19. This paper is a good overview and analysis of the many methods of simulating a patient.

Questions for Discussion

  1. 1.

    In developing effective educational interventions, you are often faced with a choice of instructional methods. Which of the instructional methods listed below would best match the instructional goals listed? Please justify your selection. Note: For some instructional goals, more than one instructional method might be appropriate.

    Instructional Goal

    1. 1.

      Be able to intubate an unconscious patient

    2. 2.

      Memorize the terminology used in neuroanatomy

    3. 3.

      Recognize the symptoms of a patient with probable mental illness

    4. 4.

      Describe the pathophysiological process of hypertension

    5. 5.

      Detect histopathologic variations on histology slides

    Instructional Method

    1. 1.

      Case-based scenarios that include video

    2. 2.

      Physical simulation with computer-based feedback

    3. 3.

      Didactic material that includes text, images and illustrations

    4. 4.

      Intelligent tutoring system

    5. 5.

      Drill-and-practice program

  2. 2.

    You are developing a software application for interprofessional education to teach participants about managing patients with advanced Type 2 Diabetes. Your audience includes students representative of the clinicians who are typically involved in the care of such patients: primary care physicians, specialists such as ophthalmologists and podiatrists, and nurses. Your software application is focused on the care of individual patients, and you have put together a set of clinical case studies as a basis. How could you leverage current collaborative technologies to help the team manage each case in a way that resembles what they would do in real life?

  3. 3.

    Select a topic in physiology with which you are familiar, such as arterial blood–gas exchange or filtration in the kidney, and construct a representation of the domain in terms of the concepts and sub-concepts that should be taught for that topic. Using this representation, design a teaching program using one of the following methods: (1) a didactic approach, (2) a simulation approach, or (3) a game approach.

  4. 4.

    You are a junior faculty member at a major medical center and you just were appointed director for a course on clinical patient examinations. You decide to check out several sharing sites for curricular material, such as MedEdPORTAL, to try to find relevant teaching materials. What kind of issues/problems would you expect in integrating material from those sites in your course?

  5. 5.

    As Chief of Quality Improvement at the Veterans Administration, you are attempting to improve fairly poor outcomes of patients with Post-traumatic Stress Disorder (PTSD). You would like to develop a computer-based educational tool for patients and caregivers to help them cope with PTSD. Most of the patients and caregivers are quite unfamiliar with the disorder, and health literacy varies widely in your target audience. In conceptualizing your approach, you are focused on the following questions:

    1. (a)

      What are the instructional goals of the program?

    2. (b)

      What kind of digital content should you use?

    3. (c)

      How do you assess baseline knowledge of patients and caregivers about PTSD, and how do you measure knowledge gains after they have used the program?