Keywords

FormalPara Learning Objectives

At the end of the chapter, the reader should be able to:

  • Define Clinical Decision Support (CDS) and Clinical Decision Support System (CDSS)

  • Compare and contrast the different types of decision support

    • Alerts

    • Reminders

    • Corollary Orders

    • Guidelines

    • CPR

  • Identify the components of a CDSS

  • Explain the challenges of implementing effective CDS and barriers to effective CDS

FormalPara Practice Domains: Tasks, Knowledge and Skills

Domain 2: Improving care delivery and outcomes

Tasks

  • 2.01. Develop, implement, evaluate, monitor, and maintain clinical decision support (CDS), in alignment with the Five Rights of CDS (information, person, intervention formats, channel, and point/time in workflow).

Knowledge and skills

  • K027. Clinical decision support standards and processes for development, implementation, evaluation, and maintenance

  • K028. Five Rights of clinical decision support (i.e., information, person, intervention formats, channel, and point/time in workflow)

  • K029. Legal, regulatory, and ethical issues regarding clinical decision support

FormalPara Case Vignette

You are the Ambulatory Associate CMIO of a large academic hospital. You are tasked with creating a structure and system to understand the current state of clinical decision support in your hospital, identifying pain points and creating a road map for future clinical decision support governance, optimization and maintenance. How would you begin? What are the key aspects of decision support you’ll need to focus on? How do you anticipate creating a structure for your healthcare system in managing CDS?

Introduction

Despite a robust history of using clinical decision support (CDS) since the 1970s, the effectiveness of CDS remains in question. Changes in clinician behavior are demonstrated in some but not all CDS studies. Demonstration of change in clinical outcomes is lacking due the number of patients needed to generate enough power to show statistical difference and challenges in designing and conducting such studies in operational settings. The design, workflow integration, and usability of clinical information systems are all factors in creating effective CDS.

In this chapter, we will review the definitions of Clinical Decision Support and Clinical Decision Support System, the different types of active CDS, (e.g., alerts, reminders, corollary orders), briefly review the impact of care settings and vendor on CDS design, methods of implementation for success, and future challenges and opportunities.

Fundamentals:

  • Defining and delineating Clinical Decision Support (CDS) and Clinical Decision Support Systems (CDSS)

  • Types of CDS

  • Governance

  • Factors for effective and successful implementation of CDS.

Defining Clinical Decision Support and Clinical Decision Support Systems

Clinical decision support (CDS) can be defined as anything that offers patient specific information in a timely fashion, in alignment with workflows, to aid and improve patient care. CDS is cited as improving clinician’s performance but also processes within healthcare [1]. CDS encompasses a gamut of aides from colleagues to electronic alerts. Clinical decision support systems (CDSS) are computerized systems designed to impact clinical decision making about individual patients. For purposes of this chapter, we are only referencing electronic or computerized forms of CDS for use in patient care. Since so much of the CDS referenced and studied today is clearly CDSS, the distinction between CDSS and CDS may be lost. Therefore, we use the terms interchangeably.

In this chapter, we address the history and architecture of clinician facing CDS and prior classification schemes for CDS tools. As health information technology (IT) evolves, delivery methods as well as classification schemes for CDS no longer fit neatly within prior frameworks. Finally, we describe the implementation and maintenance of CDS with newer frameworks in mind.

Architecture of CDS Systems

As Associate CMIO, you’re being tasked initially with learning about the architecture of CDS within your system broadly. Can you identify your Clinical Decision Support System’s Rules Engine, Knowledge Base and Clinical Repository?

History of CDS

The earliest CDS evolved in the 1970s. The Leeds abdominal pain system , developed in 1970, sought to identify causes of abdominal pain. Timely and accurate diagnosis is essential as pain can be managed either medically or surgically. This branch point is critical. The tool used Bayesian probability to identify the level of certainty for each diagnosis.

Internist-I broadened this scope of diagnostic decision support tools and attempted to provide diagnostic support across 500 disease states; this tool was also able to interact with the clinician to provide follow up questions thereby narrowing a diagnosis. However, its limitations included an inability to take anatomical or temporal information into account. Internist-I was also unable to provide the user an explanation or reasoning behind its recommendation.

Dxplain intended to build upon Internist-1 by offering explanations for its diagnostic reasoning, a feature that was not built into Internist-I. MYCIN and HELP were two forms of therapeutic decision support; rather than offering diagnostics, they aimed to help clinicians derive appropriate next therapeutic steps. The HELP system employed CDS that analyzed events directly from the electronic health record (EHR) and presented them to clinicians. HELP was initially used in cardiac catheterization labs and later aided in reducing medical errors, as well as antibiotic prescribing features [2].

CDS Evolution

CDS tools have evolved since the 1970s. There have been four major phases of CDS evolution since the 1970s.

  1. 1.

    Stand-alone CDS systems which are generally limited to one area of medicine (Internist I, Dxplain, MYCIN) as described in the history section

  2. 2.

    Integrated systems which draw data from the CPOE or EHR (HELP)

  3. 3.

    Standards for CDS rules development (ARDEN SYNTAX)

  4. 4.

    Service models which separate the clinical information system and the CDSS and subsequently integrate using an application programming interface (API) (Sage and SEBASTIAN) [3]

Many early CDS tools employed decision tees and Bayesian probabilities. However, evolution of the clinical information systems that CDS often reside in and the capabilities of computer systems and standards that support CDS tools has forced changes in how we design and construct these tools. Integrated rule based and standards-based alerts offer clarity, precision and less ambiguity with respect to why an alert fired. Finally, service models (external CDS tools that function through an API and transmit data to a commercial EHR) offer modularity and performance benefits as advances in computer technology grow; although there are vendors who offer such services, this is currently not a consistently adopted practice. We will now focus on rule based CDS as well as CDS standard for Rules and Knowledge Representation.

CDS Components

CDS design employs three major components that, in aggregate, form a CDS System:

  1. 1.

    The clinical event monitor detects new information as it comes into the system [4, 5].

  2. 2.

    The rules engine tells the clinical event monitor that there is, or is not, a rule that pertains to the clinical data entering the system.

  3. 3.

    The knowledge base is a database of rules.

This modular design allows one to change the rules without having to redevelop the entire CDS tool.

CDS Standards for Rules and Knowledge Representation

Arden Syntax is a widely recognized standard initially created in 1989. Rules within the Arden Syntax model are called Medical Logic Models [6]. Each rule is comprised of three sections called the “maintenance”, “library” and “knowledge” sections. The maintenance section contains meta-data about the rule including who owns it, when it was created and when it was last reviewed. The library section contains meta-data describing the purpose of the rule, as well as a citation to the guideline or data supporting the rule. Finally, the knowledge section contains multiple subsections which encode computable parts of the rule; this includes a subsection of logic, and actions as well as urgency.

Arden syntax is patient specific and event driven; therefore, it cannot be used for population-based decision support nor can it be used as a point of care reference. Arden syntax however can be used for drug-drug interactions and critical lab alerts. Secondarily, the vocabulary within Arden syntax is not defined; institutions define the manner in which labs and procedures are coded, perhaps creating challenges with interoperability across institutions. However, Arden Syntax was designed to be shared.

Notably, Arden syntax has been revised since its inception in 1989 and the most recent version is an ANSI and HL7 standard. In fact, some vendors support CDS based on the Arden Syntax model and non-EHR vendors sell CDS tools which follow the Arden syntax. Specifically, the Medical Logic Modules are sold and have the capacity to interface with EHRs.

GLIF (guideline interchange format) was first introduced in 1998 and was developed to support guideline modeling as a flowchart or on complex decision-making steps. To the best of our knowledge, this has not been implemented or integrated within any commercial EHR to date.

With the advent of the HITECH Act in 2009 which lead to the creation of the Meaningful Use program, EHRs were required to demonstrate creation and use of increasing number of CDS alerts. Several measures or criteria in the Meaningful Use program suggested the creation of CDS alerts. Stage 1 of Meaningful Use simply required demonstrating use of one CDS tool (e.g., alert). Stage 2 required multiple CDS alerts. Stage 3 specified that CDS alerts must be tied to quality measures or, in other words, demonstrably changed behavior. The emphasis on outcomes seemed to encourage the development of complex CDS tools.

Types of Clinical Decision Support

As Ambulatory Associate CMIO you want to better understand the use of CDS in ambulatory care and decide to begin by focusing on improving the rate of screening mammograms. Medical group leadership recommended an alert for all outpatient visits, not those with a history and physical. The doctors within the group have resisted, insisting that they always ordered a mammogram if it was due.

How would you identify the best way to deliver the CDS for all visit types?

Active (Push) Versus Passive (Pull) CDS

CDS can be delivered either passively or actively. Passive decision support is CDS in which the user actively seeks information via a ‘pull’ format. The tool requires first, the user to provide an action and seek out the information voluntarily. Passive CDS is considered non-interruptive in that it does not disrupt workflow as it is requested by the user during their desired workflow. It may be available for use within the EHR or outside the EHR (e.g., a website).

Examples of passive decision support are when a clinician happens to be charting in the EHR and realizes they would like additional information to help with diagnosis. A button then redirects the clinician to UptoDate or DynaMed. This information is not patient specific.

Alternatively, InfoButtons within the EHR can also aid in passive decision support. These can be patient or disease specific [7, 8]. An example of a “pull” type decision support is when a clinician is presented with a patient with low calcium. As the clinician, you’d like to correct for low albumin; there is an integration in the EHR with a medical calculator however the clinician has to insert the calcium as well as the albumin data into the calculator.

“Push” clinical decision support, which can also be called active decision support, is wherein the receives information and guidance that they neither expecting nor requesting. For example, the clinician while writing a note the clinician receives an alert telling them that the patient’s potassium is low and directing them to supplement it has low potassium. This alert occurs while the clinician is writing their note or reviewing the problem list. This is equivalent to notifications on a smartphone wherein the user is presented data without seeking it out [9].

Active Decision Support: Actionable Versus Non-actionable

Active decision support can be divided into two categories, actionable and non-actionable. Actionable is best defined as CDS wherein information and options are presented that can be acted upon. Ideally all of the information required to act upon the information is provided within the decision support. Non-actionable is simply information being presented to the clinician, but this may not be patient or disease specific and require interpretation by the user. For example, the clinician is notified that a patient’s potassium level is high however there is no text, orders, or options guiding the clinician on steps to remedy the elevated potassium value.

For the purposes of this chapter, we will focus on only active and actionable decision support. This includes alerts, reminders, corollary orders, and guidelines.

Alerts are anything that requires the user to act without delay. An example of an alert is notification of a critical potassium result on a patient who is admitted; an alert appears to show the clinician this value. Reminders are used to inform the user of something that needs to be acted upon, although not necessarily emergently. Examples of reminders are to prompt the clinician to schedule a screening colonoscopy, act upon an elevated hemoglobin A1c value or cholesterol value. When an initial order is placed, corollary orders are provided as additional suggestions; for example, the user places a warfarin order and subsequently a corollary order for a daily INR is suggested by the CDS tool.

Guidelines “are systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances” [10]. Guidelines can be challenging to computerize as often; they consist of multiple pieces of discrete information and pathways [use citations within Kannry Framework paper]. An example is whom should be screened for lung cancer, or when and how should patients be vaccinated for rabies. Guideline interpretation may require large amounts of data, complex decision trees and multistep data input and output tools. GLIF is a format that is built specifically for computerization of guidelines; however, most commercial EHRs are unable to effectively integrate it.

Hybrid Decision Support

Hybrid decision support has characteristics of two or more forms of classic decision support, and is usually an artifact of commercial EHR design. Examples of hybrid support are health maintenance, order sets and panels, as well as soft hard stops in commercial EHRs. Inline alerts, too, are a type of hybrid decision support tool.

Pop-up alerts fit the classic model of alerts being brought to the clinicians’ attention. These qualify as active and actionable model as stated above. Inline alerts, though they meet the hybrid criteria, are passive as they require the user to look for the alert and active in that the user was expecting or requesting the information.

Order sets are also a form of hybrid CDS as they allow clinicians to view and act upon additional orders for a specific diagnosis. Order sets can also have embedded decision support such that specific diagnoses or clinical conditions prompt orders to appear; an example would be a generic order set for a urinary tract infection in a male versus female prompts differing duration of antibiotics. On the one hand, this resembles corollary orders in that additional suggested orders are presented.

There is really no analogous form of classic CDS for the guided documentation that vendor systems provide. Through the use of smart forms and templates, users are directed to generate documentation by choosing from suggested lists

Health maintenance and headers are vendor specific functionality and do not necessarily fit neatly into a classic CDS structure. They are hybrid forms of CDS with characteristics of multiple classical forms. Health maintenance topics as we define them are screenings and immunizations that are performed and tracked. Health maintenance is passive in that the clinician must look for it in an inline menu. It is active in that actions can be taken; recommendations are patient specific and orders can be easily written. Headers in the classic model did not provide decision support; however, they can be specific to a care setting and triggered by specific conditions almost like an alert or reminder. Headers are present within the EHR’s user interface, don’t necessarily require input from the end user to seek out the CDS via a pull phenomenon, nor are they necessarily presented to the end user via a “push” phenomenon. Headers are present within the screen and provide decision support via a hybrid format. Similarly, Health Maintenance tools offer decision support for routine screening tools. These forms of CDS are present in the chat, visible for the clinician to see; however, they are not presented in a pull or push format. This underscores the importance of both vendor specific functionality and user interface and screen design.

These hybrid decision support tools do not directly push information to the clinician, nor do they require the clinician to seek out decision support. These tools have a mix of both modes of CDS delivery (active and passive), as well as a mix of various types of decision support (reminders and alerts).

Care setting specific CDS tools also emerge as vendors recognize the variation in workflows between ambulatory, inpatient, and emergency care settings and vendors provide comprehensive solutions that span multiple care settings. For example, health maintenance and screenings are only available in ambulatory settings. There is a chronic (ambulatory) problem list, and hospital problem list (acute care), and emergency department impressions. A pop-up or interruptive alert for colonoscopy screening for is extremely helpful in ambulatory setting, but it is likely a hindrance to inpatient or intensive care-unit clinicians. In light of this, restrictions surrounding where and when the CDS Tool appears within the user interface is important.

Hard stops are another type of hybrid decision support. These classically mean the clinician cannot take any action in the EHR until they perform the necessary action that the alert or reminder recommends. An example is a hard stop for venous thromboembolism assessment orders in an inpatient admission order set; there are times wherein chemical venous thromboembolism prophylaxis is inappropriate or not appropriate to order at that time. Forcing the clinician to decide at the time of placing the admission order set may not be appropriate.

An inline “soft-hard stop” is one that prompts the user to perform an action; however, the user can easily navigate around the decision support on the screen. Unless the clinician chooses to click on the alert, they are able to perform alternate aspects of workflow with the inline hard stop being present on the screen. This functions as a passive tool as the user could choose to enter into the hard stop by clicking on the inline alert, but is also an active tool as if selected, the user has to address the soft hard stop.

These Hybrid tools are increasingly common, prompting the question as to whether our prior classification schemes are applicable with the advent of commercial systems and which lessons learned apply. Perhaps further study is needed of these new forms of CDS.

CDS Delivery Mechanism: Internal Versus External

Ultimately your institution considered an interruptive (pop-up) alert in ambulatory practices to ensure an increase in appropriate mammography orders. Clinicians were inordinately unsatisfied stating these alerts disrupted workflows. A non-interruptive alert was settled on. A yellow box appeared in a section titled alerts when a patient was due for the mammogram. One click ordered the mammogram, associated the order with the diagnosis of routine screening, and provided the patient with instructions and a map to the imaging center.

What mode of CDS delivery is this? Are there ways to leverage externally delivered CDS for this case?

Traditionally Delivered CDS

Traditionally delivered clinical decision support is provided to the clinician through the EHR. Alerts, reminders, corollary orders may be delivered through multiple mechanisms including the computer screen, texting, or paging. However, the processing of rules and logic occurs solely within the EHR.

Externally Delivered CDS

In externally delivered decision support clinical data leaves the EHR and is processed external to the EHR in a separate CDSS (Clinical Decision Support System), then ideally returns to the EHR with specific recommendations as well as actions (e.g., orders) for the EHR end user to perform. Alternatively, content is provided external to the EHR, delivered to the EHR, and subsequently processing is completed within the EHR. A critical characteristic of external CDS is both its modular capacity and ability to standardize CDS across enterprises and users.

Delivering External CDS

The vision amongst clinicians as well as the AMIA Board of Directors in their roadmap for national action on clinical decision support cites the need for a “robust infrastructure for developing and delivering CDS interventions” that are not necessarily vendor specific, but interoperable in nature [11]. Clinicians also expressed interest in making CDS available through a “public knowledge repository” as shown in a survey and interview-based study by Kawamoto et al. [12]. In alignment with these goals, there are multiple consortiums that aim to develop Clinical Decision Support tools and systems that are shareable and standards-based on a national platform including CDS initiatives through the Agency for Healthcare Research and Quality (AHRQ), Open CDS, CDS Hooks, and Protecting Access to Medicare Act (PAMA)

AHRQ

In 2016, the AHRQ launched a series of grants which aimed to advance CDS by supporting clinicians and informaticists in developing CDS tools [13]. The AHRQ goal was to create freely available, interoperable tools which aim to promote collaborative models of CDS. The CDS Connect Project (Fig. 7.1) allows clinicians and provider organizations, health IT vendors as well as federal health research organizations to collaborate. This approach was successful in piloting several external CDSS, including one that increased the adoption of preventative service guidelines for patients with chronic conditions in Indianapolis and Boston [14].

Fig. 7.1
figure 1

CDS connect life cycle. Source: https://cds.ahrq.gov/cdsconnect/about

Full size image
Open CDS

Open CDS is another multi-institutional and collaborative efforts to develop standards based CDS tools licensed under the Apache 2 license. This project was initially envisioned by Dr. Kensaku Kawamoto in 2010 with the major goals of :

  1. 1.

    Transforming proprietary sources of data from the EHR

  2. 2.

    Evaluate data using a set of rules based on the latest medical knowledge

  3. 3.

    Return appropriate treatment suggestions (OpenCDS.org)

OpenCDS clients can use CDS Hooks and HL7 FHIR (Fast Healthcare Interoperability Resources) as a data model for input and output connecting to their EHR. Rules within OpenCDS can be written using Java, Drools (a business rules management system), HL7 CQL and any other custom rules language. The OpenCDS community is open and freely available for all to join encouraging active engagement.

CDS Hooks

CDS Hooks (Fig. 7.2) is an open-source system that builds a CDS service; it divides its built into a CDS client (EHR, CPOE or clinical workflow system) and a CDS Service (any external service that responds to the CDS client request through cards) or a SMART app (an application which utilized SMART a reusable medical programming technology as described below) [15]. CDS Hooks creates a simplified process by which an EHR triggers a “CDS HOOK” thereby invoking a remote CDS service that is external to the EHR/CPOE. The CDS service processes its own logic and rules and obtains data through a FHIR API (see Chap. 13). The CDS Hooks service then returns “CDS Cards” which are displayed by the EHR. Figure 7.2 is an example of CDS Hooks.

Fig. 7.2
figure 2

An example of CDS Hooks. Source: https://cds-hooks.org/#how-it-works

Full size image
SMART on FHIR

SMART (Substitutable Medical Applications, Reusable Technologies) began at Boston Children’s Hospitals and Harvard Medical School’s department of Biomedical Informatics through a grant from the ONC (Office of the National Coordinator for Health Information Technology) [16]. The purpose was to build standard frameworks that allow interchangeable healthcare applications. SMART on FHIR allows for applications to easily and interchangeably develop, install, and update clinical decision support modules, taking advantage of the FHIR standard. Notably, SMART on FHIR standards can be used for non-CDS tools as well.

Protecting Access to Medicare Act and Imaging (PAMA)

Protecting Access to Medicare Act is an initiative through Medicare and Medicaid services that helps clinicians consult use criteria for every Medicare Part D advanced imaging order. PAMA uses pre-existing guidelines to curate content, and subsequently present the knowledge to the clinician [17]. This CDSS uses clinical guidelines to ensure the most appropriate image is ordered, as well as ensures that optimal quality of care and lower imaging costs are pursued (Fig. 7.3).

Fig. 7.3
figure 3

Alert for Influence Vaccine administration

Full size image

The Reality

Despite access to these CDS tools, getting consensus on the content as well as the process and technical details for implementation remains difficult [18]. Implementing these technologies remains difficult and there remain questions surrounding the practical implementation of these tools.

Knowledge Maintenance

You were informed the guidelines for mammograms have changed. You recollect that previously an alert was implemented for ambulatory clinicians. You struggle to tell your CMIO what version of guidelines you used for the alert, when it was last updated and how you plan to keep track of this information.

What Is Knowledge Maintenance?

Knowledge Maintenance is defined as how an organization “develops, disseminates, maintains, and evaluates its clinical knowledge content” [19]. This is an ongoing task and requires a considerable number of resources as well as tools. Its importance to CDS cannot be estimated, as it this knowledge which supplies the content for CDS.

Knowledge generation and knowledge acquisition occurs prior to knowledge maintenance. First, there must be agreement on the clinical knowledge content by clinical experts using tools to support this practice—this is knowledge generation and acquisition. The knowledge must be consistently represented and stored. Second, there must be consensus on tracking and maintaining both the knowledge within the decision support but also the initial need and vision behind the decision support tool or system as well as the granular changes made within the decision support tool.

Despite varied EHRs, a survey of multiple, geographically varied practice centers (academic and community), found all considered advanced knowledge management systems critical to maintaining CDS [20]. However, in vendor specific EHR systems, knowledge maintenance is a task delegated to each institution or health system. However, this lack of consistent knowledge maintenance complicates CDS design.

There are numerous models suggested in the informatics literature, including creating multidisciplinary teams to maintain the content within your organization, purchasing knowledge from third party vendors, and using online collaborative tools across organizations, to review, aggregate, and maintain the knowledge [19, 21, 22]. Notably, practice patterns vary from academic institutions to community and this distinction surrounding workflows, governance and support is essential in designing a knowledge maintenance infrastructure that is sustainable [22]. Secondarily, although many EHRs have clinical knowledge editors for users to create CDS, few have inbuilt knowledge and content management [23]. Third, although many EHR vendors do offer the capacity to build within their own infrastructure, there remains limited capacity to share and maintain knowledge in a collaborative fashion. Fourth, storing the knowledge in a structured, shareable, and sustainable fashion is critical. Current vendor EHR systems generally do not support this functionality.

What Evidence Is the ‘Right’ Evidence?

Regardless of having a structured knowledge maintenance methodology and infrastructure, there must be organizational consensus surrounding which evidence to use within the CDS tool. This requires consensus amongst key stakeholders, risk and safety, as well as operational leadership. Second, optimizing and aligning knowledge with changing evidence as new knowledge arises is critical; this underscores the importance of representing the knowledge using standards such as Arden Syntax, Health Quality measure format (HQMF) or Cassandra Query Language (CQL). Standards allow for organization of clinical content and evidence. Standards facilitate a clear way to locate, organize and subsequently find the data.

Efficiency and Usability

Your non interruptive alert has been in place for 6 months. You would like to analyze some metrics surrounding its use. How do you plan to analyze the efficiency of your alert? What will you measure? Second, you hope to understand if the user finds the CDS intervention useful and usable. For example, does the user understand the purpose of the CDS intervention? How do you plan to research and provide this information to leadership?

Efficiency and usability are principles applicable to more than CDS. However, CDS specifically has measurement and assessment challenges that are unique. Both efficiency and usability are critical to effective implementation and subsequent analysis of CDS tools.

What Is Efficiency?

Efficiency can be defined as the percentage of time a CDS intervention results in the user taking the desired action. However often times measuring this efficiency is difficult due to the limitations of EHR vendor’s reporting systems.

When one thinks about CDS, the most common tool that comes to mind is an alert. When an alert fires, the user can either accept, adopt ignore, the alert. Acceptance is acknowledging the alert and pursuing the recommended action. Adoption means the user interact with the alert by acknowledging the alert or canceling the alert or unchecking the recommended action [9]. Theoretically this methodology of analyzing acceptance and adoption of the CDS tool can be used for all types of CDS including but not limited to banners, flags, as well as order sets and panels if EHR functionality and reporting supports it.

Assessing the number of firings, or the number of times the CDS tool appeared to the end user is a starting point for assessing efficiency. High rates of alert firing per user or per patient suggest alert being ignored. When assessing CDS acceptance and adoption for high firing alerts, CDS acceptance rates (i.e., following the recommended action) are remarkably low ranging and ignore rates are quite high; in fact, medication alerts have been shown to have ignore rates as high as 96% [9, 24]. In fact, alerts shown in excess can cause a distraction and result in providers missing other critical information and the same finding has been found in medication alerts [25, 26].

Efficiency is impacted by whether or not CDS appeared in the appropriate context, for the appropriate user and at the appropriate time in the workflow One could perform analysis of whether an alert is effective by doing usability testing, structured interviews with users, as well as surveys. Given these variables, an objective number of firings may not fully capture the picture of whether a CDS tool is firing effectively.

Second, while it is comparatively easier to assess user interaction with the CDS intervention, it is much harder to assess the relationship between CDS and clinical care outcomes. This can aid in determining whether the alert truly is changing clinician behavior—one goal of any and all CDS. Traditional clinical measures such as number needed to treat are difficult to obtain. For example, how many times did the mammogram alert fire before one mammogram order was placed? Did the total number of mammogram orders go up and how many were a direct result of the alert? These queries are actually difficult to do with current EHR vendor functionality. Finally, the ultimate comparison of alert firings and improved detection of breast cancer due to screening is even harder to do.

Studies have attempted to measure the efficiency of clinical decision support systems. Many of the studies are on medication alerts given that acceptance or adoption of the alert and subsequently following recommendations the CDS provides is more easily measurable [27]. A few studies have attempted to quantify changes in scheduling and follow up patterns as well as clinical outcomes such as screenings and lab metrics [28]. Other studies have attempted to use QI methodology to analyze interruptive alert burden and found that a systematic methodology allows for a targeted way to reduce alert burden [29].

In addition, studies have attempted to solicit end user feedback on both the efficiency and usability of an alert [30]. The study identified alerts that required override comments and analyzed the content of each comment in context of where the alert fired and the ultimate actions taken; they found that alert override comments provide a wealth of data surrounding alerts that are broken and/or malfunctioning, therefore can be improved. Some of these studies have also begun to look at whether alerts are broken [30].

Usability

Of course, ultimately the goal is to create an EHR that is easy to use and a joy to use [31]. Usability entails three key principles:

  1. 1.

    Usefulness—meaning the tool is something that is desired and provides utility

  2. 2.

    It’s ‘usable’—meaning the user can navigate the tool easily and effectively

  3. 3.

    Does the tool fit into a workflow, or a pattern of use

Some also include safety (i.e., is it safe to use) within usability frameworks.

Usability engineering is critical, especially in CDS given high interaction with the tool and an expected interaction and subsequent action to the tool. Focus groups have been tested and clinicians as well as end users have previously been involved in studying the CDS tool as well as performing usability testing [32, 33] In addition, usability engineering techniques such as usability testing can be used. The user is systematically walked through a series of steps and their responses are analyzed.

Governance

Finally , in order to effectively support measuring both usability as well as efficiency, an institution must create and implement an effective governance structure. Governance includes an institutional structure and framework to monitor and regulate new CDS implementation, maintain the CDS that currently exists in the system, and finally ensure that malfunctioning CDS or CDS that is simply ineffective is essential. However, too rigid of a governance structure, can slow and limit agility and capacity to make changes quickly. A balance between flexibility and rigidity is essential.

Wright et al. conducted site visits at five organizations and reviewed best practices from these institutions which were: creating committees that included CDS related staff, creating and sustaining a process for knowledge management as well as customization and finally creating a process for review and monitoring [34]. A second study by Kawamoto et al provides a “pragmatic guide” to establishing CDS governance; again, site visits were conducted and each organizations’ respective resources allocated to CDS, committees / working groups as well as an individual alert and efficiency as well as metrics were reviewed in the paper [35]. Ultimately however there are few systematic reviews or large-scale studies on best practices for CDS governance models given variation in resources, budgeting and organizational structure. There are however major themes that emerge from the studies that do exist—notably (1) development of a CDS committee or working group, (2) engagement of critical stakeholders, (3) A structured intake, maintenance, and expiration process, and finally (4) systematic maintenance of knowledge and data within the CDS.

Successful Implementation of CDS

You met with your boss, the CMIO, and overall, the implementation of the interruptive alert was deemed a moderate success. The data you presented demonstrated that the alert has a 10% acceptance rate in the ambulatory practice (meaning only 1 out of 10 users pursued the recommended action presented and ordered a mammogram).

What factors do you feel could be improved for better utilization? How would you systematically review usage and look to the future to improve upon the low acceptance rate?

Beyond the Five Rights

The implementation of a CDS tool must be well thought out and planned both from an efficiency standpoint and a usability standpoint. The “Five Rights” of CDS offers a framework for factors to consider in a successful implementation [36]. The five factors to consider are:

  1. 1.

    Getting the right information

  2. 2.

    To the right person

  3. 3.

    In the right format

  4. 4.

    Through the right channel

  5. 5.

    At the right time

This framework in short states that the decision support tool should present the appropriate, evidence based, patient specific information to the appropriate clinician, at the right time. For example, a patient who is a high falls risks enters the emergency department. The patient is registered and a pop up appears indicating he is high falls risk however does not include a risk score or any information on prior falls not does it include a way to place an order or alert other staff of this risk. This underscores the importance of the first of the Five Rights; it is critical that the data is appropriate. Second, it is critical that the data is provided to the appropriate user in the right format; the registration staff can ensure the patient’s status is known to other hospital staff as well and throughout their ED stay but cannot order and ensure the patient has a wheelchair and is safe. A more appropriate way to communicate the information would be to send the alert to the nurse or physician at the time of admission to ensure they could place an order for this along with a physical therapy referral if need be. The channel of delivery here is likely appropriate in that the alert is delivered through the EHR; however alternate options include secure delivery through email which is not urgent enough, nor is it within the clinician’s workflow. Finally, the alert must be delivered at the appropriate time; delivering this alert during admission is appropriate however delivery at discharge would have far less utility.

Although the Five Rights model is effective at analyzing an individual CDS tool such as one alert, the five rights model does not address the larger role of governance and maintenance. Furthermore, it does not explicitly address the critical role that workflow analysis plays in determining for whom, what format, and what channel through which to display a CDS tool. It also does not address the role that usability plays in assessing effectiveness of an alert.

Factors for successful implementation.

While the above Five Rights framework allows for a broad framework to assess CDS, successful implementation heavily depends upon IT infrastructure, governance and organizational culture. Major factors apart from content and usability are analysis of alert fatigue and metrics post implementation, workflow integration as well as role-based distribution of alerts to ensure targeted decision support.

In addition to the Five Rights framework, there are additional and complementary frameworks that consider active and actionable alerts, training as well as resources, workflow context, and usability [9].

Legal, Regulatory and Ethics

Members of your CDS governance committee are concerned that the non-interruptive alert aren’t prompting them enough to order screening mammograms. They are concerned about liability surrounding the alert if they choose not to follow the recommendations provided. Alternatively, what if the CDS misses a patient who should, in fact, be screened and they later develop breast cancer?

Providers wonder if the vendor will be sued. Some members seem to feel the FDA is regulating the electronic health record and CDS. They are concerned and would like to ensure compliance with government regulations as well as delivering superior patient care.

The Role of the FDA in Regulatory Oversight of CDS

As software continues to become integrated with EHRs and the app economy grows, the term Software as a Medical Device (SaMD) has emerged. This is defined as software intended to be used for one or more medical purposes that perform these purposes without being part of a hardware medical device (Software as a medical device). This definition created by the International Medical Device Regulators, a consortium of medical device regulators from around the world, aims to classify the impact of these software applications as well as their risk posed to clinical practice, quality management and clinical evaluation (Software as a medical device).

Notably, however a large distinction is that the FDA has no formal role in evaluating CDS that is internal to the EHR. With external devices sending information to EHRs through an increased availability of APIs, a critical piece that is missing is how the FDA “intends to distinguish machine learning software that can explain its recommendations to a physician from software that cannot” [37]. These challenges will continue to rise as technology evolves.

Legal Model for CDS

The legal model upon which lawsuits are generated in relation to CDS is one of negligence; meaning the clinician pursues a plan of care that is contrary or opposed to that of common practice [38].Therefore, unless gross negligence is proven, the CDS tool or vendor is usually not implicated in patient outcome.

Vendors and CDS Content

Despite this, vendors are hesitant to provide the content within the CDS tool—although some vendors provide a starter pack of CDS content [19].This leaves a gap in the market for CDS content providers. Commercial products have emerged that both provide evidence-based guidance and clinical content, but also maintain, update and monitor the use of these tools.

Impact of the 21st Century Cures Act

The 21st Century Cures Act impacts CDS through the information blocking section of the regulation. This releases all appropriate data to patients through electronic means. The information to be shared electronically includes notes as well as lab results, imaging results, and routine screening information. Given this, the implication is that data will be available and freely shared from disparate clinical sources. The information available to guide and recommend CDS will be far more robust. This prompts questions surrounding first how to obtain the data, whether the data will be shareable and transferrable.

Ethical Challenges of Adaptive CDS

Adaptive CDS is defined as CDS that can “learn and change performance over time, incorporating new clinical evidence” new data and new methods for interpreting data [39]. There are not only legal challenges with AI and Machine learning based CDS but also ethical challenges. As access to data and information grows, adaptive CDS, or CDS that can “learn and change performance over time” is increasingly incorporated into clinical practice [39]. An AMIA position paper discusses specific recommendations surrounding transparency metrics, communication standards, ongoing maintenance and in situ evaluations and testing. Concerns surrounding hidden biases due to poorly defined training sets, as well as AI generated bias to increase health disparities. Notably there remain concerns surrounding racial, ethnic and gender disparities inherent in AI models [40].

Emerging Trends

CDS has developed significantly since the time of being a component of internally developed clinical information systems. However most current CDS still employs the same architecture of a rules engine and knowledge base. Complex multistep algorithms are still challenging to implement despite existing standards such as GLIF that can handle these complex decision support trees.

Three themes that are evolving in CDS are: (1) Commercial solutions for knowledge maintenance, (2) Adaptive CDS and (3) workflow integration for new types of devices that deliver CDS to the user CDS.

EHRs are not equipped with easy to use and robust content management functionality nor do they have good management tools for CDS tracking. For example, there is no easy way to monitor who build a CDS tool, why it was built and the intent behind the tool. Increasingly there are commercial solutions to manage CDS assets, assess efficiency and determine outcomes. These commercial solutions come at a cost and are packaged and not yet well-integrated with EHR vendors.

Adaptive CDS in which artificial intelligence and machine learning grow and change recommendations based on their new data sets is another area of growth. The biggest challenge will be determining appropriate training sets, whether training sets are representative of a population, and ensuring bias is addressed.

Finally, a critical piece of CDS is its placement within individual workflows. Increasingly there is technology that goes beyond delivering alerts solely within an EHR. CDS can be delivered through mobile applications, and wearable devices as well.

Patient centered CDS, or CDS delivered directly to the patient, is in its early phases. As we develop CDS tools, we expect this area of CDS to grow.

Summary

Clinical decision support (CDS) is any tool that aids in clinical decision making about individual patients. Clinical decision support systems (CDSS) however are computer systems that perform this function. CDS evolved in the 1970s and has developed significantly since this time. Standards such as Arden Syntax were developed to ensure standard programming logic and knowledge representation in the development of CDS tools. CDS can be presented to the end user either in an active—or push—methods of delivery versus passive; this can further be developed into actionable versus non actionable CDS which guides the user to perform a subsequent action. Increasingly commercial EHRs pursue hybrid tools which do not fit neatly under either classification. CDS can also be delivered internal to the EHR versus externally; there are many initiatives that aim to integrate external modules with the EHR to bolster commercial CDS. Despite the availability of these tools, the implementation of CDS ultimately depends on much more than the tool itself; frameworks for implementation such as the Five Rights model discuss the “who, what, when, where and how” of CDS. Other frameworks emphasize the importance of workflows, governance and documentation [9]. In addition, the efficiency of the CDS tool depends heavily on factors discussed in the prior frameworks as well as usability, an area that is under study in CDS and workflow context. Increasingly however newer technologies such as machine learning and artificial intelligence are being used to generate CDS; with this comes questions surrounding regulatory affairs as well as ethics of tools that recommend and guide clinicians to pursue specific practices in the healthcare space. As these technologies grow and develop at a rapid pace, CDS will be an area of intense focus

Questions for Discussion

  1. 1.

    How has clinical decision support evolved since its inception in the 1970s?

  2. 2.

    What are the different modes and methods of CDS delivery? Can you describe scenarios wherein the manner in which CDS was delivered was inappropriate?

  3. 3.

    What frameworks can be used to improve implementation of CDS?

  4. 4.

    Describe ways in which knowledge management systems impact implementation and management of CDS

  5. 5.

    Brainstorm ways in which CDS can be optimized. How would you measure changes?