Introduction

Clinical decision support systems are used in medicine to assist in making the best decision regarding patient care. A few studies on clinical decision support have confirmed that its use with medication administration results in improved quality of patient care [1]. The rationale for using clinical decision support in medical imaging is to improve patient care and decrease the number of unnecessary imaging studies.

Because of new and improved imaging technologies, the volume of medical imaging examinations has increased substantially during the last 20 years [2, 3]. Between 2000 and 2007, use of imaging studies grew faster than that of any other medical service in the Medicare population [4]. Some of the growth has been driven by appropriate use of new imaging technologies. However some of the increase might be attributed to “defensive medicine,” self-referral for imaging studies from clinicians, fragmented care across multiple hospital systems, inexperienced ordering physicians, and patient expectations that might not match evidence-based imaging algorithms [5]. A report by America’s Health Insurance Plans claimed that 20% to 50% of all “high-tech” imaging provides no useful information and might be unnecessary [6].

Pediatric imaging has increased in tandem with this overall growth in diagnostic imaging. One of the roles of a pediatric radiologist is to guide use of imaging. However the imaging volumes outstrip a commensurate increase in fellowship-trained pediatric radiologists [7].

The United States government has identified clinical decision support software as a vehicle to improve patient care, decrease inappropriate imaging exams, and promote judicious use of imaging resources [8]. To help catalyze this change, the 2014 Protecting Access to Medicare Act requires referring providers to consult appropriate use criteria prior to ordering advanced diagnostic imaging services, which include CT, MRI, general nuclear medicine and positron emission tomography (PET) [9]. In November 2017, the Medicare Physician Fee Schedule final rules were published and stated that ordering providers must consult appropriate use criteria through clinical decision support mechanisms for studies ordered on or after Jan. 1, 2019 [10]. The Protecting Access to Medicare Act program starts with a voluntary reporting period from July 2018 through December 2019. On Jan. 1, 2020, ordering professionals must consult specified applicable appropriate use criteria through qualified clinical decision support mechanisms for applicable imaging services, and report the appropriate use criteria consultation information on the Medicare claim. The onus of appropriate use criteria consultation documentation is currently the responsibility of the ordering providers. These guidelines will only affect Medicare providers at first. Standalone pediatric hospitals will not immediately be affected because Medicare is a relatively small payer for them. However many insurance companies generally adopt Medicare payment guidelines, and therefore pediatric institutions are likely to be subject to similar requirements in the future.

The clinical decision support imaging software takes input from an ordering provider — generally a clinical sign or symptom — and, based on this input, suggests a list of imaging examinations that would be most appropriate to perform as part of this diagnostic workup. The provider is then able to select from this list of examinations. The backbone of the clinical decision support software is the appropriate use criteria. The appropriate use criteria are guidelines that are developed by a qualified provider-led entity [11]. Appropriate use criteria are evidence-based clinical practice guidelines that assist professionals who order and furnish advanced diagnostic imaging services (CT, MRI and nuclear medicine, including PET) to make the most appropriate imaging procedure choice for a specific clinical condition [12]. One prominent set of appropriate use criteria is the American College of Radiology (ACR) Appropriateness Criteria [13]. Finally, a qualified provider-led entity is a national medical society or other organization composed primarily of medical professionals that has applied to and been approved by the Centers for Medicare and Medicaid Services to develop appropriate use criteria. These entities can be divided into two broad groups: medical societies and large medical systems (Table 1) [11].

Table 1 Qualified provider-led entities as of June 2017 [11]
Full size table

Role of the American College of Radiology

The ACR has been developing appropriate use criteria since 1994. The initial goal of the Appropriateness Criteria was to define national guidelines for the use of imaging technology. Currently, these criteria cover 176 diagnostic radiology topics with more than 883 clinical variants. In the pediatric-specific arena, the coverage is more limited. The ACR Appropriateness Criteria cover 12 pediatric-specific topics spanning 56 clinical variants. The key audiences for Appropriateness Criteria documents are medical trainees and referring clinicians.

The mission of the Appropriateness Criteria committees is to systematically review the evidence to develop guidelines to assist referring clinicians in ordering the most appropriate imaging for specific clinical conditions. The hope is that by generating these guidelines and encouraging their use, the ACR will help enhance the overall quality of patient care and contribute to the most efficacious use of radiologic resources. The ACR Appropriateness Criteria methodology is based on the RAND/University of California at Los Angeles (UCLA) Appropriateness Method User’s Manual [14], where “the expected health benefit (e.g., increased life expectancy, relief of pain, reduction in anxiety, improved functional capacity) exceeds the expected negative consequences (e.g., mortality, morbidity, anxiety, pain, time lost from work) by a sufficiently wide margin that the procedure is worth doing, exclusive of cost” [15]. The goal of this method is to allow a group of experts and stakeholders to objectively determine the benefits and harms of performing imaging based on a systematic review of the evidence.

Development of ACR Appropriateness Criteria

The ACR has two or three panels for each diagnostic radiology and interventional radiology subspecialty (e.g., cardiac imaging, pediatric imaging) as well as a committee to review final Appropriateness Criteria documents and subcommittees on radiation exposure and methodology. Each specialty has a chairperson and comprises more than one panel. The pediatric specialty has two panels: one focusing on neurological topics and the other on body, cardiac and musculoskeletal imaging. Each panel is supervised by a chairperson and contains approximately a dozen members. The panel members are invited to serve by the panel chair. The panel chairs recruit ACR members who reflect the practitioners of their specific panel topic. Additional panel members are also assigned by the ACR staff to represent the expertise of other subspecialties (e.g., the nuclear medicine specialty assigns one of its members to serve on a pediatric specialty panel). Consequently, each panel consists of members from a variety of practice environments including private practice, hybrid academic, hospital-employed and large academic settings. The experience of panel members ranges from recently fellowship-trained to nearing retirement. Members often have differing areas of expertise, resulting in a diverse array of perspectives for the clinical condition at hand. Additionally, for each topic there is a primary care representative nominated as a member from a medical specialty society. For example, a consultant from the Societies for Pediatric Urology was involved in the development of the pediatric hematuria Appropriateness Criteria document in 2018. By involving members of other specialty societies, the appropriate use criteria are more clinically integrated and better aligned with the needs of the target audience.

The ACR panels have two main tasks. The first task is to review existing ACR Appropriateness Criteria topic documents. The Centers for Medicare and Medicaid Services requires that each topic be reviewed annually. The yearly review generally consists of the ACR staff and panel chairs conferencing to see whether either is aware of any new evidence that could affect the most current rating recommendations. Members of the panel might be asked for their input on this process, as well. If there is no new evidence, then a full topic review is tabled until the next scheduled full review. If there is new evidence, then the document is scheduled for a full review that year. By default, every Appropriateness Criteria topic is fully reviewed every 3 years. The second task of the panels is to develop new topic documents.

Developing specific topics for ACR Appropriateness Criteria

For each new topic, a standard process is followed. A topic is selected based on a discussion between the chairs and ACR staff regarding the clinical need for appropriate use criteria on a specific topic. Next, a panel member is assigned as the primary author. This primary author may recruit a research author who is a practicing radiologist or radiologist-in-training, who is not a panel member, to assist with literature review (Fig. 1), variant development and document authorship. The primary author is ultimately responsible for the content and therefore he or she mentors and guides the research author. The research author participates in the entire development process for that specific topic. The overall process is summarized in Fig. 2.

Fig. 1
figure 1

Screen shot shows search strategy used for developing American College of Radiology (ACR) Appropriateness Criteria. This was the search strategy used for the topic of suspected physical abuse in a child. JACR Journal of the American College of Radiology

Full size image
Fig. 2
figure 2

Flow diagram shows steps in the development of an ACR Appropriateness Criteria topic

Full size image

The next step of the process is to define the variants. A variant is a clinical scenario in which imaging might be considered to guide optimal management. The variant also defines the patient population by age group, gender and other specific clinical characteristics. The preliminary variants are generated by the primary author and revised by the chairs.

The number of variants varies by topic; there are often 4–8 clinical variants in a given ACR Appropriateness Criteria document. The goal is to create variants that encompass the most common clinical scenarios in which imaging might be considered. During this step, the list of possible imaging procedures for the clinical topic is also defined for each variant.

The variant definitions form the basis for the key words for the literature search. The literature search is performed by the ACR staff with guidance and input from the primary author. The staff and author identify keywords and medical subject headings (MeSH) terms to be searched in the public MEDLINE database. Other fields such as patient age, type of article, language of publication and year of publication might also be used to further refine the literature search. Generally, the literature search is limited to articles published within the last 10 years to keep the search current. Articles older than 10 years can be included at the author’s discretion if they provide evidence that cannot be retrieved by more current literature. Each literature search might undergo multiple iterations to refine search criteria to obtain the most relevant papers for the topic.

ACR Appropriateness Criteria document

Most ACR Appropriateness Criteria documents consist of four main sections. The first section contains the variant tables that indicate the recommendation for each clinical condition and procedure. The second section is the introduction. The introduction defines the scope of the clinical scenario to be analyzed in the variants, disease epidemiology, pathogenesis, common clinical scenarios and management. The third section is a brief summary of evidence on the benefits and potential risks of each clinical condition and procedure. The fourth section is a summary of the key recommendations. The evidence tables and supporting documents accompany the main ACR Appropriateness Criteria narrative document.

The first author writes the initial draft of the topic based on the literature search results. This document is revised by the chairs and later by the ACR staff. The ACR staff completes the summary of evidence section, the special considerations in pregnant patients section and the relative radiation level information section, and provides links to other ACR Appropriateness Criteria topics that are referenced in the current draft. The evidence table is a document that lists all included references with a summary of the name of the reference, the type of study, the number of patients included, the study objective, the study results and a formal assessment of the study quality based on GRADE (grading of recommendations, assessment, development, and evaluation) methodology. After the evidence table and the preliminary draft have been reviewed by the ACR staff, the documents are distributed to the panel members for review. Each panel member is asked to review and comment on the documents, and these comments are then conveyed to the primary author. The primary author makes changes to the draft documents based on the panel’s recommendations.

The next step in the document development process is conducting rating rounds whereby panel members assess the potential risks and benefits of the procedure for each specific variant. Before the first rating round, the updated draft document is distributed to all the panel members. The panel members then all vote on each procedure for each variant on a scale from 1 to 9, where a rating of 1 through 3 indicates an examination that is usually not appropriate, 4 through 6 indicates examinations that might be appropriate, and 7 through 9 indicates examinations that are usually appropriate. Of note, the perceived benefit-versus-harm tradeoff is independent of radiation dose, availability of imaging procedure, examination cost and availability of radiologist expertise when panel members score the examinations. After the first round of rating, a conference call is scheduled where the author discusses the available evidence for benefits and risks for any variant-procedure combination where there is disagreement among the panel members. Disagreement is indicated when the individual ratings vary too greatly from the group median rating as determined by a combination of interpercentile ranges and the asymmetry of the ratings. The exact method is detailed in Chapter 8 of the RAND/UCLA manual [14]. After the conference call discussion, there is a second rating round for those variant-procedure combinations where there was disagreement. After the second rating round, the variant-procedure combinations that still have disagreement are given a rating of 5, with the category “may be appropriate” and a notation that there was group disagreement.

After the rating rounds, the primary author finalizes the document text with the assistance of the ACR staff. Currently, the ACR employs 6.5 full-time equivalent technical and 3 full-time equivalent administrative staff whose duties are to assist with the preparation and review of the ACR Appropriateness Criteria documents. These employees are involved throughout the topic development process and are always available to panel members as a professional resource. Some general roles of the staff members include on-boarding and training of new panel members, making sure that panel members have up-to-date conflict-of-interest statements, helping organize conference calls, and assisting with the publication of the topics in a Journal of the American College of Radiology supplement every 6 months.

The document is then reviewed by the rest of the panel, the professional editor and the oversight committee for ACR Appropriateness Criteria. Once finalized, the Appropriateness Criteria document is made available on the ACR website and published in a Journal of the American College of Radiology supplement, issued biannually.

Challenges

The process of clinical decision support derived from appropriate use criteria for pediatric patients provides new opportunities but faces some challenges. There is a large gap between the number of the pediatric ACR Appropriateness Criteria variants and the number of topics that need to be covered in a clinical decision support system. One of the main limitations of using ACR Appropriateness Criteria for clinical decision support is that the guidelines are related to patient populations and do not consider the numerous patient- and medical-provider-specific variables that can modify appropriate imaging. Costs, ionizing radiation and availability of expertise and imaging modalities are not included in the ACR Appropriateness Criteria considerations but certainly might impact imaging requests. There is also no uniform way to define a topic. It can be defined by symptoms, clinical findings or disease. A physician ordering a study might do so at different stages of the clinical evaluation that do not correspond to the defined topics and variants.

The other challenge is the limited evidence of the benefits and risks of imaging to overall patient outcomes. Appropriate exam ordering and accurate imaging-based diagnosis and risk stratification are often early steps in a child’s disease course. After these steps, the child goes on to treatment and follow-up [16]. All of these steps contribute to overall patient outcomes and there is a paucity of research linking imaging procedures to improved patient outcomes [17]. Additionally, there is limited evidence on quantification of risks related to ionizing radiation, delay in management, incidental findings, costs and anxiety.

Conclusion

We reviewed the rationale behind clinical decision support software and outlined the comprehensive process of deriving the ACR Appropriateness Criteria, which will likely be the basis for decision support software in the years to come. As more adult hospitals implement clinical decision support software to comply with Medicare requirements, pediatric hospitals are likely to follow suit and adopt similar decision and reimbursement strategies. As this process proceeds, there should be many opportunities to decrease inappropriate imaging exams and improve overall patient care.