INTRODUCTION

Antibiotics are often inappropriately prescribed for acute respiratory infections (ARIs) in primary care settings.1 The majority of ARIs treated by primary care providers are caused by viral infections and do not respond to antibiotic treatment,2 yet over half of all ARI visits result in a prescription for an antibiotic, with increasing use of broad spectrum antibiotics.1 This antibiotic overuse leads to the emergence of antibiotic resistant bacteria, creating a growing public health problem.3,4

Over the past few decades, strategies to promote judicious antibiotic prescribing have achieved varying degrees of success. Because multiple factors contribute to a provider’s decision to prescribe antibiotics for ARIs, including awareness of prescribing guidelines,5 perceived patient expectations, uncertainty of diagnosis and concern for potential complications,6 single component interventions only marginally impact prescribing behaviors. Multi-faceted interventions, combining physician, patient and public education in a variety of formats may be more effective at changing prescribing.7 Another strategy sometimes used to reduce antibiotic prescribing is delayed prescribing, where a prescription is given, but the patient is advised to delay using it unless symptoms do not improve. When compared with immediate prescribing, delayed prescribing may decrease use of antibiotics, yet patient satisfaction is also slightly reduced.8

Clinical decision support systems (CDSS) that can target multiple factors impacting antibiotic prescribing have previously been piloted; however, low use by providers has limited assessment of their effectiveness.9,10 The Reducing Inappropriate Prescribing of Antibiotics by Primary Care Clinicians (ABX-TRIP) study was a 27-month demonstration project designed to assess the impact of a CDSS on antibiotic prescribing for ARI in primary care practices, using a multi-method intervention to facilitate its implementation. We have previously reported that, during the first year of the study, the CDSS was widely adopted among the nine participating practices;11 median practice use of the CDSS for adult patients was 58.2 % and 68.6 % for pediatric patients, an order of magnitude greater than in similar studies.9,10 Also, a preliminary analysis conducted after 15 months demonstrated that prescribing of broad spectrum antibiotics in these practices was lower than that of a contemporaneous cohort.12 The purpose of this paper is to present the primary results from the entire 27-month intervention to describe its impact on antibiotic prescribing.

METHODS

Study Practices

The ABX-TRIP study was conducted within PPRNet, a practice-based research network of primary care practices across the United States, whose members use a common electronic health record (EHR) (McKesson Practice Partner® (PP), San Francisco, CA) and pool data for quality improvement and research purposes. Nine primary care practices located in nine states and representing 27 physicians, six nurse practitioners, and six physician’s assistants volunteered to participate in this demonstration project in response to a PPRNet listserv recruitment email. All providers agreed to use the CDSS when seeing patients presenting with ARI symptoms for the study duration. The study was approved by the Institutional Review Board at the Medical University of South Carolina.

The ABX-TRIP Clinical Decision Support System

The ABX-TRIP CDSS has previously been described.11,12 Briefly, the ABX-TRIP CDSS was designed by the research team as a sophisticated point-of-care progress note template embedded within the PP® EHR. Branch logic was built into the template to trigger different diagnostic pathways, based on the patient’s age and presenting symptoms. All diagnosis and treatment recommendations were based on the Centers for Disease Control and Prevention (CDC) “Get Smart” program.13 The CDSS provided assistance with proper diagnosis of ARIs, using assessment questions in the history prompted by a patient’s predominant symptom (including cough, sore throat, sinus complaints or general upper respiratory infection (URI) symptoms). If a patient’s main presenting symptom was “sore throat,” then a reminder with the Centor criteria for diagnosis of streptococcal pharyngitis was included in the subjective portion of the note.14 Additional management recommendations for further testing with rapid streptococcal antigen testing and/or antibiotic were provided, based on the patient’s calculated score. For other presenting symptoms, prompts helped providers make the proper diagnosis based on history (e.g. symptoms lasting 7 days or more when considering a diagnosis of sinusitis) and physical exam findings (e.g. absence of vital sign abnormalities or asymmetrical lung sounds when differentiating acute bronchitis from pneumonia), according to CDC guidelines. Embedded treatment prompts included recommendations about when narrow spectrum antibiotics were appropriate, suggestions for symptomatic management such as antihistamines and/or decongestants, and reminders to providers about the expected length of symptoms (i.e. 10 days for URIs, 3 weeks for acute bronchitis). The CDSS also included hyperlinks to CDC patient education handouts that could be printed and given to the patient at the point of care.

ABX-TRIP Intervention

The ABX-TRIP quality improvement intervention was conducted in two phases: January 1, 2010 to March 31, 2011 and July 1, 2011 to March 31, 2012. During the initial phase, the intervention broadly focused on improving judicious use of antibiotics for ARIs. We conducted an interim analysis between April 1, 2011 and July 1, 2011. Based on our findings, we emphasized limiting antibiotic use for adult patients with acute sinusitis and acute bronchitis during the second phase.

The intervention had three components: 1) meetings for project introduction, CDSS review and “best-practice” dissemination; 2) practice site visits for academic detailing and CDSS training; and 3) EHR-based audit and feedback. Two project meetings attended by two representatives of each practice (one provider and one clinical staff member) were held. The initial project meeting was in December 2009, immediately prior to the start of the intervention. At this meeting, the goals of the project were described, antibiotic guidelines for ARIs were reviewed, and the CDSS was presented. Project representatives provided feedback and suggestions to improve the CDSS. The CDSS was subsequently revised to incorporate these suggestions prior to being disseminated to practices. A follow-up meeting was held in September 2010, at which practice representatives discussed their experiences with implementing the CDSS, including the barriers and facilitators to using the CDSS to improve antibiotic prescribing.

During the first phase of the intervention, practices also hosted two half-day site visits. The first visits were conducted during the first 2 months of the intervention; second site visits were held during months 9 through 11, to coincide with the second ARI season (2010–2011). During site visits, attended by the clinical staff at each practice and conducted by two members of the research team (CBL, SMO, and AMW), guidelines for antibiotic prescribing for ARIs were presented, CDSS training was provided, and practice performance on antibiotic prescribing measures was reviewed. The research team was also able to make minor modifications to the CDSS to accommodate the practices’ workflow. For example, providers at some practices chose to add additional choices for diagnoses to the CDSS, or add specific nursing prompts.

Delayed prescribing was presented to study participants as a potential strategy to decrease antibiotic prescribing. At site visits, providers opting to use delayed prescribing were assisted with adding an extended signature template to the EHR prescription writer with a notation such as “fill in 3 days if not better.”

During the second phase of the intervention, practices were given the option of hosting a final on-site visit or attending a web-based conference. Four practices requested an additional on-site visit; webinars were held with the remaining practices. During these visits and webinars, practice use of the CDSS and antibiotic prescribing for ARIs from the first phase of the project were presented, along with reviews of evidence for appropriate management of acute sinusitis and acute bronchitis in adults.15,16 Prompts were added to the CDSS to help providers set realistic patient expectations for symptom duration, reinforce the viral etiology of bronchitis,17 and discuss adverse effects of antibiotics with patients. Additional patient education materials for sinusitis and bronchitis were linked to the CDSS.

All participating practices continued the usual PPRNet procedure of data extracts. To promote use of the CDSS for improvement, practices received quarterly performance reports documenting their extent of judicious antibiotic prescribing for ARIs. Reports included use of antibiotics to treat ARIs, use of broad spectrum antibiotics to treat ARIs, use of delayed prescriptions for antibiotics, and use of the CDSS for ARI diagnoses.

Study Measures

An ARI encounter was defined as an encounter at which the CDSS was used and at least one respiratory diagnosis was made. Diagnoses that were considered respiratory diagnoses are listed in Table 2. The PP® EHR allows diagnoses to be entered either as International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9) codes or as free text. Although the CDSS included ICD-9 codes for common respiratory diagnoses, providers were also able to enter free text diagnoses when using the CDSS. PPRNet employs both computer algorithms for pattern matching and expert review to assign ICD-9 codes to free text strings.

Antibiotic use was defined as a prescription for an antibiotic written on the day of the ARI encounter. Antibiotics were further classified as broad or narrow spectrum; quinolones, amoxicillin/clavulanate, second and third generation cephalosporins, and azithromycin and clarithromycin were classified as broad spectrum antibiotics. All other antibiotics, including amoxicillin, penicillin, first generation cephalosporins, tetracyclines, erythromycin, and trimethoprim/sulfamethoxazole, were classified as narrow spectrum.18 Intra-venous formulations, polymyxins and aminoglycosides were excluded. A delayed prescription was defined as a prescription written on the date of an ARI encounter for an antibiotic with an extended signature, as described above.

Main study measures included: 1) use of antibiotics for encounters at which diagnoses for which antibiotics are rarely appropriate (URI, acute bronchitis or bronchiolitis, acute non-strep pharyngitis, laryngitis or tracheitis, influenza, non-suppurative otitis media, asthma or allergic rhinitis) were made; 2) use of broad spectrum antibiotics for encounters at which any respiratory diagnosis except pneumonia was made;19 and 3) use of delayed prescriptions for ARI encounters. All measures were calculated quarterly for both adults (patients aged 18 or older) and children. Secondary study measures were as follows: 1) use of antibiotics/broad spectrum antibiotics for adult encounters at which diagnoses of bronchitis were made, and 2) use of antibiotics/broad spectrum antibiotics for adult encounters at which diagnoses of sinusitis were made (without concomitant diagnoses for which antibiotics may be considered appropriate). For all of these measures, use of antibiotics was calculated as the percentage of encounters at which any antibiotic was prescribed out of all encounters. Use of broad spectrum antibiotics was calculated as the percentage of encounters at which a broad spectrum antibiotic was prescribed out of all encounters at which any antibiotic was prescribed. Use of delayed prescriptions was calculated as the percentage of encounters at which a delayed prescription was prescribed out of all encounters at which any antibiotic was prescribed.

Statistical Analyses

Study outcomes were reported quarterly for each practice. Relative weights were assigned to practice-level observations based on the practices’ numbers of ARI encounters during the quarter. In this manner, more weight was given to practices with greater numbers of relevant encounters, and to time points (within practices) that involved greater numbers of relevant encounters. Weighted means and 95 % confidence intervals were determined for practices’ outcome measures during the first quarter of the intervention, separately for adults and children. General linear mixed models (GLMMs) for longitudinal analyses20 were then used to compare changes over time across the 27-month study time period. All models included time (as a continuous variable) as the primary independent variable of interest, and random practice effects were included in the GLMMs to account for the clustering of repeated measures on practices over time. To assess the sustainability of the intervention between phase 1 and phase 2 of the study, the GLMMs also included a linear spline effect for time, with a knot at the transition point between phase 1 and phase 2, allowing the slope of the time trend to change during the last year of the study. All statistical analyses were performed using SAS 9.3 (SAS Institute Inc., Cary, NC), and p-values < 0.05 were considered statistically significant. No adjustment was made for multiple comparisons, as all comparisons were part of a priori pre-specified hypotheses.

RESULTS

Practice characteristics and a tally of each practice’s use of the CDSS are described in Table 1. Practices ranged in size from two to nine providers, and practice use of the CDSS ranged between 286 and 10,850 times. During the last year of the study, one practice closed and withdrew from the study. This practice’s data was included in the analysis through July 1st, 2011.

Table 1 Practice Characteristics and Use of the CDSS Across the 27-Month Study Period
Full size table

The CDSS was used at 38,592 encounters during the 27-month intervention. Figure 1 shows CDSS use by month through the study duration. As expected, CDSS use peaked during the winter months. The CDSS continued to be substantially used throughout the study, with 5,477 uses in the first quarter of 2010, 7,111 uses in the first quarter of 2011, and 4,710 uses in the first quarter of 2012. At least one ARI diagnosis was made at 35,872 (93 %) of encounters at which the CDSS was used; in 84 % of these ARI encounters, only one respiratory diagnosis was made, in 13 %, two diagnoses, and in 3 %, three or more diagnoses. In the 7 % of encounters during which the CDSS was used but no respiratory diagnosis was made, common diagnoses included nonspecific symptoms, such as “cough” or “fever unspecified,” or billing notations, such as “office visit level 3.”

Figure 1.
figure 1

Clinical decision support system (CDSS) use by month (January 1, 2010–March 31, 2012).

Full size image

Table 2 presents the frequency of respiratory diagnoses made using the CDSS for both adults and children at the practice and patient level. The most frequent respiratory diagnoses made using the CDSS in adults were URI, acute sinusitis and acute bronchitis, and, for children, URI, suppurative otitis media and streptococcal pharyngitis. Overall, at least one diagnosis for which an antibiotic was appropriate was made in 9,715 encounters in adults and 5,478 encounters in children; there were 13,232 encounters at which no such diagnosis was made in adults and 7,447 at which no such diagnosis was made in children.

Table 2 Respiratory Diagnoses Made When CDSS Was Used During 27-Month Study Period
Full size table

The primary study outcomes are presented in Table 3. Antibiotics were prescribed inappropriately for about two-fifths of encounters in adults and one-fifth in children, with wide variation among the study practices. There were no significant changes in this outcome during either phase of the study period. When antibiotics were used for encounters without a diagnosis of pneumonia, broad spectrum antibiotics were prescribed about half the time in adults and more than one quarter of the time in children, again with wide variation among the study practices. For both adults and children, there were large statistically significant declines in broad spectrum antibiotic use during both phases of the study, with larger rates of decline during the first period. Delayed prescriptions for antibiotics were used infrequently for both adults and children.

Table 3 Primary Outcome Measures at Patient Level, Practice Level, and Changes Modeled Over Time
Full size table

Table 4 presents the secondary study outcomes. Antibiotics were prescribed for acute sinusitis in adults most of the time, with wide variation among the study practices, and no evidence of change throughout either study phase. However, broad spectrum antibiotic use for acute sinusitis declined substantially during both study phases. Antibiotics were prescribed for acute bronchitis in adults about half of the time, with wide variation among the study practices and no evidence of decline throughout either study phase. There was a non-statistically significant trend toward decreased broad spectrum antibiotic use for acute bronchitis during the study period.

Table 4 Secondary Outcome Measures at Patient Level, Practice Level, and Changes Modeled Over Time
Full size table

DISCUSSION

In this study, we were able to successfully develop and implement a CDSS for ARIs that was widely and consistently used across the nine participating practices. The broad use of our CDSS provides confidence that its impact on prescribing can be reliably assessed. Although the intervention did not significantly impact the overall use of antibiotics for ARIs, use of broad spectrum antibiotics for children and adults declined an estimated 16 %, a relative reduction of about 30 % for adults and 45 % for children. Because the use of broad spectrum antibiotics has been associated with increasing antibiotic resistance,1,21 this substantial reduction of broad spectrum antibiotic use has promising public health implications. We have also shown that our intervention was sustainable over the 27-month study period.

Notable among our findings is the great variability in antibiotic prescribing for ARIs by practice, highlighting the multiple factors that likely affect prescribing of antibiotics for ARIs. Despite using the CDSS with embedded guidelines, providers at some practices reported patient expectations for antibiotics, concern about missing a more serious diagnosis, and some disagreement about antibiotic prescribing guidelines as reasons for prescribing antibiotics when not indicated or selecting broad spectrum antibiotics for conditions for which narrow antibiotics should be first line treatment. In contrast, when reflecting on improvements in the use of narrow spectrum antibiotics, providers cited the positive impact of CDSS reminders and increased awareness from quarterly performance reports.

To our knowledge, there has been one other study assessing an EHR-based CDSS to improve antibiotic prescribing for ARIs, which failed to significantly improve prescribing.9 However, in this study, the CDSS was used at only 6 % of encounters, limiting the ability to assess its efficacy.

There are several important limitations to this study. First, there was no concurrent control group and no way of ensuring that the observed changes were not due to secular trends. Second, it was conducted within a small group of volunteer practices, limiting the generalizability of the findings; however, initial prescribing patterns were similar to that of national surveys.1,18 Finally, although the CDSS was the primary intervention, it was not possible to implement it in practices without providing the context for its use. Academic detailing, sharing of “best practices,” and regular review of performance reports in study practices may also have influenced antibiotic prescribing, limiting the ability to isolate the independent effect of the CDSS.

Despite the study limitations, the intervention appears to have had a sustained impact on reducing the use of broad spectrum antibiotic prescribing for ARIs. If our findings are confirmed in controlled trials, CDSS tools should be more widely used to promote judicious antibiotic use for ARIs.