Introduction

To achieve the goals of quality-driven health care, clinicians must demonstrate benchmark performance on national quality measures. In 2011, through collaboration with the Crohn’s and Colitis Foundation of America (CCFA), the American Gastroenterological Association (AGA) published a set of 10 process-based quality measures for the care of adults with inflammatory bowel disease (IBD) [1]. Eight of these measures were adopted by the Centers for Medicare & Medicaid Services (CMS) for the Physician Quality Reporting System (PQRS) program in 2013 and 2014 [2]. The PQRS program originally provided incentive payments for eligible health care professionals who met criteria for reporting quality measures. As of 2015, the program imposes annual increasing reimbursement penalties for failing to report quality measure data according to CMS requirements [3]. In health care systems such as accountable care organizations, quality assessment is partly determined by compliance with patient-centered measures that align with the six priorities of the National Quality Strategy (NQS) [4]. These priorities entail improving patient safety, engaging patients in their care, enhancing care coordination and communication, using the most effective prevention and treatment methods, promoting community health initiatives, and making health care more affordable.

The first studies on US gastroenterologists’ compliance with PQRS quality measures for IBD have recently been published [5–8]. Feuerstein and coinvestigators reviewed the charts of IBD patients who received care from gastroenterologists at a tertiary medical center. Performance on all 8 PQRS quality measures for IBD was documented for only 6.5 % of patients [5]. We previously conducted 2 quality improvement (QI) programs in which charts of IBD patients who received care from community-based gastroenterologists were reviewed before and after the physicians participated in accredited continuing medical education (CME) activities. The programs focused on the care of patients with Crohn’s disease [6] or ulcerative colitis [7, 8]. Baseline chart reviews revealed low and variable rates of compliance with PQRS measures for IBD and NQS-related measures [6–8].

Leaders in the US gastroenterology community have called for incorporating national IBD quality measures in QI programs that engage clinicians in rigorous documentation, along with performance analysis and feedback [9–12]. The educational interventions in our previously reported IBD QI studies included individualized feedback on chart reviews. The education was associated with significant improvements in documented performance of some IBD and NQS-related quality measures, especially among gastroenterologists whose baseline compliance rates were low [7, 8]. However, due to the pragmatic nature of these programs and studies, the findings are somewhat limited by methodological issues such as a lack of nonintervention control groups.

This article reports a QI program and study in which compliance with IBD and NQS-related quality measures was assessed in a group of community-based gastroenterologists before and after they participated in a series of QI-focused CME activities. Performance on the measures was evaluated through baseline and follow-up (post-education) chart reviews of patients with ulcerative colitis. To evaluate the influence of the education, we also reviewed baseline and follow-up charts of patients who received care from a nonintervention control group of community-based gastroenterologists.

Methods

The QI study received independent institutional review board approval (Sterling IRB, Atlanta, GA; IRB ID #4613).

Physician Recruitment and Baseline Review

Randomly identified community-based gastroenterology practices across the US were contacted to provide information about the QI project and to inquire about interests in participating. Recruitment phone calls were made until 40 gastroenterologists in different practices were enrolled. In order of their agreement to participate, 20 gastroenterologists were assigned to the educational intervention group, and then 20 were assigned to the nonintervention control group. All of the physicians signed consent forms for their participation in the educational program and study.

The study was designed to retrospectively review 300 charts at baseline, for the 1-year period of January 1, 2013 to December 31, 2013. We planned to review 10 charts and 5 charts, respectively, for each gastroenterologist in the intervention and control groups. Administrative staff in each practice randomly selected charts of patients who met the following inclusion criteria: age ≥18 years; diagnosis of moderate-to-severe ulcerative colitis confirmed by ICD-9 codes and the Montreal classification system [13]; and at least 1 visit with the physician during the 1-year baseline review period. In each practice, a list of eligible patients was alphabetized and numbered. Random sampling tables were used to select an oversample of up to 25 eligible charts. An administrative fee of $500 was offered to each practice to reimburse costs for identifying patient charts and providing chart access to the reviewers.

Charts were abstracted for patient demographics and the gastroenterologists’ documented performance of (1) the 8 IBD quality measures included in the 2013 and 2014 PQRS programs and (2) patient-centered measures aligned with NQS priorities, including assessment of side effects and provision of patient counseling about IBD-related topics (Table 1). For each PQRS measure, denominators were adjusted and exclusions were applied. We chose the patient-centered measures based on their alignment with NQS priorities for ensuring that patients are engaged in their health care, improving communication, promoting effective prevention and treatment practices, or making care safer. Documentation of performance on each measure was recorded for analysis in Statistical Package for the Social Sciences (SPSS), version 22.

Table 1 Measures assessed through patient chart audits
Full size table

Educational Interventions

The educational interventions comprised a series of accredited CME activities. After their baseline charts were reviewed, gastroenterologists in the intervention group participated in the first activity, a private audit feedback session. Administered through web conference software, the individualized sessions were led by a clinician trained in interpreting and presenting quality measures abstracted from patient charts. Each session was organized by the presentation of slides with graphs showing the participating gastroenterologist’s rates of compliance with the PQRS quality measures for IBD and the NQS-related measures. Presented as mean percentages of charts documented for each measure, the graphs also included de-identified, aggregated compliance rates for the other gastroenterologists in the intervention group. The audit feedback sessions were designed to guide participants in identifying measures for which baseline performance was suboptimal. For these measures, the presenter asked the participating gastroenterologist to reflect and comment on barriers to performance and documentation. In addition, the presenter and participant discussed an individualized action plan for improving performance and documentation. During the audit feedback sessions, a facilitator took notes to record key discussion points, including participants’ barriers to compliance with IBD quality measures and strategies for addressing them.

Within 4 weeks after his/her audit feedback session, each gastroenterologist in the intervention group participated in an accredited small-group webinar. A total of 5 webinars were offered on different dates to accommodate scheduling and promote interaction. The webinars were led by expert gastroenterologists who were selected based on their involvement in national QI programs and their established records of IBD research. One of the co-authors of this article (PDRH) served as a presenter for 3 of the webinars. The expert gastroenterologists led the study participants in discussions about the evidence-based rationale for applying IBD quality measures in practice and effective methods for performing and documenting the measures.

To reinforce the education provided in the audit feedback sessions and small-group webinars, we developed an online/mobile toolkit that included accredited CME activities on improving the quality of IBD care and nonaccredited resources for supporting the gastroenterologists in performing, documenting, and reporting PQRS quality measures for IBD. The accredited activities included four 30- to 60-min interactive videos that addressed applications of quality measures to various aspects of ulcerative colitis and Crohn’s disease care, including diagnosis, treatment decision-making, and ensuring patient safety. The toolkit also included a 20-page monograph that presented the evidence-based rationale for applying quality measures in IBD practice.

Follow-up Chart Review and Analysis

Six months after the intervention group completed the CME activities, follow-up chart reviews were conducted. According to the same methods used for the baseline period, we planned to review 10 charts and 5 charts, respectively, for each gastroenterologist in the intervention and control groups. Charts were randomly identified for patients with ulcerative colitis who met the previously listed inclusion criteria and had at least 1 visit with the gastroenterologist in the 6-month period after the educational activities. The period for follow-up chart review was 6 months, from December 1, 2014 to May 31, 2015.

Statistical Analysis

Chi-square tests or t tests were conducted to compare demographic characteristics of physicians in the intervention and control groups and to determine whether patient demographics differed in the chart samples selected for baseline and follow-up reviews, as well as between the intervention and control groups. To assess the influence of the CME activities, we calculated the percentage of patient charts with documented performance of each PQRS and NQS-related measure at baseline and follow-up for each gastroenterologist. Overall mean percentages were then calculated for the 2 periods in the intervention and control groups. Independent t tests were performed to compare the differences in mean provider-level compliance rates from baseline to follow-up between the 2 groups. For all analyses, p values less than 0.05 were considered significant.

Results

The 40 gastroenterologists practiced in Alabama, Florida, New York, Texas, New Jersey, Arizona, Illinois, Indiana, Massachusetts, Missouri, Tennessee, Virginia, or Washington. The control group had significantly more clinical experience than the intervention group based on mean years in gastroenterology practice (25 vs. 18 years, p = 0.04; Table 2). There were nonsignificant differences in geographic distribution and percentages of males and females between the intervention and control groups. However, post hoc multiple regression analysis indicated that none of these demographic variables was significantly associated with changes in rates of compliance with the IBD quality measures or NQS-related measures. Before the follow-up chart reviews, one of the gastroenterologists in the intervention group dropped out of the study.

Table 2 Demographic characteristics of intervention and control groups
Full size table

The baseline analysis included 199 charts for the intervention group (mean = 10.5 charts per physician, range = 5–25 charts) and 100 charts for the control group (mean = 5.0 charts per physician, range = 4–10 charts). The follow-up analysis included 200 charts for the intervention group (mean = 10.5 charts per physician, range = 5–25 charts) and 100 charts for the control group (mean = 5 charts per physician, range = 2–10 charts). Several of the gastroenterology practices provided fewer eligible charts than were targeted. Because these practices were enrolled in the QI program, their charts were included in the analysis. All but 1 of the 40 gastroenterologists used electronic health records.

Within-group analyses indicated no significant differences between the baseline and follow-up chart samples for patients’ ages, body mass index (BMI), proportion of females and males, and disease duration. In addition, there were no significant differences between patient demographics in the intervention and control group charts, respectively, for age (45.5 and 48.5 years), proportion of females (48.5 and 49 %), BMI (27.5 and 27.4), and disease duration (10.0 and 7.5 years).

Baseline and follow-up rates of documented compliance with the PQRS quality measures for IBD are presented in Table 3. Across the 2 periods, the difference in the percentage of documented charts was significantly greater in the intervention versus control group for 5 of the 8 PQRS measures. The differences in changes for the 5 measures, reflecting greater magnitudes of improvement in the intervention group, were as follows: assessment of IBD type, anatomic location, and activity (+14 %, p = 0.009); influenza vaccination (+13 %, p = 0.025); pneumococcal vaccination (+20 %, p = 0.003); testing for latent tuberculosis before initiating anti-TNF-α therapy (+10 %, p = 0.028); and assessment of hepatitis B virus status before initiating anti-TNF-α therapy (+9 %, p = 0.010). The differences in baseline to follow-up compliance rates did not differ significantly between groups for smoking cessation intervention in current smokers and bone loss assessment and corticosteroid-sparing therapy in patients using corticosteroids over prolonged periods.

Table 3 Baseline and follow-up rates of compliance with PQRS quality measures for IBD
Full size table

Table 4 presents the percentages of charts that were documented for measures aligned with NQS priorities. Compared with the control group, the intervention group had significantly greater magnitudes of improvement for 2 of the 4 measures: assessment of medication side effects (+17 %, p = 0.048) and counseling patients about cancer risks (+13 %, p = 0.013). The differences in baseline to follow-up compliance rates did not differ significantly between groups for counseling patients about colorectal surgery or medication risks/benefits and adherence.

Table 4 Baseline and follow-up rates of compliance with NQS-related measures
Full size table

Table 5, which reflects notes taken during the audit feedback sessions, summarizes participants’ most commonly discussed barriers to compliance with IBD quality measures and the education strategies that we employed to address the barriers.

Table 5 Quality improvement (QI) education strategies for addressing barriers to compliance with IBD quality measures
Full size table

Discussion

Recent studies have indicated low and variable rates of compliance with IBD quality measures among gastroenterologists practicing in tertiary care and community-based settings [5–8]. In the present study, the mean baseline percentages of patient charts with documented performance on the 2013–2014 PQRS measures ranged from 5 % for bone loss assessment in patients using corticosteroids to 81 % for assessment of disease type, location, and activity. For the NQS-related measures, mean baseline compliance rates ranged from 18 to 85 %. In a previous QI education study, we reviewed charts of adults with Crohn’s disease who received care from 20 community-based gastroenterologists who did not participate in the present study [6]. Baseline rates of compliance with the PQRS measures for IBD ranged from 3 % for pneumococcal immunization to 98 % for corticosteroid-sparing therapy. Feuerstein and coinvestigators reviewed the electronic health records of 367 patients with Crohn’s disease or ulcerative colitis who received care from general gastroenterologists or IBD specialists at a large tertiary medical center [5]. Rates of compliance with the PQRS measures ranged from 21 % for pneumococcal immunization to 96 % for smoking cessation intervention. In our previous studies, rates of documented testing for latent tuberculosis and hepatitis B virus before initiating anti-TNF-α therapy did not exceed 29 % [6, 8]. The mean baseline rates reported here (72 and 52 %, respectively) and by Feuerstein and coinvestigators (67 and 74 %, respectively) [5] are considerably higher, reflecting progress in the gastroenterology community for these 2 measures. However, the initial reports indicate that performance of IBD quality measures is suboptimal and variable across measures and physicians.

The QI-focused educational activities in this study were associated with significantly greater improvements in the intervention versus control group for 5 of the 8 PQRS quality measures for IBD. In the intervention group, compliance rates either decreased or did not increase significantly for smoking cessation intervention, bone loss assessment, and corticosteroid-sparing therapy. These 3 measures depend on eligibility criteria of current smoking status or prolonged use of corticosteroids. As noted in Table 2, a small proportion of patient charts met these criteria; thus, the data for these measures are limited.

This study adds to a series of QI programs and outcome studies in which we reviewed charts of patients with IBD before and after their gastroenterologists participated in accredited CME activities [6, 8]. This study is unique in its inclusion of a nonintervention control group and its focus on assessing and improving the quality of care for patients with ulcerative colitis. In our previous study involving 20 different community-based gastroenterologists and reviews of 400 charts of Crohn’s disease patients, education was associated with improved documentation of PQRS IBD measures only in “low-performing” physicians, designated by compliance rates in the lowest quartile [6]. The educational interventions included individualized audit feedback sessions; however, there were no follow-up small-group webinars.

Given its context in a pragmatic QI program, this study was not designed to determine the extent to which each educational activity influenced the outcomes. The audit feedback sessions and small-group webinars were designed to flexibly address the performance gaps and educational needs of individual participants. More time was devoted to discussing barriers and solutions to performing quality measures for which the individual’s baseline compliance rates were low compared with absolute standards and mean rates for the other participants. Systematic reviews indicate that audit and feedback education generally elicits small-to-moderate, but meaningful improvements in clinical performance [14, 15]. Many of the gastroenterologists in the intervention group reported a lack of standardized processes in their clinics for accomplishing the PQRS quality measures, which led to discussion about ways to establish reliable processes to identify gaps, act on these gaps, and document completion of the quality measures for each patient, often by leveraging new capabilities of their electronic health record systems.

The study outcomes were process-based quality measures rather than patient outcomes. Thus, we assessed the performance of the same physicians, based on randomly selected patient charts, across the baseline and follow-up periods. This design was intended to control for participant-related extraneous variables. In addition, the inclusion of chart reviews for a nonintervention group was intended to control for the Hawthorne effect and secular factors that may have influenced compliance with the quality measures. The gastroenterologists were not blinded to the study goals or to their group assignments; thus, bias related to this factor may have influenced the study outcomes. The study may also have been limited by its short follow-up period, which was 6 months. Although all patients had at least 1 visit with their gastroenterologist in the follow-up period, the relatively short duration may have precluded performing some of the quality measures. It is also possible that, without continual reinforcement, performance on quality measures may revert to baseline levels.

The IBD quality measures on which the study was based were included in the 2013–2014 PQRS program. In 2015, changes to the measures set were enacted through collaboration between the AGA, the National Committee for Quality Assurance, and the American Medical Association Physician Consortium for Performance Improvement. A major revision was retirement of the measure for assessment of IBD type, anatomic location, and activity [16]. Other recent developments include a new AGA system for reporting IBD quality measures and alternative methods for meeting CMS reporting requirements [17]. Future revisions may include the addition of outcomes-based IBD quality measures [12]. Given the early stage and evolving nature of value-based IBD care, to address issues of sustainability and clinical relevance, future QI programs and studies should be designed to account for ongoing and anticipated revisions to quality measures as well as reporting systems and requirements. Moreover, bias in the appropriateness of the quality measure may influence compliance. For example, many gastroenterologists may consider surgical counseling as not applicable or inappropriate for the majority of their ulcerative colitis patients.

This study demonstrates that QI-focused CME can improve community-based gastroenterologists’ compliance with quality measures for IBD and measures aligned with NQS priorities. Our findings suggest that the QI education interventions positively influenced performance on some quality measures to a greater degree than others. Future studies are thus needed to identify factors that explain the potential for QI interventions to promote positive practice changes, especially for IBD quality measures deemed to be most clinically important. New studies are also needed to identify the most effective types of educational interventions for QI programs and to develop strategies for scaling interventions so that greater numbers of gastroenterology clinicians can become involved in national efforts to improve the quality of care for patients with IBD.