Surgical site infections (SSIs) rank among the most common hospital-acquired infections in surgical patients [1, 2] and represent a common source of post-operative morbidity and mortality [3]. Their development leads to increased hospital length of stays, hospital costs, readmissions, and subsequent procedures, as well as decreased quality of life [4,5,6,7,8,9,10,11,12,13]. Aside from negatively affecting patients, SSIs after colectomy are associated with significant economic costs. Research has demonstrated that each SSI after colectomy increases the cost of care by $18,000, burdening the health care system with an estimated annual cost of $315 million [13,14,15,16]. With the establishment of a financial penalty for SSI after colectomy in 2017 as a component of the Centers for Medicare and Medicaid Services Hospital-Acquired Condition Reduction Program, perioperative care bundles have been promoted as a method of decreasing SSIs after colectomies [14, 16,17,18,19]. Several studies utilizing surgical bundles to reduce colorectal SSI rates have demonstrated mixed results [20,21,22,23,24,25,26], although meta-analyses [27, 28] have shown that use of evidence-based surgical care bundles can effectively reduce the risk of SSIs by fostering a cohesive environment, standardization, and reduction in operative variance. Zywot et al. [27] evaluated 23 studies involving 17,557 patients and revealed an overall SSI risk reduction of 40%. Bundles including sterile closure trays, mechanical bowel prep with oral antibiotics, pre-closure glove changes, and enhanced recovery after surgery (ERAS) protocols are correlated with greater SSI risk reduction [27, 29,30,31,32]. Despite data supporting the use of such approaches, physician compliance with bundles and post-colectomy SSI remain quality problems for many health systems.

In this report, we describe a multi-year, multi-institutional quality improvement program designed to evaluate the degree of checklist compliance for both elective and emergency colectomies needed to achieve a reduction in NHSN-identified post-colectomy SSIs. The decision to classify procedures as elective or emergency was left to the operating surgeons. Similarly, issues of co-morbidity and risk adjustment were not part of our primary objective and were left to the discretion of the operating surgeons. The relationship between infection and specific checklist and provider compliance was the discriminating factor of interest across the patient population so as to evaluate the impact of process intervention alone on outcomes.

Materials and methods

Organizational structure

System leads (AA, MES, DA, KC, SD) abstracted data from NHSN downloads for 2016 at the outset of the study and annually thereafter. Demographic reports including age, sex, American Society of Anesthesiologists (ASA) classification, presence or absence of diabetes mellitus (DM), body mass index (BMI), procedure context (elective versus emergency), closure technique, pathogens, and type of SSI were prepared and distributed to all sites. Pre-study colectomy practice survey results and annual bundle compliance (system-, hospital-, and surgeon-specific) data were also distributed. A multi-disciplinary team implemented a prospective system-wide surgical bundle checklist monitoring system for colorectal surgeries involving collection of checklist compliance data into a relational database; monthly communication of outcome and checklist compliance data to practitioners; and analysis of which checklist items deficiencies are associated with SSI reduction. A detailed table of organization was established for the entire health system, identifying hospital physician leads, quality supervisor, perioperative liaison, infection control liaison, and quality liaison for each site. System meetings were held once per quarter and each site was responsible for identifying and evaluating the management of cases with SSIs according to a root cause analysis format.

National Health Safety Network (NHSN) data abstraction

The Centers for Disease Control National Healthcare Safety Network [33] is the nation’s most widely used healthcare-associated infection (HAI) tracking system. NHSN now serves approximately 25,000 medical facilities tracking HAIs. Current participants include acute care hospitals, long-term acute care hospitals, psychiatric hospitals, rehabilitation hospitals, outpatient dialysis centers, ambulatory surgery centers, and nursing homes, with hospitals and dialysis facilities representing the majority of facilities reporting data. CMS and other payers use these data to determine incentives for performance. The NHSN is a secure, Internet-based surveillance system that expands and integrates patient and health care personnel safety surveillance systems. Data are submitted and validated by hospital Infection Control personnel according to NHSN criteria. NHSN data were downloaded annually from 2016 to 2019 and compiled into a series of analytic reports to define our health system’s colectomy infection rates.

In this study, our complication reporting database [34,35,36] was used to integrate heterogeneous internal and external data sources (clinical electronic medical record, checklist, administrative, financial, and NHSN data) into a single view for analysis and reporting on surgical activity. The cohort of NHSN surgical cases was matched to surgeons without identifying National Provider Identifier (NPI) numbers using a text matching algorithm to the date of surgery, hospital facility, procedure description, and surgeon name or identifier code.

Numerator and denominator data for both elective and emergency procedures were paired to specific physician NPI numbers and hospital sites. All available data from NHSN files were imported and linked to the appropriate hospital site and surgeon. Reports provided information regarding surgeon-specific numbers of infections (elective versus emergency), number of denominator (elective versus emergency) cases, pathogens, and operative time. Data were distributed to task force leads for site and surgeon review. These reports allowed for quarter to quarter and year to year comparison, identification of outliers at the site and surgeon level, and investigation of process issues. SSI’s were defined by NHSN criteria [37]. These data were used to establish baseline rates for surgeon and hospital practice patterns within our health system. It also allowed us to identify areas of responsibility for processes of care, i.e., Surgeon, Anesthesia, and Nursing.

All data were PHI protected and HIPAA compliant and built on an SQL platform with data repositories maintained within VPN firewalls. These data served as the basis for infection rates for the health system, hospital sites, specific physicians, as well as checklist compliance reports. A “Human Subjects Research Determination Request” was filed with our IRB committee and IRB approval was not required.

Bundle checklist monitoring

Checklists were compiled from the literature and in consultation with senior colorectal surgeons for both Elective (24 items) and Emergency (21 items) colectomies (Table 3). Emergency colectomy checklist items excluded pre-operative chlorhexidine (CHG) shower, oral antibiotic, and mechanical bowel preps. Data were collected manually on 1694 random colectomies (1274 elective; 420 emergency) and entered into our Morbidity and Mortality Adverse Event Reporting System (MARS; Outcome Management System, Greenwich, Ct) database. Reports were generated that profiled system, hospital, and surgeon-specific checklist compliance (CC) and non-compliance rates by checklist item. A compliance rate of < 90% for any particular checklist item was arbitrarily highlighted as non-compliant so as to establish a minimal target compliance rate.

NHSN data from 2016 were used to establish the baseline incidence of SSI. CC was prospectively compared in two timeframes: 1/1/17–5/30/18 (793 checklists; “Roll Out”) and 6/1/18–5/30/19 (901 checklists; “Study”). The “Roll Out” period was used to introduce the project, develop the table of organization, educate staff, and implement checklists, databases, and workflow to collect reliable data. The “Roll Out” period was also used to establish baseline compliance levels for checklist items following elective colectomy (EC) and emergency colectomy (EmC). “Study” data were used to calculate changes in SSI and checklist compliance.

Statistics

For each checklist item relevant to both elective and emergency operations, a logit analysis was done to model whether the item was or was not in compliance as a function of: (1) data accrual period (“Roll Out” versus “Study”); (2) acuity (elective versus emergency); and (3) the interaction between period and acuity. A logit model was fit using the R package glm for analyzing generalized linear models; this is a special case of logistic regression in which all the predictors are categorical versus at least one being continuous. R is the public version of the original program S produced by Bell Labs [38]. Chi-square was calculated using the R procedure cross-tab in the described package for each checklist item versus period of data accrual (“Roll Out” versus “Study”) and between year versus infection. Rates are expressed as the number of per 1000 cases. The significance levels are from cross-tabulations of each category of infection (yes/no) by year or study period.

Results

Bundle checklist compliance

Table 1 presents the proportion of checklists obtained versus the number of operative procedures for each data accrual period (“Roll Out” versus “Study”) and acuity (elective versus emergency). A total of 1694 checklists were collected following 7616 NHSN-identified colectomy cases between 2017 and 2019, representing a checklist sample size of 22%. A total of 570 “Roll Out” elective colectomy checklists (EC) and 223 emergency colectomy checklists (EmC) were compared to 704 “Study” EC and 197 EmC. The percentage of compliant checklist items (Table 2) improved from “Roll Out” to “Study” periods for both EC (66.6% to 87.5%) and EmC (47.6% to 90.5%).

Table 1 Checklist sample size for “Roll Out” vs. “Study” periods
Full size table
Table 2 Checklist compliance: “Roll Out” vs. “Study”
Full size table

Table 3 lists the changes in checklist compliance for all items between “Roll Out” and “Study” periods. In elective cases, there were significant period effects indicating a ceiling compliance level (95–100%) with greater compliance during the “Study” period than “Roll Out” period for all items except wound protector use (decreased to 84.2%) and pre-operative chlorhexidine prep (decreased to 73.6%). Oral antibiotic prep (increased to 83.7%), IV antibiotics re-dosing (increased to 92%), and mechanical bowel prep (increased to 90.9%) improved but did not achieve > 95% compliance. Similarly, for emergency cases, there were significant period effects indicating greater compliance during the “Study” period than “Roll Out” period for all items. However, although increased, a ceiling compliance level (95–100%) was not achieved for chlorhexidine prep in OR (increased to 92.4%), Glucose monitoring in PACU (increased to 91.3%), IV antibiotic re-dosing (increased to 73.3%), wound protector use (increased to 72.1%), and tray/instrument/gown change after anastomosis (increased to 93%).

Table 3 Checklist compliance improvement per study period—elective/emergency
Full size table

Demographics and infection rates

Between 2016 and 2019, a total of 10,300 (8327 elective; 1973 emergency) NHSN-identified colectomies were performed within our health system. Data from 2016 established baseline SSI rates and data from 2017 through 2019 comprised “Roll Out” and “Study” datasets. Case mix indices (CMI) for elective colectomies during the “Roll Out” and “Study” period were 2.82 and 2.87, respectively. Elective laparoscopic/robotic versus open procedure CMIs were “Roll Out” 2.39 and 3.05 and “Study” 2.44 and 3.13, respectively, and were not statistically different. Emergency colectomy CMIs were 4.46 “Roll Out” and 4.94 “Study.” Emergency laparoscopic/robotic versus open procedure CMIs were “Roll Out” 4.45 and 4.46 and “Study” 3.14 and 5.24, respectively, and were not statistically different. The study groups were comparable for acuity, severity, and minimally invasive approaches.

A statistically significant reduction (32.7%; chi-square = 9.6906, p < 0.002) in annual elective colectomy infection rates was observed beginning in 2017 and persisting through 2019 (Table 4). All SSI categories were improved with a 45% overall reduction in intra-abdominal abscess. When data were separated into “Baseline (2016),” “Roll Out” and “Study” time frames, elective, emergency, and all case SSI’s were significantly improved (Table 5). Elective colectomy infection rates comparing 2016 to “Study” for deep incisional primary (DIP), superficial incisional primary (SIP), and intra-abdominal abscess (IAB) were reduced by 61.5%, 64.2%, and 81.3%, respectively. Emergency colectomy infection rates were similarly improved.

Table 4 Reduction elective infections
Full size table
Table 5 % Rate reduction NHSN colectomy infections following checklist usage
Full size table

For elective colectomy, the infection rates were significantly reduced between “Baseline (2016)” and “Roll Out” (chi-square = 30.5, df = 1; p < 0.0001) and from “Roll Out” to “Study” (chi-square = 7.46, df = 1; p < 0.006). For emergency colectomy, infection rates were significantly reduced from “Baseline (2016)” to “Roll Out” (chi-square = 13.49, df = 1, p = 0.0002), but the drop from “Roll Out” to “Study” period did not achieve significance (chi-square = 3.41, df = 1, p = 0.06).

Elective colectomy infection rates, checklist compliance, and specialization

Since checklist compliance appeared to improve overall post-colectomy infections rates and since our data suggest that maximal compliance (95–100%) was achieved in all but a handful of checklist items, we wondered whether items that failed to achieve a ceiling compliance level may differentiate surgeons with high and low infection rates. An arbitrary elective colectomy infection rate of <  > 3% was chosen to differentiate low and high infection rate surgeons as a function of below ceiling checklist compliance. Therefore, we studied the compliance of 250 surgeons with specific checklist compliance data in order to determine whether differences in their elective colectomy infection rates could be a function of below ceiling checklist item compliance. The checklist items with below ceiling compliance evaluated were oral antibiotic prep, mechanical bowel prep, and IV antibiotic re-dosing. One hundred ninety-five surgeons with infection rates < 3% performed 3016 elective surgical procedures and accounted for 20% of infections. However, 55 surgeons with infection rates of > 3% performed 1661 procedures and were responsible for 80% of elective infections (Table 6). Overall infection rates between these groups were 0.99% vs. 7.53%, respectively (p < 0.0001). Analysis of compliance between low (< 3%) and high (> 3%) infection rate surgeons demonstrated greater use of oral antibiotic prep (85.0% vs. 64.9%; p < 0.03) and mechanical bowel preps (89.4% vs. 55.3%; p < 0.02), respectively (Table 7). The use of IV antibiotic re-dosing did not achieve statistical significance (p = 0.06).

Table 6 Elective colectomy infection rates: surgeons < 3% and > 3%
Full size table
Table 7 Elective colectomy compliance rates with checklist items as function of infection rates
Full size table

In addition, we noted a difference in the number of cases per surgeon between low infection rate and high infection rate surgeons and wondered if specialty may be playing a role. The distribution of Colorectal (CR) and Non-Colorectal (Non-CR) surgeons and their respective infection rates was evaluated (Table 8). CR surgeons performed increasing numbers of cases per surgeon than non-CR surgeons from 2016 (3:1) through “Roll Out” (5:1) and “Study” (7:1) periods and has fewer infections. A cross-tab analysis of period (2016, “Roll Out,” “Study”) by infection rates for CR surgeons showed dependence of period and infection (chi-square = 21, df = 2, p < 0.001). The same cross-tab analysis for Non-CR surgeons showed independence of period and infection (chi-square = 1.58, df = 2, p = 0.454). These data suggest that CR surgeons accounted for the reduction in infection rates which was apparent immediately with institution of the checklist program at “Roll Out” and maintained through “Study” period. Infection rates for Non-CR surgeons did not improve until the “Study” period and the improvement was less than CR surgeons.

Table 8 Infection rates; colorectal surgeons v. non-colorectal surgeons & volume
Full size table

Discussion

Public reporting of outcome data and, in particular, the pay-for-performance reimbursement programs [31], which limit revenue based on potentially preventable complications, have motivated health systems to utilize care bundles as a strategy for quality improvement [28]. Compared with other elective surgical procedures, colon resection is among the highest in major morbidity and mortality [39]. More than 25% of elective colon resections among Medicare patients have an adverse outcome when followed for 90 days after discharge [36]. Complication data from 2017 to 2019 extracted from the MARS database [40] for three academic centers within our health system identified 1026 complications following 3925 colectomies with a complication rate of `26.1%. Among the most common complications were intra-abdominal abscess, readmission, return to the OR, return to the interventional suite, anastomotic leak, and death. Nearly all complications were directly or indirectly linked to surgical site infection. Any reduction in surgical site infection should have a substantial impact on related co-morbid complications.

The reduction in infection rates following implementation of a colectomy bundle checklist program for both elective and emergent cases was observed immediately during the “Roll Out” period and sustained through the “Study” period. As checklist compliance increased, infection rates decreased. The improvement in compliance to ceiling compliance levels (95–100%) was consistent from “Roll Out” to “Study” periods for both elective and emergency colectomy. Further improvement in SSI reduction may be achieved by optimizing compliance with checklist items not achieving a ceiling level of 95–100% compliance. CR surgeons accounted for the greatest number of colectomies per surgeon, the largest percentage reduction in SSI, and thus, the overall reduction in infection rates. High-volume practices and protocolization of checklist compliance within office workflow were thought to be responsible.

A question raised by these data was whether high infection rates were associated with the failure to achieve a ceiling effect for specific checklist items. Our data suggest that following elective colectomy there was a significant association between high infection rates and the decreased compliance with two of three specific below ceiling compliance checklist items (oral antibiotic prep, mechanical bowel prep, and IV antibiotic re-dosing). Using specific surgeon identifiers, we showed that the compliance of high infection (> 3%) versus low infection rate (< 3%) surgeons demonstrated a Pareto Principle [41] effect: 20% of the surgeons were responsible for 80% of elective infections and those surgeons were significantly less compliant with oral antibiotic and mechanical bowel preps. Furthermore, these data suggest that identification of high infection rate NHSN outlier surgeons as a first step coupled with identification of below ceiling level checklist compliance may improve the efficiency of a SSI reduction quality improvement project by identifying surgeons with the greatest likelihood of non-compliance. The identified surgeons may benefit from interventive quality improvement assistance so as to maximize checklist compliance prospectively. In addition, quality improvement resources could be allocated in a more focused and efficient manner. It appears that quality improvement efforts should target ceiling compliance levels (95–100%) as a goal.

Clearly, there may be other ways to foster improvement. First, an electronic checklist platform, which we are working to implement, would allow data collection contemporaneous with care and immediately flag any non-compliant cases. Based on the data presented here, an electronic monitoring system would be able to intervene in a timely manner to ensure compliance and accelerate the timeline for improvement. Secondly, accurate NHSN identification of surgeons with elective infection rates > 3%, and the allocation of quality improvement resources aimed directly at supporting and optimizing their practice improvement initiatives, would likely accelerate improvement. Thirdly, since infection rates seem to be correlated to lack of compliance, a more transparent approach to disseminating infection rates and compliance within the institution may be appropriate. And lastly, it may not be appropriate to perform elective cases that are non-compliant with bundle checklists and may better be deferred until compliance is achieved. At the current time, throughout much of the country, health care facilities are paid for cases with complications. In many respects, for avoidable complications, i.e., cases with low bundle compliance, an argument can be made to disincentivize institutions from performing cases that are non-compliant. As public reporting of hospital and surgeon outcomes progress and “bundled” payment or “pay-for-performance” strategies become the norm, health care facilities and providers will require actionable data with which to evaluate outcomes and best serve their patients. The suggestions outlined above are recommended to achieve optimal outcomes.

There are several important limitations of this study. The primary one being potential selection and sampling bias based on the original paper-based checklist data system. Despite a random 23% sample rate, our observational dataset may suggest associations rather than causality and in the absence of a checklist for every infection cannot definitively imply causality.

The electronic format currently in development and being piloted at one of our institutions will significantly improve sample size and provide immediate data feedback. An additional limitation is that we chose to limit comparison groups to acuity (elective and emergency) procedures as defined by providers and not to further risk or co-morbidity adjust the patient population. Unfortunately, we are also unable to identify whether the surgical approach (MIS vs. Open) played a role in post-colectomy surgical infections. However, our goal was to evaluate the impact of checklist compliance alone as a discriminating process factor impacting outcomes. As such, we accepted the “real-world” provider-determined decision to proceed with resection based on their assessment of co-morbidity and acuity risk. Similarly, we included all providers, not limited to colorectal specialty practitioners. In fact, we showed that the greatest improvement in outcomes was with colorectal practitioners, presumably because compliance improvement impacted the largest number of patients. Nonetheless, the complexity of the study population as related to disease, procedure, surgeon, and other factors leaves open the distinct possibility that some other issue may have positively or negatively influence our outcomes.

Conclusion

We report on a multi-year, multi-institutional quality improvement project which resulted in a substantial and sustained decline in elective and emergency post-colectomy total SSIs (57–62%) and IAB (60–72%) infection rates by increasing compliance with a colectomy bundle checklist. We achieved > 95% compliance with all but 5 checklist items and demonstrated that improvement in compliance over time was associated with improved infection rates. Our data suggest that 20% of the surgeons were responsible for 80% of elective infections, and high infection rate surgeons were significantly less compliant with oral antibiotic and mechanical bowel prep use. CR surgeons demonstrated the greatest reduction in SSI by virtue of their compliance and high volume of cases per surgeon. Identification of high infection rate NHSN outlier surgeons coupled with identification of below ceiling level checklist compliance may improve the efficiency of a SSI reduction quality improvement projects. It appears that quality improvement efforts should target ceiling compliance levels (95–100%) as a goal. We aim to apply this approach to a contemporaneous electronic checklist collection model in the near future.