Keywords

The methods by which we measure quality surgical care have evolved exponentially over the past 30 years with the growth of technology, demand for accountability, and pursuit to provide the highest-quality care for our patients. However, the framework by which we assess healthcare quality has long been rooted in the Donabedian principles of structure, process, and outcomes [1]. Structure represents the physical, technological, and human resources of a healthcare system. These include measures beyond the physical facilities but also include availability of an electronic medical record and data on the provider to patient ratio. While often the easiest to assess, structural measures are the most indirect indicator of quality. Process measures are related to the way systems and providers deliver healthcare, such as compliance with evidence-based guidelines and efficiency of delivering care. One of the most familiar process measures is the Surgical Care Improvement Project (SCIP), which, although initially promising, has had some mixed results with regard to how much compliance equates to reductions in postoperative morbidity [2, 3]. Outcomes are the metric by which we most closely scrutinize the quality of the care we provide and is arguably the most valuable of measures. It is also the most complex, making “apples to apples” comparisons incredibly challenging across healthcare systems. Multifactorial influences are present at every level, and the success of even the most common procedures is equally affected by the skill of the surgeon, the patient’s health status, and the ability of a system to deliver that care and to protect the patient from inadvertent harm.

The emergence of national surgical databases has provided us with platforms to more readily track our outcomes and make meaningful comparisons through the use of statistical modeling that allow us to evaluate data in a risk-adjusted fashion. This enables hospital systems to identify areas of deficiency, enact a plan of action, and assess the effect of that plan on defined quality metrics. This chapter describes the currently existing data registries in surgery and how they have impacted surgical practice.

Society of Thoracic Surgeons (STS)

For better or worse, the incentive to track quality has often been driven by payers, most notably the United States Federal Government. Cardiac surgery was at the heart of the development of national surgical databases. In 1986 the Health Care Financing Administration (HCFA), the predecessor to the Centers for Medicare and Medicaid Services (CMS), released mortality reports on hospitals performing as outliers in cardiac surgery [4]. These reports fueled concern from both the public sector and surgical societies. The validity of these reports was highly criticized for lacking appropriate risk adjustment, in particular when it came to evaluation of outcomes after coronary artery bypass surgery.

The STS recognized the need to better assess quality of care as a surgical specialty and took charge of developing a national database in 1989 [5]. This was among the first surgical databases to include granular clinical data and provide timely risk-adjusted feedback to participating institutions. At the same time, the Department of Veterans Affairs (VA) had begun a nationwide quality improvement project in cardiac surgery, the Continuous Improvement in Cardiac Surgery Program (CICSP). These programs would ultimately lay the foundation for tracking surgical outcomes and promoting data-driven quality initiatives across the country.

National Surgical Quality Improvement Program (NSQIP)

NSQIP was created in response to a federal mandate in 1985 (Public Law 99-166) aimed at improving outcomes for VA hospitals. At the time, the VA was under public scrutiny for high rates of postoperative morbidity and mortality. To address this issue, they first needed to create a system that would allow them to track risk-adjusted outcomes. The National VA Surgical Risk Study was conducted, which collected prospective data at 44 major VA surgical centers. This data established predictive models for risk-adjusted outcomes comparisons that would facilitate assessment of VA hospital performance and the development of the VA NSQIP [6, 7]. Through participation in this program, VA hospitals noted reduction in 30-day mortality after major surgery by 45% and reduction in 30-day mortality by 31% [8]. The federal mandate also required the VA to compare their surgical outcomes to the national average, prompting a pilot study in three academic centers in the private sector which confirmed the predictive models of the VA NSQIP could be applied to other systems. The American College of Surgeons (ACS) then partnered with the VA to conduct the Patient Safety in Surgery (PSS) Study which included 18 non-VA sites and provided further evidence of the validity of NSQIP for hospitals across the nation. Over the study period, participating private sector hospitals noted significant reductions in 30-day postoperative morbidity by 8.7%, surgical site infections (SSIs) by 9.1%, and renal complications by 23.7% [9]. The culmination of these findings along with positive feedback from the participating sites led to the official establishment of the ACS NSQIP for public enrollment in 2004.

ACS NSQIP became the first nationally validated, risk-adjusted, outcomes-based program for measuring outcomes in a variety of surgical subspecialties, with the ultimate goal of improving the quality of surgical care. Since its creation, ACS NSQIP has become an instrumental tool in quality improvements, outcomes research, and the development of an affective risk calculator. Today over 700 hospitals participate in ACS NSQIP. Preoperative data and 30-day outcomes are recorded for a variety of general and subspecialty surgeries by trained surgical clinical reviewers (SCRs). Hospitals are given semiannual reports on their own risk-adjusted outcomes and offered a blinded comparison to other participating hospitals. This has led to establishment of national benchmarks and various efforts by the ACS to support quality improvement efforts across institutions. Long-term participation in NSQIP has been associated with a reduction in 30-day morbidity and mortality. Studies by both Hall and Cohen found reductions in mortality in 66–69% and reduced morbidity in 79–82% of participating hospitals [10, 11]. However, others have been critical that mere participation in NSQIP is not enough to improve outcomes. Two studies comparing outcomes of NSQIP hospitals to nonparticipating centers found no statistically significant difference in postoperative morbidity over time, suggesting that improved outcomes may be more reflective of regression to the mean over time for certain outliers [12, 13]. In particular, Osbourne et al. [12] found no differences in Medicare payments before and after participation in ACS NSQIP when using nonparticipating hospitals as a control.

It is clear that hospital systems must be committed to improving care and implementing quality improvement projects to make a meaningful impact. There is a wealth of data demonstrating that NSQIP data can serve as a catalyst for change and facilitate monitoring the influence quality improvement initiatives have on targeted outcomes. Examples include a single center initiative at decreasing ventilator time, leading to an eventual zero pneumonia rate, and numerous programs aimed at reducing surgical site infections, in particular after colorectal surgery [14,15,16,17]. Beyond local feedback, ACS NSQIP empowers change through the development of collaboratives and best practice guidelines. Currently there are over 65 collaboratives that vary in size and function. These range from health system-wide, regional, to virtual collaboratives. Among the most notable, the Michigan Surgical Quality Collaborative (MSQC) demonstrated improved morbidity in participating hospitals when compared to a non-Michigan ACS NSQIP cohort. This was particularly true when it came to reductions in sepsis, pneumonia, septic shock, cardiac arrest, and need for prolonged mechanical ventilation [18]. The Tennessee Surgical Quality Collaborative also showed significant improvements in surgical site infections (SSIs), decreasing prolonged ventilation, AKI, and wound disruption [19]. Additionally, they estimated a cost savings of over $2,000,000 per 10,000 general and vascular surgery cases.

It is important to note that there is a significant investment incurred by the participating centers. The annual fee ranges from $10,000 to $29,000 a year, but the majority of the cost is in the salary for the SCR, which can range anywhere from $40,000 to $100,000 a year. While NSQIP does not capture cost data, several studies have deduced a cost savings from participating in ACS NSQIP by reducing the incidence of complications [17, 20,21,22,23].

Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (MBSAQIP)

Tracking quality outcomes has become the cornerstone of accreditation for centers of excellence (COE) in bariatric care. In 2012 the ACS Bariatric Surgery Center Network and the American Society for Metabolic and Bariatric Surgery (ASMBS) Bariatric Centers of Excellence combined their respective programs to form MBSAQIP. This now serves as the accreditation body and has created a single bariatric database which over 800 participating institutions currently contribute to. The MBSAQIP database captures high-quality data for the majority of bariatric surgeries that take place in the United States and Canada. An important distinction between NSQIP and MBSAQIP is that while NSQIP randomly samples cases, and thus can miss outliers, MBSAQIP is required to include 100% of bariatric cases at participating centers, thus ensuring a more robust data set.

In addition to bariatric-specific perioperative variables and 30-day outcomes, long-term follow-up data are recorded at 6 months, 1 year, and annually thereafter. The MBSAQIP has recently developed a patient-reported outcome measures (PROMs) program which will send surveys to patients preoperatively, 1-year post-op, and then annually. Semiannual site-specific reports are provided to participating institutions allowing them to benchmark their outcomes to the national average. Accredited sites are required to develop at least one quality improvement initiative per year, and centers who are high outliers for any given measure must address and implement an initiative geared toward reducing that outcome.

MBSAQIP not only stimulates quality improvement initiatives on an individual hospital level, but it has demonstrated that it can facilitate them on a much larger scale. The first national quality improvement collaborative out of the MBSAQIP was aimed at decreasing rates of readmissions. The “Decreasing Readmissions through Opportunities Provided” (DROP) program implemented a bundle at 128 hospitals and demonstrated a 10% reduction in 30-day readmissions overall, with even larger reductions at 32% in centers with the highest rates [24]. Subsequently, the Employing Enhanced Recovery Goals in Bariatric Surgery (ENERGY) study showed successful implementation of an enhanced recovery program across 36 sites with high rates of extended length of stay after bariatric surgery. Adherence to the protocol at targeted centers led to significant decreases in extended length of stay without compromising other outcomes [25].

A recent article by Clapp et al. highlighted the impressive volume of research that has emerged since the release of the first Participant Use Data File (PUF) in 2015, citing 55 published manuscripts and 126 abstracts [26]. It is clear that MBSAQIP has had a resounding impact on surgical research and has proven to be a valuable resource to evaluate outcomes in an evolving field. Over the last 10 years, we have seen a rapid growth in the number of laparoscopic sleeve gastrectomies, becoming the most common bariatric procedure performed. The detailed data captured regarding sleeve gastrectomy has allowed for large-scale analysis of specific technical elements on outcomes. While MBSAQIP data can be utilized to change practice, it is not immune to reporting conflicting results. For example, while one study comparing staple line reinforcement (SLR) to non-reinforced staple lines [27] noted no difference in leak rates after LSG, another study [28] noted a paradoxical increase in leak rates after LSG.

This database like any is not without its limitations. Changes in practice over time have likely led to an element of treatment bias that cannot be accounted for when making comparisons. An evaluation of the 2015 PUF found various data quality issues with data completeness, accuracy, and consistency, which could potentially lead to losing as much as 20% of the entered cases [29]. The majority of these were related to how weight and BMI were recorded, which is clearly an important metric in bariatric surgery.

The aggregate data collected through this robust database has allowed for the creation of a bariatric surgical risk/benefit calculator which provides individualized estimates of postoperative weight loss, resolution of comorbidities, and risk of developing postoperative complications from either sleeve gastrectomy or Roux-en-Y gastric bypass.

Abdominal Core Health Quality Collaborative (ACHQC)

The ACHQC, previously known as the Americas Hernia Society Quality Collaborative, was established in 2013 with the aim of improving the quality of care delivered to patients with ventral hernias. The database formed by this collective is unique in that it provides continuous real-time, risk-adjusted data to participating institutions [30]. It was designed to prospectively collect demographics, granular perioperative details, as well as long-term follow-up data using validated patient-reported outcome measures. It was also intended to facilitate multi-institutional investigations of mesh types and other medical devices in the treatment of hernia disease. A comparison of biosynthetic to polypropylene mesh in clean-contaminated and contaminated wounds using this database elicited some interesting and unexpected results. While there was no significant difference in overall surgical site occurrences between the two types of mesh, biosynthetic mesh was associated with higher rates of major wound complications and unplanned reoperations [31]. By integrating the use of the registry into their routine clinical practice, the Cleveland Clinic Center for Abdominal Core Health has found that the process of conducting a randomized clinical trial was efficient and ensured high-quality data, as the surgeon who was most familiar with the patient’s course was the one recording the data. Through these studies [32, 33], the ACHQC has shown one example of how disease-specific databases can be implemented to further surgical science.

Summary

This chapter highlights the history and contributions of some of the most notable surgical databases that are widely used by general surgeons but by no means encompasses the entire spectrum of high-quality surgical registries that exist. There are numerous programs in almost every surgical subspecialty that contribute to advancing global research and quality initiatives. The ACS Trauma Quality Improvement Program (ACS TQIP), Vascular Quality Initiative, Organ Procurement and Transplantation Network, and the National Cancer Data Base (NCDB) are some of the extensive list of quality improvement programs that are currently utilized in surgical practice.

The American Board of Surgery (ABS) recognizes the importance of tracking quality not only on a national level but also for the individual surgeon. As part of the continuous certification process for diplomates, participation in practice improvement is required, either through contributions to one of the national quality improvement registries or by creating an independent practice improvement plan. The goal of the practice improvement requirement is “for diplomates to regularly assess their performance, by reviewing their outcomes, addressing identified areas for improvement, and evaluating the results” [34]. The ABS provides access to the Surgeon Specific Registry (SSR), an online quality improvement tool, where surgeons can track their own individual cases and outcomes independently. This not only facilitates individual practice improvement but has the added potential of meeting certain CMS requirements.

Reliance on databases to measure the quality of surgical care has its own inherent limitations. Despite the robustness of the major surgical databases described, there are no doubt unaccounted risk factors that cannot be adjusted for. True severity of comorbidities, socioeconomic factors, and treatment biases are almost impossible to capture accurately. Outcomes measured are often limited to a 30-day postoperative time frame, when many outcomes of interest may not be evident for a much longer period. This is true for both the NSQIP and MBSAQIP PUF files that are utilized in most published studies. While the SCRs undergo rigorous training and attempts are made to maintain standardized definitions, the way we define certain events, such as ventilator-associated pneumonia, has evolved, making comparisons over time challenging. As practices continue to evolve, there will also be a need to add new variables. We are already seeing this in bariatric surgery where there is not a variable in the MBSAQIP to accurately capture revisions from sleeve to gastric bypass.

Databases have historically lacked quality of life assessments and cost data which are critical for clinicians and interesting to researchers and payers. It is important that surgeons remain engaged with how those measures are recorded and evaluated. The CMS and many other payers have shifted toward pay-for-performance and use performance data to adjust future payments. Both MBSAQIP and ACHQC are approved as qualified clinical data registries in the CMS Merit-based Incentive Payment System (MIPS), which offers surgeons in independent practice opportunities to achieve a higher level of reimbursement that is afforded to larger medical centers that participate in larger programs such as NSQIP. It is important to note that while the benchmarks established by data registries are intended to motivate improvements, oftentimes energy is more focused on “beating” the quality metric than on actually improving the quality of care. We must also be critical of the statistically significant differences we may find when using large data sets that are in reality of minimal clinically significance.

Despite the costs and limitations, we must always remember the overarching goal and purpose of participating in national surgical databases: to continuously strive to provide the highest quality of care to the patients entrusted to us.

Editor’s Note

SAGES is and has been interested in data registries for some time. As you have read in this review of registries, many surgical societies have invested time and resources into registry creation. The AHSQC was unique in that it was entirely funded by industry. Mesh manufacturers were keenly interested in product performance and likely were willing to invest in comparative data that showed that their own product performed superiorly to others. While some other product-focused procedures might also benefit from such robust data collection, few other disease-specific registries have yet been developed. One could imagine that an investment into data that showed inferior performance of a product would serve as a disincentive for our industry partners to heavily invest in this concept.

As detailed elsewhere in this textbook, SAGES has been the leader in the prevention of bile duct injuries (BDI), and as such, the concept of a cholecystectomy registry was explored in depth, led by the SAGES Quality, Outcomes, and Safety Committee. This review spanned 2 years and even led to a formal meeting with SAGES leadership and a data company in 2016. The conclusion from this exploration was that there were two major barriers to creating a SAGES data registry: cost and the human cost of data entry. A registry tracking cholecystectomy outcomes (with the aim of prevention of BDI) would require granular data collection, and as such, the surgeons themselves would most likely have to be the inputters of data. Given how ubiquitous cholecystectomy is in the general surgical world, this would impose significant work burden onto surgeons with little tangible benefit (e.g., this would not have been required by CMS for reimbursement). As such, there was concern about how well utilized such a registry would have been, and without a high percentage of usage, there was a likelihood that cases with BDI might not have been entered (with acknowledged concern for inducing medicolegal risk) – thus nullifying the value of such a registry. Again, the AHSQC had initial success because a highly motivated group of academic surgeons with a career focus on hernia surgery committed to the laborious task of data entry, but there was skepticism that surgeons would be similarly motivated with cholecystectomy. Further, the cost of creating the registry would have been between $1.5 and 2 million, which is exorbitant and beyond the realistic ability for a society to fund; data maintenance and storage over time would also have been well over $1 million. As such, the registry plans were abandoned.

Further, SAGES explored the concept – and remains interested in – tracking outcomes of anti-reflux surgery. While this would have been less populated than cholecystectomy in terms of case volumes, we postulated that this would be of high interest to the foregut surgeons that comprise a significant percentage of SAGES’ membership. While we pivoted away from a pure data registry due to the aforementioned reasons, a project to conduct video-based assessments of fundoplications is underway and may ultimately serve as a data repository for SAGES members to access. It is the hope of the group conducting this work that we can all learn from one another in terms of technical pearls and that over time this will serve to improve outcomes of the procedure. Much work remains until this is commonplace among SAGES members, but with the ever-increasing computing power and advanced video capture systems and cloud technology, this may become a “twenty-first century” data registry that can be accessed for continual quality improvement.