Introduction

In recent years, surgeons and hospitals have developed an increased interest in assessing quality of care to improve surgical practices while reducing costs of treatment. New surgical standards and policies [11, 17, 18] that require surgeons to monitor their performance and address quality indicators have encouraged the use of systematic quality measurement systems, including the National Surgical Quality Improvement Program (NSQIP). NSQIP remains the only national, risk-adjusted, quality improvement system that is both developed and validated by surgeons [9]. In contrast to other large databases, NSQIP contains preoperative, intraoperative, and 30-day patient data that are prospectively collected by clinical reviewers through a standardized methodology. Since its inception, NSQIP has been utilized by hospitals to identify perioperative risk factors and develop surgical practices that decrease the frequency of complications and adverse events [8, 9, 19]. After the Veterans Affairs (VA) Healthcare System first established the NSQIP, quality improvement initiatives based on NSQIP data led to a 31% decrease in 30-day mortality across all VA hospitals [9].

Upon expansion from the VA system, the NSQIP database had demonstrated initial success in private sector hospitals in the program operated by the American College of Surgeons (ACS) [7]. Since then, several studies have evaluated the effectiveness of the ACS NSQIP database in the development of quality improvement practices [3–6, 8, 13, 19]. For example, the University of Virginia identified risk factors for surgical site infections using NSQIP data and designed a targeted prevention protocol that led to a 36% decrease in these infections [19].

Although the ACS NSQIP database originally focused on general and vascular surgery, it has since developed into a multispecialty model [3]. With this recent expansion, orthopaedic surgeons can now use the expansive clinical data provided through the ACS NSQIP to conduct large-scale, multiinstitutional studies. Despite the potential applications of the current ACS NSQIP database, research has shown that surgeons remain skeptical on publically reporting ACS NSQIP data and applying it beyond the institutional level [13]. In fact, only a few orthopaedic surgical studies have already employed the NSQIP database [2, 12, 15, 16, 21, 22]. These studies limited their analysis to either single orthopaedic procedures or reported rates of adverse events without considering the effect of patient characteristics and comorbidities. Since the number of orthopaedic patients and procedures available for analysis remains unknown, NSQIP has been largely underutilized by orthopaedic researchers. A comprehensive analysis of the orthopaedic component of the ACS NSQIP can help us identify the most common adverse events and procedures with the highest rates of complications. This would allow us to focus future orthopaedic outcomes research in areas that require the most improvement in surgical quality. By using the ACS NSQIP to investigate the highest-risk orthopaedic procedures and exploring risk factors for adverse events, quality improvement initiatives targeting these procedures can be developed.

We therefore performed a comprehensive evaluation of the use of the ACS NSQIP database within the field of orthopaedic surgery. Our specific purposes included (1) investigating the most common orthopaedic procedures and 30-day adverse events, (2) analyzing the proportion of adverse events in the top 30 most frequently identified orthopaedic procedures, and (3) identifying patient characteristics and clinical risk factors for adverse events in patients undergoing hip fracture repair.

Patients and Methods

Data Extraction

This investigation was initiated after institutional review board approval. Access to the NSQIP dataset collected between 2005 and 2011 was granted by the ACS. The dataset included 462 hospitals across the United States and 34 hospitals in other countries, including Saudi Arabia, Canada, Lebanon, United Kingdom, and the United Arab Emirates. The 135 patient variables reported within the database included preoperative risk factors, intraoperative variables, and 30-day postoperative mortality and morbidity outcomes for patients undergoing major surgical procedures in both the inpatient and outpatient setting. At each participating institution, two risk assessment nurses trained as surgical clinical reviewers were appointed to collect data directly from the patient’s medical record. Interrater reliability disagreement of less than 5% per site was considered acceptable. Audit reports have identified disagreement rates of less than 1.8% [1].

Surgical procedures performed by an orthopaedic surgeon were found by querying the Surgical Specialty variable within the ACS NSQIP database using Microsoft® Excel® 2010 (Microsoft Corp, Redmond, WA, USA). The patients found were transferred to Stata® 12 (StataCorp LP, College Station, TX, USA) for statistical analysis. Of the 1,979,084 surgical patients identified in the ACS NSQIP database, 146,774 underwent orthopaedic procedures (7%).

Independent and Dependent Variables

Independent variables included patient characteristics, such as age, sex, race, BMI, alcohol abuse, active smoking, and history of any of the following: greater than 10% weight loss in the 6 months before surgery, dyspnea, chronic obstructive pulmonary disease, congestive heart failure (CHF), hypertension, peripheral vascular disease, esophageal varices, disseminated cancer, steroid use, bleeding disorders, dialysis, chemotherapy, and radiotherapy. Other variables included surgery in the 30 days before the injury, American Society of Anesthesiologists (ASA) physical status class, preoperative functional status, resident involvement, and operative time. Laboratory values collected included preoperative sodium, blood urea nitrogen, hematocrit, and white blood cell and platelet count.

Outcomes were grouped by severity into minor and major complications and were selected on the basis of previously published studies using the ACS NSQIP database [2, 12, 15, 16, 21, 22]. The major complications included deep wound infection, wound dehiscence, cardiopulmonary complication, thromboembolic disease, pulmonary embolism, deep venous thrombosis, cerebrovascular accident, postoperative neurologic deficit or sepsis, return to the operating room, and death. Minor complications included superficial wound infection, pneumonia, and urinary tract infection (UTI). A third outcome variable combined all complications into a single group.

Data Analysis

Only the 30 most common orthopaedic surgical procedures in the ACS NSQIP dataset were included in our final analysis. We did not include more than 100 surgical procedures that contributed less than a maximum of 500 patients per procedure.

The 30 most common orthopaedic procedures were ordered by frequency. All adverse events were identified and ranked by frequency and separated into minor and major complications. We calculated the relative contribution of each complication to the total adverse event pool and identified the total number of complications per patient.

We identified which demographic and clinical independent variables were risk factors for any complication (major or minor), minor complications, and major complications. The surgical procedures grouped into this analysis and their corresponding Current Procedural Terminology (CPT®) codes are the following: percutaneous skeletal fixation of femoral fracture, proximal end, neck (27235); open treatment of femoral fracture, proximal end, neck, internal fixation or prosthetic replacement (27236); open treatment of intertrochanteric, peritrochanteric, or subtrochanteric femoral fracture (27244); and treatment of intertrochanteric, peritrochanteric, or subtrochanteric femoral fracture, with intramedullary implant (27245).

We performed a bivariate unadjusted analysis by analyzing categorical variables using the chi-square test and Fisher’s exact test. For continuous variables, we used the Wilcoxon rank-sum test. Results were tabulated and statistical significance was set at p values of less than 0.05.

All clinically relevant variables and statistically significant variables, as identified by our bivariate analyses, were included in the multivariate models of patients undergoing hip fracture repair. Three multivariate models were performed, one for each of the outcome variables: major complications, minor complications, and any form of complication in patients undergoing hip fracture repair. Individual patients with missing data were excluded from this analysis.

Results

Most Common Procedures and Adverse Events

The top 30 most frequently identified orthopaedic surgical procedures included a total of 101,862 patients (Table 1). The three most common procedures were TKA, THA, and knee arthroscopy with meniscectomy and together they comprised 55% (55,575 of 101,862) of these patients. TKA alone accounted for 29% (29,139 of 101,862) of the top 30 procedures.

Table 1 Top 30 orthopaedic surgery procedures in order of frequency
Full size table

We identified 6742 30-day complications in 5368 patients (1.3 complications per patient with a complication), resulting in a 5% (5368 of 101,862) complication rate (Table 2). There were 3907 major complications in 3283 patients (1.19 major complication per patient with a complication), resulting in a 3.2% major complication rate. There was a 2.8% rate of minor complications (2835 of 101,862), with approximately 1.35 complications per patient with a complication. The most common 30-day minor and major complications were UTI (n = 1534) and death (n = 850), respectively.

Table 2 Total number and overall rate of adverse events
Full size table

Procedures With the Highest Rates of Adverse Events

Open reduction and internal fixation (ORIF) of a femoral neck fracture, partial hip hemiarthroplasty, and intramedullary nailing of an intertrochanteric femur fracture had the highest rates of adverse events. Overall, the top 10 procedures generated 48% (3224 of 6742) of all adverse events (Table 3). The procedure identified as having the most adverse events was open treatment of femoral neck fracture with internal fixation or prosthetic replacement (CPT® code 27236), which also had the highest complication rate of 27% (767 of 2798). Partial hemiarthroplasty of the hip (CPT® code 27125) had the second highest complication rate of 26% (477 of 1822). The difference between these two procedures is that the former is generally performed in an elective manner compared to the latter, which is used only for hip fractures treated with either hemiarthroplasty (unipolar or bipolar) or ORIF [10]. ORIF of an intertrochanteric hip fracture with an intramedullary implant had the second highest number of adverse events and third highest complication rate, 26% (695 of 2667).

Table 3 Top 10 procedures with highest number of adverse events and adverse event rates
Full size table

Risk Factors Among Hip Fracture Repairs

Patients with esophageal varices, CHF, functional dependence, a higher age or a higher ASA class were more likely to develop any type of complications after surgery within 30 days after discharge (Table 4). Esophageal varices proved to be the greatest risk factor for developing a 30-day complication, with those who had a history of esophageal varices almost four times as likely to develop any complication than those who did not (odds ratio [OR]: 3.73; 95% CI: 1.20–11.6). Functional dependency doubled the risk of any complication after surgery (OR: 2.33; 95% CI: 2.12–2.56). Patients with a history of CHF (OR: 1.85; 95% CI: 1.31–2.62) were almost two times more likely to develop any complication. Patients older than 65 years were more likely to develop any complication than those younger than 65 years (OR: 1.37; 95% CI: 1.07–1.75). Patients with an ASA class of 3 or 4 were more likely to develop any complication than those with an ASA class of 1 or 2 (OR: 1.26; 95% CI: 1.72–2.78).

Table 4 Multivariate analysis of patients with and without any complications (major and minor)
Full size table

Patients undergoing radiotherapy or with a higher age were more likely to develop a minor complication after surgery within 30 days after discharge (Table 5). Patients receiving radiotherapy were three times more likely to develop a minor complication (OR: 3.20; 95% CI: 1.35–7.58). Patients older than 65 years were more likely to develop a minor complication than those younger than 65 years (OR: 1.50; 95% CI: 1.07– 2.09).

Table 5 Multivariate analysis of patients with and without minor complications
Full size table

Patients with a higher ASA score, CHF, disseminated cancer or requiring dialysis were more likely to develop a major complication, including death, after surgery (Table 6). Patients with an ASA class of 3 or 4 were three times more likely to develop a major complication than those with an ASA class of 1 or 2 (OR: 3.05; 95% CI: 2.19–4.26). A history of CHF doubled the risk of developing a major complication (OR: 2.17; 95% CI: 1.50–3.14). Patients on dialysis were almost two times more likely to develop a major complication (OR: 1.95; 95% CI: 1.26–3.01). Finally, patients diagnosed with disseminated cancer were more likely to develop a major complication (OR: 1.63; 95% CI: 1.03–2.59).

Table 6 Multivariate analysis of patients with and without major complications or mortality
Full size table

Discussion

The goal of the ACS NSQIP is to improve patient safety. The database has been used by hospitals across the United States to decrease the rate of adverse events and improve surgical outcomes, including dramatic decreases in 30-day mortality, morbidity, and complication rates. However, only a few orthopaedic surgical studies have employed the ACS NSQIP database, all of which have limited their analysis to either single orthopaedic procedures or reported rates of adverse events, without considering the effect of patient characteristics and comorbidities. Our purpose was to demonstrate the potential of the ACS NSQIP to improve patient outcomes in orthopaedics. We did this by identifying which orthopaedic procedures were most represented within the database, providing an in-depth analysis of adverse events in orthopaedics, and providing an example of how these data can be applied in future research.

The NSQIP database has several strengths and limitations. Administrative databases are often developed for billing purposes and therefore may only represent procedures that are also of financial interest. In contrast, NSQIP is a clinical database collected specifically for the improvement of surgical quality. The data are entered by surgical clinical reviewers, who are nurses and other medical professionals. Misclassification of data is reduced by the use of the ACS NSQIP Data Definition Committee, who defines all of the variables used in the database. All surgical clinical reviewers undergo rigorous preliminary and continuous training. Because of this, interrater reliability disagreement within the ACS NSQIP is less than 1.8% [1]. The main limitation of the ACS NSQIP database is that it only collects data up to the 30th postoperative day. Deep wound infection, dehiscence, cardiopulmonary complications, and death are major complications that may occur beyond the 30-day postoperative period and therefore were underestimated in our results. Secondly, the variables captured by this database are not specific to the orthopaedic patient. Implant failure, nonunion, range of motion, time to preinjury weightbearing, and malunion are relevant long-term (> 12 months) outcomes not collected by the database. Furthermore, although NSQIP is a reliable source of clinical data for quality improvement at an institutional level, most surgeons do not believe that the risk adjustment mechanisms are reliable enough to be used for surgeon-specific outcomes or for reporting data publicly to payers or patients [13]. However, new risk-adjusted modeling is being developed by the ACS advisory board to improve the reliability of their methodology.

The most common orthopaedic surgery procedure identified from the NSQIP dataset from 2005 to 2011 was TKA, representing 29% of the top 30 orthopaedic procedures. Pugely et al. [15] evaluated differences in anesthetic methods of patients undergoing TKA and initially identified a total of 15,849 procedures in a 5-year period (2005–2010). The updated dataset almost doubles the previously reported TKAs after only an additional year of data capturing, indicating the exponential increase in the number of hospitals adopting the NSQIP dataset into their quality improvement efforts. With an increasing number of hospitals implementing NSQIP, this dataset will contain perioperative outcomes for more than 50,000 TKAs, 40,000 THAs, and 20,000 hip fracture repair procedures within the next year, which can be utilized by orthopaedic researchers to observe complication rates on a large scale. In addition, the exponential growth will provide an increase in the total number of rare procedures that may not be obtained even by large multicenter trials. Also, several literature reports have shown that quality improvement can be brought about by identifying the most common adverse events. For example, when Hall et al. [8] analyzed reduction in risk-adjusted adverse events in 118 hospitals participating in NSQIP, they found that 66% of hospitals improved risk-adjusted mortality and 82% improved risk-adjusted complication rates. These hospitals used ACS NSQIP data on adverse events to create best practices guidelines, develop case studies, and monitor progress.

Our second purpose was to identify the orthopaedic procedures with the highest complication rates. Five hip fracture procedures, THA, and TKA were the top seven procedures with the greatest complication rates. Analyzing 7970 orthopaedic procedures from the 2005 to 2007 NSQIP database, Schilling et al. [20] identified that the top 10 procedures were responsible for 70% of adverse events and 65% of excess hospital days. With only 3 additional years of data, we found that the complication rate after hip fracture repair increased exponentially while the complication rate for all other procedures remained the same. In addition, myocardial infarction, sepsis, UTI, and pneumonia were the most common complications after hip fracture repair. As has been demonstrated, identifying common adverse events in high-risk surgeries can dramatically improve patient outcomes [8, 16]. For example, Massachusetts General Hospital identified a 7.0% rate of UTIs in the vascular surgery department using NSQIP. After making changes to the Foley catheter removal algorithm and implementing educational campaigns for clinicians, they reduced the UTI rate to 1.8% in 1 year. This example highlights how NSQIP provides a highly reliable data system that allows hospitals to develop targeted approaches for quality interventions, as well as close monitoring and evaluation of results.

Our third purpose was to provide an example of using the NSQIP data to evaluate risk factors in patients undergoing hip fracture repair. An American Society of Anesthesiologists class of 3 or higher was a consistent risk factor for all three categories of complications in patients undergoing hip fracture repair. Similar risk factors were reported by Radcliff et al. [16], who provided risk-adjusted data in older male veterans undergoing hip fracture repair using the VA NSQIP database. Identifying risk factors for adverse events is essential for the development of quality improvement protocols. For example, the University of Virginia identified that a BMI of greater than 25 kg/m2 was a risk factor for surgical site infections in patients undergoing colorectal surgery. Further review resulted in the implementation of a clinical protocol including wound wicking, which led to a 36% decrease in surgical site infections in 3 years. Recently, Pugely et al. [14] published the development of a risk calculator for short-term morbidity and mortality after hip fracture surgery using the NSQIP, which can be used to identify high-risk patients and compare risk-adjusted outcomes between institutions.

Due to the large number of patients in this study, our methodology, and the paucity of data in the existing literature, we consider these results as a sound foundation for information regarding adverse events in orthopaedic surgery and the best available evidence on this topic. It is important for the orthopaedic community to be aware of the remarkable information contained within this database. Although orthopaedic procedures only account for 7% of the entire ACS NSQIP database, there are more than 1000 CPT® codes involving more than 145,000 patients from which valuable information is gathered. It is up to us as orthopaedic surgeons to closely evaluate these available data and prioritize quality improvement efforts in orthopaedic procedures for specific patient populations. This approach has shown clear benefits in general surgery subspecialties and it is only a matter of time before we see quality improvement in orthopaedic NSQIP participating institutions.