Introduction

Variation in physician practice results in preventable complications and inappropriate procedures, tests, and medications, which not only harm patients but come at high cost. According to the Institute of Medicine, $130 billion of US health care spending can be attributed to “inefficiently delivered services,” including preventable complications (1 PMID 24901184). Variation in medical quality of care contributes to between 210,000 and 400,000 deaths each year (2 PMID 23860193). If variation in quality was a disease, it would rank as the thrid leading cause of death in the US(Makary MA, Daniel M. Medical error: the thrid leading cause of death in the U.S. BMJ 2016).

Surgery is one of the most high-priced encounters in the health care system, and previous studies have found that the quality and cost of surgical care vary widely by hospital. Hospital mortality and 30-day readmission rates after common surgical procedures differ greatly by hospital (3 PMID 19797283 4 PMID 24350963). Furthermore, despite evidence that minimally invasive (laparoscopic) colectomy is safer than open colectomy when patients are a candidate for both operations (5 PMID 16034888), laparoscopic utilization varies markedly. For colectomy, hospital utilization of laparoscopy ranges from 7 to 50 % in the USA (6 PMID 25005264). In addition, the cost of colectomy differs widely by hospital, with complications being the leading driver of cost (7 PMID 25807207).

Quality metrics in health care have been developed historically to evaluate hospital performance, not physician performance. In fact, little is known about the relationship between the two. We designed a study to evaluate surgeon-level variation in risk-adjusted postoperative complications and the surgeon factors that are associated with this variation.

Methods

Population

We performed a retrospective analysis of variation in surgeon outcomes using patient data from the Health Services Cost Review Commission (HSCRC), Maryland’s all-payer administrative claims database. The database includes patient-level hospital discharge data from all 62 hospitals in the state. We studied hospitalizations for elective partial colectomy between July 1, 2012 and September 30, 2014. Elective procedures were identified from the database as those performed during a planned admission. Patient characteristics, such as age, race, and payer status, are included in the database. Payer status was divided into Medicare, Medicaid, commercial, and other, which included uninsured patients. We also linked each patient’s zip code of residence to the US Census Bureau’s American Community Survey for 2013 for median household income in the zip code. Low-income areas were defined as zip codes with median household income less than $50,000. For colectomy, we included all hospitalizations with an International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) code for partial colectomy (laparoscopic code = 17.3, open code = 45.7) combined with a diagnostic code for colon cancer (153). Surgeons performing fewer than ten colectomy operations over the study period were excluded, as were colectomy procedures combined with another procedure.

Outcomes

Postoperative complications were assessed using the Agency for Healthcare Research and Quality’s (AHRQ) Patient Safety Indicators (PSIs) for surgical care (8 PMID 20734521). These included wound infection, wound dehiscence, deep vein thrombosis, pneumonia, sepsis, postoperative hemorrhage or hematoma, postoperative metabolic derangements, accidental puncture or laceration, and inpatient death during the index hospitalization. For the Patient Safety Indicators, we used the specified numerators and denominators provided by AHRQ, modifying the code to summarize the information at the level of the surgeon rather than the hospital. We also studied inpatient death using the discharge disposition code provided by HSCRC.

Physician and Hospital Characteristics

Issued by the Centers for Medicare and Medicaid Services, the National Provider Identifier (NPI) uniquely identifies individual physicians. Surgeons included in our study were linked using their NPI to publicly available records made available by the Centers for Medicare and Medicaid Services (https://data.medicare.gov/data/physician-compare 9), which include characteristics such as sex, years in practice, and medical school. Top ten medical schools were identified by the US News and World Report ranking in 2015. Hospital size, location, and academic status were obtained from the American Hospital Association.

Statistical Analysis

We used the Elixhauser comorbidity score to adjust for patient risk factors (10 PMID 9431328). These adjustments allowed us to calculate an observed over expected (O/E) ratio of complications to describe the number of events that occurred per surgeon given the expected number of events considering the complexity of their patients, as described previously (11 PMID 23628227). Briefly, a multivariable logistic regression model with complication as the outcome variable and the Elixhauser score and patient demographic characteristics as predictors was created with a post-estimation analysis to generate a complication propensity score for each patient. This hierarchical model accounted for variation at the hospital level using a random intercept for each hospital. These scores were summed up by surgeon to calculate the expected number of complications. Adjusted complication rates were calculated by multiplying surgeon O/E ratios by the population complication rate.

We evaluated high-performing and low-performing surgeons, defined as those having risk-adjusted complication rates above and below the population complication rate, respectively. The characteristics of patients operated upon by higher performing and lower performing surgeons were compared using the chi-squared test. We included all physicians in calculating the expected number of complications but only report on surgeons contributing at least ten procedures. Characteristics of surgeons associated with high or low risk of complications were identified using patient-level multivariable logistic regression with complication as the outcome.

Results

A total of 2525 patients met the inclusion criteria (Table 1). Forty-five percent were male, 64.9 % were white, and more than half were over 65 (58.6 %). A majority of patients scored 2–4 on the Elixhauser comorbidity scale. Patients operated upon by higher performing surgeons (top 50 % by adjusted complication rate) differed significantly from those operated upon by lower performing surgeons (lower 50 % by adjusted complication rate). In particular, a greater proportion of patients operated upon by lower performing surgeons were from low-income areas (21.5 vs 16.5 %, p < 0.001). Forty percent of the operations were performed using a laparoscopic approach. Patients were operated upon by 276 surgeons, of whom 15.2 % were female and 6.2 % graduated from a top ten medical school (Table 2). The most common primary specialty was general surgery (88.8 %), followed by colorectal surgery (6.9 %) and surgical oncology (4.4 %). Approximately half of the surgeons had at least 21 years of experience (54.3 %), and 66.3 % performed fewer than ten colectomy procedures (49.3 %). Forty-one percent of surgeons worked in hospitals larger than 400 beds; 62 % were part of academic hospitals.

Table 1 Patient characteristics (N = 2525)
Full size table
Table 2 Surgeon characteristics (N = 276)
Full size table

The overall population colectomy complication rate was 5.4 %. Inpatient death by surgeon varied from 0 to 16.7 %. Postoperative complications varied more by surgeon (range 0 to 30.0 %) than by hospital (range 0 to 18.2 %). The O/E ranged by surgeon from 0.0 to 9.9, and five surgeons had an O/E greater than 5 (Figs. 1 and 2). Following adjustment, the range of complication rates varied from 0.0 to 61.0 %. The complication with the greatest surgeon-level variation was failure to rescue (range = 0 to 100 %) (Tables 3 and 4). Surgeon-level laparoscopy utilization ranged from 0 to 100 %, with 25th, 50th, and 75th percentiles for laparoscopy use by surgeon being 14.3, 47.2, and 67.1 %, respectively (Fig. 3). The overall average length of stay was 8.5 days. Surgeon-level patient length of stay ranged from 3.1 to 16.6 days with 25th, 50th, and 75th percentiles being 6.6, 8, and 10.2 days, respectively.

Fig. 1
figure 1

Unadjusted risk of complication by hospital (N = 42) and surgeon (N = 93)

Full size image
Fig. 2
figure 2

Case-mix adjusted risk of complication by surgeon (N = 92)

Full size image
Table 3 Incidence of postoperative complications by surgeon
Full size table
Table 4 Predictors of postoperative complications by surgeon characteristicsb
Full size table
Fig. 3
figure 3

Laparoscopic surgery utilization rate by surgeon (N = 93)

Full size image

After risk adjustment of patient factors, including age and Elixhauser comorbidity score, surgeon experience (p = 0.89), surgeon medical school ranking (p = 0.35), surgeon gender (p = 0.25), and surgeon colectomy volume (p = 0.86 for volume 11–20 vs <10; p = 0.85 for volume 21+ vs <10) were not associated with risk of complications. Patients were more likely to experience complications in academic hospitals (OR = 1.58 (1.02–2.43)). Compared to the fourth quartile of surgeons (0–14 %) ranked by laparoscopic surgery utilization, those in the first (67–100 %) and second quartiles (47–67 %) had reduced risk of complications (OR = 0.40 (0.22–0.74) and OR = 0.55 (0.30–1.00)), respectively). Stratifying to laparoscopic procedures only, the laparoscopic surgery utilization quartile was not associated with complications (p > 0.05). Stratifying to open procedures only, greater laparoscopic surgery utilization by the surgeon was associated with reduced risk of complications (vs fourth quartile indicating 0–14 %, second quartile indicating 47–67 % OR = 0.40 (0.18–0.94), first quartile indicating 67–100 % 0.39 (0.11–1.41).

Discussion

In this study of quality indicators among surgeons performing elective colectomy in Maryland, we found wider variation in complication rates by surgeon than by hospital. Some surgeons had an O/E indicating that, even after case-mix adjustment, the risk of complications for their patients was tenfold higher than for patients operated on by the average surgeon in the study. Patients and referring physicians should be aware that surgeon outcomes vary greatly and may depend on specific skillsets for specific groups of patients, especially the use of laparoscopic surgery. Moreover, traditional indicators of surgeon quality, including years of experience and the prestige of training locations, were not found to be significant and may not provide an accurate representation of a surgeon’s skill. The strongest predictor of surgeon quality was use of laparoscopic surgery. Moreover, greater surgeon use of laparoscopic surgery was also associated with reduced risk of complications when stratifying to open procedures only, suggesting that surgical skill with laparoscopy may also be independently associated with surgical skill on open procedures.

Whereas most of the research on surgical outcomes has focused on assessing hospitals, surgeon-level variation has been largely unexplored. Our results are consistent with a previous study of 20 surgeons showing that skill among practicing bariatric surgery can vary greatly, accounting for much of the variation in complication rates between the fourth and first quartiles of surgeons in the study (5.2 and 14.5 %, respectively) (12 PMID 24106936). Our study builds on these findings by exploring complication rates by surgeon at a population level using administrative claims data, which can be readily operationalized across state-level and national data sets. Recently, ProPublica published risk-adjusted surgery-associated readmission rates for common low-risk procedures in the Medicare population (https://projects.propublica.org/surgeons/ 13). Our results provide another benchmark, namely laparoscopic utilization, for estimating outlier and exemplar physician performance in relation to their peers for colectomy and can be used in future efforts to improve data transparency in health care.

We believe that internal data transparency by sharing data with physicians represents a major opportunity for quality improvement. In fact, this approach is the focus of the Improving Wisely campaign in which expert clinicians will identify and endorse a quality metric within their specialty, create boundaries of outlier performance, and then notify physicians of their outlier status in a confidential and peer-to-peer fashion. This quality improvement effort seeks to address outlier variation noted in this study in a way that involves physicians and their clinical wisdom. Developing metrics at the physician level could be used in quality collaboratives to address variation in outcome (14 PMID 22542349). The New England Vascular Association and the American College of Mohs Surgery have adopted programs to share physician-level performance data with their clinician members with the goal of reducing outcome variation and improving patient safety.

Our study has several important limitations. First, administrative claims data and the AHRQ PSIs do not capture all adverse events. It has been observed that PSIs demonstrate high positive-predictive value but vary in sensitivity (15 PMID 24114372). If PSIs are undercoded, they may not account for all the complications or variation in complications. Second, patient demographics and Elixhauser scores may not adjust fully for patient characteristics that affect surgeon use of laparoscopy and outcomes, such as tumor characteristics and stage. Based on these limitations, we caution against using making conclusions about physician performance at a very granular case level but instead believe that these data can be used to identify patterns of extreme outliers. Importantly, we do not believe that outcomes should be reported on surgeons unless they perform at least ten qualifying procedures because of the hazards of a low sample size. Third, it may be that surgeons serving a unique patient population or who specialize in a particular type of surgical care may be unfairly characterized. Thus, we suggest that these data be used as a screening tool a more detailed evaluation of physician outliers. Finally, we were not able to measure patient-reported or long-term outcomes after the hospitalization that the procedure occurred. Future studies should look at patient-reported outcomes, long-term prognosis, as well as patient satisfaction, as makers of the quality of surgical care.

Based on our findings, we suggest that addressing physician variation represents a major opportunity to reduce patient harm on a population level and reduce medical costs associated with complications. As consumer and health care organizations seek to make this data transparent to patients seeking care, the hospital community can build on previous models of quality improvement success using the metrics we present. While we describe a new quality metric for colectomy, other areas of medical care may be amenable to the same type of quality metric development process. Such a broadening of quality metrics from the hospital level to the physician level may enable greater strides in a safer and less costly health care system.