Introduction

The current population is aging rapidly which contributes to the growing number of elderly patients presenting to undergo surgical procedures. It is estimated that approximately 53% of all surgeries are performed on patients greater than the age of 65. 1 The growing number of elderly surgical patients cannot necessarily be treated in the same way as the younger patient population. Patients below the age of 65 tend to be healthier, while older patients tend to have chronic medical problems. Yet chronological age does not necessarily predict outcome; instead, physiological age should be considered. Older patients often have one or more comorbidities, ultimately changing how physicians should view and treat them. 2

Developing a generalizable yet accurate method to risk stratify patients that takes into account their physiological age for treatment has been an ongoing task for many years, beginning with the Charlson Comorbidity Index (CCI). CCI was developed in 1987 as a way to risk stratify patients based on their comorbidities to predict mortality. 3 The work developed the index with patients admitted to a hospital’s medical service, then validated the index in a population of women with breast cancer. 3 The index is weighted, meaning it accounts for not only the number of diseases but their severity as well. For example, having a mild liver disease gives a score of 1, whereas having a moderate or severe liver disease gives a score of 3. The score for each comorbidity is summed and can give a total score up to 37. CCI has been used in numerous situations including inpatient services, 4,5 elective surgery, 6,7 and surgical oncology. 8,9,10 Despite its continued use, some physicians believe that it is outdated, especially for specific populations for which the original index was not intended. 11,12 There are several reasons that the a newer risk stratification tool should be utilized over the CCI. First, patients now survive longer than they did in 1987 when the original weights for the CCI were developed due to advancements in treatment options. 13 Second, we have found that the CCI score for surgical oncology patients is often homogenous and does not allow for sensitive risk stratification with regard to the commonality of co-morbid conditions in those diagnosed with the same type of cancer. Current research highlights that the CCI is not accurate for specific surgical patient populations and thus researchers have developed their own scoring systems to better risk stratify particular subsets of patients. 7,14

Many variations of the CCI have been developed including the Age-Adjusted CCI (ACCI) which takes into account chronological age, the Charlson-Deyo score (CCI-D) which is a 1992 CCI revision that allowed for use with ICD-9 codes, and the Elixhauser Co-morbidity Index (ECI). The ECI is a more recent model compared to the CCU that includes 31 unique co-morbidities. 15 Contemporary indices have been developed focusing on the variable measure of frailty versus categorical co-morbidities alone. Frailty is a measure is a state of limited physiological reserve that can have tremendous impact on surgical outcomes. 16 One such index is the 5-item modified Frailty Index (mFI-5) which has slowly been gaining popularity since its development in 2017. 17 One of the promising features of the mFI-5 over traditional options such as CCI is the inclusion of a measure of the patient’s specific functional dependent status (FDS). 17 Frailty and FDS has been shown in recent studies to be a strong predictor of surgical outcomes including complications, mortality, and adverse peri-operative events. 16,18,19 Indices that capture a patients FDS may prove to be more effective compared to co-morbidity-based indices alone. 20 The aim of this systematic review and meta-analysis was to identify the optimal comorbidity index that can be reliably measured from the electronic medical record for short term outcome prediction following surgical intervention.

Methods

Search Strategy and Exclusion Criteria

A literature search was performed in a comprehensive manner according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses guidelines (PRISMA guidelines). 21 An electronic search of PubMed, Embase, and EBSCO databases was performed using the combinations of “comorbidity index,” “comparison,” “surgery,” “mortality,” “morbidity,” “prediction,” and “outcome” in the keyword and title fields. The search was limited to contemporary studies published between 2014 and 2022 unless hand-selected through citation searching, in order to include recent more clinically relevant studies. Examiners reviewed all abstracts for potential inclusion. Inclusion criteria included studies that focused upon measuring the predictive capability of comorbidity indices prior to surgical intervention. Articles were excluded according to the following criteria: comorbidity indices that are not able to be measured/extracted from medical record data, non-English language, outcomes assessed other than in surgery, and review or meta-analysis articles. The full text of each article was then reviewed for data related to the key clinical outcomes outlined below; studies not including adequate data regarding surgical outcomes after surgery or trauma were excluded from the final cohort of studies for the meta-analysis. The surgical outcome data for the remaining articles was extracted. The systematic review included retrospective studies if they measured a comorbidity index or indices able to be calculated retrospectively from hospital database data and assessed outcomes after surgery or trauma. Additionally, studies were included in the meta-analysis if they assessed at least one of the key outcomes. Non-English studies, reviews, and meta-analyses were excluded. The majority of frailty instruments were excluded from this study. Indices that rely on frailty depend heavily on factors not consistently available in the medical record. 22,23 The five-item modified frailty index (mFI-5) is a less complex option that has shown to be just as effective in outcome prediction when compared to the more comprehensive 11 item mFI-11 and is closer in nature to a co-morbidity index rather than a multi-dimensional frailty index. 24

Quality Assessment of Studies

One researcher independently reviewed each study for strength of data utilizing the following data points: author(s), year, number of patients in cohort, indication for surgery, type of procedure, and surgical outcomes following the procedure (Table 1). The studies were assigned a score using a modified scoring system for minors. 25 Up to two points were assigned for each of the following standards: a clearly stated aim (+ 2), inclusion of consecutive patients with no exclusion or information detailing the reason for exclusion (+ 2), prospective data collection that details a clear collection plan (+ 2), endpoints relevant to this study assessed (+ 2), an unbiased assessment of the study endpoint through proper controls (+ 2), proper follow-up after study period (+ 2 for median and range of follow-up, + 1 for inexplicit information regarding follow-up), loss to follow-up less than 5% (+ 2 if all patients with missing medical record data were excluded), and prospective calculation of study size (+ 2 if they include data regarding the power of the study).

Table 1 Studies assessing age-adjusted Charlson Co-morbidity Index (ACCI)
Full size table

Outcomes of Interest

The primary outcome of this study was mortality within 3 months (in-hospital, 30-day, or 90-day mortality). Secondary outcomes were the rates of any complication (in-hospital, 30-day, or 90-day morbidity) and severe complications typically defined according to Clavien-Dindo (CD) guidelines or predefined by the author. 26 The outcome of overall survival (OS) was not reviewed as this measure has confounding variables that may not accurately capture morbidity and mortality due to the physical insult of surgery. For interpretation, mortality was divided into two groups (in-hospital/30-day and 90-day) as was morbidity (in-hospital/30-day and 90-day). Damhuis et al. has shown that the rate of morbidity and mortality vastly differs between the 30-day and 90-day time intervals and thus should be considered as separate groups. 27 For severe complications, outcomes defined as CD ≥ III or IV were grouped together and CD complications < III considered minor.

Data Extraction

The following data was extracted from all eligible studies: number of patients that underwent procedure, comorbidity indices measured, index cutoff, disease type, disease site, type of surgery, and surgical specialty. Data for each comorbidity indexes’ ability in predicting the following primary surgical outcomes were extracted when applicable: post-operative 30-day morbidity and/or mortality, 90-day morbidity and/or mortality, in-hospital morbidity and/or mortality, major complications (Clavien-Dindo grade III and above or Clavien-Dindo grade II and above) and overall complications.

Statistical Analysis

Meta-analysis was performed for the results of included studies when applicable. For the chosen outcomes of interest, odds ratios and 95% confidence intervals were extracted if available. Only the standard indices with sufficient combined results from all the studies were included in the meta-analysis (ECI, CCI, ACCI, CCI-D, mFI-5). Additionally, whether the index was assessed on a continuous predictor, or a cut-off was recorded. The co-morbidity indices utilizing continuous predictors were pooled in a group labeled scale. The co-morbidity indices using a cut-off value were pooled together in group labeled categorical. The statistical programming language R was then used to generate forest plots for the extracted odds ratios.

Results

Study Selection

The initial search revealed 748 records with an additional three records attained through citation searching (PRISMA Fig. 1). From the combined 751 articles, 581 articles were excluded after being screened via title and abstract yielding 170 articles after duplicates were also removed. The group of 170 studies were then sought for retrieval and the full text reviewed for each, 7 reports were unable to be retrieved leaving 163 studies for the assessment. After reviewing the full text for 163 articles, 111 studies were included in this review (Tables 1, 2, 3, 4, 5, 6).

Fig. 1
figure 1

PRISMA diagram of literature search

Full size image
Table 2 Studies assessing Co-morbidity Polypharmacy Score (CPS)
Full size table
Table 3 Studies assessing 5-item Modified Frailty Index (mFI-5)
Full size table
Table 4 Studies assessing Charlson Co-morbidity Index (CCI)
Full size table
Table 5 Studies assessing Charlson-Deyo Co-morbidity Index (CCI-D)
Full size table
Table 6 Studies assessing Elixhauser Co-morbidity Index/Measure (ECI/ECM)
Full size table

Study Characteristics

All 111 included studies were published between 2014 and 2022. The total cohort size of all studies assessed was 25,011,834 (cohort sizes ranged from 55 to 14,007,813). The operation of highest frequency was oncologic resection (n = 50/111, 45%). The other procedure types were amputation (n = 1/111, 0.9%), arthroplasty (n = 10/111, 9%), augmentation (n = 1/111, 0.9%), bypass (n = 2/111, 1.8%), fusion/decompression (n = 7/111, 6.3%), graft (n = 1/111, 0.9%), interventional (n = 1/111, 0.9%), reconstruction (n = 7/111, 6.3%), repair (n = 6/111, 5.4%), revascularization (n = 1/111, 0.9%), trauma (n = 8/111, 7.2%), and various others (n = 12/111, 10.8%). There were 25/111 studies included in this review that compared multiple comorbidity indices in the same cohort.

Comorbidity Instruments

Limiting included comorbidity indices per study to only those that meet the criteria for ease of use and reproducibility yielded 12 unique co-morbidity indices. In total, there were 136 co-morbidity indices assessed. The most common index assessed was the Charlson Comorbidity Index (CCI) (n = 42/136). Other indices assessed included the age-adjusted Charlson Comorbidity Index (ACCI) (n = 17/136), five-item modified frailty index (mFI-5) (n = 40/136), Elixhauser Comorbidity Index/Measure (ECI/ECM) (n = 17/136), modified ACCI (mACCI) (n = 1/136), Charlson-Deyo Comorbidity Index (CCI-D) (n = 9/136), comorbidity-polypharmacy score (CPS) (n = 7/136), modified Charlson Comorbidity Index (mCCI) (n = 1/136), enhanced ECI (n = 1/136), and age-adjusted mFI-5 (aamFI-5) (n = 1/136).

90-Day Morbidity and Mortality

Three studies reported data for 90-day mortality. There was not enough combined data to perform meta-analysis on these results. Franko 2018 28 reported a statistically significant association between CCI-D and 90-day mortality for patients undergoing esophagectomy (OR: 1.58, 95% CI: 1.01–2.49, p = 0.046). McFerrin 2019 29 reported a statistically significant association between CCI and oncologic cystectomy (HR: 1.42, 95% CI: 1.27–1.57, p < 0.001). Chiu 2015 30 reported a statistically significant association between CCI-D and 90-day mortality in patients undergoing oncologic hepatic resection (HR: 1.05, 95% CI: 1.023–1.068, p < 0.001). One study reported data for 90-day morbidity. DH Park 2022 31 reported a statistically insignificant association between CCI and 90-day morbidity in patients undergoing genitourinary oncologic resection (OR: 1.24, 95% CI: 0.8–1.04, p = 0.341). One study also reported data for 90-day major morbidity. Pujara 2015 32 found a statistical association between a polypharmacy score of greater than or equal to five and 90-day major morbidity among patients undergoing oncologic gastrectomy (OR: 2.36, 95% CI: 1.08–5.17, p = 0.3). The same study did not find a statistical association between CCI and 90-day major morbidity.

In-Hospital/30-Day Mortality

Fifty-four studies reported outcomes on in-hospital/30-day mortality, twenty-one of those studies reported odds ratios that could be pooled for meta-analysis. Of the twenty-one studies, there were four studies 24,33,34,35 that reported unadjusted data while the remaining seventeen studies reported adjusted data. Eleven studies 24,35,36,37,38,39,40,41,42,43,44,45 (20%) reported odds ratios for the mFI-5 and were pooled for meta-analysis. Of the studies reporting mFI-5, there was a statistical association with an increase in the odds of in-hospital/30-day mortality (OR: 1.97, 95% CI: 1.55–2.49, p < 0.01) (Fig. 2). There were five studies 39,46,47,48,49 (9.4%) that reported odds ratios for CCI and were pooled for meta-analysis. Of the pooled CCI results, there was an increase in the odds for in-hospital/30-day mortality (OR: 1.44, 95% CI: 1.27–1.64, p < 0.01) (Fig. 2). Pooled results for ACCI (n = 2, p = 0.38),33,34 CCI-D (n = 1), 28 and ECI (n = 1) 50 showed a positive trend but were not significant (Fig. 2).

Fig. 2
figure 2

Comparative performance of indices for in-hospital/30-day mortality. Shown: age-adjusted Charlson (ACCI), Charlson Co-morbidity Index (CCI), Charlson-Deyo score (CCI-D), Elixhauser Co-morbidity Index (ECI), and 5-item Modified Frailty Index (mFI-5). The overall effect of mFI-5 and CCI showed a statistical association with in-hospital/30-day mortality

Full size image

In-Hospital/30-Day Complications

Fifty-two studies reported outcomes for in-hospital/30-day morbidity. There were 26 (50%) studies that reported ORs for in-hospital/30-day morbidity that could be pooled for meta-analysis. Of the pooled studies, eleven studies 24,34,35,51,52,53,54,55,56,57 (21.2%) reported unadjusted odds ratios. Of the pooled unadjusted results, there was a statistical association between co-morbidities and an increase in the odds of in-hospital/30-day morbidity (OR: 1.62, 95% CI: 1.30–2.02, p < 0.01) (Fig. 3). Sixteen studies 36,38,40,41,42,43,44,45,58,59,60,61,62,63,64 (30.8%) reported adjusted odds ratios that were able to be pooled for meta-analysis. It should be noted that all studies (n = 16) from which adjusted data was obtained utilized the mFI-5. Of the pooled adjusted results combining data for all of the co-morbidity indices, there was a statistical association between the co-morbidity instruments and an increase in the odds of in-hospital/30-day morbidity (OR: 1.47, 95% CI: 1.31–1.64, p < 0.01) (Fig. 3). Ten studies 24,34,35,36,42,43,44,51,54,56 (19.2%) reported odds ratios assessing co-morbidity instruments utilizing a continuous predictor (scale) and were pooled for meta-analysis. Pooled results for co-morbidity indices utilizing a scale-based predictor were significantly associated with an increase in the odds of in-hospital/30-day morbidity (OR: 1.32, 95% CI: 1.20–1.46, p < 0.01) (Fig. 4). Sixteen studies 38,40,41,45,52,53,55,57,58,59,60,61,62,63,64,65 (30.8%) reported odds ratios assessing co-morbidities on a categorical basis. On pooled analysis, the categorical results showed a higher odd for in-hospital/30-day morbidity (OR: 1.74, 95% CI: 1.50–2.02, p < 0.01) (Fig. 4). One study (Marya et al. 2016) reported an increased odds of 30-day morbidity utilizing the ACCI (OR: 4.16, 95% CI: 1.87–9.26) and CCI (OR: 3.35, 95% CI: 1.08–10.40) that showed increased statistical power compared to the other studies (Fig. 4).

Fig. 3
figure 3

Comparative performance of indices using a continuous predictor (scale) and studies that use a cut-off (categorical) for in-hospital/30-day morbidity. Shown: age-adjusted Charlson (ACCI), Charlson Co-morbidity Index (CCI), and 5-item Modified Frailty Index (mFI-5). The overall effect of scaled and categorical indices showed a statistical association with in-hospital/30-day morbidity

Full size image
Fig. 4
figure 4

Comparative performance of adjusted and unadjusted study results for in-hospital/30-day morbidity. Shown: age-adjusted Charlson (ACCI), Charlson Co-morbidity Index (CCI), and 5-item Modified Frailty Index (mFI-5). The overall effect for adjusted and unadjusted data predicted in-hospital/30-day morbidity

Full size image

Severe Complications

Eighteen studies 34,41,44,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79 reported outcomes on severe complications. Severe complications were typically defined as CD grade III/IV and higher. 26 Eleven studies 34,41,44,65,68,71,72,73,74,77,78 (61%) reported odds ratios that were able to be pooled for meta-analysis (Fig. 5). Six studies 41,43,65,68,77,78 (33%) reported odds ratios for mFI-5 that were able to be pooled for meta-analysis. Of the pooled results, mFI-5 was significantly associated with severe complications (OR: 3.31, 95% CI: 1.13–9.67, p < 0.04) (Fig. 5). Pooled results for ACCI (n = 3, p = 0.19) and CCI (n = 2, p = 0.38) showed a positive trend toward severe complications but were not significant (Fig. 5).

Fig. 5
figure 5

Comparative performance of indices for severe complications. Shown: Age-adjusted Charlson (ACCI), Charlson Co-morbidity Index (CCI), and 5-item Modified Frailty Index (mFI-5). The overall effect for the mFI-5 showed a statistical association with severe complications

Full size image

Discussion

In this systematic review and meta-analysis of 111 studies, we found that, the mFI-5 outperformed the other traditional comorbidity indices (CCI, ACCI, ECI, CCI-D) in predicting in-hospital/30-day mortality and severe complications post-operatively. There was a great degree of statistical reporting heterogeneity among studies included which prevented two of the key outcomes (90-day mortality and morbidity) from being included in meta-analysis. Additionally, studies utilizing CPS were unable to be included in the meta-analysis due to how few studies reported data and based on their level of heterogeneity.

Additionally, it was found that Marya et al. 2016 reported extremely high ORs for 30-day morbidity utilizing the CCI and ACCI (Fig. 3). The reason for the high statistical power in this study are probably related to the fact that it represents one of the largest single-center, single-surgeon bilateral simultaneous total knee arthroplasty (BSLKA) studies done in the Asian population. Thus, the variation in management was kept for the most part homogenous among all participants which could have allowed for a more controlled study environment to measure CCI and ACCI. It was found that the ACCI out-performed CCI probably due to the inclusion of age since this is a major risk factor in BSLKA. 52 Unfortunately, there was limited reporting of studies utilizing the ACCI that were able to be included in the meta-analysis making the comparison to mF-5 somewhat limited. However, our meta-analysis still found the mFI-5 to be superior to the ACCI in predicting severe complications and in-hospital/30-day mortality.

To date, this is the first systematic review and meta-analysis to compare the effectiveness of co-morbidity indices versus the newer mFI-5 index. Functional dependent status (FDS) captured by mFI-5 could be the reason it out-performed the other indices on meta-analysis. FDS could better capture health status and risk for complications than traditional co-morbidity lists. The role of frailty in surgical outcomes has been highlighted in a recent well conducted systematic review and meta-analysis that found frailty indices significantly associated with 30-day mortality, adverse discharge disposition, post-operative complications, long-term mortality, and hospital length of stay. 16

The strengths of this study include the comprehensive nature of the literature search and the sound statistical comparisons made when available. Limitations of this review include the fact that the meta-analysis was limited to studies reporting ORs and thus many important data points may have not been included. The heterogeneity among reporting styles included in this study signal a need for more uniformity among the studies investigating frailty and co-morbidity indices in surgical patients. This would allow a more comprehensive meta-analysis to be performed to compare each index. Additionally, there may have been cohort overlap between studies which could have potentiated a bias in the results and our conclusions. In the severe complications category, some studies used their own definition and severe and this could have differed from the CD grading system of complications. Determining the best index per specialty would provide more value to this work. However, while attempting to make a statistical comparison of the different indices within a specialty, we realized that we do not have the data to make such a quantitative analysis. Within each specialty group, we only have multiple papers for mFI-5 but not any of the other indices. This is something that we would like to address in a future literature review as more publications become available. Finally, quality assessment of the studies found that generally the studies included were sound in methodology.

The results of this review highlight the value of newer frailty indices like mFI-5 in predicting surgical outcomes. There is a need for an increased degree of statistical homogeneity among studies attempting to validate co-morbidity indices. Specifically, the co-morbidity polypharmacy score (CPS) needs for studies to be conducted to allow a better comparison of the predictive capability of an index focused on capturing polypharmacy. As society ages, both frailty and polypharmacy are becoming significant factors in surgical performance and are in need of simple, yet accurate tools for their measure.

Conclusion

In conclusion, the cumulative data of this study suggest that frailty indices like mFI-5 are more predictive over traditional co-morbidity indices that do not include a capture of functional dependent status regarding surgical outcomes. Further studies are warranted to obtain a wider picture of how mFI-5 compares to other indices in predicting surgical outcomes including a comparison of both frailty and polypharmacy-based indices.