Objectives

With an aging population and increasing life expectancy, there has been a significant increase in the demand for intensive care unit (ICU) admission among older patients (1–3). It is urgent to identify those at high risk of adverse outcomes after critical illness (4). Commonly used scoring systems in the ICU include the Acute Physiology and Chronic Health Evaluation II (APACHE II) and the Sequential Organ Failure Assessment (SOFA) score (5, 6), which are effective in predicting the mortality risk of critically ill patients. However, further exploration is needed to identify other risk factors for adverse outcomes in critically ill patients.

Frailty is a biological syndrome characterized by decreased biological reserves and increased vulnerability to stressors (7). A recent meta-analysis showed that the prevalence of frailty among older patients in the ICU was as high as 45% (8). Compared with non-frail patients, frail patients are less likely to tolerate critical illness, more likely to develop complications and die, and less likely to fully recover (9, 10). It is important to understand the relationship between frailty and adverse outcomes in older critically ill patients.

There is no consensus on the best method for screening frail patients in the ICU (11). The Clinical Frailty Scale (CFS) is the most commonly used in research but may introduce observer bias (12–14). Electronic medical record-based methods offer objectivity and convenience. The Hospital Frailty Risk Score (HFRS) is a validated tool based on International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) codes and can be easily evaluated using electronic medical records (15). However, studies on its validity in critically ill patients have been inconsistent (16–18). The Frailty Index based on physiological and laboratory tests (FI-lab) is another objective indicator that can be calculated using electronic medical record data; it can be used alone or in combination with other indicators to detect an increased risk of adverse outcomes in community populations (19, 20). However, there are few studies of application of FI-lab in critically ill patients (21). Additionally, information is limited on the impact of frailty status on long-term outcomes in older critically ill survivors.

In this study, we aimed to explore the predictive ability of two electronic medical record-based frailty assessment tools, the HFRS and FI-lab, for long-term mortality in older critically ill survivors.

Methods

Study design and data source

This was a retrospective observational study using the Medical Information Mart for Intensive Care (MIMIC)-IV database 2.0, which is an openly accessible critical care database that has received pre-existing institutional review board approval for data derivation (22, 23). This database comprises 76,943 ICU admission records between 2008 and 2019 from the Beth Israel Deaconess Medical Center in Boston, Massachusetts (USA). Access to this database was granted upon successful completion of the National Institutes of Health’s web-based course on Protecting Human Research Participants (certification number: 11153471).

Study population and data collection

We analyzed only the first-time ICU admission of each patient and excluded those who met any of the following criteria: (1) age less than 65 years; (2) died during the ICU or hospital stay; or (3) missing key variables, including demographic data, admission location, Sequential Organ Failure Assessment (SOFA) scores(6), items necessary to construct the HFRS or FI-lab, and survival status at 1 year after hospital discharge.

We extracted the following data from the database: demographic data (age, gender, and race and ethnicity categorized as White, Black or African American, and other); admission location (emergency department, operating room, peripheral hospital, and other); SOFA scores; comorbidities (cerebrovascular disease, congestive heart failure, dementia, diabetes mellitus, malignancy, myocardial infarction, and kidney disease); discharge location; and survival status at 1 year after hospital discharge.

Assessment of frailty

Hospital Frailty Risk Score (HFRS)

The HFRS score was calculated for each patient included in the study according to their ICD diagnoses documented during their current hospitalization, using methods previously described in the literature (15). Patients were then classified into one of three categories according to their HFRS score: low frailty risk (HFRS < 5), intermediate frailty risk (HFRS 5–15), and high frailty risk (HFRS > 15). Further details on the method used for constructing HFRS and the prevalence of the 109 ICD codes can be found in the supplemental materials (see «Detailed methods of construction of the Hospital Frailty Risk Score» and Table S1).

Frailty Index based on physiological and laboratory tests (FI-lab)

To construct the FI-lab, 33 items were tested or measured during a 24-hour period before and after admission to the ICU (19, 20). The FI-lab included 30 laboratory test items from venous blood, arterial blood gas, and urine samples, as well as three vital signs: systolic blood pressure, diastolic blood pressure, and heart rate. For items with multiple measurements, the mean value was used. Each item was dichotomized using the normal reference intervals provided in the database, and a score of 1 was given to any value lying outside of the reference range, indicating a deficit. The FI-lab score was calculated by summing the deficits present and dividing by the number of items included. The FI-lab score theoretically ranges from 0 to 1; in this study, the score ranged from 0 to 0.82. Only patients with fewer than 20% missing items (n ≥ 27) were included in the study. We divided patients into three frailty risk groups using tertiles: low frailty risk (FI-lab < 0.36), intermediate frailty risk (FI-lab 0.36–0.48), and high frailty risk (FI-lab > 0.48). The reference value, mean/median value, and defect prevalence for each FI-lab item are presented in Table S2.

Outcomes and follow-up

The primary outcome was 1-year mortality after hospital discharge. The secondary outcome was post-discharge care needs (post-discharge care needs including hospice, home care, and other healthcare facilities versus discharge home without further need for nursing care) (24). Follow-up started on the date of discharge and patient deaths occurring more than one year after hospital discharge are censored.

Statistical analyses

Continuous variables are presented as mean with standard deviation (SD) or median with interquartile range (IQR); categorical variables are presented as count and percentage. The Student t-test, analysis of variance, the Mann–Whitney U test, and χ2 test were used for descriptive analysis, as appropriate. Diagnostic consistency of the two frailty assessment tools was checked using Kappa analysis.

For survival analysis, univariate Kaplan–Meier and multivariate Cox regression models were conducted to investigate the relationship between frailty and 1-year mortality after hospital discharge. The Kaplan–Meier curves were compared using the log-rank test. Frailty was assessed as both a continuous and categorical variable in the multivariable Cox model, which was adjusted for age, gender, race and ethnicity, admission location, and SOFA score. Furthermore, five-knot restricted cubic spline curves were used to display the association of each frailty score with 1-year mortality. Logistic regression model was used to examine the association between frailty and post-discharge care needs.

To evaluate the incremental predictive value of change in each frailty score (as a continuous variable) and the combination of HFRS and FI-lab scores beyond the SOFA score-based model for the risk of 1-year mortality, we calculated and compared the C-index, integrated discrimination improvement (IDI), and net reclassification improvement (NRI).

To validate the robustness of our findings, we performed two sensitivity analyses. The first analysis involved repeating the multivariate analyses in 23,668 patients with all variables required to construct the HFRS. The second analysis involved adjusting the simplified Acute Physiology Score (SAPS) II (25), which is one of the most commonly used disease severity scores for critically ill patients and available from the MIMIC database, instead of SOFA to further account for the impact of disease severity.

All statistical analyses were performed using Stata version 17.0 (StataCorp LLC, College Station, TX, USA) and R version 4.1.2 (The R Project for Statistical Computing, Vienna, Austria). A two-tailed p value < 0.05 was considered statistically significant.

Results

Baseline characteristics

In total, 9,082 first-time ICU admission records for eligible adults aged 65 years or older were analyzed among 76,943 ICU admission records between 2008 and 2019 at Beth Israel Deaconess Medical Center in Boston. Figure S1 shows the selection procedures of the study. Table 1 presents the characteristics of included patients stratified by their survival status within 1 year of follow-up. In brief, the mean age of included patients was 77.2±7.8 years, and 4,753 patients (52.3%) were men. Patients who died within 1 year of discharge were older; more likely to have been admitted to the ICU via the emergency department; had higher SOFA scores; and had a higher proportion of chronic obstructive pulmonary disease, congestive heart failure, dementia, malignancy, myocardial infarction, and kidney disease compared with survivors. The kappa value showed a low diagnostic consistency of two frailty assessment tools (kappa=0.11, p < 0.001; Table S3). The distributions of HFRS and FI-lab scores for the study population are shown in Figure 1. Patients who died had significantly higher HFRS and FI-lab scores and a significantly higher proportion of frailty than survivors. There were no significant differences between groups in terms of gender, race and ethnicity, and prevalence of cerebrovascular disease and diabetes mellitus. Table S4 and S5 provide further detailed characteristics of patients stratified according to each frailty measure.

Table 1 Characteristics of included patients according to survival status within the 1-year follow-up period
Full size table
Figure 1
figure 1

Association of each frailty measure with 1-year mortality

Spline curves showing the association of each frailty measure with 1-year mortality; a) HFRS and b) FI-lab. The density curve describes the distribution of each frailty measure for more than 99% individuals in the study population. The dotted line indicates the cutoff score for low frailty risk and intermediate frailty risk/ high frailty risk which is also the reference standard for association between frailty and 1-year mortality: 5 points for HFRS and 0.36 points for FI-lab. Spline curves were adjusted for age, gender, race and ethnicity, admission location, and SOFA. None of the patients with FI-lab 0 to 0.06 died within the 1-year follow-up period. The colored background strip of each line represents the confidence interval of the line.

Full size image

Frailty and 1-year mortality

A total of 2,586 patients died within 1 year of follow-up. The 1-year mortality increased significantly with an increase in frailty severity, as measured using both the HFRS and the FI-lab (Table 2). The cumulative probability of mortality according to the category of frailty as assessed using the HFRS and FI-lab is presented using Kaplan–Meier curves (Figure 2). Univariate Cox regression analysis showed that both HFRS and FI-lab scores (assessed as a continuous variable) were significantly associated with an increased risk of 1-year mortality in surviving patients after discharge (HFRS: per point increase, hazard ratio [HR] 1.06, 95% confidence interval [CI] 1.06–1.07; FI-lab: per 0.01-point increase, HR 1.03, 95% CI 1.03–1.03). Each additional point on the HFRS was associated with a 6% increased risk of mortality (HR 1.06, 95% CI 1.05–1.06) after adjusting for age and gender, and each 0.01-point increase in the FI-lab was associated with a 3% increased risk of mortality (HR 1.03, 95% CI 1.03–1.03). The associations between both frailty measures and mortality remained significant after further adjustment for race and ethnicity, admission location, and SOFA in the multivariable model (Table 2, Figure 1). In the fully adjusted model, intermediate frailty risk patients had a 2-fold higher risk of mortality (HR 2.00, 95% CI 1.78–2.25) and a 1.6-fold higher risk (HR 1.59, 95% CI 1.44–1.76) compared with low frailty risk patients (assessed using the HFRS and FI-lab, respectively). High frailty risk patients had even higher risks of mortality, with HRs of 3.06 (95% CI 2.68–3.50) and 2.30 (95% CI 2.06–2.57) assessed using the HFRS and FI-lab, respectively.

Table 2 Association between frailty and 1-year mortality
Full size table
Figure 2
figure 2

Kaplan–Meier survival curves for mortality in critically ill patients with different frailty categories

Kaplan–Meier survival curves for mortality when frailty assessed using the a) HFRS and b) FI-lab. The colored background strip of each line represents the confidence interval of the line.

Full size image

Post-discharge care needs

The multivariate model showed that the HFRS was significantly associated with a higher probability of postdischarge care needs (per point, odds ratio [OR] 1.11, 95% CI 1.10–1.13; intermediate frailty risk, OR 1.79, 95% CI 1.55–2.06; high frailty risk, OR 5.97, 95% CI 4.29–8.30); the FI-lab was not associated with post-discharge care needs (per 0.01 points, OR 1.01, 95% CI 1.00–1.01; intermediate frailty risk, OR 1.08, 95% CI 0.92–1.26; high frailty risk, OR 1.07, 95% CI 0.88–1.29) (Table S6).

Improvement in mortality prediction

The addition of either the HFRS or the FI-lab to risk-adjustment model improved the discrimination for 1-year mortality beyond that provided by the SOFA score-based model. Among the two frailty measures, the HFRS performed better than the FI-lab for predicting 1-year mortality (Table 3, Table S7). Combining the HFRS and the FI-lab into a multivariable model resulted in a further increase in the A C-index to 0.042 (95% CI 0.036–0.047), which was superior to the addition of each frailty assessment alone. Moreover, the combined model showed the greatest improvements in IDI at 0.055 (95% CI 0.048–0.062) and NRI at 0.434 (95% CI 0.401–0.473).

Table 3 Incremental value of frailty for 1-year mortality
Full size table

Sensitivity analysis

Table S8 shows the baseline characteristics of 23,668 patients who had all variables needed to construct the HFRS. Sensitivity analyses confirmed the robustness of the associations between frailty, as assessed using the HFRS, and 1-year mortality (Table S9).

Furthermore, multivariate analyses, adjusted for SAPS II instead of SOFA, demonstrated similar associations between frailty and 1-year mortality, assessed using both frailty measures as both a continuous and categorized variable (Table S10).

Discussion

In this large single-center retrospective cohort study, we showed that both the HFRS and FI-lab could independently predict 1-year mortality in critically ill survivors aged 65 years and over in the ICU setting. Using either of these two frailty indicators could provide additional discriminatory value beyond SOFA for long-term survival after discharge, with addition of the HFRS showing greater improvement. The improvement was most evident when both frailty indicators were used in combination. In addition, the HFRS but not the FI-lab was associated with higher probability of post-discharge care needs.

The HFRS was initially developed for hospitalized populations and has been validated in several Western countries (15, 26–29). Limited studies have used the HFRS to assess frailty in ICU patients and predict adverse outcomes, but results have been inconsistent. Redfern and colleges reported that 71.5% of unplanned ICU admissions among hospitalized patients were intermediate frailty risk to high frailty risk, a proportion similar to that found in our study (18) (73.6% for the final study population or 59.1% for the population included in sensitivity analysis). However, Subramaniam et al. found that only 26.2% of patients were assessed as having intermediate frailty risk to high frailty risk using the HFRS (16). Differences in the frailty prevalence may be owing to reliance of the HFRS on counting comorbidities based on ICD-10 codes. Subramaniam et al. only used diagnoses at the time of admission for the current hospitalization to calculate HFRS scores, which may have led to a serious underestimation of frailty in critically ill patients; the prevalence of frailty among patients in the ICU is generally considered to be approximately 45% (8). Redfern and colleagues reported that the HFRS was not associated with in-hospital mortality among ICU patients (18). Bruno et al. also found that the HFRS was not associated with in-ICU mortality in multivariable models adjusted for APACHE II or SAPS II scores (17). Despite potentially underestimating the prevalence of frailty, Subramaniam et al. still found that the HFRS could predict in-hospital and 1-year mortality among critically ill patients (16). Our study found similar results in a larger population; frailty assessed using the HFRS could predict 1-year mortality among older critically ill survivors after discharge. The accuracy and completeness of ICD code records may vary among countries or hospitals of different levels, and a limited sample size may also affect results. Further research is needed to explore application of the HFRS in ICU settings. The HFRS is constructed based on claims data. Similarly, widely used comorbidity indices like the Charlson Index and Elixhauser Index, which are prognosis prediction tools, are also constructed based on claims data and show significant associations with adverse outcomes in critically ill patients (30). However, the construction of HFRS is more complex than that of comorbidity indices. Currently, there is no research comparing the predictive abilities of HFRS and comorbidity indices for adverse outcomes in critically ill patients and warrant further exploration in future study.

The FI-lab was initially developed for community populations and its predictive value for adverse outcomes in both community and hospitalized populations has been reported (19, 20, 31). This study provides the first report on the predictive value of FI-lab for long-term prognosis in critically ill patients with non-specific diseases. When applied in the ICU, FI-lab scores are higher than those in community populations (0.40 vs. 0.27) (20). Owing to the overlap between components of the FI-lab and markers of acute or subacute illness, the FI-lab applied in ICU settings reflects the severity of illness, to some extent. In contrast, the components of the HFRS include more comorbidities associated with long-term outcomes. This may explain why, in this study, the addition of the FI-lab to a multivariate model based on SOFA scores did not improve model performance as much as the addition of the HFRS. Moreover, the HFRS but not the FI-lab was associated with a higher likelihood of post-discharge care needs. Pre-hospital laboratory examination results may be a better choice for constructing the FI-lab but are difficult to implement. Additionally, we found that when two frailty indices were used in combination, model discrimination was improved most significantly. This result suggests that the HFRS and FI-lab may measure different aspects of frailty.

The CFS is the most commonly used frailty assessment tool in research in the ICU. However, the subjectivity of questionnaire-based frailty assessment methods may introduce observer bias. Electronic medical record-based frailty assessment methods can perform simple and rapid assessments for all patients, reducing the workload of physicians and avoiding subjective bias. However, the limitation of using ICD-10 codes is that differences in diagnostic records and coding may result in measurement errors. The FI-lab can more accurately reflect the current state of decline in a patient’s biological reserve. Further research is needed to explore use of the FI-lab among patients in ICU. The findings of our study suggest that utilizing frailty assessment to identify the high-risk population for adverse prognosis among critically ill survivors after discharge can facilitate timely interventions, such as nutritional support, medication management, and functional rehabilitation et al., to improve their long-term prognosis(32). Focusing on whether older critically ill survivors can return home directly after discharge is crucial for reallocating healthcare resources to better meet the growing medical needs of the older adult, especially those who have experienced critical illnesses.

This study has several important advantages. First, this study explored for the first time the predictive value of FI-lab for long-term adverse prognosis in critically ill patients with non-specific diseases. Second, this study has the largest known sample size for frailty assessment using the HFRS in an ICU setting, providing further evidence for validation of the HFRS. Finally, we found that the combined use of different frailty assessment methods may capture different aspects of a patient’s frailty characteristics, which can help guide future practice and research on frailty assessment in ICU settings.

We also acknowledge several limitations in this study. First, the HFRS is based on ICD-10 codes, and both ICD-9 and ICD-10 codes are present in the MIMIC-IV database. We used ICD-9 to ICD-10 conversion to construct the HFRS, which may result in slight inaccuracies in the resultant HFRS scores. Second, pre-hospital laboratory test results may be a better choice for constructing the FI-lab but these are not easily available. Lastly, because APACHE II scores are not available in the MIMIC database, we included SOFA scores in the multivariate analysis and replaced them with SAPS II scores in sensitivity analysis to confirm the robustness of our results.

Conclusion

Both the HFRS and FI-lab could independently predict 1-year mortality in critically ill older survivors aged 65 years and over in the ICU setting. Using either of these two frailty indicators could provide additional discriminatory value beyond SOFA for long-term survival after discharge, with addition of HFRS showing greater improvement. The improvement was most evident when both frailty indicators were used in combination. These findings suggest that electronic medical record-based frailty assessment methods can be useful tools for predicting long-term outcomes in older critically ill patients.