Introduction

Frailty is a clinical condition characterized by increased vulnerability to stressors caused by a cumulative decline in functioning across multiple physiological systems and its incidence is increasing [1, 2]. Frail individuals face elevated risks of adverse outcomes, such as falls, hospitalization, disability, need for long-term care, and mortality [1,2,3,4,5]. Moreover, frailty is associated with higher healthcare costs and increased consumption of healthcare resources, which are often limited [3]. Recognizing frailty in older people promptly during healthcare encounters is essential as it enables practitioners to evaluate the benefits and risks of any intervention and allows well-informed shared decision-making based on the individual patient’s goals. Additionally, it allows for targeted interventions to optimize the patient’s functional status [1,2,3,4, 6,7,8].

In acute care settings, premorbid frailty has demonstrated good prognostic value and is considered better in guiding decision-making for older patients than age alone [2, 5, 9,10,11,12,13]. When validated tools are utilized, the clinical frailty scale (CFS) is the most commonly employed tool [8]. The CFS was first developed in 2005 as a 7-point scale, with the aim of developing a tool that could effectively assess frailty, predict mortality or need for institutional care, and be easy to use. Later, the CFS was revised to a 9-point scale, redefining the highest grade of frailty into three distinct groups (severely frail, very severely frail, and terminally ill). Both versions of the scale were validated in real-life studies [4, 14].

Numerous studies have examined agreement of the CFS across multiple settings, yielding mixed results. For example, good agreement was observed in the emergency department when comparing an emergency physician and a study team that received prior education and training on assigning CFS scores. However, agreement was poor when comparing triage nurse scoring and inpatient assessment [7, 15]. In a critically ill population, agreement between geriatricians and intensivists was poor even after receiving prior education and training. Conversely, comparisons between critical care doctors, nurses, and physiotherapists showed good agreement [16, 17].

These findings demonstrate inconsistent interrater reliability of CFS scoring, which may be influenced by factors, such as experience, timing of administration, and available information. In response to this, Theou et al. developed a classification tree to improve CFS scoring reliability, particularly when employed by less-experienced raters [18].

Our objective was to validate the results of the study conducted by Theou et al., comparing CFS scoring performed by a senior geriatrician, a junior geriatrician, and using the classification tree for patients admitted to an acute geriatric unit. In contrast to the study of Theou et al., CFS assessment was performed without previous Comprehensive Geriatric Assessment (CGA) in order to mimic daily practice in which CFS is often based on clinical judgment alone. We asked the attending senior and junior geriatricians to assess premorbid CFS within the first 72 h of admission and determined agreement with the classification tree. Additionally, we evaluated the predictive value of the premorbid CFS in predicting 6-month mortality after admission.

Methods

Study design and setting

This is a prospective multi-center study performed in two general hospitals in Belgium. Patients were recruited at the acute geriatric units. In Belgium, each acute care hospital has a geriatric care program where older patients living with frailty are primarily admitted to acute geriatric units that are managed by geriatricians. These units are characterized by their focus on CGA by an interprofessional team, early rehabilitation, early discharge planning, and person-centered care. Patients were eligible for inclusion if they were aged 75 years and older and were admitted for at least 24 h. Exclusion criteria included failure to obtain informed consent, language or communication barriers, or patients who were dying on presentation. We refrained from using exclusion criteria based on certain patient characteristics to retain a study population representative of a standard population at an acute geriatric unit.

The study protocol was approved by the Ghent University Hospital Ethical Committee (reference number THE-2023-0187). The study was performed in accordance with the Declaration of Helsinki. Before enrollment, all included patients or their legal representative provided written informed consent.

Data collection

Data collection took place between September 2021 and March 2022. Medical and administrative data were extracted from administrative records. Further data were obtained during a structured face-to-face interview conducted by geriatric residents (junior geriatricians) using a standardized questionnaire [18]. In cases of mental incapacity, the interview was performed with the patient’s legal representative. Six months after the initial admission to the acute geriatric unit, junior geriatricians gathered survival status by telephone follow-up. Initial contact was attempted with the patient or their legal representative, followed by contacting the general practitioner if necessary.

Attribution of CFS

The premorbid CFS, based on the baseline health state (2 weeks before admission), was determined by the attending senior geriatrician based on clinical judgment. Independently, the junior geriatrician assigned a judgment-based premorbid CFS score. Finally, another junior geriatrician who did not have a treatment relationship with the patient assigned a premorbid CFS score using the classification tree. CFS assessment by attending senior and junior geriatrician was based on clinical judgment alone and happened within the first 72 h of admission.

Statistical analysis

All statistical analyses were performed with SPSS version 29. A value of p lower than 0.05 was considered statistically significant. We strived for a comparable sample size as the initial study performed by Theou et al., where a power analysis estimated a need for 40 ratings [18].

Level of agreement

We determined the level of agreement by calculating the intraclass correlation coefficient (ICC), comparing the classification tree CFS with those of the senior and junior geriatrician. ICC was based on a single measurement, absolute agreement, and a one-way random effects model. There are no standard values for acceptable reliability using ICC. Following the guidelines established by Koo and Li, we maintain the following cut-offs: values less than 0.5 indicate poor reliability, values between 0.5 and 0.75 moderate reliability, values greater than 0.75 good reliability [19].

Prognostic value for 6-month mortality

A receiver operator characteristic (ROC) curve and the area under the curve (AUC) determined diagnostic accuracy. Cox regression analysis based on the CFS was performed for time-dependent analysis, with adjustments made for age and gender, to predict mortality within 6 months of admission to the hospital.

Results

Study participants

This study included 97 patients with a mean age of 86 years (SD 5.2; range 75–100), among whom 64 patients (66%) were female. The main reasons for admission were fall/trauma (36.1%), infectious disease (24.7%), and pulmonary disease (19.6%). Additional characteristics of the participants are summarized in Table 1. Premorbid CFS scores of 5 or greater were attributed to 76.2% of the patients by the senior geriatrician, 57.8% by the junior geriatrician, and 86.6% using the classification tree (Table 2, Online Resource 1).

Table 1 Patient characteristics, N = 97
Full size table
Table 2 Proportion of patients assigned to each premorbid CFS level, N (%)
Full size table

CFS agreement

The scores obtained using the classification tree coincided with those from the senior geriatrician in 39.2% of cases, with 75.3% showing the same score or a difference of ± 1. The junior geriatrician scores were the same as the senior geriatrician scores in 26.8% of cases, with 75.2% of cases being within one level of each other. Similarly the scores obtained using the classification tree coincided with those from the junior geriatrician in 33% of cases, with 75.2% of cases being within one level of each other (Fig. 1).

Fig. 1
figure 1

Agreement of the CFS scoring

Full size image

The reliability of the CFS when determined by the senior geriatrician and the classification tree was moderate (ICC 0.526, 95% CI [0.366–0.656]), which was similar to the agreement between the CFS scores assigned by the senior and junior geriatricians (ICC 0.643, 95% CI [0.510–0.746]), as well as the agreement between the scores obtained using the classification tree and those from the junior geriatrician (ICC 0.573, 95% CI [0.423–0.693].

CFS and 6-month mortality

Five patients (5.2%) were lost to follow-up for survival analysis six months after hospitalization. Overall mortality within the study population was 26.1% (24 of 92 patients). The AUC for the classification tree CFS was 0.719, 95% CI [0.592–0.846]. Using a premorbid CFS cut-off value of 6 or higher, the sensitivity to predict 6-month mortality was 0.833, with a specificity of 0.426. A cut-off value of 7 resulted in a sensitivity of 0.375 and specificity of 0.956. Comparatively, the senior and junior geriatrician CFS yielded similar results with an AUC respectively of 0.774, 95% CI [0.673–0.875] and 0.774 95% CI [0.665–0.882]. The ROC curves are represented in Online Resource 2.

For Cox regression analysis, we divided the CFS into three groups: non-frailty (CFS 1–4), mild–moderate frailty (CFS 5–6), and severe–very severe frailty (CFS 7–8). When analyzing the classification tree CFS, this showed that severe–very severe frailty was associated with a higher risk of mortality compared to mild–moderate frailty (hazard ratio 6.274, 95% CI [2.613–15.062], p < 0.001) (Fig. 2). Age and gender were not significantly associated with a higher risk of mortality (p = 0.684 and p = 0.696, respectively). Six-month mortality rates were 16.7% in the non-frail group (2 of 12 patients), 17.2% in the mild–moderate frail group (11 of 64 patients) and 68.7% in the severe–very severe frail group (11 of 16 patients) (Fig. 2). Comparatively, the senior and junior geriatrician CFS rendered similar results (respectively hazard ratio 3.476, 95% CI [1.531–7.888], p = 0.003 and 4.851, 95% CI [1.891–12.442], p = 0.001) [Online Resource 3].

Fig. 2
figure 2

Survival analysis by classification tree CFS

Full size image

Discussion

The CFS demonstrates moderate agreement when determined by a senior geriatrician, junior geriatrician, or using the classification tree. Most scores were either the same or differed by only one level. However, discrepancies in the ratings were observed. Junior geriatricians tended to judge patients as more fit compared to the senior geriatricians, who in turn attributed lower CFS scores than the classification tree. Furthermore, the observed ICCs in this study are remarkably lower than the results in the initial study by Theou et al. There are several factors that can contribute to this. In the study by Theou et al., CFS was determined after CGA which may explain the higher level of agreement. In our study, the raters attributed a CFS score within the first days of admission after initial clinical evaluation to mimic daily practice. Given that the senior and junior geriatrician CFS were based on clinical judgment alone, junior geriatricians may lack the experience necessary to critically evaluate the functionality of the patient, leading to an overestimation of their fitness. Clinicians generally consider more nuanced factors, such as the patient’s will to live, severity and impact of comorbidities, and social support [14]. These factors are hard to incorporate into a classification tree. While small differences in the CFS score may be of little importance in certain situations, they become significant when a predetermined cut-off for CFS score is used to direct treatment plans. Therefore, it is essential to be attentive to its limitations, certainly when it is based on clinical judgment alone [10, 20]. The classification tree could be used as a way to better standardize CFS scoring.

Our results indicate that fit patients exhibit favorable 6-month survival rates, which rapidly decline with increasing premorbid CFS scores. Although we observed a significant increase in 6-month mortality in people living with mild–moderate frailty, it is only when using a CFS cut-off value of 7 that we can predict 6-month mortality with high specificity. These findings are consistent with previous studies [20, 21]. Notably, this study confirms that chronological age alone lacks predictive value in this context [9, 11]. These findings emphasize the need for developing individualized care plans where shared decision-making processes play a central role [9]. This process can be supported by the CFS though it should not be used as a sole determinant for treatment planning, as the CFS was not intended to replace CGA [10, 22].

A strength of our study is that our study population is heterogenous, representative of patients admitted to an acute geriatric unit. Patients with comorbidities or cognitive problems were not excluded since these patients encompass a prevalent population in daily practice. More than 75% of patients were considered frail when using a cut-off CFS of 5. These results are similar to the findings of Theou et al. [18]. Our study design, attributing a CFS score early during hospitalization, was developed to simulate real-life situations where the CFS score is often used early during hospitalization and based on clinical judgment alone. A limitation is that the sample size of this study was relatively small.

Validation of these findings through larger-scale trials is warranted. Furthermore, future research should investigate whether similar results can be obtained when the classification tree is administered by raters with no geriatric experience. We would suggest comparing the results from the CFS classification tree with the results of CFS after CGA to further investigate its validity.

Conclusion

In conclusion, the CFS classification tree exhibits moderate agreement with the assessments made by a senior geriatrician and provides predictive value for 6-month mortality among patients admitted to an acute geriatric unit. These findings suggest that using the classification tree can help standardize CFS scoring, improving reliability when used by less-experienced raters.