Introduction

Frailty has been associated with several adverse outcomes, such as disability, fall, hospitalization, and mortality, imposing heavy burden on frail older adults, care professionals and health care systems (1, 2). Previous studies have indicated that institutionalized older adults usually have high prevalence of frailty and are prone to a number of adverse outcomes in the short term (3, 4). Early identification coupled with effective interventions would offer opportunities to maximize their independence and quality of life and reduce health care costs.

The Comprehensive Geriatric Assessment (CGA) is a multidimensional and multidisciplinary diagnostic process, which focuses on determining the clinical profile, pathological risk, residual skills, short- and long-term prognosis, and personalized therapeutic and care plans of the frail older subjects (5, 6). The CGA consists of a set of screening tools, including assessment of cognitive and functional status, physical performance, nutritional conditions, co-morbidity and medication use, psychological state and social support (7). Different from the standard medical evaluation, the CGA concentrates on frail older people with complex problems, emphasizes on functional status and quality of life, and uses interdisciplinary teams and quantitative assessment scales to enable detection of individuals’ strengths, needs, and potential risks to inform individualized care planning and monitoring (8, 9). Currently, the CGA has been applied in long-term care facilities and other healthcare settings, such as primary care and hospitals, and has presented to be beneficial for health outcomes, including reduced hospitalization events, physical independence, cognitive functions, mortality, and quality of care (6, 9). However, it is difficult to integrate the CGA into the routine practice on account of the time, expertise and resources required in the process of the CGA. Furthermore, long-term care facilities, particularly in China, are lacking in healthcare professionals and resources. As a result, it is not practical to use the CGA as a routine care measurement among institutionalized older adults. A simple but accurate frailty screening instrument is urgently needed that can be quickly administered to identify frail older adults who may benefit from high-intensity CGA, which could optimize the process of frailty diagnosis and management.

At present, several frailty screening instruments with distinct properties have been used in the institutionalized setting, such as the Physical Frailty Phenotype (PFP) (10–12), FRAIL (fatigue, resistance, ambulation, illnesses, and loss of weight) (13), the Tilburg Frailty Indicator (TFI) (10, 14), and Groningen Frailty Indicator (GFI) (10). These frailty instruments are developed on the basis of different conceptual models: The PFP was proposed from a biological model (2); the FRAIL was developed by combining the components in biological, deficit accumulation, and functional models (15); the TFI and GFI were based on the integrative concept of losses in different domains of functioning (16, 17). Also, these instruments vary in measurement ways. The PFP includes objective physical measurements (grip strength and gait speed), requiring specific devices and expertise, and face-to-face interview; the FRAIL, TFI and GFI, purely self-report questionnaires, could be easily administrated by mail and telephone besides face-to-face interview.

However, these screening instruments are originally developed for community-dwelling and/or clinical populations. The FRAIL-NH scale has been recently developed for specific use in nursing homes (18). It draws on the strengths of the FRAIL, but was adapted to better fit the institutionalized older adults. Additionally, the FRAIL-NH includes core characteristics of the PFP and Frailty Index classification systems (2, 15, 19). It is easy to administer and contains seven potentially reversible conditions of frailty, which are fatigue, resistance, ambulation, incontinence or illness, weight loss, nutritional approach, and help with dressing. Previous studies have focused on comparing the FRAIL-NH with other frailty screening instruments in assessing frailty prevalence, associated factors, and predictive validity for adverse outcomes (e.g., falls, hospitalization and mortality) in nursing homes (11, 20–22). However, it remains unclear whether the specific FRAIL-NH and other screening instruments widely used in the institutionalized setting demonstrate good diagnostic test accuracy (DTA) against the CGA as a reference standard. The DTA of screening instruments is particularly important for health workers and decision-makers to accurately identify frail individuals and early initiate tailored care plans to decrease adverse outcomes.

Therefore, we examined the DTA of the specific FRAIL-NH and four frailty screening instruments (PFP, FRAIL, TFI, and GFI) with distinct properties against the CGA, aiming to find an appropriate screening instrument that could accurately identify frail older adults that may benefit from the high-intensity CGA in the institutionalized setting.

Methods

Design, Setting, and Participants

We conducted a cross-sectional study among institutionalized adults aged 60 years or older between July 2015 and August 2016 in Jinan City, Shandong Province, China. Residents were selected from 23 nursing homes in 5 districts (Lixia, Licheng, Shizhong, Tianqiao, and Huaiyin District) in Jinan. We excluded participants with severe physical deficits and severe cognitive impairment (≥ 8 on the Short Portable Mental Status Questionnaire (SPMSQ)) which precluded older adults to complete physical performance examinations (23). A total of 305 older adults living in the nursing homes for at least 1 month were enrolled. Participants received structured questionnaire interviews and physical measurements conducted by researchers. This study was approved by the Institutional Review Board of Shandong University, Jinan, China. All participants signed written informed consent forms.

Measurements

Data were collected on nursing home characteristics (i.e., ownership and affiliation), demographic characteristics (i.e., age, sex, marital status, and education), and cognitive status evaluated by the SPMSQ (23).

Frailty Assessment

The FRAIL-NH (18) includes seven items (i.e., fatigue, resistance, ambulation, incontinence, weight loss, nutritional approach, and help with dressing). The range of possible total score is between 0 and 14, and a cut-point of 2 was used to identify frailty (11). The PFP (2) includes five components: exhaustion, weight loss, low physical activity, weakness, and slowness, with a score of 3–5 representing frail, 1–2 for pre-frail, and 0 for robust, respectively. The FRAIL (15, 24) scale comprises five self-report dichotomous items: frail (3–5), pre-frail (1–2), and robust (0). The TFI comprises 15 self-report items across physical, psychological, and social domains (range 0–15), with a cut-point of ≥ 5 (16). The GFI includes 15 self-report items focusing on the loss of functions and resources in physical, cognitive, psychological, and social domains, with a cut-point of ≥ 4 (17).

According to previous systematic reviews using the CGA as the reference standard of frailty among older cancer patients, the median domain of CGA was seven (range 3–9), involving cognition, function, mobility, nutrition, mood, polyparmacy, and social support (25, 26). The CGA in this study consists of seven domains with separate cut-offs indicating their abnormalities: cognition (SPMSQ, cut-off: ≥ 3) (23), function (Activity of Daily Living, ADL, having difficulties on at least one item of ADL) (27), mobility (Timed “Up & Go” Test, TUG, cut-off: > 20s) (28), depression (the 5-item Geriatric Depression Scale, GDS-5, cut-off: ≥ 2) (29), nutritional status (the Mini Nutritional Assessment-Short Form, MNA-SF, cut-off: < 12) (30), polypharmacy (≥ 5 prescription medications) (31), and social support (the Social Support Rating Scale, SSRS, cut-off: ≤ 31) (32). Consistent with previous studies (25, 26), frailty was defined as the presence of abnormalities in two or more CGA domains.

Statistical Analysis

Means (with standard deviations) and frequencies (with percentages) were used for continuous and categorical variables to characterize the sample, respectively. The receiver operating characteristic curves (ROCs) were plotted to evaluate the areas under the ROC curves (AUCs) for the FRAIL-NH, PFP, FRAIL, TFI, and GFI against the reference standard CGA. We considered an AUC > .70 as an indicator of good diagnostic test accuracy (DTA) (33). The optimal cut-point was determined by the maximum value of the Youden index (YI, calculated as sensitivity + specificity − 1). Diagnostic properties were calculated on sensitivity, specificity, positive predictive values (PPVs), negative predictive values (NPVs), correctly classified rates (CCRs) and kappa values of the five frailty screening instruments at the optimal and original cut-points. The strength of agreement related to the kappa value is as follows: < 0, poor; 0 to 0.20, slight; 0.21 to 0.40, fair; 0.41 to 0.60, moderate; 0.61 to 0.80, substantial; 0.81 to 1.00, almost perfect (34). The nonparametric method described by DeLong et al.(35) was used to calculate standard errors and compare differences in AUCs. Analyses were performed using the IBM SPSS Statistics 21.0 (IBM Corp., Somers, NY) and Stata 12.0 (Stata Corp, College Station, Texas). A probability level of P < .05 was considered statistically significant in a 2-tailed test.

Results

Of 23 nursing homes, the majority were private (78.3%) and freestanding (91.3%) (Table 1). Older participants had a mean age of 79.3 years, and the majority were female (57.0%), mateless (80.3%), and received primary school or lower educational level (57.4%). The prevalence of frailty ranged from 25.9% (FRAIL) to 56.4% (GFI). The CGA defined 73.8% of the participants as frail.

Table 1 Characteristics of the Participants
Full size table

The AUCs (Table 2) ranged from 0.80 (FRAIL: 95% CI 0.74–0.85) to 0.83 (PFP: 95% CI 0.78–0.88) for five frailty screening instruments against the CGA in the detection of frailty. At their original cut-points, all the five frailty screening instruments presented low sensitivity (32.9%–69.3%) and high specificity (80.0%–93.8%), as well as high PPVs (94.9%–96.7%) and low NPVs (33.2%–48.1%). The CCRs of frailty ranged from 48.9% (FRAIL) to 72.1% (GFI), with varied agreement with the CGA (kappa: 0.168–0.407). At their optimal cut-points, the sensitivity and specificity of the FRAIL-NH, PFP, and FRAIL tended to be balanced, and their CCRs (76.1%–81.3%) and kappa values (0.465–0.523) increased a lot, but their PPVs were persistently high and their NPVs were persistently low. The optimal cut-points of the TFI and GFI were identical with their original cut-points, respectively.

Table 2 Diagnostic Properties† of Five Frailty Screening Instruments at the Original and optimal‡ Cut-points using the CGA as Reference Standard (n = 305)
Full size table

On ROC contrasts (Figure 1), there were no significant differences in the AUCs between any screening instruments (χ2: 0.0003–1.38, P > .05).

Figure 1
figure 1

The ROC curves for five frailty screening instruments against the reference CGA in detection of frailty

ROC contrast: FRAIL-NH vs. PFP, χ2 = 0.85, P = .356; FRAIL-NH vs. FRAIL, χ2 = 0.02, P = .898; FRAIL-NH vs. TFI, χ2 = 0.0003, P = .985; FRAIL-NH vs. GFI, χ2 = 0.01, P = .934; PFP vs. FRAIL, χ2 = 1.38, P = .240; PFP vs. TFI, χ2 = 0.78, P = .378; PFP vs. GFI, χ2 = 0.61, P = .436; FRAIL vs. TFI, χ2 = 0.02, P = .878; FRAIL vs. GFI, χ2 = 0.04, P = .847; TFI vs. GFI, χ2 = 0.01, P = .924.

Full size image

Discussion

The prevalence of frailty varied widely by five frailty screening instruments among institutionalized older adults (25.9%–56.4%), consistent with previous studies among older residents of assisted living facilities (4) and nursing homes (10, 11), as well as other diverse populations including community-dwelling older adults (36–38), hospitalized patients (39, 40) and patients in medical emergency units (41). Also, the multidimensional TFI and GFI identified more frail older adults than did other purely physical dimensional instruments of the FRAIL, FRAIL-NH, and PFP. The reason may be that the multidimensional instruments include psychological and social components of frailty that were neglected by the purely physical dimensional instruments. That is, the wide variation in dimensions or components inherent in frailty screening instruments may explain why they identified differential frail older adults.

ROC analyses showed that all the five frailty screening instruments had good diagnostic accuracy against the CGA in the detection of frailty. However, their sensitivity (32.9%–69.3%) and specificity (80.0%–93.8%) at the original cut-point varied widely. Previous research among older cancer patients also observed the wide range of sensitivity (25%–92%) and specificity (39%–100%) when comparing the DTA of frailty screening instruments against the CGA (26). Besides, all the five frailty screening instruments had high specificity and high PPVs but low sensitivity and low NPVs, consistent with the results among older cancer patients (42, 43). This may be related to the higher prevalence of frailty among institutionalized older adults and older cancer patients (3, 25). In practice, the selection of frailty screening instrument with high sensitivity or with high specificity should be based on the specific context. A frailty screening instrument with high sensitivity and high NPV could have high accuracy to exclude frail individuals, while a screening instrument with high specificity and high PPV could accurately identify frail individuals. In the institutionalized setting lack of healthcare professionals and resources, a frailty screening instrument with high specificity and high PPV may be particularly important, because the limited resources can be allocated to really frail older adults who need high-intensity CGA and frailty interventions.

At their original cut-points, five frailty screening instruments had slight to moderate agreement with the CGA. However, at their optimal cut-points, the agreement increased a lot, especially for the FRAIL that increased from slight to moderate; their sensitivity and specificity were more balanced. Therefore, it is necessary to adjust the cut-off value of frailty for screening instruments in the institutionalized setting. Additionally, the cut-off value for screening instruments could be conditional on healthcare resources. A lower cut-off value with higher sensitivity may be adopted to reduce missed diagnoses and ensure more frail older adults to benefit from the CGA and care plans in the case of adequate healthcare resources, while a higher cut-off value with higher specificity may be adopted to reduce misdiagnoses and improve efficiency of resource allocations in short of healthcare resources.

Also, ROC contrasts for the AUCs against the CGA showed that the specific FRAIL-NH, self-report FRAIL, TFI, and GFI, as well as hybrid PFP had similarly good diagnostic accuracy. Although the PFP is widely recognized, it contains objective physical performance measures, such as gait speed and grip strength, which could not be assessed among older residents experiencing serious physical deficits. In addition, the CGA also involved physical performance measures, such as the TUG test. Therefore, we did not include those older residents with serious physical deficits when comparing the DTA of five frailty instruments. Going beyond the AUCs, the specific FRAIL-NH, which captures reversible conditions in nursing home residents, and self-report FRAIL, TFI, and GFI may be more applicable for their simplicity, flexibility and easy administration in the institutionalized setting where a substantial proportion of older residents experience dependencies.

To our knowledge, this is the first study to compare the diagnostic accuracy of the specific FRAIL-NH and several screening instruments against the CGA among institutionalized older adults. However, some limitations of our study should be noted. First, our study excluded a number of older adults with severe physical deficits and cognitive impairment, which may contribute to low identification of frailty. Also, the sample from one city may limit the generalisability of the results.

In conclusion, the prevalence of frailty among institutionalized older adults varies widely by five frailty screening instruments. The specific FRAIL-NH, self-report FRAIL, TFI and GFI, as well as hybrid PFP show similarly good diagnostic properties in identifying frailty against the CGA.