Abstract
Background/objectives
The Mini Nutritional Assessment Short-Form (MNA-SF) is the recommended screening tool for older persons. Data on interrater reliability in clinical routine are rare. Thus, we wanted to quantify the interrater reliability of the MNA-SF in hospital.
Subjects/methods
This observational cross-sectional study was undertaken retrospectively. The study population comprised 105 participants. Risk of malnutrition was measured twice with the routine MNA-SF performed by nurses (within 24 h after admission) and a dedicated dietician (one to three days after the first MNA-SF). The MNA-SF score was analyzed for interrater reliability between nurse and dietician.
Results
Participants’ mean age was 82.4 (±7.1) years and 71 (68%) were women. The mean total MNA-SF score was 7.4 (±2.4) assessed by dietician and 7.8 (±2.3) assessed by nurse. The intra-class correlation coefficient between the total MNA-SF scores was 0.74 (0.61; 0.82), indicating moderate reliability. For the MNA-SF nutritional status, Cohens Kappa was 0.37 (p < 0.05) showing a fair agreement.
Conclusion
Multiple misclassifications were observed between malnutrition and risk of malnutrition. Because mean scores were near the border between malnutrition and risk of malnutrition, we recommend to consider the total MNA-SF score in addition to the three risk groups to assess nutritional risk in geriatric hospital patients.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Introduction
The prevalence of malnutrion in geriatric hospital patients is high. Malnutrition has serious consequences for health outcomes [1]. Malnourished patients show e.g., higher complication rates, such as health care associated infections, increased length of hospital stay, unfavorable rehabilitation outcome and higher risk of mortality. Many trials proved, that treating malnutrition is capable to improve health outcomes [2,3,4]. The high prevalence of malnutrition and the treatment effectiveness justify the recommendation to integrate screening for risk of malnutrition into the geriatric assessment [5]. In this regard, the Mini-Nutritional-Assessment (MNA) is the recommended tool for older patients, for whom it has been specifically validated [6]. One of the problems of the full MNA is the fact that it takes about 15−20 min, which is not realistic for routine screening in hospitals, nursing homes and in the home care area. Therefore, the first six screening questions of the 18 items of the full form have been validated as a short form (SF). The results of the MNA-SF do not substantially differ from the results of the full form [7]. The MNA-SF takes only very few minutes to be performed and is therefore acceptable and practical for clinical routine. However, a reduction of items may lead to more susceptibility for errors, because the impact of every single question increases. Studies under research conditions found excellent test-retest reliability [8,9,10]. However, data on interrater reliability are rare, especially from clinical routine. Coincidentally, we observed during an ongoing clinical trial (Optimizing refeeding in malnourished frail older patients, DRKS-ID: DRKS00017324; https://www.drks.de/drks_web/setLocale_EN.do) that the German MNA-SF nutritional status performed by a dietician (nutritional specialist) frequently differed from the one that has been performed in clinical routine by the nurses on the ward. We noticed the lack of evidence regarding the interrater reliability of screening for risk of malnutrition using MNA-SF in routine geriatric hospital setting. Thus, we wanted to quantify the interrater reliability between the screening results for risk of malnutrition with MNA-SF performed by the nurse and the dietician, differentiated by the total MNA-SF score and the categories of MNA-SF nutritional status. Moreover, we investigated if single items of the MNA-SF are especially vulnerable for errors.
Subjects and methods
This observational cross-sectional study was undertaken retrospectively between July 2020 and October 2020 in the acute care geriatric hospital department of the university hospital Marien Hospital Herne, Germany. The MNA-SF is part of the routine geriatric assessment in the geriatric department since many years. The study population comprised 105 older participants. Participants were consecutively admitted to the geriatric acute care ward. Inclusion criteria were admission to the acute care geriatric hospital department, 60 years or older, and two screening results for malnutrition with MNA-SF performed by nurse and dietician. Exclusion criteria were severe cognitive impairment and palliative situation. Data were extracted from medical records. The study protocol had been approved by the ethical committee of Ruhr University Bochum (no 20-7077-BR approved on 29.10.2020). The study is registered at German Clinical trial register (DRKS-ID: DRKS00024658; https://www.drks.de/drks_web/setLocale_EN.do).
Geriatric assessment was routinely performed at hospital admission including (among other things) patients’ date of birth, sex, height, and weight. Body weight was assessed in light clothing on a seat scale with an accuracy of 0.1 kg, and height was measured to the nearest 0.5 cm with a stadiometer during hospitalization. If the measurement of weight and/or height was not possible, weight and/or height were estimated based on the patient’s report. Activities of daily living were determined using the Barthel-Index (BI) [11]. The point’s range for the German version of the BI is 0–100 pts, with 100 pts indicating independence in all activities of daily living. Cognitive function was measured with the Montreal Cognitive Assessment (MoCA) [12] in its translated German version. The point’s range for the German version of the MoCA is 0–30 pts, with higher pts. indicating normal cognitive function.
Risk of malnutrition was measured according to the MNA-SF [7], which is a validated tool for the screening for risk of malnutrition of geriatric patients across settings. We used the translated German version of the MNA-SF, which was not especially validated after translation. In our hospital, nurses receive individual unstructured training how to perform the MNA-SF from a more experienced nurse. MNA-SF nutritional status was assessed twice, each in a personal interview with the patient. A nurse performed the routine screening with MNA-SF first within 24 h after hospital admission. The screening by nurse was carried out by different nurses, depending on who was on duty at the ward at the day of admission. A dietician (always the same person) performed the second screening with MNA-SF one to three days after the first screening. In the presented paper following, screening by dietician is assumed the “expert” opinion.
The MNA-SF consists of six items (A–F) with a scoring up to 3 points per item: (A) Has food intake declined over the past 3 months due to loss of appetite, digestive problems, chewing or swallowing difficulties? (B) Involuntary weight loss during the last 3 months? (C) Mobility? (D) Has the patient suffered psychological stress or acute disease in the past 3 months? (E) Neuropsychological problems? (F1) Body Mass Index (BMI) (F2) Calf circumference (CC) in cm. BMI (F1) was read off a BMI disc (nurse) or calculated (dietician). If height or weight could not be captured, calf circumference (CC) in cm was measured and categorized as CC < 31 or ≥31 instead, as recommend by the MNA-SF. The screening result of the MNA-SF is the calculated total MNA-SF score (0–14). The total MNA-SF score is then transferred to the corresponding MNA-SF nutritional status (categories: normal nutritional status [12,13,14], risk of malnutrition [8,9,10,11], malnourished (0–7)).
The statistical analysis was completed using SPSS statistical software (SPSS Statistics for Windows, 137 IBM Corp, Version 27.0, Armonk, NY, USA). Means and standard deviations (SDs) were used for continuous data. Categorical variables are demonstrated as n (%). Participants were divided into three groups according to the MNA-SF classification (malnourished, risk of malnutrition and normal nutritional status). Participant’s age was calculated for the day of hospital admission. If the assessment with MoCA could not be fully recorded for non-cognitive reasons, the result was extrapolated according to the questions answered.
In the present work, the MNA-SF score was analyzed for interrater reliability between routine screening for risk of malnutrition with MNA-SF of the nurse and of the dietician in two analyses: interrater reliability for the total MNA-SF score (interval-scaled) and for the MNA-SF nutritional status (categorical variable).
A Bland-Altman plot was used to graphically demonstrate the consistency of the total MNA-SF score. The Bland-Altman plot compares graphically the two total MNA-SF scores (values 0–14). It also shows possible relationships between measurement inaccuracies (estimated via the differences between the two total MNA-SF scores) with the level of the true value (estimated via the mean value of the two total MNA-SF scores) [13]. Therefore, the Bland-Altman plot allows identification of any systematic difference between the measurements or possible outliers. 95% limits of agreement for each comparison (average difference ± 1.96 standard deviation of the difference) were computed. To calculate the interrater reliability between the interval-scaled total MNA-SF score of the nurse and the dietician, the intra-class correlation coefficient (ICC) estimates and their 95% confident intervals were used in the variant ICC (model two-way mixed, absolute agreement) [14]. An ICC of 1 indicates excellent interrater reliability, whereas an ICC of 0 indicates no interrater reliability [15].
Cohens Kappa was used to assess the interrater reliability between the categorical MNA-SF nutritional status of the nurse and the dietician. It was calculated for the MNA-SF nutritional status (normal nutritional status, risk of malnutrition, malnourished) and for each single item (A–F). A Kappa of 1 indicates perfect agreement, whereas a kappa of 0 indicates agreement equivalent to chance. A p < 0.05 was considered as the limit of significance.
Results
The study population comprised 105 older participants aged between 62 and 97 years (mean age 82.4 ± 7.1 years) with 68% being women. Reasons for hospital admission in this multimorbid and frail cohort were predominantly falls, fractures, infections, cardiovascular and neurodegenerative disease. Participants had a mean MoCA of 17.7 (±5.9) and a mean BI of 47.4 (±16.7). All baseline characteristics of study participants are summarized in Table 1.
MNA-SF total score and interrater reliability
The mean total MNA-SF score was 7.4 (±2.4) assessed by dietician with a range from 2 to 12. The mean total MNA-SF score was 7.8 (±2.3) assessed by nurse with a range from 2 to 13. The mean difference of the total MNA-SF score (nurse - dietician) was 0.3 (±2.1) ranging from −6 to 6. The highest frequency of differences was 0 in 21.9% of patients, followed by 1 with 21.0% of patients (Fig. 1).
The 95% limits of agreement for the Bland-Altman plot were 4.4 (=0.3 + 2.1*1.96) and −3.8 (=0.3 − 2.1*1.96). The Bland-Altman plot shows no systematic bias (e.g., higher range with increasing total MNA-SF score) between total difference and mean value of the two total MNA-SF scores (Fig. 2). The mean difference (+0.34) between the two total MNA-SF scores indicates that there are minor deviations, visible from the solid line near 0 in the Bland-Altman plot. Measurements are equally distributed in between upper and lower limits of agreement, visible from the dashed lines. The estimate for the ICC with 95% confidence interval between the total MNA-SF score of the nurse and the total MNA-SF score dietician was 0.74 (0.61; 0.82), indicating moderate reliability [15].
MNA-SF nutritional status and interrater reliability
According to MNA-SF nutritional status assessed by the dietician, 3 (2.9%) had a normal nutritional status, 51 (48.6%) were at risk of malnutrition, and 51 (48.6%) were malnourished. Nurses categorized 3 (2.9%) having a normal nutritional status (±0% compared to dietician’s nutritional status), 62 (59.0%) were at risk of malnutrition (+9.2% compared to dietician’s nutritional status), 40 (38.1%) malnourished (−10.5% compared to dietician’s nutritional status) (Fig. 3). The BMI of two patients could not be calculated by nurse due to missing weight and height (MNA-SF item F1), therefore CC was measured (MNA-SF item F2). The dietician calculated BMI of all patients. For the MNA-SF nutritional status, Cohens Kappa was 0.37 (p < 0.05) showing a fair agreement between MNA-SF nutritional status assessed by dietician and nurse [16]. One participant (1%) was categorized by dietician to have a “normal nutritional status”, and the nurse categorized this patient to be “malnourished”. Another participant (1%) was categorized by dietician to be “malnourished”, and the nurse categorized this patient to have a “normal nutritional status” (Table 2). Cohens Kappa for the single items of the MNA-SF (A-F) ranges from 0.21 to 0.69 (Table 3).
Discussion
We found a fair to moderate interrater reliability for the screening for risk of malnutrition in routine screening in hospital. Overall, this is a satisfactory result and underlines that the screening for the risk of malnutrition can sufficiently be performed by nurses, since they know many characteristics of the patient (e.g., cognition, mobility). In contrast to other studies [8,9,10], our study represents the actual “real life” routine in hospital, and not an artificially created situation under study conditions. Studies under research conditions examined test-retest reliability. Kather et al. (2020) found excellent test-retest reliability (ICC = 0.93) of the MNA-SF in patients aged 75 years and older after catheter-based interventions undergoing cardiac rehabilitation [8]. These results are comparable to previous research by Bleda et al. (2002) and Lin et al. (2019), who examined the test-retest reliability of the original full MNA in institutionalized older people and patients with stroke, respectively. Bleda et al. (2002) reported an excellent test-retest reliability (ICC = 0.89) and a substantial agreement (Kappa = 0.78) in 67 institutionalized patients [9]. Lin et al. (2019) found excellent test-retest reliability (ICC = 0.91) in test-retest reliability in 59 patients with stroke in the full MNA [10].
In our study, the mean total MNA-SF scores assessed by dietician (7.4) and nurse (7.8) revealed a minor difference of 0.3 showing a moderate interrater reliability. The mean scores, however, are exactly on the border between the two categories “risk of malnutrition” [8,9,10,11] and “malnourished” (0–7). This partly explains why the interrater reliability of the MNA-SF nutritional status only shows fair agreement. Although the overall interrater reliability between the assessment by a dedicated dietitian and routine assessment by the nurse in charge was only fair (Cohens Kappa: 0.37 (p < 0.05)), the present study demonstrates, that severe misclassification with possible clinical consequences happened rarely. Only one participant, who was classified as malnourished by the dietician was rated as normal nutritional status in routine assessment. The opposite misclassification, which was also seen in one other patient, would barely be harmful for the patient, because it led to nutritional counseling first, where this kind of misclassification would be discovered. Even if oral nutritional supplements were given without further counseling, this would not harm any patient, but lead to avoidable costs. Multiple misclassifications were observed between malnutrition and risk of malnutrition. Their clinical relevance depends on the routine management of patients at risk of malnutrition and is likewise probably low. If there is a clinical suspicion of categorical misclassification, the total MNA-SF score should also be taken into account for nutritional risk grading (particularly if the value is near the borderline between the two categories).
On the level of individual items of the MNA-SF, the first two questions about food intake and weight loss demonstrated the lowest level of agreement, with a Cohens Kappa of 0.23 (p < 0.05) and 0.21 (p < 0.05). Respectively, pointing out that it possibly would increase accuracy if moderate and severe decrease of food intake would be defined more precisely, as done in other screening tools. To the contrary, the weight loss item is clearly defined but frequently classified as unknown in the routine assessment. In contrast to the nurse in charge, the dedicated dietician often has gathered more information about the nutritional situation of the patient, including current weight and weight loss. The psychological stress/acute disease item should have always been rated as “yes” in this study, since hospitalized patients always suffer an acute disease. About 13% of this item was rated with “no” exclusively by the nurse. Thus, we conclude that nurses need more structured training on correctly performing the screening. In addition, it could be hypothesized that the accuracy of the screening for risk of malnutrition with MNA-SF could be improved by performing it on the second or third day of hospital stay, after obtaining more information about the patient, after body weight has been measured and food intake has been observed for more than one meal. Therefore, we recommend a re-screening after a few days for patients with missing or uncertain data to prevent only relying on the subjective information of the patient.
This study has some limitations. First, it is a single center study with a limited number of patients, which may affect the generalizability of the results. Although our study is a retrospective data analysis and some of the patients were subjects in different studies, the fraction of people being at risk of malnutrition or malnourished are comparable with a consecutively recruited study group [17, 18]. In addition, the routine assessment performed by nurses and the assessment performed by the dedicated dietician were not performed at the same day, so the dietician could have more information about the patient. All of these limitations may cause a bias. On the other hand, this situation reflects best the weakness of routine screening at time of hospital admission. A general problem is that only few patients had a normal nutritional status, which is expected in geriatric hospital patients, but which reduces the chance to detect misclassifications. Furthermore, the order of the screening was not random. The first screening was always carried out by the nurse and the second screening by the dietician. This may lead to a bias, since the patient had time after the first screening to reflect on the questions and to answer them more carefully during the subsequent screening by the dietician. Moreover, the day of the hospital admission is assumed stressful for the patient, and at the time of the screening by the dietician, the patient might be a little bit more used to the situation in the hospital and therefore able to answer the screening questions well considered.
We recommend taking into account the MNA-SF total score in addition to the MNA-SF categories for grading the (the risk of) malnutrition in each category. Moreover, we recommend performing the MNA-SF with sufficient information about the patient, even if screening is postponed to the second or third day of the hospital stay. Overall, nurses need to receive a structured trained in performing the screening for the risk of malnutrition using MNA-SF.
Data availability
Additional data are available from the corresponding author on reasonable request.
References
Leij-Halfwerk S, Verwijs MH, van Houdt S, Borkent JW, Guaitoli PR, Pelgrim T, et al. Prevalence of protein-energy malnutrition risk in European older adults in community, residential and hospital settings, according to 22 malnutrition screening tools validated for use in adults ≥65 years: a systematic review and meta-analysis. Maturitas. 2019;126:80–9.
Deutz NE, Matheson EM, Matarese LE, Luo M, Baggs GE, Nelson JL, et al. Readmission and mortality in malnourished, older, hospitalized adults treated with a specialized oral nutritional supplement: A randomized clinical trial. Clin Nutr. 2016;35:18–26.
Kaegi-Braun N, Mueller M, Schuetz P, Mueller B, Kutz A. Evaluation of nutritional support and in-hospital mortality in patients with malnutrition. JAMA Netw Open. 2021;4:e2033433.
Schuetz P, Fehr R, Baechli V, Geiser M, Deiss M, Gomes F, et al. Individualised nutritional support in medical inpatients at nutritional risk: a randomised clinical trial. Lancet. 2019;393:2312–21.
Volkert D, Beck AM, Cederholm T, Cruz-Jentoft A, Goisser S, Hooper L, et al. ESPEN guideline on clinical nutrition and hydration in geriatrics. Clin Nutr. 2019;38:10–47.
Kondrup J. ESPEN guidelines for nutrition screening 2002. Clin Nutr. 2003;22:415–21.
Kaiser MJ, Bauer JM, Ramsch C, Uter W, Guigoz Y, Cederholm T, et al. Validation of the Mini Nutritional Assessment short-form (MNA-SF): a practical tool for identification of nutritional status. J Nutr Health Aging. 2009;13:782–8.
Kather F, Hadzic M, Hehle T, Eichler S, Klein J, Völler H, et al. Test-retest reliability of the Mini Nutritional Assessment-Short Form (MNA-SF) in older patients undergoing cardiac rehabilitation. J Geriatr Cardiol. 2020;17:574–9.
Bleda MJ, Bolibar I, Parés R, Salvà A. Reliability of the mini nutritional assessment (MNA) in institutionalized elderly people. J Nutr Health Aging. 2002;6:134–7.
Lin S-C, Lin K-H, Lee Y-C, Peng H-Y, Chiu E-C. Test-retest reliability of the Mini Nutritional Assessment and its relationship with quality of life in patients with stroke. PLoS ONE. 2019;14:e0218749.
Mahoney FI, Barthel DW. Functional evaluation: the barthel index. Md State Med J. 1965;14:61–5.
Nasreddine ZS, Phillips NA, Bédirian V, Charbonneau S, Whitehead V, Collin I, et al. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc. 2005;53:695–9.
Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res. 1999;8:135–60.
McGraw KO, Wong SP. Forming inferences about some intraclass correlation coefficients. Psychol Methods. 1996;1:30–46.
Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15:155–63.
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.
Pourhassan M, Wirth R. Seasonal variation in vitamin D status among frail older hospitalized patients. J Frailty Aging. 2018;7:95–9.
Pourhassan M, Wirth R. An operationalized version of the Mini-Nutritional Assessment Short Form using comprehensive geriatric assessment. Clin Nutr ESPEN. 2018;27:100–4.
Funding
Open Access funding enabled and organized by Projekt DEAL.
Author information
Authors and Affiliations
Contributions
Conceptualization, RW and MP; methodology, RW and MP; formal analysis, UST; investigation, DD; resources, RW; data curation, UST; writing—original draft preparation, UST and RW; writing—review and editing, DV and RW; visualization, UST; supervision, DV and RW; project administration, RW; funding acquisition, no funding. All authors have read and agreed to the published version of the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Ethics approval and consent to participate
This study was conducted according to the guidelines laid down in the Declaration of Helsinki. The study protocol had been approved by the ethical committee of Ruhr University Bochum.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Trampisch, U.S., Pourhassan, M., Daubert, D. et al. Interrater reliability of routine screening for risk of malnutrition with the Mini Nutritional Assessment Short-Form in hospital. Eur J Clin Nutr 76, 1111–1116 (2022). https://doi.org/10.1038/s41430-022-01080-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41430-022-01080-y
- Springer Nature Limited
We’re sorry, something doesn't seem to be working properly.
Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.