Introduction

Malnutrition is a problem in residential facilities for elderly. Literature about malnutrition in nursing homes report a prevalence between 19% and 53% depending on population and applied study design (1-3). It is estimated that 39% to 60% of the elderly in nursing homes are at risk for malnutrition (2-4).

The consequences of malnutrition are numerous and include a decline in functional status and psychosocial wellbeing, increased health care costs and increased mortality, with negative impacts on quality of life (5, 6).

An important risk factor for malnutrition in these settings is inadequate food intake (7). Various reasons for this are described in literature, for example: poor appetite, alterations in taste and smell, cognitive and functional impairment, poor oral/dental health, chronic diseases, and polypharmacy (4, 6, 8, 9, 10). Nevertheless, research has indicated that the specific needs and preferences of the elderly in nursing homes, associated with reasons of inadequate food intake, are not sufficiently met (11, 12). Nijs et al. (13) described that optimizing meal quality by offering a homestyle environment, choices, a longer time to eat, more dignified mealtime assistance and stimulating independence, can improve food intake.

The optimization of meals and meal service is an important quality improvement target. To guide quality improvement strategies it is essential to gather valid and reliable information. This information can be obtained by the registration of quality indicators (14).

Although the problem of inadequate food intake in elderly residential facilities is well known (10, 15, 16), there were no indicators available which could help guide strategies to improve the quality of meals and meal service.

The aim of this study was to develop a content validated set of quality indicators evaluating the quality of meals and meal service in residential facilities for elderly.

Methods

The ‘quality of meals and meal service’ set of indicators has been developed according to the Indicator Development Manual of the Dutch Institute for Health Care Improvement (CBO) (17). This manual is based on the instrument Appraisal of Indicators Through Research and Evaluation (AIRE) (18) which was derived from the instrument Appraisal of Guidelines Through Research and Evaluation (AGREE) (19).

Using the Indicator Development Manual (17) as a guide, this study focused on the development and validation of a set of quality indicators via nine consecutive steps: 1) establishing the overall goal, 2) composing a working group, 3) re-confirming the overall goal with the working group, 4) clearly defining the scope, 5) searching for indicators, 6) listing potential indicators, 7) summarizing potential indicators, 8) elaborating indicators into factsheets, and 9) composing a reading guide. The following steps include feasibility and pilot testing of the set of indicators.

Composition of the working group

The working group (step 2) was made up of three nurse researchers and a gastrology expert. (Gastrology is evidence, practice and creative based knowledge of specific methods and techniques concerning tasty, healthy and safe food, food service and choice to meet the needs and preferences of elderly residents.) The group had expertise in elderly care, malnutrition, indicator development, and food quality.

Defining the purpose and scope

The set of indicators had to measure the quality of meals and meal service in residential facilities for elderly where at least a principal meal is offered (step 1, 3, 4). The information obtained from the indicator set needs to be appropriate to guide quality improvement processes which are focusing on food intake. Furthermore the indicator set had to cover the three broad domains affecting meal satisfaction in elderly: food, food service and choice, as well as nutritional screening (20).

Search for potential indicators

Preferably, indicators should be based on an evidence based guideline. In the absence of such a guideline, the best available evidence has to be used. (17)

In this study, potential indicators were identified via existing sets of indicators, evidence based guidelines, scientific literature search and working group’s expertise (step 5). The following search terms were combined: nursing home, elderly, nutrition, nutritional status, satisfaction, food, meal, food services, indicator, quality account indicator and quality.

Existing indicator sets and guidelines were searched via national and international organizations, which are specialized in the development or publication of quality indicators or guidelines, and are recommended by the CBO (17). Furthermore the websites of following projects and organizations were consulted: Assessing Care of Vulnerable Elders (ACOVE) (21), the Resident Assessment Instrument (22), the Belgian’s Federal Council on the Quality of the Nursing Activities (14), the Flemish Indicator Project Nursing Homes (23) and the Dutch Health Care Inspectorate (24).

The electronic databases of PubMed, Cumulative Index to Nursing and Allied Health Literature (CINAHL) and Web of Science were used to search for literature.

Listing potential indicators

The potential indicators retrieved from the search were classified according to structural, process, and outcome indicators (step 6). The scientific rigour required to support the relationships among each indicator, the quality of meals and meal service, and residents’ outcomes, was provided. Next, for each indicator, criteria were described (e.g. Is an action plan for malnourished residents available?). Criteria are measurable elements of care through which the result of an indicator can be calculated (25). The result of an indicator was expressed as a percentage. For structural indicators the numerator was the sum of the results of the criteria (No/absent = 0; Yes/present = 1), the denominator the number of criteria. For process and outcome indicators the numerator was the number of residents in accordance with the indicator and the denominator the number of included residents.

Summarizing potential indicators

The relevance of each indicator was evaluated in a double Delphi procedure (step 7). A panel of 11 experts in elderly meal care was formed, according to Lynn’s recommendations for establishing content validity (26). Almost half of the experts were professionals with expertise in elderly care (a head nurse, a speech therapist, two dieticians, a quality manager) and half of the experts were chefs de cuisine. The chefs had received supplementary education in cuisine for the elderly with a focus either on cooking or on food service and choice. Through their education and their expertise in nursing homes, their feedback was appropriate to complement our findings from literature. The experts independently reviewed the list of potential indicators. In the first consultation round, each expert received a form by mail with the indicators, supporting scientific evidence, and the criteria. The experts were asked to rate each indicator on a 10-point scale for relevance (to residents’ health or well-being) (27). A rating of at least 7 points (upper tertile) was previously determined as ‘relevant’ (28). For each indicator, suggestions or additional evidence could be written on the form. At the end of the first consultation round, the indicator set was modified based on these expert comments. Modifications were discussed within the working group.

In the second round, experts were also asked to rate the relevance of the criteria measuring each indicator. The experts also received an overview of the comments and the ratings from the first round.

Analysis of the panel ratings

Panel ratings were analyzed according to the definition of agreement of the RAND-group (28) and Lynn’s (26) method of establishing content validity. First, for each expert, all indicators/criteria rating at least 7 points on a 10-point scale (upper tertile) were marked as ‘relevant’ (28). Subsequently, the proportion of experts who agreed about relevance was calculated for each indicator/criterion (26, 28). This proportion was expressed as the content validity index (CVI) (26). Finally, all indicators/criteria with a CVI of at least 80% were accepted as being content valid (26, 28). Indicators and criteria were then adapted based on the written comments from the experts.

Table 1
figure 1

‘Quality of meals and meal service’ set of indicators: structural indicators

Full size image

Ethical approval

This study was approved by the ethical committee of Ghent University Hospital (B/670201420070). All experts were informed by e-mail about the purpose and the procedure of the study. A form to assess and comment on the indicator set was attached. The return of a completed form was considered as informed consent to participate. All information from the assessment forms was combined in one file that did not include any identifying information. The summary of the results and comments, returned to the experts after each Delphi round, was checked for references to individual experts (e.g. by name or characteristics of the organization). References were deleted, as necessary.

Results

Listing potential indicators

Based on the literature review, a preliminary list of 20 indicators (13 structural, 3 process and 4 outcome), with one to five criteria per indicator (total 45), was compiled. The indicators covered the domains of food, food service, choice and nutritional screening.

Two process indicators were derived from the indicator set of the Belgian Federal Council on the Quality of Nursing Activities (14): ‘The proportion of residents with documented results of a malnutrition screening’ and ‘The proportion of residents whose weight change was documented’. The first indicator was also mentioned in the indicator set of the Dutch Health Care Inspectorate (24), and the guidelines of the American Dietetic Association (29), The European Society for Clinical Nutrition and Metabolism (30), and The American Society for Parenteral and Enteral Nutrition (31). One outcome indicator was derived from the Dutch Health Care Inspectorate: ‘The prevalence of residents expressing mealtime satisfaction’ (24).

Table 2
figure 2

‘Quality of meals and meal service’ indicator set: process and outcome indicators

Full size image

Because few existing indicators and guidelines were found addressing the quality of meals and meal service, we formulated 17 new indicators based on scientific literature and the working group’s expertise.

Summarizing potential indicators (first Delphi round)

In the first Delphi round, 11 experts, previously identified, reviewed the primary list of 20 potential indicators. The content validity index (CVI) ranged from 0.50 to 1. Sixteen relevant indicators (CVI ≥ 0.80) were identified. Results and experts’ comments were discussed within the working group. One indicator, ‘Cold drinks were available for the residents’ (CVI = 0.70), was removed because of the expert group’s assessment. Two other indicators, which were assessed with a low CVI, were maintained following discussion in the working group: ‘The proportion of residents with risk of malnutrition’ (CVI = 0.60) and ‘The proportion of residents with malnutrition’ (CVI = 0.70). These outcome indicators were considered important to measure the effect of meal care on the residents. One last indicator, with a low CVI, was rewritten: ‘The proportion of residents with abdominal obesity’ (CVI = 0.56). The indicator ‘A policy for the amount of food is established’ (CVI = 0.82) was removed, although the expert group assessed this indicator with a CVI above the threshold. However, according to the experts’ written comments, the measurability of this indicator was questionable.

Table 3
figure 3

Method for calculating the result of structural indicators

Full size image

Based on the experts’ comments, four indicators and seven criteria were reformulated, 12 criteria were added and 29 criteria were removed. Five indicators were rewritten as a criterion measuring a broader indicator (e.g. ‘Is a validated screening and monitoring instrument available?’ became a criterion for ‘A procedure for screening and caring for malnourished residents is established.’). Two new indicators were added.

The first Delphi round resulted in a list of 15 potential indicators (seven structural, four process and four outcome) including 33 criteria.

Summarizing potential indicators (second Delphi round)

In the second Delphi round, six experts reviewed the list of 15 potential indicators from the first Delphi round: a speech therapist, a dietician and four chefs de cuisine. In addition to the first Delphi round, experts also rated each criterion.

The CVI for the indicators ranged from 0.83 to 1. The CVI for the criteria ranged from 0.33 to 1. All 15 indicators and 28 of the 33 criteria were found to be relevant (CVI ≥ 0.80).

The results and the experts’ comments were again discussed in the working group. Two indicators were removed, although they were assessed as being relevant by the expert group. The indicator ‘Food and drinks are fresh’ (CVI = 0.83) was removed because three of the four criteria were assessed as being less relevant and, according to written comments, there was no agreement about the definition of ‘fresh’ food. The indicator ‘The proportion of residents with abdominal obesity’ (CVI = 0.83) was removed because obesity in elderly is of minor importance for health and difficult to measure because of altered body composition.

The working group decided to remove three criteria, which were assessed as being relevant by the expert group. The criterion ‘Is a procedure for screening residents for over-nutrition available?’ (CVI = 1.00) was similar to the criterion ‘Is a validated screening and monitoring instrument available?’. The criterion ‘Are meals prepared with fresh or frozen ingredients?’ (CVI = 0.80) was removed because of ongoing discussion about the benefit of fresh or frozen ingredients for meal quality. The criterion ‘Number of residents with abdominal obesity’ (CVI = 0.83) was removed for the same reason as the corresponding indicator (see above).

Figure 1
figure 4

Development process of the ‘quality of meals and meal service’ set of indicators

Full size image

Based on the experts’ comments 6 criteria were rewritten. No indicator or criterion was added.

The second Delphi round resulted in a list of 13 potential indicators (six structural, four process and three outcome) including 25 criteria (see Tables 1 and 2). Table 3 gives an example of the calculation for a structural indicator.

Discussion

In this study, a set of 13 quality indicators measuring the quality of meals and meal service in elderly long-term facilities was developed and content validity established. The indicator set covers the three broad domains affecting meal satisfaction in elderly: food, food service, choice, as well as nutritional screening. Furthermore, the information obtained through the application of the indicator set is appropriate to guide quality improvement processes which focus on food intake in order to reduce malnutrition.

We focused on developing three types of quality indicators: structural, process and outcome indicators. Other studies on quality indicators in elderly care focused on structural or process indicators. Kröger et al. (40) selected process indicators because these indicators assess actual care given and help to detect care and service processes which need improvement. Van Nie et al. (41) selected structural indicators based on Donabedian’s framework on quality of care, to identify factors influencing malnutrition. Donabedian’s framework shows that the outcome of care is dependent on the process and structural aspects of care (42). Structural and process indicators are essential sources of information to guide quality improvement processes. Moreover they are less influenced by confounding factors and give results quickly (17). Nevertheless, structural and process indicators do not give direct information about the goal of the quality improvement project (reducing malnutrition) and the perception of the residents (meal satisfaction). Meal and meal service quality improvement projects in residential facilities are in se resident-focused, so staff should continually monitor the impact of the project on the residents. Quality of meals and meal service sets of indicators should contain structural, process, and outcome indicators. This is in line with the indicator set from this study, which contains six structural, four process, and three outcome indicators.

To our knowledge this is the first report on the development of a short and simple quality of meals and meal service set of indicators for residential facilities for the elderly. Other studies only reported the necessity to adapt meals to meet the needs of elderly residents in order to decrease the risk of malnutrition and to increase quality of life, but did not develop quality indicators (11, 12, 43). However, to guide residential facilities in initiating and evaluating meal and meal service quality improvement processes, reliable information which is easy to gather is needed. This information can be obtained by the measurement of quality indicators. Chao et al. (44) developed a comprehensive list of 57 indicators encompassing the domains ‘dining room’, ‘food service’, ‘general nutrition’ and ‘therapeutic nutrition’. Only two criteria from structural indicators (2b: involving residents in compiling the menu; 2c: procedure for systematically inquiring residents about food, service and choice) and two process indicators (7: weight evolution documented; 8: results of malnutrition screening documented) from our indicator set were comparable. The main reason for this finding is that our purpose was to develop a short and simple instrument. Only in case of room for improvement do we recommend more detailed assessment. The information needed to measure most of Chao’s indicators can, according to our assessment, best be gathered by a meal and meal service satisfaction survey. Our indicator set gives an easy to gather indication of the quality of meals and meal service. A validated meal satisfaction survey encompassing at least the domains of food, service, and choice can be used to gain a deeper insight in some of the results, for example ‘The prevalence of malnourished residents’ (see also indicator 13).

A meal and meal service quality improvement process should be multidisciplinary. In this study, both kitchen staff and health care professionals were involved in selection and refinement of the indicators and criteria and so, a multidisciplinary vision on meal quality was established. The ‘quality of meals and meal service’ set of indicators could encourage the initiation and evaluation of quality improvement strategies as a multidisciplinary responsibility.

The strength of this study is that a broad range of multidisciplinary experts from various fields in Belgium and the Netherlands were involved in the development of the quality indicators. However this study has some limitations. The ‘quality of meals and meal service’ set of indicators was developed according to the Indicator Development Manual of the Dutch Institute of Health Care Improvement (17). This manual ensured a systematic approach, using literature and an expert panel, to compose the indicator set. Nevertheless the manual was restricted to the phase of indicator development and validation. Our validated indicator set has not been tested for feasibility. Feasibility of data collection could be a problem for the indicators concerning malnutrition screening, because validated screening instruments for malnutrition are limited to screening for undernutrition. However, the prevalence of overweight and obesity in nursing homes is expected to rise in line with the increasing prevalence in the community at all ages (45). Furthermore the indicator: ‘The amount of residents per meal assistant, who need help with the principal meal’ has to be further elaborated in discussion with other interested parties. No guidelines were found to estimate the true need for meal assistance in a long term care facility which limited our development of this indicator. Nevertheless this important indicator is gaining attention (38). It is reasonable to accept that the amount of meal assistance needed in a facility depends on the global degree of dependence of the residents.

Another limitation was that almost half of the experts from the first Delphi round did not participate in the second round. These experts agreed with the adjustments made at the end of the first Delphi round and had no further suggestions. Subsequently the feedback from the other experts was incorporated.

Future research should address the feasibility of assessing the ‘quality of meals and meal service’ set of indicators in practice. In addition, other interested parties (e.g. professional organizations, patient organizations) should be consulted in order to fine-tune the indicator set and to improve its applicability and acceptability. Finally, strategies could be developed and tested to implement the indicator set in longterm care facilities for elderly.

Conclusion

The ‘quality of meals and meal service’ set of indicators is a resource to map meal quality in residential facilities for elderly. As soon as feasibility tests in practice are successfully completed, the set can be used to guide meal and meal service quality improvement projects developed in close collaboration with kitchen staff and health care professionals. These improvement projects are necessary to improve food intake and reduce the risk of malnutrition among elderly in residential facilities.

Conflicts of interest statement: Authors 1-4 have no conflicts of interest known. Author 5 received a grant form Unilever Food Solutions. Authors 6 received a grant from Center for Gastrology.