Introduction

Almost all dietary intake measurements are self-reported and therefore prone to distortion by subjects inaccurately or incompletely reporting their diets. On the basis of the fundamental principles of energy balance, it is now generally accepted that under-reporting, or misreporting, of food intake is widespread if not universal.1, 2, 3, 4 Many subjects in diet surveys misreport their food intake to an extent that may distort the relationships between diet and health that inform policy decisions (for example, Rennie et al.5).

Aside from technical errors in the recording of food intake (such as inadequate descriptions of foods, accuracy of food weighing scales or unclear instructions given to participants), the misreporting of food intake can be considered as having two components. Firstly, participants choose different foods from normal when they are aware that their diet is being monitored (the observation effect), either to report a diet that they believe is closer to the recommended or for convenience as some foods and meals are simply easier to weigh than others.6 Secondly, participants fail to record all of the foods that they actually consume, either consciously or accidently (the recording effect).7

It is usually assumed that misreporting of food intake is biased more towards reporting lower rather than higher energy intakes, and there is indirect evidence to support this when reported energy intakes are compared against energy expenditure (see below). More direct evidence is harder to find, although weight-stable obese subjects under-reported energy intake from a buffet meal, whereas normal weight subjects accurately reported intakes.8 Perhaps unsurprisingly weight-restored patients with anorexia nervosa over-reported energy intake in the same study.8 When a measure of the true food intake is available for periods of a day or 2 weeks, group average reported energy intakes are lower than actual energy intakes, and most individuals under-report their food intake, although a small number do over-report.7, 9

When direct observation of food intake is not possible, the most widely used methods of identifying individuals suspected of reporting low-energy intakes are the Goldberg cut-off method and by comparison with energy expenditure through indirect calorimetry, namely, the doubly labelled water technique.10 A major problem is that these methods rely on measures of energy expenditure that are imperfect, or estimates of energy expenditure based on assumptions about levels of physical activity and regression equations to estimate basal metabolic rate (BMR). The Goldberg cut-off method aims, statistically, to identify subjects who report implausibly low-energy intake to BMR ratios either for long-term habitual intake (cut-off 1) or for intake over the measurement period (cut-off 2).11 The cut-off values are based on the assumption that subjects are in energy balance and that their energy requirements have been accurately estimated, with the cut-off value being adjusted to account for the uncertainty in estimating BMR and the duration of the diet-recording period. Predicting BMR can be difficult, especially so in the obese as common regression methods overestimate BMR at higher body weights,12 and assumptions have to be made about physical activity levels. Subsequent recommendations were made that measurements or estimates of individual physical activity levels are necessary.13 In addition, higher reported intakes may also be affected by misreporting and higher intakes are more likely in those with higher activity levels. Furthermore, most subjects tend to be in a negative energy balance (as estimated by change in body weight) when completing food intake records.14, 15, 16

The use of energy intake to BMR ratios to identify low reported energy intakes has also been compared with that of using biomarkers of diet, the most widely used being the ratio of urinary to dietary nitrogen,17 a method that is also not without its limitations. Thus, self-reported dietary intakes have been compared with indirect measures of energy expenditure (as an indirect measure of energy intake assuming energy balance) or indirect measures of protein intake (as an indirect measure of energy intake). What is missing, and is needed, is a direct, precise and concurrent measure of food intake against which the ability of energy intake to BMR ratios to identify misreporting of energy intake can be tested.

We have previously developed and validated a ‘gold standard’ method of measuring food intake, and used it to quantify the nature and extent of misreporting of diet in the laboratory, albeit under conditions that were as close to free-living as practicable, that is, in a residential metabolic facility.7 This gold standard method, the laboratory weight intake (LWI), allows a direct comparison to be made between food intake reported by subjects and their actual food intake. The current study aimed to assess the validity of self-reported weighed food intake records completed in a laboratory setting and that would be considered plausible using the criterion of reported energy intake to BMR ratios. Effects of recording food intake under more usual, real world, diet survey conditions on reported energy intake were then considered.

Methods and materials

Study design

Subjects

Sample size of n=180 was calculated to allow detection of differences in psychological measures (not reported here) between valid and under-reporters of energy intake. On the basis of values from a previous study,7 a sample size of 180 would allow detection of a 7% difference between true and reported energy intake with a power of 80% and a two-tailed alpha of 0.05. One hundred and eighty, apparently healthy, men and women were recruited from the Aberdeen area. The real purpose of the study was, necessarily, not explained to the subjects and they were informed that it was to examine the relationships between diet and lifestyle.

Recruitment and ethics

Prospective volunteers were invited to the Human Nutrition Unit (HNU) of the Rowett Institute of Nutrition and Health where all procedures involved in the study, and any discomfort or risk they may have posed, were explained. This study was conducted according to the guidelines laid down in the Declaration of Helsinki and all procedures involving human subjects were approved by the Joint Ethical Committee of the Grampian Health Board and the University of Aberdeen. Written informed consent was obtained from all subjects.

Protocol

Each subject was studied using a randomised cross-over design for two consecutive days in the laboratory and four consecutive days in their natural environment (home). The days of the week on which subjects completed the measurements was balanced between the laboratory and home phases. All randomisation was achieved using a computer-generated randomisation list.

Laboratory phase

Subjects each completed a 1-day maintenance period (at home) during which they were provided with a fixed diet designed to maintain energy balance estimated at 1.6 and 1.5 times BMR for men and women, respectively. For the following 2 days (1 week-day and 1 weekend-day, randomised to Friday and Saturday or Sunday and Monday), subjects were resident at the HNU where food intake was covertly quantified on a daily basis by the investigators, using a previously described LWI method.7

Each subject was provided with an individual larder and had ad libitum access to variety of familiar foods, and food intake was continuously and covertly monitored and quantified by trained staff. All food items were weighed by research staff before they were placed into each subject’s personal larder. Each subject received bottled water for drinking, and their own individual kettle, to allow an estimate of water consumption. Full verbal and written instructions regarding the kitchens including information on waste and packaging and use of kettles and water were given to each subject. Subjects were instructed not to throw any waste away including packaging of food items and peelings, and uneaten food from meals. Every kitchen contained a special bin for all waste and packaging, with all waste items being individually wrapped. Subjects were also instructed not to wash any dishes.

An investigator entered the kitchen each morning before the subject awoke and re-weighed all food items, any leftovers, including peelings, and packaging found in the subjects’ individual bins. This enabled accurate estimates of 24-h food intake to be calculated. Subjects were unaware of this procedure, and this constituted the ‘gold standard’ against which self-reported food intakes were compared.7 Each subject was asked to weigh and record all food items eaten and all fluids drunk using the weighed dietary record (WDR) method.18 Full written and verbal information on how to carry this out was given at the beginning of the study.

Thus, the LWI was investigator-measured actual food intake and the WDR was food intake as self-reported by subjects during the residential stay in the laboratory (WDR-L). The difference between the LWI and WDR-L was therefore the reporting effect (the difference between what subjects actually ate and reported eating). The observation effect (change in diet) as a result of the subject being aware that their diet was being evaluated was not measured and would have been an additional source of misreporting error.7

Home phase

The 5-day home study consisted of a 1-day maintenance, with the same maintenance diet as during the laboratory phase, and 2 weekdays and 2 weekend days (randomised to Thursday–Sunday or Saturday–Tuesday) within the subject’s natural environment (that is, at home). Subjects were asked to complete a 4-day WDR (WDR-H) on days 2–5 using the same procedure as during the laboratory phase.

Dietary analysis

Dietary data for all methods were analysed using Diet 5 (Robert Gordon University, Aberdeen), a computerised version of McCance and Widdowson composition of foods and supplements.19

Basal metabolic rate

Respiratory exchange was measured using a ventilated hood system (Deltatrac II, MBM-200, Datex Instrumentarium Corporation, Helsinki, Finland) under standardised conditions in subjects who were fasted for 12 h from the previous night. BMR was calculated using the equations of Elia and Livesy.20

Anthropometry

Body weight was measured on each morning of the study when subjects were resident in the HNU, and at the start and end of the WDR-H period when subjects were at home, using a digital platform scale (DIGI DS-410 CMS Weighing Equipment, London, UK) to the nearest 0.01 kg after voiding and before eating. Subjects were weighed in dressing gowns of a known weight and body weight was then corrected back to nude.

Height was measured to the nearest 0.5 cm before subjects started the study using a portable stadiometer (Holtain Ltd., Crymych, Dyfed, Wales).

Statistics

The cut-off value for weighed intake records and measured BMR was calculated as 1.03*BMR for the 2-day WDR-L and 1.10*BMR for the 4-day WDR-H following the method of Goldberg et al.11 All analyses were performed using Statistical Package of Social Sciences software (SPSS Inc., Chicago, IL, USA; Version 21.0.0.1). Normality of data was evaluated using Shapiro–Wilk’s test and histograms; equality of variances was assessed using Levene’s test. Two-sided t-tests were used for comparison of the reporting effect (WDR-L—LWI) between groups of male and female, and lean and overweight subjects. Pearson’s correlations were used to assess the strength of the relationship between energy intake and energy requirements. Differences were accepted as statistically different at the 5% level.

Results

Table 1 gives the age, height, weight, body mass index (BMI) and BMR of the subjects. Mean daily absolute energy intakes, and energy intake relative to BMR from subjects’ self-reported food intakes (WDR-L) were significantly lower than those from the LWI (Table 2). Both actual (LWI) and reported energy intakes (WDR-L) were positively correlated with BMR (r=0.487, P<0.001 and r=0.516, P<0.001, respectively).

Table 1 Baseline characteristics of participants by sex, age and BMI group (mean values with their standard deviations)
Full size table
Table 2 Average daily energy intake and energy intake relative to BMR over the WDR-L and WDR-H measurement periods
Full size table

The reporting effect (WDR-L—LWI) was significantly greater in males than it was in females (P=0.025). There was no significant difference in the reporting effect between lean (BMI⩽25 kg/m2) and overweight (BMI>25 kg/m2) subjects (P=0.539).

Six subjects (3.3%) reported energy intakes that were below the Goldberg cut-off value of 1.03*BMR. Of these, five had actual energy intake that were less than 1.03*BMR. Mean change in body weight over the 2 days was significantly different from zero for males (+0.21 kg, P=0.001) and all subjects combined (+0.09 kg, P=0.025), but not for females (−0.02 kg, not significant).

Figure 1 shows the difference in mean daily energy intake calculated from each subjects’ self-reported food intake and that calculated from the investigator-measured intake (WDR-L—LWI). Values less than zero show those subjects who under-reported their food intake (67% of subjects), and values greater than zero show those subjects who over-reported their food intake (33% of subjects). The appropriate cut-off value (1.03*BMR) is shown by the vertical line, values to the left of this line would be considered as implausible measures of the food consumed over the 2-day recording period, whereas values to the right would be considered as acceptable. The same data are presented in Figure 2 but with the WDR-L expressed as a percentage of the LWI for each subject.

Figure 1
figure 1

Difference in mean daily energy intake calculated from each subjects’ self-reported food intake and that calculated from the investigator measured intake (WDR-L—LWI) against estimated energy requirements. Section A, subjects identified as low-energy reporters by the Goldberg method, but with valid/over reports of energy intake. Section B, subjects identified as acceptable reporters by the Goldberg method, and with valid/over reports of energy intake. Section C, subjects identified as low-energy reporters by the Goldberg method, and under-reported energy intake. Section D, subjects identified as acceptable reporters by the Goldberg method, but with under-reported energy intake. LWI, laboratory weighed intake; WDR-L, weighed dietary record - laboratory.

Full size image
Figure 2
figure 2

Reporting effect against estimated energy requirements. Section A, subjects identified as low-energy reporters by the Goldberg method, but with valid/over reports of energy intake. Section B, subjects identified as acceptable reporters by the Goldberg method, and with valid/over reports of energy intake. Section C, subjects identified as low-energy reporters by the Goldberg method, and under-reported energy intake. Section D, subjects identified as acceptable reporters by the Goldberg method, but with under reported energy intake. LWI, laboratory weighed intake; WDR-L, weighed dietary record - laboratory.

Full size image

Self-reported energy intakes during the home phase (WDR-H) were significantly lower than the WDR-L energy intakes (Table 2). Few people (20 or 11%) who reported implausible energy intakes (<1.10*BMR) during the home phase of the study had also reported energy intakes that were less than the LWI during the laboratory phase (Figure 3). Almost half (101 or 56%) of the participants who under-reported energy intake in the laboratory reported plausible levels of energy intake at home.

Figure 3
figure 3

Reported energy intake during the home phase of the study relative to BMR against reported energy intake relative to actual energy intake during the residential phase of the study. LWI, laboratory weighed intake; WDR-L, weighed dietary record - laboratory.

Full size image

Mean change in body weight over the 4-day WDR-H period was similar to the WDR-L period with males gaining a small, and borderline statistically significant, amount of weight (+0.14 kg, P=0.057). Change in weight for females and all subjects combined was not significantly different from zero (−0.08 kg and +0.03 kg, respectively).

Discussion

This study explored whether plausible reports of energy intake, as determined by energy intake to BMR ratios, are always valid and accurate under residential laboratory conditions. Low reported energy intakes—those that would normally be considered implausible—can be valid, and of greater concern is that the majority of plausible food intake records are under- or over-reported to varying degrees. It is not simply a case of too lenient a cut-off value. Increasing it does not solve the problem of misreporting, which is a continuous trait that is not easily accounted for by categorical cut-offs. Misreporting of food intake under free-living conditions appears to be greater than that in the laboratory.

In a prior study, when a different group of subjects recorded their food intake, they changed their diet such that energy intake decreased by 5.3% (the observation effect), the difference between what they ate and what they reported was a further decrease in energy intake of 5.1% (the reporting effect).7 In the current study, the reporting effect was a similar 3.8% of actual energy intake.

The prevalence of low-energy reporting as determined using an energy intake to BMR cut-off value was only 5% in our previous study and 3% in the current study (and 18% when subjects were at home), considerably lower than the average of 33% (range 14–39%) reported by Poslusna et al.10 in a review of misreporting of energy intakes, and when considering weighed food records. It appears, therefore, that subjects in both studies, reported more complete food records, or at least higher energy intakes, than is typical during free-living studies. It is quite possible that the residential nature of the study, with fewer of the usual day-to-day distractions, increased the completeness of food recording. It is also likely that subjects were in positive energy balance over the 2-day residential stay as the nature of the protocol meant that subjects were sedentary, whereas the average observed energy intake was 1.82*BMR. This is higher than the estimated physical activity level of 1.78*BMR of groups judged to be more active than average.21 This is supported by the small, but statistically significant, average change in body weight, although using change in body weight as an estimate of change in energy balance over such a short period is only an approximation. Therefore, reported energy intakes were more likely to be above the misreporting cut-off than would be expected, as any misreporting was from a level that was probably higher than habitual. Even when low-energy reporting was much less than usual, there was still a large discrepancy between the numbers of people identified as reporting implausible levels of energy intakes and actually misreporting food intake.

Under-reporting, and even over-reporting, were evident in both plausible and implausible food records, not just below or near the low-energy reporting cut-off value. Under-reporting of 12 MJ/d was seen in one subject with a reported energy intake of almost 3*BMR (subject X in Figure 1). In contrast, another subject accurately reported an energy intake that was less than half of BMR (subject Y in Figure 1).

Most studies report an association between BMI and misreporting; subjects with higher BMIs being more likely to be classified as low-energy reporters, or a positive correlation between BMI and the difference between energy intake calculated from reported food intake and either estimated energy requirements or measured energy expenditure.10 An effect of BMI on the degree of misreporting was not apparent in the current study, or our previous study.7 The few studies that have used a covertly measured food intake as the reference have shown mixed results—either no effect of BMI on the degree of misreporting,9, 22 that obese subjects are more accurate in reporting their food intake than are overweight or lean subjects,23 or less accurate.8 Most of these studies have used diet recalls completed after the covert food intake measurement rather than concurrent measures thereby introducing a further source of uncertainty into the dietary intake method because the recall method relies on the ability and motivation of subjects to remember what was eaten. The difference in the apparent effect of BMI on the degree of misreporting when using estimated energy requirements compared with actual food intake may reflect a difficulty in estimating energy requirements in individuals with higher BMIs. BMR is often estimated using well-established linear regression equations.24, 25 These equations tend to overestimate BMR at higher body weights because the increase in BMR with body weight is curvilinear. Increases in metabolically active fat-free mass and metabolically less-active fat mass do not occur at a linear rate as body weight increases.12 Overestimating BMR will lower the ratio of reported energy intake to BMR and result in subjects with higher BMIs being more likely to be identified as low-energy reporters than are lean subjects. Additionally, the Schofield equations underestimate BMR at lower body weights25 resulting in leaner subjects being more likely to have reported energy intake to BMR ratios within the plausible range. However, the overweight and obese are still more likely to be classified as low-energy reporters than are the ‘normal’ weight after accounting for differences in body composition by estimating BMR from estimated fat-free mass.26 Therefore, the difference in prevalence of misreporting between the lean and overweight may still exist, but might not be as great as is generally reported.

It has been argued that removing subjects who report implausibly low-energy intakes introduces bias into any analyses,10 because subjects with higher energy requirements are also likely to under-report their food intake. The current study supports this.

Reported energy intakes were lower over the home phase than the residential phase, possibly because the residential environment of the HNU encouraged more complete food records, or the home environment hindered record keeping—or both. It is also possible that subjects altered their behaviour when in the HNU, which resulted in higher than habitual energy intakes. Food and drink were provided free to the subjects, and they probably had more time to prepare and eat meals than they would have had at home.

That so few subjects reported low-energy intakes during both the home and laboratory phases suggests that people cannot be classified as consistently plausible reporters or consistently implausible reporters. Furthermore, misreporting of food intake is continuous and is not resolved with categorical cut-offs. Plausible records that are invalid present difficulties for intervention and epidemiological studies, to the extent that some have argued that reliance on self-reported dietary intakes should be discontinued.27

Limitations

The results of this study, and therefore the conclusions drawn from it, are subject to a number of limitations. Actual, and reported, energy intakes were higher during the laboratory phase than would be expected for sedentary subjects, and it is likely that the cut-off value would have identified more subjects with low reported energy intakes had subjects been studied in their natural environment. This would, however, have precluded an accurate measure of true food intake. A lack of a covert and objective measure of food intake during the home phase of the study is an unavoidable limitation.

In the present study, energy expenditure was not measured during the time that subjects were completing the food records. However, energy intake when subjects were resident in the HNU was measured under identical conditions to a previous study where measured energy intake matched measured energy expenditure.7

Summary

Comparing reported energy intakes to estimates of energy expenditure has become an established method to identify implausible food intake records. We have previously shown that low-energy reporting, when compared with the gold standard LWI method, occurs at all levels of energy turn-over.7 In this study, we demonstrated that misreporting occurs at all levels of energy intake and found that the many plausible records of energy intake were inaccurate to variable degrees. The method of using energy intake to BMR ratios probably introduces bias by only excluding misreporters with low reported energy intakes and retaining misreporters with higher reported energy intakes. It may also have given researchers, and readers of the literature, a false confidence in the completeness of dietary data.