Introduction

Overweight or obesity now occurs in three of four adults in the USA [1], which is the nation’s highest combined prevalence to date. Nutrition is a major underlying factor in the development and treatment of obesity as well as its many comorbidities [2]. A healthy lifestyle including a healthy diet, physical activity, and maintaining a healthy weight is also estimated to prevent 34 % of cancers in the USA [3]. Certainly, the increasing prevalence of overweight and obesity is complex in etiology, but these numbers suggest inadequacy in knowledge, motivation, and/or resources among this large percentage of the population. For many, these inadequacies may be rooted in poor health literacy with regard to food and nutrition. While research has elucidated components of health literacy and consequences of inadequate health literacy, little attention has been given to nutrition literacy, despite the clear importance of a healthy diet for preventing and treating chronic disease.

Silk and colleagues proposed that “nutrition literacy” should be defined as “the degree to which individuals have the capacity to obtain, process, and understand nutrition information and skills needed in order to make appropriate nutrition decisions” [4]. Our previous research identified specific domains of nutrition literacy that represent both conceptual nutrition knowledge and functional capabilities [5]. These concepts were then incorporated into the original design of the Nutrition Literacy Assessment Instrument (NLit), which was subsequently pilot tested for content validity and usability in nutrition consultation settings with registered dietitians and critiqued by a separate sample of registered dietitians online [6].

The NLit combines measures of print literacy and numeracy, consistent with measures of health literacy, while also including measures of nutrition knowledge and skills needed for following a healthy diet. Domains of nutrition literacy represented in the NLit include an appreciation of the influence of nutrition on health, understanding of the macronutrient (carbohydrate, fat, and protein) content of foods, skill with household food measurements, ability to read the Nutrition Facts Panel of a food label, and the ability to place foods into functional categories. No other health literacy instrument addresses all of these domains critical to nutrition literacy.

Nutrition is a critical lifestyle factor for the prevention of cancer [7], for optimizing weight to prevent breast cancer recurrence and mortality [8], and for improving quality of life in breast cancer survival [9]. Nutrition is also necessary for prevention and treatment of other chronic diseases associated with breast cancer survival, such as cardiovascular disease, diabetes, and metabolic syndrome [10]. Yet uncertainty related to dietary choices and body weight increases distress in female cancer survivors and may reduce quality of life [11].

Diets that provide caloric intakes supportive of ideal body weight, low intakes of processed and red meats, avoidance of alcohol, and high intakes of fruits, vegetables, and whole grains are important for cancer prevention and survivorship [7, 8, 12]. However, the pervasive Western diet does not demonstrate these dietary choices [12]. Complicating consumer understanding of nutrition is the ubiquitous bombardment of nutrition information that is often unreliable. Thus, improving nutrition literacy may be a critical target for improving overall health and quality of life for the breast cancer population.

The purposes of this pilot study were to revise the Nutrition Literacy Assessment Instrument (NLit) [6] for use in primary and secondary breast cancer prevention populations and to estimate its validity and reliability. The adapted instrument was developed to be content valid, reliable, and clearly understood by the high-risk and breast cancer survivor population. We examined construct validity by investigating associations with diet quality and education, with the hypothesis that those with poor diet quality and lower education would score lower. A secondary aim was to evaluate the instrument’s sensitivity to intervention effects.

Methods

Study Overview

We first engaged experts in cancer nutrition and breast cancer survivors to inform instrument design prior to pilot testing. Content experts reviewed and rated each item of the instrument, and suggested edits were made. The modified instrument was then reviewed by breast cancer survivors using a cognitive interviewing approach. The instrument was then tested for reliability and construct validity in samples of two populations: (1) breast cancer survivors concurrently enrolled in a weight loss intervention study and (2) non-intervention participants: subdivided into breast cancer survivors and women at high risk for breast cancer. The university’s institutional review board deemed the study exempt, and all data was collected between August 2013 and September 2014.

Draft Instrument: Nutrition Literacy Assessment Instrument for Breast Cancer

The original 40-item version of the NLit represented five domains of nutrition literacy as described above. This version was modified to incorporate concepts from the American Cancer Society’s Diet & Cancer Prevention guidelines [7] and adding a sixth original domain that seeks to measure consumer food shopping skills. Four sections of the NLit were expanded to twice their original item pool, to better ensure internal consistency in the final instrument, resulting in a 75-item instrument that represented six domains of nutrition literacy for breast cancer patients.

Scale Content Validity

Content experts (n = 3) were recruited based upon their published expertise in cancer nutrition and were compensated for their completed review. Experts ranked each item in the expanded pool using a 4-point scale for relevance to the content domain, clarity, and whether or not (yes/no) the item should be deleted. Item rankings of relevance were transformed into a mean content validity index, which was then compiled to form a scale-content validity index (S-CVI) for each domain using methods outlined by Polit and Beck [13].

Content experts recommended deletion of 10 items and suggested modifications to 21 items and instructions for 2 domains. S-CVI for the remaining 64 items confirmed content validity with 0.93 S-CVI for the instrument overall.

Cognitive Interviewing

The remaining item pool was evaluated through cognitive interviewing of 18 breast cancer survivors recruited as a convenience sample of an on-going weight loss intervention trial. After consent, participants were given the instrument to read aloud in short segments, after which they were asked to “think aloud” about their thoughts, feelings, and ideas for each instrument item and response option using verbal probing techniques. Participants were compensated $25 for their interviews.

An iterative approach was used so that insight gained from each participant was incorporated into subsequent interviews. Interviews were conducted by two trained research staff and were evaluated by researchers separately using content analysis, a qualitative data analysis process for identifying themes in the responses. After individual review, researchers met to resolve any differences in the interpretation of comments.

Through cognitive interviewing, survivors helped to modify the instrument’s language in the instructions of two domains and five items. For example, although our previous survey of experts debated the use of “portions” instead of “servings” in the household food measurement domain, most participants indicated these terms are synonymous to them. This finding underscores the importance of investigating the word choice within instruments with the intended population because of the potential for differing interpretations. As a result of this finding, the word “portion” was kept in the instrument.

Pilot Testing

Two groups of participants were recruited to estimate construct validity and reliability. Twenty-five of 31 women (80.6 % of those invited) participating in one cohort of an on-going weight loss intervention in the rural Midwest were recruited as a convenience sample for this study [14]. This population was of interest in order to evaluate the instrument’s sensitivity to detecting intervention effects. The completed study measures were taken at baseline prior to the start of the intervention and again at 6 months. Eligible participants for this group were post-menopausal female breast cancer survivors, BMI of 27–45 kg/m2, age ≤ 75 years old, diagnosed with stage 0–IIIc (except stage 0 with mastectomy only) disease within the past 10 years, completed all local and systemic therapy at least 3 months prior to entry, obtained clearance from their oncologist or current medical provider to participate in a weight control study, resided in a rural area according to the Rural-Urban Commuting Area Codes, and were able to walk briskly unassisted and without serious medical risk. Subjects were ineligible if they did not speak and read in English, demonstrated overt psychiatric illness, visual acuity insufficient to read the testing instrument, or had a cognitive impairment. All participants provided informed written consent and received $10 for completing the additional measures associated with this study.

For the non-intervention group, a convenience sample of 30 survivors and 17 women at high breast cancer risk were recruited from Midwestern metropolitan breast cancer prevention and survivorship clinics via flier or from a patient registry by phone. This group received no nutrition education during study participation. Eligible participants were either female breast cancer patients >21 years who had been diagnosed with breast cancer and had completed all local and systemic therapy at least 3 months prior or were at risk for breast cancer, defined as genetic breast cancer susceptibility, family history of breast cancer in one primary relative or multiple secondary relatives, a prior biopsy, and/or high breast density. Individuals were ineligible if they demonstrated overt psychiatric illness, visual acuity insufficient to read the testing instrument, or cognitive impairment. Participants were compensated $25 for two visits separated by 4 weeks.

Primary Outcomes of the Pilot Study: Reliability and Validity

We estimated the Nutrition Literacy Assessment Instrument for Breast Cancer’s (NLit-BCa’s) reliability both as internal consistency using confirmatory factor analysis and 4-week test-retest reliability. Because a weight-loss intervention involves diet education, we evaluated test-retest in the non-intervention group only.

We estimated the NLit-BCa’s construct validity in two ways, by (1) confirmatory factor analysis and (2) convergent validity. It was hypothesized that diet quality and educational level would be correlated with nutrition literacy such that those with poorer diet quality and lower education would demonstrate lower nutrition literacy. We also examined changes in the NLit-BCa scores and their correlations with weight loss and change in diet quality subsequent to a successful weight loss intervention.

Pilot Testing Measures

After consent, participants completed a 10-item demographic online survey via REDCap Software (Version 5.7.7) and the 64-item NLit-BCa (pencil/paper format), and provided two 24-h dietary recalls using the USDA multiple-pass method [15]. Non-intervention participants returned for a second visit at a 4-week interval for a re-test of the NLit-BCa and to provide a second 24-h recall. Two diet recalls were necessary to evaluate diet quality as described below. Participants in the intervention group were retested at their 6-month study appointment.

Diet Quality

There is no reference measurement for nutrition literacy, and health literacy measurements differ in construct to nutrition literacy. Thus, we chose to use diet quality, as measured by the Healthy Eating Index (HEI-2010), as our validation standard. The Nutrition Data System for Research (NDSR version 2013) software was used to calculate total energy, food group, and macronutrient intake for two 24-h dietary recalls. NDSR data were used to calculate HEI-2010 scores based on a method previously developed by Miller et al. [16]. Participants who completed only one 24-h dietary recall (n = 2 in the intervention group and n = 4 in the non-intervention group) were included in the analysis, and HEI-2010 scores for these individuals reflect 1 day of intake. Higher HEI-2010 scores are indicative of better diet quality.

Data Analysis

The relationship of constructs via subscales of NLit-BCa and its respective items were analyzed by item response theory via binary confirmatory factor analysis (CFA). Binary CFA is a generalization of Rasch models [17]. The binary CFA analysis was conducted using the Lavaan package from R2.15.3. When fitting the model for each subscale, we used a one-factor model and treated the response of each item as a binary variable. Missing data were coded as incorrect. The model fit was evaluated by two statistical fit indexes: comparative fit index (CFI > 0.90) and root mean square error of approximation (RMSEA < 0.06) [18]. The composite reliability was estimated with the output obtained by binary confirmatory factor analysis. According to Shrout’s adjectives, the interpretation of reliability is 0.00–0.10 as virtually none, 0.11–0.40 as slight, 0.41–0.60 as fair, 0.61–0.80 as moderate, and 0.81–1.0 as substantial reliability [19].

In the non-intervention group only, test-retest reliability was evaluated using Pearson’s correlation coefficient with interpretation of reliability as described by Shrout’s adjectives above [19].

For the group that participated in a separate weight-loss intervention study and the present study, we explored interventional effects by conducting paired t tests to evaluate for significant difference between baseline and 6-month follow-up nutrition literacy scores. We also evaluated Pearson’s correlation to test the relationship between change in nutrition literacy and percent weight loss at 6-month follow-up.

Six general linear models (GLM) were built to test the relationship between diet quality and nutrition literacy and subsequently, controlling for education and race. Responses for education were collapsed into two categories—“below college” and “college and above.” There were two categories for race—“African American” and “White.” The dependent variable is diet quality (HEI) and the independent variables include six NLit-BCa domains (Nutrition & Health, Macronutrients, Household Food Measurement, Food Label & Numeracy, Food Groups, and Consumer Skills), education, and race. Significance was set at P < 0.05.

Results

The 71 participants in the study were primarily white (75 %) and had an average age of 60 years, and 46 % had a bachelor’s degree (Table 1).

Table 1 Demographic characteristics of study participants (n = 71)
Full size table

Test-retest reliability ranged between fair and substantial for all domains. Entire reliability was substantial (>0.80) for three domains (Food Label & Numeracy, Food Groups, and Consumer Skills) while the remaining three domains were reliable or approached acceptable reliability as measured by root mean square of approximation (≤0.06 is acceptable), including Nutrition & Health, Macronutrients, and Food Portions (Table 2). All data reported reflects inclusion of all items and does not reflect removal of non-discriminating items.

Table 2 Reliability statistics by domain
Full size table

General linear modeling of the relationships between nutrition literacy (as measured by the NLit-BCa) and diet quality (as measured by HEI-2010) demonstrates a significant positive relationship (P < 0.05) between five domains of the NLit-BCa and HEI-2010, including Macronutrients, Household Food Measurement, Food Label and Numeracy, Food Groups, and Consumer Skills. This relationship remains significant (P < 0.05) for three domains with education and race included in the model, including Food Label and Numeracy, Food Groups, and Consumer Skills (Table 3).

Table 3 Strength of relationships between domains of the instrument and Healthy Eating Index-2010 (estimate and P value)
Full size table

Seventeen participants in the weight loss intervention group completed all baseline and 6-month follow-up measures for the present study. In this group, weight loss at 6 months averaged 12.3 % (±6.11 %), which was significantly different from baseline (P < 0.001). HEI-2010 also improved an average of 7.5 points (±17.5 points), and this approached statistical significance (P = 0.096). Mean scores on the NLit-BCa were improved for all domains after 6 months of intervention except Food Label & Numeracy; however, these improvements were only significant for the Household Food Measurement section (mean difference = 0.71; P = 0.047). No significant relationships were seen between change in NLit-BCa scores and percent weight change at 6 months.

Discussion

A strength of this study is our novel use of diet quality as a standard of comparison for nutrition literacy rather than using health literacy measurement tools. Nutrition education efforts generally target improvements in diet quality as an intermediary step to improving measures of health, such as anthropometrics (i.e., body mass index, body fat, lean muscle mass, etc.) and biomarkers (i.e., insulin resistance, A1c, LDL-cholesterol, etc.). Therefore, a nutrition literacy tool is most useful if it can predict increasing diet quality as nutrition literacy increases. Otherwise, there is little practical value for assessing nutrition literacy at the outset of nutrition education endeavors.

In this case, all NLit-BCa domains with the exception of Nutrition & Health were significantly related to diet quality, and this relationship persisted after controlling for education and race in three domains. This finding is consistent with the body of literature investigating health outcomes in low health literate populations, where low health literacy appears to be a mediator of disparities and poor health outcomes [20, 21]. A potential explanation for the lack of relationship between Nutrition & Health and diet quality is that because this domain is modeled after the TOFHLA [22], asking questions about a one-page text describing the American Cancer Society dietary recommendations [7], its construct may be more related to reading comprehension than dietary knowledge or skills. Reading comprehension, while foundational to literacy and the collection of knowledge, may not be indicative of dietary action. The other domains included in this instrument may be skills and knowledge sets more proximal to the implementation of nutrition guidelines.

It is not surprising that the weight loss intervention significantly improved nutrition literacy with regard to Household Food Measurement. Dietary interventions aimed at weight loss require calorie reduction, which involves portion control through weighing and measuring of food [23]. While the intervention did involve some education that relates to other sections of the instrument, the intervention was not designed to improve nutrition literacy per se but for weight loss, and these other skills may not be as necessary for weight loss as is skill with portion sizing. However, the small sample size precludes strong conclusions regarding relationships between nutrition literacy and weight loss, and this should be explored in future research.

While our sample demonstrated a range of diet quality, mean HEI-2010 was 63.3 compared with 52.7 in women of the reference 2003–2004 National Health and Nutrition Examination Survey population in which the HEI-2010 was validated [24]. Thus, the lower spectrum of diet quality was not as well represented in our sample, which may reflect a greater inclination of those interested in nutrition to participate in nutrition research. This is an important consideration for future recruitment efforts because greater interest in nutrition may be a factor in nutrition literacy attainment, given that greater information seeking efforts are demonstrated in more health literate cancer populations [25]. Additionally, although we controlled for education and race, the majority of our sample was white, educated women. Thus, results are not generalizable and validation in other populations that deviate from this sample is recommended.

Conclusion

The NLit-BCa demonstrates content validity and was interpreted correctly after modification in our sample of breast cancer survivors. The instrument also demonstrates varying degrees of test-retest reliability, and entire reliability in three of six domains. A high ceiling effect on the first instrument administration may explain lower correlations on test-retest reliability. The remaining three domains meet the RMSEA standards for reliability and approached reliability by other standards. Thus, larger sample size in combination with the removal of non-discriminating items is expected to improve the reliability of these domains.

Practice Implications

The results of this study suggest that the NLit-BCa is a tool with potential for comprehensively measuring nutrition literacy in primary and secondary breast cancer prevention populations, which is the first of its kind. While a larger sample is needed to confirm its validation and reliability, such a tool can provide objective basis for determining educational needs related to nutrition and prevention of breast cancer recurrence, can have application in public health programs targeting diet quality in breast cancer, can be used as an outcome measure for efforts targeting improved nutrition literacy in breast cancer populations, and can be used as the basis of research tools for identifying nutrition literacy in breast cancer populations.