Abstract
Background
Scholarly investigation into orthorexia nervosa is relatively new, leading several scholars to develop new measures to adequately assess the concerns thought to underlie this condition. A promising new measure of orthorexia, the Teruel Orthorexia Scale (TOS), was developed using a Spanish sample.
Purpose
This study was the first to investigate the psychometric properties of the English version of the TOS in a U.S. sample. This study also examined the long-hypothesized relationship between orthorexia and health anxiety.
Methods
We used exploratory factor analysis (EFA) to evaluate the factor structure of the TOS. We also used correlational methods to test theorized relationships with related concerns.
Results
ESEM supported a 16-item, 2-factor model. Correlations between the modified TOS (i.e., TOS-16) subscales and measures of obsessive–compulsive and eating disorder symptoms, perfectionism, emotional distress, and body image disturbance demonstrated convergent and discriminant validity of the TOS-16. There was a moderate positive correlation between orthorexia nervosa and health anxiety and, together, pursuit of healthy eating and preoccupation with healthy eating accounted for a significant amount of variance in health anxiety after controlling for eating disorder symptoms.
Conclusion
Overall, this research suggests that the TOS-16 demonstrates psychometric properties as good as or better than other measures commonly used to assess the construct in a U.S. sample. In addition, findings support the consideration of health anxiety as a possible feature of orthorexia nervosa.
Level of evidence
Level V, descriptive research.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Recent research documents the extent to which health-care professionals identify orthorexic eating behavior as a concern within their practice and believe orthorexia should be a distinct diagnosis [1,2,3]. Although not an official diagnosis, authors have proposed criteria for orthorexia nervosa, which generally emphasize obsession and extreme preoccupation with healthy eating, distress resulting from failure to adhere to self-imposed criteria for healthy eating, and resulting impairment in areas of life to include health and psychosocial functioning (for a review see [4]). Bratman [5], the individual who first named the phenomenon of orthorexia, emphasized obsessive striving for dietary purity and feelings of superiority over those who do not espouse the dietary practices. Other suggested features include obsessive–compulsive traits [6], worry about food quality [7], phobic avoidance of “unhealthy” food [4, 8], overvaluing “healthy” dietary practices relative to empirically documented evidence of health benefits [8, 9], unusual concern with health [10], escalating dietary restrictions [11], distress and/or self-harm in response to breaking self-imposed dietary rules [4, 9, 11], intolerance of others’ food beliefs [9], impaired functioning and health [1, 4, 7, 9,10,11], and potential weight loss [10, 11]. Although some include a lack of body image disturbance among the key distinguishing features of orthorexia [4], Dunn and Bratman [11] suggest that positive body image is dependent on complying with the self-imposed rules.
Measurement of orthorexia
One area of the existing literature on orthorexia with particular limitations is measurement of the construct. Some prior measures yielded mixed psychometric findings (e.g., [12,13,14,15,16,17,18]), and the effort to identify strong measures is needed [14, 19,20,21] and ongoing (e.g., a recent revision of the ORTO-15 [22], the most commonly used measure in orthorexia research, illustrates these efforts towards continuous improvement [23]). One orthorexia measure with promising psychometric properties (i.e., αs > 0.80 and test–retest reliabilities > 0.70) is the Teruel Orthorexia Scale (TOS [17]). The TOS contains two subscales assessing healthy eating and negative consequences of the preoccupation with healthy eating—Healthy Orthorexia and Orthorexia Nervosa. Within the development study, the Orthorexia Nervosa factor related to measures of disordered eating, negative affect, obsessive–compulsive symptoms, and perfectionism, whereas the Healthy Orthorexia factor was less strongly related to these indicators of psychopathology and weakly correlated with adaptive characteristics when removing variance associated with unhealthy aspects of the construct. Thus, the latter scale illustrated a residual effect (see [24] for a discussion of residual effects) in that the pattern suggest that interest in healthy eating has the potential to serve as a measure of a protective factor once statistically removing the overlap it shares with disordered orthorexic eating. As such, the authors of the measure subsequently defined the subscale as assessing “healthy interest in diet, healthy behavior with regard to diet, and eating healthily as a part of one’s identity” [25] (p. 2). In a study investigating its psychometric properties [26], a confirmatory factor analysis (CFA) provided modest support for the bidimensional structure with some fit indices clearly supporting the model (e.g., CFI > 0.95) and others offering less support (e.g., RMSEA > 0.06 with several cross-loadings). Recent CFAs with Portuguese [27], Arabic [28], German [29] translations provided mixed support for the two-subscale structure in populations outside of Spain. Although the TOS has been translated from Spanish to English, its psychometric properties have yet to be examined with an English-speaking sample.
Despite the lack of formal diagnostic criteria and the measurement concerns, prevalence studies permeate the literature and yield point prevalence estimates ranging incredibly, from 3.3 [30] to 88.7% [31]. Methodological issues, especially the measurement challenges, render many of these studies unreliable [11]. Thus, the prevalence of orthorexia is quite uncertain and availability of sound measures is critical.
Orthorexia and related disorders
Relationships between orthorexic symptoms and other psychological conditions are also unclear, and disagreement exists as to whether orthorexia nervosa deserves its own classification (e.g., [5]) or constitutes a subtype of another disorder [32, 33]. For example, researchers theorize an overlap between obsessive compulsive disorder (OCD) and orthorexia nervosa in that both can include intrusive thoughts and rituals around food and health, exaggerated concern with impurity, and rules about food [18]. However, unlike with OCD, these experiences are thought to be ego-syntonic in orthorexia nervosa [8, 18, 34]. Similarly, orthorexia nervosa and eating disorders both include restrictive eating habits and the potential for weight loss as well as traits of perfectionism and OCD symptoms [18, 33, 35]; however, the motivation for dietary rules with orthorexia nervosa is thought to be health and purity [9, 18] rather than pursuit of thinness and body image disturbance (though studies with measures of orthorexic eating call this theoretical motivation into question [25, 36]).
Case studies (e.g., [37]) and quantitative research link orthorexic symptoms to somatic disorders [38, 39], and researchers have suggested that hypochondriasis should be a differential diagnosis for orthorexia nervosa [7, 10]. Despite the theorized connection (e.g., [8, 18, 40]), scant research explores the relationship between somatic concerns and orthorexia nervosa.
The articulated features (i.e., perfectionism, distress) of orthorexia nervosa along with the theorized relationships and distinctions between orthorexic symptoms and other disorders suggest that a valid measure of orthorexia nervosa would relate to measures of obsessive–compulsive and eating disorders symptomology, perfectionism, and distress. A valid measure should also produce patterns of relationships that support the distinction between orthorexic symptoms and disordered eating such that body image disturbance is less strongly associated with orthorexic symptoms. In addition, the theorized link between orthorexia nervosa and somatic concerns in the literature suggest greater endorsement of orthorexic symptoms might be found among those with more somatic concerns.
Purpose
Mixed performance of extant measures of orthorexia nervosa limits confidence in conclusions drawn from scholarly work. One purpose of our study is to examine psychometric properties of the TOS, a promising new instrument not yet examined with an English-speaking sample. We also evaluated the theorized, but rarely investigated, relationship between orthorexia and health anxiety, a key component of somatic symptom and illness anxiety disorders.
We expected that an English-speaking U.S. sample would replicate the TOS’s two-factor structure (Hypothesis 1). We expected the TOS subscales to yield good internal consistency (α ≥ 0.80; Hypothesis 2). Given the relationships between other measures of orthorexia nervosa and eating disorder symptoms, often above 0.75 (e.g., [41, 42]), as well as the initial psychometric findings for the TOS [17], we predicted the Orthorexia Nervosa subscale would yield a strong correlation with a measure of eating disorder symptoms (i.e., 0.70 > r > 0.50) and that Healthy Orthorexia would yield a significant, moderate correlation with a measure of disordered eating (i.e., 0.30 < r < 0.50; Hypothesis 3). We also expected that scores on the Orthorexia Nervosa subscale would moderately relate to scores on measures of obsessive–compulsive symptoms, negative reaction to mistakes, and subjective distress (i.e., 0.50 > rs > 0.30; Hypothesis 4). Given past findings where Healthy Orthorexia showed near-zero relationships (unless controlling for its overlap with Orthorexia Nervosa, in which case residual effects sometimes emerge; e.g., [17, 26]), we did not set any hypotheses for the relationship between Healthy Orthorexia and measures of obsessive–compulsive symptoms, negative reaction to mistakes (i.e., perfectionism), and subjective distress. In support of the validity of the subscales as sufficiently distinct from a measure of disordered eating, we expected both subscales of the TOS would be less strongly related to body image disturbance than would be a measure of eating disorder symptoms (Hypothesis 5). Finally, we expected symptoms of health anxiety would relate more to symptoms of pathological preoccupation with healthy eating than interest/engagement in healthy eating (Hypothesis 6) and that, together, the two dimensions of orthorexia measured by the TOS would account for a significant amount of variance in health anxiety after controlling for eating disorder symptoms (Hypothesis 7).
Methods
Participants
We set a minimum sample size of 200 to ensure 10 participants per parameter [43]. We restricted participation to college students enrolled at a large southeastern U.S. university who were at least 18 years of age.
Of the 450 participants who responded to the study, 94.9% completed the study in full. We removed 123 participants who failed one or more attention checks and 23 who did not finish the study. This left 304 participants who responded to all questions, passed all attention checks, and were included in the analyses. Within our final sample, most participants (n = 261, 85.9%) identified as White/Caucasian (no other racial/ethnic identity exceeded 8% in the sample). Participants’ ages ranged from 18 to 45 (M = 20.63; SD = 2.52), with 94.7% under the age of 24. Most participants identified as cisgender women (n = 218, 71.7%) or men (n = 80, 26.3%). Undergraduate students comprised the vast majority (97.0%) of participants.
Participants’ mean BMI (calculated from self-reported height and weight) was 24.37 (SD = 4.61), falling within the “normal” range [44]. Few participants (11.8%) reported a professionally diagnosed food allergy. A history of dieting was common among participants, with 62.2% having engaged in at least one diet; rarely were the diets recommended by health-care professionals. Nearly 10% of participants reported a previous diagnosis of an eating disorder (4.9%), obsessive–compulsive disorder (4.0%), or both (0.1%); and 60% (n = 18) of these individuals had received treatment.
Measures
Teruel Orthorexia Scale (TOS)
Barrada and Roncero [17] created the two-subscale TOS to assess orthorexia symptoms. The Healthy Orthorexia subscale examines interest and engagement in a healthy diet (e.g., “I mainly eat foods that I consider to be healthy”). The Orthorexia Nervosa subscale assesses negative consequences of fixation on a healthy diet, including self-punishment, negative emotions, and interference with daily life (e.g., “If, at some point, I eat something that I consider unhealthy, I punish myself for it”). Participants rate 17 statements on a 4-point Likert scale ranging from 0 (completely disagree) to 3 (completely agree). Items are summed to create subscale scores, with higher scores indicating more orthorexic symptoms. The original measure (in Spanish) was validated on a sample of over 900 individuals aged 18 and above (mostly students). The authors provided an English translation of the TOS in which a psychologist and professional translator with 20 years of translation experience translated the measure. To our knowledge, the English version was not back-translated or evaluated psychometrically.
Measures used to evaluate concurrent validity
For all but the measure of body image disturbance, we created total scores, by summing across items, on the measures used to evaluate concurrent validity. We averaged items to compute the total score for the measure of body image disturbance. For all measures, higher scores indicate greater endorsement of symptomology/concerns.
We used the Eating Attitudes Test-26 (EAT-26 [45]) to assess eating disorder symptoms. The EAT-26 consists of 26 items rated on a 6-point Likert scale ranging from always to never (e.g., “I am preoccupied with a desire to be thinner”). The three most pathological responses receive scores of 3, 2, or 1, depending on severity, with the remaining responses all scored 0. Research supports the reliability (α > 0.90 [46]) and validity [47] of the EAT-26. Current sample alpha was 0.85.
We used the Obsessive–Compulsive Inventory-Revised (OCI-R [48]), an 18-item shortened version of the original OCI [49], to assess OCD symptoms. Participants rated how much 18 experiences (e.g., “I check things more often than necessary”) distressed or bothered them during the past month using a 5-point scale ranging from 0 (not at all) to 4 (extremely). Research supports the reliability and validity of the OCI-R [48, 50], and the OCI-R effectively discriminates between individuals with OCD, those with other anxiety disorders, and healthy controls [48]. Current sample alpha was 0.89.
We used the Frost Multidimensional Perfectionism Scale–Concern over Mistakes Scale (CMS [51]) to measure respondents’ perfectionism, specifically their tendency to experience negative emotions as a result of mistakes, equate mistakes to failure, and believe that mistakes cause a loss of respect. Participants rated their agreement with the 9 CMS items (e.g., “If I do not do well all the time, people will not respect me”) using a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). Research supports the reliability and validity of the CMS [17, 51, 52]. Current sample alpha was 0.90.
The Positive and Negative Affect Schedule–Negative Affect Scale (PANAS–NAS [53]) was used to measure subjective distress. Participants rated the extent to which they experienced 10 markers of negative affect (e.g., “distressed” and “ashamed”) during the past few weeks using a 5-point scale ranging from 1 (very slightly or not at all) to 5 (extremely). Research supports the reliability and validity of the PANAS-NAS in a college sample [53]. Current sample alpha was 0.88.
We used the Situational Inventory of Body-Image Dysphoria-Short Form (SIBID-SF [54]) to assess a component of body image disturbance—dysphoric body image emotions. The SIBID-SF, based on the original 48-item version [55], contains 20 items. Participants rated each item based on how often they experienced negative feelings about their appearance in the given situation (e.g., “When I look at myself in the mirror”) using a 5-point Likert scale ranging from 0 (never) to 4 (always or almost always). Research supports the reliable and valid use of the SIBID-SF [54]. Current sample alpha was 0.97.
Short Health Anxiety Inventory (SHAI)
We used the SHAI [56], an 18-item short form of the Health Anxiety Inventory, to assess health anxiety. Each item contains four statements (a–d). Participants were instructed to select the statement which best captures their feelings over the prior six months [e.g., “(a) I usually feel at very low risk for developing a serious illness; (b) I usually feel at fairly low risk for developing a serious illness; (c) I usually feel at moderate risk for developing a serious illness; (d) I usually feel at high risk for developing a serious illness”). Items are scored from 0 to 3, where a = 0, b = 1, and so on. Participants can select more than one statement and the statement with the highest score is used in summing items to compute the total score. Higher scores represent greater levels of health anxiety. Research supports the internal consistency (α ≥ 0.86 [56, 57]), and correlations with a measure of attitudes towards illness and medical utilization support the validity [57], of the SHAI. In addition, the SHAI effectively differentiates between people diagnosed with hypochondriasis and comparison groups [56]. Current sample alpha was 0.90.
Demographics
The survey included a demographics questionnaire to gather background information. Participants indicated their age, gender identity, racial/ethnic identity, height, weight, food allergies, dieting history, and history of obsessive–compulsive and eating disorders diagnoses.
Procedure
To complete a back-translation of the TOS, we provided the English version of the TOS to a second-year Spanish graduate student teaching assistant with a bachelor’s degree from Spain. The individual was blinded to the original Spanish version of the TOS, per the accepted procedure [58] and translated the English version of the TOS to Spanish. A second graduate student in the same department, who was a native of Spain, compared that back-translated version to the original to determine comparability of language and ensure that any differences would result in equivalent interpretation [58]. Most items were considered equivalent in meaning despite minor differences in language choice. However, slight differences in meaning for items 3 and 13 were found. We chose to use the published English TOS as is given its existence in the literature and our goal to evaluate the psychometric properties of the published measure.
Once Institutional Review Board approval was gained, we recruited participants through SONA (a research participation system used at the University). Upon selecting the study, participants rerouted to the survey in Qualtrics, an online software, where they viewed a letter containing information for informed consent. Those who consented proceeded with the study and completed the measures in a randomized sequence followed by the demographics questionnaire. We embedded nine attention check items in measures throughout the study (e.g., “I am reading carefully and therefore will select agree as my response”). We carefully worded attention checks to mirror length of items in our measures and match the structure of the items within the measure (e.g., if most items in the measure started with “I,” the attention check also started with “I”). Thus, our attention checks avoided easy detection by a visual scan and required reading each item. We did not include attention checks on some measures due to the difficulty to structure our attention checks to align with the measure and the goal to embed them within measures. In no case did we include more than two attention checks within any measure. Participants received extra credit in a course in exchange for participation.
Statistical analyses
We used R to conduct a parallel analysis and exploratory factor analysis (EFA) and used MPlus to conduct exploratory structural equation modeling (ESEM) to test the factor structure of the English-language TOS in a U.S. sample. Specifically, in evaluating the internal structure of the TOS, we followed the process used by Sánchez-Carracedo et al. [59] by first conducting an EFA prior to the ESEM. We used parallel analysis and extant research to extract factors for the EFA. For our EFA, we used weighted least squares (WLS) extraction with oblimin rotation. We evaluated model fit using the Tucker–Lewis index (TLI), standardized root mean squared residual (SRMR), and root mean square error approximation (RMSEA). For our ESEM, we used WLS extraction with geomin rotation. We evaluated fit using the same indices as for the EFA and with the comparative fit index (CFI). Although recommendations for cutoffs vary, we set the criteria for good fit as RMSEA and SRMR < 0.06 and 0.08, respectively, and values of > 0.95 for the CFI and TLI [60]. In addition, in evaluating factor structure, we considered items with loadings > 0.30 as factor indicators and sought clean factor loadings (differentiation > 0.20). Using SPSS 27, we tested our remaining hypotheses using Cronbach’s alphas, correlations, z tests of the beta weights for dependent correlations, and a hierarchical regression.
Results
Internal structure of the TOS
The parallel analysis indicated a three-factor solution. Given the TOS authors developed the measure with two factors, we conducted a set of EFAs with two and three-factor solutions. Bartlett’s Test of Sphericity (χ2 = 2366.67; df = 136; p < 0.001) and the KMO of 0.91 supported use of EFA with the data. See Table 1 for EFA factor loadings. Initially, we included all 17 TOS items in the EFA. The three-factor EFA accounted for 51% of the variance and fit the data well (SRMR = 0.04; RMESA = 0.05, CI = 0.04–0.07; and TLI = 0.95). However, there were several cross-loadings between factors, only three items loaded cleanly onto factor 3 and five items lacked clean loadings (differentiation < 0.20; items 1, 9, 13, 15, and 16). Correlation between factors ranged from 0.32 to 0.58. The two-factor EFA accounted for 47% of the variance and produced mixed fit indices (SRMR = 0.05; RMSEA = 0.07, CI = 0.06–0.08; and TLI = 0.90). In this case, only one item demonstrated cross loading (i.e., item 13) and two items (i.e., 13 and 15) lacked clean loadings. The correlation between the factors was 0.41.
Next, we used ESEM to evaluate model fit, following procedures reported by the TOS authors [26]. Given the number of cross-loadings on the three-factor EFA and the theoretical structure for the TOS, we tested the two-factor model published elsewhere. Table 2 contains the factor loadings for our ESEM analyses. The 2-factor ESEM with all 17 TOS items fit the data well (CFI = 0.98, TLI = 0.97; SRMR = 0.04; RMSEA = 0.06 [CI = 0.05–0.07). In this case, only one item produced a loading > 0.30 on the non-primary factor. However, item 13 continued to show poor differentiation between the factors, even with this more robust analysis, and produced a loading on its primary factor that was weaker than the correlation between the factors (r = 0.51, p < 0.001). In addition, the wording used in item 13 confounds quantity and health when quantity is said to be unimportant in orthorexia nervosa (e.g., [7]). With the concerns with poor differentiation with item 13 (drawing on problems also found with the EFA) and the confounding of two constructs within the item wording, we ran a second ESEM eliminating that item. This modified ESEM with 16 TOS items fit the data well (CFI = 0.98; TLI = 0.97; SRMR = 0.04; RMSEA = 0.06 [CI = 0.05–0.08]). One item produced a loading > 0.30 on the non-primary factor but demonstrated good differentiation, and all loadings on their primary factor exceeded the correlation between the factors (r = 0.50, p < 0.001).
We used this 16-item version of the measure, the TOS-16, with 2 subscales, Healthy Orthorexia and Orthorexia Nervosa, when testing the remaining hypotheses.
We examined the reliability of each TOS-16 subscale. The Cronbach’s alpha was 0.88 for the eight-item Healthy Orthorexia subscale and was 0.86 for the eight-item Orthorexia Nervosa subscale. As predicted, both alpha values were satisfactory.
Examining the TOS-16 and theoretically related psychological constructs
Table 3 contains means, standard deviations, and intercorrelations for each measured variable. Our third hypothesis was that scores on measures of disordered eating (EAT-26), obsessive–compulsive symptoms (OCI-R), perfectionism (CMS), and emotional distress (PANAS-NAS) would moderately to strongly correlate (i.e., r would exceed 0.30, but would not exceed 0.70) with scores on the TOS-16 Orthorexia Nervosa. As expected, all relationships with the Orthorexia Nervosa subscale fell within the expected, moderate range, lending support to both the convergent and discriminant validity of the TOS-16 by demonstrating that the instrument is measuring a construct that is related, but distinct from disordered eating, obsessive–compulsive disorder, perfectionism, and general emotional distress, respectively. Except for eating disorder symptoms, TOS-16 Healthy Orthorexia was not significantly related to the indicators of psychopathology, and consistent with our third hypothesis, the relationship between TOS-16 Healthy Orthorexia and a measure of disordered eating was < 0.50.
Body image disturbance, eating disorder symptoms and orthorexia
Because body image disturbance is said to be less relevant for orthorexia (e.g., [7, 11]), we hypothesized that scores on the SIBID-SF (i.e., body image disturbance) would relate more strongly to scores on the EAT-26 (i.e., eating disorder symptoms) than to scores on both TOS-16 subscales. As anticipated and supporting discriminant validity, the z tests of the beta weights showed the relationship between body image disturbance and eating disorder symptoms was significantly stronger than that between body image disturbance and interest/engagement in (z = 5.91, p < 0.001) or pathological preoccupation with (z = 1.98, p = 0.024) healthy eating.
Health anxiety and orthorexic eating
For Hypothesis 5, we predicted that symptoms of health anxiety would relate more to symptoms of pathological preoccupation with healthy eating than interest/engagement in healthy eating. We found a moderate correlation between scores on the SHAI and scores on the Orthorexia Nervosa subscale and a near-zero relationship between scores on the SHAI and scores on the Healthy Orthorexia subscale (see Table 3). A z test of the beta weights supported our hypothesis (z = 7.34, p < 0.001).
We regressed health anxiety on a measure of eating disorder symptoms in Step 1, R2 = 0.20, p < 0.001, and orthorexia (using both TOS-16 subscales) in Step 2, R2Δ = 0.05, p < 0.001, to test our sixth hypothesis. TOS-16 subscales explained a significant amount of variance in health anxiety after accounting for the relationship between eating disorder symptoms and health anxiety, with both interest/engagement in healthy eating (i.e., TOS-16 Healthy Orthorexia; sr = − 0.16, p = 0.002) and pathological preoccupation with healthy eating (i.e., TOS-16 Orthorexia Nervosa; sr = 0.22, p < 0.001) each explaining a significant amount of unique variance.
Exploratory analyses
Although we made no hypotheses about the relationship between TOS-16 Healthy Orthorexia and measures of obsessive–compulsive symptoms, perfectionism, or negative affect, we examined the partial correlations in light of prior research sometimes illustrating a residual effect. Specifically, the residual effect reflects a statistically pure construct in which the significant shared variance (i.e., r = 0.49) between TOS-16 Orthorexia Nervosa and TOS-16 Healthy Orthorexia is removed (essentially, holding participants’ scores on TOS-16 Orthorexia Nervosa constant). In such case, we too found residual relationships between TOS-16 Healthy Orthorexia and measures of obsessive–compulsive symptoms (r = − 0.23, p < 0.01), perfectionism (r = − 0.15, p = 0.009), and negative affect (r = 0.32, p < 0.001). Similarly, partialing out the shared variance with TOS-16 Orthorexia Nervosa resulted in a negative relationship between TOS-16 Healthy Orthorexia and a measure of body image concerns (r = − 0.35, p < 0.001) and a near-zero relationship with a measure of disordered eating (r = 0.01, p = 0.853).
Discussion
Implications of findings
The English version of the TOS was translated from Spanish to English by an experienced professional before its publication in 2018 [17], but never back-translated. We followed the recommended procedure for back-translation [58] and found slight differences in meaning for items 3 and 13. We used the published version of the TOS to evaluate its psychometric properties given its availability in the literature. However, the discrepancies raise questions about what might be measured across the two versions of the TOS.
Using EFA and ESEM, we identified a 16-item 2-factor solution congruent with theory and supported by the data. These two subscales yielded reliability coefficients that suggest good reliability and equaled or exceeded those published for other orthorexia measures. The strong internal consistencies of the TOS-16 subscales suggest that the TOS-16 could serve as a high-quality assessment tool, and thus, is appropriate for use in future research in U.S. samples.
Consistent with our hypothesis and past research [17], the TOS-16 Orthorexia Nervosa subscale correlated moderately with obsessive–compulsive and eating disorder symptoms, perfectionism, and emotional distress. This supports the convergent validity of the TOS-16. Although the TOS-16 Orthorexia Nervosa subscale significantly related to the EAT-26 (r = 0.61), the magnitude of the relationship was at least as modest (suggesting assessment of related but distinct constructs) as found with other promising measures (e.g., Gleaves et al. [41] found the EHQ and EAT-26 were significantly correlated; r = 0.79). The TOS-16 Healthy Orthorexia subscale was relatively unrelated to obsessive–compulsive disorder symptoms, perfectionism, and emotional distress except when examining residual effects as described below. Although there was a moderate correlation between TOS-16 Healthy Orthorexia and eating disorder symptoms, it was not as strong as the relationship between the subscale that assesses pathological preoccupation with healthy eating and eating disorder symptoms. Segura-García et al. [40] proposed an interesting theory that orthorexia nervosa represents a “residual” symptom of anorexia and bulimia that allows individuals to have a continued sense of control after treatment. Our results, in which both the TOS-16 subscales measuring pathological preoccupation with (i.e., Orthorexia Nervosa) and interest/engagement in (i.e., Healthy Orthorexia) healthy eating related to a measure of disordered eating, are not inconsistent with this possibility. If orthorexia nervosa is residual syndrome, it could interfere with eating disorder recovery. This possibility also suggests precaution against use of the TOS-16 Healthy Orthorexia subscale in isolation as an indicator of recovery among individuals with documented eating disorders (i.e., without a low TOS-16 Orthorexia Nervosa score, the TOS-16 Healthy Orthorexia score may reflect overlap with pathological preoccupation with healthy eating; see [61] for consideration of potential types of TOS responders).
Theorists contend that body image disturbance is absent or unimportant in orthorexia [7, 11], distinguishing it from anorexia and bulimia. Supporting discriminant validity of the TOS-16, the relationship between body image disturbance and disordered eating symptoms was significantly stronger than that between body image disturbance and TOS-16 subscale scores. The moderate association between the TOS-16 Orthorexia Nervosa subscale and body image disturbance does call into question the notion that body image disturbance is absent in orthorexia. Indeed, other scholars suggest that thinness, weight, and appearance concerns seem to be a motivator in orthorexia nervosa (e.g., [62]). Such findings suggest a need to further determine whether orthorexia nervosa is a subtype of an existing disorder or warrants its own diagnosis, and whether a lack of body image concerns should be included in differential diagnosis [62]. Of course, without a recognized diagnosis, studies on orthorexia nervosa tend to be correlational and samples may include individuals with other, established eating disorders (e.g., anorexia nervosa).
We found a moderate correlation between pathological preoccupation with healthy eating and health anxiety, and this relationship was significantly stronger than that between interest/engagement in healthy eating and health anxiety. This is consistent with the results of the known studies that previously examined the relationship between the two constructs [38, 39, 63, 64]. We also found that the TOS-16 subscales together accounted for a significant amount of variance in health anxiety after controlling for variance accounted for by eating disorder symptoms with each explaining a significant amount of unique variance. The finding that the relationship between a focus on healthy eating and health anxiety is significant and negative when controlling for measures of disordered eating and preoccupation with healthy eating suggest a residual relationship. In that case, holding pathological interest in healthy eating and eating disorder symptoms constant, those with an interest in engaging in healthy eating may be less likely to experience health anxiety. Taken together, these relationships suggest that the pathological focus on healthy eating is the factor that may reflect an attempt to manage health anxiety, but a general interest in healthy eating does not seem to relate to health anxiety or could be protective. These findings are some of the first to provide research support for the long-hypothesized connection between orthorexia and health anxiety, and the first to do so in the U.S.
Consistent with prior research, we found residual effects in which the TOS-16 Healthy Orthorexia related to reduced concerns with obsessive compulsive symptoms, perfectionism, negative affect, and body image disturbance, as well as being unrelated with disordered eating, when we held participants’ scores constant on the TOS-16 Orthorexia Nervosa subscale. This suggests that the TOS-16 Healthy Orthorexia subscale assesses, in part, the potential for an interest in healthy eating to serve as a protective factor. At the same time, we caution clinicians in using this subscale in isolation or as an indicator that symptoms of orthorexia nervosa are not problematic. Individuals who endorse items on the TOS-16 Healthy Orthorexia subscale also tend to endorse items on the TOS-16 Orthorexia Nervosa subscale (there is 25% shared variance between the scales in our sample), suggesting that for an important subgroup, the focus on healthy eating is a part of their pathological orthorexic eating.
Limitations and strengths
Limitations to this study warrant acknowledgment. First, our sample was relatively homogenous, limiting the generalizability of results. Furthermore, we did not assess socioeconomic status which can have a significant impact on one’s ability to access foods considered “healthy” and “pure.” Second, the cross-cultural validity of the TOS remains uncertain. We cannot be sure items mean the same thing to U.S. and Spanish participants. Our findings support the need for evaluating measurement invariance across samples. Third, this study cannot address causation between symptoms of orthorexia and other psychological conditions, such as health anxiety. In addition, it is impossible to know whether the relationships found between symptoms of orthorexia and other relevant constructs (e.g., perfectionism, emotional distress) were independent of other conditions such as anorexia or OCD.
The key strengths of our study were examining the back-translation and evaluating the TOS in an English-speaking sample in the U.S. which was similar to samples used in studies with the Spanish version. We found psychometric support for the theorized relationship between orthorexic eating, particularly orthorexia nervosa, and health anxiety.
Conclusions
We found support for use of a modified English version of the TOS with 16 items comprising two subscales. We found a pattern of moderate relationships between the TOS-16 subscale measuring pathological preoccupation with healthy eating with related constructs such as perfectionism, symptoms of eating and obsessive–compulsive disorders, and negative affect, whereas the measure of interest and engagement in healthy eating only related to eating disorder symptoms. Therefore, the TOS-16 has psychometric support and appears to be a promising measure for use in English-speaking U.S. samples. Thus, the TOS-16 warrants further research, and such research is especially needed with clinical populations, in U.S. samples.
Overall, our results also speak to the distinctiveness of the subscales and how orthorexic eating goes beyond interest/engagement in healthy eating—it represents a level of preoccupation that is unhealthy and associated with psychological distress and impairment. For example, orthorexia nervosa may impact one’s ability to engage socially or to allow for flexibility in one’s diet without subsequent shame, anxiety, or self-punishment. For those practicing in health-related fields, this is an important distinction when considering whether a person’s behavior may be problematic.
Ours was also one of the first studies to examine the relationship between symptoms of health anxiety and symptoms of orthorexia, and was the first known study to do so with an English-speaking, U.S. sample. We found a moderate association between health anxiety and orthorexia, and demonstrated that symptoms of orthorexia accounted for a significant proportion of the variance in health anxiety. Thus, it is possible that pathological preoccupation with healthy eating reflects, at least to some extent, anxiety about health.
What is already known on this subject?
The TOS produced moderately good, relative to other measures of orthorexia, psychometric properties in Spanish-, Portuguese-, and Arabic-speaking samples, including yielding a pattern of relationships with measures of related constructs (e.g., obsessive–compulsive and eating disorder symptoms) fitting with theory. In addition, scant research examined the theoretical relationships between orthorexia and somatic symptoms.
What does this study add?
The English-language TOS required slight modification in an English-speaking, U.S. sample. Our study provided support for use of the English-language TOS-16 as a valid and reliable measure of orthorexic symptoms in a U.S. sample. In addition, somatic symptoms related to orthorexic symptoms in our sample.
Availability of data and materials
Data are available from the corresponding author upon request.
References
Barthels F, Lavendel S, Müller R, Pietrowsky R (2019) Relevance of orthorexic eating behavior in nutrition counseling and nutrition therapy. Results of a nationwide survey among German nutritionists. Ernahrungs Umschau 66:236–241. https://doi.org/10.4455/eu.2019.048
Reynolds S, McMahon S (2020) Views of health professionals on the clinical recognition of orthorexia nervosa: a pilot study. Eat Weight Disord. https://doi.org/10.1007/s40519-019-00701-z
Ryman FVM, Cesuroglu T, Bood ZM, Syurina EV (2019) Orthorexia nervosa: disorder or not? Opinions of Dutch health professionals. Front Psychol. https://doi.org/10.3389/fpsyg.2019.00555
Cena H, Barthels F, Cuzzolaro M et al (2019) Definition and diagnostic criteria for orthorexia nervosa: a narrative review of the literature. Eat Weight Disord 24:209–246. https://doi.org/10.1007/s40519-018-0606-y
Bratman S (1997) The health food eating disorder. Yoga J 42:50
Donini LM, Marsili D, Graziani MP, Imbriale M, Cannella C (2004) Orthorexia nervosa: a preliminary study with a proposal for diagnosis and an attempt to measure the dimension of the phenomenon. Eat Weight Disord 9:151–157. https://doi.org/10.1007/bf03325060
Varga M, Dukay-Szabó S, Túry F, van Furth EF (2013) Evidence and gaps in the literature on orthorexia nervosa. Eat Weight Disord 18:103–111. https://doi.org/10.1007/s40519-013-0026-y
Barthels F, Meyer F, Pietrowsky R (2015) Orthorexic eating behavior: a new type of disordered eating. Ernaehrungs Umschau 62:156–161
Moroze RM, Dunn TM, Holland JC, Yager J, Weintraub P (2015) Microthinking about micronutrients: a case of transition from obessions about healthy eating to near-fatal “orthorexia nervosa” and proposed diagnostic criteria. Psychosomatics 56:397–403. https://doi.org/10.1016/j.psym.2014.03.003
Vandereycken W (2011) Media hype, diagnostic fad or genuine disorder? Professionals’ opinions about night eating syndrome, orthorexia, muscle dysmorphia, and emetophobia. Eat Disord 19:145–155. https://doi.org/10.1080/10640266.2011.551634
Dunn TM, Bratman S (2016) On orthorexia nervosa: a review of the literature and proposed diagnostic criteria. Eat Behav 21:11–17. https://doi.org/10.1016/j.eatbeh.2015.12.006
Brytek-Matera A, Krupa M, Poggiogalle E, Donini LM (2014) Adaptation of the ORTHO-15 test to Polish women and men. Eat Weight Disord 19:69–76. https://doi.org/10.1007/s40519-014-0100-0
Missbach B, Hinterbuchinger B, Dreiseitl V, Zellhofer S, Kurz C, König J (2015) When eating right, is measured wrong! A validation and critical examination of the ORTO-15 questionnaire in German. PLoS ONE 10:e0135772. https://doi.org/10.1371/journal.pone.0135772
Moller S, Apputhurai P, Knowles SR (2019) Confirmatory factor analyses of the ORTO 15-, 11- and 9-item scales and recommendations for suggested cut-off scores. Eat Weight Disord 24:21–28. https://doi.org/10.1007/s40519-018-0515-0
Missbach B, Dunn TM, König JS (2017) We need new tools to assess orthorexia nervosa. A commentary on “prevalence of orthorexia nervosa among college students based on Bratman’s test and associated tendencies.” Appetite 108:521–524. https://doi.org/10.1016/j.appet.2016.07.010
Chard CA, Hilzendegen C, Barthels F, Stroebele-Benschop N (2019) Psychometric evaluation of the English version of the Düsseldorf Orthorexie Scale (DOS) and the prevalence of orthorexia nervosa among a U.S. student sample. Eat Weight Disord 24:275–281. https://doi.org/10.1007/s40519-018-0570-6
Barrada JR, Roncero M (2018) Bidimensional structure of the orthorexia: development and initial validation of a new instrument. An Psicol 34:283–291. https://doi.org/10.6018/analesps.34.2.299671
Koven NS, Abry AW (2015) The clinical basis of orthorexia nervosa: emerging perspectives. Neuropsychiatr Dis Treat 11:385–394. https://doi.org/10.2147/NDT.S61665
Alvarenga MS, Martins MCT, Sato KSCJ, Vargas SVA, Philippi ST, Scagliusi FB (2012) Orthorexia nervosa behavior in a sample of Brazilian dietitians assessed by the Portuguese version of ORTO-15. Eat Weight Disord 17:e29–e35. https://doi.org/10.1007/bf03325325
Meule A, Holzapfel C, Brandl B et al (2020) Measuring orthorexia nervosa: a comparison of four self-report questionnaires. Appetite 146:104512. https://doi.org/10.1016/j.appet.2019.104512
Roncero M, Barrada JR, Perpiña C (2017) Measuring orthorexia nervosa: psychometric limitations of the ORTO-15. Span J Psychol 20:1–9. https://doi.org/10.1017/sjp.2017.36
Donini LM, Marsili D, Graziani MP, Imbriale M, Cannella C (2005) Orthorexia nervosa: validation of a diagnosis questionnaire. Eat Weight Disord 10:e28–e32. https://doi.org/10.1007/bf03327537
Rogoza R, Donini DM (2021) Introducing ORTO-R: a revision of the ORTO-15. Eat Weight Disord 26:887–895. https://doi.org/10.1007/s40519-020-00924-5
Stoeber J, Gaudreau P (2017) The advantages of partialling perfectionistic strivings and perfectionistic concerns: criticial issues and recommendations. Personal Individ Diff 104:379–386. https://doi.org/10.1016/j.paid.2016.08.039
Depa J, Barrada JR, Roncero M (2019) Are the motives for food choices different in orthorexia nervosa and healthy orthorexia. Nutrients 11:1–14. https://doi.org/10.3390/nu11030697
Barthels F, Barrada JR, Roncero M (2019) Orthorexia nervosa and healthy orthorexia as new eating styles. PLoS ONE 14:e0219609. https://doi.org/10.1371/journal.pone.0219609
Roberto da Silva W, Cruz Marmol CH, Nogueira Neves A et al (2020) A Portuguese adaptation of the Teruel Orthorexia Scale and a test of its utility with Brazilian young adults. Percept Mot Skills. https://doi.org/10.1177/00315125211029240
Mhanna M, Azzi R, Hallit S et al (2021) Validation of the Arabic version of the Teruel Orthorexia Scale (TOS) among Lebanese adolescents. Eat Weight Disord. https://doi.org/10.1007/s40519-021-01200-w
Strahler J (2020) The dark side of healthy eating: links between orthorexic eating and mental health. Nutrients 12:1–12. https://doi.org/10.3390/nu12123662
Depa J, Schweizer J, Bekers SK, Hilzendegen C, Stroebele-Benschop N (2017) Prevalence and predictors of orthorexia nervosa among German students using the 21-item-DOS. Eat Weight Disord 22:193–199. https://doi.org/10.1007/s40519-016-0334-0
de Souza QJOV, Rodrigues AM (2014) Risk behavior for orthorexia nervosa in nutrition students. J Bras Psiquiatr 63:200–204. https://doi.org/10.1590/0047-2085000000026
Dell’Osso L, Carpita B, Muti D et al (2018) Prevalence and characteristics of orthorexia nervosa in a sample of university students in Italy. Eat Weight Disord 23:55–65. https://doi.org/10.1007/s40519-017-0460-3
Mathieu J (2005) What is orthorexia? J Am Diet Assoc 105:1510–1512. https://doi.org/10.1016/j.jada.2005.08.021
Brytek-Matera A, Fonte ML, Poggiogalle E, Donini LM, Cena H (2017) Orthorexia nervosa: relationship with obsessive–compulsive symptoms, disordered eating patterns and body uneasiness among Italian university students. Eat Weight Disord 22:609–617. https://doi.org/10.1007/s40519-017-0427-4
Kaye WH, Bulik CM, Thornton L, Barbarich N, Masters K (2004) Comorbidity of anxiety disorders with anorexia and bulimia nervosa. Am J Psychiatr 161:2215–2221. https://doi.org/10.1176/appi.ajp.161.12.2215
Plichta M, Jezewska-Zychowicz M, Malachowska A (2020) Relationship between psychosocial impairment, food choice motives, and orthorexic behaviors among Polish adults. Nutrients 12:1–16
Catalina Zamora ML, Bote Bonaechea B, García Sánchez F, Ríos Rial B (2005) Orthorexia nervosa. A new eating behavior disorder? Actas Esp Psiquiatr 33:66–68
Barthels F, Müller R, Schüth T, Friederich H, Pietrowsky R (2019) Orthorexic eating behavior in patients with somatoform disorders. Eat Weight Disord. https://doi.org/10.1007/s40519-019-00829-y
Barthels F, Horn S, Pietrowsky R (2021) Orthorexic eating behavior, illness anxiety and dysfunctional cognitions characteristic of somatic symptom disorders in a non-clinical sample. Eat Weight Disord. https://doi.org/10.1007/s40519-020-01091-3
Segura-García C, Ramacciotti C, Rania M et al (2015) The prevalence of orthorexia nervosa among eating disorder patients after treatment. Eat Weight Disord 20:161–166. https://doi.org/10.1007/s40519-014-0171-y
Gleaves DH, Graham EC, Ambwani S (2013) Measuring “orthorexia”: development of the eating habits questionnaire. Int J Educ Psychol Assess 12:1–18
Oberle CD, De Nadai AS, Madrid AL (2021) Orthorexia nervosa inventory (ONI): development and validtation of a new measure of orthorexic symptomatology. Eat Weight Disord 26:609–622. https://doi.org/10.1007/s40519-020-00896-6
Schreiber JB, Nora A, Stage FK, Barlow EA, King J (2006) Reporting structural equation modeling and confirmatory factor analysis results: a review. J Educ Res 99:323–338. https://doi.org/10.3200/JOER.99.6.323-338
Centers for Disease Control (2020) How is BMI interpreted for adults? https://www.cdc.gov/healthyweight/assessing/bmi/adult_bmi/index.html. Accessed 24 Jan 2021
Garner DM, Olmsted MP, Bohr Y, Garfinkel PE (1982) The eating attitudes test: psychometric features and clinical correlates. Psychol Med 12:871–878. https://doi.org/10.1017/S0033291700049163
Mazzeo SE (1999) Modification of an existing measure of body image preoccupation and its relationship to disordered eating in female college students. J Couns Psychol 46:42–50. https://doi.org/10.1037/0022-0167.46.1.42
Mintz LB, O’Holloran MS (2000) The eating attitudes test: validation with DSM-IV eating disorder criteria. J Personal Assess 74:489–503. https://doi.org/10.1207/S15327752JPA7403_11
Foa EB, Huppert JD, Leiberg S, Langner R, Kichic R, Hajcak G, Salkovskis PM (2002) The obsessive–compulsive inventory: development and validation of a short version. Psychol Assess 14:485–496. https://doi.org/10.1037//1040-3590.14.4.485
Foa EB, Kozak MJ, Salkovskis P, Coles ME, Amir N (1998) The validation of a new obsessive–compulsive disorder scale: the obsessive–compulsive inventory. Psychol Assess 10:206–214. https://doi.org/10.1037/1040-3590.10.3.206
Hajcak G, Huppert JD, Simons RF, Foa EB (2004) Psychometric properties of the OCI-R in a college sample. Behav Res Ther 42:115–123. https://doi.org/10.1016/j.brat.2003.08.002
Frost RO, Marten P, Lahart C, Rosenblate R (1990) The dimensions of perfectionism. Cogn Ther Res 14:449–468. https://doi.org/10.1007/BF01172967
Frost RO, Heimberg RG, Holt CS, Mattia JI, Neubauer AL (1993) A comparison of two measures of perfectionism. Personal Individ Diff 14:119–126. https://doi.org/10.1016/0191-8869(93)90181-2
Watson D, Clark LA, Tellegen A (1988) Development and validation of brief measures of positive and negative affect: the PANAS scales. J Pers Soc Psychol 54:1063–1070. https://doi.org/10.1037/0022-3514.54.6.1063
Cash TF (2002) The situational inventory of body-image Dysphoria: psychometric evidence and development of a short form. Int J Eat Disord 32:362–366. https://doi.org/10.1002/eat.10100
Cash TF (1994) The situational inventory of body-image Dysphoria: contextual assessment of a negative body image. Behav Ther 17:133–134
Salkovskis PM, Rimes KA, Warwick HMC, Clark DM (2002) The health anxiety inventory: development and validation of scales for the measurement of health anxiety and hypochondriasis. Psychol Med 32:843–853. https://doi.org/10.1017/S0033291702005822
Abramowitz JS, Deacon BJ, Valentiner DP (2007) The short health anxiety inventory: psychometric properties and construct validity in a non-clinical sample. Cogn Ther Res 31:871–883. https://doi.org/10.1007/s10608-006-9058-1
Sperber AD (2004) Translation and validation of study instruments for cross-cultural research. Gastroenterol 126:S124–S128. https://doi.org/10.1053/j.gastro.2003.10.016
Sánchez-Carracedo D, Barrada JR, López-Guimerà G et al (2012) Analysis of the factor structure of the sociocultural attitudes towards appearance questionnaire (SATAQ-3) in Spanish secondary-school students through exploratory structural equation modeling. Body Image 9:163–171. https://doi.org/10.1016/j.bodyim.2011.10.002
Sun J (2005) Assessing goodness of fit in confirmatory factor analysis. Meas Eval Couns Dev 37:240–256. https://doi.org/10.1080/07481756.2005.11909764
Yakin E, Raynal P, Chabrol H (2021) Distinguishing between healthy and pathological orthorexia: a cluster analytic study. Eat Weight Disord. https://doi.org/10.1007/s40519-021-01178-5
DeBois K, Chatfield SL (2020) Misinformation, thin-ideal internalization, and resistance to treatment: an interpretive phenomenological analysis of the experience of orthorexia nervosa. Eat Weight Disord. https://doi.org/10.1007/s40519-020-01049-5
Kiss-Leizer M, Tóth-Király I, Rigó A (2019) How the obsession to eat healthy food meets with the willingness to do sports: the motivational background of orthorexia nervosa. Eat Weight Disord 24:465–472. https://doi.org/10.1007/s40519-019-00642-7
Tóth-Király I, Gajdos P, Román N, Vass N, Rigó A (2019) The associations between orthorexia nervosa and the sociocultural attitudes: the mediating role of basic psychological needs and health anxiety. Eat Weight Disord. https://doi.org/10.1007/s40519-019-00826-1
Acknowledgements
None.
Funding
No funding was received for conducting this study.
Author information
Authors and Affiliations
Contributions
SC (lead author) had primary responsibility for conceptualization, methodology, analysis, interpretation of the data, and writing. AK supervised the research and supported the conceptualization, design, analysis, interpretation of data, and review and editing of the manuscript. Both authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors have no financial or non-financial conflicts of interest to declare.
Ethics approval and consent to participate
The study was approved by the Institutional Review Board (IRB) at Auburn University prior to initiation of participant recruitment or data collection. Participants provided informed consent and the authors followed the protocol as approved by the IRB.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Chace, S., Kluck, A.S. Validation of the Teruel Orthorexia Scale and relationship to health anxiety in a U.S. sample. Eat Weight Disord 27, 1437–1447 (2022). https://doi.org/10.1007/s40519-021-01272-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40519-021-01272-8