Introduction

Definition and diagnostic criteria for ON

Orthorexia nervosa (ON) refers to a speculated condition that is characterized by compulsive behaviors and obsessive thoughts concerning healthy eating [1]. Although ON is not currently recognized as a disorder in the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) [2] or in the International Classification of Diseases, Eleventh Revision (ICD-11) [3], an increasing number of patients are seeking help from psychologists, dieticians, and other health professionals for ON-related impairments [4, 5], and four sets of diagnostic criteria for ON have been proposed [4,5,6,7]. Consistent among all four sets are the following: (a) obsessive behaviors and preoccupation with healthy nutrition that includes rigidly following a restrictive “healthy” diet (that the individual believes to be healthy and pure) with strict avoidance of foods believed to be unhealthy; (b) violations of their restrictive dietary rules resulting in extreme emotional distress with feelings of guilt, shame, and/or anxiety; (c) physical impairments, whereby nutritional deficiencies may lead to significant weight loss, malnutrition, and/or physical health complications; and (d) psychosocial impairments in social, vocational, and/or academic functioning that may result from the other-diagnostic criteria.

Past research on associations with ON

Many studies have found that ON symptomatology is greater among vegetarians and/or vegans compared to omnivores [8,9,10,11,12,13,14,15,16,17]. These findings make sense given that people with ON are motivated by their pursuit of perfect physical health, and that a nutrient-rich diet consisting predominantly of fruits and vegetables may decrease one’s risk of such life-threatening diseases as cardiovascular disease and cancer [18, 19]. On the unhealthy side, however, greater levels of ON symptomatology are consistently associated with higher scores on measures of disordered eating [9, 15, 20,21,22,23,24,25,26,27,28,29,30,31]. Although these results suggest that ON may be comparable to anorexia nervosa (AN), the motives for eating behaviors may potentially differ between the two, with AN individuals predominantly guided by the adherence to a distorted body image and ON individuals predominantly guided by the desire to be as healthy as possible, at least according to theories of ON [1, 4,5,6,7]. Further, ON seems to be distinct from AN in relation to two key demographic variables: body mass index (BMI) and gender. AN is defined by low body weight per the DSM-5 [2], and this disorder is more common in women compared to men at an average ratio of 9:1 [32, 33]. In contrast, comprehensive literature reviews reveal that ON symptomatology is generally not negatively correlated with BMI [34] and not greater in women than men [35].

Given that the eating behaviors of people with ON are motivated by their pursuit of perfect physical health, their commitment to (what they perceive to be) healthy eating may be just one part of a more inclusive strict lifestyle driven by the goal of being as healthy as possible. Research supports this supposition, with studies revealing greater scores on measures of ON symptomatology among people who do not smoke [36] or consume alcoholic beverages [37], and among people with greater levels of sport and physical activity [29, 36,37,38,39,40], as well as overall exercise addiction that includes a compulsion to follow a rigid pattern of intense exercise activities even when the person experiences an injury, illness, or other problems [41, 42]. The latter findings seem sensible when considering the ON individuals’ preoccupation with healthy eating and compulsions regarding what, how, and when they eat. In fact, their obsessions and compulsions regarding both eating and exercise may at least partly be attributed to a more general neuropsychological personality profile, a claim that is supported by several studies showing that greater levels of ON symptomatology are associated with greater levels of obsessive–compulsive tendencies that are not specific to eating or exercise [17, 20, 25,26,27, 31, 43,44,45].

Assessment of ON

There are four ON assessments that have been typically used in past research. First, the Bratman Orthorexia Test (BOT) [1] includes 10 items with “yes” or “no” response options. Unfortunately, with only 10 dichotomous-response items, internal consistency has been found to be comparatively low with Cronbach α values of 0.60 to 0.67 [23, 46]. Second, the ORTO-15 [47] includes 15 items using a 4-point Likert response scale. Despite its widespread use, the ORTO-15 suffers from questionable reliability, with Cronbach α values ranging from 0.14 to 0.83 (M = 0.55, SD = 0.27) across six studies [48]. Moreover, in their attempts to validate the factorial structure of the measure, a number of researchers found several items to be unrelated to the ON construct [22, 37, 43, 49,50,51,52]. For all these reasons, researchers are now advising against using the ORTO-15 [4, 50, 53]. Third, the Eating Habits Questionnaire (EHQ) [25], which includes 21 items using a 4-point Likert scale, has good reliability with Cronbach α values of 0.81 to 0.92 [25, 31, 48, 54] and test–retest coefficients of 0.72 to 0.81 [25]. Finally, the Duesseldorf Orthorexia Scale (DOS) [6], which includes 10 items using a 4-point Likert scale, also has good reliability with Cronbach α values of 0.80 to 0.88 [6, 14, 55,56,57] and test–retest coefficients of 0.67 to 0.79 [6, 57].

While the EHQ and DOS seem to be reliable assessments of ON symptomatology, none of these measures include any items assessing physical impairments associated with ON. Given that ON is not currently recognized as a disorder in the DSM-5 [2] or the ICD-11 [3], complete validation may be difficult. However, what could and should be done is to ensure that a good ON measure has an adequate number of items representing each of the consensus diagnostic criteria previously discussed. As may be seen in Table 1, although all four commonly used ON measures include a sufficient number of items assessing Criterion A (obsessive behaviors and preoccupation with “healthy” eating), none include enough items assessing Criterion B (emotional distress resulting from violations of their dietary rules), none include any items assessing Criterion C (physical impairments resulting from nutritional deficiencies), and only the EHQ includes an adequate number of items assessing Criterion D (psychosocial impairments resulting from the other criteria). Moreover, a good measure of ON must include items that will maximally differentiate those with ON involving pathological behavior and associated impairments from those with non-pathological healthy behavior [58].

Table 1 Commonly used ON assessments—number of items assessing each diagnostic criterion
Full size table

Purpose of the current study

This study’s purpose was to develop a reliable measure of ON symptomatology that includes an adequate number of items to assess each of the four consensus diagnostic criteria for ON, as well as items worded in such a way to better differentiate those who are non-pathological healthy eaters [58] from individuals who experience pathological impairments resulting from their extreme eating behaviors. We aimed to empirically establish the measure’s intended four-factor structure, its internal consistency and test–retest reliability, and its associations with related constructs. Regarding the latter associations, based on the past research discussed in the preceding literature review, we expected scores on this measure of ON to be (a) greater among vegetarians/vegans and among those who consume more fruits and vegetables but less meat and sweets, (b) greater among those exhibiting higher levels of disordered eating and among those with a prior diagnosis of AN, (c) greater among those exhibiting higher levels of obsessive–compulsive tendencies and among those with a prior diagnosis of obsessive–compulsive disorder (OCD), (d) greater among those who spend more time exercising and who exhibit higher levels of compulsive exercise, (e) greater among non-smokers and among those who drink lower levels of alcohol, and (f) statistically equivalent across gender and levels of BMI.

Method

Participants

Participants were recruited from several sources in an effort to obtain a large and varied sample. First, participation invitations were emailed to undergraduate Texas State University students enrolled in Nutrition and Psychology courses. Second, advertisements were posted in Facebook and Instagram, with the audience interests set to target both healthy eaters (e.g., “clean eating”, “healthy eating”, “paleo”) and normal or unhealthy eaters (e.g., “Cooking Panda”, “desserts”, “comfort food”). Of the 847 people who completed the survey, 47% were students (33 Nutrition majors, 154 Psychology majors, 222 with a different unspecified major), and the majority were female (82%) and either White (54%), Hispanic/Latino (21%), Black (7%), or bi/multiracial (9%). Their ages ranged from 18 to 75 years (M = 21.72, SD = 6.74), and based on self-reported weight and height to calculate BMI (in kg/m2), 8% were underweight (BMI < 18.5), 58% were normal weight (18.5 ≤ BMI < 25), 20% were overweight (25 ≤ BMI < 30), and 14% were obese (30 ≤ BMI).

Procedure

For this study, which was approved by Texas State University’s ethical committee (Institutional Review Board, protocol #6394), participants completed an online survey through Qualtrics. Upon clicking on the link to the survey (from the recruitment email or advertisement), participants were asked to provide their informed consent before commencing with the survey questions. The survey included all of the questionnaires described in the “Measures” section, in the same order as they are described below. At the end of the survey, participants were asked to optionally provide their email address, and for those participants who did so, a link to a second briefer survey with only the ON items (to determine test–retest reliability) was sent to them 2 weeks later.

Measures

Health-related behaviors

The demographic questionnaire included items asking about smoking status, alcohol consumption, and time spent exercising. For alcohol, participants were asked to estimate the number of alcoholic drinks consumed during a typical 7-day period, and they were told that one drink may be a 12-oz can of beer, 5-oz glass of wine, or 1.5-oz shot of hard liquor. For exercise, participants were asked to estimate, again for a typical 7-day period, the number of hours spent doing moderate or strenuous aerobic exercises with the heart beating rapidly, and the number of hours spent doing strength training exercises.

Diet and food consumption

The remaining portion of the demographic questionnaire included items about dietary behaviors. First, they were asked to select the diet category that best describes their current eating habits, with the response options being “non-vegetarian: will eat red meat, poultry, or fish on a regular basis”, “semi-vegetarian: mostly vegetarian but will eat red meat, poultry, or fish on occasion”, “vegetarian: no red meat, poultry, or fish; will eat eggs and/or dairy”, and “vegan: no red meat, poultry, fish, or any animal byproducts including eggs and dairy”. Next, they were asked how many servings they typically consume of dairy foods, meats, fruits, vegetables, grains, sweets, and sweetened non-alcoholic beverages. For each item, participants were provided with examples of serving sizes (e.g., “1 cup of yogurt”, “1/2 cup of blueberries”), and they were given the following response options: “none” (1), “1–2 servings per week” (2), “1–2 servings every other day” (3), “1–2 servings per day” (4), “3–4 servings per day” (5), and “5 or more servings per day” (6).

ON

For consideration in the ON inventory, we included statements (see Table 2) to which participants indicated how true the statements are for them based on their current eating habits, using a 4-point Likert scale with the following response options: “not at all true” (1), “slightly true” (2), “mainly true” (3), and “very true” (4). Given the established reliability of the DOS and EHQ that was discussed previously, we began by selecting items from these two measures that specifically relate to the consensus diagnostic criteria for ON, and we then modified some of the items for the purpose of better differentiating pathological ON from non-pathological healthy eating. For example, instead of using the item, “I have made efforts to eat more healthily over time” that does not seem to assess the more extreme dietary behaviors associated with ON, we included the following item: “Over time, my diet has come to include elimination of entire food groups that I believe are unhealthy.” Finally, we added items that assessed aspects of the consensus diagnostic criteria not already addressed with the other items.

Table 2 ONI factor analysis results
Full size table

Regarding those diagnostic criteria, first, 11 items pertained to Criterion A (obsessive behaviors and preoccupation with “healthy” eating), including 3 items modified from the EHQ (EHQ #18, EHQ #7/20, EHQ #3/21), 2 items modified from the DOS (DOS #1, DOS #2), 1 item asked in a separate study [59], and 5 new items, 3 of which refer to dietary restrictions increasing over time, which Dunn and Bratman [5] argue is a key characteristic of ON. Second, 5 items pertained to Criterion B (emotional distress resulting from violations of their dietary rules), including 2 items modified from the DOS (DOS #6, DOS #10) and 3 new items. Third, 5 items pertained to Criterion C (physical impairments resulting from nutritional deficiencies), with all items being new. Finally, 9 items pertained to Criterion D (psychosocial impairments resulting from the other criteria), including 5 items modified from the EHQ (EHQ #6, EHQ #8, EHQ #10, EHQ #14, and EHQ #2/17), 1 item modified from the DOS (DOS #4), and 3 new items.

Upon completing these items, participants were directed to a new page with the following description of ON provided: “Orthorexia is a term that is used to describe a condition that includes the following characteristics: (a) obsessive behaviors and mental preoccupation with healthy nutrition that includes rigidly following a restrictive diet with strict avoidance of foods believed to be unhealthy; (b) violations of these restrictive dietary rules resulting in extreme emotional distress with feelings of guilt, shame, and/or anxiety; (c) physical impairments, whereby nutritional deficiencies may lead to significant weight loss, malnutrition, and/or physical health complications; and (d) psychosocial impairments in social, vocational, and/or academic functioning that may result from the three aforementioned characteristics of the condition.” Below this description were two questions, one asking the participant how much they believe that those characteristics describe them, and the other asking them to think of the person who knows them best and to speculate what that person would say regarding how much those characteristics describe the participant. Both questions used a 5-point Likert scale ranging from “not at all” (1) to “completely” (5). Responses to these self-diagnosis and other-diagnoses rating questions were used only to help establish criterion-related validity. Unfortunately, because ON is not currently recognized as a disorder in the DSM-5 [2] or in the ICD-11 [3], researchers are limited in their ability to find individuals with a formal diagnosis of ON.

Disordered eating

The Eating Attitudes Test-26 (EAT-26) [60] is a frequently used measure to assess risk of an eating disorder. It includes 13 items that address dieting (e.g., “I avoid foods with sugar”, “I feel uncomfortable after eating sweets”), 6 items that address bulimia and food preoccupation (e.g., “I have the impulse to vomit after meals”, “I feel that food controls my life”), and 7 items that address oral control (e.g., “I display self-control around food”, “I cut my food into small pieces”). The response options for these items range from “never” (1) to “always” (6). For this study, Cronbach’s α was 0.93, demonstrating excellent internal consistency. In addition to completing the EAT-26, participants were also asked to indicate whether they had ever been diagnosed with the following eating disorders: anorexia nervosa (AN), bulimia nervosa (BN), binge eating disorder (BED), and avoidant/restrictive food intake disorder (ARFID).

Obsessive–compulsive tendencies

The self-report version of the Yale-Brown Obsessive Compulsive Scale (Y-BOCS) [61, 62] first defines obsessions followed by 5 multiple-choice questions (e.g., “How much of your time is occupied by obsessive thoughts?”, “How much distress do your obsessive thoughts cause you?”) and then defines compulsions followed by 5 multiple-choice questions (e.g., “How much do your compulsive behaviors interfere with your work, school, social, or other important role functioning?”, “How would you feel if prevented from performing your compulsions?”). The five response options are specific to each question, but on a scale such that higher responses/scores indicate higher levels of obsessive–compulsiveness. For this study, Cronbach’s α was 0.90, demonstrating excellent internal consistency. In addition to completing the Y-BOCS, participants were asked to indicate whether they had ever been diagnosed with OCD.

Compulsive exercise

Two scales of the Compulsive Exercise Test (CET) [63] were used in the current study. The CET Rule-Driven Behavior scale includes 8 items that assess the compulsive need to exercise (e.g., “I feel extremely guilty if I miss an exercise session”, “I usually continue to exercise despite injury or illness unless I am very ill or too injured”), and the CET Exercise Rigidity scale includes 3 items that assess the compulsive need to structure exercise activities [e.g., “My weekly pattern of exercise is repetitive”, “I follow a set routine for my exercise sessions (e.g., walk or run the same route, particular exercises, same amount of time, and so on)”]. The response options for all items range from “never true” (1) to “always true” (5). For this study, Cronbach’s α was 0.94 for the CET Rule-Driven Behavior scale and 0.85 for the CET Exercise Rigidity scale, demonstrating great internal consistency.

Depression

The Patient Health Questionnaire-9 (PHQ-9) [64] comprises 9 items that present depression-related problems (e.g., “little interest or pleasure in doing things”, “feeling bad about yourself—or that you are a failure or have let yourself or your family down”). Participants are asked to indicate the frequency with which they have been bothered by those problems during the last 2 weeks, using a Likert scale with response options ranging from “not at all” (1) to “nearly every day” (4). For this study, Cronbach’s α was 0.91, demonstrating excellent internal consistency.

Social desirability bias

The Lie Scale (LS) of the Eysenck Personality Questionnaire-Revised, Short Form (EPQ-R-SF) [65] has often been used as a measure of social desirability bias. It includes 12 “yes” or “no” questions asking about common or uncommon things (e.g., “As a child were you ever disrespectful to your parents?”, “Are all your habits good and desirable ones?”). The items are coded or reverse-coded accordingly, such that higher scores reflect a greater tendency to lie for the purpose of making oneself appear more socially desirable. For this study, Cronbach’s α was 0.69, demonstrating acceptable internal consistency.

Statistical analyses

Factor analysis, using the maximum likelihood extraction method and varimax rotation, was used to assess the factorial structure of the ON Inventory (ONI), and Cronbach’s α and test–retest reliability analyses were conducted to assess the reliability of the ONI and its scales. To assess how the ONI and its scales correlate with the continuous variables in this study, Pearson correlation analyses were conducted. To assess group differences on the ONI and its scales, Student’s t tests and Fisher’s ANOVA (with Tukey HSD post hoc tests) were conducted in cases of homogenous variances, and Welch’s t tests and Welch’s ANOVA (with Games-Howell post hoc tests) were conducted in cases of heterogeneous variances. A standard α of 0.05 was used to determine statistical significance. For effect size, r and ω were computed for the t tests and ANOVA, respectively, and Cohen’s [66] guidelines were followed for interpretation: 0.1 = small, 0.3 = medium, and 0.5 large. Finally, a regression analysis was used to determine the greatest predictors of ONI scores.

Results

Factorial structure of the ONI

Although four factors were hypothesized, the scree plot indicated the presence of only three factors. For comparison and per the recommendations of Costello and Osborne [67], two initial factor analyses were conducted (to test a three-factor model suggested by the scree plot and a four-factor model based on a priori hypotheses), both using maximum likelihood extraction and varimax rotation. The Chi square test was significant for both the three-factor model (χ2 = 1188.33, p < 0.001) and the four-factor model (χ2 = 955.24, p < 0.001), suggesting that either three or four factors would be adequate to explain the covariance among items. For both models, the same 10 items loaded onto the first factor (physical and psychosocial impairments), the same 9 items loaded onto the second factor (behaviors and preoccupation), and the same 5 items loaded onto the third factor (emotional distress). The only difference was that the four-factor model included a fourth factor with 2 of the 6 items that did not adequately load (i.e., loadings < 0.50) onto any of the factors from the three-factor model. We chose to go with the three-factor model because the physical and psychosocial impairment items loaded onto a single factor in both models, and because those two items loading onto a fourth factor did not seem conceptually related beyond both assessing ON.

Upon deleting the 6 items with factor loadings below 0.50, we conducted a final factor analysis, again using maximum likelihood extraction with three factors and varimax rotation. The solution was the same (see Table 2), with 10 items loading onto the first factor (physical and psychosocial impairments) accounting for 44% of the variance, 9 items loading onto the second factor (behaviors and preoccupation) accounting for 9% of the variance, and 5 items loading onto the third factor (emotional distress) accounting for 5% of the variance. The item communalities ranged from 0.37 to 0.66, suggesting that each item pertained to the same overall ON construct, a conclusion that is further supported by strong positive correlations among the composite scales for the sample of 847 people: the ONI Impairments scores correlated with the ONI Behaviors scores (r = 0.62, p < 0.001) and the ONI Emotions scores (r = 0.67, p < 0.001), and the ONI Behaviors scores correlated with the ONI Emotions scores (r = 0.68, p < 0.001).

Reliability and criterion-related validity of the ONI

As shown in Table 3, Cronbach’s α was 0.94 for the overall ONI and ranged from 0.88 to 0.90 for the three ONI scales, demonstrating good internal consistency. The 2-week test–retest reliability was also good at 0.91 for the overall ONI and ranging from 0.86 to 0.87 for the ONI scales. Further, helping to establish criterion-related validity, we found large-magnitude positive correlations between the ON self- and other-diagnosis ratings and the scores on the ONI and its scales. On a final note, the ONI and its scales’ scores were not significantly correlated with the Lie scale scores, suggesting that participants responded to the ONI items truthfully.

Table 3 Descriptive statistics and reliability of the ONI and its scales
Full size table

ONI associations with other variables

Regarding associations with health-related behaviors, the ONI and each of its scales were positively correlated with aerobic exercise, CET Exercise Rigidity scores, and CET Rule-Driven Behavior scores (see Table 4). In addition, the ONI and the ONI Behaviors scale were positively correlated with strength training exercise, albeit these correlations were small in magnitude. Contrary to the hypotheses, the absolute value of the correlations with alcohol consumption were all less than 0.1, and scores did not significantly differ between smokers and non-smokers.

Table 4 ONI relationships with health-related behaviors
Full size table

As expected, the ONI also exhibited associations with healthy eating behaviors regarding food consumption, with its overall score and each of its scales’ scores positively correlated with the consumption of vegetables and fruits, and negatively correlated with consumption of meat, dairy, sweets, and sweetened beverages (see Table 5), although the correlations between the ONI Impairments scores and the latter three variables failed to reach statistical significance. Likewise, regarding group differences for the ONI, the non-vegetarians (M = 38.52, SD = 12.30) scored lower than the semi-vegetarians (M = 47.16, SD = 16.70, p < 0.001), the vegetarians (M = 47.21, SD = 15.75, p = 0.006), and the vegans (M = 48.78, SD = 14.51, p < 0.001). None of the latter three groups significantly differed from one another (all p’s > 0.8). This same pattern of results was found for each of the ONI scales, although for the ONI Impairments scale, two of the differences did not reach significance (p = 0.21 for non-vegetarians vs. vegetarians, p = 0.12 for non-vegetarians vs. vegans).

Table 5 ONI relationships with eating behaviors
Full size table

Moving from being healthy to being unhealthy, the ONI was also significantly associated with levels of disordered eating, with large-magnitude positive correlations between the EAT-26 scores and the scores on the ONI and each of its scales (see Table 6). Likewise, regarding group differences, ONI scores were greater for those with a prior diagnosis of AN (M = 60.60, SD = 19.29) versus not (M = 39.18, SD = 11.72), greater for those with a prior diagnosis of BN (M = 60.62, SD = 16.85) versus not (M = 40.19, SD = 13.16), greater for those with a prior diagnosis of BED (M = 50.39, SD = 16.27) versus not (M = 40.66, SD = 13.73), and greater for those with a prior diagnosis of ARFID (M = 58.50, SD = 15.84) versus not (M = 39.99, SD = 13.11). These group differences also held for each of the ONI scales, with consistently large-magnitude associations with AN, BN, and ARFID.

Table 6 ONI relationships with disordered eating
Full size table

Regarding other clinical associations, the scores on the ONI and each of its scales were correlated with PHQ-9 scores and with Y-BOCS scores (see Table 7). The latter correlations, which were mostly large in magnitude, are consistent with group differences, whereby ONI scores were greater among people with a prior diagnosis of OCD (M = 51.90, SD = 19.36) versus those without this diagnosis (M = 40.43, SD = 13.30). These differences were also significant for each of the ONI scales.

Table 7 ONI Relationships with Other Clinical Variables
Full size table

Demographic associations

Regarding demographic relationships, age was negatively correlated with the ONI Impairments score and ONI Emotions score; and BMI was negatively correlated with the ONI total score, ONI Impairments score, and ONI Behaviors score (see Table 8), although all these correlations were small in magnitude. Regarding group differences, men and women did not significantly differ on any scale. Finally, we found that ONI total scores were greater among nutrition majors (M = 41.48, SD = 11.89) in comparison to psychology majors (M = 35.66, SD = 10.17), and these differences also held for the ONI Behaviors and ONI Emotions scales.

Table 8 ONI relationships with demographic variables
Full size table

Regression analysis

The previous results revealed that ONI scores were significantly correlated with key demographics (age and BMI), exercise activity (aerobic exercise, strength training exercise, CET Exercise Rigidity, and CET Rule-Driven Behavior), food and drink consumption (vegetables, fruits, meats, dairy products, sweets, and sweetened beverages), and clinical symptomatology (EAT-26, Y-BOCS, and PHQ-9). To determine the impact of each, as well as which variables are the strongest predictors of ONI scores, a regression analysis was conducted. The variables were entered in steps: demographic variables in Step 1, exercise variables in Step 2, food and drink variables in Step 3, and clinical symptomatology variables in Step 4. A histogram revealed normally-distributed residuals, and the residuals-by-predicted plot revealed a random dispersal of residuals around the horizontal axis. Moreover, although some of the predictor variables were related, multicollinearity was not an issue, with all tolerance values greater than 0.3 and all VIF values lower than 3. As indicated in Table 9, the addition of each set of predictor variables (i.e., exercise activity, food and drink consumption, clinical symptomatology) resulted in a significant change in R2. Inspection of the full model revealed that the best predictor of ONI scores was the EAT-26, followed by the CET rule-driven exercise behavior scale, and the Y-BOCS.

Table 9 Standardized coefficients from regression analysis
Full size table

Recommended “Diagnostic” criterion

The authors of the BOT [1], the ORTO-15 [47], and the DOS [6] all offer diagnostic criteria based on their scales. Although ON is not yet recognized as a disorder in the DSM-5 [2] or in the ICD-11 [3], and although diagnosis of this speculated disorder would also entail a clinical interview, providing such a scale criterion would be useful in making general comparisons across studies with different samples. For the 24-item ONI, a mean Likert rating of 3 (“mainly true”) out of 4 would equate to a minimum total score of 72 on the ONI. Using this criterion in the current study, 4.5% (38 out of 847) of the sample would be considered to have, or to at least be at high risk for, ON. That prevalence rate is comparable with the 1–8% prevalence rates found with the DOS [6, 14, 42, 55, 68].

Discussion

Factorial structure and reliability of the ONI

The purpose of the present research was to develop a more reliable measure of ON symptomatology that includes items assessing aspects of all four of the currently agreed-upon diagnostic criteria for ON [4,5,6,7]. Although four corresponding factors were hypothesized, only three emerged from the factor analysis, with items loading onto a physical and psychosocial impairments factor, a behaviors and preoccupation factor, and an emotional distress factor. These findings are consistent with the diagnostic criteria for ON, such that all criteria were represented. The only difference is that items assessing physical and psychosocial impairments loaded onto a single impairments factor rather than two separate factors. Reinspection of the items revealed that the physical impairment items included wording specific to physical health (e.g., “physical symptoms,” “illness,” “physical health”) and that the psychosocial impairment items included wording specific to social or vocational/academic functioning (e.g., “less time … with my family and friends,” “stress in my relationships,” “missed time at work or missed classes at school”). Perhaps the single factor with the physical and psychosocial impairment items reflects a more inclusive impairments factor representing the overall debilitating impact of the more extreme restrictive eating behaviors and obsessions with what they believe to be proper nutrition. This finding from the factor analysis may have implications for the diagnostic criteria for ON, such that these two criteria should be combined into a single impairments criterion, as is common in the DSM-5 [2].

Regarding reliability, with the current sample, the ONI demonstrated excellent internal consistency with a Cronbach’s α of 0.94 and excellent 2-week test–retest reliability with a coefficient of 0.91. It is worth noting that, based on the current sample, the ONI’s internal consistency and test–retest reliability are greater than that of previously used ON measures including the BOT (Cronbach’s α = 0.60–0.67) [23, 46], the ORTO-15 (Cronbach’s α = 0.14–0.83; r = 0.73) [48, 71], the EHQ (Cronbach’s α = 0.81–0.92; r = 0.72–0.81) [25, 31, 48, 54], and the DOS (Cronbach’s α = 0.80–0.88; r = 0.67–0.79) [6, 14, 55,56,57]. Moreover, the ONI is the only ON measure to include items assessing physical impairments, and the ONI includes more items assessing emotional distress than the other measures.

ONI associations with other variables

The current research also partially supports the validity of the ONI, based on its associations with other key variables. Regarding dietary practices, ONI scores were greater among vegetarians and vegans in comparison to non-vegetarian omnivores, consistent with findings from other studies using different measures of ON [8,9,10,11,12,13,14,15,16]. On the unhealthy side; however, ONI scores were positively correlated with the EAT-26 scores, consistent with the results of numerous studies indicating that greater ON symptomatology is associated with greater disordered eating [9, 15, 20,21,22,23,24,25,26,27,28,29,30,31]. It is worth noting that the EAT-26 was the greatest predictor of ONI scores in the regression analysis, supporting the notion that ON involves disordered eating above and beyond a simple commitment to healthy eating behaviors. Regarding other disordered functioning, ONI scores were positively correlated with PHQ-9 and Y-BOCS scores, consistent with the past research revealing that greater ON symptomatology is associated with greater levels of both depression [14, 45] and obsessive–compulsive tendencies [18, 20, 25,26,27, 31, 43,44,45]. Thus, the ON individuals’ obsessions with nutrition and compulsions with eating may be part of a more general neuropsychological personality profile characterized by negative affect and obsessive–compulsive tendencies. Regarding lifestyle variables, ONI scores were positively correlated with amount of exercise activity, replicating several previous investigations [29, 36,37,38,39,40]. In the full regression model; however, the only significant exercise predictor was the CET rule-driven behavior scale, suggesting that ON involves seemingly unhealthy exercise behaviors including a compulsion to follow a rigid pattern of intense exercise activities even in the case of injury, illness, or other problems [41, 42]. Regarding gender, ONI scores did not significantly differ between men and women, consistent with numerous studies showing that ON symptomatology is not greater in women than men [35], a finding that makes ON unique from AN.

Two finding from the current study were unexpected. First, regarding a “healthy” lifestyle, ONI scores did not significantly differ between smokers and non-smokers. Interestingly, another study also failed to find any differences based on smoking status [69], and two groups of researchers even found that ON symptomatology was greater in smokers than in non-smokers [24, 70]. Future research is needed to further investigate these unexpected findings and to perhaps additionally assess whether ON symptomatology is associated with addiction proneness. Second, regarding body mass, ONI scores were negatively correlated with BMI, although this relationship was no longer significant in the full regression model. Although multiple research studies show that ON symptomatology is not negatively correlated with BMI [34], the consensus diagnostic criteria include physical impairments with significant weight loss as a result of nutritional deficiencies [4,5,6,7]. The reason that the current study’s findings regarding BMI partially conflict with past research is that the ONI is the only scale to include items that specifically assess physical impairments that may result from an individual’s eating habits. Moreover, one of our goals with the ONI was to include items worded in such a way to better differentiate non-pathological healthy eaters from individuals with ON. Thus, we believe that this deviation from past research reflects that the ONI may be superior to the other ON instruments at measuring ON above and beyond a simple commitment to healthy eating.

Limitations

This research is not without its limitations. First, regarding the item selection process, we only relied on aspects related to the content of the items, selecting items that specifically relate to the consensus diagnostic criteria for ON. Although this method is suitable to improve content validity, it is also prone to biases due to the subjective choice of the items, which influences the interpretability of the final scale. Second, regarding the sample selection process, we chose to include in the sample participants who reported that they had previously been diagnosed with an eating disorder. Given that this self-reported diagnosis was not confirmed by a clinician, the analyses comparing participants with and without the disorder are subject to the self-report bias. In addition, it is up for debate whether it is helpful to include individuals with previous diagnoses in a sample used for the construction of a new scale. Third, still regarding the sample, the generalizability of the findings is limited due to the sample composition for the current study. Fourth, regarding the measure’s validity, because both researchers and clinicians lack a clear clinical standard for diagnosis, a signal detection analysis or other-diagnostic sensitivity measure for the ONI is not yet possible. Finally, regarding the proposed cutoff score for the ONI, the value is somewhat arbitrary and of questionable meaning beyond it representing a mean Likert rating of 3 for that particular person, corresponding to their self-report that the statements are overall “mainly true” for them. In the absence of a clear clinical standard for diagnosis, however, this value cannot be an indication of ON diagnosis. Rather, it should only be used to assess risk for ON and to make general comparisons for ON risk across different populations.

Conclusion

The ONI is a new, 24-item measure of ON symptomatology that demonstrated excellent internal consistency and test–retest reliability for this study’s sample. Moreover, in comparison to other measures, the ONI is the only one to include any items pertaining to physical impairments, which comprise one of the consensus diagnostic criteria for ON. Consistent with the past research using other ON measures, ONI scores were unrelated to gender; positively correlated with fruit and vegetable consumption and negatively correlated with meat and sweet consumption; and positively correlated with scores on measures of disordered eating, depression, obsessive–compulsive tendencies, and compulsive exercise activity.

What is already known on this subject?

The most commonly used ON measures are lacking either in reliability or in items assessing two important diagnostic criteria: emotional distress and physical impairments.

What does this study add?

The ONI is a new, reliable ON measure with items assessing all diagnostic criteria that have been agreed upon by researchers.