Introduction

US physician Steven Bratman first made the remarkable observation that some people were becoming so obsessive about eating healthfully that they might be causing themselves harm [1]. Bratman coined the term “orthorexia nervosa” (ON) to describe this fixation. He went on to write a book, Health Food Junkies, about people whose quest for the perfect healthy diet had become obsessional and dysfunctional [2]. Despite a favorable book review in the prestigious US medical journal JAMA supporting the inclusion of ON in the medical lexicon [3], the disorder is still not well understood [4]. The first peer-reviewed article on ON appeared in the literature in 2004. Donini et al. characterized ON as a “maniacal obsession” with the pursuit of healthy eating [5]. However, the first formal proposal for diagnostic criteria only appeared in 2015 [6]. While most of the literature is dominated with studies regarding prevalence rates in non-clinical samples, convincing case studies are also reported [6–9].

With increased interest in ON, many researchers have sought to detect its prevalence in a number of different samples. The first to do so was Donini and colleagues who constructed the “ORTO-15,” a measure based on 10 yes/no items originally constructed by Bratman in his 2000 book [2, 10]. Scores on the ORTO-15 of less than 40 are thought to be in the ON range, and using this cutoff score, the authors were able to achieve 100 % sensitivity, 73.6 % specificity, with a positive predictive value (PPV) of 17.6 % and a 100 % negative predictive value (NPV). Traditional data regarding the psychometric properties of the ORTO-15, such as data regarding reliability and validity, are not reported. Donini et al. report a prevalence rate of 6.9 % [10]. In another Italian study, Ramacciotti et al. report 57.6 % of a community sample of 107 scored in the ON range [11]. This group suggests using a cutoff score of 35, a number that gave them a “sensibly lower” prevalence rate of 11.9 %. A third Italian study reports prevalence rates of ON in 577 athletes, reporting one in three scoring in the ON range (using a cutoff score of 35) [12].

Additionally, the ORTO-15 has also been translated into Turkish, Portuguese, Spanish, and Polish. Bosi et al. translated the ORTO-15 into Turkish and administered it to 318 resident physicians; 45.5 % of them suffered from ON [13]. Alvarenga et al. used a complex multi-step translation method to translate both the Italian ORTO-15 items and the published English version into Portuguese [14]. The translated instrument was then administered to 392 Brazilian dietitians and found 81.9 % suffered from ON. Alvarenga et al. report believing that the cutoff score is inappropriate given such a high prevalence rate and wonder about the cultural differences between the original Italian instrument and a Brazilian sample [14]. Finally, Brytek-Matera et al. investigate the “ORTHO-15”Footnote 1 by a simple translation of the instrument from English into Polish [15]. After confirmatory factor analysis, they determined that nine items were useful, but kept the other six questions as buffer items. This translated and transformed instrument, the “Polish ORTHO-15”, was then used in a second study of 327 students; 68.6 % of women and 43.2 % of men scored in the ON range [16].

There are other instances in the literature where a group has translated the ORTO-15 into another language, modified its content, and generated a revised instrument. Arusoğlu et al. translated the English items of the ORTO-15 into a Turkish version and administered the resulting instrument to a non-clinical sample of 994 individuals. Their intent was to validate a Turkish version of the ORTO-15, so this group makes no statements about prevalence of ON [17]. They did, however, perform a confirmatory factor analysis on the ORTO-15 and found 11 items were statistically powerful. Four items were discarded and they proceeded with the “ORTO-11.” The ORTO-11 has been put to use evaluating 878 Turkish medical students. Fidan et al. found a prevalence rate of 43.6 % [18]. Similarly, a Hungarian group made concerted efforts to assure meaning and content of the ORTO-15 by translating the English items into Hungarian [19]. They found that 74.2 % of their sample scored below 40, in the ON range. Varga et al. report being surprised by this finding and raise concerns about the validity of the ORTO-15 [19]. After confirmatory factor analysis, Varga et al. decided that the instrument could be improved by discarding four items, naming the result the “ORTO-11-Hu.” Similarly, Missbach et al. translated the ORTO-15 into German and found 69.1 % of their sample showed “orthrectic tendencies” [20]. They also decided that six of the ORTO-15 items did not show sufficient factor loading, so discarded them, resulting in the ORTO-9-Ge [20]. Across all of these measures, all original ORTO-15 items are represented.

The point prevalence of ON presents as being impossibly high. Point prevalence rates of anorexia and bulimia nervosa are estimated to be no higher than about 2 % [21]. It is counterintuitive to believe that a phenomenon of restricted eating that is not well understood has point prevalence rates found to be as high as 88.7 %, with repeated findings of 30–80 %. The ORTO-15, and its derivatives in other languages, are all based on the ten-item yes/no questionnaire appearing in Bratman’s Health Food Junkies [2]. Bratman did not validate these items, and indeed, only included them as an afterthought and with the suggestion of an editor (Bratman, personal communication, May 2015). As others suggest, it is possible that such high point prevalence rates are due to a flaw in these items or in test construction of the ORTO-15, particularly since there are no traditional psychometric properties reported for this instrument [11, 19]. Additionally, cultural factors may be playing a role. Given that the ORTO-15’s development was strongly influenced by a US physician observing Americans, perhaps validating the ORTO-15 using an Italian sample helps explain questionable prevalence rates. A US sample might perform differently on the ORTO 15.

To our knowledge, data regarding the ORTO-15 from a US sample have never been published. Doing so would help identify whether Americans would have similar point prevalence rates for ON based on the ORTO-15. Nor has the ORTO-15 been administered to a US sample in conjunction with questions regarding health or conflict with others because of diet, or other features that would suggest an individual’s quest for healthful eating has become pathological. Accompanying data regarding disordered eating and attitudes toward healthful dieting may help quantify the phenomenon of ON.

Five research hypotheses have been formulated. First, given that atypically high prevalence rates for ON are found in samples outside of the United States, ORTO-15 data from a US sample would add to the literature on the occurrence of this phenomenon. A vastly different prevalence rate found in the US might suggest that the construct of pathologically healthful eating has cultural implications and that the ORTO-15 (based on its American heritage) is biased for international samples. The remaining hypotheses concern whether the ORTO-15 identifies constructs of pathological dieting, particularly dietary choices leading to medical problems or conflict with others. It is possible that high prevalence rates of ON based on ORTO-15 data are simply due to this instrument incorrectly identifying conscientious dieting as being harmful without also confirming accompanying pathology. One reason for this is that, to our knowledge, the ORTO-15 was constructed without the benefit of diagnostic criteria regarding ON. For this study, we followed the 2015 Moroze et al. criteria to identify ON [6], particularly the belief that those with ON hold healthy eating in very high regard, that they perceive their diet as being healthier than others, and that their dieting leads to life disruption, including medical problems and difficult interactions with friends or family. To summarize:

Hypothesis 1

Prevalence rates of ON based on ORTO-15 data from a US sample will be no different than from samples from Europe and South America.

Hypothesis 2

Significant group differences will be found using the ORTO-15 between those who endorse that healthy eating is very important, those who believe their diet is healthier than others, and those with medical problems or social conflict because of healthful eating, and those who do not present with any or all of the foregoing.

Hypothesis 3

Mean ORTO-15 scores will be significantly different between groups of participants who exercise at higher rates than those who do not, as well as those who restrict their diets and those who do not.

Hypothesis 4

The ORTO-15 will show acceptable sensitivity and specificity in identifying which participants are categorized as endorsing that healthy eating is very important, those who believe their diet is healthier than others, those with medical problems or social conflict because of their diet, and those in treatment for an eating disorder.

Hypothesis 5

The prevalence rates of identifying ON based on the Moroze et al. criteria will be comparable to a prevalence rate generated by the ORTO-15.

Method

Participants

Students from a medium-sized US university were recruited through email invitations and by research assistants sitting at a table at a campus fitness center. Of 294 participants, 19 were removed from data analysis. Of those 19, 10 identified as non-US citizens and nine did not complete the study protocol, leaving 275 participants.

Materials and procedure

Identification of ON was done through two means: first, the ORTO-15 was reproduced from the article introducing the scale [10]. As previously discussed, the ORTO-15 is an instrument that purportedly measures ON using 15 items with a 1–4 “always, often, sometimes, never,” scale. Higher scores on the ORTO-15 indicate more moderate dieting practices, with lower scores indicating ON. Data regarding reliability and validity are not reported in the article introducing the measure [10]. Second, all participants were also asked to complete a questionnaire regarding demographic information; diet and exercise; food preparation habits; and attitudes regarding healthy eating. These items were measured on a Likert scale. Participants were also asked whether dieting had led to health problems, conflict with friends/family, or treatment for an eating disorder. These questions were answered either “yes” or “no.” With direct data regarding healthful eating and possible problems because of it, this permits an evaluation of the sensitivity and specificity of the ORTO-15 to identify participants with possible ON.

After institutional review board approval regarding the ethical treatment of humans in research, 178 participants completed the ORTO-15 and questionnaire online. These participants were recruited through a research support initiative where email addresses of 400 random undergraduate students at a Rocky Mountain West US university were supplied to the first author. All online participants were sent an email explaining the study and the need for volunteers. A link to an online data collection website was included for those who wished to participate. Five days after the first email was generated, a follow-up invitation to participate in the study was sent. To assure having sufficient participants who were interested in physical health, an additional 97 participants were recruited as they entered or exited the same university’s campus fitness center. This was accomplished by having study authors and a research assistant set up a table in the fitness center lobby. Signs were created about a healthy eating study and participants were asked to participate by completing printed materials. Participants were asked not to complete the study twice. All participants gave their informed consent.

After data collection was complete, we considered how specific and sensitive the ORTO-15 was when it came to five areas: How important healthy eating was as rated by participants (“Healthy Eating is Important to Me”), whether the participant believed their diet was healthier than others (“My Diet is Healthier Than Others”), whether a participant endorsed that their diet led to health problems (“My Diet Has Led to Health Problems”), whether a diet led to conflict with friends or family (“My Diet has led to Conflict Friends/Family”), and whether the participant was in treatment or had been in treatment for an eating disorder (“Treatment for Eating Disorder”). For three variables, My Diet has led to Health Problems, My Diet has led to Conflict with Friends/Family, and Treatment for Eating Disorder, the data were already dichotomous, sorted by yes/no answer. Dichotomous data were created for Healthy Eating is Important to Me by placing those endorsing “Healthy eating is among the most important things in my life,” or healthy eating being rated as “important” or “very important” into a single group. Those endorsing “somewhat important” or “not at all important” were placed into a second group. Similarly, participants rating “Relative to other people, how healthy are your meals?” as being “significantly more healthy” or “more healthy” were placed into one group, with those responding as “no different,” “less healthy than others,” or “not nearly as healthy” into a second.

Finally, all participants classified both as Healthy Eating is Important to Me and My Diet is Healthier than Others categories were collapsed into a single group. It was then possible to see how many of the participants also endorsed that their diet has led to health problems, whether their dietary choices have led to conflict with friends/family, and those endorsing both. This helped identify participants who likely would meet most of the criteria for Moroze et al. conceptualization of ON [6].

Results

Of the 275 participants, 188 (68 %) identified as female and 2 (<0.1 %) identified as “other.” The mean age was 21.7 years (SD = 4.8). More than 75 % of the sample reported being in the third year of undergraduate study or beyond. When identifying ethnicity, 214 (78 %) reported being white, 41 (15 %) as Latino, 11 (4 %) as African-American, 5 (2 %) as Asian-American, and 4 (1 %) declined to answer.

The mean ORTO-15 score was 37.5 (SD = 4.4). Using the original cutoff score of 40, 71.2 % scored in the ON range, 22.1 % did when a cutoff score of 35 was applied.

Table 1 shows mean (and SD) scores across different demographic categories. Means testing for gender was conducted using an independent samples t test. The differences between men and women were not significant: t = (273) 1.37, p = 0.86, d = 0.41. A one-way ANOVA was used to compare mean ORTO-15 scores based on the type of identified eating habit; these differences were also not significant: F (4, 271) = 0.44, p = 0.79, η 2 = 0.04. Further, 80 % of the sample endorsed “no restrictions” when it comes to their food preference. Given a five-point difference of mean ORTO-15 score between those endorsing “No restrictions” to their eating, and those identifying as “Vegan,” a direct comparison between these two groups was made using an independent samples t test. This difference was significant, using a Bonferonni-corrected alpha level of 0.02: t (223) = 2.54, p = 0.01, d = 0.34. Finally, groups were created by examining the number of times per week of exercise. A one-way ANOVA was used to compare mean ORTO-15 scores. There were no significant differences between groups: F (4, 271) = 1.97, p = 0.08, η 2 = 0.03.

Table 1 Mean (and standard deviation) ORTO-15 scores by group
Full size table

Table 2 shows the number of participants endorsing Healthy Eating is Important, My Diet is Healthier than Others, My Diet has Led to Health Problems, My Diet has Led to Conflict with Friends/Family, and Treatment for an Eating Disorder. Additionally, the mean (and SD) ORTO-15 scores for each group are also reported. Significance testing based differences of ORTO-15 scores between participants who endorse healthy eating, having a healthy diet, those who endorse health or family problems due to diet, as well as those in treatment for an eating disorder was conducted using independent samples t tests. Those results are also reported in Table 2. Participants endorsing Healthy Eating is Important to Me had significantly lower ORTO-15 scores, as did those who indicated that their Diet is Healthier than Others. However, when addressing whether there may be disordered eating among the sample, there were no significant differences based on mean ORTO-15 score between those who did and did not endorse whether their diet had caused medical problems, whether their diet was the source of conflict with friends/family, or whether the individual had ever been in treatment for an eating disorder.

Table 2 Number of participants by category, mean (SD) ORTO-15 score, t (degrees of freedom), p, and Cohen d values
Full size table

By having dichotomous data, analysis for sensitivity and specificity was possible, as well as positive predictive and negative predictive values. These values were calculated using both a 40 and 35 cutoff score for the ORTO-15 and can be found in Table 3. When reviewing sensitivity and specificity of the ORTO-15 to classify membership in the Healthy Eating is Important to Me category, the sensitivity of the instrument is in an acceptable range, with higher sensitivity at a cutoff score of 35. Specificity, however, is quite poor. PPV and NPV vary widely as a function of cutoff score. When classifying membership to the Diet Is Healthier than Others category, the ORTO-15 is moderately sensitive at both cutoff scores, with slightly better specificity. Again, great variability is seen with PPV and NPV. Finally, on the remaining categories, Diet Health Problems, Diet Friends/Family Problems, and Eating Disorder, the ORTO-15 had abysmal sensitivity and high specificity at both cutoff scores. PPV and NPV scores followed the same pattern as with the previous categories.

Table 3 Sensitivity, specificity, positive predictive value, and negative predictive value by group
Full size table

Last, three additional groups were considered by first identifying participants classified as both “Healthy Eating is Important to Me” and “My Diet is Healthier than Others” and collapsing them into a single group. It was then possible to see whether participants falling in both of these categories also endorsed either “My Diet has Led to Health Problems” or “My Diet has Led to Conflict with Friends/Family.” Presumably, this would identify those who had pathological diet problems and were also serious about healthful eating consistent with the Moroze et al. criteria for ON [6]. These data are given in Table 4. Only a small number of those in the sample who endorse importance of healthy eating (by falling into both the “Healthy Eating is Important to Me” and the “My Diet is Healthier than Others” categories) are also identified with possible dysfunction of either conflict with others about their diet or health problems. Indeed, less than half of 1 % of the sample cites both problems with their health and conflict with friends/family. It is likely that, based on the Moroze et al. criteria [6], those suffering from ON would have issues both with their health status because of their diet and conflicts with family and friends.

Table 4 Number (and percent of sample) for those classified as both “Healthy Eating is Important to Me” and “Diet Healthier Than Others” and also endorsing either “My Diet has Led to Health Problems” or “My Diet has Led to Conflict with Family/Friends”
Full size table

Discussion

Consistent with other studies using the ORTO-15, a sample of US college students shows an extraordinarily high point prevalence rate for ON when compared against established eating disorders. That 71 % of the sample scores below 40 is consistent with many other studies that have been conducted in other countries, including those in Turkey [17], Chile [22], Poland [15], Germany [20], and Italy [11]. This gives support for Hypothesis 1, suggesting that it is unlikely that cultural differences alone can account for stable findings of high point prevalence rates for ON with studies that use the ORTO-15 outside the United States. It is more likely that, as some suggest [11, 14, 19], there may be issues with the validity of the ORTO-15.

Flaws in test construction could certainly account for the counterintuitive findings in this study, although several researchers have created instruments based on the ORTO-15 that are believed to have improved psychometric properties [16, 17, 19, 20] that also report high point prevalence rates. In this study, it is remarkable that 80 % of the sample endorse that they do not restrict the type of food that they consume, yet the ORTO-15 identifies over 70 % of the sample suffering from ON. More surprising is that when comparing sub-groups of participants based on diet and exercise, there are no significant differences based on ORTO-15 score. However, our curiosity was piqued with the unlikely finding that the small subgroup of participants (n = 6) identifying as having a vegan diet had the least pathological ORTO-15 score (42.5). Therefore, a second analysis (with a Bonferonni correction) comparing vegans to those endorsing no restrictions with their diet was run. This showed that the vegan group had a significantly higher mean ORTO-15 score (and away from the ON range) than the group endorsing no particular diet. This is extraordinary considering one would intuitively believe that individuals with highly restricted diets scored in a healthier direction on the ORTO-15 than those with those who eat a broad diet with many food types. This tends to suggest that the ORTO-15 is flawed. Indeed, Hypothesis 2 was not supported.

There are significantly lower mean ORTO-15 scores, however, when examining groups of individuals who endorse healthy eating as being important and those who perceive their diet as being healthier than others. This suggests that the ORTO-15 is identifying features of ON. These significant differences, however, disappear when comparing mean ORTO-15 scores based on groups identified as having a diet that has led to health problems, conflict, or with an eating disorder. It is likely that the ORTO-15 is effective in identifying individuals who are serious about healthy eating, but ineffective when also identifying individuals whose healthful dietary choices are associated with pathology. Mixed support is found for Hypothesis 3.

There are challenges when it comes to deciding whether a particular behavior is merely unusual or extreme, but not pathological. Certainly, it would not be prudent to suggest that individuals who follow a strict diet to be healthy are in danger. Instead, common practice determining “clinical significance” is whether the behavior is interpersonally distressing, or causes impairment in daily areas of functioning, such as at work, among friends/family, or at school [23]. Both Moroze et al. and Dunn and Bratman have proposed diagnostic criteria that suggest for healthy eating to cross the line into a clinically significantly syndrome, there should be some evidence of pathology, such as medical issues secondary to dieting and conflict with others due to dietary choices [6, 24]. For this study, proxy categories for ON were created by identifying individuals who endorsed both a seriousness about healthful eating and whether their diet caused medical or social problems. A very small number of persons endorsed both high expectations about their dieting while also endorsing either social conflict or medical problems secondary to dieting. The ORTO-15 performs poorly when used to classify individuals into such categories. Given that the ORTO-15 reliably sorts participants who value healthy eating from those who do not, it is likely that the ORTO-15 detects healthful eating, but not pathological dietary restriction. Support was not found for Hypothesis 4.

When considering whether eating behavior was clinically significant, far fewer participants meet criteria than the ORTO-15 classifies as having ON. Indeed, our sense is that less than one half of 1 % of this sample suffers from ON. Slightly more than that, about 10 %, have behaviors that place them at risk for ON. It is quite noteworthy that Lopes and Kirsten, one of the few studies regarding the prevalence of ON that does not use the ORTO-15 (or a derivative of Bratman’s original yes/no items) found a 0 % prevalence rate for ON with 200 Brazilian college students when using open-ended questions generated from the “scientific literature” [25]. Prevalence rates based on the ORTO-15 are likely quite inflated and should be considered very carefully. No support was found for Hypothesis 5.

Our study has a number of limitations. Among them is that body mass indexes (BMI) of the subjects were not measured. Elevated BMIs have been associated with increased risk of ON [18]. Assessing exercise was limited in that its assessment was only by self-report, a validated test for exercise and physical activity was not used, nor was there probing further into the type, intensity, and motivation leading to that exercise. Further, despite literature suggesting co-occurring psychopathology with ON, there was no investigation regarding obsessive–compulsive behavior, anxiety, or symptoms along the psychotic spectrum [10]. Similarly, the questions we asked about attitudes toward diet, healthful eating, and clinical significance, while based on the Moroze et al. criteria for ON [6], were not given to participants in a manner that formally mapped onto their diagnostic features. Finally, there was no assessment of socioeconomic status, racial, or religious backgrounds in consideration when conducting our analysis.

Given that the ORTO-15 (and the other measures that are based on it) is likely detecting those who wish to follow healthy diets, but is not sensitive to pathology, additional research in this area is needed. We would suggest that further study regarding prevalence rates among community samples using the ORTO-15, however, will not add to the literature. Nor will further development of the ORTO-15 into other languages and countries, as has been the trend for the last decade. Constructing a valid instrument that can reliably detect ON is a high priority, but so is the need for additional research conducted in clinical settings. Qualitative studies of the shared experiences of those with pathologically healthful eating could help sharpen diagnostic criteria. Measures that follow traditional principles of test construction would be very beneficial, as well as identifying treatment modalities that address orthrectic practices, rather than following regimens typically reserved for those who have anorexia nervosa. Finally, exploration of the differences and similarities of ON with the established DSM-5 eating disorder of avoidant/restrictive food intake disorder should be examined.

The case studies regarding ON are quite compelling. Additionally, discussion in the lay press suggests that a significant number of individuals suffer from ON [24]. It is assertion that there are individuals with pathological diets who maintain a belief that they are being healthy, but whose behavior may lead to significant health issues and/or life-disrupting events. It is prudent for eating disorder clinicians to be familiar with ON and to determine whether treatments typically reserved for anorexia nervosa are effective in those whose body weight is significantly low due to a drive for healthy eating as opposed to a diet based on disordered sense of body weight or shape.