Introduction

Orthorexia nervosa (ON) is characterized by an obsession or pathological fixation with a diet considered healthy. In is work, Bratman, who first explored ON in 1997 [1], explains that this eating-related difficulty starts of as a simple interest in healthy eating, and through time develops into a pathological one [2]. Unlike individuals with anorexia nervosa, individuals with ON do not want to lose weight, or at least that is not the focus of their diet. It is the pursuit of what the individual considers healthy eating that makes ON, meaning that the focus is on the quality of the food, not the quantity [3,4,5].

An essential part in the development of ON is the presence of associated rules and behaviours (orthorexic behaviours). These orthorexic behaviours could be anything that is associated with the individual’s efforts for healthy eating, from the time and energy spent on purchasing, preparing and eating food, to the obsessive thoughts, guilt, punishments and restrictions imposed, or even the colour of the food consumed [1, 2, 5, 6]. ON is linked to clinical impairment or distress, with some studies reporting cases where physical and mental health was compromised [5, 7, 8]. Regarding its prevalence, previous research reported rates vary extremely, from less than 1% to more than 80% [9], however this variance may be explained by the use of flawed assessment methods [9] or by the differences in the assessed samples. Research investment to better understand ON is essential.

Though recent literature has focused on ON since its first mentioning in 1997 [1]; its definition and recognition as a mental disorder are still disputed [10]. In fact, this lack of consensus is the reason why assessment tools regarding ON and its behaviours are still scarce and, more importantly, so different from one another [11].

In 2019 [11], a systematic review of all the existing measuring tools for ON was published. The authors found six different tools that assess the presence of ON: the Orthorexia self-test (BOT; [12]), the ORTO-15 test [13], the Eating Habits Questionnaires (EHQ; [14]), the Dusseldorf Orthorexie Scale (DOS; [15]), the Barcelona Orthorexia Scale (BOS; [16]) and the Teruel Orthorexia Scale (TOS; [17]). From these tools, the ORTO-15 has been widely used, having been adapted for at least six different languages. However, what was once considered the most reliable tool to assess ON, has raised concerns regarding its validity, reliability, and internal consistency [18,19,20]. It is important to add that, despite this, recent studies have explored and validated the revision of the ORTO-15, the ORTO-R, with success [21]. However, for the Portuguese population, a valid ON assessment tool still does not exist. A recent study [22], attempted to evaluate the psychometric properties of the Portuguese version of the ORTO-15, but the psychometric evaluation failed to provide proof of validity and consistency.

Recently, validation and adaptation studies using the DOS have had great success [15, 23, 24]. Therefore, the aim of this study, was to translate and validate the DOS for the Portuguese population, possibly offering a reliable and essential tool for the research and clinical development of ON in Portugal.

Additionally, since ON has been associated with disordered eating [25, 26] we explored DOS associations with different body and eating-related indicators. Furthermore, ON has been linked to psychological and clinical distress [5, 25], so other psychopathological indicators (e.g.: stress, anxiety, depression and shame) were also studied on their relationship with DOS.

ON has been studied in terms of its relationship with different sexes and BMI, with results being inconsistent [5, 27, 28] so, in order to expand our research contribute, the study explored these relationships as well.

Finally, ON’s relationship and association with different dietary patterns has been an important research focus, and diets like veganism and vegetarianism can be seen as possible risk factors [5]. Once again, results are incongruous [29, 30], so the present study explored the differences regarding the prevalence of ON, assessed by DOS, in groups with different dietary patterns.

Materials and methods

Participants

To test the factor structure and psychometric properties of DOS a sample of 513 participants (sample 1) was analyzed, comprised of 454 women and 59 men from the Portuguese general population. The age of this sample ranged from 18 to 62 (M = 27.65; SD = 9.16), with a mean of 14.84 (SD = 2.97) years of education. The participants’ Body Mass Index (BMI) mean was 23.44 (SD = 4.43), which corresponds to a normal BMI according to WHO [31]. This sample was also used to explore DOS’s prevalence, sex and BMI differences and DOS’s association with different psychopathology indicators.

Additionally, another sample was used (sample 2) to understand the differences and similarities between dietary patterns in their relationship with DOS and to explore DOS’s relationship with body and eating-related indicators. This sample consisted of 541 participants (447 women) from the Portuguese population, with an age mean of 34.66 (SD = 11.81) and a mean of 14.43 (SD = 4.14) years of educations. The BMI average corresponded to a normal BMI (M = 24.33; SD = 4.57; [31]).

Procedures

DOS’s translation and adaptation process for the Portuguese population began after authorization was granted from the authors of the original scale [15]. Firstly, a bilingual researcher translated the scale. Then, another bilingual researcher back-translated the scale, in order to compare its’ accuracy. After this process, a first sample of the Portuguese adaptation of DOS was drafted; 12 college students completed this draft and reviewed it. The students revealed no difficulties completing the scale, so only minor wording adjustments were made. Following these steps, the final version of the DOS’s Portuguese adaptation was elaborated.

The data collection for both samples was conducted in identical ways. All ethical requirements regarding the present study were respected, receiving the approval of the Ethics Committee of the Faculty of Psychology and Educational Sciences of the University of Coimbra. Participants were invited to enter in the study through different online social networks, by clicking the survey link. The voluntary and confidential nature of the study was informed immediately, and participants could stop their participation at any time. Individuals who agreed to take part in this research gave their written informed consent before completing the questionnaire, which took approximately 20–25 min. The questionnaires were built so that participants could not skip questions, meaning that there were no missing data.

Sample 1: the final sample corresponded to the initial sample, since no participant was younger than 18 years old or older than 65, the only exclusion criteria applied.

Sample 2: the initial sample comprised 560 participants but exclusion criteria were applied, excluding (a) participants out of the 18–65 years old range and (b) participants who did not identify with one of the following dietary patterns: omnivore, vegetarian, vegan or paleo. This corresponded to 3.39% of the initial sample being excluded from the study.

Instruments and measures

Düsseldorf Orthorexie Scale (DOS; [15]). DOS assesses the presence of ON and orthorexic behaviours, through a 10-item self-report questionnaire (e.g.: “I can only enjoy eating foods considered healthy”; “I have certain nutrition rules that I adhere to.”). The scale is ranked by a four-point Likert scale (from 1 = “never” to 4 = “always”), and a higher total score (in a maximum of 40) corresponds to the presence of higher levels of orthorexic behaviors. The maximum score is 40, and the preliminary cut-off point is 30. Scores between 25 and 29 indicate possible risk of ON. DOS presented good psychometric properties in its original study (α = 0.84).

Depression anxiety and stress scales-21 (DASS21; [32, 33])

DASS21 is a self-report measure that assesses psychopathology symptomology. The scale consists of three subscales (depression, anxiety and stress) each with 7 items. Participants answer according to a 4-point Likert scale (from 0 = “Did not apply to me at all” to 3 = “Applied to me very much or most of the time”). The subscales – depression (e.g.: “I felt down-hearted and blue.”), anxiety (e.g.: “I felt I was close to panic.”) and stress (e.g.: “I found it difficult to relax.”)—presented good psychometric properties. The subscales Cronbach’s alpha values in the original version were 0.88, 0.82, and 0.90; and 0.85, 0.74, and 0.81 in the Portuguese version, respectively.

External and internal shame scale (EISS; [34])

EISS is constituted by 8 items that assess the experience of external (4 items; e.g.: “Other people see me as not being up to their standards.”) and internal (4 items; e.g., “I am different and inferior to others.”) shame. Participants answer according to a 5-point Likert scale (0 = “never” to 4 = “always”) with higher scores representing more accentuated experience of shame. In the original Portuguese version EISS presented a Cronbach’s alpha of 0.89.

Body image shame scale (BISS; [35])

BISS measures body image shame in its’ internal and external dimensions. Constituted by 14 items—7 items for internal shame (e.g.: “There are parts of my body that I prefer to hide.”) and 7 items for external shame (e.g.: “I feel uncomfortable in social situations because I feel that people may criticize me because of my body shape.”), answered according to a 5-point Likert scale (from 0 = “never” to 4 = “almost always”). The scale presented good psychometric properties, with a Cronbach´s alpha of 0.92 in the original Portuguese version.

Inflexible eating questionnaire (IEQ; [36])

IEQ measures the inflexibility of eating-related rules. It consists of an 11-item scale (e.g.: “I feel proud when I can rigorously follow certain food rules.”), where higher values reflect more inflexibility regarding dietary rules. The answers are evaluated according to a 5-point Likert scale (from totally disagree (1) to totally agree (5)). IEQ has good psychometric qualities (Cronbach's alpha is 0.90 for the Portuguese population).

Intuitive eating scale -2 (IES-2; [37, 38])

IES-2 is a self-report scale that measures intuitive eating, or ones’ conscious perception of hunger and satiety, leading to the ability to understand internal physiological signs. The scale consists of 23 items (e.g.: “I rely on my hunger signals to tell me when to eat.”), with answers ranging from 1 (“Strongly Disagree”) to 5 (“Strongly Agree”) and higher scores represent higher levels of intuitive eating. With a Cronbach’s alpha of 0.87 in the original study and a Cronbach’s alpha of 0.97 in the Portuguese version, IES-2 has presented good psychometric characteristics.

Binge eating scale (BES; [39, 40])

BES evaluates the presence and severity of different attitudes and behaviours regarding binge eating. It consists of 16 items (e.g.: “At times, I tend to eat quickly and then, I feel uncomfortably full afterwards.”), each with 3 or 4 options, in which the individual must indicate the one they identify with the most. A more superior score, in a maximum of 46, represents higher severity of binge eating, with the scores above 16 as the cut-off clinical indicator. BES has good psychometric qualities with a Cronbach's alpha of 0.85 in the original study and with a Cronbach's alpha of 0.88 for the Portuguese one.

Eating disorder examination questionnaire (EDE-Q; [41, 42])

EDE-Q is a self-report measure, with 36 items. The scale assesses disordered eating attitudes and behaviours in the last 28 days. The questionnaire provides four different subscales, these being: Restraint (e.g.: “Have you been deliberately trying to limit the amount of food you eat to influence your shape or weight?”; five items), Eating Concern (e.g.: “Have you had a definite fear of losing control over eating?”; five items), Shape Concern (e.g.: “Have you had a definite desire to have a totally flat stomach?”; eight items) and Weight Concern (e.g.: “Have you had a strong desire to lose weight?”; five items). Subscales are ranked by frequency and severity, according to a 7-point Likert scale. Additionally, the total score provides a final measure regarding disordered eating. In the original study and in the Portuguese version EDE-Q presented good psychometric properties (α = 0.97).

Body mass index (BMI)

Body mass index was calculated by dividing the self-reported current weight (kg) by the height squared (m).

All measures presented good psychometric properties, with Cronbach’s alpha ranging from 0.86 to 0.96 (Table 2).

Data analysis

All data were analyzed using SPSS software version 22 (Chicago, IL) and AMOS software [43], with p values of < 0.05 indicating statistical significance. With the aim of exploring the dimensionality of the Portuguese version of the DOS, first order confirmatory factor analyses (CFA), with maximum likelihood estimation method, testing the theoretical model proposed by the original authors [15]. This CFA was conducted in sample 1. Model fit was assessed through several goodness-of-fit indicators, such as: the overall model Chi square (χ2), Goodness-of-Fit Index (GFI) > 0.90, Adjusted Goodness-of-Fit Index (AGFI) > 0.90, Comparative Fit Index (CFI) > 0.90, Tucker and Lewis Index (TLI) > 0.90, the (Standardized) Root Mean Square Residual (SMSR) < 0.08 and Root Mean Square Error of Approximation (RMSEA) < 0.08 [44]. Local adjustments indices were explored, and the indicators used were the standardized regression weights (SRW) and the squared multiple correlations (SMC). Moreover, Cronbach’s alpha values, item-total correlations and Skewness and Kurtosis were calculated.

To explore DOS’ relationship with other measures (general psychopathology and body and eating-related indicators) product-moment Pearson correlations analyses were conducted where correlations ranging between 0.1 and 0.3 were considered weak, above 0.3 to 0.5 moderate and strong when equal to or superior than 0.5, considering a significance level of 0.05 [45]. ON’s prevalence was also explored.

Finally, sex differences on ON were examined through a t-test for two independent samples, while BMI and dietary pattern differences on ON were examined through a One-way ANOVA.

Results

Confirmatory factor analysis

To confirm the DOS’s one-dimensional structure, a CFA was conducted (Sample 1; N = 513). Results indicated a significant chi-square goodness of fitness (χ2(35) = 419.882, p < 0.001), but since this fit index has been regarded as leading to biases in results due to sample size [46], we considered other goodness of fit indices to attest for the adequacy of the structure under analysis. Results suggested a poor model fit (GFI = 0.86; AGFI = 0.78; CFI = 0.80; TLI = 0.74; NFI = 0.79; SMSR = 0.08; RMSEA = 0.15). The analysis of the modification indices suggested the correlation of the errors of items 6 and 10 (206.426) and 1 and 3 (57.972), which resulted in an improvement of the model to good fit (χ2(33) = 119.982, p < 0.001; GFI = 0.95; AGFI = 0.92; CFI = 0.96; TLI = 0.94; NFI = 0.94; SMSR = 0.04; RMSEA = 0.07).

Local adjustment indicators analysis confirmed the adequacy of the DOS (Table 1). All items revealed adequate standardized regression weights (SRW), which ranged from 0.50 (item 1) to 0.79 (item 8). Thus all values were above the recommended cut-off point of 0.40 [47]. The individual items reliability was also corroborated through the values of the squared multiple correlations (SMC; with a recommended minimum cut-off point of 0.25 [47] which varied between 0.25 (item 1) and 0.63 (item 8).

Table 1 Items’ means (M), standard deviations (SD), standardized regression weights (SRW), squared Multiple Correlations (SMC) and Cronbach’s alpha if item deleted (α if item deleted) in sample 1 (N = 513)
Full size table

Regarding reliability, DOS revealed a Cronbach’s alpha of 0.86, disclosing a good internal consistency [48]. Additionally, the elimination of any item would not increase the scale reliability, suggesting that all items are relevant. Item-total correlations were also high, ranging from 0.48 to 0.72 (Table 1).

DOS items’ skewness values varied between − 0.30 (item 2) and 2.39 (item 7), and kurtosis values ranged from -1.38 (item 10) to 5.12 (item 7) indicating no severe violation of normal distribution (Sk <|3| and Ku <|10|; [44]).

DOS relationship with general psychopathological and body and eating indicators

Product-moment Pearson correlation coefficients (Table 2) revealed positive weak correlations between DOS and different general psychopathological indicators (DASS21 Depression, Anxiety and Stress subscales and EISS).

Table 2 DOS’s correlations with psychopathological indicators (sample 1; N = 513) and body and eating indicators (sample 2; N = 541) and Cronbach’s alphas
Full size table

Correlations were also explored in Sample 2, regarding body and eating indicators (BISS, IES-2, IEQ, BES and EDE-Q). Results revealed positive weak associations with BISS and BES, positive moderate association with EDE-Q, a positive and strong association with IEQ and a negative weak association with IES-2 (Table 2).

ON prevalence in the Portuguese population, assessed by DOS

Using the original version’s cut-off points [15], where 30 is the clinical indicator and scores from 25 to 29 indicate risk of ON, the prevalence of ON was explored in Sample 1.

The total sample DOS mean score was 20.73 (SD = 6.60). Results found that 10.52% of participants scored 30 or higher, 15.01% scored between 25 and 29, and 74.46% scored lower than 25.

Sex differences on ON

To examine sex differences, a smaller group with similar demographic characteristics (t(163)age = 0.538, p = 0.591; t(163)education = -2.410, p = 0.017) to the male sample (n = 59) was randomly selected from the total female sample. This smaller female group included 106 women.

Results indicated that women presented higher scores of orthorexic tendencies (M = 20.75; SD = 6.94), in comparison to men (M = 16.75; SD = 5.18), and these differences were statistically significant (t(149.40) = -4.196, p < 0.001). The magnitude of these results was calculated (d = 0.65) and interpreted considering Cohen’s guidelines [50], where values equal or superior to 0.5 correspond to a moderate magnitude.

BMI group differences on ON

With the aim of exploring BMI differences regarding ON, Sample 1 was divided into four BMI categories, following the BMI category ranges defined by WHO (2011). The four categories were: Underweight (n = 27), Normal weight (n = 351), Pre-obesity (n = 95) and Obesity (n = 40).

The groups’ DOS means were the following: Underweight (M = 23.85; SD = 10.39), Normal weight (M = 20.77; SD = 6.65), Pre-obesity (M = 20.00; SD = 5.62) and Obesity (M = 20. 73; SD = 4.32). These differences were analyzed through a one-way ANOVA and results found no significant differences [F(3, 509) = 2.58, p = 0.053].

Differences between dietary patterns on ON

Sample 2 was used to compare the differences between four dietary patterns regarding ON, assessed by DOS.

One-way ANOVA was conducted to compare the levels of orthorexic tendencies on different dietary patterns: Omnivores (n = 357), Vegetarian (n = 66), Vegan (n = 60) and Paleo (n = 58). Results found a significant effect regarding the four different patterns [F(3, 537) = 21.51, p = 0.000]. Post hoc comparisons using the Tukey HSD test showed that the mean score for the Omnivore group (M = 16.94, SD = 6.08) was significantly different from the Vegetarian group (M = 20.39, SD = 5.31), from the Vegan group (M = 21.65, SD = 7.29) and from the Paleo group (M = 22.03, SD = 6.05). These results show that Omnivores present significantly lower levels of orthorexic tendencies when compared with the other dietary patterns.

Discussion

The aim of the present study was to further extend the knowledge on ON. In order to do this, several options for measuring ON were explored, concluding that the DOS is a measure with overall past great results in different studies and validations [15, 23]. This is the first study, as far as we know, to successfully adapt and validate an ON measure for the Portuguese population. Besides this important validation, the study also explored ON’s relationship with different general psychopathology and body and eating indicators. Finally, ON’s prevalence and differences regarding sex, BMI and dietary patterns were also reported.

A CFA was conducted to test DOS’s structure, and initially results did not reveal a good model fit. However, after correlating items 6 and 10 and items 1 and 3, a great model fit was achieved. When examined closely, items 6 (“If I eat something I consider unhealthy, I feel really bad.”) and 10 (“I feel upset after eating unhealthy foods.”) are indeed predicted to present such high correlation, since both items refer to negative feelings after eating “foods considered unhealthy”. Regarding items 1 (“Eating healthy food is more important to me than indulgence/enjoying the food.”) and 3 (“I can only enjoy eating foods considered healthy.”), these do not present such high correlations since they do not directly assess the same characteristic or dimension of ON. However, these items seem to indicate the “need to put eating healthy above pleasure and enjoyment”, explaining the correlation found between them. In the future, these high correlations should be further explored, and possibly proceed to the adaptation of the mentioned items. With a Cronbach’s alpha of 0.86, the scale’s internal consistency was very good and local adjustments revealed items’ robustness.

Previous studies have been inconsistent regarding the prevalence of ON in the general population, with results ranging from less than 1% to more than 80% [9]. In the present study the prevalence of ON, assessed by DOS, using the original study’s cut-off point [15], was 10.52%. Despite significantly lower than several previous studies, these results are coherent with the ones obtained by the English adaptation of DOS (8%; [23]). Even so, this prevalence rate seems to be very high for the general population, especially when considering the prevalence rates of other eating disorders in the general population (e.g.: Anorexia nervosa (0.21%), bulimia nervosa (0.81%) and binge eating disorder (2.22%); [50]). This may be explained by the lack of consensus regarding the proper definition of ON and to the different assessment tools used.

DOS was positively linked with all general psychopathology and body and eating indicators in the study, including measures of depression, anxiety and stress, shame, body-image shame, inflexible eating, intuitive eating, binge eating and disordered eating. The correlation with IEQ was strong, which is not surprising since IEQ measures inflexible eating attitudes and behaviours (e.g.: “Eating according to certain rules gives me a sense of control.”; “To me, having a balanced eating plan means fulfilling certain rules rigorously”), which is an important characteristic of orthorexic behaviours. Additionally, the correlation of DOS with EDE-Q was moderated. This is consistent with previous literature [25, 26] but, as stated by Mccomb & Mills [5], it is still unclear whether ON is a risk factor for disordered eating or if disordered eating predicts the development of ON.

To further extend this study’s research contribute, we explored ON’s differences regarding sex, BMI and dietary pattern. Sex differences were significant, with women presenting higher levels of orthorexic tendencies than men. Even though previous studies have been inconsistent regarding sex differences in ON, this result is not surprising since women have been shown to be at higher risk of developing eating-related difficulties. Similarly, Strahler [28], in a systematic review of sex differences in orthorexic behaviours, found that pathologically healthful eating is slightly more pronounced in women. Regarding ON’s differences in BMI, sample 1 was divided into four groups (underweight, normal weight, pre-obesity and obesity) according to the guidelines of WHO [31], and no significant differences between the groups were found. Nevertheless, the group with higher levels of orthorexic tendencies was the underweight group. Previous results regarding BMI’s relationship with ON have been inconsistent. Our results might indicate that ON is equally present in all BMI categories, which might indicate that ON is uncorrelated to weight. Finally, sample 2 was divided into four different dietary patterns (omnivore, vegan, vegetarian and paleo) and significant differences were found regarding orthorexic tendencies, assessed by DOS. The vegan, vegetarian and paleo group had significant higher levels of orthorexic tendencies when compared with the omnivore group. Vast research exists regarding ON’s relationship with vegan and vegetarian patterns, showing that individuals with these patterns have greater tendency to develop ON [5], which confirms our results. Still, this is one of the first studies that includes individuals who identify as paleo, and shows that orthorexic tendencies for this group are similar to those who identify with veganism/vegetarianism.

These results should be interpreted considering certain limitations. Despite the considerable sample sizes, they are not representative of the total Portuguese population. Future studies should explore these results in samples with a higher percentage of men. Additionally, the study should be replicated in a clinical sample. This study was conducted online, through social network advertisements, which may be associated to certain biases. However, the online character of the data collection gives participants an enhanced sense of anonymity, which may be linked to more honest answers. Moreover, DOS is a self-report measure, therefore assessing certain aspects of ON solely based on this measure may not be adequate. Specifically, the assessment of prevalence for the Portuguese population should be considered as merely exploratory data. More importantly, a limitation of this study is the lack of consensus that still exist around ON and its conceptualization. More research is necessary to reach an accord.

Overall, the present study fulfills its goal to extend the current research on ON and is the first to successfully validate a measure that assesses ON and orthorexic behaviours for the Portuguese population, offering DOS as a valid and reliable tool.

What is already known on this subject?

Orthorexia nervosa is considered a pathological fixation with healthy eating. Despite recent research focus, there is still a lot of inconsistent information concerning ON, including its definition and validity of its measuring tools.

What does this study add?

The present study is the first to successfully validate an ON assessment tool for the Portuguese population, and adds to the current knowledge around ON by exploring its relationship with different factors, including BMI, dietary pattern and sex.