Introduction

Weight stigma is defined as “the social devaluation and denigration of people perceived to carry excess weight and leads to prejudice, negative stereotyping and discrimination toward those people” [1]. Weight stigma is widespread, as individuals with excess weight experience discrimination not only in employment, education, healthcare, media but also in their interpersonal relationships with significant others [2]. The cyclic obesity/weight-based stigma (COBWEBS) model characterizes weight stigma as perpetuating a “vicious cycle” of weight gain: weight stigma is a stressor that can increase eating behavior and the secretion of the hormone cortisol, ultimately leading to weight gain and difficulty losing weight [1].

Weight stigma can be classified into three related types of stigma: experienced stigma, perceived stigma, and weight-related self-stigma [3]. Experienced stigma refers to an individual receiving and experiencing prejudice or discrimination from others. Furthermore, the individual may be aware that this prejudice or discrimination happened due to his or her weight condition, and develops perceived stigma. When the individual accepts and endorses weight stigma and believes that negative weight-related stereotypes apply to himself or herself, weight-related self-stigma (also referred to as weight bias internalization or internalized stigma) occurs [3].

Numerous negative health outcomes of weight stigma have been reported, including metabolic dysfunction, psychological distress, body image dissatisfaction, low self-esteem, poor health-related quality of life, and disordered eating behaviors [4,5,6,7]. Moreover, recent studies suggest that weight stigma is related to less enjoyment and avoidance of physical activity and sports, and low levels of physical activity [8,9,10]. Recent evidence has shown that weight stigma, most frequently in the forms of teasing and bullying about body weight, is highly prevalent among youths across different sociodemographic groups [5]. For instance, the first national study in the Middle East found a high prevalence of weight stigma (46.4%) and its association with increased risk of depression [11]. Another study in Iran showed that weight self-stigma was associated with poor quality of life and psychological distress among women with overweight or obesity [12]. A recent study in Hong Kong and Taiwan also reported that university students with overweight or obesity experienced weight stigma and became vulnerable to eating disturbances such as uncontrolled eating and emotional eating [7]. Given the global impact of weight stigma, scholars and researchers underscore the importance of assessing weight stigma, along with weight status, among children and adults [13].

Measurements assessing weight stigma are important to develop prevalence profiles and identify services needed to prevent stigmatization among those with excess weight. Measurement scales with good psychometric properties are vital for both research and clinical purposes, to provide valid and reliable assessment results to generate recommendations for practice and policies [14]. However, no validated scales are available in measuring weight stigma in Malaysia, particularly perceived weight stigma and weight-related self-stigma. The latest National Health and Morbidity Survey (NHMS) in Malaysia reported that one in two Malaysian adults is overweight and obese [15]. In light of the high prevalence of overweight and obesity and the lack of studies investigating weight stigma in Malaysia, it is crucial to examine the psychometric properties of scales measuring weight stigma. Two different self-reported questionnaires are internationally available for assessing weight stigma, the Weight Self-Stigma Questionnaire [16] and the Perceived Weight Stigma Scale [17].

Validating both WSSQ and PWS among Malaysians will benefit healthcare providers and healthcare recipients in Malaysia. Furthermore, validation of these instruments would greatly facilitate the assessment of weight stigma in cross-cultural studies. For example, comparing weight stigma across Malaysians and other Asian individuals would require valid psychometric instruments. Thus, evaluating the psychometric properties of WSSQ and PWS is essential. This evaluation can be performed using classical test theory methods (e.g., factor analysis and internal consistency) or modern test theory methods (e.g., based on Rasch models). Application of the Rasch model for assessing the psychometric properties of instruments has increased in recent decades [18,19,20]. It is a powerful tool to evaluate the validity and reliability of an instrument, and it can provide different information from psychometric testing in the classical test theory and provide more accurate findings [21]. To the best of our knowledge, no studies have applied a Rasch model in examining the psychometric properties of the WSSQ in Malaysia and PWS worldwide. Consequently, we proposed to examine the WSSQ and PWS simultaneously using the classical test theory (i.e., confirmatory factor analysis) and modern test theory (i.e., Rasch model) to strengthen the scientific evidence for both instruments. Psychometric evidence may help healthcare providers and researchers in selecting valid and reliable instruments and subsequently using them appropriately.

The purpose of this study was to examine the psychometric properties of the WSSQ and PWS among Malaysian university students. First, the validity and reliability of the WSSQ and PWS were examined using classical test theory and modern test theory. Second, the concurrent validity of the total scores of WSSQ and PWS with depression, anxiety, stress, and body mass index was also investigated.

Methods

Participants and procedure

This study was approved by the Ethics Committee for Research Involving Human Subjects in Universiti Putra Malaysia (JKEUPM; Reference Number: JKEUPM-2021-455). Participants were recruited through convenience sampling between 19 August 2021 and 30 September 2021 using Google Form via online social media platforms including “Facebook”, “WhatsApp”, “Instagram” and “Telegram”. The inclusion criteria for the participating university students included: (1) Malaysian; (2) age 18 and above; and (3) studying in a university in Malaysia. The exclusion criteria included: (1) pregnant or lactating women; (2) self-reported to have chronic diseases such as cardiovascular diseases and cancer; and (3) physically disabled. For the students who agree to participate in this study with terms and conditions provided in the informed consent form (which is in the first page of the online questionnaire), they are required to click the option “I agree to participate in the study” and proceed to completing the questionnaire.

Measures

All the measures were conducted in English given that English is widely understood and the main language used in universities and colleges in Malaysia. Moreover, there are different ethnic populations in Malaysia (e.g., Malays, Chinese, and Indians) and different languages have been used across these ethnic populations. Therefore, it is appropriate to use English for all the measures in the present study. Although the present study did not translate English into local languages in Malaysia, cultural adaptions have been done according to the guidance from international guidelines on patient-reported outcomes [22,23,24]. Moreover, supplementary Table S1 provides the item descriptions for all assessment instruments used in the present study (i.e., Weight Self-Stigma Questionnaire [WSSQ], Perceived Weight Stigma Scale [PWS], and Depression, Anxiety, Stress Scale-21 [DASS-21]).

Demographic and anthropometric information

A background information sheet was asked for the participants to obtain their demographic and anthropometric information. Specifically, the demographic questions include age, gender (male or female), ethnic group (Malay, Chinese, or other), study program (undergraduate or postgraduate), and marital status (answered in single, married, or other). Anthropometric questions include height (cm) and weight (kg). Height and weight were used to calculate body mass index (BMI; kg/m2); subsequently, BMI with international classification was used to classify the participants’ weight status into underweight (< 18.5 kg/m2), normal weight (between 18.5 and 24.9 kg/m2), or overweight (> 25 kg/m2) [25].

Weight self-stigma questionnaire (WSSQ)

The Weight Self-Stigma Questionnaire (WSSQ) is used to measure weight-related self-stigma within the last week [16]. It contains 12 items including self-devaluation subscale (first 6 items) and fear of enacted stigma subscale (last 6 items). The self-devaluation subscale includes items assessing how individuals devalue and feel ashamed of themselves because of their weight; an example item is “I caused my weight problems.” The fear of enacted stigma subscale includes items assessing how individuals perceive and fear enacted stigma and discrimination due to their weight; an example item is “I feel insecure about others’ opinions of me.” All 12 items are rated on a 5-point Likert scale rating from 1 (completely disagree) to 5 (completely agree). A WSSQ total score and two WSSQ subscale scores can be calculated using the average of item scores, with higher scores indicating higher levels of weight-related self-stigma [16]. It showed good validity and reliability with internal consistencies of α = 0.88 (WSSQ total score), α = 0.87 (WSSQ fear of enacted stigma), and α = 0.81 (WSSQ self-devaluation), respectively [16].

Perceived weight stigma scale (PWS)

The Perceived weight stigmatization Questionnaire (PWS) is used to measure the severity of perceived weight stigma within the last week [17]. It is assessed using 10 dichotomous items (yes scores = 1 and no scores = 0). A higher PWS sum score (possible range between 0 and 10) indicates a higher level of perceived weight stigma. However, to the best of our knowledge, there is no suggested cutoff to indicate moderate or high level of perceived weight stigma for the PWS. It showed good internal consistency (α = 0.84) and convergent validity [26].

Depression, anxiety, stress scale-21 (DASS-21)

The DASS-21 contains 21 items assessing 3 types of psychological distress, including depression (7 items), anxiety (7 items), and stress (7 items) [27]. All the items in the DASS-21 were assessed using a four-point Likert scale (0 = did not apply to me at all; 3 = applied to me very much or most of the time). The three subscale scores can be calculated using the sum of item scores multiplied by two, with higher scores indicating higher levels of psychological distress. Moreover, suggested cutoffs were 10–13 (mild level), 14–20 (moderate level), 21–27 (severe level), and over 28 (extremely severe level) for depression; 8–9 (mild level), 10–14 (moderate level), 15–19 (severe level), and over 20 (extremely severe level) for anxiety; and 15–18 (mild level), 19–25 (moderate level), 26–33 (severe level), and over 34 (extremely severe level) for stress [27]. The psychometric properties of the DASS-21 have been widely investigated and supported [28, 29]. It has been also validated among Malaysian university students [30]. In the present study, the internal consistency of the DASS-21 was satisfactory (α = 0.92 for depression subscale; 0.86 for anxiety subscale; and 0.91 for stress subscale).

Statistical analysis

First, we used descriptive statistics (including mean and frequency) to characterize the present sample and the score distributions of both WSSQ and PWS. After obtaining an initial understanding of the performance of the WSSQ and PWS, we examined their psychometric properties: internal consistency, confirmatory factor analysis (CFA), Rasch analysis, and concurrent validity with DASS-21 and BMI.

The internal consistency was examined using Cronbach’s α, and an acceptable value was 0.7 or above [31]. The CFA was conducted using a diagonally weighted least squares estimator. Moreover, the tested structures in the CFA include a two-factor structure (for the WSSQ) and a unidimensional structure (for the PWS). The following fit indices were used together to evaluate whether the proposed factor structure fits with WSSQ and PWS: a nonsignificant χ2 test, a standardized root mean square residual (SRMR) < 0.08, a root mean square error of approximation (RMSEA) < 0.08, a Tucker–Lewis index (TLI) > 0.9, and a comparative fit index (CFI) > 0.9 [32, 33]. Apart from the data-model fit, factor loadings in the CFA were used to examine whether each item fits in its embedded factor, and a factor loading larger than 0.4 is expected [34].

The Rasch analysis was conducted using a partial credit model (PCM). Moreover, the item fit with its embedded factor was tested via three statistics: a significant χ2 test, an outfit mean square (MnSq), and an infit MnSq. A nonsignificant χ2 test together with both outfit and infit MnSq ranged between 0.5 and 1.5 support an item fit in its embedded factor. When an outfit MnSq or an infit MnSq is smaller than 0.5, it implies the redundancy of that specific item. When an outfit MnSq or an infit MnSq is larger than 1.5, it implies the misfit of the specific item [35, 36]. Lastly, concurrent validity of WSSQ and PWS was assessed using Pearson correlation coefficients with the DASS-21 subscales and BMI.

R software and IBM SPSS 20.0 (IBM Corp.: Armonk, NY) were utilized for statistical analyses. Specifically, lavaan package [37] in the R software was used for conducting CFA; eRm package [38] for Rasch analyses.

Results

Table 1 presents the characteristics of the participants (N = 380; mean [SD] age = 24.0 [5.1] years). The majority were females (n = 272 [71.6%]), ethnic Chinese (n = 200 [52.6%]) and studying an undergraduate program (n = 277 [72.9%]). Mean BMI was 22.1 (4.5) kg/m2 with nearly two-thirds were normal weight (n = 237 [62.4%]) and nearly one-fifth were overweight (n = 74 [19.4%]).

Table 1 Characteristics of the participants (N = 380)
Full size table

Table 2 presents how the WSSQ and PWS items were distributed among the 380 participants. In general, the most common response was completely disagree for the WSSQ and no for the PWS. Moreover, all the WSSQ items had their skewness values between − 0.03 and 1.17; kurtosis values between − 1.32 to 0.30, which implied acceptable normal distributions.

Table 2 Distributions for the items of the weight stigma instruments
Full size table

The internal consistency was satisfactory for the WSSQ (Cronbach’s α = 0.89 for factor 1 [self-devaluation]; = 0.91 for factor 2 [fear of enacted stigma]; and = 0.94 for the entire WSSQ) and the PWS (Cronbach’s α = 0.83). In addition, the two-factor structure of the WSSQ was supported by the CFA results, including the fit indices (p-value of χ2 = 0.15; SRMR = 0.049; RMSEA = 0.023; 90% CI of RMSEA = 0.000, 0.042; TLI = 0.998; CFI = 0.998) and the strong factor loadings (ranged between 0.59 and 0.89). Similarly, the unidimensional structure of the PWS was supported by the CFA results with good fit indices (p-value of χ2 = 0.73; SRMR = 0.067; RMSEA = 0.000; 90% CI of RMSEA = 0.000, 0.028; TLI = 1.009; CFI = 1.000) and relatively strong factor loadings (ranged between 0.39 and 0.74) (Table 3).

Table 3 Psychometric results derived from confirmatory factor analysis on the internet-related instruments
Full size table

The good psychometric properties of both WSSQ and PWS items were further illustrated by the Rasch analysis results. Specifically, only three items had a significant χ2 test: WSSQ item 5 in factor 1 (“I would never have any problems with weight if I were stronger”; p < 0.001; outfit MnSq = 1.40 and infit MnSq = 1.34), WSSQ item 7 in factor 2 (“I feel insecure about others’ opinions of me”; p < 0.001; outfit MnSq = 1.58 and infit MnSq = 1.58), and PWS item 3 (“People act as if they are afraid of you”; p < 0.001; outfit MnSq = 1.58 and infit MnSq = 1.43). All other items in both WSSQ and PWS had a nonsignificant χ2 test with both outfit and infit MnSq values between 0.5 and 1.5 (Table 4).

Table 4 Psychometric results derived from Rasch analysis on the internet-related instruments
Full size table

With the psychometric properties largely supported by the internal consistency, both WSSQ and PWS demonstrated good concurrent validity: moderate correlations were observed between WSSQ factors and DASS-21 subscales (r = 0.33 to 0.45; p < 0.001); between WSSQ total score and DASS-21 subscales (r = 0.39 to 0.45; p < 0.001); and between PWS and DASS-21 subscales (r = 0.40 to 0.42; p < 0.001). Moderate correlations were also observed between BMI and the WSSQ (r = 0.31 to 0.42; p < 0.001); however, no correlation was observed between BMI and the PWS (r = 0.08; p = 0.12). Moreover, WSSQ and PWS had moderate correlations (r = 0.24 to 0.38; p < 0.001), while the two WSSQ factors and the WSSQ total score were highly correlated (r = 0.77 to 0.94; p < 0.001) (Table 5).

Table 5 Concurrent validity of the weight stigma instruments using Pearson correlations
Full size table

Discussion

This is the first study in Malaysia to examine the psychometric properties of the WSSQ and PWS among a group of Malaysian university students by applying both classical and modern test theories. Consistent with previous findings, our study also supported that the WSSQ is a two-factor structure instrument [26, 39, 40] and the PWS is a single-factor structure instrument [17, 26]. Furthermore, concurrent validity of both the WSSQ and PWS was confirmed by moderately associations with depression, anxiety, and stress. Given the increasing prevalence of overweight and obesity and high weight stigma directed at individuals with excess weight, especially in the Southeast Asian Region [5], this study adds to the body of knowledge regarding validity and reliability of both weight stigma instruments in this region. Nevertheless, conclusions from these findings may not be generalizable to samples of other ages, geographic regions, or ethnic backgrounds.

Although the Rasch findings are almost in line with the CFA findings and previous evidence [15, 28, 29], WSSQ item 5 (“I would never have any problems with weight if I were stronger”), WSSQ item 7 (“I feel insecure about others’ opinions of me”), and PWS item 3 (“People act as if they are afraid of you”) were found to have some psychometric problems, with relatively high infit and outfit MnSq values. Although the three items showed significantly misfit in the Rasch analysis, the CFA results indicated that the three items embedded in their corresponding factor structures with acceptable factor loading (0.59 for WSSQ item 5, 0.65 for WSSQ item 7, and 0.39 for PWS item 3). Together with the consideration that the three items only marginally exceeded infit and outfit MnSq in the Rasch analysis, we propose currently retaining the three items in their corresponding factor/scale, pending additional research. However, future studies are needed to further examine the infit and outfit of these three items.

The problem of WSSQ item 5 shown in the present study was comparable to the findings obtained in a group of Hong Kong children and adolescents, in which their result showed a weak factor loading for WSSQ item 5 [26]. A possible reason might be due to the almost equal percentage of underweight (18.2%) and overweight (19.4%) found in our sample, in which the content of this item might not relate to those who were not overweight [26]. On the other hand, the psychometric properties findings from a French study found that WSSQ item 7 functioned differently in males and females, in which the researchers suggested that this item should be excluded when computing the fear of enacted stigma subscale as males may feel more secure than females about others’ opinions [41]. Given the limited evidence on the psychometric properties of the PWS, findings of this study supported the previous two studies in a group of university students from Hong Kong and Taiwan [17] as well as a group of children and adolescents from Hong Kong [26].

Our findings further indicated that both the WSSQ and PWS correlated significantly with depression, anxiety, and stress, which were consistent with previous studies [3, 42,43,44]. For instance, a meta-analysis and systematic review found that the pooled associations between weight-related self-stigma and psychological distress were moderate [3]. A sample of Saudi adult population with overweight or obesity also showed a significant association between weight-related self-stigma with the development of severe depression and anxiety [42]. The WSSQ correlated slightly more strongly with depression, anxiety, and stress than the PWS in the present study. These findings suggested that WSSQ may be a more sensitive marker of psychological distress than the PWS, consistent with previous findings from Iranian adolescents [39]. Moreover, the WSSQ was correlated significantly with BMI, consistent with a recent study among adults in Saudi Arabia showing a significant association between WSSQ and BMI, particularly among adults with obesity [42]. The high correlations found between WSSQ factors and WSSQ total score in the present study are consistent with the findings of the WSSQ development study, r = 0.53 to 0.88 (p < 0.001) [16]. However, PWS was not correlated significantly with BMI in the present study. This might be due to the majority of the present sample not being overweight (80.6%). The items in PWS are more related to overweight and obesity compared to underweight and normal weight.

Strengths and limitations

To our knowledge, this is the first study in Malaysia to evaluate the psychometric properties of the WSSQ and PWS. This study was also novel in applying classical test theory as well as modern test theory to examine the psychometric properties of the WSSQ and PWS. A potential implication of using an English-language WSSQ and PWS is that the two instruments can assess weight stigma in this population because such weight-related instruments have never been tested and used among Malaysian young adults. Given that Malaysia universities use English as a medium in knowledge delivery, English-language WSSQ and PWS are useful and reliable instruments. Moreover, both questionnaires have been pre-tested by a smaller pilot sample of young adults from the same university student population as those in the present study. There are several limitations to be considered in this study. First, we used convenience sampling, which would affect the representativeness of the study sample and limit the generalizability of the study findings. Second, self-reported weight and height were used, which could be biased. Third, this was a cross-sectional study and thus could not address predictive validity, test–retest reliability, or longitudinal measurement invariance. Lastly, the WSSQ may possibly have different factor structures among males and females, as a previous study showed that WSSQ item 7 functioned differently in males and females [41]. However, given that the present study did not have sufficient sample size (i.e., at least 200 participants for each subgroup [45], whereas the present study had only 108 male participants), the present study could not examine if the factor structure of WSSQ is different between males and females. Future studies using larger sample sizes are thus needed to examine the factor invariance of the WSSQ.

Conclusion

In conclusion, both WSSQ and PWS were found to have adequate validity and reliability; therefore, they can be used to assess weight stigma among Malaysian university students. Factor structures were confirmed for both instruments. Nevertheless, special attention needs to be paid to items 5 and 7 for WSSQ and item 3 for PWS due to the somewhat unsatisfactory fit statistics in Rasch model. Overall, our findings suggest that the use of both instruments may benefit healthcare professionals and patients. The findings may also increase our understanding of weight stigma among those with excess weight and help to prevent and combat weight stigma.

What is already known on this subject?

Weight stigma and its adverse health outcomes are crucial issues that can affect psychological distress and physical health behaviors. Good psychometric properties in assessing weight stigma are increasingly important for healthcare professionals to enable them to evaluate this issue, and further prevent weight stigma development.

What this study adds?

This present study indicated that both WSSQ and PWS are suitable for assessing weight stigma among Malaysian university students. However, some specific WSSQ and PWS items (i.e., items 5 and 7 in the WSSQ; item 3 in the PWS) may need to be further investigated given that they have relatively poorer properties than other items.