Introduction

The COVID-19 pandemic has caused an increase in mental disorders globally. According to the World Health Organization (WHO) (World Health Organization, 2022), approximately one in eight people has a mental disorder, which entails alterations in thought, emotional regulation, or behavior. In addition, research indicates that a significant group of patients who recovered from COVID-19 (33%) tend to develop mood changes or suffer from mental disorders such as depression and anxiety (Broche-Pérez et al., 2020). Regarding America, in Mexico, the symptoms of depression (20.8%) and anxiety (35.1%) have increased due to factors associated with the loss of loved ones, the diagnosis of COVID-19, or a previous diagnosis (Álvarez Bermúdez & Meza Peña, 2022; Morales-Chainé, 2021). In Colombia, the prevalence of depression is 5%, and anxiety is 19.3% in the adult population (Ministerio de Salud y Protección Social, 2021). While in Brazil, it has been observed that 8% of people presented a diagnosis of depression, 15.5% of generalized anxiety, and 13.6% of post-traumatic stress disorder (Damiano et al., 2022). In Peru, in 2021, a prevalence of 15.6% in mild depressive symptoms and 4.3% in moderate depressive symptoms were reported (Villarreal-Zegarra et al., 2023).

In relation to the above, it can be seen that the population’s mental health is affected, which is why intervention programs are necessary to improve mental health. However, some barriers prevent access to timely care, including internalized negative beliefs regarding mental health disorders (Larrahondo et al., 2021). As well as the reduced self-esteem experienced by a person when labeling themselves as socially unacceptable for deciding to seek help from a mental health professional; these beliefs are called self-stigma for seeking help (Larrahondo et al., 2021; Vogel et al., 2006).

Various studies have shown that help-seeking self-stigma is related to self-sufficiency, unwillingness to seek help (Latalova et al., 2014), stress, depression (Lokhee & Hogg, 2021), fear of social exclusion, thoughts of self-devaluation, shame (Wallin et al., 2018), self-pity (Heath et al., 2018) and anxiety (Cheng et al., 2015). Therefore, this construct is a factor that is associated with various mental health problems, which is why it is essential to have a valid and reliable instrument.

There is only one scale to measure this construct, Self-Stigma of Seeking Help (SSOSH) (Vogel et al., 2006), comprising ten items. Regarding the psychometric properties of the scale, in the original study, a Confirmatory Factor Analysis (CFA) showed that the SSOSH presents a unidimensional factorial structure with satisfactory fit indices (0.04). In addition, the original study showed that the scale presents adequate reliability indices (α > 0.80). As can be seen in in the supplementary information file (Appendix 1), adaptations of the SSOSH scale were carried out in several countries such as the United States (Eagle et al., 2020; Hammer et al., 2013; Vogel et al., 2006), Turkey (Kaya et al., 2014), Greece (Efstathiou et al., 2019) and Colombia (Larrahondo et al., 2021).

Subsequently, considering the negative impact of the inverse items on the factorial structure and the ease of application of brief measures, two new versions of the SSOSH were developed: the revised version of the SSOSH (SSOSH-7) composed of 7 items and the ultra-brief version of the SSOSH (SSOSH-3), composed of 3 items (Brenner et al., 2021). The study was developed in the United States with a sample of 857 adults and 661 university students. The study performed an Item Response Theory (IRT) analysis using a graded response model, which showed adequate functioning of the items of the SSOSH-7 and SSOSH-3 scales to measure the latent trait. Regarding reliability, good values were obtained in the SSOSH-7 (α = 0.89) and SSOSH-3 (α = 0.87) (Brenner et al., 2021).

Despite the psychometric evidence of the SSOSH, some studies have found variations regarding the scale’s dimensionality (Eagle et al., 2020; Kaya et al., 2014). Therefore, it is necessary to use statistical methods such as greater accuracy and precision, such as Exploratory Graph Analysis (EGA), which allows for determining the number of dimensions in multivariate data (Golino & Epskamp, 2017). The EGA stands out for its effectiveness and multiple advantages over traditional methods of exploratory factor analysis. Provides a straightforward and quick visual interpretation of which items belong to each factor through a color-coded network graph, eliminating the need to decode factor loading matrices. Furthermore, not requiring decisions about the type of rotation to use simplifies the process by avoiding the complexity associated with selecting the appropriate factorial rotation. Also, being a one-step approach eliminates the need for additional steps to verify the factors, significantly reducing the margin of error and bias in dimensionality estimation compared to traditional factor analysis methods (Golino et al., 2020).

It is important to mention that no psychometric studies of the SSOSH scale and its abbreviated versions, SSOSH-7 and SSOSH-3, have been found in the Latin American context. In addition, the factorial invariance of the scale according to the sex of the participants has not yet been evaluated. Obtaining this evidence will allow reliable comparisons (Association American Educational Research et al., 2014) and the development of future comparative studies.

Therefore, the present study has the following objectives: (a) to evaluate the factorial structure of the SSOSH, SSOSH-7, and SSOSH-3 in the Peruvian sample, (b) to identify evidence of validity based on the relationship with other variables, (c) to carry out factorial invariance according to sex and (d) determine the degree of reliability.

Method

Participants

The sample comprised 404 Peruvian citizens (65.84% women and 34.16% men). Age ranged from 18 to 58 years (M = 24.27; SD = 7.9). Most participants lived in an urban area (87.6%), and only 12.4% were in a rural area. The vast majority were single (84.2%), 14.6% married, and a small minority were divorced (0.3%) and widowed (0.9%). Regarding chronic diseases, 96.7% did not report having suffered from any, and 3.2% indicated that they suffered from some chronic disease.

For the study, a non-probabilistic sampling was carried out for convenience whose inclusion criteria were: (a) being over 18 years of age, (b) being of Peruvian nationality (c) giving informed consent.

Measures

Self-stigma of seeking help (SSOSH)

The version adapted to Colombia (Larrahondo et al., 2021) was used, which measures prejudices related to seeking help. Regarding the psychometric properties, the one-dimensional model showed satisfactory fit indices (CFI = 0.97, TLI = 0.98, RMSEA = 0.07). Regarding the revised version (SSOSH-7), it showed a good value in reliability (α = 0.89) and the ultra-brief version (SSOSH-3) showed satisfactory values in internal consistency (α = 0.87) (Brenner et al., 2021). The SSOSH is a scale of 10 items, the SSOSH-7 is of 7 items (1, 2, 3, 6, 7, 8, and 10), and the SSOSH-3 is of 3 items (1, 6, and 8). These instruments present a Likert-type response scale (1 = Strongly disagree, 2 = Disagree, 3 = Neither agree nor disagree, 4 = Agree, 5 = Strongly disagree).

Patient health questionnaire (PHQ-9)

The version validated in Peru (Villarreal-Zegarra et al., 2019) was used, which measures the presence of depressive symptoms during the last two weeks. Regarding the psychometric properties, the one-dimensional model showed adequate fit indices (CFI = 0.93; RMSEA = 0.089; SRMR = 0.039). It also presented adequate reliability indices (α = 0.87, ω = 0.87). This questionnaire is made up of 9 items designed based on DSM IV criteria and with a Likert-type response (0 = not at all, 1 = several days, 2 = more than half the days, 3 = almost every day). The instrument does not have inverse items; therefore, a higher score in the questionnaire shows a greater presence of symptoms of depression.

Rosenberg self-esteem scale (EAR-P)

The version adapted to Peru (Vilca et al., 2022) was used, which measures the personal assessment of oneself. Regarding the psychometric properties of the scale, the one-dimensional model showed adequate fit indices (CFI = 0.98; TLI = 0.97; RMSEA = 0.138 [IC 90% = 0.123–0.154], SRMR = 0.049) and internal consistency (α = 0.96; ω = 0.96). The scale is made up of 10 items with four response categories (Strongly agree = 4, Agree = 3, Disagree = 2, Strongly disagree = 1). The scale does not present inverse items; therefore, a higher score shows a higher level of self-esteem.

Procedure

This study has followed the standards and provisions of the Declaration of Helsinki (World Medical Association, 2013). Also, it has obtained the approval of the Ethics Committee of the Faculty of Health Sciences of the Universidad Peruana Unión (2022-CE-FCS - UPeU-042). In adapting the content of the items to the Peruvian context, two processes were used: content validity and face validity. Seven experts evaluated the construct’s relevance, coherence, clarity, and context in the first process. In the second process, twelve adults evaluated the items’ clarity and made some modifications. These processes made it possible to correct any cultural biases that might exist in the content of the items.

In data collection, a virtual form created in Google Forms was used and distributed nationwide through social networks. It is important to note that this form was designed with informed consent which details that the data obtained will be anonymous, voluntary, and for only research purposes.

Data analysis

In the first place, the clarity, coherence, and relevance were evaluated through the evaluation of judges using the V of Aiken, whose value is considered adequate when it is greater than 0.70. In addition, descriptive analyzes of mean, standard deviation, as well as skewness (As) and kurtosis (Ku) were estimated, whose values are adequate when As < ± 2; Ku < ± 7 (Finney & DiStefano, 2006). Subsequently, to perform the Exploratory Graph Analysis (EGA), a Gaussian Graphic Model (MGG) was used, which was estimated using the Graphical Least Absolute Shrinkage and Selection Operator (GLASSO) (Friedman et al., 2008), and the Walktrap algorithm (Pons & Latapy, 2005) used to determine the number of factors. In addition, Random-Intercept EGA (riEGA) was used, which is recommended when negative and positive items are used (Garcia-Pardina et al., 2022). Subsequently, Confirmatory Factor Analysis (CFA) was performed with the Diagonally Weighted Least Squares with Mean and Variance corrected (WLSMV) estimator due to the ordinal nature of the items. The RMSEA and SRMR indices were used to evaluate the model’s fit, where values less than 0.08 are considered acceptable. Also, the CFI and TLI indices were used, whose values greater than 0.95 indicate a good fit (Schumacker & Lomax, 2015).

Regarding the validity based on the relationship with other variables, the Structural Equation Modeling (SEM) approach was used. The relationship between self-stigma by seeking help with depression and self-esteem was evaluated. For this, the WLSMV estimator was used, given the ordinality of the data. In addition, the model was assessed using the same fit indices used in the CFA. Multigroup Confirmatory Factor Analysis (MGCFA) was used to assess factorial invariance by gender. A series of hierarchical variance models were carried out, starting with configural invariance as the reference model, followed by metric invariance, scalar invariance, and finally, strict invariance. A modeling strategy was used to compare the sequence of models, for which the differences in the RMSEA (ΔRMSEA) were used, where differences less than ≤ 0.010 evidence the invariance of the model between the groups. The differences in the CFI (ΔCFI) were also used where values less than ≤ − 0.005 show the invariance of the model between the groups. Scientific literature recommends using these criteria when the sample size of the comparison groups is unequal (Chen, 2007). Finally, to evaluate the internal consistency of the scale, the omega coefficient was used, whose value greater than 0.70 is considered adequate (Viladrich et al., 2017).

All statistical analyzes were performed using the “lavaan” package for the CFA and the “semTools” package for factorial invariance and reliability. Finally, the “EGAnet” package performed the EGA and riEGA. For all the analyses, the R program (version 3.5.0) and the R Studio Team environment were used.

Results

Content-based validity

The judges’ evaluation of the SSOSH can be found in the Supplementary Information file (Appendix 2) where satisfactory results were found in the criteria of clarity, relevance, coherence, and context (V > 0.70). Likewise, adjustments were made in the content of items 4,5,9 and 10 based on the recommendations of the judges. In addition, the face validity assessment (focus group) carried out by university students revealed that the items were clear and understandable.

Descriptive analysis

Table 1 shows that item 5 (“The concept I have of myself would not change because I decided to consult a therapist”) presents the highest average score (M = 2.72). While item 3 (“Seeking psychological help would make me feel less intelligent”) shows a lower average score (M = 1.79). In addition, the skewness and kurtosis values show that all the items have good values (skewness < ± 2; kurtosis < ± 7).

Table 1 Descriptive analyzes of the items
Full size table

Validity evidence based on internal structure

Figure 1 shows the exploratory graph analysis of the three versions of SSOSH. It is appreciated that the analysis of EGA and rieEGA showed evidence of a one-dimensional structure in the SSOSH, SSOSH-7, and SSOSH-3 (see Fig. 1). Considering the results of the EGA and riEGA, an CFA was carried out to evaluate the fit of a one-dimensional model for each of the scale versions. Table 2 shows that the SSOSH (version composed of positive and negative items) did not show adequate fit indices (χ2 = 509.366; df = 35; p = < 0.01; CFI = 0.86; TLI = 0.82; SRMR = 0.12; RMSEA = 0.183 [IC90% 0.169– 0.198]). Subsequently, the SSOSH-7 (version composed of positive and negative items) whose indices were primarily adequate (χ2 = 132.048; df = 14; p = < 0.01; CFI = 0.96; TLI = 0.94; SRMR = 0.06; RMSEA = 0.145 [IC90% 0.123–0.168]), was tested. Finally, the SSOSH-3 (version composed of positive items) was tested, which provided adequate values in the fit indices (χ2 = 0.000; df = 0; p = < 0.01; CFI = 1.00; TLI = 1.00; SRMR = 0.000; RMSEA = 0.000 [IC90% 0.000–0.000]). Table 2 shows that the factorial loads of the SSOSH-7 and SSOSH-3 were mostly high (λ > 0.70), unlike the SSOSH, which presented items with very low factorial weights (λ < 0.30). For all these reasons, the SSOSH-7 and SSOSH-3 were used in subsequent analyses.

Fig. 1
figure 1

Exploratory graph analysis of the SSOSH versions. Note: rieEGA = Exploratory graph analysis with random intercept model

Full size image
Table 2 Fit indices and factor loadings of the scales
Full size table

Reliability

The scale showed acceptable reliability values in the total sample for the SSOSH-7 (ω = 0.80) and SSOSH-3 (ω = 0.75). Similar results were found in the male sample in the SSOSH-7 (ω = 0.78) and SSOSH-3 (ω = 0.69). As well as in the sample of women of the SSOSH-7 (ω = 0.83) and SSOSH-3 (ω = 0.85).

Invariance by sex

Table 3 shows that the SSOSH-7 in men and women does not show strict invariance in the different invariance models proposed: metric (ΔCFI = 0.036, ΔRMSEA = − 0.022), scalar (ΔCFI = 0.001, ΔRMSEA = − 0.005), and strict (ΔCFI = 0.008, ΔRMSEA = − 0.008). On the contrary, Table 4 shows that the SSOSH-3 shows evidence of being strictly invariant in men and women: metric (ΔCFI = 0.000, ΔRMSEA = 0.000), scalar (ΔCFI = 0.000, ΔRMSEA = 0.000), and strict (ΔCFI = 0.000, ΔRMSEA = 0.000).

Table 3 Adjustment indices and invariance models according to sex of the SSOSH-7
Full size table
Table 4 Adjustment indices and invariance models according to sex of the SSOSH-3
Full size table

Validity evidence based on relations to other variables

SEM models were developed in the study to assess the relationship between help-seeking self-stigma, depression, and self-esteem. Regarding the SSOSH-7, it was found that the structural model (model 1) presented adequate fit indices (χ2 = 964.52; df = 296; p = < 0.01; CFI = 0.97; TLI = 0.97; SRMR = 0.06 RMSEA = 0.075 [IC90% 0.070– 0.080]). Figure 2 shows that the SSOSH-7 positively predicts levels of depression (0.27; p < .01) and negatively predicts the level of self-esteem in the participants (− 0.44; p < .01).

Fig. 2
figure 2

Relationship of the SSOSH-7 and SSOSH-3 models with other variables

Full size image

Regarding SSOSH-3, the study found that the structural model (model 2) showed adequate fit indices (χ2 = 910.65; df = 206; p = < 0.01; CFI = 0.97; TLI = 0.96; SRMR = 0.00; RMSEA = 0.092 [IC90% 0.086–0.098]). Figure 2 shows that the SSOSH-3 positively predicts levels of depression (0.32; p < .01) and negatively predicts the level of self-esteem (− 0.44; p < .01). Considering these results, the SSOSH-7 and SSOSH-3 present validity based on the relationship with other variables.

Discussion

Exposure to stressful factors implies a greater risk of mental health problems; in this context, adults are more vulnerable to psychosocial disorders such as anxiety and depression, affecting their physical and mental health (Álvarez Bermúdez & Meza Peña, 2022; Damiano et al., 2022; Morales-Chainé, 2021). However, people do not seek treatment due to the self-stigma of seeking psychological help (Larrahondo et al., 2021; Medeiros et al., 2016; Vogel et al., 2017). In this sense, it is necessary to have a measure of help-seeking self-stigma that is useful, valid, and reliable. For this reason, this study aimed to evaluate the psychometric properties of the SSOSH versions in Peruvian adults.

Regarding the validity based on the internal structure, the three versions of the SSOSH were evaluated from a network approach. The findings from the EGA and riEGA confirm the existence of a one-dimensional model and agree with previous research (Brenner et al., 2021; Vidales et al., 2023). The implementation of EGA represents a significant improvement over conventional exploratory factor analysis method. From this approach, the strongest connections between the nodes tend to group forming communities in the network, which are mathematically analogous to the factors (Golino & Epskamp, 2017). The EGA provides a direct visual representation of the factor structure, simplifying the interpretation of the results. Furthermore, unlike other traditional methods, its high precision is significant (Golino et al., 2020). In turn, the study used a recent implementation of the EGA called riEGA, which has demonstrated better precision in estimating dimensionality in scales with positive and negative items (Garcia-Pardina et al., 2022).

On the other hand, the results found through the CFA reported that the original version of the SSOSH did not have an adequate adjustment; this is probably due to the presence of negative items since the combination of direct and inverse items in the same instrument can influence its psychometric properties (Garcia-Pardina et al., 2022), and altering the discriminative power of the items (Józsa & Morgan, 2017). Furthermore, the dimensionality will also be affected due to the internal consistency irregularities of the test. This result is consistent with previous research, which showed that the inclusion of negative items mainly affects one-dimensional models, deteriorating the fit of the model and increasing the rejection of these (DiStefano & Motl, 2009; Essau et al., 2012; Suárez-Alvarez et al., 2018). Consequently, the SSOSH-7 and SSOSH-3 had a one-dimensional factorial model with adequate fit indices. These findings empirically support the theoretical proposal about help-seeking self-stigma (Vogel et al., 2006) and are similar to that reported by previous studies (Brenner et al., 2021; Vidales et al., 2023). Notably, the SSOSH-3 demonstrated even better fit indices compared to the SSOSH-7. This difference is likely due to the absence of negative items in the SSOSH-3.

Regarding reliability, adequate values were obtained for both versions, both for the SSOSH-7 (ω = 0.80) and the SSOSH-3 (ω = 0.75). Results that are favorable, considering the advantages of the omega coefficient. First, it is more robust and resistant to samples in a heterogeneous population and is not affected by the number of items, reducing the risk of overestimating reliability (Waller, 2008). In addition, it has higher precision than other estimators (Zinbarg et al., 2005).

Regarding factorial invariance, it was found that the SSOSH-7 items showed that men and women understand help-seeking self-stigma differently. It is important to note that the lack of invariance in this study was not an expected result and represents an issue that should be addressed before conducting future research (Vandenberg & Lance, 2000). The lack of invariance could affect the comparability between the results obtained in both sexes in future studies on the construct. This finding may be attributed to differences in the understanding of the self-stigma associated with seeking help, especially in the case of men, where masculine social norms significantly influence (Hammer et al., 2013). In this sense, help-seeking in men can be perceived as a loss of status and autonomy and damage to identity (Möller-Leimkühler, 2002). Furthermore, another possible explanation is attributed to the inequality between the participants since there was a greater representation of women than men in the study. The unequal distribution between groups affects the levels of invariance, leading to values not being obtained according to the adjustment indices (Chen, 2007). However, the SSOSH-3 did report evidence of strict invariance according to sex, making its use possible in comparative studies.

Regarding the validity based on the relationship with other variables, the results reported that self-stigma positively predicts levels of depression. Therefore, it generates an increase in depressive symptoms; that is, the more self-stigma in people, the greater it will be the presence of symptoms of depression. This finding is consistent with previous studies (Cheng et al., 2015; Lokhee & Hogg, 2021). In addition, the scientific literature suggests that people who do not seek professional help suffer a greater impact on their health and psychological well-being (Vogel et al., 2006). Also, a negative influence of self-stigma due to seeking help on self-esteem was found, suggesting that the greater the individual’s perception of self-stigma toward mental health treatment, the latter will experience reduced self-esteem. This result is similar to that reported in previous studies (Villar et al., 2021). It could be explained by the fact that the search for psychological help requires people to feel self-esteem and deserving of help (Link et al., 1989); instead, when it does not occur through this, people could incorporate more negative attitudes about seeking help that could affect their level of self-esteem.

Regarding the strengths, it is the first study in Latin America, specifically in Peru, to provide psychometric evidence from an SEM and Network Psychometrics approach of SSOSH, SSOSH-7, and SSOSH-3 in Peruvian adults. Another strength is that evidence is provided that the SSOSH-3 has greater evidence of validity and reliability than the SSOSH and SSOSH-7 in the Peruvian sample. However, the study has limitations. First, non-probabilistic sampling was used, which does not allow for generalizing the results. Secondly, the study needed to provide evidence of longitudinal invariance, which makes it impossible to verify whether the one-dimensional structure varies over time. Third, reliability was not estimated with test-retest, so its temporal stability could not be verified. Fourthly, the relationship between the variables depression and self-esteem was analyzed; however, it is suggested that future studies analyze the relationship with other constructs. Specifically, it is suggested that discriminant validity with other constructs be obtained in future studies. Fifth, although the Peruvian sample included adults with a chronic disease, no specific analysis was performed since it was a very small sample (3.2%). Therefore, future studies should examine the scales’ psychometric performance in clinical samples. Sixth, the performance of the items should have been evaluated from the perspective of the Item Response Theory (IRT). Therefore, it is suggested that future studies show evidence of the psychometric performance of these scales from the IRT.

Despite these limitations, the study’s results in a Peruvian sample represent a significant advance in research on self-stigma in help-seeking in Latin America. This study fills a gap in the scientific literature, as most cross-cultural research on SSOSH has focused on countries in Europe and North America (Vogel et al., 2013). The study also provides new possibilities for cross-cultural research on help-seeking self-stigma, allowing comparison of self-stigma experiences in different cultures. By having a valid and reliable instrument, studies can be carried out to identify possible risk or protective factors associated with this construct, which will favor understanding the causes and consequences of this phenomenon and the development of effective intervention strategies.

In conclusion, the present study identified that the SSOSH-3 has greater psychometric properties than the other versions of the SSOSH. The SSOSH-3 is invariant both in men and women and shows adequate reliability. In addition, it has evidence of validity based on the relationship to other variables, such as depression and self-esteem. Therefore, the SSOSH-3 proves to be a brief, valid and reliable instrument that allows the adequate measurement of self-stigma due to help-seeking in the Peruvian adult sample.