Abstract
Rumination is a relevant mechanism for experiencing depressive symptoms. To date, one of the most commonly used measures to assess rumination is the Ruminative Response Scale (RRS), which contains two dimensions: brooding and reflection. The current study aimed to examine the factorial structure and measurement invariance of a modified version of the RRS. Exploratory factor analysis (EFA), confirmatory factor analysis (CFA), and multigroup confirmatory factor analysis (MGCFA) were conducted. Also, we evaluated the relationship between the RRS, its dimensions, and the Beck Depression Inventory–II. Two samples of Peruvian college students were used (n1 = 1044 and n2 = 441). EFA and CFA provided greater support for a bifactor model rather than a two-factor model. MGCFA revealed that the bifactor model was not invariant according to sex. A structural equation model determined that brooding predicts depression better than reflection and global rumination. However, a significant negative association between reflection and depression was also observed in females.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Rumination is a complex, multi-faceted construct, which has been studied from different theoretical perspectives (Smith & Alloy, 2009). Perhaps the most prolific of these is the response styles theory (RST), which defines rumination as “repetitively focusing on the fact that one is depressed; on one’s symptoms of depression; and on the causes, meanings, and consequences of depressive symptoms” (Nolen-Hoeksema, 1991, p. 569). As becomes evident from this definition, the RST conceptualizes rumination as essentially maladaptive and focused on depression (Lyubomirsky & Tkach, 2004). Certainly, even though rumination is present in a myriad of mental health problems, it is more strongly related to depression (Cernvall et al., 2016; Iqbal & Dar, 2015).
Rumination was originally proposed as a mediator of the association between gender and depression (Nolen-Hoeksema, 1987). Indeed, small but consistent differences have been found between men and women’s levels of rumination (Johnson & Whisman, 2013), and the mediation hypothesis has been successfully tested in some studies (Grant et al., 2004; Ricarte Trives et al., 2016). Consequently, rumination is part of state-of-the-art models of gender differences in depression (Hyde & Mezulis, 2020). Moreover, it seems to mediate the relationship between gender and other disorders such as eating pathology (Opwis et al., 2017).
The Ruminative Response Scale
Many measures of rumination have been proposed; however, the Ruminative Response Scale (RRS) is the one that has received the greatest attention (Luminet, 2004). It was originally part of the Response Styles Questionnaire (RSQ), a wider instrument that also assessed other three styles of reacting to negative mood (Nolen-Hoeksema & Morrow, 1991). As time went by, however, researchers started to focus on the 22 items of the RRS, which were eventually studied as an independent measure.
The 22-item RRS was used extensively to study rumination in relation to depression levels (Nolen-Hoeksema et al., 1994), the onset of depression (Just & Alloy, 1997), and the gender differences observed in this disorder (Nolen-Hoeksema et al., 1999). These results notwithstanding, some authors also raised concerns regarding the dimensionality of the scale. Specifically, they pointed out to the fact that many items of the 22-item RRS seemed to be confounded with depression itself (Cox et al., 2001; Roberts et al., 1998; Segerstrom et al., 2000). In response to these criticisms, Treynor et al., (2003) proposed a new version of the scale, which was composed of only 10 items and had two underlying dimensions: brooding and reflection. According to these authors, the latter—characterized by self-focus—was an adaptive aspect of rumination in the long term, since it was longitudinally associated with lower levels of depression. This short form of the RRS (hereafter simply referred to as “the RRS”) will be the focus of the present report.
Factor Structure of the RRS
Several studies of the RRS reported the use of principal component analysis (PCA) as a technique to perform exploratory factor analysis (Cowdrey et al., 2011; Dinis et al., 2011; Hernández-Martínez et al., 2016; Lee & Kim, 2014; Roelofs et al., 2006; Shin et al., 2015; Thanoi & Klainin-Yobas, 2015). Although this is a variable reduction technique, it considers both the common variance and the error variance in the analysis, while EFA only includes the common variance between the item and the rest of the items (Lloret-Segura et al., 2014). Thus, the use of PCA could also affect the results of the factorial structure.
Exploratory research, either with PCA or EFA, has repeatedly identified a two-factor structure of the RRS (Armey et al., 2009; Dinis et al., 2011; Hernández-Martínez et al., 2016; Lee & Kim, 2014; Shin et al., 2015; Thanoi & Klainin-Yobas, 2015; Zanon et al., 2018), even though most of these studies had to remove indicators in order to achieve a clear factorial solution. This suggests that the RRS is still subject to improvement by adding some new items and deleting others. For instance, it has been observed, both in exploratory and confirmatory analyses, that item 5 (Write down what you are thinking and analyze it) performs poorly, thus its elimination seems justified (Dinis et al., 2011; Parola et al., 2017; Xavier et al., 2016).
Many confirmatory studies found mediocre fit (both CFI < 0.90 and RMSEA > 0.08) of the proposed two-factor structure (Arana & Rice, 2017-Argentinian sample; Schoofs et al., 2010; Whisman et al., 2018; Xavier et al., 2016). Others had better fit but still far from the desired CFI > 0.95 and RMSEA < 0.06 (Arana & Rice, 2017-American sample; Hasegawa, 2013; Lucena-Santos et al., 2018; Ruiz et al., 2017; Thanoi & Klainin-Yobas, 2015). Finally, only a small set of results were close to the desired criteria of good fit (Lei et al., 2017; Parola et al., 2017). Due to the lack of clarity regarding validity evidence based on the factorial structure, the original model proposed by Treynor et al., (2003) still needs further assessment.
Measurement Invariance of the RRS
Rumination was proposed to help explain the gender differences in depression (Nolen-Hoeksema, 1987). It is, therefore, necessary that rumination measures function equivalently between genders, so that meaningful comparisons can be made (e.g., Johnson & Whisman, 2013). In factor analytical terms, such equivalence is named measurement invariance and implies three aspects: metric (equal loadings), scalar (equal intercepts or thresholds), and strict (equal residual variances) invariance (Bowen & Masa, 2015; Dimitrov, 2010), even though some authors suggest that metric and scalar invariance be tested together (Sass, 2011). Some authors found evidence of metric invariance of the RRS but did not test any further (Ruiz et al., 2017; Xavier et al., 2016). Others, on the other hand, found scalar (Whisman et al., 2018) and even strict invariance (Lei et al., 2017; Liang & Lee, 2019) of the RRS between genders.
Latent Correlation Between Brooding and Reflection
Not all studies reported the latent correlation between the two factors (i.e., brooding and reflection). However, those who did found large or very large correlations, ranging from 0.63 to 0.95 (Lucena-Santos et al., 2018; Ruiz et al., 2017; Thanoi & Klainin-Yobas, 2015; Xavier et al., 2016). This raises concerns as to whether the significant correlations of the RRS’s factors and other variables are due to their specific or shared variance. For instance, it has been observed that brooding has larger correlations with depression, even though the correlation of reflection and depression is significant too (Armey et al., 2009; Dinis et al., 2011; Hernández-Martínez et al., 2016; Lee & Kim, 2014; Schoofs et al., 2010; Shin et al., 2015). However, it remains unclear whether these correlations are due—at least partly—to an underlying general dimension. This issue can be addressed by modelling both specific (i.e., brooding and reflection) and general (i.e., global rumination) factors.
The RRS and Depression
Rumination as a construct was originally developed in depression research (Nolen-Hoeksema, 1987, 1991). Therefore, it is a valid question to ask whether brooding and reflection are equally related to depression. A first answer can be found in Treynor et al., (2003) work itself. These authors found that brooding was more associated with depression when measured cross-sectionally; however, when depression was measured 1 year later, and after adjusting for various confounders, reflection was the only significant predictor. The former result has been repeatedly replicated: In all but one of the studies reviewed by the authors, brooding had a stronger correlation to depression than reflection (Armey et al., 2009; Dinis et al., 2011; Lee & Kim, 2014; Schoofs et al., 2010; Shin et al., 2015; but Lei et al., 2017).
The Bifactor Approach
As reported above, large correlations are routinely observed between brooding and reflection. This suggests that the variance shared by the factors may be larger than each one’s separate variance (Hair et al., 2019). Furthermore, it is possible that an unmodelled general factor explains this shared variance; in other words, a global rumination factor seems plausible (Furr & Bacharach, 2014). On the other hand, as already mentioned, brooding and reflection are not equally related to relevant outcomes. Therefore, a case can be made that they should not be completely subsumed under a single global dimension.
The situation just described can be addressed taking a bifactor approach. It involves creating a model with both a general factor (i.e., rumination) and specific factors (i.e., brooding and reflection), all of which are orthogonal to each other (Markon, 2019; Reise, 2012). One advantage of modelling the RRS as bifactor is that each factor’s unique contribution can be evaluated while simultaneously controlling for the others (Chen et al., 2006). Thus, it is possible to examine whether the specific factors are still relevant when the general factor is included in the model (Reise et al., 2010; Rodriguez et al., 2016). Moreover, if a relevant outcome (e.g., depression) is added to the model, its associations with both the general and specific factors can be estimated (Chen et al., 2006).
In spite of the popularity of the RRS, few studies have analyzed its items with a bifactor methodology. For example, Topper et al., (2014) used only the brooding items of the RRS as part of a bifactor model that included worry as a second specific factor. These authors found that brooding, as a specific factor, did not predict depression after controlling for the global factor of repetitive negative thinking. On the other hand, McEvoy and Brans, (2013) did find that both brooding and reflection predict depression even after controlling for a general factor. Given the limited evidence, further research is needed to examine the RRS from a bifactor perspective.
Purpose of the Present Study
This study sought to evaluate the dimensionality of a modified version of the RRS by means of examination of its factor structure, measurement invariance, and association with depression. Thus, the following objectives were developed: (a) to examine the factor structure of the RRS with exploratory factor analysis (EFA); (b) to compare possible factor structures with confirmatory factor analysis (CFA); (c) to appraise the relevance of each factor of the bifactor model; (d) to assess evidence of measurement invariance of the RRS with regard to sex (as a proxy for gender); and (e) to estimate how well each dimension of rumination predicts depression.
Method
Sample 1
A total of 1096 Peruvian undergraduate university students from a public university located in Lima (Peru) completed the questionnaire. Prior to data collection, we decided that the sample size should be at least 1000, so that each institutional academic field be represented by approximately 200 students. No formal power analysis was conducted. Fifty-two participants were excluded due to missing data, leaving a study sample of n = 1044. This was 50.1% male and 49.3% female; six participants did not report their sex. The mean age of the participants was 21.02 (SD = 3.19; range: 18–57). Some of them (18.6%) had a paid job. Also, most were freshmen (29.5%) or sophomores (31.4%). The majority belonged to programs in the engineering (25.6%), economic-business (20.8%), and healthcare areas (19.5%). Sample 1 was randomly split into two equally sized subsamples (named 1a and 1b) for exploratory and confirmatory analyses.
Sample 2
The sample consisted of 441 Peruvian undergraduate university students (46.6% female) from the same university as sample 1. This was a convenience sample with no a priori sample size determination. Ten individuals were excluded, leaving 431 participants as the final sample. Their mean age was 21.40 (SD = 2.52; range: 18–43). Most of them were second- or third-year students (33.2% and 33.6%, respectively) and enrolled in programs in the economic-business area (43.40%). Participants were selected through convenience non-probability sampling.
Measures
RRS (Treynor et al., 2003)
The RRS is a self-administered rumination questionnaire. For this study, the Spanish version of the RRS proposed by Cova et al., (2007) was used. Given the reported problems with the psychometric properties of the RRS, these authors added four new items to the original 10-item RRS (Cova et al., 2009). One of these items, however, performed poorly in their study (F. Cova, personal communication, August 23, 2016). Therefore, we decided to replace this item with a different one, which was developed based on rational criteria by the first author. The final 4 additional items were as follows: I try to understand why I feel this way (Reflection, created by the first author), I think over and over again about the bad things that have happened to me (Brooding), I try to find out why I feel like this (Reflection), and I remember all the times when bad things happened to me (Brooding). Also, minor adjustments were made to some items after unstructured preliminary testing. The resulting 14 items are responded on a 4-point Likert-type scale (1 = almost never to 4 = almost always). As will be described in the “Results” section, the scale was later shortened to 12 items with good psychometric properties, the details of which will be presented throughout this article.
The Beck Depression Inventory–II (BDI–II; Beck et al., 2006)
The BDI–II is a self-report questionnaire measuring depressive symptoms in the past 2 weeks. In this study, the Argentinian version, validated by Brenlla and Rodríguez (2006), was employed. It consists of 21 items that are scored from 0 to 3, and its psychometric properties have been examined in Peruvian undergraduate samples before (e.g., Sánchez-Villena & Cedrón, 2019). Even though the dimensionality of the BDI-II has been under debate, the evidence suggests that it is an essentially unidimensional measure (Brouwer et al., 2013). For this study, we conducted a series of exploratory and confirmatory analyses to select a set of items that best represented depression as a global latent variable. These items (1, 3, 7, 10, 12, 13, 14, and 21) reflected a single dimension with acceptable reliability (ωNL = 0.76).
Procedure
Data from both samples were collected in paper-and-pencil format. Potential participants were approached in their classrooms after asking the instructor in charge for permission. A booklet that contained the RRS was handed. It also included other measures that were not part of the present study: the Acceptance and Action Questionnaire II, the Positive and Negative Affect Schedule, the Emotion Regulation Questionnaire, and the Escala de Bienestar Psicológico para Adultos. Different versions of the booklet were randomly handed, so that the instruments were presented in different order. Overall, participants took approximately 15 min to complete all the scales. The first page of the booklet described the aims of the study, and explicitly stated that participation was voluntary and anonymous. Only participants who consented to participate completed the questionnaires. They received no financial or academic reward for their participation. This study was part of a project approved by the Vice-Rectorate for Research of Universidad Nacional Mayor de San Marcos.
Data Analysis
Two EFAs were conducted on subsample 1a. First, two factors were extracted using the MINRES method with Oblimin rotation. Second, a bifactor EFA was conducted with the Direct Schmid-Leiman approach proposed by Waller, (2018), which, according to a recent simulation study, is a recommended method (Giordano & Waller, 2020). In both cases, polychoric correlations were used.
After two possible solutions (oblique and bifactor) were derived from the EFA, they were tested on subsample 1b with CFAs using a robust weighted least squares estimator (WLSMV). Model fit was considered acceptable based on the following criteria: CFI > 0.95, TLI > 0.95, RMSEA < 0.06 (Hu & Bentler, 1999), and WRMR < 1 (DiStefano et al., 2018). Additionally, the bifactor model was also examined with the additional indices suggested by Rodriguez et al., (2016). Specifically, (1) if omega hierarchical (ωH) of the general Rumination factor was large compared to omega hierarchical subscale (ωHS) of the specific factors (Brooding and Reflection), (2) construct reliability (H) was larger than 0.70 for the general but not for the specific factors, and (3) explained common variance (ECV) by the general factor was 0.60 or larger (Reise et al., 2013), then the scale could be judged to be essentially unidimensional.
Next, measurement invariance was tested via multigroup CFA using sample 1 as a whole. The configural (baseline) model was compared to the more restricted scalar model, in which both factor loadings and item thresholds were constrained to be equal. Following best practices for ordinal indicators, a separate metric model (with only factor loadings set to be equal across groups) was not tested (Chen et al., 2020; Sass, 2011). A significant reduction in the Δχ2 test would be considered evidence of noninvariance. Approximate fit indices (e.g., ΔCFI) were not used for evaluating invariance because they have shown to be inadequate when categorical indicators are used (Sass et al., 2014). However, they are reported for the interested reader.
Finally, a full structural equation model was tested in sample 2. A new latent variable (Depression) was added to the bifactor model, and regression paths from both Rumination (general factor) and Brooding (specific factor) were estimated. Finally, a regression path from Reflection to Depression was also tested in an exploratory way. Since measurement invariance between sexes could not be established in sample 1, these analyses were conducted on males and females separately.
All the statistical analyses were computed in R 4.0.3. The following packages were used: psych 2.0.8, fungible 1.96.3, and lavaan 0.6–7.
Results
Exploratory Factor Analysis
The oblique EFA showed a pattern of factor loadings very similar to the original structure of the RRS. The only exceptions were items 4 (a Reflection item which had a stronger loading on Brooding) and 5 (a Reflection item which had negligible loadings on both factors). As shown in Table 1, similar results were found for the bifactor EFA, where each item (except for the two aforementioned problematic items) had non-negligible factor loadings on both the general Rumination factor and one specific factor.
Confirmatory Factor Analysis
The two competing models obtained from the previous analyses were tested in subsample 1b with CFA. The oblique two-factor model had acceptable fit according to the CFI and the TLI, but not according to the RMSEA and the WRMR, χ2(53) = 267.46, p < 0.001, CFI = 0.96, TLI = 0.95, RMSEA = 0.09, WRMR = 1.33. On the other hand, the bifactor model had better fit on most indices (except for the RMSEA), χ2(42) = 198.88, p < 0.001, CFI = 0.97, TLI = 0.95, RMSEA = 0.09, WRMR = 0.99. The factor loadings of both models are presented in the last columns of Table 1.
The bifactor model was further examined with additional indices. First, the omega hierarchical of the general Rumination factor (ωH = 0.75) was notably larger than the omega hierarchical subscale of both Brooding (ωHS = 0.17) and Reflection (ωHS = 0.41). On the other hand, construct reliability was strong for Rumination (0.88) and Reflection (0.70), but not for Brooding (0.55). Finally, explained common variance of the general factor was also large (ECV = 0.65). All in all, these results give partial support to the essential unidimensionality of the RRS. However, given that the specific factors do not seem to be purely residual, we decided to keep the bifactor model for the subsequent analyses.
Measurement Invariance
When tested separately on females (n = 515), the bifactor model had acceptable or marginally acceptable fit on most indices, χ2(42) = 191.32, p < 0.001, CFI = 0.97, TLI = 0.95, RMSEA = 0.08, WRMR = 1.02. Males (n = 523), on the other hand, showed results that were slightly better, χ2(42) = 135.94, p < 0.001, CFI = 0.98, TLI = 0.97, RMSEA = 0.07, WRMR = 0.80. A configural model in which both groups were included in a single CFA had similar acceptable but suboptimal fit, χ2(84) = 330.60, p < 0.001, CFI = 0.98, TLI = 0.96, RMSEA = 0.08, WRMR = 1.30. However, when scalar invariance was tested by constraining loadings and thresholds to be the same in both groups, model fit decreased significantly, Δχ2(45) = 100.64, p < 0.001, ΔCFI ≈ 0, ΔRMSEA ≈ 0.01. Even though we tried to achieve partial invariance by freeing several parameters, we were not able to do so. Consequently, measurement invariance between sexes was not supported.
Full Structural Equation Models
A new latent variable (Depression) was added to construct a full SEM. Two models were evaluated separately in men and women with sample 2 data. First, Depression was regressed on both Rumination and Brooding, but not on Reflection. Second, a model that also included a regression path from Reflection to Depression was also tested in an exploratory way. This second model did not significantly improve fit in the male sample (n = 228), Δχ2(1) = 0.05, p = 0.816. Thus, the final model for men was the one presented in Fig. 1, χ2(156) = 234.78, p < 0.001, CFI = 0.97, TLI = 0.96, RMSEA = 0.05, WRMR = 0.88.
When the second SEM was compared to the first one in the female sample (n = 198), model fit did show a significant improvement, Δχ2(1) = 5.15, p = 0.023. As shown in Fig. 2, the additional coefficient of this model indicates an inverse association between Reflection and Depression (β = − 0.25). Model fit was acceptable for this SEM, χ2(155) = 211.02, p = 0.002, CFI = 0.98, TLI = 0.98, RMSEA = 0.04, WRMR = 0.78.
Discussion
This study aimed to examine the factor structure and measurement invariance of the RRS, as well as its relationship with depression. The bifactor approach allowed us to examine the role of brooding and reflection while controlling for the effect of global rumination. Our results suggest that this bifactor structure is adequate and has better fit than the original two-factor oblique solution. However, measurement invariance of the bifactor model could not be established with regard to sex, so the RRS may not be measuring the same constructs in male and female participants. Two structural equation models were evaluated in men and women separately. In both cases, brooding had the strongest association with depression, followed by general rumination. With regard to reflection, it had a negative and significant association on the women sample, while it was nonsignificant for men.
The fact that the bifactor model was the best-fitting one is in line with previous research that found large correlations between brooding and reflection, thus suggesting the existence of a strong general factor (i.e., rumination; Lucena-Santos et al., 2018; Ruiz et al., 2017; Thanoi & Klainin-Yobas, 2015; Xavier et al., 2016). This fact notwithstanding, it should be also acknowledged that bifactor models tend to have better fit per se, even in situations in which the data are not bifactor (Markon, 2019). Consequently, it is important to examine additional indices that evaluate the relevance of all the factors of such a model (Rodriguez et al., 2016). When this was done, we found partial evidence that the RRS may be a unidimensional measure. However, it was also observed that the specific factors (especially Reflection) did explain an important amount of variance even after controlling for global rumination. Moreover, since we intended to test the claim that brooding and reflection show different associations with mental health criteria (Treynor et al., 2003), we decided to keep the bifactor model for further analyses.
This bifactor model was not invariant in men and women. This finding is in contrast to previous studies that found measurement invariance of the RRS across genders (Lei et al., 2017; Liang & Lee, 2019; Ruiz et al., 2017; Whisman et al., 2018; Xavier et al., 2016). Two possible explanations arise. First, it should be acknowledged that our model (i.e., bifactor) is different from the model tested in previous studies (i.e., oblique). Our model has additional parameters, is more complex and, therefore, it is less likely that all of them should be invariant across groups. Moreover, measurement invariance of such complex bifactors models has still been scarcely studied (Khojasteh & Lo, 2015). Second, it should also be noted that most existing studies on the invariance of the RRS were conducted using the maximum likelihood estimator (Lei et al., 2017; Liang & Lee, 2019; Whisman et al., 2018; Xavier et al., 2016). However, it is debatable whether such an approach is appropriate for Likert indicators that only have four response options (Brown, 2015; Rhemtulla et al., 2012).
It is clear from our data that the lack of invariance between men and women is statistically significant. There are no clear guidelines, however, to judge the practical significance of this difference. Even though the change in approximate fit indices (e.g., ΔCFI) is routinely used for this purpose (Cheung & Rensvold, 2002), these guidelines may be inappropriate for nonlinear models with categorical indicators (Sass et al., 2014). Thus, the lack of empirical criteria to evaluate the practical importance of the observed invariance prevents us from making claims in this regard. In consequence, we decided to proceed in a conservative way and our full SEM was analyzed in men and women separately.
The first notable finding of the SEM was that brooding was a stronger predictor of depression than global rumination. Thus, contrary to our hypothesis, it seems that the distinction between brooding and reflection is not a methodological artifact. Instead, as Treynor et al., (2003) initially proposed, they are distinct constructs that remain important even after their shared variance is taken into account.
The second surprising finding of the SEM was that reflection had a significant negative association with depression in the female sample. This result was unexpected and should not be taken for granted before it is replicated with larger samples. To the best of our knowledge, it is the first time that such a result is reported from cross-sectional data. Even though Treynor et al., (2003) did find an inverse association between reflection and depression, this was only observed in longitudinal data. Indeed, when these variables were examined cross-sectionally, the vast majority of studies found a positive association between them (Armey et al., 2009; Dinis et al., 2011; Lee & Kim, 2014; Schoofs et al., 2010; Shin et al., 2015).
It is possible that the negative association between reflection and depression was not evident in previous studies due the influence of global rumination on simple composite scores. Indeed, one advantage of the bifactor approach is that each factor can be examined separately while controlling for the others (Chen et al., 2006). As an approximate test of this possibility, we also examined the correlation between the composite scores of reflection and depression on the same female sample. Similar to previous studies, this correlation was positive and small-to-moderate, r(199) = 0.20, p = 0.005, thus giving some support to our tentative hypothesis.
The fact that the negative association between reflection and depression was only significant for women eludes a simple interpretation. First, it should be borne in mind that the bifactor model of the RRS was not invariant between sexes. Therefore, it is possible that the latent variable named “reflection” is not the same in both groups (i.e., we are measuring a different construct in men and women) (Dimitrov, 2010). Second, if we assume that the lack of invariance in our data is of no practical relevance, more substantial explanations can be forwarded. The following commentaries, however, shall remain speculative until further research addresses this point in more detail.
Past research found that the association between positive reframing and depression is significantly stronger in women, but sex was not a significant moderator of the relation between self-blame and depression (Kelly et al., 2008). Indeed, it has been suggested that engaging in self-reflection does not influence self-clarity in men, contrary to what is observed in women (Csank & Conway, 2004). Also, people who score high on masculinity tend to have less complex representations of their emotional life (Conway, 2000). These findings seem to imply that culturally based gender differences exist regarding the way in which people experience and regulate their emotions (Nolen-Hoeksema, 2012). Moreover, these differences seem to develop throughout the lifespan beginning with early interactions with parents (Root & Denham, 2010). For instance, it has been observed that parents discuss more emotional content with their daughters than with their sons (Fivush et al., 2000). Therefore, it could be that women are better trained to reflect on their emotional experiences and thus regulate their distress levels.
Limitations
Some limitations of the present study need to be considered. First, it should be noted that the participants from both samples were undergraduate students from one single university. This limits the generalization of our findings and requires them to be replicated in different populations. Second, all the variables were measured with self-reports. Even though this is a common practice in the field, future studies would benefit from including other data collection methods. Third, we did not collect data on gender, but relied on self-reported biological sex instead. Fourth, the sample size of sample 2 did not allow us to conduct a new test of invariance for the full structural equation models. However, since invariance was not found for the measurement model in sample 1, it can be assumed that the structural model was not invariant either. Finally, it should be noted that the two SEMs conducted on males and females separately may be under-powered; thus, caution should be taken when interpreting nonsignificant results. These limitations notwithstanding, the present study provides a new approach and new exploratory findings that should encourage further research.
Conclusion
The present study achieved a clear factorial structure of the RRS after some slight modifications. However, the fact that the bifactor model was not invariant between sexes should be examined in more detail in future studies. Users of the RRS should be aware that it may not be measuring the same constructs in men and women. Finally, the unexpected finding that reflection predicts depression after controlling for brooding and global rumination must be replicated in other samples before substantial conclusions can be made.
Data Availability
The data that support the findings of this study are available from the corresponding author upon request.
References
Arana, F. G., & Rice, K. G. (2017). Cross-cultural validity of the Ruminative Responses Scale in Argentina and the United States. Assessment. https://doi.org/10.1177/1073191117729204
Armey, M. F., Fresco, D. M., Moore, M. T., Mennin, D. S., Turk, C. L., Heimberg, R. G., Kecmanovic, J., & Alloy, L. B. (2009). Brooding and pondering: Isolating the active ingredients of depressive rumination with exploratory factor analysis and structural equation modeling. Assessment, 16(4), 315–327. https://doi.org/10.1177/1073191109340388
Beck, A. T., Steer, R. A., & Brown, G. K. (2006). Inventario de Depresión de Beck: BDI-II [Beck Depression Inventory II]. Paidós.
Bowen, N. K., & Masa, R. D. (2015). Conducting measurement invariance tests with ordinal data: A guide for social work researchers. Journal of the Society for Social Work and Research, 6(2), 229–249. https://doi.org/10.1086/681607
Brenlla, M. E., & Rodríguez, C. M. (2006). Adaptación argentina del Inventario de Depresión de Beck (BDI-II) [Argentinian adaptation of the Beck Depression Inventory II]. In A. T. Beck, R. A. Steer, & G. K. Brown (Eds.), Inventario de Depresión de Beck: BDI-II (pp. 11–38). Paidós.
Brouwer, D., Meijer, R. R., & Zevalkink, J. (2013). On the factor structure of the Beck Depression Inventory–II: G is the key. Psychological Assessment, 25(1), 136–145. https://doi.org/10.1037/a0029228
Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). Guilford Press.
Cernvall, M., Skogseid, E., Carlbring, P., Ljungman, L., Ljungman, G., & von Essen, L. (2016). Experiential avoidance and rumination in parents of children on cancer treatment: Relationships with posttraumatic stress symptoms and symptoms of depression. Journal of Clinical Psychology in Medical Settings, 23(1), 67–76. https://doi.org/10.1007/s10880-015-9437-4
Chen, F. F., West, S., & Sousa, K. (2006). A comparison of bifactor and second-order models of quality of life. Multivariate Behavioral Research, 41(2), 189–225. https://doi.org/10.1207/s15327906mbr4102_5
Chen, P.-Y., Wu, W., Garnier-Villarreal, M., Kite, B. A., & Jia, F. (2020). Testing measurement invariance with ordinal missing data: A comparison of estimators and missing data techniques. Multivariate Behavioral Research, 55(1), 87–101. https://doi.org/10.1080/00273171.2019.1608799
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233–255. https://doi.org/10.1207/S15328007SEM0902_5
Conway, M. (2000). On sex roles and representations of emotional experience: Masculinity, femininity, and emotional awareness. Sex Roles: A Journal of Research, 43(9), 687–698. https://doi.org/10.1023/A:1007156608823
Cova, F., Rincón, P., & Melipillán, R. (2009). Reflexión, rumiación negativa y desarrollo de sintomatología depresiva en adolescentes de sexo femenino [Reflection, negative rumination and development of depressive symptomatology in female adolescents]. Terapia Psicológica, 27(2), 155–160. https://doi.org/10.4067/S0718-48082009000200001
Cova, F., Rincón, P., & Melipillán, R. (2007). Rumiación y presencia de sintomatología ansiosa y depresiva en adolescentes [Rumination, anxiety and symptoms of depression in adolescents]. Revista Mexicana de Psicología, 24(2), 175–183. Retrieved July 4, 2020, from https://www.redalyc.org/articulo.oa?id=243020637003
Cowdrey, F. A., & Park, R. J. (2011). Assessing rumination in eating disorders: Principal component analysis of a minimally modified ruminative response scale. Eating Behaviors, 12(4), 321–324. https://doi.org/10.1016/j.eatbeh.2011.08.001
Cox, B. J., Enns, M. W., & Taylor, S. (2001). The effect of rumination as a mediator of elevated anxiety sensitivity in major depression. Cognitive Therapy and Research, 25(5), 525–534. https://doi.org/10.1023/A:1005580518671
Csank, P. A. R., & Conway, M. (2004). Engaging in self-reflection changes self-concept clarity: On differences between women and men, and low- and high-clarity individuals. Sex Roles: A Journal of Research, 50(7–8), 469–480. https://doi.org/10.1023/B:SERS.0000023067.77649.29
Dimitrov, D. M. (2010). Testing for factorial invariance in the context of construct validation. Measurement and Evaluation in Counseling and Development, 43(2), 121–149. https://doi.org/10.1177/0748175610373459
Dinis, A., Gouveia, J. P., Duarte, C., & Castro, T. (2011). Estudo de validação da versão portuguesa da Escala de Respostas Ruminativas—Versão Reduzida [Validation study of the Portuguese version of the Ruminative Responses Scale—Short form]. Psychologica, 54, 175–202. https://doi.org/10.14195/1647-8606_54_7
DiStefano, C., Liu, J., Jiang, N., & Shi, D. (2018). Examination of the weighted root mean square residual: Evidence for trustworthiness? Structural Equation Modeling: A Multidisciplinary Journal, 25(3), 453–466. https://doi.org/10.1080/10705511.2017.1390394
Fivush, R., Brotman, M. A., Buckner, J. P., & Goodman, S. H. (2000). Gender differences in parent–child emotion narratives. Sex Roles: A Journal of Research, 42(3–4), 233–253. https://doi.org/10.1023/A:1007091207068
Furr, R. M., & Bacharach, V. R. (2014). Psychometrics: An introduction (2nd ed.). SAGE.
Giordano, C., & Waller, N. G. (2020). Recovering bifactor models: A comparison of seven methods. Psychological Methods, 25(2), 143–156. https://doi.org/10.1037/met0000227
Grant, K. E., Lyons, A. L., Finkelstein, J.-A.S., Conway, K. M., Reynolds, L. K., O’Koon, J. H., Waitkoff, G. R., & Hicks, K. J. (2004). Gender differences in rates of depressive symptoms among low-income, urban, African American youth: A test of two mediational hypotheses. Journal of Youth and Adolescence, 33(6), 523–533. https://doi.org/10.1023/B:JOYO.0000048066.90949.be
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate data analysis (8th ed.). Cengage Learning.
Hasegawa, A. (2013). Translation and initial validation of the Japanese version of the Ruminative Responses Scale. Psychological Reports, 112(3), 716–726. https://doi.org/10.2466/02.08.pr0.112.3.716-726
Hernández-Martínez, A., García Cruz, R., Valencia Ortiz, A. I., & Ortega Andrade, N. A. (2016). Validación de la Escala de Respuestas Rumiativas para población mexicana [Validation of the Ruminative Response Scale for Mexican population]. Revista Latinoamericana De Medicina Conductual / Latin American Journal of Behavioral Medicine, 6(2), 66–74.
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
Hyde, J. S., & Mezulis, A. H. (2020). Gender differences in depression: Biological, affective, cognitive, and sociocultural factors. Harvard Review of Psychiatry, 28(1). https://doi.org/10.1097/HRP.0000000000000230
Iqbal, N., & Dar, K. A. (2015). Negative affectivity, depression, and anxiety: Does rumination mediate the links? Journal of Affective Disorders, 181, 18–23. https://doi.org/10.1016/j.jad.2015.04.002
Johnson, D. P., & Whisman, M. A. (2013). Gender differences in rumination: A meta-analysis. Personality and Individual Differences, 55(4), 367–374. https://doi.org/10.1016/j.paid.2013.03.019
Just, N., & Alloy, L. B. (1997). The response styles theory of depression: Tests and an extension of the theory. Journal of Abnormal Psychology, 106(2), 221–229. https://doi.org/10.1037/0021-843X.106.2.221
Kelly, M. M., Tyrka, A. R., Price, L. H., & Carpenter, L. L. (2008). Sex differences in the use of coping strategies: Predictors of anxiety and depressive symptoms. Depression and Anxiety, 25(10), 839–846. https://doi.org/10.1002/da.20341
Khojasteh, J., & Lo, W.-J. (2015). Investigating the sensitivity of goodness-of-fit indices to detect measurement invariance in a bifactor model. Structural Equation Modeling, 22(4), 531–541. https://doi.org/10.1080/10705511.2014.937791
Lee, S., & Kim, W. (2014). Cross-cultural adaptation, reliability, and validity of the revised Korean version of Ruminative Response Scale. Psychiatry Investigation, 11(1), 59–64. https://doi.org/10.4306/pi.2014.11.1.59
Lei, X., Zhong, M., Liu, Y., Xi, C., Ling, Y., Zhu, X., Yao, S., & Yi, J. (2017). Psychometric properties of the 10-item Ruminative Response Scale in Chinese university students. BMC Psychiatry, 17(1), 152. https://doi.org/10.1186/s12888-017-1318-y
Xavier, A., Cunha, M., & Pinto-Gouveia, J. (2016). Rumination in adolescence: The distinctive impact of brooding and reflection on psychopathology. The Spanish Journal of Psychology, 19, E37. https://doi.org/10.1017/sjp.2016.41
Liang, L., & Lee, Y.-H. (2019). Factor structure of the Ruminative Response Scale and measurement invariance across gender and age among Chinese adolescents. Advances in Applied Sociology, 9(6), 193–207. https://doi.org/10.4236/aasoci.2019.96016
Lloret-Segura, S., Ferreres-Traver, A., Hernández-Baeza, A., & Tomás-Marco, I. (2014). Exploratory Item Factor Analysis: A practical guide revised and updated. Anales de Psicología / Annals of Psychology, 30(3), 1151–1169. https://doi.org/10.6018/analesps.30.3.199361
Lucena-Santos, P., Pinto-Gouveia, J., Carvalho, S. A., da Oliveira, M., & S. (2018). Is the widely used two-factor structure of the Ruminative Responses Scale invariant across different samples of women? Psychology and Psychotherapy: Theory, Research and Practice, 91(3), 398–416. https://doi.org/10.1111/papt.12168
Luminet, O. (2004). Measurement of depressive rumination and associated constructs. In C. Papageorgiou & A. Wells (Eds.), Depressive rumination: Nature, theory and treatment 185–215. John Wiley & Sons, Ltd. https://doi.org/10.1002/9780470713853.ch10
Lyubomirsky, S., & Tkach, C. (2004). The consequences of dysphoric rumination. In C. Papageorgiou & A. Wells (Eds.), Depressive rumination: Nature, theory and treatment 21–41. John Wiley & Sons, Ltd. https://doi.org/10.1002/9780470713853.ch2
Markon, K. E. (2019). Bifactor and hierarchical models: Specification, inference, and interpretation. Annual Review of Clinical Psychology, 15, 51–69. https://doi.org/10.1146/annurev-clinpsy-050718-095522
McEvoy, P. M., & Brans, S. (2013). Common versus unique variance across measures of worry and rumination: Predictive utility and mediational models for anxiety and depression. Cognitive Therapy and Research, 37(1), 183–196. https://doi.org/10.1007/s10608-012-9448-5
Nolen-Hoeksema, S. (1987). Sex differences in unipolar depression: Evidence and theory. Psychological Bulletin, 101(2), 259–282. https://doi.org/10.1037/0033-2909.101.2.259
Nolen-Hoeksema, S. (1991). Responses to depression and their effects on the duration of depressive episodes. Journal of Abnormal Psychology, 100(4), 569–582. https://doi.org/10.1037/0021-843X.100.4.569
Nolen-Hoeksema, S. (2012). Emotion regulation and psychopathology: The role of gender. Annual Review of Clinical Psychology, 8, 61–87. https://doi.org/10.1146/annurev-clinpsy-032511-143109
Nolen-Hoeksema, S., & Morrow, J. (1991). A prospective study of depression and posttraumatic stress symptoms after a natural disaster: The 1989 Loma Prieta earthquake. Journal of Personality and Social Psychology, 61(1), 115–121. https://doi.org/10.1037/0022-3514.61.1.115
Nolen-Hoeksema, S., Parker, L. E., & Larson, J. (1994). Ruminative coping with depressed mood following loss. Journal of Personality and Social Psychology, 67(1), 92–104. https://doi.org/10.1037/0022-3514.67.1.92
Nolen-Hoeksema, S., Larson, J., & Grayson, C. (1999). Explaining the gender difference in depressive symptoms. Journal of Personality and Social Psychology, 77(5), 1061–1072. https://doi.org/10.1037/0022-3514.77.5.1061
Opwis, M., Schmidt, J., Martin, A., & Salewski, C. (2017). Gender differences in eating behavior and eating pathology: The mediating role of rumination. Appetite, 110, 103–107. https://doi.org/10.1016/j.appet.2016.12.020
Parola, N., Zendjidjian, X. Y., Alessandrini, M., Baumstarck, K., Loundou, A., Fond, G., Berna, F., Lançon, C., Auquier, P., & Boyer, L. (2017). Psychometric properties of the Ruminative Response Scale-short form in a clinical sample of patients with major depressive disorder. Patient Preference and Adherence, 11, 929–937. https://doi.org/10.2147/PPA.S125730
Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47(5), 667–696. https://doi.org/10.1080/00273171.2012.715555
Reise, S. P., Moore, T. M., & Haviland, M. G. (2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92(6), 544–559. https://doi.org/10.1080/00223891.2010.496477
Reise, S. P., Scheines, R., Widaman, K. F., & Haviland, M. G. (2013). Multidimensionality and structural coefficient bias in structural equation modeling: A bifactor perspective. Educational and Psychological Measurement, 73(1), 5–26. https://doi.org/10.1177/0013164412449831
Rhemtulla, M., Brosseau-Liard, P. E., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17(3), 354–373. https://doi.org/10.1037/a0029315
Ricarte Trives, J. J., Bravo, B. N., Postigo, J. M. L., Ros Segura, L., & Watkins, E. (2016). Age and gender differences in emotion regulation strategies: Autobiographical memory, rumination, problem solving and distraction. The Spanish Journal of Psychology, 19, E43. https://doi.org/10.1017/sjp.2016.46
Roberts, J. E., Gilboa, E., & Gotlib, I. H. (1998). Ruminative response style and vulnerability to episodes of dysphoria: Gender, neuroticism, and episode duration. Cognitive Therapy and Research, 22(4), 401–423. https://doi.org/10.1023/A:1018713313894
Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21(2), 137–150. https://doi.org/10.1037/met0000045
Roelofs, J., Muris, P., Huibers, M., Peeters, F., & Arntz, A. (2006). On the measurement of rumination: A psychometric evaluation of the ruminative response scale and the rumination on sadness scale in undergraduates. Journal of Behavior Therapy and Experimental Psychiatry, 37(4), 299–313. https://doi.org/10.1016/J.JBTEP.2006.03.002
Root, A. K., & Denham, S. A. (2010). The role of gender in the socialization of emotion: Key concepts and critical issues. In A. K. Root & S. A. Denham (Eds.), Focus on gender: Parent and child contributions to the socialization of emotional competence (pp. 1–9). Jossey-Bass.
Ruiz, F. J., Suárez-Falcón, J. C., Sierra, M. A., Barreto Montero, K., García Martín, M. B., Bernal, P. A., & Ramírez, E. S. (2017). Psychometric properties and factor structure of the Ruminative Responses Scale-Short Form in Colombia. International Journal of Psychology & Psychological Therapy, 17(2), 199–208.
Sánchez-Villena, A. R., & Cedrón, E. F. (2019). Análisis factorial exploratorio del Inventario de Depresión de Beck (BDI-II) en universitarios cajamarquinos [Exploratory factor analysis of Beck Depression Inventory (BDI-II) in college students from Cajamarca]. Interacciones. Revista de Avances en Psicología, 5(3), e177. https://doi.org/10.24016/2019.v5n3.177
Sass, D. A. (2011). Testing measurement invariance and comparing latent factor means within a confirmatory factor analysis framework. Journal of Psychoeducational Assessment, 29(4), 347–363. https://doi.org/10.1177/0734282911406661
Sass, D. A., Schmitt, T. A., & Marsh, H. W. (2014). Evaluating model fit with ordered categorical data within a measurement invariance framework: A comparison of estimators. Structural Equation Modeling: A Multidisciplinary Journal, 21(2), 167–180. https://doi.org/10.1080/10705511.2014.882658
Schoofs, H., Hermans, D., & Raes, F. (2010). Brooding and reflection as subtypes of rumination: Evidence from confirmatory factor analysis in nonclinical samples using the Dutch Ruminative Response Scale. Journal of Psychopathology and Behavioral Assessment, 32(4), 609–617. https://doi.org/10.1007/s10862-010-9182-9
Segerstrom, S. C., Tsao, J. C. I., Alden, L. E., & Craske, M. G. (2000). Worry and rumination: Repetitive thought as a concomitant and predictor of negative mood. Cognitive Therapy and Research, 24(6), 671–688. https://doi.org/10.1023/A:1005587311498
Smith, J. M., & Alloy, L. B. (2009). A roadmap to rumination: A review of the definition, assessment, and conceptualization of this multifaceted construct. Clinical Psychology Review, 29(2), 116–128. https://doi.org/10.1016/j.cpr.2008.10.003
Thanoi, W., & Klainin-Yobas, P. (2015). Assessing rumination response style among undergraduate nursing students: A construct validation study. Nurse Education Today, 35(5), 641–646. https://doi.org/10.1016/J.NEDT.2015.01.001
Topper, M., Molenaar, D., Emmelkamp, P. M. G., & Ehring, T. (2014). Are rumination and worry two sides of the same coin? A structural equation modelling approach. Journal of Experimental Psychopathology, 5(3), 363–381. https://doi.org/10.5127/jep.038813
Treynor, W., Gonzalez, R., & Nolen-Hoeksema, S. (2003). Rumination reconsidered: A psychometric analysis. Cognitive Therapy and Research, 27(3), 247–259. https://doi.org/10.1023/A:1023910315561
Waller, N. G. (2018). Direct Schmid-Leiman transformations and rank-deficient loadings matrices. Psychometrika, 83(4), 858–870. https://doi.org/10.1007/s11336-017-9599-0
Whisman, M. A., Miranda, R., Fresco, D. M., Heimberg, R. G., Jeglic, E. L., & Weinstock, L. M. (2018). Measurement invariance of the Ruminative Responses Scale across gender. Assessment. https://doi.org/10.1177/1073191118774131
Zanon, C., Dellazzana-Zanon, L. L., & Menga Junior, E. (2018). Adaptação e evidências de validade da Escala de Resposta Ruminativa no Brasil [Adaptation and validity evidence of the Ruminative Response Scale in Brazil]. Avaliação Psicológica, 17(2), 170–179. https://doi.org/10.15689/ap.2018.1702.13559.02
Shin, K. M., Cho, S.-M., & Kim, K.-H. (2015). A validation study of the Korean-Ruminative Response Scale in Korean adolescents. Psychiatry Investigation, 12(4), 508–515. https://doi.org/10.4306/pi.2015.12.4.508
Funding
This study was part of a project partially funded by the Vice-Rectorate for Research of the National University of San Marcos (RR No. 00876-R-16).
Author information
Authors and Affiliations
Contributions
Both authors have participated in the elaboration of the idea and the writing of the article.
Corresponding author
Ethics declarations
Consent to Participate
Informed consent was obtained from all individual participants included in the study.
Conflict of Interest
The authors declare no competing interests.
Rights and permissions
About this article
Cite this article
Valencia, P.D., Paredes-Angeles, R. Revisiting the Factor Structure of the Ruminative Response Scale: a Bifactor Approach. Trends in Psychol. (2022). https://doi.org/10.1007/s43076-022-00214-2
Accepted:
Published:
DOI: https://doi.org/10.1007/s43076-022-00214-2