A bifactor analysis of the Difficulties in Emotion Regulation Scale - Short Form (DERS-SF) in a sample of adolescents and adults

Abstract

The Difficulties in Emotion Regulation Scale (DERS) is a measure of emotion regulation that has been extensively used in research and clinical settings, both with adolescents and adults. However, its length has been recognized as an important limitation. In addition, its adequacy as a multidimensional measure and the utility and interpretability of the Awareness subscale has been questioned. The goal of this study is to contribute to the clarification of these issues through the examination of the factor structure and psychometric properties of a short form of the DERS in a large sample of Portuguese adolescents and adults from the community. Two studies were conducted. The first study comprised 1314 adults and the second study comprised 612 adolescents, who completed the DERS-SF and additional measures to assess validity evidence for DERS-SF scores in relation to other relevant variables (self-compassion, attachment orientations, anxiety and depressive symptoms, dispositional mindfulness, perceived stress, emotion suppression, and quality of life). Several models were examined in two studies as well as bifactor model-based psychometric indices. In both studies, bifactor models, particularly the one that excluded the Awareness items, exhibited a good fit to the data. ECV and OmegaH indices suggest a strong general factor of emotion dysregulation and do not support individual use of the Objectives, Strategies, Impulse, and Nonacceptance subscales. The DERS-SF scores correlated with the other measures as expected. Our findings suggest that the Awareness subscale should be excluded when computing the DERS-SF total score and that the total score can be considered essentially unidimensional. This study supports the use of a total score to assess emotion dysregulation but does not support the use of most subscale scores.

Emotion regulation can be broadly described as an ability to modulate the experience and expression of emotional states and responses. Gross (1998) defined emotion regulation as the “processes by which individuals influence which emotions they have, when they have them, and how they experience and express these emotions” (p. 275). For Thompson (1994), emotion regulation is “the extrinsic and intrinsic processes responsible for monitoring, evaluating, and modifying emotional reactions, especially their intensive and temporal features, to accomplish one’s goals” (pp. 27–28).

The ability to adaptively regulate emotions is a fundamental aspect of an individual’s adaptive functioning and mental health (Cole et al. 1994). Difficulties in regulating emotions or emotional dysregulation have been consistently described as a transdiagnostic vulnerability factor that is linked to numerous difficulties and forms of psychopathology across the lifespan (Sheppes et al. 2015), including alcohol use (Dragan 2015), depression (Joormann and Stanton 2016; Schäfer et al. 2017), bipolar disorder (Van Rheenen et al. 2015), borderline personality disorder (Salsman and Linehan 2012), anxiety disorders (Cisler and Olatunji 2012), eating disorders (Brockmeyer et al. 2014), and post-traumatic stress (McLean and Foa 2017). Given the prevalence of emotion regulation difficulties in several forms of psychopathology and their importance for the individual’s psychological functioning (Berking and Wupperman 2012), a psychometrically robust and brief measure of emotion dysregulation that can be applied to adolescents and adults is needed both in clinical and in research contexts.

The Difficulties in Emotion Regulation Scale

The Difficulties in Emotion Regulation Scale (DERS; Gratz and Roemer 2004) is one of the most widely used measures of emotion regulation problems. This scale is based on an integrative conceptualization of emotion regulation and some important assumptions were considered in its development: 1) the ability to monitor and evaluate the emotional experience (i.e., the awareness and understanding of emotions) is an important aspect of an adaptive emotion regulation (Thompson and Calkins 2009). Therefore, difficulties in emotion regulation might be related to a diminished capacity to experience and differentiate the full range of emotions and does not necessarily imply emotional control or an attempt to diminish negative affect or attenuate and modulate strong negative emotions (Thompson 1994; Gross and Muñoz 1995; Cole et al. 1994); 2) avoidance of internal experiences and attempts to control the experience and expression of emotions may paradoxically increase the dysregulation of emotions (Hayes et al. 2006), whereas the acceptance of emotions may contribute to emotion regulation (Cole et al. 1994); 3) adaptive emotion regulation involves changing the intensity or duration of an emotion rather than changing or eliminating the emotion (Thompson 1994). This ability to modulate the intensity or duration of an emotion can reduce the urgency associated to that emotion, allowing the individual to inhibit impulsive behaviors and act in accordance with their goals (Linehan 1993); and 4) the effectiveness of emotion regulation strategies depends on the context, i.e., on the demands of the situation and the goals of the individual (Thompson 1994; Cole et al. 1994).

Based on this conceptualization, Gratz and Roemer (2004) sought to develop a multidimensional scale to measure difficulties in four main dimensions of emotion regulation: (a) awareness and understanding of emotions; (b) acceptance of emotions; (c) the ability to engage in goal-directed behavior and to control impulsive behavior in the presence of negative emotions; and (d) access to emotion regulation strategies perceived by the individual as effective. An exploratory factor analysis, conducted among a sample of college students, resulted in a six-factor solution slightly different from this initial theoretical conceptualization. Specifically, the authors found that difficulties engaging in goal-directed behaviors and impulse control difficulties were distinct factors, as well as the lack of emotional awareness and the lack of emotional clarity. The following six dimensions were found: 1) Lack of Emotional Awareness (Awareness; assesses a tendency toward inattention to and a lack of awareness of emotions); 2) Nonacceptance of Emotional Responses (Nonacceptance; assesses a tendency to display negative secondary reactions to negative emotions and/or distress); 3) Difficulties Engaging in Goal-Directed Behavior (Goals; assesses one’s difficulties in completing tasks and in concentrating when experiencing negative emotional states); 4) Limited Access to Emotion Regulation Strategies (Strategies; assesses the belief that little can be done to regulate emotions when one is upset); 5) Impulse Control Difficulties (Impulse; assesses one’s difficulties in controlling behavior when experiencing negative emotional states; and 6) Lack of Emotional Clarity (Clarity; assesses the degree of individuals’ knowledge and clarity about their emotion states). Cronbach’s alphas were above .80 for all subscales.

The DERS has been used worldwide and has been translated into several languages, including European Portuguese (Coutinho et al. 2010), Brazilian Portuguese (Miguel et al. 2017), Greek (Mitsopoulou et al. 2013), Italian (Giromini et al. 2012), Turkish (Ruganci and Gencoz 2010), Dutch (Neumann et al. 2010), French (Dan-Glauser and Scherer 2013), Korean (Cho and Hong 2013), and Spanish (Gomez-Simon et al. 2014). It has demonstrated adequate psychometric properties (e.g., good internal consistency and construct validity) across different cultures and populations, including adults (e.g., Salsman and Linehan 2012; Anderson et al. 2016) and adolescents (e.g., Neumann et al. 2010) and clinical (Van Rheenen et al. 2015) and non-clinical (Giromini et al. 2017) samples. The European Portuguese version of the DERS presented adequate reliability, with Cronbach’s alphas raging from .74 (Awareness) to .88 (Strategies) and test-retest correlations ranging from .67 (Awareness) to .81 (Strategies). The factor structure was explored through PCA and was shown to be very similar to the structure of the original scale (only two items loaded on a different factor than the original scale). In addition, positive and significant associations were found between all DERS subscales and symptoms of psychopathology.

Despite the extensive use of the DERS, its length has been identified as one important limitation (Kaufman et al. 2015; Bjureberg et al. 2016). Many of the items within each subscale are quite similar and may be perceived by participants as repetitive and tiresome, which may compromise the validity of participants’ responses and decrease their response rates. Therefore, a shortened version of this instrument could be very useful in research and clinical settings. Recently, Kaufman et al. (2015) proposed a short form of the DERS with 18 items (DERS-SF). This shortened version confirmed the original six-factor structure of the scale and presented very good psychometric properties, including adequate reliability (Cronbach’s alphas ranged between .78 and .91) and concurrent validity, in samples of adults and adolescents aged between 12 and 18 years from the United States. In addition, correlations between the DERS and the DERS-SF subscales ranged from .90 to .98, reflecting 81% to 96% of shared variance between the short and original versions of the DERS.

Despite the advantages of a short version of the scale, research on its psychometric properties and cross-cultural suitability has been lacking. Therefore, in the present study, we intend to validate the Portuguese version of the DERS-SF and contribute to its investigation and dissemination.

Psychometric Limitations of the Difficulties in Emotion Regulation Scale

Notwithstanding the utility and widespread use of DERS, some important issues have been raised regarding its factor structure. In particular, the Awareness subscale has shown some problematic issues, such as lower internal consistency and weaker correlations with the DERS total score and with the other subscales, as opposed to the other DERS subscales, which usually have adequate reliability and correlate strongly with each other (e.g., Gratz and Roemer 2004; Neumann et al. 2010; Tull et al. 2007; Marques et al. 2018; Bardeen et al. 2012). This subscale has also demonstrated weaker correlations than the other subscales with variables that are expected to be correlated with emotion regulation (e.g., Salters-Pedneault et al. 2006; McDermott et al. 2009; Marques et al. 2018). These results suggest that the Awareness subscale might not assess the same construct as the remaining five subscales and therefore may not contribute to an emotion dysregulation total score. To investigate this hypothesis, Bardeen et al. (2012) examined several models through confirmatory factor analyses (CFA), including the original correlated six-factor model (Gratz and Roemer 2004), one-factor and second-order models with all items, and alternative five-factor models that excluded the Awareness subscale (including a one-factor, correlated, and second-order models). The authors concluded that the Awareness subscale seemed to not assess the same higher-order emotion regulation construct as the other five subscales and that a revised five-factor model without the Awareness items could be a better fit to the data. Therefore, they recommended that the Awareness items should be removed when computing the DERS total score. Bardeen et al. (2012) suggested that if, on the one hand, the results found may indicate that in the temporal sequence of emotional regulation, Awareness (and Clarity) precede the effective use of strategies, which may not necessarily lead to the regulation of emotions (we may be aware of emotions and do nothing to regulate them), on the other hand, the results may be due to a method effect (i.e., to the fact that the Awareness subscale is the only DERS subscale that includes only reverse-coded items).

To confirm whether the results regarding the Awareness subscale were due to a method effect, Bardeen et al. (2016) tested a modified version of the scale in which all reverse-coded items were rewritten in a straightforward manner. Through exploratory and confirmatory factor analyses, the authors found that a modified five-factor version with 29 items (none of which was reverse coded) with the Awareness and Clarity items loading on the same factor (named Identification) was reliable and valid. Interestingly, the Identification factor was consistent with the original conceptualization of Gratz and Roemer (2004), according to which awareness and understanding of emotions are part of the same dimension. Cronbach’s alphas ranged from .88 to .95 and significant correlations were found between the DERS subscales, including the Identification subscale, and measures of emotion regulation, experiential avoidance, anxiety, and depression. These results support the hypothesis that a method error related to reverse-coded items could explain the problems found with the Awareness subscale.

Recent studies tested bifactor models to better examine the factor structure of the long and short forms of the DERS and determine the adequacy of computing a total score of emotion dysregulation and of using different subscales. Bifactor models allow for the examination of whether an instrument comprises a general factor explaining some proportion of common item variance for all items (in this case, an emotion dysregulation factor) as well as multiple domain-specific factors (in this case, the different DERS subscales) accounting for the unique influence of the specific dimension or subscale (i.e., the shared variance in their set of items) over and above the general factor (Chen et al. 2006; Reise et al. 2010).

Osborne et al. (2017) were the first to test a bifactor model in the DERS scale to explore the issue of the multidimensionality versus the unidimensionality of the scale and the suitability of computing a DERS total score, as well as to better examine the problematic issues previously found with the Awareness subscale. The authors tested several models (unidimensional, correlated, higher-order, and bifactor) in a clinical sample of patients receiving dialectical behavior therapy and found that a modified bifactor model was the model that best fit the data. In this model, the Awareness items were not allowed to load on the general factor, and the Awareness factor was only allowed to correlate with the Clarity factor. In general, items loaded more strongly on the general factor than on the specific factors, which supports the computation and utilization of a total score of emotion dysregulation. However, for the Clarity and Nonacceptance subscales, most of the item loadings were stronger on the specific factor than on the general factor, which also supports the utilization of these subscales.

In addition, in the Osborne et al. (2017) study, the omega coefficient for the 30-item total score was .96 and ranged from .85 to .91 for the subscales, indicating that both the total score and the subscales scores are internally consistent. The authors also found that the omega hierarchical (OmegaH) was .83 for the total score, which suggests that most of the variance in the total score is explained by the general factor. The omega hierarchical subscale (OmegaHS; i.e., the proportion of reliable systematic variance of a given subscale score after partitioning out variability attributed to the general factor) ranged between .16 and .70. Specifically, Clarity and Nonacceptance were the factors that presented higher values (.70 and .54, respectively), which supports the individual use of theses subscales. Lower values were found for the Goals and Impulse subscales (.34 and .36, respectively), but the authors still contend that “enough variance was explained by the respective group factors to retain their use as subscales” (p. 364). Finally, the omegaHS for Strategies was only .16, which calls into question the adequacy of using this subscale. Therefore, Osborne et al. (2017) concluded that the Awareness items should be excluded when computing the total DERS score and recommended caution when using the DERS subscales. Based on their results, they concluded that “there was solid evidence to support the use of two of the subscales (Clarity and Nonacceptance), fair support for the use of two additional subscales (Goals and Impulse), and minimal support for the use of the Strategies subscale” (p. 369). Therefore, they recommend using the DERS total score as an index of emotion regulation problems in clinical settings and suggest that some subscales may be more useful in research settings.

Benfer et al. (2018) also tested several models, including a bifactor model, in the original DERS and in the modified version proposed by Bardeen et al. (2016) (i.e., a five-factor version with 29 items, none of which reverse coded, and with the Awareness and Clarity items loading on Identification; DERS-M) in a sample of adults from the general community. Contrary to Osborne et al. (2017), these authors examined the bifactor model allowing all items to load onto the general factor. In this study, none of the models tested in the original DERS presented a good fit to the data. In addition, consistent with previous investigations, the Awareness subscale presented small to medium correlations with all subscales, except with the Clarity subscale (with which it correlated strongly). In contrast, when using the DERS-M, the correlated, second-order, and bifactor models demonstrated an adequate fit to the data. As the best-fitting model was the bifactor model, several bifactor statistical indices were analyzed. The high omega hierarchical for the total score (.92) and the low omega hierarchical for subscales (from .09 for the Strategies subscale to .56 for the Identification subscale) suggest that most of the reliable variance in DERS-M scores is attributable to the general factor. However, according to the authors, an ECV of .68 suggests that although the general factor accounts for most of the variance, it is not large enough to indicate unidimensionality. Therefore, the authors concluded that the DERS-M should be used instead of the original DERS and that the degree of multidimensionality of this modified version is considerable enough to justify the use of the subscales, with the exception of the Strategies subscale, even if the general factor is strong.

Hallion et al. (2018) also tested a bifactor model with one general factor of emotion dysregulation and five uncorrelated specific factors (excluding the Awareness subscale) in a sample of treatment-seeking adults with emotional disorders. The model fit was acceptable and all items loaded significantly on the general factor (excluding two items from the Clarity subscale) and on their specific factor (excluding two items from the Strategies subscale). The bifactor model was also tested in three short forms of the DERS, including the DERS-SF (Kaufman et al. 2015). The bifactor model excluding the Awareness subscale presented a very good fit to the data and Cronbach’s alphas ranged from .92 (Clarity) to .98 (Nonacceptance and total score). These authors concluded that when excluding the Awareness items, the DERS and its short forms are robust measures of the difficulties in emotion regulation in a clinical population.

In addition to the problems identified in the Awareness subscale, some authors suggested that the Strategies subscale might also not adequately represent the construct evaluated by the remaining DERS subscales. Medrano and Trógolo (2016) argued that Strategies items reflect emotional self-efficacy and, particularly, emotional self-efficacy beliefs (i.e., the extent in which individuals believe they are able to reduce negative emotions). According to these authors, emotional self-efficacy is a precursor of emotion regulation and, therefore, a more accurate operationalization of DERS should exclude the Strategies items. In fact, Gratz and Roemer (2004) explained that the Strategies dimension assessed the subjective appraisal of the effectiveness of contextually appropriate emotion regulation strategies to modulate emotional responses, rather than specific strategies (in an effort to consider the contextually dependent nature of emotion regulation strategies). To test their hypothesis, the authors tested several correlated and hierarchical models including and excluding the Awareness and the Strategies subscales. The results of the CFA showed that the four-factor models that excluded the Awareness and Strategies subscales adjusted to the data better than the models including those subscales.

In addition, based on studies from affective neuroscience, Medrano and Trógolo (2016) suggested that Awareness, Nonacceptance and Clarity subscales could represent a higher-order factor assessing difficulties in emotion processing, whereas Goals and Impulse subscales could represent a higher-order factor assessing difficulties in the regulation of the emotional response. Based on these hypotheses, these authors tested a hierarchical model with two higher-order factors (difficulties in emotion processing and difficulties in the regulation of the emotional response) and the five or four second-order factors corresponding to the DERS subscales (one model included Awareness and the other model excluded this subscale; both excluded Strategies). The best fitting model was the one in which Clarity and Nonacceptance were accounted by a second-order factor of “difficulties in emotion processing”, and Goals and Impulse were accounted by a second-order factor of “difficulties in the regulation of emotional response”. As both the four-factor correlated model and the second-order model with two higher-order factors (excluding Awareness and Strategies) presented a similarly good fit to the data, the authors suggested that difficulties in emotion regulation could be conceptualized as involving lack of emotional clarity and of emotional acceptance and difficulties in goal-directed behavior and in impulse control; or as a construct involving difficulties in emotion processing and in the regulation of the emotional response. To date, these findings were not yet replicated.

Taken together, the findings from the abovementioned studies suggest that the Awareness and Strategies subscales might not assess the same emotion regulation construct as the remaining subscales. However, additional studies are needed to clarify these issues and to investigate whether problems with these subscales hold for different sample and for the short version of the DERS.

The Present Study

The present study intends to examine the factor structure and psychometric properties of the Portuguese version of a short form of the DERS (DERS-SF; Kaufman et al. 2015) in two independent studies: one study with a sample of adults from the general community and another study with a sample of adolescents aged between 12 and 19 years from the general community. Since this instrument, including the short form, can be administered to both adults and adolescents, we intend to examine whether the Portuguese version of DERS-SF is equally appropriate for both age groups. In addition, this study aims to extend previous investigations on the factor structure of the scale and examine whether the previously identified psychometric limitations of the Awareness and Strategies subscales hold for the DERS-SF. The present study also intends to contribute to the cross-cultural validation of the DERS-SF. The psychometric properties of this short form have not been analyzed in non-English speaking populations, which is essential to establishing the psychometric robustness of a scale and to enabling its utilization in other cultures.

Therefore, the first goal of this study is to comprehensively examine the factor structure of the DERS-SF in the adolescents and adults samples. Specifically, we intend to analyze the usefulness and adequacy of the Awareness and Strategies subscales and test the adequacy of computing a total score or calculating subscale scores. Therefore, several competing models that have been suggested in previous research, including bifactor models, will be investigated.

The second goal of this study is to examine the validity of the DERS-SF scores in relation to other variables. The validity will be examined by analyzing the correlations between the DERS-SF and variables that are expected to be associated with emotion regulation (self-compassion, attachment, dispositional mindfulness, perceived stress, anxiety and depression symptoms, quality of life, and emotion suppression). Based on previous investigations, we expect higher levels of emotion dysregulation to be associated with lower levels of self-compassion (Finlay-Jones et al. 2015; Barlow et al. 2017), mindfulness (Desrosiers et al. 2013; Roemer et al. 2009), and quality of life (Phillips et al. 2014), and with higher levels of attachment avoidance and anxiety (Marques et al. 2018), stress (Finlay-Jones et al. 2015), and anxiety and depression symptoms (Desrosiers et al. 2013; Marques et al. 2018).

Study 1

In Study 1, we examine the factor structure of the DERS-SF in a sample of Portuguese adults recruited in different settings. Additionally, we explore the validity of the scale scores by exploring the associations between the DERS-SF total and subscale scores and the scores from the measures of self-compassion, attachment representations, anxiety and depression symptoms, dispositional mindfulness, emotion suppression and perceived stress.

Method

Participants and Procedure

Participants in Study 1 were 1314 adults participating in three different studies that used the DERS-SF. Authorization for the sample collection in the three studies was obtained from the Portuguese Data Protection Authority and from the Ethics Committee of the Faculty of Psychology and Education Sciences of the University of Coimbra.

Sample 1

The participants in sample 1 were 748 women participating in a study about parenting issues. They had a mean age of 41.26 years old (SD = 5.54, range = 22–60), and the majority were married or living with a partner (n = 648, 86.6%), lived in rural areas (n = 590, 80.7%), had completed basic or secondary education (n = 534, 71.4%), were employed (n = 597, 79.8%), and had a monthly household income less than 2000€ (n = 632, 86.2%). This sample was collected in two public school units in central Portugal. Parents received a letter through their children explaining the study, an informed consent form, and a packet with the questionnaires to be completed at home and returned a week later. All respondents provided written informed consent. Authorization for the sample collection was also obtained from the Board of Directors of Schools.

Sample 2

The participants in sample 2 were 285 mothers of a child or adolescent with overweight or obesity who were participating in a study about parenting and emotion regulation in pediatric obesity. The participants had a mean age of 41.28 years old (SD = 5.91, range = 24–58), and the majority were married or living with a partner (n = 245, 86%), lived in rural areas (n = 194, 68.6%), had completed basic or secondary education (n = 235, 82.5%), were employed (n = 221, 77.5%), and had a monthly household income less than 2000€ (n = 260, 91.2%). Participants were collected in two pediatric departments of two public hospitals in central Portugal. The only criterion to participate in the study was having a child with a body mass index (BMI) between the 85th and the 97th percentile (overweight) or equal to or above the 97th percentile (obesity) for children and teens of the same age and sex according to the WHO Child Growth Standards (WHO, 2006) who was being followed at a nutrition consultation to lose or control weight. After the nutrition consultation, a research assistant approached the parents and invited them to participate in the study. Those who agreed to participate in the study completed the questionnaires at the hospital or took them home and returned them later by mail. All participants provided written informed consent. The Ethics Committees and the Board of Directors of both hospitals approved the study.

Sample 3

The participants in sample 3 were 281 women participating in an online study about parenting issues. They had a mean age of 36.99 years old (SD = 5.40, range = 23–54), and the majority were married or living with a partner (n = 258, 91.8%), had completed a college degree (n = 239, 85.1%), were employed (n = 244, 86.8%), and had a monthly household income less than 2000€ (n = 165, 59.1%). All women reported living in an urban area and were from all regions of Portugal, particularly from the Lisbon Metropolitan Area (n = 119, 42.3%) and Northern (n = 68, 24.2%) and Central Portugal (n = 58, 20.6%). The only criteria to participate in the study were being between the ages of 18 and 65 years old and having at least one child between the ages of 1 and 18 years old. Participants completed the questionnaires through a data collection website (LimeSurvey®). The survey link was shared on social networks and through email. The first page of the online protocol provided a brief description of the study objectives, the inclusion criterion, and the ethical issues of the study. The participants were assured that their participation in the study was anonymous and that no identifying information would be collected. Those who provided informed consent by clicking on the option “I understand and accept the conditions of the study” were granted access to the assessment protocol.

Measures

Participants from samples 1 and 2 completed measures assessing difficulties in emotion regulation, self-compassion, attachment representations, anxiety and depression symptoms, and dispositional mindfulness. Participants from sample 3 completed measures assessing difficulties in emotion regulation, self-compassion, attachment representations, emotion suppression, and perceived stress. All measures are validated to the Portuguese population and present adequate psychometric properties.

Difficulties in emotion regulation

The Difficulties in Emotion Regulation Scale – Short Form (DERS-SF; Kaufman et al., 2015) consists of 18 items that load onto six subscales: (1) Nonacceptance of Emotional Responses (e.g., “When I’m upset, I feel guilty for feeling that way”), (2) Difficulties Engaging in Goal-Directed Behavior (e.g., “When I’m upset, I have difficulty concentrating”), (3) Impulse Control Difficulties (e.g., “When I’m upset, I become out of control”), (4) Lack of Emotional Awareness (e.g., “I pay attention to how I feel”), (5) Limited Access to Emotion Regulation Strategies (e.g., “When I’m upset, it takes me a long time to feel better”), and (6) Lack of Emotional Clarity (e.g., “I am confused about how I feel”). Each subscale has three items, which are answered on a five-point Likert scale that ranges from 1 (almost never) to 5 (almost always). The subscale scores consist of the mean of the items, with higher scores indicating more difficulties in regulating emotions. Items from the Awareness subscale are reverse coded. Items on the Portuguese short form version of the DERS were extracted from the Portuguese 36-item scale (Coutinho et al. 2010).

Self-compassion

The Self-Compassion Scale – Short Form was used to assess self-compassion (SCS-SF; Castilho et al. 2015; Raes et al. 2011). This instrument has 12 items (e.g., “I try to be understanding and patient toward those aspects of my personality I don’t like”) rated on a five-point Likert scale ranging from 1 (almost never) to 5 (almost always). The total score is the mean of the items, with higher scores indicating higher levels of self-compassion. In the combined sample of the present study, Cronbach’s alpha was .82. In the subsamples, Cronbach’s alphas were .97 (Sample 1), .76 (Sample 2), and .91 (Sample 3).

Attachment representations

The Experiences in Close Relationships – Relationship Structures questionnaire (ECR-RS; Moreira et al. 2015; Fraley et al. 2011) was used to assess attachment-related anxiety (e.g., “I worry that this person won’t care about me as much as I care about him or her”) and avoidance (e.g., “I don’t feel comfortable opening up to this person”) in close relationships in general. The ECR-RS is composed of nine items rated on a seven-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree). The subscale scores consist of the mean of the items, with higher scores indicating higher attachment avoidance and anxiety. Cronbach’s alphas in the combined sample of the present study were .86 for anxiety and .77 for avoidance. In the subsamples, Cronbach’s alphas were .85 (Sample 1), .90 (Sample 2), and .87 (Sample 3) for anxiety; and .72 (Sample 1), .77 (Sample 2), and .86 (Sample 3) for avoidance.

Anxiety and depression symptoms

The Hospital Anxiety and Depression Scale (HADS; Pais-Ribeiro et al. 2007; Zigmond and Snaith 1983) was used to assess levels of depressive and anxious symptomatology in the previous seven days. The scale contains 14 items and uses a four-point scale that ranges from 0 (not at all/only occasionally) to 3 (most of the time/a great deal of the time), with higher scores indicating higher levels of symptomatology. In the combined sample of the present study, the Cronbach’s alpha coefficients were .78 for anxiety symptoms and .75 for depression symptoms. When considering the subsamples, Cronbach’s alpha were .79 (Sample 1) and .76 (Sample 2) for anxiety symptoms; and .75 (Sample 1) and .73 (Sample 2) for depression symptoms.

Dispositional mindfulness

Dispositional mindfulness was assessed by the Mindful Attention and Awareness Scale (MAAS; Brown and Ryan 2003; Gregório and Pinto-Gouveia 2013). This measure has 15 items (e.g., “I rush through activities without being really attentive to them”) answered on a six-point Likert-type response scale ranging from 1 (almost never) to 6 (almost always). Higher scores reflect higher levels of dispositional mindfulness. In this study, the Cronbach’s alpha coefficient was .91 for the combined sample and for Sample 1 and .90 for Sample 2.

Perceived stress

The Perceived Stress Scale (PSS; Trigo et al. 2010; Cohen et al. 1983) was used to assess the extent to which different situations in an individual’s life were appraised as stressful, unpredictable and uncontrollable in the last month. This unidimensional self-report questionnaire comprises ten items (e.g., “In the last month, how often have you felt nervous and ‘stressed’?”) answered on a five-point Likert-type response scale ranging from 0 (never) to 4 (very often). The total score is the sum of the items, and higher scores reflect higher levels of perceived stress. In the present study, Cronbach’s alpha was .89 (Sample 3).

Emotion suppression

The Emotion Suppression subscale of the Emotion Regulation Questionnaire (ERQ; Gross and John 2003; Vaz 2009) was used to assess emotion suppression. This subscale has four items (e.g., “I keep my emotions to myself”) answered on a seven-point Likert-type response scale ranging from 1 (strongly disagree) to 4 (strongly agree), with higher scores indicating higher levels of suppression. In the present study, Cronbach’s alpha was .81 (Sample 3).

Statistical analyses

Preliminary analyses were performed to assess whether missing values were missing completely at random and whether the data followed a normal distribution. Following the recommendations of Kline (2011), normality was assessed through examination of skewness and kurtosis of each individual item. The data distribution was considered non-normal if the values of skewness and kurtosis were above 3 and 10, respectively. The presence of outliers was determined by examining leverage indices and Mahalanobis distance (D²) statistic for all participants. A leverage score five times greater than the sample leverage value (Brown 2006) and a D² value that particularly departs from all the other D² values (Byrne 2010) were considered as an outlier.

Several models were examined through confirmatory factor analyses (CFA) conducted in AMOS© 22 using maximum likelihood estimation (see Fig. 1). The first set of models tested the structure of the complete DERS-SF (18 items) and included a unidimensional model in which all the items loaded on a single factor of Emotion Dysregulation (Model 1); the original correlated six-factor model in which all DERS-SF dimensions were allowed to correlate (Gratz and Roemer, 2004; Model 2); a second-order model in which items were organized into the six dimensions and in a second-order factor of Emotion Dysregulation (Model 3); and a bifactor model in which all items loaded on a general factor (emotion dysregulation) and had nonzero loadings on the domain-specific factor that they were designed to measure and zero loadings on the other factors; and in which specific factors were not correlated with each other and error terms that were associated with each item were not correlated (Model 4).

Fig. 1

Models examined through confirmatory factor analyses

The second set of models included modified versions of the scale. Following recommendations to exclude the Awareness subscale (Bardeen et al. 2012; Osborne et al. 2017; Medrano and Trógolo 2016; Hallion et al. 2018), several models excluding this subscale were examined: a unidimensional model (Model 5), a correlated five-factor model (Model 6), a second-order model (Model 7), and a bifactor model (Model 8; Hallion et al. 2018). Following recommendations to exclude the Strategies subscale (Medrano and Trógolo 2016), a correlated five-factor model excluding the Strategies subscale (Model 9) was also tested. Based on the suggestion that the DERS subscales could be organized into two higher order factors assessing difficulties in emotion processing and difficulties in the regulation of emotional response (Medrano and Trógolo 2016), a bifactor model including two correlated general factors and all DERS subscales was also tested (with Strategies, Impulse, and Goals loading on the “difficulties in the regulation of emotional response” factor, and Awareness, Clarity, and Nonacceptance loading on the” difficulties in emotion processing” factor; Model 10). Based on the initial conceptualization of the scale proposed by Gratz and Roemer (2004), which combined Awareness and Clarity into one factor, we tested a model similar to Model 10, but only Awareness and Clarity loaded on the general factor “ difficulties in emotion processing “ and the remaining subscales loaded on the “ difficulties in the regulation of emotional response” factor (Model 11). Finally, the Osborne et al.’s (2017) modified bifactor model, in which the Awareness factor remained in the model and correlated with Clarity but the three items were not allowed to load on the general factor (Model 12), was tested.

The fit of the models was assessed through the comparative fit index (CFI), the Tucker-Lewis Index (TLI), the root-mean-square error of approximation (RMSEA), and the standardized root-mean-square residual (SRMR). The cut-offs for adequate and good model fit were CFI and TLI values ≥ .90 and ≥ .95, RMSEA values ≤ .08 and ≤ .06, and SRMR values ≤ .10 and ≤ .08, respectively (Browne and Cudeck 1993; Hu and Bentler 1999). The Akaike Information Criterion (AIC; Akaike 1987) was used to compare the models. The model with the smallest AIC values was considered the best-fitting model (Kline 2011). Factor loadings of .32 or above were considered meaningful (Tabachnick and Fidell 2007).

Several bifactor model-based psychometric indices were computed: the explained common variance (ECV; Sijtsma 2009; Ten Berge and Sočan 2004), the percentage of uncontaminated correlations (PUC; Reise et al. 2013b; Bonifay et al. 2015), and the omega reliability coefficients, including the omega (ω), omega subscale (ωS), omega hierarchical (ωH), and omega hierarchical subscale (ωHS) (McDonald 1999; Reise 2012; Zinbarg et al. 2005). The ECV is an index of the degree of unidimensionality and assesses the relative strength of the general factor or the proportion of all common variance explained by the general factor (Rodriguez et al. 2016a, 2016b). Higher values of ECV indicate a strong general factor and suggest unidimensionality. According to Rodriguez et al. (2016a), “values closer to 1.0 indicate a strong general factor and that the common variance is essentially unidimensional” (p. 231). The ECV for specific factors is only relative to items loading on that factor. The PUC, which is another strength index (Rodriguez et al. 2016a, 2016b), was also computed. Higher values of PUC suggest that the parameter estimates in a unidimensional model are less likely to be biased. According to Rodriguez et al. (2016a), “when ECV is > .70 and PUC > .70 relative bias will be slight and the common variance can be regarded as essentially unidimensional” (p. 232).

The omega index (ω) is a factor-analytic model-based index of internal reliability (Reise et al. 2013a; Rodriguez et al. 2016b). For the general factor, all items were considered (ω); for specific factors, only items loading on that factor were considered (ωS). According to Rodriguez et al. (2016a) “high omega values indicate a highly reliable multidimensional composite” (p. 224). Omega hierarchical (ωH) and omega hierarchical subscale (ωHS) compare the variance of only one construct (general factor or domain-specific factor) to the total score variance. Higher values of omegaH indicate that the general factor is the dominant source of systematic variance. Rodriguez et al. (2016a) suggested that omegaH >.80 indicates that total scores can be considered essentially unidimensional. We also computed the relative omega, which is the OmegaH divided by omega. For the general factor, the relative omega represents the percentage of the reliable variance in the multidimensional composite due to the general factor. For specific factors, this index represents the proportion of reliable variance in the subscale composite that is independent of the general factor.

Finally, correlations between the DERS-SF subscale scores and between the total score and the subscale scores were computed in addition to correlations between each item and the scale’s total score. The validity of the DERS-SF scores based on their relation to variables expected to be associated with emotion regulation was explored. Correlations around .10 were considered small; correlations near .30 were considered medium; and correlations of .50 or higher were considered large (J. Cohen 1988). The extent to which the DERS-SF subscale scores contribute incrementally in the prediction of these variables beyond the total score was explored through hierarchical regression analyses. The total score was introduced in the first step and the DERS-SF subscale in the second step.

Results

Preliminary Analyses

Missing values in each item varied between 0 and 0.1% and were missing completely at random, Little’s MCAR test: χ²(136) = 143.46, p = .314. The expectation-maximization imputation procedure was used to estimate missing values to avoid losing cases in the analyses. Ten outlier cases were detected and, therefore, eliminated from the sample, which resulted in the final sample of 1314 participants. The analysis of skewness and kurtosis values suggested that none of the items were significantly skewed or highly kurtotic. Skewness values ranged from 0.36 (item 2) to 1.88 (item 32) and kurtosis values ranged from −0.03 (item 26) to 3.46 (item 32).

Confirmatory Factor Analyses in the Complete DERS-SF

One-factor, correlated and hierarchical models

First, we tested a one-factor model with all DERS-SF items (Model 1). As presented in Table 1, this model presented a poor fit to the data. Next, we tested a correlated six-factor model consistent with the scale’s original model (Model 2; Gratz and Roemer 2004), which exhibited a good fit to the data. In this model, correlations between factors were high, with the exception of the correlations between Awareness and all the other factors (Strategies: r = .18; Nonacceptance: r = .06; Impulses: r = .14; Goals: r = .07; and Clarity: r = .39). The second-order model that included all DERS-SF items presented a poor fit to the data (Model 3).

Table 1 Goodness of Fit Statistics in Study 1 (Adult Sample) and Study 2 (Adolescent Sample)

Bifactor model

The bifactor model with all DERS-SF subscales presented a good fit to the data (Model 4). In this model, all items loaded significantly on each specific factor and on the general factor, with the exception of item 10 from the Awareness subscale (which presented a nonsignificant loading on the general factor) and items from the Strategies subscale (which did not load significantly on the specific factor). Items belonging to the Awareness factor presented weaker factor loadings on the general factor (from .02 to .13) and stronger factor loadings on the specific factor (.53 to .82). In the Clarity subscale, two of the three items (4 and 5) loaded more strongly on the specific factor (.52 and .68, respectively) than on the general factor, although loadings on the general factor were also significant and generally meaningful (.25, .42, and .50 for items 4, 5, and 9, respectively). For the remaining subscales (Strategies, Nonacceptance, Impulse, and Goals), all items loaded more strongly on the general factor than on each specific factor. Loadings for the general factor of the items from the Strategies, Nonacceptance, Impulse and Goals subscales ranged from .43 (item 12) to .78 (item 16). With regard to the loadings for each specific subscale, all factor loadings for the Nonacceptance, Impulse, and Goals subscales were significant (p < .001), ranging from .20 (item 13) to .55 (item 18). Loadings for the Strategies subscale were weaker (from .06 to .19) and nonsignificant. This pattern of results suggests that most of the items’ variance is shared with the general factor, with the exception of the items belonging to the Awareness subscale and, less clearly, the items from the Clarity subscale. The ECV for the general factor was .599 and the PUC was .882. As presented in Table 2, Omega values were of .70 or above for all subscales. The OmegaH index was .821 for the total score, .715 for the Awareness factor, .445 for the Clarity factor and ranged between .013 (Strategies) and .274 (Goals) for the remaining specific factors. The relative omega indicates that 98.2% of the reliable variance of the Awareness subscale is independent of the general factor. With regard to the Clarity subscale, although 64% of the reliable variance of this subscale is independent of the general factor, loadings on the general factor were significant and generally meaningful. For the remaining subscales, only 1.6% (Strategies), 29.2% (Nonacceptance), 27% (Impulse), and 31.9% (Goals) of the variance was independent of the general factor.

Table 2 Omega and OmegaH Indices in Study 1 (Adult Sample) and Study 2 (Adolescent Sample) for the Bifactor Models

Confirmatory Factor Analyses in Modified Versions of DERS-SF

One-factor, correlated and hierarchical models excluding the Awareness subscale

As presented in Table 1, the one-factor (Model 5) and second-order models (Model 7) presented a poor fit to the data, contrary to the correlated five-factor model (Model 6), which presented a good fit.

Bifactor model excluding the Awareness subscale

The fit of the bifactor model excluding the Awareness subscale (Model 8) was very good. This model was the model that presented the lowest AIC values. As presented in Fig. 2, all items loaded significantly on the general factor and on the specific factor, with the exception of items 16, 28 and 35, which did not load significantly on the specific Strategies factor. Loadings for the general factor were stronger than loadings for the specific factors of Strategies, Nonacceptance, Impulse, and Goals. The only exceptions were items 4 and 5 from the Clarity factor, which loaded more strongly on the specific factor than on the general factor. Almost all items presented factor loadings above .32 on the general factor (with the exception of item 4) and on each specific factor (with the exception of items 16, 28 and 35 from the Strategies subscale and item 13 from the Goals subscale). The PUC was .857 and the ECV of the general factor was .699. The ECV values for the subscales, as well as the omega, omegaH, and relative omega indices are presented in Table 2. The omega index was .933 for the total score and ranged from .696 (Clarity) to .858 (Goals) for the subscales. These results indicate that the majority of total variance in the scores can be attributed to both the total score and the subscales. As in the bifactor model that included all subscales (Model 4), the omegaH index was high for the total score (.859), indicating that a large proportion of the total score variance could be accounted for by the general factor (92.1%). The OmegaH indices of the subscales were much lower, ranging from .015 (Strategies) to .449 (Clarity). Specifically, 1.9% (Strategies), 29.2% (Nonacceptance), 26.8% (Impulse), 31.5% (Goals), and 64.6% (Clarity) of the variance of these subscales was independent of the general factor. These results suggest a strong general factor and support the computation of a DERS-SF total score. The amount of reliable systematic variance of the subscale scores of Strategies, Nonacceptance, Impulse and Goals after partitioning out variability attributed to the general factor was low, which does not support the use of these factors as independent subscales. Enough variance was explained by the Clarity subscale to support its use as an independent subscale.

Correlated model excluding the Strategies subscale

As presented in Table 1, the fit of Model 9 was good.

Bifactor models with two correlated general factors

The fit of the bifactor model that included two correlated general factors with Strategies, Impulse, and Goals loading on the “difficulties in the regulation of emotional response” factor and Awareness, Clarity, and Nonacceptance loading on the” difficulties in emotion processing” factor (Model 10) was good. The fit of Model 11 (equal to Model 10 but only Awareness and Clarity loaded on the general factor assessing difficulties in emotion processing) was also good and very similar to the fit of Model 10.

Osborne et al.’s (2017) bifactor model

Consistent with Osborne et al.’s (2017) model, we tested a modified bifactor model with the Awareness subscale not loading on the general factor but correlated with the Clarity subscale (Model 12). As presented in Table 1, the fit of this modified model was also good.

Item Descriptives and Correlations between Subscales

The means and standard deviations of each item and its correlations with the scale total score (including and excluding the Awareness items) are presented in Table 3. Almost all items presented strong correlations with the scale total score, with the exception of the items from the Awareness subscale, which presented weaker and some nonsignificant correlations with the DERS-SF total score. In addition, as presented in Table 4, all subscales presented moderate to strong correlations between each other and with the scale’s total score, with the exception of the Awareness subscale, which presented weak correlations with the Strategies and Impulse subscales and nonsignificant correlations with the Nonacceptance and Goals subscales.

Table 3 Item Descriptives and Item-Total Correlations in Study 1 (Adult Sample) and Study 2 (AdolescentSample)

Table 4 Correlations between the DERS Subscales and Total Score in Study 1 (adult sample) and Study 2 (adolescent sample)

Validity Evidence of the DERS-SF scores in Relation to Other Variables

Because no evidence was found to support the use of DERS-SF subscales, with the exception of the Clarity and Awareness subscales, correlations with other measures were only computed for Clarity, Awareness, and the DERS-SF total score. As presented in Table 5, significant and moderate to strong correlations were found between the Clarity subscale and the total score and self-compassion, mindfulness, perceived stress, anxiety and depression. Significant and moderate correlations were found between Clarity and the total score and attachment anxiety, and weaker correlations were found between Clarity and the total score and attachment avoidance and emotion suppression. The correlations between the Awareness subscale and the other measures were mostly moderate and in the same direction as the correlations found for Clarity and total score.

Table 5 Correlations Between Awareness, Clarity and Total Score (15 items) and Related Measures in Study 1 (Adult Sample)

Hierarchical regression analyses were performed to test whether the Clarity and Awareness scores contributed incrementally to the prediction of these variables over and above the total score. Clarity significantly contributed to the explanation of attachment anxiety [ΔR² = .007, F(1, 1311) = 10.60, p < .001; total R² = .159]; attachment avoidance [ΔR² = .012, F(1, 1311) = 10.60, p < .001; total R² = .031]; depression symptoms [ΔR² = .005, F(1, 1311) = 7.18, p = .007; total R² = .258]; and suppression [ΔR² = .013, F(1, 278) = 3.99, p = .047; total R² = .075]. In contrast, Clarity did not contributed incrementally in the prediction of anxiety symptoms [ΔR² = .001, F(1, 1311) = 1.85, p = .175; total R² = .274]; mindfulness [ΔR² = .000, F(1, 1311) = 0.34, p = .560; total R² = .233]; stress [ΔR² = .003, F(1, 278) = 1.10, p = .295; total R² = .378]; and self-compassion [ΔR² = .000, F(1, 1311) = 0.31, p = .581; total R² = .338].

Awareness significantly contributed to the explanation of attachment anxiety [ΔR² = .020, F(1, 1311) = 29.77, p < .001; total R² = .171]; attachment avoidance [ΔR² = .022, F(1, 1311) = 28.55, p < .001; total R² = .041]; anxiety symptoms [ΔR² = .037, F(1, 1311) = 53.09, p < .001; total R² = .310]; depression symptoms [ΔR² = .077, F(1, 1311) = 114.35, p < .001; total R² = .330]; suppression [ΔR² = .066, F(1, 278) = 320.94, p < .001; total R² = .128]; mindfulness [ΔR² = .032, F(1, 1311) = 43.04, p < .001; total R² = .265]; and self-compassion [ΔR² = .082, F(1, 1311) = 177.81, p < .001; total R² = .420]. In contrast, Awareness did not contributed incrementally in the prediction of stress [ΔR² = .009, F(1, 278) = 3.81 p = .052; total R² = .390].

Study 2

The aim of Study 2 was to examine the factor structure of the DERS-SF in a sample of adolescents from the general community and to explore the associations between the DERS-SF scores and the scores from measures of emotion regulation, depressive symptoms, quality of life, dispositional mindfulness and self-compassion.

Method

Participants and Procedure

The participants in Study 2 were 612 adolescents (53.9% girls) aged between 12 and 19 years (M = 14.54, SD = 1.89) recruited at two public schools in central Portugal. A research assistant visited several classes from each school and gave each student an informed consent form to be completed by the student’s parents or legal representatives. A week later, the research assistant collected the written informed consent, and those adolescents who were allowed to participate in the study completed the questionnaires in the classroom. All parents or legal representatives of adolescents provided written informed consent. The study was approved by the Portuguese Data Protection Authority, the Ethics Committee of the Faculty of Psychology and Education Sciences of the University of Coimbra, and the Board of Directors of Schools.

Measures

All adolescents completed an assessment battery composed of the DERS-SF and the SCS-SF, described in Study 1, and additional measures of emotion suppression, dispositional mindfulness, depressive symptoms, and quality of life, to assess the validity of the scale. All measures are validated to the Portuguese population and present adequate psychometric properties.

Emotion suppression

The Expressive Suppression subscale of the Emotion Regulation Questionnaire for Children and Adolescents (ERQ-CA; Gullone and Taffe 2012; Teixeira et al. 2015) was used to assess adolescents’ suppression strategy of emotion regulation. This subscale has four items answered on a five-point Likert scale that ranges from 1 (strongly disagree) to 5 (strongly agree). The subscale score is the sum of the items and ranges from 4 to 20. Higher levels indicate higher levels of suppression. In the present sample, Cronbach’s alpha was .65.

Depressive symptoms

The Center for Epidemiological Studies Depression Scale for Children (Carvalho et al. 2015; Weissman et al. 1980) was used to assess the frequency of adolescents’ depressive symptoms during the preceding week. This questionnaire has 20 items assessing different symptoms of depression (e.g., “I felt like crying”, “I didn’t sleep as well as I usually sleep”), answered on a four-point Likert response scale ranging from 0 (not at all) to 3 (a lot). Higher scores indicate higher levels of depressive symptoms. In the present sample, Cronbach’s alpha was .84.

Dispositional mindfulness

Adolescents’ mindfulness skills, particularly their present-moment awareness and their nonjudgmental and nonavoidant responses to their thoughts and feelings, were assessed through the Child and Adolescent Mindfulness Measure (CAMM; Cunha et al. 2013; Greco et al. 2011). This measure has ten items (e.g., “At school, I walk from class to class without noticing what I’m doing”; “I push away thoughts that I don’t like”) rated on a four-point Likert scale ranging from 0 (never true) to 4 (always true). The total score is the sum of the 10 items and ranges from 0 to 40, with higher scores indicating higher levels of mindfulness. In the present sample, Cronbach’s alpha was .80.

Quality of life

The self-report version of the KIDSCREEN-10 index (Matos et al. 2012; Ravens-Sieberer et al. 2010) was used to assess children’s perception of their quality of life. This questionnaire has ten items (e.g., “Have you felt full of energy?”; “Have you had fun with your friends?”) answered on a five-point Likert scale ranging from 1 (never; not at all) to 5 (always; extremely), with high scores indicating a better quality of life. In the present sample, Cronbach’s alpha was .80.

Statistical analyses

The analytical procedure was similar to Study 1.

Results

Preliminary Analyses

Missing values in each item varied between 0 and 0.2% and were missing completely at random, Little’s MCAR test: χ²(17) = 14.24, p = .650. The expectation-maximization imputation procedure was used to estimate missing values to avoid losing cases in the analyses. Three outlier cases were detected and, therefore, eliminated from the sample, which resulted in the final sample of 612 adolescents. The analysis of skewness and kurtosis values suggested that none of the items were significantly skewed or highly kurtotic. Skewness values ranged from −0.22 (item 10) to 0.87 (item 4) and kurtosis values ranged from −1.17 (item 26) to 0.92 (item 13).

Confirmatory Factor Analyses in the Complete DERS-SF

One-factor, correlated and hierarchical models

As presented in Table 1, the one-factor model (Model 1) and the second-order model (Model 3) presented a poor fit to the data. In contrast, the correlated six-factor model exhibited an adequate fit to the data (Model 2). In the correlated model, with the exception of the correlations between Awareness and all the other factors, which were low (Strategies: r = −.15; Nonacceptance: r = −.21; Impulses: r = −.11; Goals: r = −.20; and Clarity: r = −.14), the remaining correlations between factors were high and varied between .54 (Impulse and Clarity) and .91 (Strategies and Nonacceptance).

Bifactor model

The bifactor model with all DERS subscales presented a good fit to the data (Model 4). With regard to the item loadings on the general and specific factors, a pattern of results similar to those found in the adult sample was found in the adolescent sample. Specifically, all items loaded significantly on each specific factor and on the general factor, with the exception of item 2 from the Awareness subscale (which presented a nonsignificant loading on the general factor) and the three items from the Strategies subscale (which did not load significantly on the specific factor). All items belonging to the Awareness factor loaded more strongly on the specific factor (from .48 to .77) than on the general factor (from −.05 to −.28), and two items of the Clarity subscale (4 and 5) loaded more strongly on the specific factor (.58 and .63, respectively) than on the general factor. In the other subscales (Strategies, Nonacceptance, Impulse, and Goals), all items loaded more strongly on the general factor than on each specific factor. Loadings for the general factor ranged from .41 (item 4) to .77 (item 16). For the subscales of Nonacceptance, Impulse, and Goals all item loadings for each specific subscale were significant (p < .001) and ranged from .27 (item 29) to .58 (item 26). Loadings for the Strategies subscale were weaker (from .11 to .45) and nonsignificant. As in the adult sample, these results suggest that with the exception of the items of the Awareness subscale and, less clearly, the items from the Clarity subscale, the majority of the items’ variance is shared with the general factor. The ECV for the general factor was .625 and the PUC was .882. As presented in Table 2, Omega values were above .73 for all subscales. In addition, the OmegaH index was .832 for the total score, .690 for the Awareness factor, and .404 for the Clarity factor. In the remaining specific factors, OmegaH ranged between .020 (Strategies) and .283 (Goals), which concurs with the conclusion that the Awareness and Clarity subscales do not contribute to the unidimensionality of the scale. In fact, 94.8% and 51.8% of the reliable variance of the Awareness and Clarity subscales, respectively, are independent of the general factor. For the remaining subscales, only 2.5% (Strategies), 22.9% (Nonacceptance), 25% (Impulse), and 32.9% (Goals) of the variance was independent of the general factor.

Confirmatory Factor Analyses in Modified Versions of DERS-SF

One-factor, correlated and hierarchical models excluding the Awareness subscale

As presented in Table 1, the correlated five-factor model presented a good fit to the data (Model 6). In contrast, the fit was poor for the one-factor (Model 5) and second-order models (Model 7).

Bifactor model excluding the Awareness subscale

A bifactor model excluding the Awareness subscale presented a good fit to the data (Model 8). As shown in Fig. 3, with the exception of items 4 and 5 from the Clarity factor, all items loaded more strongly on the general factor than on the specific factors. All loadings were significant, with the exception of the loadings of items 16, 28 and 35 on the specific Strategies factor. Almost all items presented factor loadings above .32 on the general factor and on each specific factor (with the exception of items 16, 28 and 35 from the Strategies subscale). The PUC was .857 and the ECV of the general factor was .709. The ECV values for the subscales are presented in Table 2. As presented in Table 2, the omega index for the total score was .952, and the omegaS varied between .784 for the Clarity subscale and .891 for the Impulse subscale. These results indicate that most of the total variance in the scores can be attributed to both the total score and the subscales. The OmegaH was .886 for the total score, indicating that a large proportion of the total score variance can be accounted for by the general factor (93%). The OmegaH subscale ranged from .012 (Strategies) to .413 (Clarity). For the majority of the subscales, a small percentage of the variance is attributable to each subscale, whereas for the Clarity subscale, a substantial portion of the variance is attributable to the subscale and independent of the general factor (52.7%). For the remaining subscales, only 1.4% (Strategies), 23.9% (Nonacceptance), 24.5% (Impulse), and 33.5% (Goals) of the variance of these subscales was independent of the general factor. Consistent with Study 1, these results suggest a strong general factor and support the computation of a DERS-SF total score. While enough variance was explained by the Clarity subscale to support its use as an independent subscale, the amount of reliable systematic variance of the remaining subscale scores after partitioning out variability attributed to the general factor was low.

Fig. 2

Standardized factor loadings for the bifactor confirmatory model with five specific factors in the adult sample Note. All items presented significant factor loadings (p < .001), with the exception of items 16, 28 and 35, which did not load significantly on the Strategies factor.

Fig. 3

Standardized factor loadings for the bifactor confirmatory model with five specific factors in the adolescent sample Note. All items presented significant factor loadings (p < .001), with the exception of items 16, 28 and 35, which did not load significantly on the Strategies factor.

Correlated model excluding the Strategies subscale

As presented in Table 1, the fit of Model 9 was good.

Bifactor models with two correlated general factors

The bifactor model that included two correlated general factors (Strategies, Impulse, and Goals loading on the “difficulties in the regulation of emotional response” factor; and Awareness, Clarity, and Nonacceptance loading on the” difficulties in emotion processing” factor (Model 10) have exhibited a good fit to the data. The fit of Model 11 (equal to Model 10 but only Awareness and Clarity loaded on the general factor assessing difficulties in emotion processing) was also good and very similar to the fit of Model 10.

Osborne et al.’s (2017) bifactor model

The modified bifactor model proposed by Osborne et al. (2017) presented an acceptable fit to the data (Model 12).

Item descriptives and Correlations between Subscales

As shown in Table 3, strong correlations were found between almost all items and the scale’s total score. The exceptions were the items from the Awareness subscale, which presented weak correlations with the DERS total score. In addition, as shown in Table 4, all subscales correlated to each other and with the scale’s total score significantly and strongly. In contrast, the correlations between the Awareness subscale and the other subscales were small to moderate and in the opposite direction.

Validity Evidence of the DERS-SF scores in Relation to Other Variables

As presented in Table 6, both the Clarity subscale and the DERS-SF total score presented negative and strong correlations with self-compassion, mindfulness and depressive symptoms and moderate correlations with expressive suppression and quality of life. The Awareness subscale presented small to moderate correlations with these measures and, in contrast to the Clarity subscale and the total score, showed a positive association with mindfulness.

Table 6 Correlations Between Awareness, Clarity and Total Score (15 items) and Related Measures in Study 2 (Adolescent Sample)

Hierarchical regression analyses were performed to test whether the Clarity and Awareness scores contributed incrementally to the prediction of these variables over and above the total score. Clarity significantly contributed to the explanation of mindfulness [ΔR² = .013, F(1, 609) = 12.06, p < .001; total R² = .341] and suppression [ΔR² = .015, F(1, 609) = 10.40, p < .001; total R² = .097]. In contrast, Clarity did not contributed incrementally in the prediction of self-compassion [ΔR² = .001, F(1, 609) = 0.67, p = .413; total R² = .301], quality of life [ΔR² = .000, F(1, 609) = 0.09 p = .763; total R² = .155], and depression [ΔR² = .014, F(1, 609) = 0.55, p = .317; total R² = .564].

Awareness significantly contributed to the explanation of self-compassion [ΔR² = .112, F(1, 609) = 115.41, p < .001; total R² = .413], quality of life [ΔR² = .049, F(1, 609) = 37.17, p < .001; total R² = .204], and suppression [ΔR² = .020, F(1, 609) = 13.68, p < .001; total R² = .102]. In contrast, Awareness did not contributed incrementally in the prediction of mindfulness [ΔR² = .000, F(1, 609) = 0.22, p = .642; total R² = .328], depression [ΔR² = .000, F(1, 609) = 0.03, p = .863; total R² = .551].

Discussion

The main goal of the present study was to explore the factor structure and the psychometric properties of the DERS-SF in a large sample of Portuguese adolescents and adults. Since this scale can be applied to adolescents and adults, two independent studies were conducted: one study with adults from the general community collected in several settings and another study with adolescents aged between 12 to 19 years collected in schools. In addition, we aimed to contribute to the clarification of questions related to the factor structure of the scale raised in previous studies. In particular, we aimed to investigate, through the examination of bifactor confirmatory models, whether the DERS-SF can be used to measure a general factor of emotion dysregulation and/or specific dimensions of emotion regulation. We also wanted to clarify whether the Awareness and Strategies subscales measure the same construct of emotion dysregulation as the total score. The possibility that DERS-SF subscales could be organized in two broader domains (difficulties in the regulation of the emotional response and difficulties in the emotion processing) was also explored.

Several models were tested through CFA. In particular, the bifactor models allowed for the examination of the extent to which items reflect a common factor (i.e., emotion dysregulation) and the extent to which they reflect specific factors (i.e., different emotion regulation strategies) (Reise et al. 2010). All bifactor and correlated models showed a good fit to the data (including or excluding the Awareness or the Strategies subscales). However, considering the fit indices, and in particular the AIC, the best-fitting model was the bifactor model that excluded the Awareness subscale. The results were remarkably similar in the adult sample (Study 1) and the adolescent sample (Study 2), which supports the conclusions drawn from the models as well as the psychometric soundness of this short form of the scale and its adequacy for assessing emotion regulation difficulties in different age groups. Overall, the results of the present study support the psychometric robustness of the Portuguese version of the DERS-SF as a measure of emotion dysregulation and the use of a total score excluding the items from the Awareness subscale. The analysis of the different models and, particularly, of the bifactor psychometric indices, allows us to draw several conclusions about the questions that this study intended to clarify.

First, the results of the present study support the computation of a total score of emotion dysregulation, excluding the Awareness items. In both studies, in the bifactor models that included and excluded the Awareness subscale (Models 4 and 8), we found that all items loaded more strongly on the general factor than on each specific factor, with the exception of the three items from the Awareness subscale (in Model 4) and of two items from the Clarity subscale (in both models). In addition, the OmegaH for the total score was above .80 in both models and in both studies, which means that a large proportion of the total score variance can be accounted for by the general factor. In turn, the OmegaH subscales were much lower (for instance, in Model 8, this index ranged from .015 for the Strategies subscale to .449 for the Clarity subscale in the adult sample and from .012 for the Strategies subscale to .413 for the Clarity subscale in the adolescent sample) and are below the threshold proposed by Reise et al. (2013a) of .50. These findings suggest that the total score can be considered fundamentally unidimensional since much of the reliable variance of the subscale scores is explained by the general factor and not by what is unique to the subscales.

Further support for computing a total score of emotion dysregulation was obtained through the index of degree of unidimensionality (ECV), an index that assesses the proportion of all common variance explained by the general factor (that is, the relative strength of the general factor). According to Rodriguez et al. (2016a), ECV and PUC values above .70 for the total score indicate that the scale is essentially unidimensional and that the relative bias is slight. In the bifactor model that included the Awareness subscale, we found an ECV value of .599 in the adult sample and of .625 in the adolescent sample. These values suggest that although the general factor is strong, the degree of multidimensionality is substantial enough to justify the use of the subscales. However, when excluding the Awareness subscale, the ECV was .699 for the total score in the adult sample and .709 in the adolescent sample. The ECV values of the subscales were much lower, ranging from .038 for the Strategies subscale to .644 for the Clarity subscale in the adult sample and from .119 for the Strategies subscale to .532 for the Clarity subscale in the adolescent sample. These results suggest that when excluding the Awareness subscale, which does not appear to be contributing to the unidimensionality of the scale, the DERS-SF (with 15 items) is essentially unidimensional and that a total score of emotion dysregulation can be reliably computed.

A second important finding of this study is the clarification of the role of the Awareness subscale. The results of this study suggest that the Awareness subscale does not appear to assess the same emotion regulation construct as assessed by the total score. In particular, in the bifactor model that included all subscales, the items of the Awareness subscale presented weak factor loadings on the general factor and strong factor loadings on the specific factor. In addition, the OmegaH of the Awareness subscale was .715 in the adult sample and .690 in the adolescent sample, which indicates that a large proportion of the reliable variance in this subscale composite is independent of the general factor (98.2% and 94.8% in Study 1 and in Study 2, respectively). When comparing the fit of correlated and bifactor models that included and excluded the Awareness subscale, although all models had a good fit, the models without Awareness have a better fit (for instance, lower AIC values). We also found that in contrast with the other subscales, Awareness presented weak correlations with the DERS-SF total score as well as with the other subscales. Similarly, the three items of the Awareness subscale had much lower correlations with the total score than the items of the other subscales, which presented very strong correlations with the total score. These findings may suggest that the Awareness subscale is not valid and evaluates a construct that is different from the other subscales. Bardeen et al. (2012) argued that emotional awareness and clarity might precede and be necessary but not sufficient for the subsequent use of emotional regulation strategies (i.e., it is possible to be aware of negative and strong emotions and not implement any strategy to regulate them). This may mean that while the remaining DERS-SF subscales evaluate different emotion regulation strategies, Awareness may evaluate a process that, while necessary for emotional regulation, is not the same as emotional regulation strategies.

On the other hand, these results may be related to the fact that the Awareness subscale is the only subscale in which all the items are reverse-coded. Although the inclusion of reversed items has several advantages (e.g., disruption of nonsubstantive responding, improvement of scale validity), it can also be associated with a number of measurement problems (e.g., lower reliability, poorer model fit, smaller factor loadings) (Weijters and Baumgartner 2012). The hypothesis that a method effect may be a possible explanation for the psychometric limitations of this subscale was corroborated by previous studies showing that a modified version of the scale, in which all items are written straightforwardly, presented a good fit for the data (Bardeen et al. 2016; Benfer et al. 2018). Taken together, the results of the present study suggest that, in the current version of the scale (with the Awareness items reverse-coded), the Awareness subscale should be excluded when computing an emotion dysregulation total score. However, this subscale can be used separately, as a measure of the extent to which individuals are aware of their emotions. As we cannot rule out the possibility that these results are explained by the presence of reverse items and as the theoretical underpinnings of this scale underline that being aware of the emotions is an important aspect of emotion regulation (Gratz and Roemer 2004; Thompson and Calkins 2009), future studies should further explore the possibility that a method effect is responsible for the consistently poor results found with this subscale.

Interestingly, our results also support the individual use of the Clarity subscale, which is consistent with the initial conceptualization of DERS that combined the awareness and understanding of emotions in one unique dimension (Gratz and Roemer 2004). In fact, the OmegaH of the Clarity subscale was above .40 in both bifactor models of both studies, which means that a large amount of the reliable variance (more than 52%) in this subscale composite is independent of the general factor and can be attributable to the subscale. We also found that Clarity scores contribute incrementally to the prediction of several variables, over and above the total score (attachment anxiety, attachment avoidance, depression symptoms, and suppression in the adult sample; mindfulness and suppression in the adolescent sample), which also supports the use of this subscale as an independent measure. However, it is important to note that this subscale also contributed significantly for the total score (all loadings on the general factor were significant and above .32; Tabachnick and Fidell 2007) and therefore, contrary to the Awareness subscale, Clarity items should not be eliminated when computing the total score.

For the remaining subscales, only a small proportion of the variance can be attributable to the subscale factors, which calls into question the interpretability and utility of the Strategies, Nonacceptance, Impulse, and Goals subscales. The Strategies subscale was the subscale with the smallest OmegaH and ECV values. In addition, none of the items of this subscale loaded significantly on the specific factor (although all items loaded significantly on the general factor). While the nonsignificant loadings on the specific factor and the negative loading of item 16 (although it was positive in first-order CFA models) can be considered an anomalous result due to the application of a fully symmetrical bifactor model to structurally different facets (Heinrich et al. 2018), these findings are consistent with the results from previous studies, in which the items from the Strategies subscale also did not significantly load on the specific factor and/or presented negative loadings (Hallion et al. 2018; Osborne et al. 2017). Overall, our results indicate that the amount of reliable systematic variance of the Strategies subscale after partitioning out variability attributed to the general factor was negligible, which suggests that this subscale cannot be used independently since its content seems to be redundant with the general factor. Although Medrano and Trógolo (2016) have suggested that the Strategies subscale should be excluded from the DERS because it assesses emotional self-efficacy beliefs rather than the emotion regulation strategies themselves, our results suggest that this subscale contributes to the emotion dysregulation total score. In addition, the fit of the correlated model that excluded the Strategies subscale was practically the same as the fit of the correlated model that included all subscales. Thus, given the adequate internal consistency of this subscale (omega values around .80) as well as its contribution to the total factor of emotion dysregulation, we consider that it should not be excluded from the scale and should be considered in the computation of the total score.

The possibility that the DERS-SF subscales could be organized in two broader domains (difficulties in the regulation of the emotional response and difficulties in the emotional processing) was also explored. In fact, based on the results obtained in the previous bifactor models, one might hypothesize that Awareness and Clarity dimensions share fewer common aspects with the others DERS-SF subscales that seem to reflect a common underlying process represented by the general factor. Therefore, it can be hypothesized that Awareness and Clarity measure a different emotion regulation process (i.e., difficulties in emotional processing), whereas the dimensions that loaded strongly in the general factor may reflect another emotion regulation process (i.e., difficulties in the regulation of the emotional response). Following the procedures of Medrano and Trógolo (2016), we tested two bifactor models with two general factors corresponding to these broader domains (i.e., difficulties in the emotional processing and difficulties in the regulation of the emotional response). Both models presented a similarly good fit to the data. In one of the models, only Awareness and Clarity loaded on the factor assessing difficulties in the emotional processing and the other subscales loaded on the other general factor (Model 11). This model would allow us to test the hypothesis that Awareness and Clarity assess the awareness and understanding of emotions, as hypothesized by Gratz and Roemer (2004), and the remaining subscales assess different strategies that can be used to regulate emotions and that seem to be assessed by a common general factor of emotion dysregulation. In the other model, we followed the organization proposed by Medrano and Trógolo (2016): Awareness, Nonacceptance and Clarity loaded on the factor assessing difficulties in the emotional processing, whereas Strategies, Impulse and Goals loaded on the factor assessing difficulties in the regulation of the emotional response (Model 10). According to Medrano and Trógolo (2016), this organization is consistent with research on affective neuroscience, according to which difficulties in the emotional awareness and in understating and experiencing emotions can be associated with a lower activation of subcortical regions (van der Velde et al. 2013), whereas difficulties in the control of impulses and in task performance might be associated with higher error-related activity in specific regions of the dorsolateral prefrontal cortex (Moeller et al. 2014). This hypothesis is also consistent with the results of Osborne et al. (2017), who found solid evidence to consider the Clarity (omegaHS = .70) and Nonacceptance (omegaHS = .54) subscales as measuring a construct distinct from what is measured by the total score.

Taken together, the results from these two bifactor models may suggest that Awareness and Clarity (and possibly Nonacceptance) measure a different underlying process in the regulation of emotions (difficulties in the emotional processing) that is not covered by the other DERS subscales, which may be measuring difficulties in the regulation of the emotional response. These results are also in accordance with the initial conceptualization of Gratz and Roemer (2004), according to which awareness and understanding of emotions (i.e., clarity) were a single dimension, as well as with the results found in the studies that combined Awareness and Clarity into an Identification subscale (Bardeen et al. 2016; Benfer et al. 2018). In fact, a careful analysis of the items of the different DERS-SF subscales shows that the items in the Awareness and Clarity subscales evaluate aspects related to the ability to notice and understand the emotions (e.g. “When I’m upset, I acknowledge my emotions”, “I have no idea how I am feeling”), whereas the items of the other subscales focus on strategies that an individual can employ to regulate their emotions (e.g., “When I’m upset, I become irritated at myself for feeling that way”, “When I’m upset, I have difficulty concentrating”). Thus, we can hypothesize that while Awareness and Clarity assess an early stage of emotional regulation, the remaining subscales assess a subsequent stage, when the individual is already making efforts to regulate the emotional response. This is consistent with the modal model of emotion (Gross 2007), according to which emotions arise when an individual attends to a situation and evaluates (appraises) that situation as relevant to a particular type of goal. According to this perspective, the emotion-generative process involves a chronological sequence of events: firstly there has to be a real or imaginary situation that elicits an emotion; second, the individual pays attention to the situation and evaluates it; and finally, the emotional response arises (Mauss et al. 2007; Gross and Thompson 2007). The process model of emotion regulation builds on the modal model, and considers each stage of the emotion-generative process as a potential target for emotion regulation (Gross and Thompson 2007; Gross 1998). Therefore, the awareness and understanding of emotions seem to be a fundamental step for the subsequent implementation of emotion regulation strategies.

The analyses of the correlations between the DERS-SF total score and other measures expected to be associated with emotion (dys)regulation support the validity of the scale scores and provide additional evidence of the utility and interpretability of the total score. These correlations were in the expected directions, which supports the usefulness of the DERS-SF as a measure of emotion dysregulation. First, significant associations were found with suppression, which is another strategy of emotion regulation (Gross and John 2003). These correlations were moderate, suggesting that the DERS-SF and the questionnaires used to assess suppression (the ERQ for adults and the ERQ-CA for adolescents) measure different yet related strategies of emotion regulation. The stronger associations with depression, anxiety, perceived stress, and quality of life corroborate the view that difficulties in emotion regulation contribute to higher levels of psychopathology and lower psychosocial adjustment and that an adequate ability of emotion regulation is a key component of mental health (e.g., Finlay-Jones et al. 2015; Cisler et al. 2010). Significant associations with self-compassion and dispositional mindfulness were also expected because these are two psychological resources that help individuals adaptively cope with negative internal and external events (Diedrich et al. 2014). The correlations between emotion regulation and attachment avoidance and anxiety were also expected because it is well known that attachment insecurity is associated with secondary strategies of emotion regulation, such as hyperactivation and deactivation of the attachment system (Mikulincer and Shaver 2019).

Limitations and Conclusions

This study presents some limitations that should be considered. First, the adult sample was exclusively composed of women, which compromises the generalization of results to adult men. Second, since there was only one moment of evaluation, the test-retest reliability of the DERS-SF was not determined. Third, the validity of the results may be compromised because we used only self-report measures, which may be influenced by social desirability factors and do not reliably reflect what participants feel or think. Fourth, only individuals from the general community were included. It would be important to replicate the analyses performed in the current study in clinical samples, in order to ascertain whether the DERS-SF could potentially explain systematic individual differences in emotion regulation skills among clinical populations (e.g., by assessing the discriminatory and/or predictive value of DERS-SF scores for psychopathology and quality of life outcomes).

Despite these limitations, this study represents an important contribution to the investigation of the factor structure and psychometric properties of the short version of the DERS and clarifies some problematic issues previously identified in the Awareness subscale and the adequacy of using this questionnaire as a unidimensional or multidimensional measure. Taken together, our findings suggest that items from the Awareness subscale should not be included in the computation of the total score and that the total score can be interpreted as a measure of a single common construct (emotion dysregulation or difficulties in regulating the emotional response). In addition, while the Strategies, Nonacceptance, Impulse, and Goals subscales seem to not be suitable for use as individual subscales, our results support the use of the Awareness and Clarity subscales as individual subscales. One possible hypothesis is that Awareness and Clarity (as well as Nonacceptance) may measure difficulties in the processing of emotions, whereas the remaining DERS-SF subscales may be measuring difficulties in the regulation of the emotional responses. Overall, our results demonstrate that the Portuguese short form version of the DERS is reliable and valid and can be used to measure emotion regulation problems in adolescents and adults from the general community.