Emotion regulation deficits appear to underlie a broad range of deleterious pathological outcomes, including, but not limited to, generalized anxiety disorder (GAD; Salters-Pedneault et al. 2006), posttraumatic stress disorder (PTSD; Ehring and Quack 2010; Tull et al. 2007), depression (Tull et al. 2009), alcohol dependence (Fox et al. 2008), and borderline personality disorder (Gratz et al. 2006). Given its potential transdiagnostic status, emotion regulation has become the focus of a burgeoning area of research (see Aldao et al. 2010). To better understand relations between emotion regulation difficulties and phenomena of interest, it is necessary to have valid self-report measures that adequately assess the emotion regulation domain.

A self-report measure that provided a comprehensive assessment of the emotion regulation domain had been lacking in the available literature until Gratz and Roemer (2004) developed the Difficulties in Emotion Regulation Scale (DERS). Gratz and Roemer used both a deductive and an inductive approach to scale construction when developing the DERS. More specifically, 41 items were initially developed to assess four core facets of emotion regulation identified by Gratz and Roemer. These items were then reduced to a 36-item pool based on comparing item-level characteristics and an exploratory factor analysis (EFA). EFA results indicated that six dimensions best represented the 36 items of the DERS. These dimensions reflected the nonacceptance of emotional responses (NONACCEPTANCE), difficulty engaging in goal-directed behavior when experiencing negative emotions (GOALS), impulse control difficulties when experiencing negative emotions (IMPULSE), lack of awareness of emotions (AWARENESS), limited access to strategies for regulation (STRATEGIES), and lack of emotional clarity (CLARITY). Gratz and Roemer found that these six DERS subscales demonstrated good internal consistency (Cronbach’s α from .80 to .89) and that they significantly correlated with other measures of emotion regulation (e.g., rs ranging from −.34 to −.69 with negative mood regulation, with higher negative mood regulation scores indicating more positive mood regulation expectancies).

Gratz and Roemer (2004) intended for the DERS to assess dimensions of the same underlying construct (i.e., emotion regulation). As such, the DERS subscales should share significant intercorrelations and converge in a consistent pattern with variables theoretically relevant to the emotion regulation domain. However, whereas five of the DERS subscales shared moderate to strong intercorrelations (rs ranging from .32 to .63), DERS-AWARENESS evidenced much more modest intercorrelations with the other five DERS subscales (rs ranging from .08 to .46). This pattern of subscale intercorrelations has been replicated. For example, Neumann et al. (2010) found that the range of interrelations between DERS-AWARENESS and the other five DERS subscales was more modest in magnitude (rs ranging from −.09 to .10) relative to the interrelations among the remaining five DERS subscales (rs ranging from .34 to .54) in a sample of adolescents. Moreover, using adult samples, both Tull et al. (2007) and Tull et al. (2010) found that DERS-AWARENESS failed to correlate with four of the five DERS subscales (i.e., NONACCEPTANCE, GOALS, IMPULSE, STRATEGIES), but the other five DERS subscales all correlated significantly with one another.

In addition to sharing only modest intercorrelations with the other DERS subscales, DERS-AWARENESS, compared to the other DERS subscales, has shown a relatively divergent pattern of relations with criterion putatively relevant to the emotion regulation domain. For example, Salters-Pedneault et al. (2006) found that DERS-AWARENESS was the only DERS subscale that failed to significantly predict GAD diagnostic status. Similarly, McDermott et al. (2009) found that DERS-AWARENESS was the only DERS subscale that did not significantly predict PTSD status (see also Tull et al. 2007). Moreover, Soenke et al. (2010) proposed that the development of adaptive emotion regulation abilities is less likely when childhood abuse has occurred. Consistent with this proposal, five of the DERS subscales shared a significant correlation with a history of childhood emotional abuse. However, DERS-AWARENESS failed to significantly correlate with this criterion. Drawing from Reinforcement Sensitivity Theory (RST; Gray and McNaughton 2000), Tull et al. (2010) asserted that the behavioral inhibition system (BIS) - the neurological system thought to underlie inhibitory or avoidance behaviors—would be related to difficulty regulating emotion. Consistent with this assertion, the BIS was significantly positively correlated with five DERS subscales. Similar to the other studies described above, DERS-AWARENESS was the only DERS subscale that was not significantly associated with the BIS.

Collectively, findings indicate that DERS-AWARENESS does not cluster with the other DERS subscales and shows a differential pattern of relations with criteria of interest. As such, DERS-AWARENESS might not be assessing the same underlying construct as the other DERS subscales. Whereas the available literature points to such a possibility, there exist no known published data that specifically seek to examine whether DERS-AWARENENESS is meaningfully distinct from the other dimensions of the DERS. For example, available factor analytic work of the DERS has focused on replicating Gratz and Roemer’s (2004) original correlated six-factor structure of the measure (e.g., Neumann et al. 2010). Although these studies are important for validating the DERS, they do not speak to the tenability of alternative, and potentially more parsimonious, operationalizations of this measure (e.g., that remove DERS-AWARENESS). Existing data which suggests that DERS-AWARENESS behaves differently than the other dimensions of the measure provides relatively strong empirical evidence to suggest that such a refined operationalization of the DERS may more accurately represent the latent structure of the measure. As such, the present study sought to address this gap in the literature by further examining the latent structure of the DERS. Examining this empirical question has important implications for the conceptualization and assessment of emotion regulation. For example, if DERS-AWARENESS appears meaningfully distinct from the other dimensions of the DERS, DERS-AWARENESS might be not be best conceptualized as belonging to the same higher-order emotion regulation construct as the other dimensions represented by the DERS. Moreover, researchers often use the DERS total scale score (i.e., summation of all items comprising the six DERS subscales) in an attempt to assess the emotion regulation domain (e.g., Burns et al. 2010; Cisler et al. 2009; Fox et al. 2008; Glenn and Klonsky 2009; Gratz and Chapman 2007; Gratz et al. 2009; Heath et al. 2008; Hoffman and Kashden 2010; Johnson et al. 2008; Lavender and Anderson 2010; McDermott et al. 2009; Roemer et al. 2009; Salters-Pedneault et al. 2006; Snorrason et al. 2010; Soenke et al. 2010; Tull et al. 2007, 2009, 2010; Vujanovic et al. 2008). To the extent that DERS-AWARENESS is not best represented as being part of this domain, including the items of this subscale in the computation of the DERS total scale might be obscuring relationships between emotion regulation and variables of interest.

Clark and Watson (1995) noted that an important requirement in determining whether to use an overall score versus subscale scores is to “establish that all of the items—regardless of how they are placed in the various subscales—define a single general factor” (p. 318). One way to test for such a possibility is to conduct a higher-order confirmatory factor analysis (CFA). A higher-order CFA seeks to investigate whether a general factor accounts for the interrelations among the lower-order latent factors of a scale (Brown 2006). To our knowledge, such an approach has yet to be used to examine the latent structure of the DERS. Given that DERS-AWARENESS shares relatively small correlations with the other DERS subscales, a higher-order factor might not account for the interrelations among the lower-order DERS factors. A higher-order CFA also allows for an examination of the relative importance of each lower-order factor to the general factor, which can help quantify the contribution of DERS-AWARENESS to the putative general emotion regulation factor represented by the DERS. A markedly lower factor loading of DERS-AWARENESS on this general factor would support the notion that DERS-AWARENESS may not belong to the same domain as the other dimensions of the DERS.

The Present Study

The present study had three primary aims. First, we sought to examine the degree to which the dimensions of the DERS represent an overarching higher-order emotion regulation construct. Based on the information presented above, it was predicted that DERS-AWARENESS would show the weakest loading on this higher-order factor. Additional analyses were conducted to further examine the distinctiveness of DERS-AWARENESS from the other dimensions of the DERS. Second, on the basis of the literature reviewed above which suggests that the AWARENESS dimension may fail to converge with the other dimensions underlying the DERS, we sought to investigate the tenability of a DERS model without the AWARENESS subscale. Third, we sought to examine the concurrent validity of a revised DERS total scale (i.e., removing DERS-AWARENESS) by comparing its ability to concurrently predict pathological criteria theoretically and empirically related to emotion regulation (i.e., depression, anxiety, posttraumatic stress symptoms) to the concurrent validity of the full-length DERS total scale (i.e., including DERS-AWARENESS). Evidence of significantly attenuated correlations between the revised DERS total scale with the psychological symptom measures, relative to those obtained using the full-length DERS total scale, would cast doubt on the tenability of removing DERS-AWARENESS from the computation of the total scale score. Additional analyses were run to ensure the full-length DERS total scale does not share unique variance with pathological criteria beyond the variance accounted for by the revised DERS total scale.

Method

Participants and Procedure

One thousand and forty-five female undergraduate students from a mid-sized Midwestern University were recruited to participate in an Institutional Review Board approved study designed to examine trauma and sexual revictimization, for which they completed the DERS and measures of anxiety, depression, posttraumatic stress symptoms (PTSS), and trauma history. Data collection began during the fall of 2006 (via rolling admission), and was completed in February of 2008. Eight participants omitted responses to all of the DERS items and were thus excluded from the present study. Only those participants who endorsed experiencing a traumatic event (i.e., Criterion A1 and A2 of PTSD; DSM-IV-TR: APA 2000) were asked to complete a measure of PTSS (n = 851); therefore, any reference henceforth to the Distressing Events Questionnaire (DEQ: Kubany et al. 2000b) or PTSS will refer only to this subsample, whereas information regarding depression and anxiety will include the entire sample. Participants were required to be over the age of 18 years and fluent in English; the presence or absence of a history of trauma was not a selection factor for participant inclusion. Written informed consent was obtained from all participants.

The mean participant age was 19.4 years (SD = 2.2), and the majority of participants were freshman (71%). In terms of race, 64.7% self-identified as Caucasian, 21.9% as Black, 2.9% as Asian, 0.4% as American Indian or Alaska Native, 0.3% as Native Hawaiian or Pacific Islander, 8.8% endorsed “Other”, whereas 1.1% preferred not to respond. Additionally, 7.1% of participants identified as being from Hispanic or Latino decent.

Measures

Difficulties in Emotion Regulation Scale (DERS).

As described, the DERS (Gratz and Roemer 2004) is a 36-item self-report measure used to assess six dimensions of emotion regulation: NONACCEPTANCE, GOALS, IMPULSE, STRATEGIES, CLARITY, and AWARENESS. Each item is rated on a 5-point scale based on how often participants believe each item pertains to them (1 = almost never to 5 = almost always). Internal consistency was good for both the DERS total scale (α = .94), as well as its subscales (α values ranging from .80 to .91). The mean full-length DERS total scale score was 76.70 (SD = 21.91).

In order to test study aims, a revised DERS total scale score was also calculated by summing 30 of 36 items; the six items of the AWARENESS subscale were excluded from the calculation of the revised DERS total scale score. The AWARENESS subscale consists of items representing one’s propensity for recognizing and attending to emotions (e.g., I am attentive to my feelings or When I’m upset, I acknowledge my emotions). Internal consistency was good for the revised DERS total scale (α = .95; M = 62.59; SD = 19.82).

Depression, Anxiety, and Stress Scale-21 Item Version (DASS – 21).

The DASS-21 (Lovibond and Lovibond 1995a) is a 21-item measure of depression, anxiety, and stress. Each subscale is made up of seven items. Each item is rated on a 4-point scale based on the extent to which participants believe each statement pertained to them in the past week (0 = did not apply to me at all to 3 = applied to me very much, or most of the time). The DASS-21 Depression and Anxiety subscales – the DASS-21 subscales of interest in the present study—have shown adequate psychometric properties in prior studies (e.g., see Henry and Crawford 2005; Lovibond and Lovibond 1995b). Internal consistency for these subscales was adequate (Depression α = .86; Anxiety α = .77). The mean DASS-21 Depression and Anxiety scores were 3.26 (SD = 3.55) and 2.55 (SD = 3.12), respectively.

Traumatic Life Events Questionnaire (TLEQ).

The TLEQ (Kubany et al. 2000a) assesses exposure to 22 potentially traumatic events (e.g., natural disasters, combat or warfare, assault, sexual abuse) consistent with Criterion A1 and A2 of PTSD as specified in the DSM-IV-TR (APA, 2000; i.e., Criterion A1: exposure to a potentially traumatic event, and A2: the subjective experience of intense fear, helplessness, or horror). For each potentially traumatic event that is endorsed, follow-up questions assess lifetime frequency and whether participants experienced intense fear, helplessness, or horror in response to the event.

Distressing Events Questionnaire (DEQ).

The DEQ (Kubany et al. 2000b) is a self-report measure used to assess PTSS. The DEQ was designed to assess the three clusters of PTSD symptomatology (i.e., reexperiencing, avoidance, arousal) via 17 items. Response options are rated on a 5-point scale, indicating the extent to which respondents have experienced each symptom in the past month (0 = not at all to 4 = extremely). The DEQ has demonstrated good convergent and discriminant validity (Kubany et al. 2000b). Additionally, the DEQ has good short-term test-retest reliability and excellent internal consistency (Kubany et al. 2000b). Internal consistency was good for the DEQ total scale among the subsample of participants who reported experiencing a traumatic event (n = 851: α = .92; M = 11.29; SD = 11.74).

Data Analytic Strategy

DERS: First-Order Confirmatory Factor Analysis.

The correlated six-factor first-order model put forth by Gratz and Roemer (2004) was initially tested. This six-factor model consisted of the 36 DERS items being fixed to their respective factor, with no secondary loadings. The fixing of factor loadings followed the primary salient factor loadings identified by Gratz and Roemer. The DERS factors were allowed to intercorrelate in this six-factor model. To test alternative first-order models, both a one-factor model and an uncorrelated six-factor model of the DERS were tested. The uncorrelated six-factor model was identical to the correlated six-factor model described above, except the latent factor intercorrelations were not freely estimated. The one-factor model consisted of all of the DERS items loading on a single latent factor.

DERS: Higher-Order Confirmatory Factor Analysis.

Given the expected superiority of the correlated six-factor first-order model relative to the alternative first-order models, two higher-order CFA models were then used to further examine the latent structure of the DERS. The first higher-order model tested whether a higher-order factor could account for the first-order DERS factor intercorrelations and thus examined the tenability of using a total scale score. In this model, the first-order DERS factor intercorrelations within the correlated six-factor first-order model were removed and direct effects from a higher-order factor to each of the DERS first-order factors were added (following Brown 2006). The fit of this higher-order model was compared to the fit of the correlated six-factor first-order model.

Next, an alternative higher-order CFA model was tested to further examine the distinctiveness of DERS-AWARENESS from the other dimensions of the DERS. In this second higher-order CFA, two higher-order factors were modeled. One of the higher-order factors was identical to the higher-order factor just described, in which each of the six factors of the DERS loaded on a higher-order factor. For the second higher-order factor, five factors were allowed to freely load on the higher-order latent variable (i.e., NONACCEPTANCE, GOALS, IMPULSE, STRATEGIES, CLARITY). The two higher-order factors were not allowed to intercorrelate. This alternative higher-order model followed Geiser, Eid, and Nussbeck’s (2008) method that allows for the assessment of whether a group of indicators share covariation that is unaccounted for by their relations with another set of indicators.Footnote 1 Applying Geiser et al.’s method to the present study allowed us to further examine whether five of the DERS factors (NONACCEPTANCE, GOALS, IMPULSE, STRATEGIES, CLARITY) share unique covariation that is unaccounted for by their relations with DERS-AWARENESS. Evidence that this second higher-order model provided an adequate, and potentially better fit relative to the single higher-order CFA model described above, would support the notion that DERS-AWARENESS is relatively distinct from the other dimensions of the DERS.

Examination of A Five-Factor Model Of the DERS.

Based on our predictions that DERS-AWARENESS might not belong to the same domain as the other dimensions of the DERS, preliminary evidence related to the latent structure of a revised version of the DERS, in which DERS-AWARENESS items were excluded, was examined. This examination included investigating the adequacy of a correlated five-factor first-order model of the DERS and comparing it to alternative first-order models (one-factor and uncorrelated five-factor models). If this correlated five-factor first-order model was to provide an adequate fit to the data, a second-order model would be examined next to test the adequacy of a general factor accounting for latent factor intercorrelations among the five factors of the revised DERS. The data analytic strategy for testing these models mirrored the strategy outlined above.

All models were tested using LISREL 8.80 (Jöreskog and Sörbom 2006). Robust maximum likelihood estimation (Satorra and Bentler 1994), in which covariance and asymptotic covariance matrices are entered, was used to avoid any concerns surrounding the influence of data non-normality on maximum likelihood estimation (Brown 2006). Four commonly recommended fit indices (Brown 2006; Hu and Bentler 1999) were used to evaluate the models: the comparative fit index (CFI), the non-normed fit index (NNFI), the root mean square error of approximation (RMSEA), and the standard root mean square residual (SRMR). Hu and Bentler’s guidelines stating that the CFI and NNFI should be close to .95 (ideally ≥.95), the RMSEA should be close to .06 (ideally ≤.06), and the SRMR should be close to .08 (ideally ≤.08) to indicate an adequate fit were used.

Along with using these fit indices to evaluate model fit, model comparisons were evaluated as follows. First, the scaled difference in χ2s test (SDCS; Satorra and Bentler 2001) was used. A significant SDCS test between two comparable models indicates a significant decrement in model fit. However, because the SDCS test, as well as chi-square difference tests more broadly, is affected by sample size, model testing completed with large sample sizes might result in significant SDCS tests when differences in parameter estimates are trivial in magnitude (Cheung and Rensvold 2002). As such, and following the recommendations of Kline (2010), we also used alternative tests for comparing models. One alternative test included examining the change in CFI (ΔCFI). Meade et al. (2008) identified a ΔCFI value of less than or equal to .002 as representing functionally trivial differences in parameter estimates among models. A second alternate test for comparing models involves examining RMSEA 90% confidence intervals (CIs). Differences in model fit are considered non-significant if models have overlapping 90% RMSEA CIs (see Wang and Russell 2005).

Concurrent Validity.

Pearson zero-order correlations were used to examine the concurrent validity of the revised total scale score (i.e., that removed AWARENESS items). Measures of depression, anxiety, and PTSS were used as the criterion in our tests of concurrent validity. The magnitude of the relations between the revised DERS total scale and these criteria was compared to the magnitude of relations obtained using the full-length DERS total scale (using Meng et al. 1992, test for dependent correlations).

Incremental Concurrent Validity.

Partial correlations were used to examine the relation between the full-length DERS total scale and symptom outcome measures (DASS-Depression, DASS-Anxiety, and DEQ total scores) while controlling for the effects of the revised DERS total scale (removing the AWARENESS subscale). Evidence that the full-length DERS total scale shares non-significant relations with these criteria after controlling for the reduced DERS total scale would suggest that the full-length DERS total scale does not share unique variance with the symptom measures that is unaccounted for by the revised DERS total scale.

Results

Full-Length DERS First-Order CFA (All 36 Items)

One-Factor Model,

The one-factor model did not provide an adequate fit to the data [χ 2 = 19,416.05; Satorra-Bentler (SB) χ 2 = 14,554.17 (df = 594, p < .01); CFI = .836; NNFI = .826; RMSEA = .151 (90% CI = .149–.153); SRMR = .125]. None of the goodness-of-fit indices met or exceeded the specified guidelines.

Uncorrelated Six-Factor Model,

The uncorrelated six-factor model did not provide an adequate fit to the data [χ 2 = 7,549.56; Satorra-Bentler (SB) χ 2 = 5,824.98 (df = 594, p < .01); CFI = .939; NNFI = .935; RMSEA = .092 (90% CI = .090–.094); SRMR = .285]. None of the goodness-of-fit indices met or exceeded the specified guidelines.

Correlated Six-Factor Model,

The correlated six-factor model provided an adequate fit to the data [χ 2 = 3,780.28; Satorra-Bentler (SB) χ 2 = 3,431.53 (df = 579, p < .01); CFI = .966; NNFI = .964; RMSEA = .069 (90% CI = .067–.071); SRMR = .076]. All of the goodness-of-fit indices met or exceeded the specified guidelines. In addition, the SDCS tests [one-factor versus correlated six-factor: SDCS χ 2 = 1517.21 (df = 15, p < .01); uncorrelated six-factor versus correlated six-factor: SDCS χ 2 = 428.27 (df = 15, p < .01)] indicated that the correlated six-factor model provided a significantly better fit to the data relative to the alternative models. Moreover, the ΔCFIs were uniformly ≥.002 and there was evidence of non-overlapping 90% RMSEA CIs. Thus, both the one-factor and uncorrelated six-factor models appear to provide a significantly poorer fit to the data compared to the correlated six-factor model.

All of the factor loadings within the correlated six-factor first-order model were significant (p < .01). Completely standardized factor loadings from this model are presented in Table 1. The intercorrelations among the DERS factors in this model are presented in Table 2. Not surprisingly, given our large sample, all of the latent intercorrelations were significant (ps < .05). Consistent with prior studies, AWARENESS tended to share small intercorrelations with the other DERS factors (rs ranging from .08 to .61, average r = .27), whereas the other DERS factors shared more robust intercorrelations (rs ranging from .39 to .77, average r = .59).

Table 1 Completely standardized factor loadings from confirmatory factor analysis of items of the Difficulties in Emotion Regulation Scale
Full size table
Table 2 Latent correlations among factors of the Difficulties in Emotion Regulation Scale
Full size table

Full-Length DERS Second-Order CFA (All 36 Items)

The six-factor second-order model generally provided an adequate fit to the data [χ 2 = 3,996.77; SB χ 2 = 3,509.78 (df = 588, p < .01); CFI = .966; NNFI = .963; RMSEA = .069 (90% CI = .067–.072); SRMR = .089]. All of the goodness-of-fit indices, except the SRMR, met or exceeded the specified guidelines. The factor loadings on the higher-order factor were all significant (p < .01). Completely standardized loadings on the second-order factor were generally large: NONACCEPTANCE = .73, ENGAGEMENT = .72, IMPULSE = .78, STRATEGIES = .98, and CLARITY = .64. However, AWARENESS showed a much smaller factor loading on the higher-order factor (i.e., .26).

Whereas the SDCS test indicated that the six-factor second-order model provided a significant decrement in model fit compared to the correlated six-factor first-order model [SDCS χ 2 = 61.38 (df = 9, p < .01)], the ΔCFI was ≤.002 and these two models had overlapping 90% RMSEA CIs. As such, our evaluation of the higher-order model provided somewhat equivocal conclusions. That is, whereas the majority of the goodness-of-fit indices indicated that the higher-order factor adequately accounted for the latent intercorrelations among the factors of the DERS, all of the goodness-of-fit indices did not meet our a-priori benchmark levels indicative of adequate fit. Moreover, one of our tests of model comparisons (i.e., SCDS) indicated that the higher-order model provided a significant decrement in model fit. One tenable reason for these discrepant findings about the adequacy of the higher-order model relates to the possibility that the relatively small contribution of DERS-AWARESS to the higher-order factor attenuated model fit.

Alternative Full-Length DERS Second-Order CFA (Geiser et al. 2008, Method; 36 Items)

The alternative second-order model (Geiser et al. 2008) containing two second-order factors, including one second-order factor that had all six DERS factors load on it and one second-order factor that had all factors except AWARENESS load on it, generally provided an adequate fit to the data [χ 2 = 3,784.84; SB χ 2 = 3,443.56 (df = 583, p < .01); CFI = .966; NNFI = .964; RMSEA = .069 (90% CI = .067–.071); SRMR = .076]; all of the goodness-of-fit indices met or exceeded the specified guidelines. Whereas the SDCS test indicated that this alternative second-order model provided a significant decrement in model fit compared to the six-factor second-order model described above [SDCS χ 2 = 36.78 (df = 5, p < .01)], the ΔCFI was ≤.002 and they had overlapping 90% RMSEA CIs. The adequacy of this alternative second-order CFA, and its potential superiority relative to the other second-order model (i.e., all of the goodness-of-fit indices of this alternative second-order model met or exceed specified guidelines), suggests five of the DERS factors (NONACCEPTANCE, GOALS, IMPULSE, STRATEGIES, and CLARITY) share unique covariation that is unaccounted for by their relations with DERS-AWARENESS.

Revised DERS First-Order CFA (AWARENESS Removed)

Revised One-Factor Model.

The one-factor model of the revised DERS did not provide an adequate fit to the data [χ 2 = 11,446.31; Satorra-Bentler (SB) χ 2 = 7,972.14 (df = 405, p < .01); CFI = .899; NNFI = .892; RMSEA = .134 (90% CI = .132–.137); SRMR = .094]. None of the goodness-of-fit indices met or exceeded the specified guidelines.

Revised Uncorrelated Five-Factor Model.

The uncorrelated five-factor model of the revised DERS generally did not provide an adequate fit to the data [χ 2 = 5,778.22; Satorra-Bentler (SB) χ 2 = 4,196.76 (df = 405, p < .01); CFI = .950; NNFI = .946; RMSEA = .095 (90% CI = .093–.098); SRMR = .321]. Only the CFI and NNFI met or exceeded the specified guidelines.

Revised Five-Factor Correlated Model.

The five-factor correlated model of the revised DERS provided an adequate fit to the data [χ 2 = 2,524.34; Satorra-Bentler (SB) χ 2 = 2,038.38 (df = 395, p < .01); CFI = .978; NNFI = .976; RMSEA = .063 (90% CI = .061–.066); SRMR = .061]. All of the goodness-of-fit indices met or exceeded the specified guidelines. In addition, the SDCS tests [one-factor versus correlated five-factor: SDCS χ 2 = 966.37 (df = 10, p < .01); uncorrelated five-factor versus correlated five-factor: SDCS χ 2 = 475.40 (df = 10, p < .01)] indicated that the correlated five-factor model provided a significantly better fit to the data relative to the alternative models. Moreover, the ΔCFIs were uniformly ≥.002 and there was evidence of non-overlapping 90% RMSEA CIs. Thus, both the one-factor and uncorrelated five-factor model appear to provide a significantly poorer fit to the data compared to the correlated five-factor model. All of the factor loadings within the revised five-factor first-order model were significant (p < .01). Completely standardized factor loadings from this model are presented in Table 1.

Revised Higher-Order CFA.

The five-factor revised second-order model was tested next and it provided an adequate fit to the data [χ 2 = 2550.10; SB χ 2 = 2052.46 (df = 400, p < .01); CFI = .978; NNFI = .976; RMSEA = .063 (90% CI = .061–.066); SRMR = .063]. All of the goodness-of-fit indices met or exceeded the specified guidelines. The factor loadings on the higher-order factor were all significant (p < .01). Standardized loadings on the second-order factor were generally large: NONACCEPTANCE = .73, ENGAGEMENT = .72, IMPULSE = .78, STRATEGIES = .99, and CLARITY = .63.

Whereas, the SDCS test indicated that the five-factor second-order model provided a significant decrement in model fit compared to the six-factor first-order model [SDCS χ 2 = 16.48 (df = 5, p < .01)], the ΔCFI was ≤.002 and they had overlapping 90% RMSEA CIs. As such, our evaluation of the five-factor higher-order model provided a somewhat clearer picture than did our evaluation of the six-factor higher-order model. All of the goodness-of-fit indices of the five-factor higher-order model met our a-priori benchmark to indicate adequate fit. Moreover, although the SCDS indicated that the five-factor higher-order model provided a significant decrement in model fit, because the ΔCFIs was ≤.002 and overlapping RMSEA CIs were observed, a non-significant difference in fit between the two models is suggested.

Concurrent Validity

Bivariate Correlations.

Correlations between the two versions of the DERS total scale (full-length and revised version with the AWARENESS subscale omitted) and symptom measures are presented in Table 3. As shown, the two versions of the DERS total scale showed a similar pattern of correlations with the symptom measures. However, the revised DERS total scale had a significantly stronger relation with anxiety symptoms (z-statistic = 3.18, p < .01) and PTSS (z-statistic = 3.33, p < .01) relative to the full-length DERS total scale. As such, there was no evidence of attenuated relations between the revised DERS total scale and criterion of interest.

Table 3 Zero-order correlations between the Difficulties in Emotion Regulation Scale and psychological symptom measures
Full size table

Partial Correlations.

The full-length DERS total scale did not share significant unique relations with any of the symptom measures after controlling for the effects of the revised DERS total scale (partial rs ranged from −.04 to .03, ns).

Discussion

The DERS is a commonly used self-report measure that purportedly assesses the emotion regulation domain. Through its widespread use, extant data have revealed that one DERS subscale (i.e., AWARENESS) consistently shares only modest intercorrelations with the other DERS subscales and demonstrates differential relations with criteria of interest. Whereas a number of published studies in the emotion regulation literature have used the DERS total scale, no known examination has yet been published in which the adequacy of a higher-order model of the DERS is investigated. As noted by Brown (2006), “a goal of higher order factor analysis is to provide a more parsimonious account for the correlations among lower order factors” (p. 321). To the degree to which a higher-order factor does not account for the latent correlations among the DERS factors, then the computation of a total scale score might not be optimal when using this measure of emotion regulation difficulties. Using a series of CFAs, the present study sought to assess whether the AWARENESS dimension belonged to the same higher-order emotion regulation domain as the other dimensions of the DERS.

Our results replicated prior findings demonstrating that whereas five of the DERS factors (NONACCEPTANCE, GOALS, IMPULSE, STRATEGIES, and CLARITY) cluster strongly with one another, DERS-AWARENESS generally demonstrates much weaker latent factor intercorrelations. CFA results further revealed that DERS-AWARENESS provided a markedly lower contribution to the general DERS factor relative to the other five latent factors of the DERS and that the five DERS factors that cluster more strongly with one another seem to share covariation that is unaccounted for by their relations with DERS-AWARENESS. Overall, our CFA results support the notion that AWARENESS might not be best conceptualized as belonging to the same higher-order emotion regulation construct as the other five dimensions of the DERS.

Some researchers have suggested that the awareness of negative emotional states may not be necessary or sufficient for the adaptive regulation of emotion (Tull et al. 2007). For example, Tull et al. found that DERS-AWARENESS was the only DERS subscale that was not associated with PTSD. They hypothesized that, whereas it may seem intuitive that individuals experiencing the emotional numbing that is a part of PTSD symptomatology would be more likely to be unaware of their emotional states, it is also possible that such individuals are just as aware of, and attentive to, their emotions than those without PTSD. Attention to emotional states does not necessarily suggest a healthy response or regulation of such states. In fact, Lischetzke and Eid (2003) found that attention to emotions had a negative impact on affective well being in individuals who had low emotion regulation abilities. Moreover, as noted by Tull et al., some forms of emotional awareness may be adaptive (e.g., nonjudgmental acceptance), whereas other forms are likely maladaptive (e.g., rumination on negative emotion). Thus, DERS-AWARENESS may sufficiently measure emotional awareness, but emotional awareness may not necessarily be associated with adaptive emotion regulation.

On the other hand, it could be argued that one must be able to properly identify emotions in order to strategically alter affective states. For example, experiencing the emotion of sadness may lead one to employ cognitive (e.g., cognitive reappraisal) and behavioral (e.g., seeking social support) regulatory measures in order to reduce the potential of experiencing a prolonged negative affective state. Those who have difficulties noticing and correctly identifying the experience of sadness would likely fail to employ emotion regulation strategies, and thus, be more likely to experience prolonged states of negative mood, which may develop into psychopathological symptomatology. This view is consistent with an evolutionary perspective in which emotions developed in order to direct behavior; through emotional awareness one has the potential to up- or down-regulate emotional states as needed. Thus, it appears that in the temporal sequence of emotion regulation, emotional awareness and clarity are needed before one can employ adaptive emotion regulation strategies; however, awareness and clarity by themselves do not guarantee that one will employ such strategies. As noted above, it is possible that those who develop psychopathology are just as aware of, and attentive to, their emotions as those who do not develop psychopathological symptomatology, and that the deficits in emotion regulation which lead to pathological conditions come later in the temporal chain (i.e., impulse control, use of emotion regulation strategies, engaging in goal directed behaviors).

Another possibility is that the way in which the DERS-AWARENESS dimension is operationalized might be obscuring the relative importance of this dimension to the emotion regulation domain. For example, the AWARENESS subscale is the only subscale of the DERS in which all of the items are reverse-keyed. In fact, no other scale contains more than two reverse-keyed items. Although reverse-keyed items can serve to identify random responding, they can also increase the likelihood of systematic error, thus reducing scale validity (Hinkin 1995). Moreover, enhanced psychometric properties of self-report measures have been identified upon the removal of reverse-keyed items, which has led some researchers to advocate for the scoring of only the straightforward-worded items of these measures (e.g., see Rodebaugh et al. 2007; Weeks et al. 2005). Given Gratz and Roemer’s (2004) theoretical justification for the inclusion of the AWARENESS dimension of emotion regulation when creating the DERS, future research might seek to create and evaluate straightforward-worded items of this scale. In addition to providing a potentially more comprehensive assessment of emotion regulation, the presence of a revised AWARENESS scale might shed further light on the convergence of this dimension with the other dimensions purported to underlie emotion regulation. Nonetheless, given its divergence from the other dimensions of the measure, the present results support the removal of the current AWARENESS items when computing the DERS total scale.

However, whereas we advocate for the removal of the AWARENESS items from the DERS total scale, the AWARENESS scale may have utility on its own. For example, Neumann et al. (2010) found that, while DERS-AWARENESS was the only DERS dimension that was not significantly associated with measures of anxiety and depression, the AWARENESS dimension was positively associated with delinquent behavior in an adolescent sample. Of note, the AWARENESS dimension was the only DERS dimension found to be significantly associated with delinquency. Neumann et al.’s findings further highlight the divergent pattern of criterion-related validity evidenced by DERS-AWARENESS relative to the other dimensions of the DERS. However, such findings do provide evidence for the potential usefulness of this scale. As such, although it does not appear to belong to the same higher-order construct as the other dimensions of the DERS, future research may wish to examine the validity of the AWARENESS subscale as a construct separate from the DERS.

In addition to examining the latent structure of the DERS, we sought to provide preliminary evidence of the tenability of a revised version of the DERS that removes the AWARENESS items. First, we examined the factor structure of a five-factor model of emotion regulation (i.e., excluding the AWARENESS dimension). The five-factor model provided an adequate fit to the data. Moreover, the revised DERS total scale (removing DERS-AWARENSS) showed good internal consistency and its relations with criteria were not attenuated relative to those relations obtained using the full-length DERS total scale. Further, the full-length DERS total scale did not share unique relations with psychological symptom measures after controlling for the revised DERS total scale. As such, the revised DERS total scale does not appear to substantially sacrifice the criterion-related validity of the original total scale. Taken together, the present results suggest a benefit in reducing the current six-factor model of the DERS in favor of a five-factor model that removes the AWARENESS dimension when computing a total scale score.

Of note, associations between the revised DERS total scale and both depression and PTSS were significantly stronger than the associations between anxiety and PTSS and the full-length DERS total scale. However, it is important to note that the difference in magnitude in the relations between the full-length and revised DERS total scale with the assessed psychological symptom measures were generally small in magnitude. Moreover, these statistical differences were likely substantially influenced by our large sample size and the near-perfect zero-order correlation between the full-length and revised DERS total scale. As such, extant findings reported in the emotion regulation literature using the full-length DERS total scale would likely be quite similar to those obtained using the revised DERS total scale that excludes DERS-AWARENESS. However, the importance of using the most parsimonious available measure cannot be overstated. Thus, the reduced DERS total scale might be favored, particularly when using the DERS within long questionnaire batteries.

Limitations surrounding the present research must be acknowledged. One limitation is that the sample consisted solely of women, and thus, the present results may not generalize to male respondents. It is important to note, though, that the majority of participants in Gratz and Roemer’s (2004) two-study examination of the DERS were women (i.e., Study 1: 73%; Study 2 = 62%). Nevertheless, the present results should be replicated in a sample marked by a relatively equal gender representation. Another limitation relates to our use of a nonclinical college student sample; participants were likely functioning at a higher level than individuals from treatment-seeking populations. It should be noted, though, that the bulk of the available psychometric data on the DERS have been collected using nonclinical samples (e.g., Gratz and Roemer 2004; Neumann et al. 2010). However, to ensure generalizability of the present findings, future research should seek replication in samples that score consistently higher on the DERS, as well as other criterion of interest.

Limitations notwithstanding, emotion regulation deficits have been implicated in the maintenance and exacerbation of a broad range of deleterious outcomes. Given its potential transdiagnostic status, the necessity of having a psychometrically sound self-report measure of emotion regulation cannot be overstated. Through their creation of the first comprehensive assessment of the emotion regulation, Gratz and Roemer’s (2004) DERS has helped researchers further examine the applicability of emotion regulation difficulties to a number of phenomena of interest. However, the present results suggest that using a revised DERS total scale that removes the AWARENESS dimension may provide researchers with a more refined assessment of emotion regulation in their future assessment of this domain of interest.