Introduction

Distress tolerance (DT) refers to the ability to endure uncomfortable or distressing cognitive, physical or emotional states (Brown et al. 2005; Leyro et al. 2010; Simons and Gaher 2005). Researchers hypothesize that individuals low in DT experience distress more acutely and more intensely, and tend to seek relief from their discomfort by engaging in maladaptive escape and avoidant behaviors that are maintained through negative reinforcement (Trafton and Gifford 2011). Individuals high in DT are theorized to be able to cope effectively with their discomfort rather than avoid it. Research has implicated poor DT in the etiology and maintenance of various psychological and behavioral problems (Leyro et al. 2010; Zvolensky and Otto 2007; Zvolensky et al. 2010); including depressed mood (Clen et al. 2011), relapse of substance and nicotine use (Brown et al. 2002; Daughters et al. 2005a, b), non-suicidal self-injury (Nock and Mendes 2008), agoraphobic avoidance in panic disorder (White et al. 2006), unhealthy eating behaviors (Anestis et al. 2007; Corstorphine et al. 2007), obsessive–compulsive disorder, panic disorder, social phobia, and worry (Keough et al. 2010). Further, DT negatively impacts quality of life in samples without diagnosable psychiatric problems (Bernstein et al. 2011) and may serve as a risk factor for engaging in problematic coping behaviors (Abrantes et al. 2008; Brown et al. 2009). Given the growing empirical support for the link between problematic DT and impaired psychological functioning, there has been an interest in examining and refining the DT construct and its assessment among both clinical and non-clinical populations.

Defining Distress Tolerance

There are limitations within the DT literature that undermine the impact of this line of research. A lack of agreement on the conceptual definition of DT has led to a considerable overlap between DT and other related constructs (e.g., anxiety sensitivity, emotional dysregulation, experiential avoidance, frustration intolerance). For example, individuals experiencing anxiety tend to find their symptoms overwhelming and uncomfortable; as a result, individuals low in DT may experience higher levels of anxiety sensitivity and anxiety-related diagnoses (Keough et al. 2010). However, these links are understudied and thus—we cannot be sure whether these constructs merely co-occur or overlap. Similarly, frustration intolerance is conceptualized as a difficulty tolerating the aggravation experienced when the reality of a situation is not as we expected, and the construct is believed to have several components (e.g., emotional intolerance, demands for entitlement, comfort and achievement; Harrington 2005). While, theoretically, distress tolerance appears related to emotional intolerance and demands for achievement (Rodman et al. 2009), it is unclear how much the conceptualizations and measurements of frustration intolerance and distress tolerance overlap. Although a comprehensive analysis of the theoretical link between DT and other constructs is beyond the scope of this paper (for a review see Leyro et al. 2011), this conceptual overlap is problematic because it precludes a clear understanding of the role of DT in the development and maintenance of psychopathology.

The lack of an accepted conceptual definition of DT has resulted in a corresponding inconsistency in the measurement of the construct. Recent theoretical and empirical work characterizes and assesses different forms of distress (cognitive, physical, emotional; Leyro et al. 2010; Zvolensky et al. 2010). As a construct, DT is consistently defined as the “ability to tolerate negative affect or related aversive psychological as well as physical states” (Brown et al. 2005; Daughters et al. 2005; Leyro et al. 2010; Zvolensky et al. 2010). However, many authors refer to any portion of this definition (e.g., emotional, cognitive, physical) broadly as “distress tolerance” which creates a lack of consistency in the extant literature.

Measuring Distress Tolerance

The absence of a cohesive definition of DT, and lack of “gold-standard” measures, has made it difficult to compare results across studies and populations and prevents strong conclusions about the relationship between DT and other constructs (McHugh et al. 2011). Although both self-report and behavioral measurement modalities for DT have emerged, they measure different aspects of the construct. Self-report assessments are used to quantify one’s perceived ability to tolerate distress whereas behavioral measures assess overt and goal-directed actions in the context of distress (e.g., keeping a hand submerged in very cold water). The distinction between cognitive, physical, and emotional aversive states further divides the subset of behavioral DT measurements. Physical assessments of DT primarily measure tolerance of physiological discomfort and thermal stress. Two examples of commonly-used physical DT assessments include the cold pressor task (Burns et al. 2004; Hines and Brown 1932; Neufeld and Thomas 1977; physical task) where the participant keeps their hand submerged in very cold water and the breath holding task (Brown et al. 2002; Hajek et al. 1987; physical task) where an individual holds their breath for as long as possible. Alternatively, emotional or cognitive-based measures, including the Distress Tolerance Scale (DTS; Simons and Gaher 2005) and the mirror tracing task (Strong et al. 2003; requires tracing an object while viewing it through a mirror) assess tolerance of psychological discomfort (Leyro et al. 2010). Whereas the assessment of various domains of discomfort as measures of distress tolerance is not problematic per se, the interchangeable use of physiological and psychological discomfort obscures their potential conceptual differences.

Assessing Distress Tolerance in Clinical Populations

Concerns about the division of DT assessments and the resulting impact on construct validity have led researchers to examine the relationships between and among self-report and behavioral DT measures in clinical samples. McHugh and Otto (2011) reported theoretically inconsistent correlations between measures of distress tolerance in a mixed substance-dependent and a non-dependent control sample. Two self-report measures, the DTS and the Discomfort Intolerance Scale (DIS; Schmidt et al. 2006), were not correlated with any of the three behavioral DT tasks utilized in the study: the mirror-tracing task (Quinn et al. 1996; cognitive task), the cold pressor test, and the breath holding task. A meta-analysis of behavioral research conducted by Podsakoff et al. (2003) found a similar discrepancy among varied methods of measurement. The absence of correlations between behavioral and self-report measures may indicate true differences in the constructs underlying the respective measures or may reflect difficulties that humans have in predicting their own thoughts and feelings in the moment (Kazdin 1974). Some researchers attribute the lack of association between different assessment tools to a fundamental flaw in self-report measures, wherein participants are more often motivated to select responses to questions that they believe are socially desirable or ego strengthening rather than responding truthfully (Podsakoff and Organ 1986).

Additionally, persistence on the mirror tracing task, a cognitive-based behavioral task, was significantly correlated with cold pressor test and breath holding task, two physical behavioral tasks (McHugh and Otto 2011). Surprisingly, there was no significant relationship between persistence on the cold pressor test and the breath holding task, despite the fact that they are both classified as physical-based behavioral measures of DT (McHugh and Otto 2011). These theoretically inconsistent results create further concerns about the concurrent validity of current DT measures, and cast doubt on whether the family of “distress tolerance” assessments truly measures a single construct.

Another study by McHugh et al. (2011) examined DT measures in a drug dependent sample, two samples of cigarette smokers, and an unselected sample. In contrast to their prior study, the authors did not find a relationship between the breath holding task and two cognitive-based DT measures: the mirror tracing task and the paced auditory serial addition attention task (Lejuez et al. 2003).

Finally, Bernstein et al. (2011) examined the relationship between DT assessments in a clinical sample of individuals with various Axis I disorders (e.g., major depression, generalized anxiety, social anxiety). They found that self-report measures of DT (the DTS and the DIS) were strongly and significantly correlated with one another, but these measures were not associated with either physical or cognitive measures (the mirror tracing task, the breath holding task, and a CO2-enriched air task). Also in line with the findings of McHugh and Otto (2011) and Bernstein et al. (2011) found a strong positive correlation between mirror tracing task tolerance (cognitive) and breath holding task duration (physical), but they did not find a correlation between the mirror tracing task and the CO2-enriched air challenge (physical). Again, the two physical measures of DT (CO2 and breath holding task) were not correlated.

Assessing Distress Tolerance in Mixed Samples

In order to better understand the utility and validity of DT assessments and interpret these results, relationships among measures must be adequately examined in samples that contain both clinical and non-clinical participants. Unselected samples incorporate both types of individuals, and may reflect an important subset of the population who may also struggle with low levels of DT that either impact quality of life (Bernstein et al. 2011) or lead to future behavioral problems (e.g., smoking, drinking; Abrantes et al. 2008; Brown et al. 2009). Such samples may also reflect a wider range of DT scores than clinical samples and also represent differences in the broader population. To this end, Cougle et al. (2013) investigated the validity of the DTS and DIS as measures of emotional and physical DT in an unselected undergraduate population. Similar to the Bernstein et al. (2011) findings, the DTS and DIS were significantly correlated. However, Cougle et al. (2013) did find significant relationships between self-report measures and behavioral measures: the DIS was significantly correlated with a hand grip task (physical). Moreover, the DTS, but not the DIS, was found to be significantly associated with self-reported emotional tolerance and perceived threat following four film clips that aimed to induce specific emotions.

Summary and Aims

In sum, the research to date suggests that behavioral DT measures often do not significantly correlate with one another and that behavioral measures differentially correlate with self-report measures. Furthermore, there are inconsistent results across studies and populations, and there is very little research on the relationship between performance on measures of DT and real-world outcomes (e.g., avoidance of stressful events or emotions). Taken together, these findings raise questions regarding the best way to assess distress tolerance and whether current measures truly converge on a single, unitary construct.

Prior to addressing these larger issues, further research exploring the relationships among various measures of DT in unselected populations is needed in order to determine whether we need to re-evaluate the interchangeable use of DT assessments in future research. As such, the current study administered self-report and both cognitive and physical-based behavioral measures of DT to an unselected undergraduate population. We chose this population to replicate previous research in unselected samples (e.g., Cougle et al. 2013, McHugh and Otto 2011) and because it would reflect broader levels of DT than clinical samples. These samples also represent an important area of study for DT in particular, as it is hypothesized to predict the onset of psychological problems (Leyro et al. 2010).

We chose one physical-based DT behavioral assessment (the breath holding task), one cognitive-based behavioral DT assessment (mirror tracing), one emotional DT self-report measure (distress tolerance scale), and one self-reported measure of perceived physical DT (discomfort intolerance scale). These four measures were chosen as they are among the most widely-used in the DT research but may actually measure different latent constructs. We decided not to add additional measures due to concerns regarding participant fatigue and the effect on DT during a study visit. For this reason, we also excluded DT measures with greater levels of physical exertion (e.g., cold pressor task).

Materials and Methods

Participants

Power analyses, calculated using G*Power (Faul et al. 2007, 2009), indicated that we needed a sample of 78 individuals to obtain full power (0.80) for our analyses. We attempted to recruit more individuals than needed in case technological difficulties (mirror tracing task) or other unpredictable factors rendered some of the data unusable. Eighty-three undergraduate students (76 % female) provided written informed consent to participate. Data from all participants were usable and included in our analyses. All participants were enrolled in one or more psychology courses at a large university in Philadelphia, Pennsylvania, and ranged in age from 18 to 34 years old (M = 20.62, SD = 2.57). The sample was 68.9 % Caucasian, 23.2 % Asian American, and 5.8 % African American. Fifty percent of individuals were employed in addition to being students; 81 % of our sample were full time students. Individuals were primarily recruited through psychology course announcements in exchange for extra course credit.

Measures

Discomfort Intolerance Scale (DIS)

The DIS (Schmidt et al. 2006) is a five-item questionnaire that measures the subjective belief about one’s capacity to withstand physical discomfort. Responses consist of six-point Likert scale items ranging from “not at all like me” to “extremely like me.” The DIS has been found to have good internal consistency (Cronbach’s α = 0.70) and test–retest reliability (Schmidt et al. 2006). Mean score on the DIS in a sample of over 1000 non-clinical individuals was 11.1 (Schmidt et al. 2006). High scores on the DIS indicate low levels of distress tolerance, as the DIS focuses on perceived levels of physical DT. Cronbach’s α for our sample = 0.71.

Distress Tolerance Scale (DTS)

On the DTS (Simons and Gaher 2005), participants are asked to indicate the extent to which they agree with 15 statements about finding emotional distress intolerable. The DTS shows good internal consistency (Cronbach’s α = 0.82), as well as test–retest reliability (Simons and Gaher 2005). Average score on the DTS is 3.37 in a student sample (Simons and Gaher 2005). High scores on the DTS indicate high levels of emotional distress tolerance. Cronbach’s α for our sample = 0.91.

Mirror Tracing

This computerized task aims to measure cognitive distress tolerance (Strong et al. 2003). Participants are asked to trace geometric shapes (i.e., a star) using a mouse that moves in the opposite direction than intended. When the mouse moves off the shape, a buzzer sounds and the participant must start over. Participants are told that they will be compensated based on how far (in pixels) they are able to successfully trace the shape as an incentive. They are also told that they are allowed to terminate the task at any time. Tolerance is measured by how long the participants persist on the task before they discontinue (Strong et al. 2003).

The Breath Holding Task

This task is a commonly used physiological measure of distress tolerance (Daughters et al. 2005; Hajek et al. 1987; Zvolensky et al. 2001). Participants are asked to hold their breath for as long as they can, and to indicate (by raising their hand) when they have a strong urge to take a breath. They are also asked to raise their hand again when they discontinue the task by taking a new breath. The assessor uses a stopwatch to time (in seconds) when they begin holding their breath to when they indicate having a strong urge to take a breath and also when they end the task. The task is repeated two times with a 60 s inter-trial rest period (Zvolensky et al. 2001). Of the two trials, the average time between the urge to breathe and the actual breath is used as the measure of distress tolerance. The measure has shown good test–retest reliability (r = 0.69, p < 0.05; Zvolensky et al. 2001).

Procedure

Interested participants were directed to a secure online system that allowed them to read more about the study and sign up for participation. After providing informed consent, participants completed a demographic questionnaire. The self-report portion of the assessment was divided into three parts to prevent participant fatigue. The breath holding task was administered following the first third of the self-report questionnaire and the mirror-tracing task was administered after the middle third of the questionnaire. The order of study assessments was consistent across participants. Participants completed the computerized assessment and self-report questionnaires seated alone at a desktop computer in a small room. All participants were awarded extra credit points in a psychology course, as well as one dollar for completion of the mirror-tracing task, independent of task performance.

Results

Descriptive data for all study measures can be found in Tables 1 and 2. Skewness and kurtosis were both within acceptable limits for all variables. There was one outlier for the mirror-tracing task. However, this score was still within an acceptable range for the task and was associated with higher levels of DT across all measures for this individual. Furthermore, removing this outlier did not impact overall results. Therefore, the outlier was kept in our analysis.

Table 1 Descriptive data for distress tolerance assessments
Full size table
Table 2 Descriptive data for distress tolerance assessments by gender
Full size table

Bivariate correlations (Table 3) revealed a medium-sized, statistically significant correlation between the DIS and DTS (r = −0.31, p < 0.01; note that a negative correlation was expected because higher scores on the DTS indicate greater DT whereas higher scores on the DIS indicate lower DT). However, the remaining measures (including the two behavioral tasks) evidenced very low (and statistically insignificant) associations with one another.

Table 3 Bivariate correlations among distress tolerance self-report and behavioral measures
Full size table

We analyzed results by gender, as there is preliminary evidence that males report higher levels of DT than females (Simons and Gaher 2005). However, we found no significant gender differences on any DT measures. Furthermore, when we compared correlations between genders using the Fisher r-to-z transformation, all comparisons were non-significant (p > 0.05).

Discussion

Although empirical data related to DT has grown considerably over the last few decades, fundamental questions remain about the nature of the construct and its measurement. These inconsistencies make it difficult to draw conclusions across studies and to understand the link between DT and the development, maintenance, and treatment of psychopathology.

We examined the intercorrelation of common self-report and behavioral DT measures in an unselected undergraduate population. Our findings indicated that the self-report measures correlated with one another, but the strength of their correlations with the behavioral measures was very small. These results, in part, replicate prior research by Bernstein et al. (2011) and Cougle et al. (2013), who also reported a significant relationship between self-report measures of DT despite the fact that the DTS and DIS measure different elements of DT (e.g., perceived emotional vs. perceived physical DT). These consistent findings may indicate that these two self-reports capture different elements of perceived distress tolerance and load onto the same overarching latent construct. It is also important to note, from a measurement perspective, that there are many issues with self-reported data that may increase the likelihood that self-report measures correlate with one another independent of an actual relationship between constructs. The literature is replete with studies pointing to the methodological limitations of traditional self-report measurement (Campbell and Lee 1988), and these limitations have motivated the development of new assessment technologies (i.e., Ecological Momentary Assessment, EMA; Shiffman et al. 2008). Therefore, these results should still be interpreted with caution.

In contrast with Cougle et al. (2013), we did not find a relationship between behavioral and self-report measures in our sample. There are a few potential explanations for these discrepant findings in self-report and behavioral assessments of DT. The lack of association between one’s perceived ability to tolerate distress (i.e., via self-report questionnaires) and his or her actual ability (i.e., via behavioral measures) may reflect the difficulty people have in predicting their own thoughts, feelings, and behaviors in the moment (Kazdin 1974). Thus, behavioral and self-report measures may not converge on a single, shared DT construct. Bernstein et al. (2011b) posited that self-reported (or perceived) DT requires individuals reflect back on prior examples of their ability to regulate emotions when answering questions (even if those questions are future oriented). Alternatively, behavioral measures of DT measure acute emotion regulation ability while experiencing distress. As such, self-report and behavioral measures of DT may actually converge on different latent constructs (Bernstein et al. 2011a). Similarly, cognitive (e.g., mirror-tracing) and physical (e.g., breath holding) tasks may activate different types of experiential discomfort (emotional vs. physical), which, in turn, may require different strategies for tolerating that discomfort. This may explain why studies inconsistently find a relationship between physical and cognitive DT measures.

Although our results also revealed only a very weak and non-significant relationship among behavioral measures of distress tolerance, other authors (e.g., McHugh and Otto 2011) previously reported moderate correlations between cognitive and physical behavioral distress tolerance measures (i.e., the computerized mirror tracing persistence task, a breath holding task, and a cold pressor task). One explanation for these discrepant findings may lay in the fact that McHugh and Otto utilized a clinical sample, whereas we studied unselected undergraduate participants. Prior research indicates that psychopathology is associated with decreases in measures of DT across domains (Leyro et al. 2010), and thus, we would expect that studies across clinical, non-clinical and mixed populations might not be comparable due to distinct differences in DT levels.

Taken together, these findings argue against the conceptualization of DT as a single, unitary construct, and instead support the hypothesis that multiple domains of distress tolerance exist. It may be possible that tolerance of cognitive frustration, tolerance of physiological discomfort, and tolerance of emotional discomfort are all separate constructs. Refining our understanding of the construct is becoming increasingly important, as it has been implicated in the pathogenesis and maintenance of maladaptive behaviors (e.g., substance abuse, eating pathology). Furthermore, a variety of psychotherapeutic interventions have incorporated strategies aimed at improving DT. Improved understanding of the assessment of DT may allow researchers to clarify conceptual and theoretical issues surrounding the construct, which in turn may lead to an enhanced understanding of the relationship between DT, psychopathology, and treatment outcome. For example, researchers interested in examining treatment outcomes in an intervention that targets emotional disorders may find that a cognitive-based DT assessment is more appropriate for their study and that physical measures of DT are not related to their symptoms of interest. Clinically, a therapist might chose to assess DT in a patient with borderline personality disorder, an eating disorder, or another syndrome linked to DT in order to better assess symptom severity or treatment response. Distress tolerance is viewed as a mechanism underlying several psychiatric conditions (e.g., borderline personality disorder, substance use) and their treatments (e.g. Buckner et al. 2007; Linehan 1993). As such, a better understanding of our DT assessments may be critical in developing and interpreting future research on the development, maintenance, and treatment of these clinical conditions. Unfortunately, the state of the current DT research has not yet reached the point of determining which specific domains of distress tolerance are most influential in the development or treatment of particular psychopathological symptoms. This may be due, in part, to the challenges associated with reliable, valid DT measurements.

There are several limitations to this study that should be considered. First, our sample consisted of unselected undergraduates and may not be generalizable to other purely clinical populations. We only included one type of each subset of behavioral measures of DT (one examining physical distress and one cognitive distress), which precludes the ability to assess each of these sub-domains more closely. Furthermore, our sample was primarily composed of females. Although there were no significant differences in outcomes by gender in our sample (see Tables 2, 3), there is some evidence of sex differences in performance on DT tasks in the literature (Simons and Gaher 2005). Nevertheless, the results underscore the conclusion that existing assessment and conceptualizations of DT need refinement.

The present results, in combination with existing literature, support the re-conceptualization of distress tolerance as a term that consists of distinct domains. Future research should continue to explore correlations among distress tolerance measures using more sophisticated statistical analyses (e.g., cluster analyses, factor analyses) as well as examining the relationship between different measures of distress tolerance and real-world outcomes (e.g., rates of substance use, emotional avoidance). Given the growing inconsistencies among studies that measure the association between DT assessments, there is a need for additional research on both the measurement and theoretical conceptualization of the DT construct.