Introduction

Rational Emotive Behaviour Therapy (REBT) (Ellis 1994) is concerned with the dysfunctional ‘irrational’ beliefs that characterise psychological disturbance. REBT proposes that a central characteristic of irrational beliefs are the absolutistic “musts” we bring to events such as failure, rejection, and poor treatment by others (Ellis and Dryden 1997). Arising from these musts are two categories of psychological disturbance: ego disturbance and discomfort disturbance (Ellis 1979, 1980, 1994). Ego disturbance is characterised by irrational beliefs about the absolute conditions required for self-worth, and subsequent self-criticism when these conditions are unmet. Discomfort disturbance (also termed frustration intolerance), is characterised by irrational beliefs regarding the intolerance of discomfort and frustration, and the demand that comfortable and easy conditions must exist (DiGiuseppe et al. 2014).

Despite the fact that frustration intolerance is a central concept in REBT, specific research of this construct has been limited. One reason for this has been the relative lack of a specific measure of frustration intolerance beliefs, separate from self-worth beliefs. In addition, the concept of frustration intolerance has traditionally been considered a single dimension, limiting the analysis of specific emotional and behavioural problems and their relationship with different aspects of frustration intolerance. It has been argued that frustration intolerance involves several different content areas, such as tolerance of fairness, comfort, and uncertainty (Dryden 1999). Furthermore, that such content areas may be differentially related to distinct psychological problems (Harrington 2006).

Frustration Discomfort Scale (FDS) (Harrington 2005a) was designed to measure intolerance of frustration as a multidimensional construct. Exploratory and confirmatory factor analysis on clinical and student samples indicated that frustration intolerance was best described by four factors: Discomfort intolerance, entitlement, emotional intolerance and achievement frustration. The discomfort intolerance scale concerns the belief that life should be easy, comfortable and free of hassle (e.g., “I can’t stand doing tasks that seem too difficult”). The entitlement scale concerns the belief that wishes must be met and that other people should indulge and not thwart these wishes (e.g., “I can’t tolerate being taken for granted”). The emotional intolerance scale concerns the belief that emotional distress is intolerable and must be quickly relieved or avoided (e.g., “I can’t bear disturbing feelings”). The achievement frustration scale concerns perfectionistic beliefs and assesses the intolerance of obstructed goal achievement (e.g., “I cannot bear to move on from work I am not fully satisfied with”).

Further studies (Harrington 2005b, c, 2006) indicated that this instrument allowed for a more complex assessment of frustration intolerance. Thus, empirical evidence indicated that different frustration intolerance sub-scales were associated with specific emotional dysfunctions such as anxiety, depression and anger (Harrington 2006). This research also showed that FDS sub-scales were differentially associated with specific emotions, independent of self-esteem and negative affect. In particular, the entitlement sub-scale uniquely predicted anger, discomfort intolerance uniquely predicted depression, and emotional intolerance uniquely predicted anxiety. Different aspects of frustration intolerance were also associated with different behavioural problems. For example, and as predicted by REBT theory, two groups of procrastinators emerged. The first group reflected self-worth beliefs and fear of failure, whilst the second reflected frustration intolerance, and more specifically, discomfort intolerance beliefs regarding task aversiveness (Harrington 2005c). There were similar differential relationships with other self-control problems. For instance, behavioural avoidance significantly correlated with discomfort and emotional intolerance, but not with entitlement or achievement frustration (Harrington 2005b).

The FDS has been translated into Serbian, confirming the psychometric reliability and validity of the four-factor structure, and supporting the cross-cultural applicability of the scale (Stanković and Vukosavljevic-Gvozden 2011). This research also analysed the relationship between frustration intolerance and emotional dysfunction in a non-clinical student sample. This study differed from Harrington (2006) in employing measures of state and trait anxiety and anger. However, consistent with the previous research, hierarchical multiple regression indicated that entitlement was a unique predictor of trait and state anger; discomfort intolerance a significant predictor of state depression; and emotional intolerance a unique predictor of trait anxiety. Emotional intolerance was also a predictor of depression, but none of the FDS sub-scales were significant predictors of state anxiety.

Contrary to previous results, achievement frustration was a negative predictor of depression, indicating that stronger achievement frustration beliefs were associated with less depression. However, other research has also found positive relationships between some types of perfectionistic beliefs and increased psychological resilience (Flett et al. 1996), and between perfectionistic ‘organization’ and reduced procrastination (Stober 1998). Likewise, a previous study (Harrington 2005c) investigating the relationship between frustration intolerance and procrastination reported that higher scores on achievement frustration were associated with reduced frequency of procrastination. One possible explanation for these results is that the achievement scale is not assessing irrational beliefs, but rather a preference for high standards (Stanković and Vukosavljevic-Gvozden 2011), or that in some circumstances, irrational beliefs might have positive consequences (Harrington 2005c).

Jibeen (2013) also investigated the relationship between frustration intolerance beliefs and emotional problems in a non-clinical student sample, using an Urdu version of FDS. Again, consistent with previous research, regression analysis showed that entitlement was a unique predictor of hostility, while emotional intolerance was a unique predictor of anxiety. However, entitlement appeared as the main predictor of depression, in contrast to the previous research, which had indicated discomfort intolerance in this role (Harrington 2006). This discrepancy may reflect differences in the sample population used by these three studies. Whilst Harrington (2006) employed clinical subjects, the other studies involved college students (Stanković and Vukosavljevic-Gvozden 2011; Jibeen 2013). Such non-clinical groups may not be appropriate when investigating emotional disturbance, since most students will not be suffering from significant psychological disorders. A further possibility is that these discrepancies reflect cultural or gender variation in emotional expression and beliefs. Indeed, previous studies using different cultural groups (e.g., Serbian, Urdu, English) have reported marked variation in gender of the sample (91 % male, Jibeen 2013; 36 % male, Stanković and Vukosavljevic-Gvozden 2011; 41 % male, Harrington 2006).

Apart from procrastination, there has been little investigation of frustration intolerance beliefs in regard to specific avoidance behaviour. An important type of social avoidance is non-assertiveness (Arrindell et al. 1999). Lack of assertiveness might arise from self-worth issues, such as the belief that other people are more worthwhile and come first (Hauck 1991). For instance, Lange and Jakubowski (1976) showed that the irrational beliefs held by non-assertive individuals included a need for absolute approval (“I must be loved by everyone I care about”), awfulising (“It would be just awful if I hurt his feelings”) and perfectionistic demands (“I need to be good at everything I do”). However, as Alden and Safran’s (1978) research suggested, it is not lack of knowledge of assertive behaviour that interferes with appropriate assertiveness and increases anxiety, but the presence of irrational beliefs. However, non-assertiveness may also arise from frustration intolerance beliefs, such as intolerance of emotional distress, or intolerance of uncomfortable social conflicts. Individuals may also fail to be appropriately assertive, not by passive avoidance, but by being overly aggressive in pursuit of their demands. This aggressive non-assertiveness could be associated with entitlement beliefs, such as the belief in one’s ‘perfect rights’ (Robb 1992).

This present study used an Italian translation to investigate the psychometric characteristics of the FDS in an Italian student sample. Since well validated measures are necessary to enable cross-cultural comparisons of frustration intolerance and other REBT concepts, the study aimed to verify the factorial structure and reliability of this version. It also investigated the criterion validity, in regards to the relationship between sub-scales and specific emotions such as anxiety, depression, and anger. In addition, it aimed to investigate the relationship between the different aspects of frustration intolerance and assertive behaviour.

Method

Participants and Procedure

Participants included 250 undergraduate students from different disciplines of the University of Messina (Sicily, Italy): Educational Sciences (58 %), Medicine (35.6 %), Engineering (3.2 %), Physical Sciences-Mathematical and Natural Sciences (2.4 %) and Law (0.8 %). The sample consisted of 163 females (65.2 %) and 87 males (34.8 %) with a mean age of 23.29 years (SD = 3.54). The demographic questionnaire collected basic demographic information on the participants, including age, gender, and educational level/academic class. All participants completed five questionnaires administered during one classroom period lasting approximately 40 min and did not receive any compensation.

Instruments

The Frustration Discomfort Scale (FDS; Harrington 2005a) consisted of 28 items rated on a 5-point Likert-type scale with the following anchors: (1) absent, (2) mild, (3) moderate, (4) strong, (5) very strong. The instrument consisted of four sub-scales each containing seven items: discomfort intolerance, entitlement, emotional intolerance and achievement frustration. All statements were worded in terms of frustration intolerance (e.g., “I cannot stand/tolerate/bear”). The FDS has good evidence of reliability (Cronbach’s α = .94 for the full scale) with acceptable coefficient alphas for the respective sub-scales: .88, .85, .87, and .84. The Italian version of the FDS was developed using the back-translation method (Tables 1, 2).

The State-Trait Anxiety Inventory (STAI-X; Spielberger et al. 1970) was used for the evaluation of anxiety using the Italian version by Lazzari and Pancheri (1980). This consists of two separate 20-item sub-scales that measure trait (baseline) and state (situational) anxiety, on 4-point Likert-type measure. The State Anxiety scale was designed to measure transitory anxiety states: that is, subjective feelings of apprehension, tension, and worry, that fluctuate in intensity over time. Higher scores indicated higher anxiety. The Trait Anxiety scale (STAI-X2) measured relatively stable individual differences in anxiety proneness or differences in the tendency to experience anxiety states. High trait-anxiety subjects were more prone to respond to situations perceived as threatening with significant increases of anxiety.The reliability and validity of this scale have been documented in different countries (e.g., Muhamad et al. 2013; Deb et al. 2010), including Italy (e.g., Iliceto et al. 2011). In this study, the internal consistency (Cronbach’s α) of the scales was .92 for State Anxiety and .91 for Trait Anxiety.

The Depression Questionnaire (DQ; Bertolotti et al. 2000), consisting of 24 items, measured depression including dysphoria and depressive manifestations of subclinical significance. The subjects check whether the statements reflect their current living condition by answering “Yes” or “No”. High scores indicated the presence of depressive feelings. The reliability and validity of the depression questionnaire have been demonstrated in previous researches (e.g., Dorz et al. 2003). In the present study, internal consistency (α) of this scale was .83.

The State-Trait Anger Expression Inventory-2 (STAXI-2; Spielberger 1999) was used to evaluate anger using the Italian version by Comunian (2004). This inventory consisted of a state anger and a trait anger scale containing 15 items and 10 items, respectively, on a 4-point Likert-type scale. The state anger scale measured how angry participants feel at the time of administration, while the trait anger scale measured the degree to which participants generally feel angry. High scores on the state anger scale indicated a greater intensity of angry feelings and a greater extent of feeling like expressing anger at the time of the administration. High scores on the trait anger scale indicated a higher frequency of angry feelings experienced over time. The reliability and validity of this scale have been demonstrated in previous research (e.g., Zoccali et al. 2007; de Anda et al. 2000). In the present study, the reliability (α) was .94 for the state anger and .88 for trait anger.

The Scale for Interpersonal Behaviour (Arrindell et al. 1984) measured assertive behaviours using the Italian version short form (SIB-r) by Arrindell et al. (2004). The SIB-r is a self-report measure of difficulty (performance) and discomfort in state assertiveness (distress). This measure consist of 50 items involving two subscales: (1) performance (25 items), that is the probability of engaging in a specific assertive response; (2) distress (25 items), that is the degree of discomfort (felt anxiety/distress) associated with self-assertion in specific social situations. Participants score two separate 5-point Likert-type scales with one for distress (“not at all” to “extremely”), and performance (“I never do” to “I always do”). High scores on the distress sub-scale indicated that the person experiences anxiety or discomfort in many social situations. Low scores on the performance sub-scale indicated a low probability of engaging in assertive behaviours. The reliability and validity of this scale have been documented in different countries (e.g., Bouvard et al. 1999; Eskin 2003), including Italy (e.g., Nota and Soresi 2003). The reliabilities (α), in the present study, were .92 for distress and .93 for performance.

Table 1 Confirmatory factor analysis of the Italian frustration discomfort scale: fit indices
Full size table
Table 2 Factor analysis of the Italian frustration discomfort scale: means, standard deviations for items
Full size table

Data Analyses

All descriptive analyses, gender differences (MANOVA), and correlations, were carried out using IBM SPSS Statistic version 19.0. To verify the factorial validity, a confirmatory factor analysis was conducted employing the maximum likelihood method using Eqs. 6 (Bentler 1995). Preliminary analysis revealed that the data deviated significantly from multivariate normality (Mardia’s coefficient = 115.29). For this reason, an adaptation of the model was employed using the Satorra-Bentler correction. Several indices were used to assess the goodness of fit of the model: the model Chi square provided a measure of overall fit where a non-significant Chi square (χ2) statistic indicates a good fit. Since the Chi square test is sensitive to sample size (Bentler 1990), a Chi square normalised by degrees of freedom (χ2/df) with a ratio of <3.0 (Hair et al. 2006) indicated a good fit. On the Comparative Fit Index (CFI), a value equal to or >0.90 indicated a good fit (Bentler 1990). The root mean square of approximation (RMSEA; Browne and Cudeck 1993) indicated an acceptable fit with 90 % confidence with values ≤0.08 (Browne and Cudeck 1993; Marsh et al. 2004).

Results

Descriptive Analysis

The internal consistency (Cronbach’s α) and descriptive statistics (means, standard deviations, skewness and kurtosis) for all the instruments are shown in Table 3. The mean scores of FDS sub-scale were similar (although slightly higher) to those reported for the student samples in Harrington (2005a) (discomfort 17.63, entitlement 19.52, emotional 18.49, achievement 20.70, full scale 76.35) and Stanković and Vukosavljevic-Gvozden (2011) (discomfort intolerance 17.37, entitlement 19.83, emotional intolerance 17.86, achievement 19.67, full scale 74.86). In the present Italian sample, the internal reliability of the FDS was .92 (Compared to a full scale reliability of .94 in Harrington 2005a). The internal consistency of the sub-scales was .73 for discomfort intolerance, .74 for entitlement, .83 for emotional intolerance, .81 for achievement. The internal consistency for discomfort intolerance and entitlement subscale are lower than the other translations of the instruments, but would be considered adequate according to Nunnally (1978), who suggests that a reliability coefficient above 0.7 is acceptable.

Table 3 Reliability, means, standard deviations, skewness, and kurtosis
Full size table

The mean scores of STAI-X, STAXI-2, DQ and SIB-r were consistent with the normative data of these instruments. The descriptive analysis showed that all scales have good symmetry and kurtosis values (Table 3). Symmetry and kurtosis values for state anger and depression were higher than absolute values of 1. However, according to Curran et al. (1996), skewness and kurtosis absolute values of 0–2, and 0–7, respectively, can be taken as demonstrating sufficient univariate normality. Furthermore, these figures were similar to those reported by Stanković and Vukosavljevic-Gvozden (2011).

Gender Differences

To investigate gender differences a MANOVA was conducted, with the four FDS factors as the dependent variables and the two participant groups (males, females) as the independent variable. There was a significant multivariate main effect of gender (Wilks’ Lambda = 0.935, F (4, 245) = 4.24, p < .05, η 2p  = .065), with a significant univariate effect for emotional intolerance (F (1, 248) = 12.49, p < .001, η 2p  = .048), with females reporting higher scores. The other sub-scales did not significantly differ between genders: discomfort intolerance, F(1, 248) = .464, p > .05, η 2p  = .002; entitlement, F(1, 248) = .566, p > .05, η 2p  = .002; achievement, F(1, 248) = 1.78, p > .05, η 2p  = .007.

A second MANOVA was computed with the two STAI factors as the dependent variables and the two participant groups (males, females) as the independent variable. There was a significant multivariate main effect of gender (Wilks’ Lambda = 0.884, F (2, 247) = 16.18, p < .001, η 2p  = .116), with a significant univariate effect for state anxiety, F (1, 248) = 13.48, p < .001, η 2p  = .052), and for trait anxiety, F (1, 248) = 32.47, p < .001, η 2p  = .116, with females reporting higher scores than males.

A third MANOVA was carried out with the two factor SIB-r (distress and performance) as the dependent variables and the two participant groups (males, females) as the independent variable. There was a significant multivariate main effect of gender (Wilks’ Lambda = 0.951, F (2, 247) = 6.404, p < .05, η 2p  = .049), with a significant univariate effect for distress, F (1, 248) = 9.13, p < .05, η 2p  = .036, with females reporting higher scores than males. The other sub-scale did not significantly differ between genders for performance, F (1, 248) = 1.08, p > .05, η 2p  = .004.

A further MANOVA was carried out with the two factor STAXI (state anger and trait anger) as the dependent variables and the two participant groups (males, females) as the independent variable There was no a significant multivariate main effect of group (Wilks’ Lambda = 0.983, F (2, 247) = 2,176 p > .05, η 2p  = .017). The sub-scales did not significantly differ between gender, state anger F (1, 248) = .025, p > .05, η 2p  = .000; trait anger F (1, 248) = 3.437, p > .05, η 2p  = .014.

Finally, we computed a univariate ANOVA with QD (questionnaire depression) as the dependent variable and participant groups (males, females) as the independent variable. A significant univariate gender effect was found for depression, F (1, 248) = 20.75, p < .001, η 2p  = .077, with females reporting higher scores than males.

Correlations

Zero-order correlations between the measures are shown in Table 4. Intercorrelations among the FDS sub-scale were high (range: .59–.67) and were higher than those reported in the research of Harrington (2006) in a clinical sample, but lower than reported by Stanković and Vukosavljevic-Gvozden (2011) in a non-clinical student sample. The FDS full scale score had a very high correlation with the individual sub-scales (range: .82–.86). However, sub-scale intercorrelations were lower than their respective reliabilities.

Table 4 Correlations among measures
Full size table

Correlations between FDS sub-scales and the other measures show that discomfort intolerance was substantially correlated with state anxiety, trait anxiety, state anger, depression, trait anger, and SIB-distress; entitlement was significantly related to depression, state anxiety, trait anxiety, state anger, trait anger, and SIB-distress; emotional intolerance was correlated with state anxiety with trait anxiety, with depression, with state anger, with trait anger, and with SIB-distress; achievement was significant correlated with state anxiety, trait anxiety; depression, state anger, trait anger, and SIB-distress. None of the FDS sub-scales had a significant correlation with sib-performance. The correlations with emotional disturbance were consistent with Harrington (2006). They are also consistent with Stanković and Vukosavljevic-Gvozden (2011), in showing that trait measures had a stronger relationship with frustration intolerance than state measures.

Confirmatory Factor Analysis of the Italian Frustration Discomfort Scale

The fit indices (Table 1) indicated an acceptable fit: χ2/df = 2.51; RMSEA = .068. Furthermore, all items loaded onto their designated factors and the standardised factor loadings were all significant, with values ranging from .35 to .76. However, the Chi square (χ2 = 865.525) and the comparative fit index (CFI = .813) indicated a lack of fit.

Furthermore, inspection of the fit statistics indicated that some degree of model misfit still remained. Indeed, a review of the modification indices (MI) revealed some abnormally large values representing error covariances between various items. However, there is considerable controversy in the CFA literature regarding the interpretability of such values and their cause. Bentler and Chou (1987) have remarked that model specification that forces all error terms to be uncorrelated is rarely appropriate with real data. Incorporation of these correlated error terms into CFA models does not otherwise undermine the factorial validity of the instruments, but rather provides a more realistic factorial representation of the observed data structure (Byrne 1993; Beckstead 2002; Swisher et al. 2004).

Based on inspection of MIs associated with the correlated error variances, specific error covariance terms were freed sequentially. That is, one parameter was freed and then the likelihood-ratio test was used to assess the significance of improvement in the fit of the model. This process continued until freeing additional parameters did not produce a significant improvement in model fit. Correlated errors are often the result of redundancy in item content (Byrne 1994). Given that this explanation is conceptually plausible, the model was re-specified. The resulting model was consistent with the observed data and the various fit indices also reflected an acceptable fit of the model to the data, χ2 (313) = 485.61, p < .01; S–B χ2 (313) = 421.01, p < .01; R-CFI = .95; R-NNFI = .94; R-RMSEA = .04 (90 % CI = .03–05). The model and its parameter estimates are shown in Fig. 1, along with residual variance terms for each item. For clarity of presentation, the values of the 24 correlations among the residual variances are not shown in Fig. 1, however, all were significant (p < .05). Freeing 24 of the 378 error covariance terms significantly improved the fit of the model. These correlated errors were permitted based on the apparent item content overlap that may reflect similar/related emotional experiences and, therefore, could share a common source of variance other than the general distress factor. Finally, previous exploratory and confirmatory factor analysis (which compared five alternative models—including a one factor unidimensional model) indicated a four-factor model as the best solution. Given these considerations, it was decided that the overall model fit should be considered acceptable (Harrington 2005a).

Fig. 1
figure 1

Correlations among the residual variances

Full size image

Structural Equation Modeling

To examine the associations between intolerance frustration beliefs and state and trait anxiety, depression, state and trait anger, structural equation modeling (SEM) with manifest variables was employed (Table 5), using Eqs. 6 (Bentler 1995). In model A (Fig. 2), we tested a model estimating the direct paths from the predictors (i.e., four subscales of FDS: discomfort intolerance, entitlement, emotional intolerance and achievement, which were entered simultaneously) to all the emotions variables (state anxiety, trait anxiety, depression, state anger, trait anger). Estimation of the model, χ2 (4) = 53.11, p < .001, CFI = .96, RMSEA = .07 (90 % CI = .15–.25) showed that discomfort intolerance (β = .29, p < .05) and emotional intolerance (β = .33, p < .05) related to state anxiety. Similarly, discomfort intolerance (β = .20, p < .05) and emotional intolerance (β = .40, p < .05) were directly related to trait anxiety. Discomfort intolerance (β = .33, p < .05) and emotional intolerance (β = .25, p < .05) were related to depression. Discomfort intolerance (β = .29, p < .05) and entitlement (β = .25, p < .05) related by state anger, and entitlement (β = .46, p < .05) was directly related to trait anger.

Table 5 Fit statistics for each of the models
Full size table
Fig. 2
figure 2

Model A relationship between FDS sub-scales and unhealthy emotions

Full size image

Several models based on the theoretical and empirical literature were tested to verify the relation between frustration intolerance and assertiveness. In model B (Fig. 3), we tested a model estimating the direct paths from the predictors (i.e., four subscales of FDS: discomfort intolerance, entitlement, emotional intolerance and achievement, which were entered simultaneously) to the outcome variable (distress and performance). Estimation of the model, χ2 (1) = 14,99, p < .001, CFI = .97, RMSEA = .21 (90 % CI = .12–.32) showed that emotional intolerance (β = .37, p < .05) was related to distress.

Fig. 3
figure 3

Model B relationship between FDS sub-scales and assertive behaviour

Full size image

In model C (Fig. 4), we tested a model estimating the direct paths from the predictors (i.e., FDS full scale) to the outcome variable (state anxiety, trait anxiety, depression, state anger and trait anger). Estimation of the model, χ2 (2) = .22.28, p < .000, CFI = .97, RMSEA = .18 (90 % CI = .11–.26) showed that the FDS full scale score was related to state anxiety (β = .44, p < .05), trait anxiety (β = .57, p < .05), depression (β = .46, p < .05), state anger (β = .38, p < .05) and trait anger (β = .52, p < .05).

Fig. 4
figure 4

Model C relationship between Frustration Intolerance total score and unhealthy emotions

Full size image

In model D (Fig. 5), we tested a model estimating the direct paths from the predictors (i.e., FDS full scale) to the outcome variable (distress and performance). Estimation of the model, χ2 (1) = 15,18, p = .000, CFI = .76, RMSEA = .21 (90 % CI = .11–.32) showed that the FDS full scale score (β = .40, p < .05) was related to distress.

Fig. 5
figure 5

Model D relationship between Frustration Intolerance total score and assertive behaviour

Full size image

Discussion

This present research provides evidence of the applicability of the FDS in a different culture, supporting the usefulness of an Italian version of the instrument. Consistent with our hypothesis we found support using confirmatory factor analysis for the four-factor model, and good internal reliability of the Italian FDS. The similar pattern of results obtained with the English and the Serbian language version also supports the validity of the Italian version of the FDS. Intercorrelations among the FDS sub-scales are higher than those achieved reported by Harrington (2006) using a clinical sample, but lower than those reported by Stanković and Vukosavljevic-Gvozden (2011) using a non-clinical student sample. This would be expected, in that the FDS aims to measure differences in dysfunctional beliefs, and therefore the variation in dysfunctional beliefs in a normal student sample will be reduced. The means and Standard Deviation of Italian students sample were only slightly higher than those obtained in English student group (Harrington 2005a).

Whilst some SEM indices showed lack of fit, an alternative model that sequentially incorporated correlated error variance terms into the model, did achieve a consistent fit to the data. This suggested that researchers should make full use of the strength and flexibility of SEM to assess the FDS interrelated components. Specifically, it would be useful in future studies to test two different measurement models using the FDS: (a) the original hypothesized four-factor model in which each of the 28 items loads only on its intended subscale or factor, and (b) a model that judiciously incorporates correlated error variances between items. The likelihood-ratio test might then be used to determine if there are significant differences in fit among these measurement models, and to identify which model best fits the data (Gonzalez and Griffin 2001).

Regarding gender differences, results indicate that females scored higher than males on the emotional intolerance sub-scale of the FDS, but on the other sub-scales there were no significant gender differences. These results confirm those obtained from Harrington (2006) using a clinical sample. In terms of other measures, for measures of state and trait anxiety, and depression, females reported higher scores than males. For assertive behaviour, females expressed greater discomfort (felt anxiety/distress) with being assertive in specific social situations than males. No gender differences were found in the probability of engaging in a specific assertive behaviour (performance). Whilst this results supports earlier studies that report males are more assertive than females (Eskin 2003), other research has found that females have a significantly higher score on assertive communication and independence (Bourke 2002), or that no significant gender differences exist in general assertiveness (Karagözoğlu et al. 2008). Finally, no significant gender differences were found in measures of anger, consistent with previous studies that have indicated few differences between men and women in terms of frequency or intensity of anger (Averill 1983).

The results of this study provide further evidence for the validity of FDS, and support for frustration intolerance as multidimensional concept (Harrington 2005a). The FDS dimensions were associated with different emotional and behavioural measures, supporting previous results (Harrington 2006; Stanković and Vukosavljevic-Gvozden 2011; Jibeen 2013). Achievement frustration has the weakest relationship with the various measures, although this sub-scale is significantly correlated with trait anxiety and trait anger. Discomfort intolerance was significantly correlated with depression and state anger, entitlement with trait anger, emotional intolerance with state/trait anxiety and with non-assertive behaviour (distress). In terms of anger, the entitlement sub-scale was correlated with trait anger, consistent with these previous research studies. However, both discomfort intolerance and entitlement proved to have a significant correlation with state anger, in contrast to Stanković and Vukosavljevic-Gvozden’s (2011) results, which indicated that entitlement was the only correlation with state anger.

This present research, also explored the relationship with between the FDS and measures of assertive behaviour. Lange and Jakubowski’s (1976) research suggested that non-assertiveness is associated with self-worth beliefs, such as a need for absolute approval, although an individual may have secondary problems with ‘frustration’ as a result of being non-assertive. However, in this study, we explored the question as to whether assertiveness deficits may be the result of frustration intolerance beliefs, rather than the cause of ‘frustration’.

In terms of the structural equation models, models A, B and C showed good fit, while Model D showed inadequate fit (Fig. 1). Although model B had a good fit, the only significant pathway between the four frustration intolerance sub-scales and assertiveness was that of emotional intolerance. This finding makes sense, in that the distress sub-scale is a measure of the distress involved with being assertive, and this might be expected to be associated with emotional intolerance beliefs. On the other hand, Model D (representing the predictive path between total frustration intolerance score and assertiveness) was a poor fit. This would be consistent with the position that frustration intolerance is multi-dimensional, and the sub-scales differentially related to specific problems. In other words, not all of the sub-scales would be expected to be associated with non-assertiveness. Likewise Model B (represent the predictive path of each frustration sub-scale to assertiveness), was a better fit, suggesting that the use of individual sub-scales is useful in terms of predicting behaviour. Whilst both Model A and Model C show good fits, the multidimensional model of frustration provides more information than the total score model. Whilst Model C (total frustration intolerance score) is positively related with all the negative emotional-behavioural problems, this model cannot provide information regarding specific frustration intolerance beliefs and their relationship with particular emotional/behavioural problems. On the other hand, Model A enables comparison of the strength of predictive relationships with unhealthy emotions. For example, results show that both discomfort intolerance and emotional intolerance are closely related to state anxiety (discomfort intolerance = .29; emotional intolerance = .33), and depression (discomfort intolerance = .33; emotional intolerance = .25). However, it is emotional intolerance (.40) that is the much stronger predictor of trait anxiety when compared to discomfort intolerance (.20). Similarly, whilst both discomfort intolerance (.29) and entitlement (.25) are closely related to state anger, only entitlement (.46) is a predictor of trait anger.

Although there are some contrary results to previous studies, the overall results support a multidimensional model of frustration intolerance and the relationship between specific frustration intolerance beliefs and emotional-behavioural problems. In particular, Model A showed that there were significant relationships between discomfort intolerance and depression, between entitlement and trait anger, and between emotional intolerance and anxiety. Consistent with the original research (Harrington 2006), achievement frustration was the weakest sub-scale in relation to emotional measures. Likewise, emotional intolerance was significant related to trait anxiety and state anxiety (Harrington 2006; Stanković and Vukosavljevic-Gvozden 2011; Jibeen 2013). This last result differs from that obtained Stanković and Vukosavljevic-Gvozden (2011) in which the state anxiety subscale was not predicted by any of the FDS sub-scales. Emotional intolerance was significantly related to depression, consistent with Stanković and Vukosavljevic-Gvozden’s study (2011). In this present study, entitlement had a significant relationship with trait anger consistent with previous research (Harrington 2006). The discomfort intolerance sub-scale was found to be a significant predictor of state anger, along with entitlement, whilst Stanković and Vukosavljevic-Gvozden (2011) found only entitlement as a predictor of state anger. Harrington (2006) reported that discomfort intolerance was a predictor of depression, however, Jibeen (2013) did not find discomfort intolerance to be a predictor of unhealthy emotions, and instead, found that entitlement predicted depression. In the present study, Italian students seem to react to everyday frustration with increased state anger rather than with depressed mood. State anger was not measured in the Harrington (2006) study, and it is possible that in non-clinical groups, everyday hassles may evoke feelings of transient irritation and hostility, rather than low mood.

This study also explored the relationship of frustration intolerance beliefs with measures of assertiveness. In Model B, the results supported the hypothesis that non-assertiveness may be the result of frustration intolerance beliefs. In particular, the results show that there was a significant relationship between emotional intolerance and non-assertive behavior (distress). This suggests that difficulties in tolerating negative emotions, for instance when asking someone to stop an annoying behavior (item 14 SIB-r- Subscale distress), may inhibit individuals from expressing themselves assertively. Whilst increased anxiety levels may initially be related to self-worth beliefs (e.g., fear of rejection following being assertive), secondary intolerance of anxiety may increase avoidance, consistent with Alden and Safran (1978). Thus, although research shows that individuals with social anxiety have considerable difficulty in being assertive (Rapee 1995; Davidson et al. 1993), this may not only be due to fear of social disapproval, but also intolerance of the emotions associated with being assertive.

The present findings have several implications for therapy. Assertion training procedures have long existed as an integral part of REBT and other Cognitive Behaviour Therapy approaches (DiGiuseppe et al. 2014). The present study indicated that distress in expressing assertive behaviour is associated with discomfort intolerance beliefs and emotional intolerance. This suggests that therapy needs to focus not only on improving self-acceptance beliefs, but also on those beliefs involved in general avoidance of discomfort. Other therapeutic approaches, such as Acceptance and Commitment Therapy (ACT) have also highlighted the role that experiential avoidance plays in exacerbating emotional problems and in counterproductive behaviour (Hayes et al. 1996). Likewise, Dialectical Behaviour Therapy (DBT) has pointed to the central role of distress tolerance in maladaptive behaviour (Simons and Gaher 2005). However, these approaches have focused most on emotional intolerance, rather than other forms of intolerance. The benefit of the classification of frustration intolerance beliefs into four dimensions is that it enables a more nuanced assessment of the relationships between avoidance behaviours and specific beliefs.

There are several limitations to this study. First, we did not use a measure of self-esteem. Since REBT proposes that frustration intolerance and self-worth are separate categories of belief, this would have enabled a more detailed investigation of construct validity. For instance, when comparing the degree to which frustration intolerance and self-esteem were differentially associated with specific emotions and behaviours. Self-esteem measures would also have been useful in regard to assertiveness, which has a close relationship with social anxiety and self-esteem (Rapee 1995). Second, the sample size of 250 subjects was relatively small and predominantly female, and the study would have benefitted from a larger and more heterogeneous sample. A further limitation was that it consisted of university students, and as noted earlier, the results may not be applicable to clinical groups suffering from significant emotional disorders.