Introduction

Childhood experience of an invalidating family environment has received significant research attention with regard to its association with a variety of psychopathologies, including borderline personality disorder (BPD) [1], depressive symptomatology [2], and eating disorders (ED) [3]. Linehan [4] posits that the invalidating environment, in which the communication of emotions is ignored or punished, puts children at risk of developing maladaptive behaviors, especially in the presence of a biological predisposition to emotional vulnerability. It is possible that perceived invalidation of internal experiences results in the development of emotional dysregulation and behaviors, such as substance use, cutting, binge eating, and purging, which in turn may be used as mechanisms to regulate or tolerate emotional distress [5].

The growing research concerning invalidating environments and the dialectical behavior therapy (DBT) model [4] has been supported by development of the Invalidating Childhood Environment Scale (ICES) [6], a self-report measure that retrospectively assesses the exposure to invalidation in the family. The ICES consists of 18 items based on the descriptions proposed by Linehan [4]. 14 items assess invalidating parental behaviors (e.g., ignore emotions, over-react to emotions, and overestimate problem solving), while the other 4 items reflect three types of invalidating environment (“typical”, “perfect”, and “chaotic”) and one type of validating environment. The original version of the scale [6] demonstrated higher levels of internal consistency in the eating-disordered sample (paternal invalidation alpha = 0.796; maternal invalidation alpha = 0.772) in comparison with the non-clinical sample (paternal invalidation alpha = 0.587; maternal invalidation alpha = 0.664).

To the best of our knowledge, eight studies have used the ICES [1, 3, 6,7,8,9,10,11]. So far, two previous studies have established the reliability and validity of the ICES. The validation of the French version, with college students, revealed good psychometric qualities. Factor analyses indicated a two-dimensional structure (“personal distress” and “inability of empathy”), and a good convergent validity was demonstrated by the significant correlations between the ICES scores and a measure of depressive symptoms [7]. Another study carried out with a group of American college students also showed that the ICES had high reliability (maternal invalidation alpha = 0.90; paternal invalidation alpha = 0.88) and adequate concurrent validity by correlating positively with BPD symptoms. Regarding factor structure, the authors found poor fit for the full 14-item version of the invalidation scales. However, a nine-item version revealed improved fit, which suggested the possible exclusion of five items [10].

The ICES has been used to evaluate the relationship between invalidating family environment and eating symptomatology. The authors of the original ICES found that bulimic behaviors were positively associated with perceived paternal and maternal invalidation among the non-clinical sample. Within the eating-disordered sample, only the maternal invalidation was associated with bulimic behaviors. In addition, distress intolerance partially mediated the relationship between perceived experience of an invalidating father and the development of an ED [6]. In another study, individuals with bulimia nervosa reported the highest level of paternal invalidation. In particular, paternal invalidation was related to vomiting, and a typical family in which the focus is success, achievement, and control of one’s emotions also related to exercising excessively [3].

Although demonstrating promising results, additional research is warranted to investigate the psychometric properties of the ICES in other countries, such as Portugal. Introducing an assessment tool into Portuguese research and clinical practice would contribute to the advancement of knowledge regarding the impact of invalidating childhood environments. Moreover, to the best of our knowledge, the factor structure of the scale has not yet been examined with a clinical sample.

Thus, the current study aimed to (a) examine the psychometric properties of the Portuguese version of the ICES, including the principal component analysis in a non-clinical sample, the confirmatory factor analysis in a clinical sample of ED patients, and the internal consistency for the both samples; (b) investigate the between-sample differences regarding invalidating parental behaviors and family styles; and (c) explore the associations between invalidating childhood environments and eating pathology. Consistent with previous literature, it was hypothesized that the Portuguese version of the ICES, when applied to non-clinical and clinical samples, has adequate psychometric properties. Further predictions were that there would be significant differences between non-clinical and clinical samples regarding ICES dimensions and eating attitudes and behaviors and that perceived parental invalidation would also be positively associated with eating-disordered attitudes and behaviors, especially bulimic symptoms.

Methods

Participants

A total of 511 participants completed the current study. These participants were drawn from non-clinical (n = 410) and clinical (n = 101) samples. Two groups represented the non-clinical sample: one group of high school students (n = 49) and another of college students (n = 361). The non-clinical sample was recruited from three education institutions in the north of Portugal: a public high school, a public university, and a private university. The clinical sample consisted of 101 ED patients. Participants were recruited in a specialized ED treatment setting.

Measures

The Invalidating Childhood Environment Scale (ICES)

The ICES [6] is a self-report measure that assesses childhood invalidation. It contains 14 items that examine invalidating maternal and paternal behaviors: ignore thoughts and judgements; ignore emotions; negate thoughts and judgements; negate emotions; overreact to emotions; overestimate problem solving; over-react to thoughts and judgements; and oversimplify problems [4]. Participants are asked to rate their experience up to the age of 18 years, and each item is rated on a five-point scale (“never” to “all the time”) for each parent. The mean score for the 14 items for each parent indicates the levels of perceived parental invalidation. Higher scores reflect a greater perception of emotional invalidation by one’s father and mother. The final four items address family style during childhood. Three of these are styles of an invalidating environment: “chaotic” (parents are often unavailable, and they may have substance use problems, mental health disorders or financial difficulties); “typical” (the focus of family is controlling one’s emotions, achievement and success); and “perfect” (the focus is on hiding feelings and getting on with it). One of these items include a description of an emotionally supportive environment (“validating”). The items were rated on a five-point scale (“not like my family” to “like my family all of the time”). The ICES provides a single score for each of the four family types. Higher scores indicate greater levels of a validating environment or of the three styles of invalidating environment. In the present study, a validating family (coded as one) represents a higher score in the validating family style, whereas an invalidating family (coded as zero) represents a higher score in one or more of the three styles of an invalidating environment (“typical”, “perfect” and “chaotic”).

Eating Disorder-15 (ED-15)

The ED-15 [12] is a brief self-report measure that assesses ED features over the preceding week. The ED-15 includes ten items divided into two attitudinal subscales: weight and shape concerns, and eating concerns. The total attitudinal score is the mean of the scores on all ten items. Higher scores indicate greater levels of eating pathology. The measure also includes five behavioral items (objective binges, vomiting episodes, laxative use days, exercise days and restriction days). The original measure revealed strong internal consistency and test–retest reliability [12]. In the present study, Cronbach’s alpha for the ED-15 overall attitudinal score (non-clinical sample alpha = 0.912; clinical sample alpha = 0.932) also suggested high internal consistency.

Procedure

This study was authorized and approved by the University of Minho Ethics Commission—Subcommittee of Ethics for Social and Human Sciences and the Ethics Committee São João Hospital Centre/Faculty of Medicine, University of Porto. In the non-clinical sample, all students received an invitation to participate in the study, and after obtaining their informed consent (including legal guardians), the participants completed the test battery in their classroom.

Data for clinical participants were collected as part of psychological assessment of ED patients in treatment. Participants were informed about the research aims, and data confidentiality was assured. Legal guardians and adults provided written informed consent before participating, and adolescents under 18 years of age provided assent. All participants were diagnosed by psychiatrists according to Diagnostic and Statistical Manual of Mental Disorders (DSM-V) criteria [13].

The ICES was translated and adapted from the original English version [6]. The original version of the measure was translated by the authors of this study; a fluent bilingual (English and Portuguese) psychologist then translated the Portuguese version back into English. We compared the versions and identified discrepancies were analyzed to clarify the Portuguese ICES.

Statistical analyses

Principal component analyses, using direct oblimin rotation, were performed on the non-clinical sample to examine the factor structure of the 14 items of the ICES that address maternal and paternal behaviors. The Kaiser–Meyer–Olkin (KMO) test and Bartlett’s test of sphericity were conducted to determine the adequacy of the principal component analyses. Concerning the paternal items, the number of factors to retain was indicated by eigenvalues greater than one [14]. With respect to the maternal items, data were forced into a two-factor solution, based on the two-factor structure found in the paternal items and in a previous study [7]; items with factor loadings > 0.30 were chosen [15].

Confirmatory factor analyses, using a maximum likelihood estimation method, for the ICES items were conducted on the clinical sample to evaluate the fit of the data to the factor models. The relevant fit statistics were tested (according to Bentler and Bonett [16]; Hu and Bentler [17]). Because the chi-squared test of absolute model fit is sensitive to sample size and is affected by the distribution of variables, we also turned to other fit statistics, such as the Incremental Fit Index (IFI), the Tucker–Lewis Index (TLI), the Comparative Fit Index (CFI), and the Root Mean Square Error of Approximation (RMSEA). Criteria for a good fit model were IFI, TLI, and CFI > 0.90 and RMSEA < 0.06 (< 0.07 acceptable) [18, 19].

The ICES internal consistency was examined in both samples by computing Cronbach’s alphas. To determine between-group differences, ICES and ED-15 scores for the non-clinical and clinical samples were compared using the independent samples t test (t). Chi-squared (χ2) tests were conducted to evaluate the associations between the sample (non-clinical/clinical), the family type (validating family/invalidating family—the latter includes typical, perfect or chaotic family), and the ED type (restricting-type ED, binge-eating/purging type ED). To investigate associations between the ICES and the ED-15 among the clinical sample, we performed Spearman correlation coefficients (rs). The relevant assumptions of these statistical analyses were tested. Confirmatory factor analyses were conducted using IBM® SPSS® Amos™ 24.0, and other analyses were conducted using IBM® SPSS® Statistics 24.0. P values < 0.05 were considered significant.

Results

Table 1 presents the demographic and clinical characteristics of the non-clinical and clinical samples. In the non-clinical sample, a total of 410 participants with a mean age of 19.99 years (SD = 3.19) responded to our measures. Participants were mostly female (n = 345, 84.1%) and mean body mass index (BMI) was 21.58 (SD = 2.77).

Table 1 Demographic characteristics of the non-clinical and clinical samples
Full size table

In the clinical sample, of the 101 ED patients, 59 participants (58.4%) were diagnosed with restricting-type ED (includes anorexia nervosa restricting type) and 42 (41.6%) were diagnosed with binge-eating/purging type ED (includes anorexia binge-eating/purging type, bulimia nervosa, and binge ED). Participants were mostly female (n = 93, 92.1%), ages ranged from 14 to 55 years (M = 26.21 years, SD = 10.66 years) and mean BMI was 19.20 (SD = 4.49).

Principal component analyses

The factor structure of the ICES was examined using the non-clinical sample (n = 410). With respect to the items that address paternal behaviors, the principal component analysis extracted two factors explaining 48.13% of the variance. The first factor included nine items: 1, 3, 4, 6, 7, 9, 10, 11, and 13, and the second factor included five items: 2, 5, 8, 12, and 14. For these data, the KMO was 0.91 and Barlett’s Spherical Test was significant (p < 0.001). Therefore, principal component analysis was appropriate. Concerning the items that address maternal behaviors, the same two factors were extracted explaining 44.97% of the variance. The KMO was 0.90 and Barlett’s Spherical Test was significant (p < 0.001). Thus, principal component analysis was also appropriate. The 14 items that compose these two factors and the factor loadings are presented in Table 2.

Table 2 Principal component analysis of the ICES (n = 410)
Full size table

Confirmatory factor analyses

The data from the clinical sample (n = 101) were used in the confirmatory factor analyses. Initially, a one-factor solution was fitted; however, both the paternal (14-item) and maternal (14-item) invalidation scales had poor fit—Paternal Invalidation Scale: IFI = 0.753, TLI = 0.700, CFI = 0.746, and RMSEA = 0.132; Maternal Invalidation Scale: IFI = 0.774, TLI = 0.726, CFI = 0.768, and RMSEA = 0.132. This suggested that more than one factor underlies the ICES paternal and maternal items, which is consistent with a previous study [7] and findings of the principal component analyses with the non-clinical sample. Therefore, two-factor models (Factor I includes items 1, 3, 4, 6, 7, 9, 10, 11, and 13; Factor II includes items 2, 5, 8, 12, and 14) were fitted for father and mother. As seen in Table 3, this solution showed an improved fit—Father: IFI = 0.970, TLI = 0.962, CFI = 0.969 and RMSEA = 0.047; Mother: IFI = 0.955, TLI = 0.935, CFI = 0.953 and RMSEA = 0.065.

Table 3 Model fit statistics for confirmatory factor analyses of the ICES (n = 101)
Full size table

Internal consistency

Cronbach’s alphas of the ICES factors were calculated in both non-clinical and clinical samples for each parent. As shown in Table 4, reliability analyses showed good internal consistency for both the non-clinical sample (Father: factor I alpha = 0.796 and factor II alpha = 0.845; Mother: factor I alpha = 0.780 and factor II alpha = 0.792) and the clinical sample (Father: factor I alpha = 0.873 and factor II alpha = 0.797; Mother: factor I alpha = 0.864 and factor II alpha = 0.790).

Table 4 Internal consistency of the ICES in both non-clinical and clinical samples
Full size table

Cronbach’s alphas of the original ICES scales were also calculated. In the non-clinical sample, paternal invalidation and maternal invalidation scales had good levels of internal consistency (paternal invalidation alpha = 0.859; maternal invalidation alpha = 0.845). In the clinical sample, reliability analyses also showed good internal consistency for both the paternal invalidation (Cronbach’s alpha = 0.862) and the maternal invalidation (Cronbach’s alpha = 0.885).

Differences between samples in invalidating childhood environments

Table 5 presents mean scores on the ICES and ED-15 for the non-clinical and clinical samples. There were significant differences between the samples with respect to the ICES scores. Participants from the clinical sample perceived higher paternal [t (509) = 2.94, p = 0.003] and maternal invalidation [t (509) = 2.55, p = 0.011] than the non-clinical sample. ED participants had significantly higher scores in the chaotic family [t (506) = 6.31, p < 0.001] and in the perfect family style [t (508) = 4.91, p < 0.001], and lower scores in the validating family style [t (505) = − 3.89, p < 0.001]. There were no significant differences between the two samples in the typical family style [t (507) = 1.20, p = 0.229].

Table 5 Mean scores on the ICES and ED-15 for the non-clinical and clinical samples
Full size table

Within the clinical sample, more participants perceived an invalidating family [χ2(1) = 13.06, p < 0.001] in comparison with the non-clinical sample. The family type (validating/invalidating family) was not significantly associated with ED type [χ2(1) = 3.25, p = 0.071].

Associations between invalidating childhood environments and eating pathology

According to previous results about the pattern of differences between samples, with the clinical sample scoring higher on the ICES and ED-15, the correlations between both measures, among the ED participants, showed that paternal invalidation was not associated with ED-15 scores. On the other hand, maternal invalidation was not associated with ED-15 attitudinal scales but was significantly associated with objective binges (rs = 0.278, p = 0.005), vomiting episodes (rs = 0.204, p = 0.043) and restriction days (rs = 0.217, p = 0.021). The chaotic family style was associated with laxative use days (rs = 0.232, p = 0.031), and the perfect family style was associated with objective binges (rs = 0.231, p = 0.021). The results of correlation analyses are presented in Table 6.

Table 6 Associations between invalidating childhood environments and eating pathology among the clinical sample
Full size table

Discussion

The first aim of this study was to examine the psychometric properties of the Portuguese version of the ICES in both a non-clinical and a clinical sample of ED patients.

Previously, to the best of our knowledge, only two studies examined the factor structure of the ICES using non-clinical samples [7, 10]. In contrast to the one-factor (14-item) version of the paternal and maternal invalidation scales of the original ICES [6], our principal component analyses with a non-clinical sample provided support for a two-factor solution. Confirmatory factor analyses with a clinical sample of ED patients also showed that the two-factor model found provided a better fit to the data than a one-factor solution. Our results are in accordance with both previous studies to which we referred. That is, in both, the authors established the presence of one factor composed by the same nine items as our Factor I [7, 10]. The other five items were removed in the work by Robertson and colleagues [10], who proposed an abbreviated version of the ICES (ICES-9), or like in Compagnone and Lo Monaco [7], included a second factor, Factor II.

We believe that our results reinforce Linehan’s conceptual model [4]. Factor I includes nine items that characterize an invalidating family environment precisely around the three primary features proposed by Linehan [5]: rejection of the communication of private experiences and self-generated behaviors; punishment of emotional displays and intermittent reinforcement of emotional escalation; and oversimplification of problem solving and goal attainment. On the other hand, Factor II includes exactly four items that represent parents’ emotional and supportive behavior. The only item that does not fit with this explanation is item two, which assesses anxiety, an emotion that is frequent from childhood into adolescence [20] and that may be associated with the need of some parents to ignore anxiety in a daily routine to help the child break the threatening interpretations of harmless situations.

In the present study, the original maternal and paternal invalidation scales, as well as the two factors for father and mother, extracted in the principal component analyses were found to have good internal consistency across the samples. Considering these results and those of previous studies [3, 10], the ICES has demonstrated good internal consistency in both clinical and non-clinical samples.

A secondary aim of this study was to investigate the between-sample differences regarding invalidating parental behaviors and family styles. In line with our hypotheses and the evidence of links between family environment and eating pathology [21], scores on the ICES were found to differentiate between samples. Perceived maternal and paternal invalidation, and chaotic and perfect invalidating family styles, were significantly higher for participants in the clinical sample than in the non-clinical sample.

Consistent with the study of Mountford and colleagues [6], only maternal invalidation was associated with a higher eating pathology among the clinical group. More specifically, there were positive correlations between the maternal invalidation and binge eating, vomiting episodes and restriction days. The chaotic family style was positively related to laxative use days, and the perfect family style was related to objective binges. In accordance with prior research [3], while invalidation was associated with some bulimic behaviors, no significant associations were found between perceived invalidation and eating attitudes (weight and shape concerns and eating concerns). These findings support the hypothesis that an invalidating family environment might contribute to the development of eating pathology, especially bulimic behaviors [3, 6, 8]. According to Linehan and Dexter-Mazza [5], invalidating environments may contribute to deficits in emotion regulation, which in turn increase the likelihood of engaging in behaviors, such as binge eating and vomiting, that regulate or avoid negative emotions. Therefore, because of its emphasis on regulating emotions and using healthier coping mechanisms, Linehan’s DBT model [4] can be an effective treatment for ED patients.

Considering the fact that previous studies examining the factor structure of the ICES were carried out with non-clinical samples, this study has addressed a gap in the research literature by directing attention to the psychometric properties for the Portuguese version of the ICES using a large community sample as well as a clinical sample of ED patients. However, this study has some limitations. First, the current sample was composed mostly of women, and the findings cannot be generalized to men. Second, the ED participants were in treatment and therefore represent only part of the general population with ED. Moreover, the smaller size of the clinical sample limits our knowledge about the structure of the ICES with ED patients. Third, recall biases may have influenced retrospective assessments of invalidating parental behaviors and family styles. Fourth, the presence of a diagnosis of ED in the non-clinical sample was not rigorously assessed. Therefore, it is not possible to infer that the non-clinical sample does not include subjects with ED. In addition, the between-sample differences regarding invalidating parental behaviors and family styles should be interpreted with caution, since the mean age of participants was different across the two samples in the analysis. Finally, the cross-sectional design cannot address the causal relationships between invalidating childhood environments and ED.

Future studies should consider using samples of similar age and equal proportions of men and women to examine whether there are age and gender differences in invalidating parental behaviors and family styles. Prospective research on the role of the invalidation in the development of ED symptomatology is also needed. Finally, investigating the factor structure of the ICES among samples from different countries could be interesting.

In sum, the findings from the current study suggest that the Portuguese version of ICES has adequate psychometric properties. Thus, the Portuguese version of the ICES seems to be a useful measure of invalidating childhood environments, especially in ED settings.