Introduction

Interest in the assessment of health-related quality of life in people with eating disorders has increased over recent years [1], and measures of functional impairment secondary to eating disorder symptoms have been recognised as relevant to both baseline assessment [2] and clinical outcome [3, 4].

Several eating disorder-specific tools for assessing health-related quality of life are available [5,6,7,8], but none is completely adequate for assessing clinical impairment related to eating disorder psychopathology. Indeed, as underlined by Bohn et al. [4], these tools either do not focus completely on the impairment secondary to eating disorder psychopathology; fail to assess the impact of concerns specific to people with the eating disorders, such as extreme concerns about shape and weight; or are completely lacking in validity.

Nevertheless, the Clinical Impairment Assessment (CIA), a 16-item self-report tool, has been designed to overcome the above limitations, specifically to: (1) assess the impact of the full range of eating disorder psychopathology on clinical impairment and (2) enquire about the specific aspects of life that are affected by eating disorder psychopathology [4].

Several studies have provided data on the use of the CIA and its translated versions in different populations. In particular, the CIA has demonstrated good psychometric proprieties in a UK clinical sample [9] and in women at high risk of eating disorder onset [10]. Moreover, the CIA has been adapted, translated, and validated in female Persian non-graduates [11], in Spanish patients with eating disorders [12], in a Swedish population [13], in Norwegian students [14], and in Fijian adolescents [15]. An additional study has provided normative data from a Norway clinical sample [16].

Moreover, the factor structure of the tool has been investigated using confirmatory factor analysis in clinical samples with eating disorders [4, 9, 12], and using exploratory factor analysis in a non-clinical sample [14]. Results from these studies support the validity of the original factor structure of the measure, and all, except Jenkins et al. [9], tested and confirmed the three-factor structure of the CIA (“Personal”, “Social”, and “Cognitive” impairment) investigated in the original study [4].

To our knowledge, no previous studies have validated an Italian-language version of the CIA. Hence, due to its potential utility in clinical and research fields and the good reliability and validity demonstrated in several studies, we set out to validate a translated version of the CIA in a large Italian population, assessing several psychometric properties of the questionnaire in a clinical sample with eating disorders.

Methods

Participants and procedure

The study sample comprised 259 patients consecutively recruited by the Villa Garda Hospital Eating Disorder Clinic (Italy) between January 2015 and March 2017. Patients were aged between 16 and 60 years and met the DSM-5 diagnostic criteria for eating disorders [17]; they had been referred from all over Italy by general practitioners and eating disorder specialists. The DSM-5 diagnosis and the presence of comorbid conditions were carried out by means of a clinical interview conducted by experts in the field. Seventy-eight members of the sample (30.1%) were outpatients and 181 (69.9%) inpatients. Indications for inpatient admission were the presence of an eating disorder that could not be managed safely on an outpatient basis, or previous outpatient treatment failure(s). Exclusion criteria were daily substance abuse, acute psychotic disorders, and/or pregnancy.

One-hundred and two healthy controls (100 females and 2 males) with a body mass index (BMI) ≥ 18.5 and < 30 kg/m2 (mean self-reported BMI = 21.6 ± 2.3) aged between 18 and 50 years (mean 31.7 ± 11.4) were also recruited from the general population in various community settings. Subjects were excluded from the control group if they scored higher or equal to 20 on the Italian version of the Eating Attitudes Test-26 (EAT-26) [18, 19] and/or there was a suspicion or diagnosis of eating disorder, i.e., an affirmative answer to one or both following questions: “Do you have an eating disorder?” and/or “Do you attend a treatment for eating disorders?” (N = 3).

The ethics committee of the Local Health Unit 22-Bussolengo (Verona) approved the study (Study Protocol No. USL22#03/12-CEP329) and all participants gave informed written consent for the use of their anonymous personal data. For those under the age of 18, additional informed consent was provided by their parents.

Assessment and measures

Demographic and clinical variables

All data from the clinical sample were collected before the treatment was begun. Demographic and clinical variables were obtained by direct interview. Weight was measured on calibrated scales and height using a stadiometer. The BMI was then calculated via the standard formula—body weight in kilograms divided by height in meters squared. Patients were weighed and measured wearing underwear but no shoes.

The Eating Disorder Examination Questionnaire (EDE-Q)

The Italian validated version of the EDE-Q 6.0 self-report questionnaire was used to assess eating disorder psychopathology and behaviours [20, 21]. The EDE-Q also provides frequency data on key behavioural features of eating disorders, i.e., the number of episodes of the behaviour over the preceding 28 days and the number of days on which the behaviour has occurred. The questionnaire is made up of 22 items, rated on a seven-point forced-choice format (0–6), with higher scores reflecting greater severity or frequency. Items are grouped into subscales (Restrain, Eating Concern, Weight Concern, Shape Concern), whose scores reflect the severity of key psychopathological features of eating disorder, and the mean score of the four subscales is taken as the global score. The internal consistency in our sample was high both for the global score (Cronbach’s alpha = 0.94) and the subscales (Cronbach’s alpha Restrain = 0.87; Eating Concern = 0.78; Weight Concern = 0.78; Shape Concern = 0.88).

The Brief Symptom Inventory (BSI)

The Italian version of the Brief Symptom Inventory [22, 23], a short version of the Symptom Checklist-90 [24], was used to measure general psychopathology. The Global Severity Index (GSI) was calculated from scores assigned to patients’ responses. Cronbach’s alpha = 0.97.

The Clinical Impairment Assessment (CIA)

The CIA is a 16-item self-report measure of psychosocial impairment secondary to features of an eating disorder [4]. Items are rated on a four-point Likert scale, ranging from 0 = “Not at all” to 3 = “A lot.” A global CIA score ranging from 0 to 48 is calculated to provide a global index of the severity of psychosocial impairment due to eating disorder psychopathology over the preceding 28 days. Higher scores indicate greater impairment. Signal detection analysis on the original study revealed that a score of 16 on the CIA was a good predictor of eating disorder case status, with a sensitivity of 76% and a specificity of 86% [4]. A subsequent study conducted in clinical and community samples confirmed this cutoff [25].

The questionnaire includes three subdomains of impairment, personal, social, and cognitive, that can be used to investigate specific areas of impairment.

The Italian version was translated from the original English version by experts in the field. The translation process was conducted as follows [26]: (1) forward translation into Italian by a bilingual person; (2) blind back-translation into English by a bilingual person; (3) discussion of items by the investigator team to identify any discrepancies and to adjust any inconsistencies; and (4) final approval by the investigators.

All patients were administered the questionnaire before starting treatment—outpatients during their first treatment session assessment interview and inpatients on their first day of admission to the Unit (Time 1). However, a randomly selected subgroup of 56 patients (30 inpatients and 26 outpatients) had also been administered the questionnaire during the pre-treatment assessment interview, conducted 1–3 weeks before admission (Time 0), to assess the test–retest reliability of the tool. Treating physicians were blind to these patients’ pre-admission CIA scores.

The Eating Attitudes Test-26 (EAT-26)

The Italian validated version of EAT-26 was used as a screening tool for eating disorder [18, 19]. It furnishes a total score ranging from 0 to 78, and a score equal to or higher than 20 indicates that a subject may be at risk of an eating disorder. EAT-26 was, therefore, administered to the control group to exclude any subjects with a total score of 20 or above. Cronbach’s alpha in the non-clinical sample is 0.85.

Data analysis

Confirmatory Factor Analysis (CFA) was performed using MPlus (version 7; Los Angeles, CA) to evaluate whether the data fit the three-factor model obtained in the original CIA study [4]. Model estimation was based on robust-weighted least squares estimator. Goodness-of-fit of the relevant model was evaluated on the basis of recommended standards [27,28,29]: relative χ2 (χ2/df, criterion: 2–5) comparative fit index (CFI, criterion > 0.900), Tucker–Lewis index (TLI, criterion > 0.900), root-mean-square error of approximation (RMSEA, criterion < 0.060), and weighted root-mean-square residual (WRMR, criterion < 1.00).

Internal consistency reliability was assessed using Cronbach’s alpha and McDonald’s omega coefficients [30, 31].

Kolmogorov–Smirnov and Shapiro–Wilk tests were performed to test the normality of the distributions and indicated non-normal distributions (p < 0.05).

Spearman’s product–moment correlation was used to assess test–retest reliability in the random subgroup of 56 patients who repeated the CIA at Time 1. Moreover, an intraclass coefficient (two-way random consistency) between global CIA scores at T0 and T1 was reported.

To assess convergent validity, Spearman’s correlation coefficients were used to evaluate the association between CIA scores and EDE-Q, BSI, and EAT-26 scores.

As a test of known-group validity, the Mann–Whitney U test was used to compare CIA scores reported by patients with eating disorders with those reported by healthy controls. In the clinical sample, global EDE-Q and BSI scores were compared between those scoring above and below the suggested CIA cut-off score of 16. A receiver operating characteristic (ROC) curve analysis was also performed to evaluate the ability of the global CIA to discriminate patients from healthy controls. We calculated the area under the curve and determined the cutoff with better sensitivity and specificity. Moreover, global CIA and subscale scores were compared between groups of patients reporting behavioural expressions of eating disorders and not (as assessed by EDE-Q), between underweight and not-underweight patients (BMI < vs. ≥ 18.5) and between in- and outpatients.

The two patients who did not complete the EDE-Q and the three patients who did not complete the BSI were excluded from the analysis. All items of the CIA in the clinical and healthy control sample were completed.

Results

Table 1 shows demographic and clinical characteristics of patients with eating disorders. The majority of the 259 patients were single, female and met the criteria for anorexia nervosa.

Table 1 Demographic and baseline clinical characteristics of patients with eating disorders (n = 259)
Full size table

Confirmatory factor analysis

The overall mean score of the CIA was 32.4 (SD = 11.5, range = 0–48). Table 2 shows mean, median, standard deviation, skewness, and kurtosis of each item of the CIA.

Table 2 Mean, median, standard deviation (SD), skewness, and kurtosis of each item of the Clinical Impairment Assessment (CIA) in the clinical sample (n = 259)
Full size table

Table 3 shows factor loading of the CIA items. Fit indices (Table 3) indicated a satisfactory model fit, with a finding of significance for the relative χ2 (χ2 = 221.072, df = 101, p < 0.001). Factor loadings were all significant, ranging from 0.69 to 0.93.

Table 3 Confirmatory factor analysis (model estimation was based on robust-weighted least squares estimator, Italic) of Clinical Impairment Assessment (CIA): factor loadings and fit indices (n = 259)
Full size table

Internal consistency

Internal consistency was similar as measured by the Cronbach’s alpha and by the McDonald’s omega and was 0.94 for the global CIA score, and 0.89, 0.89, and 0.84 for the Personal, Social, and Cognitive impairment factors, respectively.

Test–retest reliability

The CIA was administered to 56 patients at Time 0 (initial assessment), and again 7–24 days (mean 13.5 days) later (Time 1, admission). The mean global CIA score at T0 was 32.7 (SD = 11.1) and at T1 31.3 (SD = 13.1; p = 0.212), indicating the good temporal stability of the tool.

Spearman’s correlation coefficient was 0.74 for the global CIA score, 0.65 for the personal impairment factor, and 0.77 for the social and cognitive impairment factors, respectively. Moreover, the intraclass correlation coefficient was 0.86 (95% CI 0.76–0.92). These data confirm a satisfactory level of test–retest reliability.

Convergent validity

Table 4 shows Spearman’s correlation coefficients between CIA scores and EDE-Q and BSI scores. All correlations were highly significant, in particular global CIA score, and global EDE-Q and BSI scores displayed correlation coefficients of 0.72 and 0.76, respectively.

Table 4 Spearman’s correlation coefficients
Full size table

In the healthy control sample, the correlations between EAT-26 and CIA subscale scores were all significant at p < 0.001 (Personal Impairment rho = 0.38; Social Impairment rho = 0.43; Cognitive Impairment rho = 0.27).

Known-groups validity

A comparison of global CIA and subscale scores between the clinical sample and healthy controls indicated significant differences between the two groups (Table 5). Similar results were obtained when controlling the analysis for age.

Table 5 Clinical Impairment Assessment (CIA) scores in patients with eating disorder and healthy controls
Full size table

Moreover, the comparison of the global EDE-Q and BSI scores between patients that scored above and below the CIA cutoff of 16 showed a mean global EDE-Q score of 3.99 (SD = 1.23) and global BSI of 2.08 (SD = 0.80) for those scoring above, in comparison with 1.23 (SD = 0.95) and 0.68 (SD = 0.42), respectively, for those scoring below (z = − 7.85, p < 0.001; z = 7.71, p < 0.001). The area under the curve of the ROC curve analysis was excellent (0.98, 95% CI 0.92–0.99) and the optimal cut-off scores discriminating patients with eating disorders from healthy controls are 16 (sensitivity = 90%, specificity = 97%), confirming the original cut-off score.

In addition, global CIA and subscale scores were significantly higher in patients who reported objective bulimic episodes, self-induced vomiting, laxative misuse and excessive exercising than in those who did not report these behavioural expressions of eating disorder psychopathology. The only exception was Social impairment, which was similar in patients with and without objective bulimic episodes (Table 6).

Table 6 Global Clinical Impairment Assessment (CIA) and subscale scores in subgroups with and without behavioural expressions of the eating disorder psychopathology, as assessed by Eating Disorder Examination Questionnaire (EDE-Q)
Full size table

Underweight patients showed significantly higher global CIA scores than not-underweight patients (32.9 ± 11.7 vs. 30.0 ± 10.1, z = 2.03, p = 0.042). CIA subscale scores indicated that the difference was significant for social and cognitive impairment (p < 0.001 and p = 0.025, respectively), but not for Personal impairment (p = 0.363).

Finally, inpatients obtained significant higher global CIA and subscale scores than outpatients (global CIA score: 35.0 ± 10.3 vs. 26.3 ± 11.8, z = 5.58, p < 0.001; personal impairment: p = 0.014, social impairment: p < 0.001; cognitive impairment: p < 0.001).

Discussion

The aim of this study was to propose an Italian translation of the CIA, and to test its internal consistency, short-term test–retest reliability, and factor structure in a real-world setting. Results from our large sample of inpatients and outpatients referred for eating disorder treatment by clinical services confirmed the factor structure proposed by Bohn et al. [4] as well as the strong reliability and validity of the tool.

First and foremost, the three-factor structure of the CIA was confirmed, supporting the findings from both the original study [4] and one assessing the Spanish version of the questionnaire [12] in a clinical sample. Our item loadings were in line with the study by Bohn et al. [4], except for the items “interfered with your relationship with others” of the social impairment factor and “made it difficult to concentrate” of the cognitive impairment factor, which were both higher in our sample. This difference, albeit small, could be attributed to the higher severity of eating disorder psychopathology in our sample, which included a subgroup of hospitalised patients.

The Italian version of the CIA showed very high internal consistency for both the global and its three subscale scores, comparable with those observed in other studies on clinical samples [9, 12]. The test–retest reliability of global CIA and subscale scores was acceptable, and to our knowledge, this is the first demonstration of the temporal stability of the CIA in a clinical sample. Indeed, the previous studies have only assessed the test–retest reliability of the questionnaire in non-clinical samples [4, 14].

Furthermore, the high correlation coefficients between CIA and both the EDE-Q and the BSI in the clinical sample and those found between CIA and EAT-26 in the healthy control sample confirm the good convergent validity of the instrument. Moreover, a cut-off CIA global score of 16, confirmed in our sample with the ROC curve analysis, permitted discrimination between patients with higher eating disorder and general psychopathology scores, confirming the cutoff suggested by its authors [4]. The Italian version of CIA also effectively distinguished between patients with eating disorder and healthy controls. Indeed, CIA subscale scores were highly related to global and subscale EDE-Q scores, and global BSI score confirming the close relationship between psychosocial impairment secondary to eating disorder and the severity of eating disorder and general psychopathology.

Our data also support the known-groups validity of the CIA, showing higher scores for psychosocial impairment in underweight patients and those with objective bulimic episodes, self-induced vomiting, laxative misuse, and excessive exercising than in patients without these behavioural expressions of eating disorder psychopathology. This finding is partially in line with those of the previous studies, which found significantly higher global CIA scores in patients with bulimic episodes or excessive exercising than those without, but failed to find a significant relationship between global CIA scores and low weight or the presence of vomiting or laxative misuse [9, 15]. This discrepancy could be related to the different characteristics of the samples considered (a non-clinical sample in Becker et al. [15] and a higher percentage of patients with bulimia nervosa and other eating disorders in Jenkins [9]).

Interestingly, we also found that the presence of objective bulimic episodes did not affect social impairment and that to be underweight was associated with social and cognitive impairment but not personal impairment–a finding that warrants further research. Another novel finding of our study is that inpatients obtained significantly higher global CIA and subscale scores than outpatients. This supports the known-groups validity of the questionnaire, and its association with the severity of psychopathology.

This study has three main strengths. First, the large sample of inpatients and outpatients with eating disorders of clinical severity we considered permits our results to be generalised to the Italian eating disorder population. Second, our assessment of both the temporal stability of the CIA in a clinical sample and the differences between in- and outpatients demonstrates that it has important but previously unknown psychometric proprieties. We also evaluated CIA subscale scores in different subgroups of patients, and validated the CIA with the primary eating disorder and general psychopathology questionnaires used in clinical practice and research into eating disorders.

The main limitation of this study, however, was that we did not use more objective measures of clinical impairment, as a clinical interview; this limited our ability to objectively assess both impairment and construct validity. Moreover, the small sample size of subjects with BED diagnosis and the scant presence of males do not permit to generalize the results to the overall population of subjects with eating disorders.

Nevertheless, our results strongly suggest that the Italian version of the CIA is a valid and reliable tool for assessing the psychosocial impairment that directly stems from eating disorder psychopathology in clinical samples.