One of the main consequences of illicit drug and alcohol use is cognitive dysfunction due to people’s disease awareness deterioration. This deteriorated or altered perception must be regarded as a continuum ranging from complete denial of the disease, when people underestimate the multiple everyday consequences of substance use (David et al. 2012; Lincoln et al. 2011), autobiographical memory deficits (Poncin et al. 2015), to the full and conscious acknowledgment that one is suffering from a chronic disease (Volkow and Li 2005).

In addition, functional neuroimaging has revealed the existence of certain brain regions which are linked to components of disease awareness; for instance, the insula is related to interoceptive and emotional awareness, the anterior cingulate cortex to decision-making processes (Goldstein et al. 2009), and the ventromedial frontal cortex to the ability to attribute personal relevance to salient stimuli (Moeller and Goldstein 2014).

Lower awareness of potentially high-risk situations or their consequences is associated with increased susceptibility to automatic drug-seeking processes and a lower likelihood of employing adequate and effective coping strategies that make it possible to pay conscious attention to one’s desire to use substances (Garland et al. 2010). In this regard, a person’s thoughts about his/her disease play a major role in his/her way of responding to risk situations. A person who is able to perceive that his/her behavior will lead to results regarded as negative will tend to implement changes; however, inability to avoid or resist risk situations despite wishing to do so will result in cognitive dissonance characterized by deep dissatisfaction, which will eventually modify healthy beliefs instead of risk behaviors. In the same vein, Bitarello do Amaral et al. (2006) propose 2 main belief types that characterize addicted patients: risk-minimizing and functional. The former involve either underestimating the likelihood of experiencing negative effects or downplaying the importance of the negative or undesirable consequences of a given behavior (e.g., “harmful effects or consequences associated with substance use do not apply to me”) (Borland et al. 2009); in contrast, functional beliefs involve thinking that it is beneficial to engage in certain behaviors regarded as positive (e.g., “alcohol can lower stress levels in people like me”).

In patients with substance use disorders, recovery rates increase as they manage to develop a higher level of awareness of their disease, which translates into a greater subjective feeling of happiness, a decrease in symptom intensity, and a positive change in their interpersonal relationships (Carter 2015; Copello and Orford 2002; Lee and Bowen 2014; Levola et al. 2014). However, this recovery also involves implementing several lifestyle changes essentially aimed at transforming one’s identity, regarded as a multidimensional, fluid, and context-dependent construct (Tracy and Trethewey 2005). Such a transformation means that the person must be able to cast aside his/her identity as a “drinker” while at the same time internalizing a new and stable “non-drinker” identity of which he/she is fully aware (Best et al. 2015; Buckingham et al. 2013; Chambers et al. 2017; Hill and Leeming 2014; Laudet 2007).

These lifestyle changes are associated with greater patient self-agency, that is, more self-efficacy regarding one’s ability to predict risky situations and reduce the likelihood of relapsing, considering that people with more disease awareness tend to repeat certain behaviors when they feel sure of their ability to make good decisions (Sakiyama et al. 2012). This stands in contrast with the concept that Bechara (2005) labeled future myopia, which precisely refers to an individual's inability to foresee certain situations and learn from his/her mistakes. In this regard, patients with strongly positive expectations with respect to their substance use behavior (e.g., “I’ll be the soul of the party again when I'm able to have a few beers”) are expected to achieve poor therapeutic outcomes, unlike those with strongly negative expectations (e.g., “I know I’ll feel awful if I use even a small amount”), precisely because they will be able to anticipate the negative consequences of their decision to use (Sawayama et al. 2012).

In this regard, the type of expectations that patients have could promote healthier behaviors associated with their commitment to treatment and their level of satisfaction with the results obtained. Patients’ ability to anticipate risk situations, based on greater awareness of their disease, could ensure better adherence to treatment if they manage to achieve more desirable results by themselves. For this reason, the present study is aimed at constructing and validating an efficient diagnostic questionnaire capable of detecting an individual’s awareness of his/her harmful use or chemical dependency at an initial screening level, especially in primary health care. In addition, this questionnaire is expected to allow therapists who specialize in substance use disorders (SUD) to monitor their patients’ psychotherapeutic processes and outcomes. The psychometric properties of the ICE-A were analyzed in order to (a) explore the unidimensionality of the questionnaire through an exploratory factor analysis and item-total correlations; (b) calculate its temporal and internal reliability; (c) calculate its discriminant validity with respect to other constructs (self-perceived risks of alcohol use and self-perceived psychological functioning); and (d) determine its sensitivity to change during treatment.

Materials and Methods

Participants and Procedure

The sample was collected in public and private mental health servicesFootnote 1. It consisted of 124 patients who were receiving treatment due to a dependency disorder (alcohol or drugs). The participants were 39.0 years old on average (SD = 14.35, Q2 = 27.0, Q3 = 51.0) and 74.2% were men.

All the subjects who participated in the study displayed a substance use disorder according to DSM-5 (American Psychiatric Association 2013). The subsample of patients included in the follow-up phase were participating in a comprehensive rehabilitation program based on a total abstinence model with a multifaceted approach that includes individual therapy sessions, family therapy, and or/couples therapy, as well as psychiatric checkups when indicated (intensive outpatient treatment).

Subjects were invited to participate in the study considering all due ethical safeguards. This study was certified by the Science Ethics Board of the Metropolitan Health Care Service - East Branch (Chile). Evaluations were performed at various points of the process: at baseline and at 1, 3, 6, 12, and 24 months after the start of the treatment.

Instruments

The Inventory of Disease Awareness for Addicted Patients (ICE-A) was constructed based on the model for generating psychological instruments proposed by Pasquali (2010), which includes the following three sequential phases: (a) operationalization of the “disease awareness” construct and production of adequately worded items (theoretical phase); (b) construction and administration of the questionnaire (experimental phase); and (c) use of statistical analysis to calculate its validity and reliability (analytical phase).

Theoretical Phase: Operationalization of the Construct

The following three actions were performed: (a) a review of the recent literature on the construct studied; (b) a national and global review of the questionnaires developed to date; and (c) individual and group interviews to explore the meanings associated with the construct, from the perspective of addiction therapists.

Literature review

Most of the sources reviewed refer to disease awareness in schizophrenic patients, noting their low treatment adherence resulting from their difficulties realizing that it is their very disease that prevents them from regarding themselves as disease sufferers. Nevertheless, it can be hypothesized that addicted patients who seek treatment are those with a minimum level of disease awareness, as long as their cognitive deterioration is not overly severe.

Instruments review

Most of the questionnaires found were designed to evaluate aspects of substance abuse other than addicted patients’ self-reported perceptions of their own disease.

The Beck Cognitive Insight Scale (Beck et al. 2004) makes it possible to measure how patients evaluate their own judgment. It comprises two dimensions: self-reflectiveness (9 items) and self-certainty (6 items). However, it is a questionnaire developed to assess cognitive deterioration in patients with severe mental pathologies, specifically schizophrenia. None of the scales reviewed were developed considering a theory aimed at explaining subjects’ awareness that addiction is a disease.

Interviews

Two interview sessions were held with a group of 13 addiction experts (3 psychiatrists and 10 psychologists) with more than 5 years of experience (Sá and Del Prette 2016). The participants were able to talk about the questions presented by the moderator, who used an interview guideline with the following topics: meaning of the disease awareness construct, clinical indicators of disease awareness, main defenses of the addicted patient, aspects of patients’ life history connected with disease awareness, the link between disease awareness and treatment adherence, and consequences of addiction, among others. Both interview sessions were audio recorded and then transcribed for content analysis.

Experimental Phase: Construction and Administration

A content analysis (Bardin 1996; Mayring 2000; Krippendorff 1990) was conducted to infer the various representations held by the therapists regarding the basic components of the disease awareness construct (Piñuel 2002). Each interview was individually analyzed by two coders with qualitative methodology expertise (Pasquali 2010). Our intention was not to perform a content analysis or a linguistic analysis, because there were no pre-established categories to guide such qualitative approaches, just as the focus was not on the semantic characteristics of therapists’ speech. Instead, this phase yielded a preliminary list of categories using an inductive logic, which were later transformed into potential items for the questionnaire. This preliminary list was presented to the same group of therapists to enrich the items through consensual discussion (Krippendorff 1990).

In parallel, the therapists (a) assigned a level of importance to each item (from not important to very important), in order to ensure the inclusion of the most relevant ones in the preliminary version of the questionnaire, and (b) reviewed the wording of the items (semantic analysis).

This second phase resulted in a list of 97 items related to the following topics: (a) recognition that one is suffering from a dependency-related disorder; (b) recognition that one needs treatment; (c) awareness of the negative consequences of addiction; (d) awareness of one’s past and present symptoms; (e) identification of one’s desire to use substances; (f) defense mechanisms; (g) connection between addiction and one’s life history; (h) acknowledgment of one’s limitations; and (i) attribution of the improvements achieved.

Analytic Phase: Administration of the Questionnaire

The first step of this stage involved making decisions about some practical aspects related to the administration of the questionnaire: precise instructions, answer format, and graphic design. Answer choices were established according to a 5-point unipolar Likert scale (agree–disagree; 1 = strongly agree, 2 = agree, 3 = undecided, 4 = disagree, and 5 = strongly disagree).

Piloting

Two pilot studies were conducted before the final validation of the questionnaire. The first of them was conducted with a total of 31 addicted patients at various stages of their treatment (74.19% men) who were 35.46 years old on average (SD = 12.50). The questionnaire used was composed of 97 items and administered individually. An exploratory factor analysis was conducted calculating the item-total correlations in order to reduce the number of items, as only those whose factor loadings or item-total correlations surpassed .40 were kept.

The second pilot study was conducted with a sample of 69 addicted patients (69.9% men) at various stages of their treatment. The participants were 37.14 years old on average (SD = 15.38). On this occasion, a version of the questionnaire was administered which contained the 39 items with the highest factor loadings as identified through the previous study. The number of items was again reduced, which resulted in a final 20-item version of the questionnaire.

Alcohol Use Disorders Identification Test

Alcohol Use Disorders Identification Test (AUDIT) is a self-report questionnaire developed by the World Health Organization (WHO) for screening excessive alcohol use (Babor et al. 2001). It is composed of 10 items grouped into 3 dimensions: hazardous alcohol use (items 1 to 3), dependence symptoms (items 4 to 6), and harmful alcohol use (items 7 to 10). In general, the overall score can range from 0 to 40 points; therefore, a person who scores 9 points or more is self-reporting harmful use or dependence.

Outcome Questionnaire (OQ-45.2)

Outcome Questionnaire (OQ-45.2) is a self-report questionnaire developed by Lambert et al. (1996) that has been validated for use in Chile (Von Bergen and de la Parra 2002). It is composed of 45 items grouped into the following dimensions: symptom distress, interpersonal relations, and social role. High overall scores reflect high levels of dissatisfaction with one’s quality of life, with 73 points being the cutoff score for Chile. This score is used to distinguish a functional population (< 73) from a dysfunctional one (> 73).

Data Processing

Missing Data Management

The percentage of missing values per variable ranged from 0 to 2.2%. The variables sex and months of treatment, as well as items 7 and 9, displayed no missing data. The case elimination method would have yielded only 195 measurements (82.3 %) for analysis. However, 24 combinations of variables with missing values were found. The percentage of cases in which each combination appeared ranged from 0.4 to 2.1%.

Little’s MCAR test (1988) provided evidence for rejecting the assumption that the loss mechanism was fully random (χ2 = 592.578, df = 509, p = .006), which indicates that the database included variables that could be used to predict losses in other variables. The missing values of each variable are related to other variables; therefore, other variables can be used to impute them.

Missing values were managed using the multiple imputation by chained equations (MICE) method, including all the variables in the imputation model. The predictive mean matching imputation technique was selected because there is evidence that it can be used to correctly identify the number of factors and factor loadings in exploratory factor analysis (McNeish 2017). Bearing in mind the percentage of cases with missing values, 20 imputations were performed with 15 iterations per imputation, which resulted in convergence according to visual inspection and \( \hat{R} \) (\( \hat{R} \) < 1.1) for the means and variances of all variables.

The analyses were performed for each imputation and combined according to the recommendations published by Marshall et al. (2009). This procedure showed that the listwise deletion method produced results comparable to those of data imputation.

Data analysis

Participants’ responses to the ICE-A were analyzed to explore the questionnaire’s unidimensionality, reliability (internal consistency and temporal stability), discriminant validity, and sensitivity to change. All analyses were performed using R 3.4.3 (R Core Team 2017) with the DA.MRFA (Navarro-Gonzalez and Lorenzo-Seva 2017), psych (Revelle 2017), mice (van Buuren and Groothuis-Oudshoorn 2011), miceadds (Robitzsch et al. 2017), and lcmm (Proust-Lima et al. 2017) libraries.

Unidimensionality of the questionnaire

The first available measurement of each participant was used to complete a sample of n = 124, which exceeds the recommended minimum size (n = 110) to perform a factor analysis considering 4 factors, a wide range of item commonality (.20 < h z .80), and a ratio of 5 items per factor (Mundfrom et al. 2005). An exploratory factor analysis (EFA) was conducted using the minimum rank factor technique (MRFA; ten Berge and Kiers 1991) based on the matrix of inter-item polychoric correlations. The MRFA extraction technique minimizes non-explained common variance, causing two elements to be positive semi-definite: (a) the diagonal of the matrix of single variances and (b) the result of subtracting the diagonal of the matrix from the matrix of observed covariances. Therefore, it displays optimum commonalities that make it possible to differentiate the explained common variance from the total common variance (Shapiro and ten Berge 2002). Given the presence of leptokurtosis in the distributions of all items, polychoric correlations were employed (Muthén and Kaplan 1985, 1992).

In addition, robust parallel analysis (PA-MRFA; Timmerman and Lorenzo-Seva 2011) was employed to assess the dimensionality of a set of variables by comparing the observed eigenvalues to randomly generated eigenvalues. PA-MRFA based on polychoric correlations is suitable for examining polytomous ordinal variables. Finally, the item-total correlations of each item were explored.

Reliability

To determine the reliability of the ICE-A, both its internal consistency and its temporal consistency were evaluated. Its internal consistency—defined as the degree to which the item set yields similar results—was explored with Cronbach’s alpha (Cronbach 1951), using the same sample used for factor analysis. Temporal stability—the degree to which scores in the questionnaire remain stable over time if no additional situations occurred which could have affected the state of the variable measured—was analyzed by examining the correlation between the measurements taken before the treatment and after 1 month of treatment had elapsed. Even though the participants had been receiving treatment during that period, the effect of the interventions was judged to be insufficient to cause major changes. There were 29 cases in which questionnaire results were available at the required moments of measurement (month 0 and month 1). With a sample of this size, it was possible to detect correlations of a moderate magnitude (r = .44) with 80% accuracy. Since level 2 predictors (differences between individuals) were not evaluated, this sample size was considered sufficient to find unbiased estimators of regression coefficients and their standard errors (Maas and Hox 2005).

Discriminant validity

To evaluate discriminant validity, a correlation technique was used to compare the participants’ ICE-A scores with their AUDIT and OQ-45.2 scores. Since these two questionnaires evaluate alcohol use risk and psychological functioning from the respondent’s perspective, it was not unexpected to find low correlations with the ICE-A. Information submitted by 66 and 62 participants was used for the AUDIT and OQ-45.2 questionnaires, respectively. With these sample sizes, it was possible to detect low magnitude correlations (r = .30 and r = .31, respectively) with 80% accuracy.

Sensitivity to change

To explore sensitivity to change, a latent process mixed models approach (Proust-Lima et al. 2007, 2011) was employed to estimate the trajectory of ICE-A scores over time. This approach estimates a structural model and a measurement model to account for the effects of curvilinearity, ceiling, and flooring that often appear in psychometric tests (Proust-Lima et al. 2007). Moreover, since it yields estimates in a maximum likelihood framework, it is robust against unbalanced samples, in which not all participants have measurements for all moments (Proust-Lima et al. 2011).

In the measurement model, a non-linear function (in this case, I-splines with five nodes located at the 0th, 25th, 50th, 75th, and 100th percentiles corresponding to ICE-A scores 1.0, 2.2, 3.9, 4.3, and 5.0.) was used to estimate the relationship between the measure (ICE-A) and the latent process (disease awareness). For the structural model, a standard mixed linear model is used with time as predictor and the latent variable as outcome. If the ICE-A is adequately sensitive to change, the slope should be positive when evaluating patients receiving treatment.

For this analysis, the cases for which records were kept for at least two measurement moments were used. This resulted in a sub-sample of 159 observations from 46 participants.

Results

Unidimensionality of the Questionnaire

The polychoric correlations matrix was adequate for factor extraction (Kaiser-Meyer-Olkin Measure of Sampling Adequacy = .94; Bartlett’s test of sphericity F(190, 8677.52) = 13.737, p < .001). The one-factor solution (eigenvalue = 12.55) represented 69.8% of the explained common variance, with N = 124. For its part, PA-MRFA is in line with this result, also recommending a one-factor solution (see Fig. 1).

Fig. 1
figure 1

Results of the parallel analysis using the minimum rank factor technique. Comparing the observed values with a distribution of random values shows that a 1-factor solution is advisable

Full size image

Finally, for all items, it was observed that the factor loading on the single factor was higher than .61 and that its correlation with the overall score was higher than .53, which provides additional evidence for the unidimensionality of the questionnaire (see Table 1).

Table 1 Descriptive statistics, factor loadings, and item-total correlations of items
Full size table

Reliability

The ICE-A displayed high internal consistency (α = .96; standard measurement error = .01) and high temporal stability, as shown by the correlation between the scores before the treatment and after 1 month (r = .85, SE = .06, p < .001, n = 29).

Discriminant Validity

Evidence was found in support of the questionnaire’s discriminant validity, as the scale measures a phenomenon other than that measured by the scales to which it is compared (AUDIT and OQ-45.2). The correlation between the ICE-A and the AUDIT was positive, low-magnitude, and statistically significant (r = .26, p = .033, n = 66). However, the correlation with the OQ-45.2 was also positive but its magnitude was very low and statistically non-significant (r = .13, p = .305, n = 62).

Sensitivity to Change

According to a visual inspection of the density plot, ICE-A presents an asymmetric and bounded distribution, which reflects the possible existence of curvilinearity problems and justifies the use of sophisticated mixed models. Therefore, the effects of treatment on ICE-A scores were analyzed using a latent process mixed model (Proust-Lima et al. 2007). Additionally, the null model estimated using I-splines as a function of the bond in the measurement model has a better fit (AIC = 400.60, BIC = 415.23) than the model produced using a linear function (AIC = [1] 489.79, BIC = 495.27).

In order to reflect the existence of variable slopes and intercepts for each participant, subjects were specified as a random factor while time was specified as both a random and a fixed factor. However, the random slopes model could not be identified due to singularity problems. In a subsample of 159 observations from 46 participants, it was detected that the time spent in treatment (b = .12, SE = .003) was significant (p < .001) for predicting ICE-A scores. After a visual inspection of LOESS estimation (Fig. 2), we further investigated whether a quadratic term was significant.

Fig. 2
figure 2

Individual trajectories of subjects’ ICE-A transformed (i-spline) scores as a function of time spent in treatment. LOESS- (solid line), linear- (dashed line), and quadratic-based (dotted line) estimation (n = 46)

Full size image

The goodness-of-fit statistics for the linear model (AIC = 377.51, BIC = 393.96) were just slightly better than those of the quadratic model (AIC = 378.05, BIC = 396.34), and the quadratic term was not statistically significant (b = − 0.002, SE = 0.002, p = .228).

Results from both models are summarized in Table 2. Normality and linearity assumptions were checked by visual inspection of the residuals VS predictions and of normal QQ plots for the marginal and subject-specific residuals, revealing no noticeable deviations.

Table 2 Results of latent-process mixed models to predict ICE-A
Full size table

A post hoc analysis was conducted to explore paired contrasts of the scores among all evaluation moments. As shown in Table 3, a comparison among the scores of months 0 (baseline), 1, 3, and 6 yielded no significant differences. However, the subjects’ scores at 12 months of treatment were significantly higher than those obtained by the same subjects at 6 months of treatment. Finally, no statistically significant differences were found between month 12 and month 24 scores.

Table 3 Analysis of the differences among ICE-A scores at each moment of evaluation
Full size table

Conclusions

The central aim of the present study was to construct and validate a questionnaire for assessing the disease awareness of patients with substance use disorders. The results obtained provide sufficient evidence in support of the unidimensional structure of the questionnaire, as all the items that comprise it measure a single variable that can be expressed as an overall score. In addition, the questionnaire was shown to have an adequate level of internal consistency, which reflects the strong correlation among its items and reinforces the evidence for its unidimensionality.

The ICE-A also displayed high temporal stability over a 1-month period, which is logical considering that the aim of the first month of treatment is to attain abstinence and symptomatic control. Also, patients’ disease awareness was expected to increase gradually and in parallel to treatment time, especially after the sixth month. It is at this moment that the patient manages to improve his/her self-esteem and reduce his/her emotional pain, which enables him/her to develop a new lifestyle. The subjects’ ICE-A score trajectories throughout their treatment showed a significant increase, particularly between the sixth and twelfth months of treatment. Therefore, the ICE-A is adequately sensitive to change, which makes it possible to detect a progressive increase in the disease awareness of patients receiving treatment.

With respect to the limitations of the study, it should be noted that the ICE-A includes items that refer to abstinence; therefore, its usefulness is limited to therapeutic models based on this requirement. The ICE-A should be administered to other samples in order to increase the generalizability of the results obtained.

Finally, the present study demonstrates the need for a specific scale for measuring disease awareness in addicted patients. Such a scale would complement the results obtained with scales that measure other aspects of the addiction phenomenon and would also enable specialists to evaluate and monitor awareness growth throughout the treatment. In brief, the ability to recognize the importance of a substance-free lifestyle, the acceptance of one’s limitations, and the adoption of the necessary measures to preserve such a lifestyle are indicators of improvement and could reflect a better prognosis.