Introduction

Over the last 20 years, a burgeoning number of studies have shown that cancer patients present high levels of distress, which is associated to maladaptive coping and impairment in social relationships. [1] However, far less than 30% of cancer patients showing psychosocial problems are recognized in clinical settings by oncologists and thus referred to proper sources of psychological support [2, 3]. Several psychometric instruments, such as the Hospital and Anxiety Depression Scale (HADS) [4–6], the Brief Symptom Inventory (BSI) [7], the Profile of Mood States (POMS) [8], and the Zung Depression Inventory [9] have been suggested as clinical tools to be routinely used in cancer settings in order to improve the detection of psychosocial morbidity, especially depression, and the referral of patients who resulted as “cases” [10, 11]. Lack of time and lack of self-confidence in exploring psychosocial dimensions and in using psychometric instruments have been raised by cancer physicians as the most frequent barriers for not detecting and referring patients [12].

Thus, brief screening instruments such as single-item questions and 0–10 point scales have more recently been suggested as a rapid, noninvasive, acceptable, and valid alternative to psychometric instruments, especially in busy cancer outpatient clinics. Within the National Comprehensive Cancer Network (NCCN), guidelines for the identification of psychosocial distress have been recently proposed to be an integral part of routine clinical care in cancer settings [13–16]. A score of 4 on a 0–10 distress tool, the Distress Thermometer (DT), was proposed as the most appropriate cutoff for identifying distressed patients and referring them for psycho-oncology consultation. Roth et al. [17], by using the HADS and the DT among prostate cancer patients, found the DT to be useful in rapidly detecting patients in need of more specific psychological assessment. Jacobsen et al. [18], by studying 380 cancer patients, showed that a cutoff of 4 on the DT yielded optimal sensitivity and specificity in comparison with “caseness,” as established by the HADS. They also found the above-mentioned cutoff was useful in identifying patients reporting high levels of physical, emotional, practical, and family problems [18, 19].

More recently, cross-cultural data have been gathered on the use of the DT in cancer settings. In a Japanese study [20] carried out on 275 cancer patients, the sensitivity and specificity of the DT in the detection of adjustment disorders and major depression were reported to be 84% and 61%, respectively. The authors also showed that a companion Impact 0–10 Thermometer (i.e., “What is the impact of illness to you”) was useful, at a cutoff of 3, in adding further clinical information.

However, some contrasting data and problems have also emerged. In a large study of more than 1,000 cancer patients, the DT was associated to a larger extent with the HADS anxiety dimension rather than HADS depression dimension, with higher identification of cases of anxiety rather than those who were depressed [21]. This suggests the need to consider further simple tools, as in the study of Akizuki et al. [20], in order to discriminate between psychosocial dimensions and detect possible morbidity. Furthermore, Muszbek et al. [22], examining the specificity and sensitivity of the DT in comparison with the HADS in Hungarian cancer patients, found problems in the translation of the word “distress” into their own language. They used multiple words such as “feeling blue” and “feeling bad,” but with the risk of measuring different dimensions. Similar problems of translation or conceptualization from English (e.g., fatalism, abnormal illness behavior) have been reported by other countries, such as Italy and Spain, where other psychosocial dimensions of cancer were examined from a cross-cultural perspective [23, 24]. However, no study has examined the use of DT and its translated versions in these countries. Thus, given the importance of routinely assessing cancer patients’ emotional distress that, along with pulse, temperature, respiration, blood pressure, and pain, has been proposed to be considered as the “sixth vital sign” [25], we considered it important to have more information on this area from a cross-cultural point of view.

This study is a part of a larger investigation (Southern European Psycho-Oncology Study—SEPOS) that evaluated psychosocial implications of cancer in southern European countries [26, 27] and the efficacy of a training model in helping physicians to recognize their patients’ psychosocial morbidity [28]. The main purpose of the study presented here was to assess the validity of two simple instruments in cancer settings in a cultural context where no data are available on the application of “emotional” thermometers.

Patients and methods

Patients

The study was conducted in three countries of the European Union (EU), namely, Italy (Hospital S. Anna, Ferrara), Spain (Hospital Duran i Reynals, Barcelona), and Portugal (Hospital S. José, Lisbon), and in a non-EU country with a similar cultural background, Switzerland (General Hospital, Lugano). A convenience sample of cancer outpatients was invited to participate in the study. Enrollment criteria entailed being outpatients with a diagnosis of cancer (at any stage) between 6 months and 2 years and no cognitive impairment due to disease or side-effects of treatment, as evaluated both at the clinical level (orientation in time, space and person, short-term memory) and through the Mini Mental State Examination (cutoff=24) [29]. Furthermore, since the original study also had the aim of evaluating the ability of physicians to assess their patients’ psychosocial morbidity, only patients reporting a score ≥80 on the Karnofsky Performance Status Scale (KPSS) [30] were recruited. This was in order to avoid the possibility of patients’ poor physical conditions negatively influencing the doctors’ ability to assess psychosocial disorders, which are more complex to evaluate in the advanced stages of illness [31]. However, the patients enrolled in the study were representative of cancer patients seeking care at the site of each respective center and region.

The study was approved by the ethical committee or related boards of the hospitals. Each patient was informed of the aim of the study, gave written consent to participate, and was individually met by research psychologists involved in the study. Each patient was given psychological measures, which were explained in detail before being completed. The following instruments were used:

  1. 1.

    The HADS [32] was used to assess anxiety, depression, and total psychosocial morbidity (caseness). The HADS was chosen both because it has been reported to be a “gold standard” for evaluation of anxiety and depression in cancer settings [5, 6] and because of the need to replicate in a different cross-cultural context the data obtained by other authors using the same methodology [18, 19, 22]. The HADS is a 14-item self-rated scale consisting of two subscales: anxiety (seven items) and depression (seven items), which together yield a total HADS score. A cutoff of 8 on each scale has been proposed for “likely or borderline cases” of both anxiety and depression among cancer patients while the cutoff of 11 has been reported to identify more severe cases [5]. On the HADS total score, the cutoff >14 has been shown to be the best predictor of general psychosocial morbidity [33, 34] while the cutoff of 19 has been shown to be the best predictor of more severe caseness (e.g., major depression).

  2. 2.

    Two “emotional thermometers” were used to evaluate the patient’s level of distress (DT) and depression (MT). The DT was used according to NCCN guidelines on distress management [13, 15] while depressed mood (MT) was chosen in order to verify its relationship with the dimension of mood. Each patient was asked to answer the question “How distressed have you been today and over the last week”? for the DT and to rate their answer on a 0–10 scale (0=no distress; 10=extreme distress). Likewise, they were asked to answer the question “How depressed have you been today and over the last week”? for the MT and to rate their answer on a 0–10 scale (0=normal mood; 10=extreme depression).

Statistical analysis

Distribution and frequency analyses were employed to describe the sample. Cronbach’s alpha was used to examine reliability and internal consistency of the HADS. Correlation analysis, Student’s t test, and ANOVA where used when appropriate. Receiver operating characteristic (ROC) analysis [35] was used to evaluate the diagnostic accuracy of the DT and MT to detect cases identified by the HADS. A series of ROC analyses were performed by using the different DT and MT cutoff scores on the total score of the HADS in order to measure the sensitivity, specificity, and positive predictive values in discriminating between cases and noncases, as identified by the HADS.

Results

Sociodemographic and clinical features of the sample

Three hundred and twenty-three patients were recruited. Mean age was 57 years (SD 13.26); 69% (n=224) were women and 21% (n=99) were men. Mean education was 9 years (SD 5.88). Most patients were married (n=233, 72%), and 41% were retired (n=133). The stage of the disease was local (n=134, 41%), loco-regional (n=87, 27%), and metastatic (n=55, 17%). Almost half of the sample was diagnosed as having breast cancer (n=154, 8%) (Table 1).

Table 1 Patient sociodemographic and clinical characteristics
Full size table

Data concerning psychological measures

Complete questionnaires were available for 312 subjects. Since the HADS was used in three different countries, factor analysis was performed in order to assess the characteristics and reliability of the scale. Two factors emerged consisting of the same items as the original version (seven items for the anxiety factor, factor loading between 0.44 and 0.78, Cronbach’s alpha=0.81; seven items for the depression factor, factor loading between 0.49 and 0.79, Cronbach’s alpha=0.81).

According to the established cutoff values of 8, 203 patients (65%) and 236 patients (75.6%) scored within the normal ranges on anxiety and depression, respectively. Fifty-five patients (17.6%) were borderline cases of anxiety, and 48 (15.4%) were borderline cases of depression (cutoff <11). Fifty-four (17.3%) were cases on anxiety and 28 (9%) on depression (cutoff ≥11). With regard to general psychosocial morbidity (HADS total score), 28.8% patients (n=90/312) emerged as being cases (cutoff >14) and 16.3% (51/312) as being more severe cases (cutoff=19).

DT correlated more significantly with HADS anxiety (r=0.50, p=0.001) than depression (r=0.40, p=0.02). MT correlated significantly both with HADS depression (r=0.61, p=0.001) and HADS anxiety (r=0.56, p=0.001). In general, MT showed more significant correlation scores with HADS subscales and HADS single items than DT (Table 2). DT and MT were significantly correlated (r=0.40, p=0.01).

Table 2 Correlation between the emotional thermometers [distress thermometer (DT) and mood thermometer (MT)] and the Hospital Anxiety Depression Scale (HADS)
Full size table

With regard to sociodemographic and clinical variables, no difference was found on the DT and MT according to gender, stage of illness, and type of cancer (breast versus other types). As far as the HADS is concerned, only HADS depression was marginally related to age (r=0.16, p=0.05) while no significant relationships were found regarding the other sociodemographic and clinical variables explored.

ROC analysis

ROC was used to identify the optimal cutoff score on the distress and depression thermometers in identifying caseness according to the HADS total score. By using the HADS total score for general psychosocial morbidity, the AUC was 0.77 (95% CI 0.72–0.83) for the DT and 0.83 (95% CI 0.78–0.88) for the MT. By using the more conservative HADS total score for more severe caseness, the AUC was 0.78 for DT (95% CI 0.71–0.85), and 0.84 for MT (95% CI 0.78–0.90). The optimal cutoff scores on DT and MT for general psychosocial morbidity and more severe caseness are presented in Figs. 1 and 2. A cutoff score >4 maximized sensitivity and specificity of DT in detecting general psychosocial morbidity (sensitivity=65%, specificity=79%) while the cutoff score >5 maximized sensitivity (70%) and specificity (73%) in detecting more severe caseness. The cutoff score >3 on the MT was associated with sensitivity of 85% and a specificity of 72% in detecting general psychosocial morbidity while a score >4 was associated with a sensitivity of 78% and a specificity of 77% in detecting severe caseness.

Fig. 1
figure 1

Receiver operating characteristics (ROC) curve of the DT (squares) and MT (circles) in detecting general morbidity according to the Hospital Anxiety and Depression Scale (HADS) (cutoff >14)

Full size image
Fig 2
figure 2

Receiver operating characteristics (ROC) curve of the DT (squares) and MT (circles) in detecting more severe caseness according to the Hospital Anxiety and Depression Scale (HADS) (cutoff=19)

Full size image

Discussion

In this study, we presented the validity of two easy-to-administer emotional thermometers in examining two dimensions of psychosocial morbidity, namely, distress and depressed mood, among cancer patients in southern Europe. The most important caveat to be considered when discussing the data relates to the fact that the evaluation of psychosocial morbidity was performed by using a psychometric instrument, such as the HADS. Although we followed the same methodology of other investigations [18, 19, 21, 22] which detect caseness by using the HADS only, because of its good factor structure and performance in cancer settings [4–6], a more detailed psychiatric interview [e.g., Structured Clinical Interview for DSM-IV (SCID), Composite International Diagnostic Interview (CIDI), Mini International Neuropsychiatric Interview (MINI)] would have avoided the risk of patient misclassification. In fact, by using the HADS, 15–20% of patients were shown to be false-positive or false-negative, respectively, in different studies [4, 5]. However, in our investigation, the HADS appeared to have acceptable levels of reliability and internal consistency. Furthermore, preliminary data examining the DT and the HADS compared with the International Classification of Diseases, Tenth Revision (ICD-10) in Italian cancer patients seem to confirm validity of both psychological tools [36]. It also has to be considered that even gold standard psychiatric nosographic systems [e.g., Diagnostic and Statistical Manual for Mental Disorders-Fourth Revision (DSM-IV) and ICD-10] lack criteria for the identification of some important dimensions of psychosocial morbidity (e.g., health anxiety, demoralization, irritable mood) in medically ill patients, including cancer patients [37, 38]. Bearing these limitations in mind, the findings of the study showed that both the DT and the MT were useful in rapidly assessing psychosocial distress in cancer patients in a different cultural area in comparison with English-speaking countries. This underscores the utility of simple instruments in their routine application in cancer services.

Distress was related to a larger extent to HADS anxiety while mood was related both to HADS depression and anxiety. This result is partially in line with what was reported by Patrick-Miller et al. [21], who found DT to be anxiety-related rather than depression-related. When examining the intercorrelation of DT and MT with the single HADS anxiety and depression items, some items of the HADS scales seemed to be more representative of the subjective evaluation of the single dimensions, such as the DT and the MT, while other HADS items were less significantly associated with the DT and MT. In general, however, MT performed better and showed higher correlation with anxiety and depression than did DT. This could be related to translation problems, as already found for other psychosocial dimensions of cancer [23] and the difficulty of finding an exact match for the word “distress” in Italian, Spanish, and Portuguese. In contrast, the word depression seems to have no problem in being understood. Cross-cultural comparisons between studies using the DT [19, 21, 22] could clarify part of these findings.

As far as psychosocial morbidity is concerned, when using the standard cutoff scores for the HADS, as the literature has repeatedly suggested as criteria for identifying cases, 17.6% appeared to be borderline cases of anxiety and 15.4% borderline cases of depression. Moreover, 17.3% and 9% of patients appeared to have more severe states of anxiety and depression, respectively (cutoff=11). For general psychosocial morbidity (HADS total score), 28.8% of patients (n=90/312) were classified as being cases (cutoff >14) while the rate of more severe general caseness (cutoff=19) was 16.3%. These data confirm data reported by other authors who indicated an average prevalence ranging from 15% to 25% [5, 6]. ROC analysis was helpful in identifying the best cutoff scores on DT and MT in detecting cases according to the HADS. More significant sensitivity and specificity levels were found on MT than DT in detecting both general morbidity and more severe caseness. On the whole, the best cutoff scores on DT were 4 (general morbidity) and 5 (more severe caseness) while the best cutoff scores on MT were 3 (general morbidity) and 4 (more severe caseness). With regard to DT, these findings confirm the results obtained by Jacobsen et al. [18], who showed that a cutoff score of 4 on DT was related to caseness as measured by both the HADS and the BSI. By using the cutoff of 5 among prostate cancer patients, Roth et al. [17] showed that 31% were cases who were referred for psychiatric evaluation. Forty-seven percent of DT cases received a DSM-IV psychiatric diagnosis. However, to our knowledge, no data are available in the literature regarding the use of a different thermometer, such as the MT, in detecting HADS caseness.

Some issues remain to be explored. Inclusion criteria in this study (i.e., outpatients with a good performance status) prevented us from drawing conclusions for patients with more advanced disease, for those admitted to the hospital, and for those followed at a domiciliary level. The usefulness of DT or MT in these settings in southern Europe should be examined. In this respect, Chochinov et al. [39] showed that a single-item screening correctly identified depressed cancer patients outperforming psychometric questionnaires ,such as the BDI, among advanced cancer patients. A further aspect refers to the fact that the study was carried out in only three centers and on a small sample of patients. Involvement of other centers in different countries and a larger patient population would have been more significant in terms of statistical analysis and clinical implications.

In conclusion, this study showed that the DT measuring distress and a new tool, the MT measuring depression, were valid instruments in detecting cancer patients who were cases or severe cases according to the HADS, with a tendency of the MT to perform better than the DT. Overall, the findings of this study might have practical implication in daily clinical activity and facilitate oncologists and nurses in focusing attention on psychosocial dimensions of cancer, in using simple tools to assess these dimensions in follow-up visits, and in improving the referral process. The possibility of misclassifying patients should, however, always be considered as a limitation of short screening instruments.