Introduction

In illness-related behaviour, such as absence from work, search for help and consultations with physicians, both physical and mental health factors may play a part. When studying such behaviour, it is important to be able to make a clear distinction between physical and mental aspects of health and illness. If mental disorders are measured in addition to other measures of illness and physical functioning, the assessment of these disorders should be as independent as possible from physical illnesses and symptoms.

However, common mental disorders, such as psychological distress (PD) or mood disorders, are often accompanied by physical symptoms or bodily complaints, which may complicate the distinction between physical and mental disorders [1].

This implies that when choosing an instrument appropriate for studies of illness-related behaviour, measures of PD contaminated with physical symptoms should be avoided, or alternatively, measures with the lowest degree of contamination should be preferred.

In this study, we will focus on the assessment of the construct validity of three measures that can be used as indicators of PD. Validity will be assessed from two perspectives:

  1. 1.

    the association of these PD measures with physical illness measures and

  2. 2.

    the performance of these PD measures to predict perceived health.

The first perspective is chosen to assess the degree of contamination of the PD measures. If the association is weak, we consider the degree of contamination to be low. The second perspective is less obvious but is related to the background of this study: our intention is to use (one of) these measures to predict help-seeking behaviour and utilisation of general, i.e. non-mental, health care. It may, therefore, seem logical to compare these measures on the strength of their associations with the utilisation of general health services. However, health service use is also influenced by characteristics of the health care system itself, such as availability and accessibility of specific services [2, 3], from which we want to refrain in this stage. For that reason, we examine the relation between PD and subjective health (perceived health status), because we consider subjective health as the nearest next link on the causal pathway to health care utilisation, which is not influenced by structural characteristics of the health care system or physician behaviour [4, 5] but is closely associated with health care utilisation [6–8]. A measure of PD that is best in predicting subjective health has the highest validity in this respect. However, subjective health will not be related to PD only but to other subject-related factors as well, especially physical (ill) health [9, 10]. At the same time physical illness may lead to PD [11, 12]. This implies that both PD and subjective health may be related to physical illness and that physical illness may act as a confounder in the association of PD and subjective health (see also Fig. 1). Therefore, we will also study the effects of physical (ill) health (and other possible confounders) on the association between PD and subjective health and adjust for their effect.

Fig. 1
figure 1

Relations between main concepts and measures

Full size image

In combining these objectives, the aim of this study is to compare three indicators of PD on the strength of their association with subjective health and to analyse to what extent these associations will change after adjustment for physical illness measures and other possible confounding variables. An indicator of PD with minimal physical health components will show less change in degree of association with subjective health if this association is adjusted for physical health, which will be seen as an indication of higher construct validity.

The main concepts and measures and their supposed relations are depicted in Fig. 1.

Method

Study population

For this study, data were used from a community-based sample of adults (from 15 to 90 years of age) from a population of enrollees of a sickness fund working in the western part of the Netherlands, who had responded to a health survey mailing (N=9,428). The design of the procedure for conducting the survey was guided by Dilman's [13] recommendations. The first mailing of the survey was in February 1993. In this sample, individuals who were hospitalised in a period before the survey were deliberately over-represented. The net response rate to the survey was 70.4%. Respondents and non-respondents differed slightly in their medical consumption. Non-response bias will result in a small overestimation of utilisation of prescribed drugs [14].

Data

As part of the survey, PD was measured using three different instruments. The first instrument was the Negative Affect Scale (NAS), i.e. the negative items of Bradburn's Affect Balance Scale [15]. The NAS is a five-item scale, which appeared to be quite reliable (Cronbach's α=0.80) in this study. It is related to factors such as anxiety and depression [15], neuroticism and psychological complaints [16]. The second instrument was the nervousness scale. This three-item scale was originally part of a larger scale designed to measure work-related stress and subjective health [17, 18]. Its reliability appeared to be satisfactory (α=0.74). It measures feelings of nervous tension and irritability and it is correlated to neuroticism [18].

As part of the survey, chronic conditions (25 types), including depressive complaints, were assessed by self-report, using the same checklist as listed in the periodical General Health Survey of the Central Office of Statistics in the Netherlands [19]. Self-reported depressive complaints (a single-item measure) were used as a third indicator of PD.

To measure physical illness, only active chronic conditions, i.e. conditions for which treatment was going on at the moment of completing the questionnaire, were used. Seven different active chronic conditions (arthritis, back problems, diabetes, heart disease, hypertension, lung disease and migraine) that had the highest prevalence in the sample were used as seven independent dummy variables. Data on the other 17 conditions (excluding depressive complaints) were used to form a co-morbidity index by counting the number of chronic conditions for which treatment had been sought.

In addition to these disease-related variables, two other possible confounding variables were included in the analyses. Two measures of disability, based on indicators developed by the Organisation for Economic Co-operation and Development [20], were used: activities of daily living (ADL) restrictions and mobility restrictions. The reliability (Cronbach's α) of these measures was 0.89 and 0.85, respectively.

Subjective health was assessed as part of the survey by asking a single question (“How is your health in general”) with five answer categories, ranging from ‘very good’ to ‘poor’.

The socio-demographic variables that were included in the analyses were age, gender, marital status (divorced/separated/widow(er) vs other), living situation (household size) and educational level.

Analyses

The analyses were conducted in two steps

First, we wanted to detect which illness and disability variables and which socio-demographic variables should be considered as confounding variables, as to the relation between PD and subjective health. For that purpose, the measures of PD and subjective health were each used as dependent variables in regression analyses, which used general and specific chronic illness variables as well as socio-demographic variables as independent variables. This was done separately for each category of independent variables as well as combined for all categories of independent variables. To be considered a confounding variable requires that the same (independent) variable is associated with the measure of PD under consideration as well as with subjective health. This implies that both (standardised) regression coefficients related to the same independent variable should have a minimal significance level of 1%.

Secondly, for each measure of PD, two regression models predicting subjective health were estimated: firstly, a basic model (unadjusted, with PD as the only independent variable); and secondly, an adjusted model with PD and the confounding variables selected in the first step as independent variables.

All models were estimated by using standard multiple regression analysis. As ‘depressive complaints’ is a binary variable, using multiple regression analysis might result in biased coefficients. We therefore checked the results by replicating the analysis related to this variable using logistic regression analysis instead of linear regression analysis.

As a final check, a structural equation model (reflecting the model depicted in Fig. 1) was tested for each measure of PD. In this model, physical illness will be considered as a latent variable, which is measured by several observed indicators. The EQS 6.1 program [21]was used for that purpose.

Results

Sample characteristics

Almost 57% of the respondents were female. The largest age category was 25–34 years (28%), whereas 17.4% were aged 65 or more. The majority of the respondents (75%) rated their health as good or very good. However, 40% had one or more chronic conditions. Nearly 16% had a chronic condition with at least one co-morbid condition. Back problems were predominant (8.8%), followed by migraine (7.3%) and arthritis (6.7%). Only 5% had some degree of ADL restrictions, mobility restrictions being more frequently seen (almost 20% had some restrictions).

Almost 13% had depressive complaints and more than 5% sought treatment for these complaints. On the nervousness scale, 19% scored two or more complaints (out of three possible complaints). On the NAS, 44% did not have any negative feeling at all and 30% had sometimes one or two negative feelings (out of five). The other 26% had at least several negative feelings on occasion, such as loneliness or being upset.

Additional sample characteristics can be found in Table 1.

Table 1 Sample characteristics
Full size table

Psychological distress and subjective health

In Tables 2 and 3, the results of the regression analyses, as described in the method section, are presented. Table 2 contains standardised regression coefficients (betas) of the regression models with the three variables indicating PD and subjective health variable as dependent variables. This table shows the strength of the associations of the indicators of PD with physical illness measures and demographic variables. For each dependent variable, the coefficients of the three separate models (demographics, general illness and specific chronic conditions) are presented in the left column, whereas the coefficients of the combined model are presented in the right column.

Table 2 Prediction of three indicators of PD and of subjective health status by socio-demographic and illness variables
Full size table
Table 3 Prediction of subjective health by three indicators of PD and possible confounding illness and demographic variables
Full size table

The relation between physical illness and PD is weak in the case where ‘depressive complaints’ is used as an indicator of PD. The general illness measures explain just 2% of the total variance, the specific chronic conditions 3% and the combined model, i.e. the combination of these measures with socio-demographic variables, explains 4% of the variance. In comparison to the other indicators, ‘negative affect’, used as an indicator of PD, has the strongest association with general illness measures (adjusted R 2=0.07; with the combined model, adjusted R 2 becomes 0.12). ‘Nervousness’ is in between these indicators.

Subjective health is much better explained by the same independent variables. The general illness measures explain 29% of the total variance; the specific measures, 19%. Both categories, combined with demographics, result in 34% explained variance.

From Table 2, we can also learn which independent variables (physical illness measures and demographics) may function as confounders. As already stated in the Method section, independent variables that have significant regression coefficients with subjective health as dependent variable, as well as with one of the PD variables as dependent variable, may be considered confounders as to the relation between PD and subjective health.

Within the category of socio-demographic variables, age is clearly a confounder (although not very convincingly related to nervousness). Low educational level is related to nervousness and subjective health as well.

Within the category of the general physical illness variables, both the number of chronic conditions and mobility restrictions may act as confounding variables related to all three PD measures.

Within the category of seven specific chronic conditions, ‘back problems’ and ‘migraine’ may act as confounders, related to each PD measure. Hypertension and lung disease may be considered confounders related to negative affect and nervousness.

In Table 3, the results can be found in the regression analyses in which subjective health was the dependent variable and one of the PD indicators (depressive complaints, negative affect or nervousness) was the main independent variable combined with other independent variables, which may act as confounders.

From Table 3, we can learn that all indicators of PD are related to subjective health, if not adjusted for confounders. Negative affect and nervousness are strongly associated with subjective health than depressive complaints.

The relationship of ‘depressive complaints’ with (worse) subjective health is attenuated after adding age, number of chronic conditions and mobility restrictions (as well as two specific chronic conditions) to the regression model, but it is still significant. The beta coefficient of depressive complaints is reduced by 29% from 0.24 to 0.17.

In case ‘negative affect’ (instead of depressive complaints) is used as indicator of PD, the addition of confounders results in a reduction of 33% of the coefficient of negative affect (from 0.33 to 0.22). ‘Nervousness’ as a predictor of subjective health is behaving almost similar as to negative affect, with a slightly greater reduction (from 0.35 to 0.22, i.e. 37%) of the beta coefficients related to nervousness, when confounders are added.

The results of testing the structural equation model are summarised in Table 4. In this model, the same variables were used, as shown in Table 3, as far as they were common in all regression models shown in that table. Consequently, educational level, lung disease and hypertension were not included in the analyses. We postulated a latent variable physical illness, which was measured by the number of chronic conditions, mobility restrictions, back problems and migraine. Physical illness was related to subjective health in two ways, directly and via PD (see also Fig. 1). Subjective health was supposed to be related to physical illness, PD and age.

Table 4 Summaries of structural equation models: fit coefficients and model parameters
Full size table

All three models (with different PD indicators) had, more or less, the same goodness of fit. However, the model parameters showed some interesting differences. The path coefficients of the PD indicators and subjective health were almost comparable for depressive complaints and nervousness and the lowest for negative affect. Factor loadings of PD indicators on the physical illness factor (latent variable) were highest for negative affect and lowest for depressive complaints.

Discussion

We assessed the construct validity of three indicators of PD by comparing the strength of their relations with subjective health before and after adjustment for mainly physical illness effects.

The relation between subjective health and PD is strongest in the case where nervousness is used as an indicator. However, the strength of that relation is almost on the same level where negative affect is used as an indicator. The measure of depressive complaints is less strong, but still substantial, as related to subjective health.

After adjustment for physical illness variables and other confounding variables, such as age, the change in strength of the association is the slightest for depressive complaints and highest for nervousness. There are only slight differences between negative affect and nervousness. These measures, which are more contaminated by physical ill health than depressive complaints, have the strongest association with subjective health both before as well as after correction for physical illness components. However, the measure of depressive complaints has the highest construct validity as a measure of PD if a measure with the lowest physical health components is required.

Our study has several limitations and possible weaknesses.

Firstly, our indicators of physical illness were all based on self-report. Theoretically, this might have inflated the observed associations between physical illness measures and subjective health perception as well as PD measures. The most important physical illness indicators were based on self-reported chronic conditions. As we not only asked about the presence of such a condition but also whether a treatment for such a condition was being sought or was still going on, it can be assumed that these indicators are not just tapping health perceptions but also more objective health differences. Nevertheless, we might have overestimated the strength of the associations between physical ill health and PD measures, thus overcorrecting their assumed contamination effects. This may affect the precise strength of the corrected associations, but this will be the case for all measures of PD, implying that it will not affect our main findings.

Secondly, we were not able to establish the reliability of the measure of depressive complaints, as it is a single item measure. If, for this or other reasons, this measure is less reliable than the other indicators of PD, which were at least sufficiently reliable, then the strength of the association with subjective health will be attenuated. This might explain the lower predictive validity of this measure in relation to subjective health.

The association of negative affect with subjective health is consistent with findings of other studies [22–25]. Most of these studies, however, relate negative affect to several forms of illness behaviour such as symptom perception, symptom reporting and somatic complaints, resulting in a stronger association than we found. Moreover, it appears that somatic complaints and symptoms reporting are better explained by negative affect than by differences in physical illness. Studies that relate subjective or perceived health (instead of symptom reporting or somatic complaints) to other measures of health, negative affect and mood disorders are less dominant compared to measures of physical ill health [9, 26]. This could mean that negative affect has more effect on the reporting of symptoms than on the perception of symptoms [27].

Both nervousness and negative affect showed similar patterns of associations with other measures of illness to a great extent. The finding that nervousness is behaving in a more or less parallel way compared to negative affectivity was not expected. The nervousness scale is a part of a larger, more comprehensive scale on work-related stress. This comprehensive scale is susceptible for both physical distress and PD [18]. Our assumption, being that the nervousness subscale would be more related to physical distress than negative affect, appeared not to be correct. A closer look at the specific items that constitute ‘negative affectivity’ and ‘nervousness’, respectively, does not provide a self-apparent explanation of the parallels of these measures. Negative affectivity comprises items related to loneliness, restlessness and depressed mood, whereas nervousness is related to agitation and sensitivity. On the two bipolar dimensions regularly used in the literature on affects and emotions [28, 29], both measures load on the negative side of the (un)pleasantness dimension, but it is not clear whether they differentiate on the second dimension of level of activation (arousal vs sleepiness), as both contain just one item related to arousal.

Depressive complaints can, presumably, be positioned further away from the arousal side than negative affectivity and nervousness. If that interpretation is correct, it might explain, as least partly, the weaker association of it with subjective health compared to the other measures of PD [30].

Many questionnaires and rating scales have been developed to measure depression. They differ largely as to their contamination with physical symptoms [31, 32]. A simple question on depressive complaints, used as a screening tool among clinical groups, appeared to be highly predictive of depressive disorder [33]. It seems that the same simple question results in a measure that is relatively free of contamination with ill health. The reliability of this measure should, however, be further examined.

In conclusion, we can state that both the negative affect measure and the nervousness scale are reliable and valid indicators of PD that can be used to predict subjective health and health-related behaviour. If used for that purpose, they should, however, be corrected for the confounding effects of physical illness variables. Depressive complaint is less strongly related to subjective health and may therefore be less valid as a predictor of health-related behaviour. It is, however, less contaminated by physical illness variables, making it a better indicator of PD if correction for physical illness variables is not possible.