Introduction

Mortality and morbidity have traditionally been the considered “hard” outcomes in evidence-based and experimental therapies. Health-related quality of life (HRQL) is considered an important health outcome, and also a risk factor as it predicts other health outcomes such as mortality [1, 2]. The European Medicines Agency [3] and the US Food and Drug Administration [4] recommend HRQL as an outcome measure in relevant research studies as well as in clinical care, and it is even mandatory in some countries, e.g., Austria [5]. Rather than using generic HRQL questionnaires that focus on the assessment of a wide range of aspects of life in a variety of health states, patient-reported outcome comparisons in a given disease with different diagnoses require disease-specific HRQL questionnaires that have been validated in each diagnosis to better understand the range of HRQL in that disease [6].

Patients with different ischemic heart disease (IHD) presentations, e.g., angina, myocardial infarction (MI), and heart failure, often receive the same therapies, e.g., medications and non-pharmacological interventions such as revascularization interventions and rehabilitation, with symptom management and improvement of patient HRQL as common therapeutic goals. In the IHD continuum, patients with angina, MI, and heart failure frequently present with marked health status deficits including poor HRQL [7]. Valid and reliable IHD diagnosis-specific HRQL instruments such as the Seattle Angina Questionnaire [8], and the Minnesota Living with Heart Failure questionnaire [9], are not designed for across-diagnosis HRQL outcome comparisons as their respective item stems refer specifically to either “chest pain, chest tightness or angina,” or “heart failure.” A solution to this limitation is the use of core disease-specific HRQL questionnaires that have only recently become available in cardiology [10, 11], even though they have been available for two decades or more years in both oncology [12, 13] and rheumatology [14]. Core disease-specific HRQL questionnaires are designed with a sufficient degree of generalizability to allow for cross-study comparisons and a sufficient level of specificity to address research questions of particular relevance in a given disease [12].

The self-administered MacNew Heart Disease HRQL questionnaire (MacNew) uses the item stem “your heart problem,” and was originally validated in English [15, 16]. There are now 39 language versions of the MacNew form, and as the questionnaire has been validated in 14 languages in patients with MI, in 11 languages in patients with angina, and in 9 languages in patients with heart failure, and with the generic stem “heart problem,” it is considered a core IHD-specific HRQL questionnaire [17]. The MacNew Questionnaire has not been validated in Italian-speaking patients with IHD, so Italian-speaking patients with IHD and a diagnosis of angina, heart failure, or MI were recruited as part of the international cross-sectional HeartQoL Project [18].

The primary objective of this report is to report on the reliability and validity of the Italian version of the MacNew Questionnaire in patients with (1) angina, (2) MI, and (3) heart failure.

Materials and methods

Patients

A convenience sample of patients ≥18 years, without serious psychiatric disorders or active substance abuse, who the referring physician considered able to complete the self-administered battery of HRQL instruments were eligible for international cross-sectional HeartQoL Project [18]. The Italian patients were recruited for the HeartQoL Projects at one of the three Northern Italian clinics (Florence, Udine, and Veruno), where they were being treated for:

  1. (a)

    Angina (Canadian Cardiovascular Society class II, III or IV) with an objective measure of IHD (e.g., previous MI, exercise testing, echocardiogram, nuclear cardiac imaging, or coronary angiography) or

  2. (b)

    MI diagnosed at least 4 weeks and <6 months previously; or

  3. (c)

    Ischemic heart failure (New York Heart Association Class II, III or IV) with evidence of left ventricular dysfunction (ejection fraction ≤40 % by invasive or non-invasive testing) and an objective measure of IHD (e.g., exercise testing, echocardiogram, nuclear cardiac imaging, or coronary angiography).

Patient-centered outcome assessment

The referring physician provided routine diagnostic data, and all patients completed a self-report sociodemographic and clinical questionnaire. The Short Form 36 (SF-36) [19], the Hospital Anxiety and Depression Scale (HADS) [20], and the MacNew Questionnaire [15, 16] were each administered on entry to the study.

SF-36

The SF-36 is a validated generic health survey consisting of 36 items with 8 subscales summarized in a physical component summary (PCS) and a mental component summary (MCS) and includes a health transition item [19]. The SF-36 uses norm-based scoring algorithms for all eight scales, and a T-score transformation (mean of 50 and standard deviation of 10). The SF-36 has been extensively used internationally in clinical trials [19], and has been validated in Italian [21].

HADS

The HADS is a validated psychological screening instrument with scores ≥8 identifying patients with symptoms of depression or anxiety, and has been extensively used internationally in clinical trials [20]. The HADS items are scored on a 0–3 scale, and it has been validated in Italian [22].

MacNew Questionnaire

The MacNew Questionnaire is designed to assess a patient’s feelings about how IHD affects daily functioning, contains 27 items with physical limitation, emotional and social function subscales that can be combined to establish a Global HRQL score [23], and was originally validated in English [15, 16]. The items are scored from 1 (low HRQL) to 7 (high HRQL) and the minimal important difference (MID) is 0.50 points [24]. Using forward–backward translation, the MacNew Questionnaire was translated into Italian as part of the international HeartQoL Project [18].

Statistical analysis

We used frequencies, means and standard deviations (SD) to describe patient clinical, sociodemographic, and scale characteristics. Comparisons among the three cardiac diagnostic groups were made with Chi-square (categorical variables), and analysis of variance (continuous variables). We performed Kolmogorov–Smirnov tests for the scored data (SF-36, HADS, MacNew), and used non-parametric statistics where appropriate. Floor and ceiling effects were determined as the proportions of the sample reporting the lowest and highest scores, 1 and 7, respectively.

The conceptual model, reliability, and validity of the MacNew Questionnaire were assessed as recommended by the Scientific Advisory Committee of Medical Outcomes Trust [25].

Factor analysis

Exploratory principal component factor analysis with varimax rotation was performed with factor loadings of ≥0.40 used to allocate items to a subscale, and to determine whether this factor structure replicated the original structure.

Reliability

We assessed the internal consistency reliability of the MacNew with Cronbach’s α; values of r ≥ 0.70 are considered sufficient for group comparisons and ≥0.90 for individual comparisons [25].

Validity

Face and content validity of the MacNew Questionnaire were assumed based on previous validation studies [17]. Convergent validity of the MacNew Questionnaire was tested with Pearson correlation coefficients (<0.10 = absent, 0.10–0.29 = weak, 0.30–0.49 = moderate and ≥0.50 = strong), using Steiger’s test for differences in correlations [26]; stronger correlations between similar SF-36 component scales and MacNew subscales and weaker correlations between dissimilar SF-36 and MacNew constructs were predicted a priori. Discriminative validity was tested for with analysis of variance using the ‘known-groups’ approach [27], with the SF-36 health transition item collapsed in three groups: deteriorated, no change, improved, and the presence or absence of anxiety and depression (HADS cut-off scores, ≥8) [20]. Post hoc comparisons were carried out using Bonferroni corrections.

Results

Patient characteristics (Table 1)

Sociodemographic and clinical data were collected on 298 patients [mean age 64.8 (± 10.6) years] with IHD (angina, n = 88; MI, n = 106; and heart failure, n = 104) recruited at the three centers in Northern Italy. The majority of patients in the total group (47.3 %) had less than high school education, and a minority (13.4 %) had more than high school education; there are significant diagnostic differences (p = 0.04) by education; patients with MI are most likely (58.2 %) to have less than a high school education, and patients with heart failure least likely (7.7 %) to have more than a high school education. Patients with angina are most likely to be hypertensive (59.1 %, p < 0.02), to have high cholesterol (62.5 %, p < 0.01), and to be physically inactive (81.8 %, p < 0.03); patients with heart failure are most likely to be either anxious (39.4 %, p < 0.001) or depressed (41.3 %, p < 0.005.)

Table 1 Sociodemographic and clinical characteristics (mean ± standard deviation or %) of the total group of patients and by diagnosis; data missing when total <100 %
Full size table

Patient-centered outcome measures (Table 2)

MacNew Questionnaire

In the total group of patients with IHD, the mean MacNew global and emotional subscale scores are 5.2 ± 1.0, the mean physical subscale is 5.2 ± 1.1, and the mean social subscale score is 5.4 ± 1.1. By diagnosis, patients with MI have higher (better HRQL) mean global and physical subscale scores than patients with either angina or heart failure than (p < 0.001), and have higher mean emotional and social subscale scores than patients with heart failure (p < 0.001); patients with angina have higher social subscale scores than patients with heart failure (both p < 0.001).

Table 2 Mean MacNew Questionnaire, SF36 Physical Component (PCS) and Mental Component Scale (MCS) and Hospital and Anxiety Scale (HADS) scores (±standard deviation) in the total group and by diagnosis
Full size table

SF-36

In the total group of patients with IHD, the mean PCS and MCS scores are 41.8 ± 9.3 and 46.6 ± 9.9, respectively. By diagnosis, the mean PCS score is higher in patients with MI than in patients with either angina or heart failure (p < 0.001). The mean PCS scores in patients with either angina or heart failure are >1 SD below the population mean of 50.0. There are no significant between-diagnosis differences on the mean MCS scores and all are <1 SD below the population mean.

HADS scores

In the total group of patients with IHD, both mean anxiety (6.1 ± 4.0) and depression (5.6 ± 3.9) scores are below the cut-off of ≥8.0 for possible caseness. By diagnosis, the mean anxiety score is lower in patients with MI than in patients with either angina or heart failure (p < 0.001); the mean depression score in patients with MI is lower than in patients with heart failure (p < 0.001).

MacNew Questionnaire missing item rates

The missing Italian MacNew Questionnaire item rate (excluding item 27, sexual activity) in the total group ranged from 0 % (item #1, frustrated; #2, worthless; #4, down in the dumps; #6, worn out) to 4.3 % (#20, restricted or limited). By diagnosis, the missing item rate in patients with angina ranges from 0 to 4.5 %, from 0 to 3.8 % in patients with MI, and from 0 to 2.9 % in patients with heart failure. Only eight of the 298 patients (<3 %) missed three or more items (maximum missing items, n = 12) meaning that MacNew Questionnaire Global and subscale scores could be calculated for each of the 298 patients.

MacNew Questionnaire floor and ceiling effects (Table 3)

There are no floor effects (score = 1) for the MacNew Questionnaire Global, physical, emotional, and social subscales in the total group and each diagnosis. Ceiling effects (score = 7) are always <1.3 % for the total group, <1.2 % for patients with angina, <2.9 % for patients with MI, and <1.1 % for patients with heart failure.

Table 3 MacNew Questionaire global and subscale floor and ceiling effects and internal consistency (Cronbach’s α) in the total group and by diagnosis
Full size table

MacNew Questionnaire factor analysis (Table 4)

Using Principal Component Analysis, a three-factor solution for the Italian MacNew Questionnaire explains 54.6 % of the observed variance (physical subscale, 17.1 %; emotional subscale, 20.7 %; and social subscale, 16.8 %). Nine of the 13 physical subscale items (69 %), 10 of the 14 emotional items (79 %), and all of the 13 social items (100 %) conformed fully to the original MacNew Questionnaire factor structure with factor loadings that are always ≥0.42. Of the 27 Italian MacNew Questionnaire items, seven load partially according to the original item loading structure; items #12 (social activities), #13 (less confidence in you), #19 (dizzy, lightheaded), #23 (burden on others), #24 (excluded), #25 (unable to socialize), and #26 (physically restricted.)

Table 4 Principal component analysis: factor loadings ≥0.40 for each MacNew Questionnaire item in the total group
Full size table

MacNew reliability (Table 3)

Using Cronbach’s α, internal consistency reliability is always ≥0.90 in the total group, always ≥0.86 in patients with either angina or MI, and always ≥0.91 in patients with heart failure.

MacNew Questionnaire convergent validity (Table 5)

In the total group and in patients with each diagnosis, the correlations between the MacNew Questionnaire physical subscale and the SF-36 PCS and between the MacNew emotional subscale and the SF-36 MCS are all in the hypothesized direction, strong (r ≥ 0.50), and significant (p < 0.001). Also as hypothesized, the correlations between dissimilar MacNew Questionnaire subscales and SF-36 component scales (MacNew Questionnaire emotional and PCS; MCS and MacNew Questionnaire physical and MCS) are lower, moderate to weak (<0.43, >0.23) and significant (p < 0.001).

Table 5 Convergent validity (using Spearman correlation coefficients) of the MacNew Questionnaire physical and emotional subscales with the Short Form-36 physical component (PCS) and mental component scales (MCS) in the total group and by diagnosis (angina, myocardial infarction, and heart failure)
Full size table

MacNew Questionnaire discriminative validity (Table 6)

Considering first the SF-36 health transition item, discriminative validity of the Italian MacNew Questionnaire is confirmed on the Global scale (<0.05) and the emotional subscale (p < 0.01) in the total cohort, on the Global scale and each subscale in patients with heart failure (p < 0.01) but is not confirmed in patients with angina or MI. With both HADS anxiety and depression, discriminative validity is confirmed in the cohort as a whole (p < 0.001) and in each diagnosis (p < 0.01).

Table 6 Discriminative validity of MacNew Questionnaire global score and subscale scores (mean ± standard deviation) by SF-36 health status transition and Hospital Anxiety and Depression Scale (HADS) anxiety and depression (score 8+) in the total group and by diagnosis
Full size table

Discussion

The Italian version of the MacNew HRQL Questionnaire demonstrates acceptable psychometric properties for reliability and validity in patients with IHD affected by angina, MI, or heart failure. Our findings with the Italian MacNew Questionnaire confirm previous language validation studies substantiating the MacNew Questionnaire as a potential core IHD-specific HRQL instrument [10, 28–32]. The mean Global score and each subscale score in the disease-specific self-report Italian MacNew Questionnaire discriminate between diagnoses with HRQL scores always highest in patients with MI and lowest in patients with heart failure, consistent with the pathophysiologic and clinical IHD continuum [33]. The absence of floor effects and the minimal ceiling effects on the subscales permits measurement of change, both deterioration and improvement, in patient-reported HRQL.

Despite patient-reported health status, which includes HRQL, being a predictor of mortality, cardiovascular events, hospitalization, and costs of care in patients with IHD, it is underutilized when assessing cardiovascular health [1, 2]. Interpreting health status should be viewed from a broader perspective than only the biologic and clinical perspectives [34]. Physician rating of disease severity, function, and HRQL apparently does not always correlate well; patient and physician assessment of disease severity have been reported as discordant in more than one-third of cases [35], with patients frequently classifying themselves in a worse functional class than physicians do [36]. Using a single IHD-specific HRQL instrument such as the MacNew Questionnaire or the recently validated HeartQoL questionnaire [11] in patients with different IHD diagnoses has important implications when determining whether treatments, such as invasive interventions or cardiac rehabilitation, are as effective from the patient’s point of view as the clinician’s. As both a risk factor and a health outcome, patient-reported HRQL is recommended as a key measure of cardiovascular health in clinical research, clinical practice, and disease surveillance [1, 2].

The three-factor physical, emotional, and social subscale structure of the MacNew questionnaire is confirmed in the Italian version of the MacNew Questionnaire. Six specific items in the Italian MacNew Questionnaire do not load as in the original factor analysis, most likely due to cultural reasons, and the item on sexual activity (item 27) was not included in the original factor analyses [15, 16], but has since been shown to load on both the physical [31, 32, 37] and the social subscales [10, 29, 38, 39]. However, the factor analysis in our study is consistent with concerns previously reported in the psychometric testing of the MacNew Questionnaire that several items load at ≥0.40 in more than one domain [40, 41], and which are being addressed.

The Italian MacNew Questionnaire has satisfactory reliability and validity substantiating previous reports in other languages as divergent as English [10, 16], Chinese [31], Farsi [42], and German [28, 29]. With Cronbach’s alpha ≥0.86 in each diagnosis, internal consistency reliability is adequate and consistent with MacNew Questionnaire internal consistency reports in other languages [10, 28–32]. Due to the cross-sectional design of the HeartQoL HRQL survey [43]. it is not possible to determine test–retest reliability. Convergent validity of the Italian MacNew Questionnaire is confirmed with strong correlations of between 0.52 and 0.77 for all eight comparisons between similar MacNew Questionnaire and SF-36 scales (MacNew Questionnaire physical and SF-36 PCS; MacNew Questionnaire emotional and SF-36 MCS). Dissimilar scales (MacNew Questionnaire physical and SF-36 MCS; MacNew Questionnaire emotional and SF-36 PCS) have lower correlations, always r ≤ 0.42. These observations are also consistent with previous reports [10, 28–32].

Using the “known groups” approach [27], discriminative validity of the Italian MacNew Questionnaire is partially confirmed with the SF-36 health transition item. Although there are statistically significant differences between those whose health has improved and those whose health has deteriorated in the total group, the differences do not meet the MID criterion; the differences are not statistically significant, and do not meet the MID criterion in patients with angina or MI, which is not consistent with reports on other language versions of the MacNew Questionnaire, e.g., Chinese [31], English [10], and Flemish [32]. We have no explanation for this. On the other hand, and as expected from previous MacNew Questionnaire studies in patients with angina, MI, or heart failure [31, 32, 44], discriminative validity of the Italian MacNew Questionnaire is fully documented with HADS anxiety and depression. There are significantly lower MacNew Questionnaire HRQL scores between those patients with and without symptoms of anxiety and depression in the total group and in each diagnostic group.

The major limitation of the present validation study of the MacNew questionnaire is the cross-sectional design of the parent study, the HeartQoL Project [18], which does not allow for the determination of responsiveness, i.e., evaluative validity. In addition, the location of the three HeartQoL sites in northern Italy may limit the generalization of the results to patients in the southern parts of Italy. Strengths of the data in this study include the minimal missing data rate meaning that MacNew Questionnaire Global and subscale scores could be calculated for each of the 298 patients, and the absence of important floor and ceiling effects on the subscales allows measurement of change, both deterioration and improvement, in patient-reported HRQL. Finally, with the MacNew Questionnaire being completed in <10 min on average [45], both respondent and staff burden are minimized.

Conclusions

In conclusion, the Italian version of the MacNew Questionnaire demonstrates satisfactory psychometric properties. The instrument is reliable and valid in Italian-speaking patients in each of the three major IHD diagnoses, angina, MI, and heart failure. Future research needs to determine the responsiveness of the Italian MacNew Questionnaire.