Introduction

Life threatening and chronic illnesses such as cancer can cause uncertainty about the future, interfere with coping and relationships [1–10]. Historically, psychosocial research has focused on cancer’s negative consequences, but a growing literature documents positive life changes including greater appreciation of life, changed priorities, closer relationships, and enhanced spirituality [11–23]. A broad conceptualization of reaction to illness, taking into account both positive and negative outcomes, has been advocated [24]. However, with few exceptions [25–28], existing measures target positive and negative life changes following cancer independently (e.g., Benefit-Finding scale [23, 29], COPE [30], Impact of Events Scale, IES [31], Perceived Benefit Scales [32], Stress-Related Growth Scale, SRGS [33], and Post-Traumatic Growth Inventory, PTGI [34]). Examining the full spectrum of the impact of the cancer experience by including both negative and positive perspectives helps to minimize potential socially desirable or self-enhancement response biases [23, 35]. Attempts have been made to address these concerns by including positive and negative items in scales such as the Life Impact Scale, LIS [25], and Impact of Cancer Scale, IOC [36, 37]. The LIS captures relationships/life outlook, health behaviors, and financial situation, with over half of the items addressing physical, behavioral, or practical issues rather than strictly psychosocial concerns. The IOC emphasizes concrete experiences across multiple domains, including physical, psychological, social, and spiritual/existential impacts.

These two measures use different approaches to produce separate positive and negative impact scores. The LIS asks participants to describe the impact of cancer using a neutral item stem (e.g., your exercise activities) and positive and negative Likert-type response options (0 = “does not apply,” 1 = “very negative” to 5 = “very positive”). Sub-scales are created using the number and magnitude of positive and negative responses. The IOC uses different positive and negative items (e.g., “worry about my future,” “take better care of self”) with a 5-point Likert scale response option (1 = “strongly disagree” to 5 = “strongly agree”) and yields both positive and negative impacts of cancer scores. These scales conceptualize positive and negative effects as coexisting within the same domain and eliminate the risk of an aggregated score yielding “no impact” if positive and negative impacts cancel one another [17, 38]. However, only a measure that includes different content assessing both positive and negative impacts (e.g., the IOC) establishes a conceptual framework in which they are not two sides of the same continuum.

Building on previous research, we sought to develop a measurement framework for positive and negative psychosocial illness impacts that capture the emotional, cognitive, and behavioral responses resulting from illness experiences, across the cancer disease/treatment continuum, from diagnosis to post-treatment survivorship. The item pool we describe here was informed by patient and expert input as well as qualitative and quantitative data. Items assess positive or negative psychosocial sequelae and highlight the contribution of the illness (e.g., “My illness has helped me become a better person”). Aiming to measure a cohesive latent construct, we focused entirely on psychosocial impact, rather than including practical changes (e.g., in finances). Given that psychosocial impact is experienced by people with various chronic diseases, items were worded in a general way to permit future adaptation to other illnesses. We also explored the relationship of demographic variables and psychosocial impact upon cancer experience in this report.

Methods

Development of the psychosocial illness impact (II) item pool

In a pilot study, we examined the relationship between positive and negative psychosocial illness impacts. Results guided development of the conceptual framework tested in the main study. Both the pilot and main studies were approved by the Institutional Review Board, and all participants provided informed consent.

Pilot study

A research team, consisting of four clinical psychologists and one occupational therapist, reviewed available scales measuring the psychosocial impact to patients from their diseases. Candidate measures included the IES, SRGS, and PTGI, and permission to include and possibly modify items was obtained from the respective authors. Candidate items were rejected or modified when deemed to be outside the scope of psychosocial impact, vague, confusing, redundant, or of uncertain valence. Wording was revised to standardize items and emphasize the impact from illness (e.g., “because of my illness”). New items were written to fill identified conceptual gaps. The resulting II item pool consisted of 46 items (23 positively worded and 23 negatively worded) including 19 newly written items, 11 from the IES [31], and 16 modified from existing scales (PTGI [34], SRGS [33], and Functional Assessment of Cancer Therapy-Cognition [39, 40]).

This item pool was evaluated using data from 205 oncology patients from the Chicago metropolitan area (average age 59.6 years, 59% female, 93.2% White). The largest proportion (25.9%) was diagnosed with breast cancer, followed by colorectal (15.1%), non-Hodgkin’s lymphoma (14.1%), leukemia (6.8%), and ovarian cancer (5.9%). Average time since diagnosis was 34 months (range: 0.5–302). All participants were receiving at least one treatment when this study was conducted (90.2% chemotherapy, 9.3% hormone therapy, and 5.9% radiation). The majority of patients were able to perform normal activities with either no (29.8%) or some (51.7%) symptoms.

Unidimensionality of items was examined using exploratory factor analysis (EFA) with varimax rotation (criterion: factor loading > 0.3; i.e., 2-tailed test, p < 0.01); Rasch analysis [41] was used to calculate item fit statistics (criterion: MnSq < 1.4). EFA results showed that all positive items had significant loadings (0.43–0.89) on Factor 1 while 19 negative items loaded on Factor 2 (0.30–0.79) and 4 negative items did not have significant loadings on either factor. We therefore analyzed negative (NII) and positive (PII) items separately using Rasch analysis. Six items did not meet the fit statistics criterion. Based upon results from both EFA and Rasch analysis, 8 items (2 positive and 6 negative) were removed from the pool. Both PII and NII scales had acceptable internal consistency (alpha = 0.95 and 0.93, respectively) and discriminated patients by the Eastern Cooperative Oncology Group performance status rating (ECOG PSR) (F (3,201) = 2.74, p < 0.05 for PII; F (3,201) = 10.03, p < 0.05 for NII). The correlation between the two scales was 0.071, p = 0.310 [42, 43]. These results suggest that negative and positive psychosocial illness impacts are two distinct concepts. We therefore chose to enhance PII and NII scales by assessing them separately in the current study.

Main study

Item pool generation

A clinician/investigator panel consisting of clinical psychologists and patient outcome researchers was formed to guide the development of the II item pool for the current study. Four psychosocial illness impact categories, representing both PII and NII, were identified through results from the pilot study, literature, and thematic review of definitions: Coping (PII)/Stress Response (NII), Social Connection (PII)/Social Isolation (NII), Self-concept (PII & NII), and Meaning and Spirituality (PII & NII). Item generation and classification into categories were conducted using the following steps. Panel members independently grouped items from the pilot study, and those generated from literature review, into the respective categories and later reached consensus on items for which there was disagreement. Cognitive interviews were conducted with cancer patients (N = 15) for their input on study themes and items. Feedback on Meaning and Spirituality was gathered from a team of chaplains to augment areas of expertise represented in the panel. After reviewing this input, the panel refined content area definitions and wrote items to cover content areas that were thought missing during the discussion. Items that were redundant, inapplicable, or vague were modified or excluded from the pool. As a result, a total of 89 items (43 positively worded and 46 negatively worded) were included for the field testing. Table 1 shows the conceptual model and example items within each category.

Table 1 Psychosocial illness impact conceptual model
Full size table
Sample

Participants (N = 754) were recruited from (1) oncology clinics in the Chicago metropolitan area (n = 249); (2) local and national support societies (CancerCare, The Wellness Community & Gilda’s Club) via internet (n = 256); and (3) an internet survey panel company (n = 249). Diagnoses and treatment information of groups 2 and 3 were collected via self-report. Eligibility criteria included a diagnosis of cancer and English fluency. Participants were primarily White (97.4%), women (61%), married, or living with a partner (61.2%) and had at least some college education (80%) and a household income higher than $50,000 (55.2%). They had an average age of 57.41 years (SD = 13.37); 30.59% were employed full time and 29.26% were retired. Participants had a wide range of cancer diagnoses: 29.3% breast, 9.9% colorectal, 8.9% prostate, 8.8% urological, and 8.1% gynecological. The average time since cancer diagnosis was 4.1 years (SD = 6.4) with 32.6% of participants diagnosed less than 1 year and 33.2% more than 5 years prior. A majority of participants (53.9%) received chemotherapy and 18.4% received radiation therapy; 23.2% had experienced a cancer recurrence. Specifically, 34% were receiving active treatment (9% with early stage, 13% with advanced stage, and 12% with unknown stage disease) and 62% were post-treatment (33% within 5 years of diagnosis and 29% more than 5 years since diagnosis). Most participants reported either no symptoms (49.8%) or being ambulatory with some symptoms (35.6%).

Analysis

We first evaluated dimensionality of items and used SAS (v9.1) [44] to perform the EFA and MPlus (v5.0) (Los Angeles, CA: Muthen and Muthen) for confirmatory factor analysis (CFA) and bi-factor analysis. Data were randomly divided into two datasets: Dataset 1 (n = 376) for EFA and Dataset 2 (n = 378) for CFA and bi-factor analysis. Using EFA, we determined the factor structure using the following criteria: (1) number of factors with eigenvalue > 1 [45]; (2) number of factors before the break in a scree plot [46]; and (3) factors that have explained variance > 5% [47]. Items with loadings greater than 0.4 were considered significant. Acceptable CFA results were indicated by Tucker-Lewis Index (TLI) [48] and comparative fit index (CFI) values of 0.90 or greater, root mean square error of approximation (RMSEA) less than 0.1, and item R-square > 0.3.

The II measurement model (Fig. 1) was determined based on the results of the bi-factor analysis, which better represents the current framework by capturing the general and sub-domains simultaneously. A bi-factor analysis is hypothesis-driven, consisting of a general factor (i.e., N-II and P-II) and local factors (4 sub-domains under the umbrella of N-II and P-II). The general and local factors are modeled as orthogonal and therefore the relationship between items and the general factor is not constrained to be proportional to the relationship between the first- and second-order factors as demonstrated in other hierarchical CFA. When the general factor explains covariance between items, uniformly high standardized loadings (i.e., >0.3) are seen for the general factor, indicating that it is appropriate to report a single score. Yet, investigators can choose to report sub-scale scores as tested in the hypothesized model. If the sub-domains represent demonstrably separate constructs, loadings on the general factor will not be uniformly high, leading one to reject the conclusion that the items are sufficiently unidimensional, making it more appropriate to report scores of sub-domains separately, but not a single overall score [49–51].

Fig. 1
figure 1

Measurement model used for bi-factor analysis

Full size image

Lastly, using analysis of variance (ANOVA), we evaluated whether significantly different psychosocial impact exists across groups or levels of the following variables: gender, education, years since diagnosis, household incomes, and functional performance (see Tables 2, 3 for categories). The relationship between psychosocial illness impact and age was evaluated using Pearson correlations. We explored whether certain demographic and clinical variables were associated with different magnitudes of positive and negative psychosocial impacts.

Table 2 ANOVA results for general positive psychosocial impact and its four components (i.e., local factors)
Full size table
Table 3 ANOVA results for general negative psychosocial impact and its four components (i.e., local factors)
Full size table

Results

Evaluation of dimensionality

All NII items were reversed to the same direction as PII items. Thus, higher PII and NII scores both represent favorable (more positive and less negative) psychosocial illness impact. EFA results showed 11 factors had eigenvalues > 1 and only two of them had explained variance > 5%. The scree plot (Fig. 2) clearly shows two factors before the break. All PII items had significant loadings on Factor 1 (0.49–0.91) and Factor 1’s correlations with Factors 2, 3, and 4 were negligible (r = 0.09, 0.10, and −0.09, respectively). In terms of NII, Self-concept and Social Isolation items both loaded on Factor 2 (0.46–0.99). Stress Response items loaded on either Factors 2 or 4, with 5 of those 13 items having loadings greater than 0.4 on both factors. Negative Meaning and Spirituality items demonstrated significant loadings on either Factors 2 or 3. Item “Because of my illness, I am less able to attend usual religious services” did not show a significant loading on any of these factors. This item was excluded from further analyses. Moderate inter-factor correlations were found between Factor 2 and both Factors 3 and 4 (r (F2–F3) = 0.46; r (F2–F4) = 0.43, respectively), suggesting that NII items might be represented by a single factor.

Fig. 2
figure 2

Scree plot from the results of EFA analysis on dataset 1 (n = 376)

Full size image

Using Dataset 2, acceptable fit indices (CFI = 0.96; TLI = 0.96) were obtained from a 2-factor CFA where PII and NII were assigned to be two independent factors. We then conducted another 2-factor CFA, in which PII and NII factors were presumed correlated (producing CFI = 0.863, TLI = 0.951). That analysis also indicated that the correlation between PII and NII factors was negligible (r = 0.076). Thus, we concluded that PII and NII should be treated as two independent factors.

In bi-factor analysis, the general factor was defined as psychosocial illness impact (either PII or NII), and local factors were Coping/Stress Response, Social Connection/Social Isolation, Self-concept, and Meaning and Spirituality (Table 1). Both PII and NII were tested using the same hypothesized model but were analyzed separately given their negligible correlation. Bi-factor analysis showed all items had significant loadings on the general factor, with factor loadings ranging from 0.610 to 0.882 for PII and from 0.492 to 0.891 for NII. Acceptable fit indices were found for NII (CFI = 0.952, TLI = 0.989, RMSEA = 0.067) while borderline acceptable fit indices for PII (CFI = 0.929, TLI = 0.992, RMSEA = 0.105), confirming that the hypothesized model is valid for both. These results support the sufficient unidimensionality of PII and NII items, and two summation scores should be produced (PII & NII) with the recognition of four sub-domains represented by local factors.

When comparing item loadings between general and local factors, some items showed significant loadings on both. Loadings on the general factor were larger than those on all local factors except Stress Response and positive and negative Meaning & Spirituality. Specifically, Meaning and Spirituality items demonstrated loadings on the general and local factors that were of similar magnitudes. Likewise, 7 of 14 Stress Response items had loadings greater than 0.4 on both general and local factors. Six of these items used a frequency rating scale (“none of the time” to “all of the time”) and five (all from the IES) used a unique time context (“in the past 7 days”), whereas other Stress Response items used an intensity rating scale (“not at all” to “very much”) and no specific timeframe. These item format artifacts may have contributed to the cross-loading of these Stress Response items, but their high loadings (0.497–0.619) on the general factor suggest that these items characterize general negative psychosocial illness impact.

To confirm this, we conducted two additional bi-factor analyses. Five PII items, which had similar loading on both general and local factors or slightly higher loading on the local factor, were treated as a separate local factor. Results showed that although all PII items now had higher loadings on the general factor than on the local factor, a borderline CFI (0.89) was obtained, suggesting that including these five items in the originally assigned local factor produces a better fit. When cross-loading NII items were separated from their original local factors and treated as two new local factors, fit statistics were similar (CFI = 0.948, TLI = 0.987, RMSEA = 0.072) and the items had comparable loadings on both general and local factors. Therefore, from a measurement perspective, these items could be either included or excluded from the general factor. We opted to keep items in their original local factors for the sake of parsimony and content coverage.

PII and NII both demonstrated excellent internal consistency (alpha = 0.98 and 0.97, respectively). Item-scale correlations ranged from 0.61 to 0.84 for PII and from 0.45 to 0.74 for NII. Correlations within sub-domains are reported in Fig. 1, in which high Spearman’s rho values for the sub-domains are found within (range: 0.755–0.920 for PII; 0.659–0.767 for NII) but not across PII and NII (range: 0.003–0.157). Results again confirmed sufficient unidimensionality within, but not between, the NII and PII item pools.

Variables associated with illness impact

The following analyses were conducted using both general factors and each local factor. Age was significantly (p < 0.001) correlated with PII and its sub-domains in a negative direction, with coefficients ranging from −0.140 (Coping) to −0.230 (Social Connection). In contrast, age was significantly correlated with NII (which was reverse scored) and its sub-domains in a positive direction, with coefficients ranging from 0.246 (negative Self-concept) to 0.359 (Stress Response). These results indicate that older participants reported both less positive and negative psychosocial impacts of illness.

ANOVA results showed that gender, income, education, time since diagnosis, and performance status were significantly (p < 0.01) associated with illness impact as a whole as well as its individual components. Results are summarized in Tables 2 (PII) and 3 (NII), and comparisons of group means are depicted in Figs. 3a–d (PII) and 4a–d (NII). Tukey’s post hoc comparisons showed that women reported more positive and less negative psychosocial illness impacts as reflected in scores on all sub-domains except negative Meaning and Spirituality, in which no significant differences were found. Likewise, participants with lower income reported more PII and less NII. Participants with less formal education reported more PII; however, no significant differences related to educational level were found for NII.

Fig. 3
figure 3

Mean comparisons across sub-groups for positive psychosocial impact (P-II) as well as its local factors (P-SEW self-concept/esteem/worth, P-SC Social Connection, P-Coping Coping, P-MS Meaning & Spirituality)

Full size image
Fig. 4
figure 4

Mean comparisons across sub-groups for negative psychosocial impact (N-II) as well as its local factors (N-SEW self-concept/esteem/worth, N-SI Social Isolation, N-SR Stress response, N-MS Meaning & spirituality

Full size image

Years since cancer diagnosis were significantly associated with NII but not PII, except for positive Social Impact. Specifically, results suggest that both NII and PII, in general, and positive social impact diminish over time. Lastly, participants with higher functional performance reported less PII in sub-domains of Social Impact and Meaning & Spirituality and less NII across all sub-domains.

Discussion

Both positive and negative psychosocial impacts of illness are undeniable realities for most patients throughout their experiences with cancer and other life threatening illnesses. As conceptualizations of psychosocial adaptation in chronic illness and subsequent interventions increasingly acknowledge that positive and negative sequelae can coexist at different levels within the same sub-domains or content areas, comprehensive measures are needed in order to more fully capture patients’ experiences. Such measures should be applicable throughout the cancer care continuum so that patients’ scores can be compared over time, as they progress through different disease and treatment phases.

Recent measures have included items assessing both positive and negative sequelae. Additionally, measures should move away from simple problem checklists and instead assess latent constructs. In this study, we sought to develop a measurement framework to capture positive and negative psychosocial effects across the disease/treatment continuum. Of particular interest was the dimensionality of the psychosocial illness impact construct and how positive and negative sequelae related to one another and to other patient variables.

We created our psychosocial illness impact item pool using expert and patient input, as well as qualitative and quantitative data, at various points in the development process. Results of our pilot study established the foundation for the current study. We developed and tested a conceptual framework of positive and negative illness impacts (NII & PII), consisting of four content areas as shown in Fig. 1: Coping/Stress Response, Social Connection/Social Isolation, Self-concept, and Meaning and Spirituality. Bi-factor analysis results verified this framework supporting one single P-II or N-II score. However, sub-domain-specific scores can be reported in a psychometrically sound manner if investigators are interested in specific sub-domains. Our results provide an empirical basis for further development of a comprehensive measurement model in which positive and negative illness impacts are two distinct factors each comprised of four sub-domains corresponding to the content areas mentioned above. While these sub-domains represent important, conceptually distinct content areas, the prevailing driver of the patients’ reports relates to whether the illness impact is positive or negative. This is consistent with the common observation that general positive and negative effects are also essentially uncorrelated [25–27].

Results of our study suggested that PII and NII are two independent concepts, are each sufficiently unidimensional, and can adequately be represented by two summation scores. Differential associations with sociodemographic and clinical variables between PII and NII further confirmed our results. For example, more positive impact was reported by study participants who were younger, female, and had lower income and education levels. More negative impact was also reported by study participants who were younger, female, and had lower income levels, as well as by participants who were more recently diagnosed and had poorer functional status. This is consistent with the literature in which, compared to older cancer survivors, younger survivors report greater positive benefits in their lives due to the cancer experience [14, 26, 37] while also reporting greater negative life disruption [25, 52, 53]. Although a cancer diagnosis and treatment can be exceedingly stressful, most individuals adapt well over time [54]. Among patients who complete treatment and whose cancer is in remission or otherwise controlled, their psychological adjustment and quality of life is comparable to the general population by 1–2 years post-diagnosis [7, 55, 56]. These findings are consistent with our findings that more recently diagnosed individuals had worse negative impacts than those who were further out from treatment.

Other findings present a more equivocal picture when compared to existing research. For example, a gender difference in cancer experiences is not consistently supported by the cancer literature [37, 57–60]. Regarding measures of socioeconomic status (e.g., income and education), associations with positive impact are often mixed, revealing positive, negative, and no associations (see Stanton et al. [60] for a review). In contrast, associations with negative impact and income level are more consistent and are likely impacted by concurrent factors such as limited access to healthcare, poor or no insurance coverage, and underutilization of services. Zebrack and colleagues [37] have noted a significant link between lower income and greater negative impact from the cancer experience. Although the exact relationship between some sociodemographic variables and illness impact remains to be clarified, our results, taken together, corroborate a framework in which negative and positive psychosocial illness impacts are independent factors, not opposite ends of a spectrum.

We consider the current study to be a starting point for future work that carefully examines how best to conceptualize and measure the psychosocial impact of illness, but it is not without limitations. First, because the II items emphasize the contribution of the illness (e.g., “My illness has helped me…” or “Because of my illness…”), the measure requires that respondents make attributional judgments; how this affects scores has not been evaluated. A qualitative study to evaluate patient perception of various formats of similar item content is warranted. Second, the term “impact” implies a degree of causality that we are unable to determine from this cross-sectional dataset. A longitudinal study tapping patient experience from diagnosis to long-term survivorship can help model the experience trajectory in a more precise manner. Third, the population is a convenience sample and the majority of participants were White. The degree and type of illness impact experiences may vary as a function of race/ethnicity; therefore, future studies should include more diverse samples. Fourth, although we sought to include participants across the entire disease/treatment continuum, we did not stratify recruitment based upon these criteria (e.g., newly diagnosed, currently on-treatment, post-treatment). Future development in this area should determine its suitability for use across the treatment continuum, which will be paramount in longitudinal studies tracking psychosocial illness impact over time.

In the current study, we gathered input from patients to improve item content coverage and format as well as refined the measurement framework. This measurement framework informed the subsequent development of a calibrated psychosocial illness impact item bank that is part of the NIH Patient-Reported Outcomes Measurement Information System (PROMIS; www.nihpromis.org) and beyond the scope of this paper. In future studies, we will expand the current item pool and calibrate illness impact item banks, using Item Response Theory (IRT) models. Although comprehensive, the item pool is too long to be administered in clinical research or practice. Yet, applications from an IRT-calibrated item bank such as dynamic computerized adaptive tests and static short-form measures can overcome this barrier by only selecting the most informative items targeting to the persons’ levels on the latent trait at the time of survey. These applications can help clinicians and researchers better understand patients’ cancer experiences longitudinally [61–65] and assist with the development of interventions to minimize negative and maximize positive psychosocial impacts of illness.