Introduction

Many patient-reported outcomes (PROs) ask patients to rate aspects of their health and well-being in evaluative terms. Interpretation of these measures is problematic because they are subject to individual differences and intra-individual changes in the interpretation of items, the salience of relevant experiences, and standards of comparison [1, 2]. In order to account for these differences, Rapkin and Schwartz [3, 4] proposed a model of quality of life (QOL) appraisal, which views the QOL score at any given time as a function of four sets of parameters: the individuals’ frame of reference, recall and sampling of experiences, evaluative comparators, and algorithm for reconciling discrepant experiences. Over a series of studies, Rapkin et al. have shown that measures of appraisal mediate and moderate the impact of illness and treatment on PROs [5,6,7,8,9,10,11,12,13]. In other words, the influence of stressors like injury, comorbidity burden, or financial difficulties on evaluative ratings of pain or financial distress depends on the ways that people understand and think about their QOL.

Detecting and accounting for such inter- and intra-individual differences in appraisal can be important for understanding the impact of health state change. In cross-sectional studies, characterizing appraisal can highlight underlying differences in how people think about QOL that impact or obfuscate score differences between groups [8]. In longitudinal studies, characterizing appraisal changes over time allows one to examine and quantify response-shift effects, reflecting adaptation to changing health [3]. Appraisal assessment can help to portray individual differences in terms that not only depict how thinking influences experience, but also may be more amenable to clinical intervention than standard PRO scores [8, 14, 15]. Person factors, such as personality [16,17,18], perseverance [19, 20], and social support [21,22,23] may also be relevant to appraisal patterns as well as to resilience to health challenges. We expect that the ways in which appraisal is influenced by person factors will be affected by health status. For people with greater burden of chronic illness, appraisal may be dominated by health concerns or reflect accommodations that individuals have had to make in light of functional decline. Conversely, those with fewer chronic conditions may be comparatively less focused on health concerns and more focused on avoiding compromises in other life domains. Of course, the interplay of health and illness, demographics, and person factors will combine to influence how individuals experience and appraise their health-related quality of life.

As a complex, multi-faceted domain, assessment of appraisal has relied on in-depth, descriptive measures that are not feasible for use in most PRO studies. Research spanning over a decade has utilized the QOL Appraisal Profile (QOLAP), a comprehensive self-report measure that includes both qualitative and quantitative measures of the four appraisal parameters [3]. Data generated by the QOLAP have been critical to demonstrating the relevance of appraisal to QOL research [5, 10, 24]. This assessment procedure requires, however, elicitation and coding of qualitative responses based on a standardized coding protocol to generate data for subsequent quantitative analysis. Multiple coders are required for establishing inter-rater reliability prior to hypothesis-testing data analysis. In addition, the QOLAP uses several formats and response sets for quantitative items and requires complex procedures for assessing change in appraisal over time. There was thus a clear need to develop a more practical appraisal measure that could facilitate appraisal assessment in a wider range of situations.

We thus developed the Brief Appraisal Inventory (BAI), a brief and close-ended tool that was easier to administer and score than the original QOLAP. Items for the BAI were developed over a series of analyses using QOLAP data from four large samples (bladder cancer [6], HIV/AIDS [5], multiple sclerosis [11], spinal surgery [25]). These analyses were used to characterize the essential themes reflected in the qualitative data [24, 26], and supported writing new items to replace the qualitative questions and hone existing close-ended items. Items were reviewed for comprehensiveness and clarity with a panel of Stakeholders including clinicians, policy makers, patient advocates, and QOL researchers. The interim result was the QOLAP-version 2 (QOLAP-v2), which comprised a pool of 83 items summarizing themes of appraisal identified in the original QOLAP [18]. The QOLAP-v2 item pool was included in an online survey of over 4000 patients with a wide variety of chronic conditions, the access to whom was provided by Rare Patient Voice LLC. First- and second-order principal components analysis of these items yielded 13 composite scores reflecting patterns of appraisal. Rather than simply selecting a subset of original items for the BAI, new items were written to capture overarching patterns of appraisal, each encompassing different combinations of appraisal parameters based on second-order components results. For example, the item, “preparing family for the ups and downs of health concerns” involves health-related frame of reference, variability of recent experiences, and salience of the impact of one’s health on the family. After reviewing and refining these new composite statements with Stakeholders, the decision was made to capture more complex second-order components with two separate items, for the sake of clarity. The end result was the 23-item BAI.

The purpose of the present work was to (1) evaluate the item distributions and structure of the BAI; (2) examine the explanatory power of appraisal as compared to personality in predicting health-related QOL by comorbidity-burden group; and (3) investigate how demographic, person factors, and personality relate to appraisal patterns in a heterogeneous sample of people with chronic illness.

Hypotheses

We hypothesize that the inclusion of the BAI will explain more independent variance in physical and emotional functioning among people with chronic illness than demographic, person factors, and personality. We expect to find differential associations among demographic, person, and personality factors and QOL appraisal in people with low, medium, and high comorbidity disease burden. Consistent with the QOL Appraisal model, we expect that the impact of exogenous influences on QOL will be both mediated and moderated by processes of appraisal. We hypothesize that appraisal mediates the influences of demographic, personality, cognitive, and social support measures on QOL.

Methods

Sample and design

This cross-sectional study recruited people from the participant panels of Rare Patient Voice, LLC, and WhatNext, recruiting people affected with rare diseases and cancer, respectively. Respondents self-identified as patients, caregivers, or both. Caregivers were included because they also have health challenges in addition to providing caregiving support (see Supplementary Text for more background on Rare Patient Voice, LLC). Eligible participants were age 18 or older, and able to complete an online questionnaire.

Procedure

The study protocol was reviewed and approved by the New England Institutional Review Board (NEIRB #15-254). Cognitive interviews were implemented with stakeholders to ensure the clarity of the BAI items. The honed BAI items were then included in a web-based study, which was administered using the HIPAA-compliant, secure SurveyGizmo engine (http://www.surveygizmo.com). We followed study procedures described by Dillman’s Tailored Design Method [27] to maximize response. Respondents were not paid for participation in the study.

Measures

Appraisal was measured using the BAI, a 23-item Likert-scaled measure that asks respondents how often they thought about the specified appraisal approach when completing the online QOL survey (1 = never, 2 = rarely, 3 = sometimes, 4 = often, 5 = always). Health-related QOL was measured using the PROMIS-10, a brief measure of general physical and emotional functioning [28]. Comorbidities were assessed using the Self-Administered Comorbidity Questionnaire [29]. Demographic characteristics collected included year of birth, gender, cohabitation/marital status, with whom the person lives, employment status, annual income categories, and whether the respondent endorsed having difficulty paying bills [30]. Person factors that may be relevant to appraisal were measured by subscales from the DeltaQuest Reserve-Building Measure [31]: the Perseverance subscale assess one’s tendency not to give up despite challenges; and the Past and Current Social Support subscales query how much the respondent’s past or current social networks provided substantive support and help in dealing with problems. Personality was measured by the Big Five Inventory-10, a 10-item measure of the NEO-Five Factor Model of Personality [32].

Statistical analysis

We began by examining frequency distributions of BAI items, as well as missing data patterns. We then examined the inter-correlation structure of the BAI using principal components analysis. This approach followed a successful data reduction strategy from earlier appraisal studies as a way of minimizing the number of statistical comparisons and Type I error rate [5]. Although this analytic approach does lose some information, it keeps the strongest signal. Recall that BAI items were based on orthogonal second-order components analysis of our original item pool, and so would not necessarily share common variance. We did not want to omit potentially important items because they were not correlated with other items, as would occur if we used factor analysis. On the basis of principal components analyses, composite scores were created. Descriptive statistics of all measures used were then computed, as well as Pearson inter-correlations among all measures. We report results using Cohen’s criteria for delineating small (0.10 < r < 0.30), medium (0.30 ≤ r < 0.80), and large (r ≥ 0.80) effect sizes, hereafter referred to as small, medium, and large correlations [33].

To examine these potential differences in person factors influencing appraisal, we created three comorbidity-burden groups: those with no comorbidities other than the index condition leading to their involvement in the Rare Patient Voice or WhatNext panel (no comorbidity burden); those with 1–3 comorbidities (moderate comorbidity burden); and those with four or more comorbidities (high comorbidity burden). Regression models were then computed separately for each stratum, considering demographic, person factor, and personality predictors of appraisal-domain scores.

We then implemented two sets of hierarchical regression models with forward stepwise selection to examine predictors of QOL. In the first set, predictor domains included (a) demographic variables (age, sex, education); (b) comorbidity variables, and (c) BAI appraisal-domain scores. Interactions of the appraisal measures with demographic and comorbidity variables were added after testing main effects. This hierarchical approach allowed us to examine the independent and incremental explanatory power of the BAI. For comparative purposes, the second set of models used standard person variables instead of appraisal measures on step (c). We also examined the incremental predictive contribution of appraisal variables as step (d), after demographic, comorbidity, and person factors were entered. These two sets of predictive models were evaluated for two different dependent measures, physical and emotional functioning. Finally, in a separate set of hierarchical regression models, we sought to understand predictors of appraisal. These models considered (a) demographic, (b) comorbidity burden, and (c) person factors as predictors of each appraisal measure. Statistical analyses were implemented using SPSS 24 [34] and Stata 14 [35].

Results

Sample

The study sample comprised 592 people, of whom 446 were patients, 103 were caregivers, and 43 were both patients and caregivers (Table 1). The sample had a mean age of 43.8 (SD = 18.5), and was 79% female. The majority of the sample was married, and most lived with their spouse/partner and/or other relative. The median income was in the range of $50,000–$100,000, and 70% percent of the sample indicated that they had somewhat to extreme difficulty paying bills. The most prevalent comorbidities were back pain, depression, and insomnia. The sample contained 60 people with no comorbidity burden (37 patients, 23 caregivers); 301 people with moderate (1–3) comorbidity burden (245 patients, 56 caregivers); and 231 with high (4+) comorbidity burden (207 patients, 24 caregivers).

Table 1 Demographic characteristics of study sample (n = 592)
Full size table

Distributional and structural properties of the BAI

BAI items were piloted with five volunteers utilizing “think aloud” cognitive interview techniques [36,37,38] to ensure that the items convey the intended meaning to different patients (see Supplementary Text for more background on cognitive interview results).

Frequency distributions

A missing-item analysis revealed 95.1% of respondents completed 19 or more items of the 23-item BAI. The remaining 4.9% of respondents that completed 18 or fewer items were omitted from the analysis. The sporadic missing items were generally not applicable and were thus treated as something that they never thought about on the response scale (i.e., lowest score). The final sample included 563 respondents who responded to 19 or more items. Participants dropped from the analysis due to missing BAI data tended to be patients and males, and there was trend that they were living with a spouse or partner (Χ 2 = 13.43, 7.48, 3.23; p = 0.001, 0.006, 0.07, respectively). Frequency analysis revealed that respondents used the full range of the response scale on every item. 52% of items had a negative skew towards the “always” response item compared to 48% of items that had a positive skew towards the “never” response item. The mean response for individual items ranged from 2.0 to 3.7, with an overall mean of 2.9 for all items (Supplemental Table 1).

Component structure

The 23 items demonstrated small to medium inter-item correlations (Supplemental Table 2). The principal components analysis with varimax rotation yielded five components that explained 59.7% of the total variance (Table 2). Item communalities ranged from 46 to 76%. The five components were characterized as follows: (1) Health Worries; (2) Interpersonal and Independence concerns; (3) Accomplishing Goals and Problem-Solving; (4) Calm, Peaceful, and Active; (5) Spiritual Growth and Altruism. Table 3 provides descriptive statistics on the appraisal factor scores along with all other measures used in the study.

Table 2 Results of Brief Appraisal Inventory principal components analysis (varimax rotation)
Full size table
Table 3 Descriptive statistics of patient-reported outcomes
Full size table

Bivariate relationship of appraisal scores with other PROs

Table 4 shows the correlation coefficients of appraisal component scores with demographics, comorbidities, health-related QOL, person factors, and personality. Health Worries had the most and largest correlations with the other constructs measured, showing medium correlations with worse physical and emotional functioning. Small correlations indicated that people who endorsed health worries tended to be women, younger, and sicker, with low past social support, low conscientiousness, and higher neuroticism. Interpersonal and Independence concerns had small correlations with physical functioning, and people endorsing this appraisal pattern tended to be sicker and have lower levels of education. Accomplishing Goals and Problem-Solving was not associated with QOL subscales despite small correlations with QOL items. People endorsing this appraisal pattern were younger and male, less likely to have cancer and more likely to endorse depression, had lower current social support, and were less conscientious. Calm, Peaceful, and Active had small correlations with better physical functioning, and people endorsing this appraisal pattern had fewer comorbidities, were less likely to endorse back pain, had higher levels of perseverance and current social support, and were more agreeable and conscientious and less neurotic. Spiritual Growth and Altruism was unrelated to the QOL subscales, and people endorsing this appraisal pattern tended to be female, with lower levels of education, and higher perseverance scores.

Table 4 Inter-correlations among appraisal and sample characteristics
Full size table

Explanatory power of person factors in predicting appraisal

Regression models were then computed separately for each comorbidity-burden stratum suggested that different person factors predicted the appraisal composites as a function of comorbidity burden (see Table 5 for summary; see Supplemental Table 3 for full results). The Health Worries pattern was associated with low conscientiousness among people with no comorbidities, but with high neuroticism among people with moderate and high comorbidity burden. Interpersonal and independence concerns was associated with having more social support among people with no comorbidities, but with high agreeableness and low education in those with moderate comorbidity burden; and with low agreeableness in those with high comorbidity burden. Accomplishing goals and problem-solving was associated with low agreeableness among those with no comorbidity burden, but with low social support, being younger, and male gender among the moderate comorbidity-burden group; and with low social support, being younger, and low conscientiousness in the high comorbidity-burden group. Calm peaceful active was not associated with any person factor in the no-comorbidity group, but was associated with high agreeableness and high social support in the moderate comorbidity-burden group; and with low neuroticism and high openness in the high comorbidity-burden group. Spiritual growth and altruism was associated with low openness in the no comorbidity-burden group, with high perseverance and high neuroticism in the moderate comorbidity-burden group, and with high agreeableness and low education in the high comorbidity-burden group.

Table 5 Summary of person factors predicting appraisal component scores
Full size table

Explanatory power of appraisal versus personality in predicting health-related QOL

Hierarchical regressions predicting physical functioning revealed that appraisal components alone explained 22% of the variance in physical functioning, as compared to 8% explained variance by personality alone (Supplemental Table 4). The penultimate models (Model 4) comprising demographics, comorbidities, appraisal, and their interactions showed that appraisal uniquely explained 8% of the variance in physical functioning, as compared to 4% explained by personality (Supplemental Table 4). This penultimate appraisal model suggested that people who reported worse physical functioning tended to endorse Health Worries or Interpersonal and Independence concerns, after adjusting for covariates. People who endorsed Calm Peaceful and Active appraisal patterns tended to have higher physical functioning, after adjusting for covariates. There were no significant interactions.

In contrast, the final personality model suggested that people who reported better physical functioning tended not to endorse neuroticism or agreeableness, and tended to endorse higher conscientiousness. There were significant interaction effects such that males higher in conscientiousness reported higher physical health than would be expected, and people with liver disease who also endorsed being higher in agreeableness or neuroticism has worse physical functioning than would be expected, after adjusting for covariates.

Results of hierarchical regressions predicting emotional functioning revealed that appraisal components alone explained 19% of the variance in emotional functioning, as compared to 24% explained variance by personality alone (Supplemental Table 5). However, the penultimate models (Model 4) comprising demographics, comorbidities, and appraisal showed that appraisal and its interactions uniquely explained 11% of the variance in emotional functioning compared to the 8% uniquely explained by personality variables and interactions (Supplemental Table 5). This appraisal model suggested that people who reported better emotional functioning tended to endorse Accomplishing Goals and Problem-Solving, but tended not to endorse Calm Peaceful Active, after adjusting for covariates. Appraisal composites had interaction effects with gender, education, depression, and liver disease as comorbidity, after adjusting for covariates. Specifically, males who endorsed Health Worries or Accomplishing Goals and Problem-Solving reported worse emotional functioning. People with higher education who endorsed higher Accomplishing Goals and Problem-Solving reported worse emotional functioning. People with depression who endorsed higher importance of Calm Peaceful Active reported better emotional functioning than would be expected. People with liver disease who endorse higher Interpersonal and Independence reported better emotional functioning than would be expected. In contrast, the penultimate personality model (Model 4) suggested that people who reported better emotional functioning tended not to endorse neuroticism, and tended to endorse conscientiousness, after adjusting for covariates. There were no significant interaction effects. The final models (Model 5 in Supplementary Tables 4 and 5) considered all person factors, appraisal, and personality and showed that appraisal explained an additional 5% of the variance in physical functioning and an additional 5% of the variance in emotional functioning than that explained by demographic, comorbidities, person factors, and personality.

To get a better sense for potential moderating effects that could be masked in the regression analysis, we examined differences in correlations between appraisal and physical functioning across the three comorbidity subgroups (Supplementary Table 6). Comparisons among correlations showed that Health worries had a similar negative association with physical functioning across all groups; that individuals in the 1–3 comorbidities group who expressed Interpersonal and Independence concerns reported worse physical functioning; and that prioritizing calm, peaceful, and active had a positive influence on physical functioning in the group with 4–6 comorbidities. Thus, moderation effects involving appraisal may be present but masked by large appraisal main effects in this sample.

Discussion

The present study supports the importance of appraisal for interpreting QOL PROs. The appraisal items evidence good distribution of responses and the appraisal components’ construct validity is supported by their zero-order correlations with demographic, health, personality, and health-related QOL measures. The five appraisal patterns showed distinct relationships with these measures. For example, Health Worries was associated with worse physical and emotional functioning, high neuroticism and low conscientiousness, and lower reported past social support. Comparisons of the correlates of appraisal among subgroups reporting different levels of comorbidity shed light on the role that appraisal may play in adapting to chronic illness. First, different personality styles played a role in appraisal processes for the three comorbidity subgroups. For example, the Interpersonal and independence appraisal component reflects respondents’ concern about the impact of their illness on others in their lives. As such, correlates of this pattern differed markedly among groups. Most notably, agreeableness was positively associated with this concern among people with 1–3 comorbidities but negatively correlated among those with four or more comorbid illnesses. It is plausible that among the group with fewer comorbid problems, more agreeable patients are reticent to place demands on others, while in the group with more problems, more agreeable individuals may have already worked through issues about the impact of illness on others. In the group with no comorbidities, Interpersonal and Independence concerns were greatest among those reporting more current social support, likely indicating greater sociability and involvement in relationships. These differences warrant further exploration regarding interpersonal relationships and chronic illness (see Supplemental Text for a more detailed discussion).

The major impetus for the development of appraisal measures is to improve our ability to account for individual differences in QOL. Findings here show that the performance of the BAI in this regard is comparable to results obtained with earlier measures [5, 18, 24, 26]. Three orthogonal appraisal measures alone—Health Worries, Interpersonal and Independence concerns, and Focus on Calm, Peaceful, and Active—explain 22% of the variance of physical functioning, and add 5% to variance explained by demographic, comorbidities, person factors, and personality. By way of comparison, personality measures alone explain only 8% of the variance of physical functioning. When appraisal measures are entered first, personality measures add no additional variance to the prediction of physical functioning. This suggests that the influence of personality on physical functioning is at least partially mediated by appraisal. With regard to emotional functioning, four of the five appraisal components alone accounted for 19% of the variance, including the three abovementioned components as well as a lesser focus on goal attainment. These appraisal main effects explained only 2% of the variance in emotional functioning beyond demographic, comorbidities, person factors, and personality (Supplemental Table 7). This is largely due to the strong relationship of Neuroticism to emotional functioning.

We had expected that measures of appraisal would also moderate the influence of health status measures on physical and emotional functioning. In fact, no interaction effects emerged in this analysis of physical functioning, but appraisal did moderate the influence of demographic and health status variables on emotional functioning. Moderating effects added an additional 4% of the variance beyond appraisal alone (Supplemental Table 7). Specifically, we found that emotional functioning was worse among women focused on Health Worries and on concerns about Accomplishing Goals and Problem-Solving. Emotional functioning was also worse among more educated respondents concerned about Accomplishing Goals and Problem-Solving. Alternatively, the impact of depression was attenuated among individuals who focused on Calm, Peaceful, and Active.

Comparison of correlations between emotional functioning and appraisal patterns helped to further clarify moderating effects within comorbidity group. As the number of comorbidities increased, the association of Health Worries on emotional functioning was increasingly negative and the association of Calm, Peaceful, and Active increasingly positive, indicating the heightened role of these appraisal dimensions in, respectively, amplifying or attenuating the emotional impact of more serious illness. Overall, these analyses demonstrate the ability of the BAI to account for important differences in the factors that influence emotional functioning.

The BAI is intended to serve as portable measure of appraisal, suitable for use in a wide range of assessment situations. It is an alternative to our earlier measures which were lengthier and more cumbersome to administer, score, and analyze. In order to evaluate the BAI’s adequacy as an alternative to the QOLAP-V2’s more in-depth appraisal assessment, we note that the 83 QOLAP-V2 items explained 8% of physical functioning and 16% of emotional functioning, after controlling demographic, health, and personality [18], compared to 8 and 11%, respectively, using the BAI. Thus, the portion of variance uniquely associated with appraisal main and interaction effects is comparable between the two instruments. As such, the BAI seems well suited to serve as an alternative to the QOLAP-V2.

The present study has a notable strength in its large and heterogeneous sample, which is useful for validating a general-purpose scale such as the BAI. The sample had important variability in comorbidity burden, which allowed for the informative stratified analyses. The limitations of this study should, however, be noted. While the sample is heterogeneous in its illness representation, it is predominantly composed of middle-aged white females who are married or living with family members. Thus, it may not be as helpful for examining the relationship of social support and appraisal or health outcomes, since the participants may be more representative of people with higher levels of social support. This sample characteristic could constrain the correlations between BAI scores and social support. It may also not address appraisal patterns in older samples (i.e., over age 65). Additionally, the use of a very brief personality measure renders the variables less reliable than a longer measure of personality.

Future research might utilize the BAI in clinical settings and/or with older samples. This application might focus on improving patient-provider communication about concerns related to health, health care, or QOL. It might also utilize the measure in longitudinal research aimed at detecting response-shift effects over time. Given its enhanced practicality, it facilitates response-shift research based on the QOL Appraisal Model [3].