Health care in the United States continues to move toward a value-incentivized system.1 As defined by Porter and Teisberg, “value” is a multidimensional relationship between health outcomes and the costs incurred to deliver those outcomes.2 To capture these multiple dimensions, they proposed an outcome metric hierarchy2,3 that highlights the perceived importance of longevity, recovery, harm caused by treatment, recurrences, and the long-term consequences of care. To address the challenge of measuring health care value across multiple outcome and cost dimensions, several comprehensive frameworks have been developed. The American Society of Clinical Oncology’s Net Health Benefit score and the National Comprehensive Cancer Network’s Evidence Blocks are prominent examples.4,5 The University of Texas MD Anderson Cancer Center favors a definition that integrates outcomes with traditional quality, safety, and patient experience measures and with costs to both payers and patients (Fig. 1).6,7 Importantly, as “value” may be poorly represented by a single number, we recently developed a novel value framework to simultaneously quantify and visualize multiple value domains.7,8,9 While our ability to measure various value metrics is improving, however, we still do not know how to weigh the importance of each metric within this framework.

Fig. 1
figure 1

Value equation at MD Anderson Cancer Center

Full size image

As patients take greater responsibility for their health care,10 it is critical that value measurements accurately reflect their perspectives. In addition to patients, other major participants in the health care system include “the 5 Ps:” providers, payers, producers, and planners.11 Their perspectives on what constitutes value should be considered as well. Each of these stakeholders would likely have different goals. For example, patients may weigh the outcome of functional independence heavily, while providers may primarily focus on factors that influence long-term survival, and payers’ main concern may be lowering costs. Having a better understanding of the views of patients and providers and developing a tool to readily assess these stakeholder preferences may help guide how we ultimately weigh these important value metrics in a value framework.

We sought to assess the importance patients place on each attribute of value (e.g., outcomes, quality of life, cost, treatment experience) within an expanded value framework (Fig. 1). We also sought to identify the attributes care providers (i.e., physicians, nurses, administrators) feel are most important in health care delivery, to assess how patients’ views differ from those of providers and to assess how patients’ and providers’ views have changed over time.

Methods

We conducted a cross-sectional survey study with approval from the Institutional Review Board at MD Anderson, Protocol #2020–0211.

Patients and Providers

For this exploratory study, our goal was to obtain a sample size of 300 patients. As we estimated approximately 25% of patients would respond, 1,200 surveys would need to be administered. We used our departmental research databases to identify surviving patients who had been treated or were anticipating treatment for biopsy-proven, potentially resectable pancreatic or gastric cancer at MD Anderson from January 1, 2000 through January 31, 2020. Patients newly diagnosed with cancer were prospectively identified by screening appropriate medical and surgical providers’ clinics. We aimed for newly diagnosed patients to constitute 10% of our entire study cohort. Patient age, demographics, histologic diagnosis, treatments, and current disease state were obtained from the relevant databases. We also identified providers involved in the care of cancer patients. Providers included surgical oncologists, medical oncologists, radiation oncologists, administrators, nurses, and advanced practice clinicians within our institution.

Survey Tool

To develop our novel survey instruments, we consulted our institution’s experts in value-based health care, decision science, health care economics, patient advocacy, and survey development. Both the patient and provider surveys included an 8-item ranking question to assess values (Fig. 2), followed by a question asking them to rate the same items in terms of their importance on an 11-point Likert scale (0: Not Important, 5: Somewhat Important, 10: Extremely Important; see Appendix A in ESM). Respondents were also asked how the importance of those items had changed since their initial cancer diagnosis (for patients) or the beginning of their health care career (for providers).

Fig. 2
figure 2

Patient [A] and provider [B] surveys, value rankings

Full size image

The 36-item patient survey also assessed respondents’ educational, financial, and social support systems and their understanding of cancer treatments and opinions of the U.S. health care system. The 31-item provider survey assessed educational levels and professional roles and elicited respondents’ views of the U.S. health care system. The patient and provider surveys were similar, with the exception that patients were asked to assess their own values, whereas providers were asked to assess factors they felt were important to patients and to the health care system. The full patient and provider surveys are presented in Appendices A and B, in ESM respectively.

Survey items were evaluated through an intensive iterative feedback phase that involved interviewing more than 100 patients and providers recruited from MD Anderson’s Patient and Family Advisory Council (PFAC) and Community Scientist Program, an initiative of the Department of Health Disparities Research. The PFAC is a group of patients, survivors, caregivers, and family members of patients who serve as an advisory resource for the institution on various projects. The Community Scientist Program provides support to researchers seeking cost-effective and time-efficient methods to gather feedback from cancer survivors about issues pertaining to research design, implementation, recruitment, retention, and other barriers to participation. Community Scientist Feedback Sessions provide an infrastructure that streamlines the process of gathering information about community needs and empowers community members to provide meaningful insight into all phases of research. The surveys were presented at meetings of both the Community Scientist Program and the PFAC to elicit iterative feedback from both patients and caregivers. These presentations were conducted in-person by a research staff member. We also obtained iterative feedback on the provider survey from 20 health care providers, including physicians, nurses, and administrators.

In adherence to a standardized survey reporting guideline, we have incorporated the American Association for Public Opinion Research (AAPOR) survey disclosure checklist (see Appendix C in ESM).

Distribution of Surveys

For patients who had provided email addresses, electronic surveys were sent out via the Research Electronic Data Capture (REDCap) system. Patients without email were mailed all content, including informed consent documents, and were encouraged to complete paper surveys. By completing and returning the questionnaires, the patients consented to participate in the research. Cancer care providers were identified through departmental listings, entered into the REDCap system, and similarly contacted to introduce and complete the survey. We completed all prospective enrollments for both the patient and provider surveys between June 1 and November 1, 2020.

Statistics

Respondent characteristics and survey items were reported as mean ± standard deviation if normally distributed or median and interquartile range if not. The relative importance of the values was calculated using the mean-rank attribute for each of the 8 items. Differences between groups were compared using the Student t test for parametric data and the Mann-Whitney U test for nonparametric data. Categorical data were compared using a Pearson chi-square; if cell counts were <5, the Fisher exact test was used. Pearson correlation tests were performed to assess associations between continuous variables. Significance was assessed at p < 0.05. Statistical analyses were performed using SPSS version 24 (IBM Corporation, Armonk, NY).

Results

A total of 547 patients and providers responded to the survey. The patient survey was offered to 1,242 patients, of whom 383 responded (31% response rate). Of the patient respondents, 72% had pancreatic cancer and 28% had gastric cancer. Most (89%) had undergone surgery more than 6 months before completing the survey, while the remaining 11% were treatment naïve, receiving preoperative therapy, or recovering from surgery (Table 1). Of the 20 patients who did not have email, 2 (10%) completed and returned the paper survey. When comparing responders to nonresponders, age (62 ± 11 vs. 64 ± 13 years), sex (54% vs. 53% male), and race (82% vs. 80% white) were similar (all p > 0.05).

Table 1 Summary statistics: patients (n = 383) and providers (n = 164)
Full size table

The provider survey was sent to 474 providers, of whom 164 responded (35% response rate). The provider respondents included radiation (20%), medical (9%), and surgical oncologists (6%), as well as advanced practice clinicians and nurses (26%), administrators (20%), and others (20%) (Table 1).

Value metric ratings based on Likert scores for both the patient and provider surveys are reported in Table 2. The most highly rated value metrics for both cohorts included functional well-being, overall survival, and emotional well-being. The lowest-rated value metric for both patients and providers was the cost to the U.S. health care system. Providers rated treatment experience and indirect costs to family and caregivers as significantly more important than patients did (both p < 0.05).

Table 2 Value ratings by patients and providers (n = 547)a
Full size table

Providers designated a unique rank (1 through 8) for each item more frequently than patients (77% vs. 36%, p < 0.001). Using only the results from only the respondents who provided unique ranks, Fig. 3 compares the mean rank-order of the value metrics ascribed by patients (n = 113) and providers (n = 101). Comparison of the rank-orders revealed that providers ranked patient experience, emotional well-being, and costs to the patient significantly higher than patients themselves did (all p < 0.05). There were no differences in mean rank-orders between provider subgroups. There was internal consistency based on correlations between the rank-order and Likert scale responses by both patient and provider.

Fig. 3
figure 3

Value rankings by patients (n = 113) and providers (n = 101). Values shown are mean ± standard deviation of the responses to the rank-order item (range: 1-8). Metrics are listed in mean rank-order for each subgroup. *Denotes significant difference (p < 0.05) between patients and providers for the indicated metrics

Full size image

When we compared how patient and provider reported values changed over time, we found that patients most commonly reported that survival had become more important to them over time, whereas providers believed that emotional well-being, treatment experience, and costs had become more important considerations for patients (all p < 0.05; Fig. 4).

Fig. 4
figure 4

Changes in perception of value metrics over time by patients and providers (n = 547). Colored bars designate the percentage of respondents who indicated that the value metric had become less important, was of the same importance, or had become more important

Full size image

Patients who had previously undergone surgery valued functional well-being more highly than did preoperative patients (mean score 2.2 ± 1.3 vs. 3.2 ± 1.7, p = 0.031). Patients with lower income and/or education levels more often felt that costs had become more important considerations throughout the course of their care (both p < 0.05). Younger patient age was correlated with greater concern for both direct (r = −0.167, p = 0.004) and indirect costs (r = −0.318, p < 0.001).

Discussion

Although there are general consistencies amongst the views of cancer patients and providers, we identified discordances in the perception of value between cancer patients and health care providers. Our findings underscore the importance of perspective on the various outcome and cost metrics in the assessment of value in health care.

As it has been difficult to effectively summarize value using various multidimensional measures, a framework for this purpose has recently been described.7,8,9,12 This may have utility in the context of shared decision-making between patient and provider as a robust framework could reflect both costs and outcomes across two or more treatment options. Institutions also may use this framework to develop and prioritize specific workflows, pathways, and algorithms or to support value-based reimbursement models. Still, the metrics to be considered in any value framework will have many components, and choosing those that matter most from multiple perspectives has been a challenge.13 It is clear from our current study that various stakeholders do not entirely agree on which health care outcomes are most important. Thus, a better understanding of stakeholder-specific views could help to devise a more comprehensive and inclusive value framework.

An ideal value framework would incorporate stakeholder perspectives to effectively quantify the value of any care process. Still, the extent to which value can be fully captured by a single number representing multidimensional data is unclear, and there is currently no reason to believe that total value would be accurately represented by the linear sum of such metrics. In cost-benefit analysis, benefit, and cost metrics are typically entered into a cost-utility model to identify optimal utility, but this seldom accounts for the perspectives of most of the participants in the health system: patients, providers, payers, producers, and planners.14 Each of these decision makers likely has distinct goals, so comprehensive frameworks to assess health care value must accommodate these different values and goals and incorporate multiple attributes associated with each option.

Models based on multicriteria analyses, which look at problems from many perspectives, might fill this void.11 Recently, the International Society for Pharmaceutical Outcomes Research task force on multicriteria decision analysis addressed the uses of multicriteria models in health care, from supporting individual patients’ decisions to justifying a universal health care system.15,16 Thus, an understanding of patient, provider, and societal preferences may help to guide not only the selection of the metrics that are important to different stakeholders, but also the incorporation of stakeholders’ preferences into aggregate indexes of value. For example, in our current study, patients and providers quantified the importance of each value metric. These responses could serve as coefficients (or weights) of each measurable value metric to quantify a specific value score that could be used to assist value-based decision making. This is an area for future research.

Before value-based decision making can even be attempted, the collection of data that is important to patients, providers, and society must be improved. There are several limitations to our current ability to measure the many value metrics.7,12 Many domains can be measured using data readily available in electronic medical records, data registries, and/or financial systems, but other metrics—especially patient-reported outcomes, such as pain, functional outcomes, quality of life, and financial toxicity—may not be routinely available. As we found in the current study, both patients and providers consistently rated some of these difficult-to-measure domains (such as well-being and long-term outcomes) as more important than treatment experiences and complication rates. Furthermore, administrative datasets are now being extensively used to assess and improve the performance of health care organizations over time. Agencies even use these data sources to rank and compare hospitals across the country.4,5 Although administrative claims data are useful, they do not include long-term survival, patient-reported outcomes, such as pain, function, and post-treatment quality of life, nor do they contain metrics that reflect costs to the patient. Thus, administrative claims data–based studies effectively assess and compare hospitals using metrics that patients and providers care relatively little about. Until we have access to multiple outcomes, each measured routinely and reliably, it will be difficult to assess total value.

Once we develop these better data, our ability to effectively communicate value will facilitate shared decision-making among all stakeholders involved in value-based health care. A value framework could be used to present multidimensional information to patients and let them decide on various options based on the endpoint that matters the most to them.17 Although substantial research has been conducted on patient preferences, numerous areas of uncertainty and opportunities for future research still exist. The methods used in patient preference research range from easy-to-understand and easy-to-administer ranking or rating surveys to more sophisticated—but also more difficult to administer and interpret—time trade-off or standard gamble methods.18 Findings from studies of patient preferences should be used to define future research strategies and to select the most appropriate and meaningful endpoints to be presented to patients during the decision-making process. Gathering standardized patient-centered data, creating patient-preference studies that use reliable and generalizable methods, and developing decision aids are top priorities for future research.

There are limitations to this exploratory study. This was a cross-sectional study and longitudinal studies are needed to fully understand how value changes across the phases of cancer care. Our patient cohort was relatively homogeneous, consisting of patients with specific gastrointestinal malignancies from our single, large, quaternary-referral institution. Because treatment of pancreatic and gastric cancer involves high physical, emotional, and financial costs yet still entails poor functional outcomes and low rates of survival, we felt this was an opportune population in which to assess the value of care delivery.19,20,21,22 Because most of the patients who responded to our survey were white, insured, no longer worked for pay yet retained a relatively high annual income, and specifically seeking care at a quaternary-referral institution, their perspectives and values may not be generalizable to the overall cancer patient population. Although we received more responses than originally expected, it is possible that death is responsible for a significant portion of nonresponse given the high mortality of pancreas and gastric cancer. There was a disproportionate representation of radiation oncologists and administrators within the provider respondents. Although there were no differences in mean rank-orders between provider subgroups, this introduces potential bias. The classification of administrators as “providers” is arguable. Administrators could be classified as “planners” or it can be argued that administrators working within a hospital system are still “providers” as they ultimately collaborate with other health professionals to provide care to patients. Overall, this project was not meant to obtain results that would define all patients’ and providers’ perspectives, but to drive further prospective work in larger, more diverse cohorts with direct patient and provider engagement. Finally, many survey respondents did not provide unique ranks for all value metrics. To this point, the survey allowed for respondents to provide equal ranks if that was their preference. Providers may have been more likely to choose a rank perhaps, because they were more comfortable allowing tradeoffs between values, while patients were possibly more resistant to tradeoffs. Still, our study assessed the responses both from all participants and from only those who provided unique ranks. These findings will, nevertheless, allow for future studies to develop an improved survey tool in assessing value in health care.

Conclusions

Although patients with cancer and their health care providers generally agreed on most dimensions of value, we identified differences in the perception of value among patients and between patients and providers. Our findings thus underscore the importance of embedding multistakeholder perspectives in the assessment of value in health care. In addition to collecting more complete outcome and cost metrics, future efforts should focus on designing patient preference studies that use reliable and generalizable methods and using the results of such studies to develop decision aids for value-based, shared decision-making strategies.