Avoid common mistakes on your manuscript.
1 Introduction
Patient reported outcomes (PROs) capture how patients feel, function, live their lives, and survive [1, 2]. PROs include outcomes such as symptoms, functional status, well-being, and health-related quality of life reported directly by the patient without interpretation by a clinician or anyone else [3, 4]. Standardized and validated PRO measures (PROMs) are used to assess PROs. Although historically PROs have been used in research settings, increasingly PROMs are also being collected as part of routine practice to monitor how individual patients are doing and inform their clinical care [5,6,7,8,9].
Vulnerable and underserved populations are defined as populations who have been systematically denied a full opportunity to participate in aspects of economic, social, and civic life [10, 11]. Evidence indicates that using PROs in clinical practice can improve patient care and outcomes in general and may be even more valuable for vulnerable and underserved populations in the USA [12,13,14]. For example, Basch et al.’s study found that PROs improved health-related quality of life, reduced emergency department visits, and increased survival for patients generally, and improved these outcomes even more for patients in the computer inexperienced subgroup, who also tended to be older, less educated, or Black [12, 13]. Similarly, another study reported that patients from underrepresented racial groups with less education or less computer usage reported more benefits from completing PRO questionnaires in their routine care [14].
However, numerous, multilevel barriers impede broad implementation of PROs in clinical practice [15], and evidence suggests that vulnerable and underserved populations face unique challenges that hinder PRO use. These challenges can span multiple levels [15,16,17,18,19,20]. Patients may face challenges such as not having access to technology. Clinicians may be hesitant to use PRO data due to workflow challenges and resource constraints [20, 21]. Systems may lack expertise on the needs of specific patient populations. Therefore, there is a need to determine how best to address the full range of key barriers.
The purpose of this letter is to present the results from an expert advisory committee convened to identify the key barriers and potential solutions for implementing PROs in vulnerable and underserved cancer populations in the USA. The intent is to provide strategies for multiple stakeholders interested in improving the use of PROs.
2 Advisory Group Formation
The advisory group was organized by the PROTEUS Consortium (Patient-Reported Outcomes Tools: Engaging Users & Stakeholders [22]) in collaboration with Pfizer. PROTEUS partners with over 50 key stakeholder groups to help patients, researchers, and clinicians navigate the use of PROs in clinical trials and clinical practice [23, 24].
By advertising widely, PROTEUS solicited applications from experts with interest in and/or experience using PROs in vulnerable and underserved cancer populations in the USA. For the purposes of the advisory group, we defined vulnerable and underserved populations as populations sharing a particular characteristic, as well as geographic communities, that have been systematically denied a full opportunity to participate in aspects of economic, social, and civic life, including: older adults; persons of color; members of religious minorities; LGBTQ+ populations; individuals with disabilities; populations impacted by persistent poverty or who may face barriers that make it difficult to get health coverage and basic health care services; and people whose first language is not English, live in rural areas, or have limited health literacy and/or numeracy [10, 11]. The advisory group focused specifically on the use of PROs to inform patient monitoring and management.
Three PROTEUS leaders reviewed a total of 86 applications from which 24 individuals across 12 states were invited and able to serve on the advisory group. The final advisory group included 29 members: the PROTEUS leadership, the 24 invited individuals, and a Pfizer representative. Multiple disciplines were represented, including clinicians, researchers, pharmacists, and program managers.
3 Advisory Group Meeting: Process and Outcomes
The advisory group met in person for a full-day meeting in May 2023 that was organized in four parts and used structured processes (brainstorming, break-out groups, and prioritization exercises) to (1) provide background information, (2) identify barriers to implementing PROs in vulnerable and underserved cancer populations, (3) prioritize the top barriers, and (4) develop solutions to address the prioritized barriers.
3.1 Part 1. Providing Background Information to the Advisory Group
The first part of the meeting provided background information to promote relevant and productive discussions. PROTEUS team leaders described PROTEUS, the rationale for establishing the advisory group, and the goal of developing solutions to advance the use of PROs in vulnerable and underserved populations. In addition, three orienting presentations highlighted a patient’s experience overcoming cancer and the importance of partnering with patients, the current literature on implementing PROs in routine clinical care for diverse patients in the USA [20], and multilevel barriers to integrating PROs in clinical care generally [15, 25].
3.2 Part 2. Identifying Barriers to PRO Implementation in Vulnerable and Underserved Cancer Populations
The second part of the meeting focused on identifying key barriers to implementing PROs in vulnerable and underserved populations in the clinical setting. To accomplish this, the advisory group members brainstormed in four break-out groups on the following topics: (1) additional barriers that were not already presented, (2) their top five barriers and challenges for using PROs in vulnerable and underserved populations, and (3) any specific barriers that applied to particular vulnerable and underserved populations (e.g., older adults, rural patients).
Following break-out group report backs, PROTEUS leaders consolidated barriers to create a list of 17 unique barriers (Table 1). Some barriers expanded on those that were already known. For example, in addition to the resources needed to train clinicians and staff on using PRO systems and using PRO data for making clinical decisions, the advisory group described how PRO use in vulnerable and underserved populations requires resources for systems and clinicians to meet patients’ culture, language, literacy, and numeracy needs.
3.3 Part 3. Prioritizing Barriers to PRO Implementation in Vulnerable and Underserved Cancer Populations
These 17 barriers were listed on flip-chart pages and the advisory group members prioritized them through “dot” voting [26, 27] where they were given four sticker dots and placed the dots next to the barriers they thought were most important and actionable. Four barriers were prioritized by the advisory group (Table 1), which included the commitment of health systems to invest in providing the resources required to address patients’ diverse needs, as well as patient capability to complete PROs.
3.4 Part 4. Developing Solutions to Address Prioritized Barriers
The last part of the meeting focused on developing solutions to address the four prioritized barriers. First, the advisory group members met in their break-out groups and brainstormed relevant solutions. Then the group developed a list of unique solutions based on the breakout group discussions. Overall, 47 different solutions were identified to address the top barriers.
Following the meeting, the PROTEUS leaders reviewed and thematically categorized these solutions into four categories (Table 2): (1) Education and Engagement; (2) Information Technology or Technological Resources; (3) Research; and (4) Incentives, Mandates, and Marketing. Within these categories key solutions included sharing resources/toolkits/best practices for addressing the needs of different populations; demonstrating the monetary and nonmonetary value of PROs; developing partnerships among patient advocates, health workers, community organizations, and practices caring for vulnerable and underserved populations; and incentivizing patients and/or compensating them for their time completing PROs.
4 Discussion
The advisory group of multidisciplinary experts identified and prioritized barriers to PRO implementation and proposed a framework of solutions to advance the use of PROs in vulnerable and underserved populations. The framework of solutions contributes to important discussions about equitably capturing PROs and covers various actions to enhance PRO implementation in vulnerable and underserved populations [28, 29]. It also aligns with established implementation strategies based on the Expert Recommendations for Implementing Change (ERIC) project [30, 31]. The ERIC project strategies are a compilation of techniques to enhance the adoption, implementation, and sustainability of interventions. Their strategies are grouped into nine clusters, and our solutions directly address five of them, including: changing the infrastructure to incentivize PRO use, using evaluative and iterative strategies to gather patient insights, adapting and tailoring PRO use to different contexts; developing stakeholder interrelationships to facilitate PRO integration, and training and educating stakeholders on the benefits and methods of PRO. The alignment of our solutions framework with the ERIC recommendations substantiates that they are well-suited to improve the use of PROs. Nevertheless, we acknowledge that the solutions proposed by the advisory group are not exhaustive.
The findings also lay a foundation for research that advances the use of PROs in routine cancer care for vulnerable and underserved populations. Although some studies have shown that using PROs in clinical practice can improve patient care and outcomes for these populations [12,13,14], there is a lack of widespread implementation of PROs in institutions that care for vulnerable and underserved populations. Future research should assess the implementation and effectiveness outcomes of the proposed solutions. Moreover, additional evidence should be generated on the value of PROs to secure critical support and resources from policymakers, administrators, and leaders. Together, these approaches aim to ensure vulnerable and underserved cancer populations experience the benefits of patient-centered care.
References
U.S. Food and Drug Administration. Clinical outcome assessment (COA): frequently asked questions 2020. Available from https://www.fda.gov/about-fda/clinical-outcome-assessment-coa-frequently-asked-questions#COADefinition. Accessed 31 Oct 2023.
Haywood KL, de Wit M, Staniszewska S, Morel T, Salek S. Developing patient-reported and relevant outcome measures. In: Facey KM, Ploug Hansen H, Single ANV, editors. Patient involvement in health technology assessment. Singapore: Springer Singapore; 2017. p. 103–20.
Acquadro C, Berzon R, Dubois D, Leidy NK, Marquis P, Revicki D, et al. Incorporating the patient’s perspective into drug development and communication: an ad hoc task force report of the patient-reported outcomes (PRO) harmonization group meeting at the food and drug administration, February 16, 2001. Value Health. 2003;6(5):522–31.
U.S. Food and Drug Administration. Patient-reported outcome measures: use in medical product development to support labeling claims 2009. Available from https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-reported-outcome-measures-use-medical-product-development-support-labeling-claims. Accessed 31 Oct 2023.
Greenhalgh J. The applications of PROs in clinical practice: what are they, do they work, and why? Qual Life Res. 2009;18(1):115–23.
Snyder CF, Aaronson NK, Choucair AK, Elliott TE, Greenhalgh J, Halyard MY, et al. Implementing patient-reported outcomes assessment in clinical practice: a review of the options and considerations. Qual Life Res. 2012;21(8):1305–14.
Snyder C, Brundage M, Rivera YM, Wu AW. A PRO-cision medicine methods toolkit to address the challenges of personalizing cancer care using patient-reported outcomes: introduction to the supplement. Med Care. 2019;57(Suppl 1):S1–7.
Stover AM, Tompkins Stricker C, Hammelef K, Henson S, Carr P, Jansen J, et al. Using stakeholder engagement to overcome barriers to implementing patient-reported outcomes (PROs) in cancer care delivery: approaches from 3 prospective studies. Med Care. 2019;57(Suppl 1):S92–9.
Lavallee DC, Chenok KE, Love RM, Petersen C, Holve E, Segal CD, et al. Incorporating patient-reported outcomes into health care to engage patients and enhance care. Health Aff (Millwood). 2016;35(4):575–82.
U.S. Centers for Medicare and Medicaid Services. Serving vulnerable and underserved populations. Available from https://marketplace.cms.gov/technical-assistance-resources/training-materials/vulnerable-and-underserved-populations.pdf. Accessed 9 Jan 2023.
U.S. Department of Agriculture and U.S. Department of Health and Human Services. Executive order on advancing racial equity and support for underserved communities through the federal government 2021. Available from https://www.whitehouse.gov/briefing-room/presidential-actions/2021/01/20/executive-order-advancing-racial-equity-and-support-for-underserved-communities-through-the-federal-government/. Accessed 9 Jan 2023.
Basch E, Deal AM, Kris MG, Scher HI, Hudis CA, Sabbatini P, et al. Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial. J Clin Oncol. 2016;34(6):557–65.
Jensen RE, Snyder CF. PRO-cision medicine: personalizing patient care using patient-reported outcomes. J Clin Oncol. 2016;34(6):527–9.
Snyder CF, Herman JM, White SM, Luber BS, Blackford AL, Carducci MA, et al. When using patient-reported outcomes in clinical practice, the measure matters: a randomized controlled trial. J Oncol Pract. 2014;10(5):e299-306.
Crossnohere N, Brundage M, Snyder C, and the Advisory Group. The PROTEUS guide to implementing patient-reported outcomes in clinical practice a synthesis of resources 2023. Available from https://theproteusconsortium.org/proteus-practice/proteus-practice-guide/. Accessed 31 Oct 2023.
Meirte J, Hellemans N, Anthonissen M, Denteneer L, Maertens K, Moortgat P, et al. Benefits and disadvantages of electronic patient-reported outcome measures: systematic review. JMIR Perioper Med. 2020;3(1): e15588.
Basch E, Stover AM, Schrag D, Chung A, Jansen J, Henson S, et al. Clinical utility and user perceptions of a digital system for electronic patient-reported symptom monitoring during routine cancer care: findings from the PRO-TECT trial. JCO Clin Cancer Inf. 2020;4:947–57.
Griffin J, Finney Rutten LJ, Ridgeway JL, Lam V, Austin J, Ruddy K, Pachman D, Fischer K, Cheville A. 29th Annual conference of the international society for quality of life research: 105.4 demographic differences in patient reported outcome (PROS) response rates of cancer-related symptoms by patient portal, in-clinic tablet and integrated voice response. Qual Life Res. 2022;31(2):9–169.
Aiyegbusi OL. Key methodological considerations for usability testing of electronic patient-reported outcome (ePRO) systems. Qual Life Res. 2020;29(2):325–33.
Hyland CJ, Guo R, Dhawan R, Kaur MN, Bain PA, Edelen MO, et al. Implementing patient-reported outcomes in routine clinical care for diverse and underrepresented patients in the United States. J Patient Rep Outcomes. 2022;6(1):20.
Liu LH, Garrett SB, Li J, Ragouzeos D, Berrean B, Dohan D, et al. Patient and clinician perspectives on a patient-facing dashboard that visualizes patient reported outcomes in rheumatoid arthritis. Health Expect. 2020;23(4):846–59.
The PROTEUS Consortium. Available from https://theproteusconsortium.org/. Accessed 31 Oct 2023.
Snyder C, Crossnohere N, King M, Reeve BB, Bottomley A, Calvert M, et al. The PROTEUS-Trials Consortium: optimizing the use of patient-reported outcomes in clinical trials. Clin Trials. 2022;19(3):277–84.
Pritchett JC, Patt D, Thanarajasingam G, Schuster A, Snyder C. Patient-reported outcomes, digital health, and the quest to improve health equity. Am Soc Clin Oncol Educ Book. 2023;43: e390678.
Crossnohere NL, Anderson N, Baumhauer J, Calvert M, Esparza R, Gulbransen S, et al. A framework for implementing patient-reported outcomes in clinical care: the PROTEUS-practice guide. Nat Med. 2024. https://doi.org/10.1038/s41591-024-02909-8.
Brown T, Wyatt J. Design thinking for social innovation. Dev Outreach. 2010;12(1):29–43.
Bridges JFP, Lynch T, Schuster ALR, Crossnohere NL, Smith KC, Aslakson RA. A review of paper-based advance care planning aids. BMC Palliat Care. 2018;17(1):54.
Calvert MJ, Cruz Rivera S, Retzer A, Hughes SE, Campbell L, Molony-Oates B, et al. Patient reported outcome assessment must be inclusive and equitable. Nat Med. 2022;28(6):1120–4.
Ortega G, Allar BG, Kaur MN, Edelen MO, Witt EE, Fayanju OM, et al. Prioritizing health equity in patient-reported outcome measurement to improve surgical care. Ann Surg. 2022;275(3):488–91.
Waltz TJ, Powell BJ, Matthieu MM, Damschroder LJ, Chinman MJ, Smith JL, et al. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the Expert Recommendations for Implementing Change (ERIC) study. Implement Sci. 2015;10:109.
Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21.
Acknowledgments
The PROTEUS Consortium is funded through unrestricted support from Pfizer. The advisory group for Patient-Reported Outcome Implementation in Vulnerable and Underserved Populations is a collaboration between PROTEUS and Pfizer Global Medical Grants. We are grateful to Emilie Smith, PharmD, BCOP, for her time and valuable insights.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
Dr. Snyder is currently or has previously received research funding through her institution from Genentech and Pfizer, and consulting fees from Shionogi and Janssen. Dr. Brundage is currently or has previously received research funding through his institution from Genentech and Pfizer. Dr. Schuster receives research funding through her institution from Pfizer. Dr. Crossnohere is currently or has previously received research funding through herinstitution from Genentech and Pfizer, and receives unrelated consulting support from Boehringer Ingelheim. Dr. Hamilton has consulting (ad hoc advisory board) fees from Incyte and Sanofi; she is on the data safety monitoring board for Angiocrine, and Adjudication committee for CSL Behring. Dr. Meyer has research funding from Merck and AstraZeneca. Dr. Mittal has received grant funding from Pfizer, has served in consulting/advisory board role for Bayer, Aveo, Dendreon, Myovant, Fletcher, Curio Science, Janssen, Dedham Group, Astra Zeneca, and has received honoraria from IntrinsiQ, Targeted oncology, Medpage, Aptitude Health, Cardinal Health; none of these activities are related to material in the commentary. Ms. Rainey sits on the board for both the Community Oncology Alliance (COA) and the Association for Community Cancers Centers (ACCC). Dr. Smith reports honoraria from GSK and serves on the advisory board for Elevance Health. Ms. Baldwin and Ms. Owens have no conflicts of interest to disclose. Drs. Adjei Boakye, Angove, Gillespie, Jackson McCleary, Mittal, Kaufmann, Lee, Mehta, Peterson, Richardson, Shapiro, Sibbitt, Vargo, Vickers, have no conflicts of interest to disclose.
Data availability
The data supporting the findings of this work are available within the article.
Additional information
Maimah Karmo—Patient-author.
Rights and permissions
About this article
Cite this article
Schuster, A.L.R., Crossnohere, N.L., Boakye, E.A. et al. A Framework to Promote Implementation of Patient-Reported Outcomes in Institutions Caring for Vulnerable and Underserved Cancer Populations. Patient (2024). https://doi.org/10.1007/s40271-024-00703-9
Accepted:
Published:
DOI: https://doi.org/10.1007/s40271-024-00703-9