FormalPara Key Points for Decision Makers

Patient-centered core impact sets (PC-CIS) are disease-specific lists of impacts patients report as most important and are proposed to help align research and other efforts with the patient voice.

In our environmental scan, we found that no other effort like PC-CIS exists but a number of past and current efforts, especially core outcome sets and patient-focused drug development, provide foundational inputs that can help move the PC-CIS initiative forward.

We noted that the foundational inputs will require some modifications to protect the patient voice as part of the process and transparently describe what engagement effort specifically entailed to help move the field forward.

1 Introduction

As patient partnership in the planning, conduct, and dissemination of research and care has expanded, improvements in research and care have been accomplished (e.g., study recruitment and retention advancements, attention on lessening burden on patients and families, enhanced focus on health equity) [1,2,3]. There is growing commitment to capturing what is most important and meaningful to patients. This is exemplified in programs such as the US Food and Drug Administration’s (FDA’s) Patient-Focused Drug Development (PFDD) initiative, as well as efforts by the Patient-Centered Outcomes Research Institute (PCORI) and European Medicines Agency [4,5,6,7].

Yet, challenges persist in identifying the impacts of disease and treatment that matter most to patients for a single disease, and subsequently developing related consistent measurement for varying downstream uses [8, 9]. Greater emphasis on patient engagement reveals the importance of understanding, not only health impacts, but also other impacts on the everyday lives of patients, families, and caregivers, including finances, ability to work, and access to education. This has been recognized in PCORI’s expanded mandate to collect data on the “full range” of outcomes such as “potential burdens and economic impacts of the utilization of medical treatments, items, and services” [10]. Similarly, the definition of patient-experience data provided in twenty-first Century Cures Act refers to the wide “range of impacts and preferences reported by patients.” Further, the legislation instructs FDA to prepare guidance that includes “approaches to identifying and developing methods to measure impacts to patients that will help facilitate the collection of patient experience data in clinical trials” [11]. To avoid confusion about terminology, we use the term “impact” to refer to any potential ramifications of disease and/or treatment on the health and lives of patients that patients report as important, including outcomes, burdens, and other ramifications [9].

Sound decision-making depends on the ability to compare and integrate study findings and learnings from different sources. However, siloed approaches for determining what concepts to collect data on, results in inconsistency. For example, study endpoints, outcomes, and outcome measures tend to be study and/or setting specific, often not reflecting the impacts on health and daily life that patients report they care about. To overcome these inconsistencies, approaches have been championed to standardize the data researchers should collect. For example, a core outcome set (COS) is an agreed-upon, standard list of outcomes that should be measured and reported, as a minimum, in all clinical trials [12].

Ideally, consistency is derived from the experiences patients report and what is meaningful to them, applied across the health ecosystem—not just in trials. Different contexts will likely require variations in endpoints and measures, as they must be fit-for-purpose for the use. However, a core set of qualitative impacts can drive alignment. That is, the core impact set provides the measurement targets, driving alignment across many potential downstream uses. The impacts are not the measures, they are the targets (or potential concepts of interest) for measurement. One would qualitatively collect data on the broad set of impacts meaningful to patients; then, that broad set would be prioritized into the core set, which would then inform future measure development that is fit-for-purpose for the intended use (e.g., pain, difficulty walking, and difficulty working could be prioritized as core impacts that inform development of clinical outcome assessment (COA) and quality measures, and value elements that are aligned but might not be identical). Thus, it would be useful to combine patient-engagement best practices (to identify impacts that matter) with approaches to gain consistency across the health ecosystem.

A patient-centered core impact set (PC-CIS), a disease-specific list of impacts reported by patients as most important to them, has been proposed for this purpose and is characterized as being [9]:

  • Patient-community led—by patient groups/communities, with patients in governance roles;

  • Patient derived and patient prioritized—with other stakeholders who can and should be involved, but who cannot “out-vote” the patient voice; and

  • Comprehensive of the patient experience—capturing how illnesses and treatments impact all aspects of daily life for patients, families, and caregivers [9].

In theory, a PC-CIS for a disease and/or population is developed and tapped as a resource, a precursor that informs various downstream applications, allowing for future apples-to-apples comparisons and obviating duplication of efforts. Potential uses include: informing COS development, selecting measures and endpoints for trials or other research (e.g., real-world evidence), developing COAs and quality-of-care measures, consideration in clinical practice guidelines, and value and health technology assessment and related economic analyses.

To support the concept of PC-CIS, our objective was to explore PC-CIS feasibility and identify conceptual overlap(s) with past or existing efforts for the purpose of gleaning from past or existing work. This paper describes key insights from an environmental scan to identify and characterize those existing and past efforts.

2 Environmental Scan Approach

The objectives of the environmental scan were to identify: (1) foundational work that can be built upon to establish PC-CIS as a viable concept, and (2) applicable methods and best practices from which we might glean sound, practical development methods and approaches to leverage in PC-CIS development.

The environmental scan was challenging as a typical search is reliant on specific terms or subcomponents of terms. Since PC-CIS do not exist as we have defined the term, this more straightforward approach was not possible. We relied upon the PC-CIS Advisory Committee (see members in Supplementary Materials Table 1) to guide PC-CIS conceptual framing and inform the environmental scan. Committee members have expertise in patient-centered outcomes research, value/health technology assessment, COS development, COAs, and/or patient engagement.

Relevant resources and initiatives were identified via: (1) advisory committee group discussions, (2) an open-ended survey fielded among committee members, and (3) a search of the academic and gray literature using terms the committee suggested as relevant and related (e.g., COS) (see search terms in Supplementary Materials Table 2). An iterative approach was used with findings presented to the committee during regularly held meetings throughout early 2021 (January to June) for further discussion, brainstorming, and elaboration. At any time in the process, a committee member could suggest a publication or resource for presentation to the group.

Results were provided to the committee for review, discussion, and synthesis. A working group was formed to review all individual documents. Insights were derived from working group and advisory committee discussions.

3 Environmental Scan Results

Findings are provided in Supplementary Materials Fig. 1 and Tables 3 and 4. The full list of 51 resources, tools, and reference documents identified in the scan is located Supplementary Materials Table 3. Materials were categorized as: core outcome sets (general) (n = 13), core outcome sets (patient perspectives/relevance to patients) (n = 19), categorizing impacts (n = 17), and miscellaneous (n = 2).

4 Key Insights Gleaned from the Environmental Scan

4.1 No Existing Efforts Were Identified that Specifically Meet the Definition of PC-CIS as We Have Specified It, in Terms of Patient Centricity

The potential for patients and patient representatives to identify unique impacts not identified by other stakeholders has been recognized for some time [13,14,15,16,17,18,19,20]. A specific goal for PC-CIS is that they not just include patients, but are patient-community driven [9]. Thus, many existing, general resources on patient engagement (e.g., PCORI resources) informed our search, but were not specific to creating patient-centered core lists or repositories.

For the purposes of our data gathering, we focused heavily on methodology documents for engaging patients in COS and other similar research efforts, as recommended by the committee [21, 22]. In general, we found COS developers have increasingly sought patient perspectives over time in COS development, and many provide guidance on how to do so [22,23,24,25,26,27,28,29]. However, none to date specifically report or outline methods for including or promoting patient-community-driven efforts with patient leadership roles or partnership governance roles, a basic premise for PC-CIS.

We identified numerous resources describing best practices for developing COS and disease-specific COS methodology documents, valuable to informing best practices for PC-CIS development. The documents provide a rich pool of resources that can be tapped to support patient involvement in PC-CIS; however, they will need to be extended to include the concept of patient-community leadership.

It is useful to note here that our scan provided information that further helped differentiate and articulate a clear role for PC-CIS as distinct from COS (Supplementary Materials Table 4). PC-CIS are patient led, derived, and prioritized, and are comprehensive of the full range of impacts (including outcomes) important to patients. This might lead one to think PC-CIS are simply COS developed by patients. However, other distinctions emerged. For example, our scan found that the role of a COS has been traditionally and predominantly to inform outcome selection in clinical research, but PC-CIS are intended to inform all health ecosystem activities that require knowledge of the patient perspective—from the patient perspective. This includes such disparate activities as quality measurement, value/health technology assessment, or outcome-based contracting. PC-CIS also can inform COS development, but probably will not replace COS in their traditional role. Over time, COS and PC-CIS (for a disease) will become more aligned as PC-CIS availability becomes more prevalent and becomes a COS input. However, they will likely never be identical since PC-CIS are an intentionally all-inclusive menu to drive alignment among, but not be prescriptive for, all downstream uses. Clinical research is but one of those uses. For example, there will be some core impacts more applicable to trials versus others more useful to value assessment, versus others more useful to quality assurance, etc. and some might apply across all uses. There should be clear patient-centered processes and rationale for selection from among impacts for a use, driving applicable, fit-for-purpose measures for that use. We envision the core as the precursor, with necessarily nuanced subsets (e.g., COS, value elements, COA concepts of interest, etc.), but all derived from knowing the most important impacts for a disease/population.

4.2 Existing COS-Development Efforts Are a Valuable Source of Foundational Resources for PC-CIS

Foundational work exists upon which to build (Table 3 in Supplementary Materials.) The PC-CIS Blueprint can be informed by previous and ongoing efforts in COS development, patient-preference work, data standardization, and the measurement taxonomies and repositories identified. Reinforced with a focus on meaningful patient engagement, from planning to dissemination, the methods and practices used across these initiatives can be leveraged in whole, or in part, for PC-CIS.

The Outcome Measures in Rheumatoid Arthritis Clinical Trials (OMERACT) Handbook provides a starting point, highlighting identification of opportunities for patient involvement. The Handbook features a chapter on partnership and a separate publication describes how partnership has evolved over time [24, 25]. To achieve adequate representation of patients, the Harmonizing Outcome Measures for Eczema (HOME) roadmap encourages “proactive approaches and possibly funding to enable participation in consensus meetings.” [26]. Clearfield et al. describe learnings from the Center for Medical Technology Policy’s (CMTP) experience partnering with patient organizations, developing orientation and training, and using their consensus process [27]. The Core Outcome Measure in Effectiveness Trials (COMET) initiative provides a number of resources, including examples of COS development protocols [12, 28]. More recently, Vanderhout and colleagues published a practical example of patient engagement in COS development [29]. These examples are very useful but must be further developed and informed by the growing science of patient engagement literature and experiences.

4.3 Existing Health-Outcome Taxonomies Can Be Augmented with Patient-Prioritized Impacts to Create a Comprehensive Impact Taxonomy

To support a PC-CIS effort, an important starting point is defining an “impact” and identifying the lexicon of impacts patients might report. A current impact taxonomy does not exist but could be realized by starting with and augmenting existing, qualitative health-outcome taxonomies. To assist researchers in selecting and describing outcomes measured in clinical trials, there are numerous tools offering ways to categorize these outcomes [30,31,32,33,34,35,36]. For example, a reported taxonomy of 38 categories includes five domains: death, physiologic/clinical, life impact, resource use, and adverse events with traditional health outcomes (e.g., cardiac, physical functioning) and other impacts (e.g., personal circumstances) [37]. Thus, there is recognition of the importance of all impacts, but in a not-yet-standardized fashion.

It should be noted that beyond expansion and reorganization of existing taxonomies to capture the broader range of impacts on patients’ health lives, there is a critical need to distinguish between “impacts” versus what might influence impacts, such as “modifiers.” For example, gender is not a resulting impact from a disease or treatment, but it could modify how a disease or treatment is experienced by a patient. Other social determinants of health can be included in this group of potential “modifiers.”

Since we intend a disease-specific PC-CIS to be used across research and other health settings, it would be useful to test a taxonomy by assessing how well real-world data or sources of patient-experience data fit to ensure the taxonomy can categorize the full range of impacts. For example, “economic” outcomes are a single category in an existing taxonomy [37]. However, a recent report commissioned by the US Department of Health and Human Services illustrates how patient/family, caregiver, employer, and payer/insurer perspectives on economic costs differ, identifying various cost types. Direct non-medical costs capture what could potentially be patient-reported and -prioritized impacts (e.g., transportation/travel costs, paid professional care, and home modifications) [38]. Thus, existing taxonomies could be combined and expanded to efficiently jump-start an all-inclusive impact taxonomy for the purposes of PC-CIS [30,31,32,33,34,35,36].

4.4 Current Approaches/Methods Can Inadvertently Result in Exclusion of Patient Priorities from Core Lists/Sets and Will Need to Be Adjusted to Protect the Patient Voice

To prioritize all potential outcomes into a more practical “core set,” COS developers typically conduct consensus-building exercises (e.g., eDelphi) with minimum levels for achieving consensus. While this approach conceptually and methodologically makes sense, it may have the unintended consequence of culling patient-prioritized outcomes from the list. Patient perspectives often differ from those of clinicians, researchers, and other healthcare stakeholders, and patients could be outvoted in a consensus process [39].

Kirwan and Hewlett describe one example of the disconnect between patient-identified impacts and traditional health outcomes sought in research or routine care. Despite a large proportion of arthritis patients reporting fatigue as important, it was originally not included in any of seven internationally agreed-upon rheumatoid arthritis COS [40]. The importance of fatigue, as well as sleep disturbance and “sense of wellbeing,” was brought to light during the first OMERACT “Patient Perspective Workshop” [18].

A basic premise of PC-CIS is that the patient voice should not be able to be outvoted. An example we identified that supports this is an approach by CMTP for protecting outcome concepts contributed by patients incorporated in COS development. Despite receiving lower vote counts in consensus voting, an outcome concept that originated from patient input is kept in the Delphi results; all others had to meet a minimum threshold [41].

4.5 There is Need for Further Clarity and Transparency on How Patients Were Engaged in Individual Efforts

An aim of PC-CIS is to promote enhanced transparency and greater sharing of detailed, patient-engagement best practices. This accomplishes at least two goals. The first is the ability to appraise the depth and quality of patient-engagement methods within a given effort. The second is to expand access to, and foster, better engagement practices. Interestingly, some of the most ardent supporters and practitioners of meaningful patient engagement can miss the transparency “mark” because their methods have become so natural and innate to them, it no longer seems to them worth mentioning in any detail. For example, the National Health Council (NHC) is a known, ardent supporter of and advocate for patient engagement. Yet, we can cite an example where we know there was strong patient engagement following NHC policies on engagement and the names of those engaged are listed in the report. But the engagement activity itself is not described in the report [42]. Until the rest of the field catches up to the pioneers, explicit and open description of patient-engagement methods will be imperative for learning and evaluation purposes.

Those developing COS are increasingly aiming to include patients and patient perspectives in their work [23]. Despite significant advancements over time, our exploration revealed a dearth of efforts that are specifically patient centered or patient led, and are also described as such [43,44,45,46]. The National Health Council (NHC) Rubric To Capture The Patient Voice, our north star for meaningful patient engagement, along with aligned guidance, includes facets of patient leadership, partnership, and representativeness, as well as transparency, among other domains [47,48,49]. In application, this definition does not simply distinguish between presence or absence of patient engagement. Rather, it identifies those practices that qualify as meaningful engagement.

Despite growing interest in and support for patient engagement, the outputs of the environmental scan revealed that clear descriptions of patient-engagement methodology are rare, even among groups or initiatives known to embrace the philosophy of patient engagement. For example, two important sets of guidance exist, one outlining standards for COS-development protocols and one outlining standards for reporting on COS development. Both facilitate transparency and rigorous methodology, but neither speak specifically to describing patient engagement and documenting how patients guided COS development [50, 51]. Both include a section on “stakeholder” engagement, including a description of who was involved and how or why. While this is a good start, patient engagement is in a nascent-enough phase that it needs to be called out specifically. It should be noted that these tools were developed with patient representatives as participants. The patient engagement may have been exceedingly robust and beneficial, but was just not described [28, 50, 51]. Adopting a standardized approach and reporting mechanism for patient and public participation, such as practices identified in the COMET Handbook, would not only shine light on patient engagement strengths and deficits, it would foster greater awareness and adoption of engagement methods and accelerate improvements in the field [12].

Thus, a complicating aspect of our approach in this scan is the possibility some patient-engagement activities were meaningful and very much in alignment with the NHC Rubric, but were not fully described as such in the public-facing materials we identified. For example, older COS may have been developed with no patient engagement, but engagement was added to newer iterations. It can be difficult to impossible to differentiate when it was introduced, in which sets, and what it actually entailed. In some cases, we know anecdotally about high-quality engagement methods being deployed. However, to maintain rigor and replicability in our approach, our review was necessarily based on publicly available materials, not anecdotes. Details we required on engagement were just not available in current documentation. This must be remedied to highlight, promote, and applaud sound patient engagement methods.

5 Limitations

Our review has several limitations that should be considered. The term “core impact set” is new to the fields of patient-centered research and healthcare. We tried to be as inclusive as possible in our search terms (i.e., keywords) to ensure we were capturing a breadth of terms that might be related to our intended definition of PC-CIS. However, it is possible we have missed related initiatives or resources. We relied on advice and guidance from our advisory committee to help ensure we captured related works. In addition, we limited our search to publicly available resources. There may be work happening that we were unable to access or that is anecdotal that could impact our findings. We believe, however, that clarity and transparency in what is considered patient engagement is required to enhance understanding of what constitutes meaningful engagement and to assess the quality of engagement to move the field forward.

6 Conclusions

The results from the environmental scan confirm PC-CIS as conceptually unique from past efforts by explicitly emphasizing patient leadership and being patient driven and, in its precursor role, necessary for downstream uses that require knowing what is most important to patients. The environmental scan found no existing efforts that specifically met our PC-CIS definition in a strict patient-centered sense. However, we did identify rich sources of work conducted with a great deal of PC-CIS alignment that can be leveraged in many ways in PC-CIS development. This is especially true for past and existing work in COS and PFDD. They provide a valuable treasure trove of methods, resources, and experiences that inform the NHC Blueprint for PC-CIS Development, which is in now process.