1 Background

Weighing the benefits and risks of new health technologies requires assessing the available scientific evidence but also making societal value judgments about the relative importance of those benefits and risks [1]. Such judgments traditionally have been delegated to physicians. Increasingly, however, patients are claiming a greater role in such assessments. This paper discusses the increasing concern of the US FDA in strengthening patient engagement by conducting health preference research (HPR) on patients’ risk tolerance.

2 Regulatory Decision Making at FDA

Regulators of drugs and devices have taken different approaches to incorporating the patient’s voice into regulatory decision making. While the legislative and regulatory frameworks under which the Center for Drug Evaluation and Research (CDER) and Center for Devices and Radiological Health (CDRH) approve drugs or devices differ considerably, their fundamental decision problems are essentially identical: how to weigh the potential benefits of health technologies against the potential risks when the beneficial and harmful outcomes are measured in different, non-comparable units.

In describing the regulatory priorities for FDA’s CDER, Center Director Janet Woodcock acknowledged that “people with chronic diseases are experts in that disease, as far as the symptoms and the impact on quality of life, and what might be acceptable tradeoffs on risk and uncertainty” [2]. However, the regulators of drugs and devices have taken different approaches to incorporating the patient’s voice into the regulatory process. For drugs, regulators responded in part to the 2012 fifth authorization of the Prescription Drug User Fee Act’s aim to “enhance its benefit-risk assessment in regulatory decision-making” by initiating a public process to better include the patient perspective [3]. The resulting Patient-Focused Drug Development Program involves a structured “semi-quantitative” approach to benefit-risk assessment. Specifically, the plan consists of convening meetings regarding specific disease areas to obtain the patient perspective on disease severity, on the current state of treatment available, on unmet medical needs, and on important elements of patients’ benefit-risk decision-making structures. Up to 24 are planned for the purposes of engaging FDA review divisions, relevant patient-advocacy communities, and other interested stakeholders. These public meetings result in disease-specific reports that FDA views as providing a general context for their regulatory reviews.

In 2014, the first ever patient advocacy-initiated draft guidance was submitted to CDER to help accelerate the development and review of potential therapies for Duchenne muscular dystrophy [4]. This guidance recommends using a quantitative approach to understand the community’s tolerance of potential risks or the uncertainty of benefit associated with new treatment alternatives. However, CDER has thus far not incorporated quantitative measures of patient preferences into its regulatory decisions, but has indicated that they will work to translate the qualitative findings from patient input into decision making [5]. In September 2016, CDER approved eteplirsen injection, the first drug approved to treat patients with Duchenne muscular dystrophy [6].

In contrast to CDER’s qualitative approach to patient engagement, FDAs CDRH has implemented a path-breaking quantitative approach to integrating patient concerns into regulatory reviews of medical devices. In 2012, CDRH issued guidance on benefit-risk evaluations that included this statement [7]:

“Risk tolerance will vary among patients, and this will affect individual patient decisions as to whether the risks are acceptable in exchange for a probable benefit. … FDA would consider evidence relating to patients’ perspective of what constitutes a meaningful benefit.”

FDA did not include a discussion of what form such evidence would take but did clearly signal their interest in quantitative evidence on patients’ willingness to accept benefit-risk tradeoffs. In 2016, CDRH subsequently issued guidance on how to conduct studies to elicit such information [8].

3 Patient Preferences as Regulatory Evidence

The 2016 guidance recommends using methods that satisfy scientific standards for validity and reliability and includes the following guiding principles for these methods:

  1. 1.

    The submission of patient-preference information to FDA is voluntary.

  2. 2.

    Preference studies should follow established good research practices for discrete-choice experiments (DCEs) such as those of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR).

  3. 3.

    Preference-elicitation methods should simulate actual choices that patients make among therapeutic alternatives and ensure that the numbers and risk-benefit attributes of each alternative are defined and understood clearly.

  4. 4.

    The sample should include the full range of patients for which the treatment is indicated, ensuring that the sample is representative with respect to demographic characteristics and disease severity levels.

  5. 5.

    Trade-off evidence should include relevant product attributes, including both the risks of potential serious adverse events and the therapeutic benefits.

  6. 6.

    Trade-off preferences can be described as the minimum acceptable benefit for the given risk levels or as the maximal acceptable risk for the given benefit levels.

  7. 7.

    The analysis of the DCE responses should account for potential heterogeneity in preferences.

The guidance endorses the DCE approach that was used in a CDRH-sponsored study of weight-loss devices [9]. One of the authors of this paper (Johnson) served as scientific lead for this study. FDA reviewers used the results of the weight-loss device study to evaluate a new weight-loss device (the Maestro Rechargeable System). Although the device failed its primary clinical trial endpoints, it was subsequently approved as a Class III device on 14 January 2015 based on the findings from the patient-preference study. This decision marks the first weight-loss device to be approved in 10 years [10].

4 FDA Draft Guidance: Implementation Challenges and Recommendations

The FDA draft guidance on quantifying patients’ benefit-risk trade-off preferences breaks new ground in regulatory science. However, as with any organizational innovation, the agency faces a number of challenges to successful implementation.

4.1 Questionable Requirements

The draft guidance includes a number of requirements, some of which could be infeasible. For example, drawing a strictly representative sample of a patient population may be technically impossible in the absence of a national registry of that population. Also, it may be more important to obtain preference evidence on certain subpopulations such as, for example, children, advanced-stage patients, minorities, or other groups whose risk tolerance may be systematically different from that of the average patient.

The draft guidance dictates that the labeling should describe the preference study, including the characteristics of the patients who considered the device’s benefits to outweigh its risks. The implication that such information will improve patients’ decision making is questionable. Psychologists and behavioral economists have found that telling people “there are many people like you” who do something can strongly influence decision making without encouraging careful evaluation of the advantages and disadvantages of alternatives [11, 12].

The guidance also states that “a specialized informed-consent process may be appropriate to facilitate use in patients who explicitly accept the probable risks in exchange for the probable benefits.” While special information requirements already exist for some high-risk medications, FDA requires that sponsors verify individual risk tolerance without providing guidance on how to do that.

The draft guidance helps to institutionalize the integration of patient concerns with evidence-based regulatory reviews. The Maestro decision sent a signal that FDA was prepared to use preference evidence in regulatory decision making. However, apart from the Gastroenterology Devices Branch, none of the other CDRH review groups currently have the experience and expertise to evaluate preference evidence if the guidance is widely adopted by sponsors.

Not only is expertise at FDA limited, but most established HPR groups are already working at capacity. The limited available expertise invites proliferation of poorly designed and executed studies. This gap led to the founding of the International Academy of Health Preference Research, which produced these commentaries. FDA should invest in similar initiatives both to better educate their own staff and to support programs to educate interested researchers in best-practice HPR methods.

4.2 The Evidence Base and Acceptable Methods

While the number of published applications of DCE methods in healthcare is growing rapidly, the peer-reviewed literature on methods and applications across therapeutic areas is still relatively small. There is a need to sponsor additional case studies, possibly under FDA direction. Many published studies are product specific or at least focus on a narrow range of product-relevant outcomes. FDA should consider providing grant and contract support for generic studies to produce transferable values for general quality-of-life benefits, common side effects, and typical serious adverse event risks that are of regulatory concern. Validated preference-elicitation instruments could be developed for administration in clinical trials.

While endorsing DCE methods, the guidance includes an extensive appendix on other approaches. FDA apparently simply intended to document survey research approaches that have been used for various purposes to elicit preference information. However, the appendix provides little help for readers in assessing the relevance of the listed methods for FDA regulatory reviews. For example, the willingness-to-pay approach is not relevant for benefit-risk assessments.

FDA does not preclude the use of approaches other than DCE, but the criteria that such methods would have to satisfy and how to deal with the different types of results that could be produced is not clear. There plainly is a need to provide greater support for consensus-building activities such as initiatives in the ISPOR and the Society for Medical Decision Making and international collaborations [13].

5 Conclusion

In line with the values of evidence-based medicine, effective patient engagement in FDA regulatory benefit-risk evaluations could contribute to “more thoughtful identification and compassionate use of the individual patients’ predicaments, rights, and preferences in making clinical decisions about their care.” Standardizing the process of designing and implementing preference studies to be analogous to the process used for clinical trials involves overcoming a number of institutional and scientific challenges. Nevertheless, there appears to be a commitment at FDA to achieve this. At the release event for the CDRH draft guidance on 13 May 2015, Dr. Rob Califf, then FDA Commissioner, observed the following:

You don’t know people’s preferences unless you ask them. How do people look at these differences? I fell in love with the discrete choice experiments, which I had heard about from the Business School, but had not seen in action and I think that provides major advantages. … I think it’s a major triumph … that we’re here today, not just talking about it, but with the FDA very involved. To the extent that FDA takes preferences seriously, I think it’s a great day.