Keywords

1 Introduction

Randomized controlled clinical trials (RCTs) have been the gold standard for the evaluation of efficacy and safety of medical interventions. However, the costs, duration, practicality, and limited generalizability have incentivized many to look for alternative ways to optimize it. In recent years, we have seen an increasing usage of real-world data (RWD) and real-world evidence (RWE) in clinical development and life-cycle management. Especially encouraged by legislations and guidance released by regulators and special interest groups in recent years, sponsors have been actively seeking guidance and application use cases. In 2016, the twenty-first Century Cures Act was signed into law [1]. It is designed to help accelerate medical product development and bring new innovations and advances to patients who need them faster and more efficiently. The Food and Drug Administration (FDA) PDUFA (Prescription Drug User Fee Act) VI, released in 2017 for fiscal years 2018–2022, enhances FDA’s ability to consider the possibilities of using “real world” (RW) data as an important tool in evaluating drug safety and efficacy [2].

In December 2018, FDA released an FDA’s RWE Framework (henceforth called Framework) [3]. The Framework defines RWD as “data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources,” and RWE as “the clinical evidence about the usage and potential benefits or risks of a medical product derived from analysis of RWD.” Examples of RWD in the Framework include data derived from electronic health records (EHR), medical claims and billing data, data from product and disease registries, patient-generated data and data from other sources, such as mobile devices. The Framework further indicates that RWD sources can be used for data collection and to develop analysis infrastructure to support many types of study designs to develop RWE, including, but not limited to, randomized trials (e.g., large simple trials, pragmatic clinical trials) and observational studies (prospective or retrospective).

More recently, the PDUFA VII Commitment letter for fiscal years 2023 through 2027 [4] provided further details on the FDA RWE program and indicated the following key aspects:

  1. (a)

    By no later than December 31, 2022, FDA will establish and communicate publicly a pilot Advancing RWE Program.

  2. (b)

    The Advancing RWE Program will include, but not be limited to, a list of activities and components, some of which include (1) FDA will solicit applications for RWE programs; (2) FDA will use structured review process to evaluate and rank applications; (3) FDA will accept one to two eligible and appropriate proposals each cycle, and several additional activities FDA will convene following the solicitation and application.

  3. (c)

    By no later than June 30, 2024, FDA will report aggregate and anonymized information, on at least an annual basis and based on available sources (e.g., information provided by review divisions), describing RWE submissions to CDER and CBER.

  4. (d)

    By no later than December 31, 2025, FDA will convene a public workshop or meeting to discuss RWE case studies with a particular focus on approaches for generating RWE that can potentially meet regulatory requirements in support of labeling for effectiveness.

  5. (e)

    By no later than December 31, 2026, experience gained with the Advancing RWE Program, as well as CDER’s and CBER’s RWE program in general, will be used to update existing RWE-related guidance documents or generate new draft guidance, as appropriate.

Chapter “Overview of Current RWE/RWD Landscape” provides more in-depth information on the regulatory guidance documents in recent years in key regions around the world. Further, there have also been increasing public and private collaborations in RWE research. Examples include the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and International Society for Pharmaceutical Engineering (ISPE) special joint task force on “good practices for RWD studies of treatment and/or comparative effectiveness (CER)” [5], and “reporting to improve reproducibility and facilitate validity assessment for healthcare database studies” [6]. Launched in October 2013, the GetReal was a three-year project of the Innovative Medicines Initiative, a Europe’s largest public-private consortium consisting of pharmaceutical companies, academia, Health Technology Assessment (HTA) agencies and regulators, patient organizations, and subject matter experts (SMEs). The efforts resulted in numerous publications including delivery of four work packages [7]. Within the statistical community in the United States, the American Statistical Association (ASA) Biopharmaceutical Section (BIOP) sponsored an RWE Scientific Working Group (SWG) that started in April 2018. The primary goal of the group is to advance the understanding of the RWE research in a precompetitive space, and the membership consists of members from FDA, academia, and industry. The group has produced or submitted six peer-reviewed publications:

  • The Current Landscape in Biostatistics of the use of Real-World Data and Evidence for Medical Product Development: General Considerations [8]

  • The Current Landscape in Biostatistics of Real-World Data and Evidence: Clinical Study Design and Analysis [9]

  • The Current Landscape in Biostatistics of Real-World Data and Evidence: Causal Inference Frameworks for Study Design and Analysis [10]

  • Estimands – From Concepts to Applications in Real-World Setting [11]

  • Statistical Consideration for Fit-For-Use Real-World Data to Support Regulatory Decision Making in Drug Development [12]

  • Examples of Applying Causal Inference Roadmap to RWE Clinical Studies [13]

With the encouragement from regulators and available guidance and literature on the use of RWD and RWE in recent years, we have seen an increased uptake of RWE in various stages of drug development. Figure 1, which is adapted from the figure in [14], depicts the various uses in different stages of drug development and their reliance on RWD in representative types of study design. Together with guidance in the Framework on the usage of RWD, we summarize key usages of RWD in clinical development and life-cycle management as follows, but the list is by no means exhaustive:

  • Generate hypothesis for testing in RCTs.

  • Identify investigators who provide care for patients with the disease or condition of interest, thereby selecting study sites with appropriate investigators.

  • Assess disease prevalence and sub-population of patients identified by phenotype or genotype, thus assisting with patient selection and enrollment.

  • Assess the prevalence of concomitant medications for a disease, along with prevalence of comorbidities of the disease.

  • Evaluate biomarker prevalence and discover target for the development of personalized medicine.

  • Evaluate indication calibration by assessing unmet medical need and whether the need is consistent across the targeted population.

  • Identify outcome measures by ascertaining background event rate in a disease and related population for incidence, duration, severity.

  • Fulfil regulatory safety commitment, safety surveillance, and safety label update.

  • Enroll patients at point of care and leverage existing RWD, such as EHR and administrative claims, to retrieve historic information and long-term follow-up, thereby employing a so-called hybrid study design to use both existing RWD and prospectively collecting additional study information.

  • Use RWD to build external control cohort for single-arm clinical studies or augment concurrent control group of an RCT.

  • Describe patient journey, treatment pattern, healthcare utilization to assess unmet medical needs and disease burden and facilitate choice of comparator.

Fig. 1
An illustration of the R W E spectrum has the following levels from top to bottom based on the increasing reliance on real-world data. 1. Traditional Randomized controlled clinical trials using R W D in planning. 2. R C T in clinical practice settings. 3. Nonrandomized, interventional study. 4. Nonrandomized, noninterventional study.

RWE spectrum

Full size image

With the above delineation, the use of RWE could lead to support approval of new molecular entities or biologics, accelerate or seek conditional approval, explore new indication or new population, make changes to dosing administration, supplement RCTs information for a regulatory submission, or provide complementary evidence for comparative effectiveness and cost-effectiveness assessment for reimbursement decisions in HTA.

For the rest of the chapter, in Sect. 2, we review the progress to date on the uptake of RWE. Even with these recent progresses, challenges remain. We interpret these challenges as opportunities for further research and development, as described in Sect. 3. The final section provides discussions on future directions and concluding remarks.

2 Where We Are Now with the Use of RWE and RWD

In the last few years, a great stride has been made in advancing the uptake of RWE in drug development. The prevailing environment from regulators is that of encouragement and guidance, along with concrete action plan, as shown by the PDUFA VII Commitment letter for fiscal years 2023 through 2027 [4]. In this section, we will discuss a few major advances as we observed in recent years, focusing primarily on the advancement as described in this book. However, there have been numerous literatures on RWE- or RWD-related publications, such as the work by ASA BIOP RWE SWG, and guidance and publications by regulators and interest groups. It should be noted that the discussion here is by no means thorough, and any further gaps remain to be filled by further observations and research.

2.1 Regulatory Advancement

In the regulatory arena, in rapid succession, FDA released four draft guidance in late 2021 on RWD on assessment EHR, medical claims, and registry data to support regulatory decisions in two guidance documents; data standard for regulatory submission is also delineated in another guidance, along with considerations for the use of RWD and RWE for drug and biologic products in the fourth draft guidance. In October 2021, EMA adopted guideline on registry-base studies. Chapter “Overview of Current RWE/RWD Landscape” provides a good coverage of guidance documents related to RWE from the United States, Japan, China, and the United Kingdom, and discussed similarities and differences between them. The European Medicines Agency (EMA)’s RWE Vision is that, by 2025, the use of RWE will have been enabled and the value will have been established across the spectrum of regulatory use cases [15]. In 2022, EMA established a Coordination Centre for the Data Analysis and Real World Interrogation Network (DARWIN EU®) [16]. DARWIN EU will deliver RWE from across Europe on diseases, populations, and the uses and performance of medicines.

Health Authorities responsible for reimbursement and pricing reviews in HTA submissions have also released draft guidance documents on the use of RWE for HTA submissions. The National Institute for Health and Care Excellence (NICE), the United Kingdom’s HTA body, released NICE RWE framework in June 2022 [17]. The key message is that RWD can improve our understanding of health and social care delivery, patient health and experience, and the effects of interventions on patient and system outcomes in routine settings. As described in NICE strategy 2021–2026 [18], NICE wanted to use RWD to resolve gaps in knowledge and drive forward access to innovations for patients. In the rest of the world, French National Authority for Health (HAS) released a guidance in June 2021 on RW studies for the assessment of medicinal products and medical devices [19]. Further, due to limited recommendations to support the appropriate use of RWE, a group of experts from top European Union (EU) academic institutions and HTA bodies in eight countries as part of the EU’s Horizon 2020 IMPACT-HTA program published a white paper on the use of nonrandomized evidence to estimate treatment effects in HTA [20]. The key messages are:

  • RWE must be relevant for the research question.

  • They recommended strategies to study design and analysis.

  • The white paper deemed transparency as essential.

  • The paper also recommended strengthening infrastructure and investing in resources to design, analyze, and interpret RWE.

Chapter “Use of Real-World Evidence in Health Technology Assessment Submissions” of this book provides more details on the use of RWE in HTA submissions. In summary, these various guidance documents all provided a similar message on fit-for-purpose use of RWE and RWD.

2.2 Advancement in Operational Considerations

Several chapters in this book covered operational considerations in implementation in the use of RWE.

In the RW studies (RWS), key variables such as exposure, treatment, outcome, disease status, or confounders may not be captured in one place, it is therefore important to ascertain these key variables using advance analytics, such as machine learning and nature language processing. Misclassification is also a concern, requiring validations. Chapter “Key Variables Ascertainment and Validation in Real-World Setting” of this book covers these topics and walks through an example study for which the ascertainment of key variables was found to be acceptable from a regulatory standpoint. Once the key variables are in place for an RWD source, assessment of fit-for-use RWD sources is a critical step in the determination of whether an RWD source could be used. Chapter “Assessment of Fit-for-Use Real-World Data Sources and Applications” provides guiding principles in the fit-for-use RWD assessment and illustrates assessment steps with an application. The authors drill down into details on the factors to consider specific to a research question and disease condition and provides sufficient details to allow practitioners to follow in their applications.

The role of health data and interoperability standards is another important element to consider in their harmonization, since lack of harmonization and common data standards would impede the foundation for a vision to achieve large-scale interoperability in supporting technical, methodological, and evidence generation, based on emerging trends. Chapter “Data Standards and Platform Interoperability” presents a discussion on the need for Findable, Accessible, Interoperable, and Reusable (FAIR) data, and the role data standards, in particular those emerging as leading with regards to regulatory decision, and emerging platforms for network, at-scale evidence generation, as unified visions for standards and platforms. It is often of great interest to aggregate and link data from several RWD sources to provide a more comprehensive longitudinal evaluation of treatments from different aspects. Chapter “Privacy-Preserving Data Linkage for Real-World Datasets” reviews privacy framework and different methods in linking data sources, while focusing on patient privacy protection, data pre-processing, linkage, and performance evaluations.

2.3 Advancement in Statistical Methodologies in Causal Inference

Tremendous progress has been made not only in the methodologies of causal inference but also in the applications of these methods in the uptake of RWE generations. Chapter “Causal Inference with Targeted Learning for Producing and Evaluating Real-World Evidence” summarizes a Target Learning roadmap as a systematic guide to navigate the study design and analysis challenges inherent in real-world studies. ICH E9 (R1) Addendum [21] presents a structured framework to strengthen the dialogue between disciplines involved in the formulation of clinical trial objectives, design, conduct, analysis and interpretation, as well as between sponsor and regulator regarding the treatment effect of interest that a clinical trial should address. Further, the guidance indicates that “The principles outlined in this addendum are relevant whenever a treatment effect is estimated or a hypothesis related to a treatment effect is tested, whether related to efficacy or safety. While the main focus is on randomized clinical trials, the principles are also applicable for single-arm trials and observational studies.” Chapter “Framework and Examples of Estimands in Real-World Studies” presents principles for the Estimand Framework for use in RW setting, highlights similarities and differences between RCTs and RWS, and provides a roadmap for choosing appropriate estimand for RWS.

Chapter “Clinical Studies Leveraging Real-World Data Using Propensity Score-Based Methods” provides a comprehensive summary of propensity score-based methods (PSM) to minimize confounding biases in clinical studies leveraging RWD sources. Beyond PSM, chapter “Recent Statistical Development for Comparative Effectiveness Research Beyond Propensity-Score Methods” presents recent statistical developments for comparative effectiveness research using methods, such as G-methods. Chapter “Innovative Hybrid Designs and Analytical Approaches leveraging Real-Word Data and Clinical Trial Data” showcases an innovative hybrid design and analytical approaches leveraging RWD and clinical trial data, while chapter “Statistical Challenges for Causal Inference Using Time-to-Event Real-World Data” highlights statistical challenges for causal inference using time to event RWD. As we know, the lack of randomization in RWD brings the potential for bias into any comparisons between groups or interventions of interest. Commonly used methods such as PSM can account only for confounding variables that are included in the analysis database, but any confounders not contained in the database are ‘unmeasured confounders’ and may result in a biased treatment effect estimate. Chapter “Sensitivity Analyses for Unmeasured Confounding: This is the Way” focuses on the challenging case of comparative analyses based on RWD and the issue of unmeasured confounding. Further, ICH E9 (R1) discusses the importance of sensitivity analysis [21]. Chapter “Sensitivity Analysis in the Analysis of Real-World Data” guides readers on how to conduct sensitivity analysis to explore the robustness of inference to deviations from the underlying assumptions.

The practice of modern medicine demands personalized medicine (PM) to improve both quality of care and efficiency of the healthcare system. Chapter “Personalized Medicine with Advanced Analytics” dives into the application of advanced analytics in addressing PM research questions, while chapter “Use of Real-World Evidence in Health Technology Assessment Submissions” covers the utility and strengths of well-developed RWE in HTA decision-making in major regions around the world.

2.4 Advancement in Real Case Applications

The concept of causal inference framework and use of causal inference roadmap is crucial in the use of RWD to generate robust RWE. Chapter “Examples of Applying Causal-Inference Roadmap to Real-World Studies” describes a few examples of applying causal inference roadmap to RWSs. Chapter “Applications Using Real-World Evidence to Accelerate Medical Product Development” summarizes six case studies that regulatory agencies considered in recent years in the use of RWE/RWD for regulatory decisions. Some of these use cases succeeded in achieving positive regulatory decisions, while a couple of others didn’t meet the principle of adequate and well-controlled study for evidentiary standard. This chapter includes rich details on the analysis of each case study. Finally, chapter “The Use of Real-World Data to Support the Assessment of the Benefit and Risk of a Medicine to Treat Spinal Muscular Atrophy” presents a detailed case study in Spinal muscular atrophy (SMA) and describes how RWD from publications and individual patient data were used to support the development of risdiplam, a medicine to treat SMA.

3 Opportunities for Further Advancement

3.1 Regulatory Context

With the release of numerous regulatory guidance documents in recent years from regions around the world, there is a prevailing need to share more use cases. Through use case studies, practitioners could understand better the regulatory contexts, key regulatory review issues, whether the use of RWE/RWD is pivotal or supplemental for the regulatory decisions, assessment of fit-for-use data sources, statistical methods employed, and whether substantial evidence of effectiveness as stated in Regulations 21CFR314.126 is met for a specific case study. With the encouragement from the Framework and more emerging literature on the changing landscape of regulatory approval processes and case examples as delineated in chapters “Applications Using Real-World Evidence to Accelerate Medical Product Development” and “The Use of Real-World Data to Support the Assessment of the Benefit and Risk of a Medicine to Treat Spinal Muscular Atrophy”, we believe that we will see more and more such use cases in the coming years. Further, through a feedback loop between sponsors and regulators, existing RWE-related guidance documents could be updated, or new draft guidance could be developed.

Considering the evolving and diverse regulatory frameworks across jurisdictions, sponsors are encouraged to engage with regulatory agencies and other stakeholders, ideally through joint scientific advice procedures, when applicable, such as EMA/FDA parallel scientific advice [22]. Further, the use of RWE for regulatory submissions and decisions is still relatively new. The FDA draft guidance on data standards for drug and biologic products submissions containing RWD provide guidance on data standards and data mapping, along with the development of review guide for such submissions [23]. It’s helpful for sponsors to gain further experience in these areas and engage regulators for further advice as needed.

3.2 Clinical Context

Up until just a few years ago, RWE has been used primarily to perform post-marketing surveillance to monitor drug safety and detect adverse events or in HTA submissions to understand disease burden, drug effectiveness, or economic modeling. To expand the use to support clinical development and life-cycle management, it is important to consider the clinical contexts regarding the clinical question of interest and whether RWS that generate RWE are sufficient and robust enough for the regulatory question at hand. We believe that RW studies should not be used as a replacement for RCTs, since all the design precautions and/or statistical techniques could still not overcome unquantifiable or poorly recorded data inherent with RWD. However, if used appropriately, RWE could be used to support regulatory decisions in certain situations.

The PRECIS-2 tool [24] is a refined tool of PRECIS (Pragmatic Explanatory Continuum Indicator Summaries) that was intended to help trialists make design decisions consistent with the intended purpose of their trial. PRECIS-2 tool contains nine domains – eligibility criteria, recruitment, setting, organization, flexibility (delivery), flexibility (adherence), follow-up, primary outcome, and primary analysis, scored from 1 (very explanatory) to 5 (very pragmatic). The authors argued that although we often refer to trials as in ideal RCT setting or in RWS, there is no simple threshold to separate the two concepts. Rather than a dichotomy, there is a continuum between the two, by adjusting the factors in either design or study conduct to make a trial more RCT or RWS. Undeniably, clinical context is critically important in determining whether the aim is to answer the question, “Can this intervention work under ideal considerations?” or “Does this intervention work under usual conditions?” The internal and external validity and generalizability can be inferenced from such considerations.

3.3 Study Design and Analysis Context

Chapter “Key Considerations in Forming Research Questions” reviews and identifies key elements of forming sound research questions in RWS. The PROTECT criteria proposed in [25] is discussed in-depth in chapter “Key Considerations in Forming Research Questions”. Further, the authors propose a roadmap for revising a research question and/or element of the PROTECT criteria if a question cannot be answered as framed. The authors’ way of setting up right research questions is quite innovative, as they use Estimand framework as “touchstone” to gauge whether a question can be answered or not.

There has been a flurry of literature on the statistical methodologies in analyzing RWD and translating data into robust RWE. Given that rich literature exists on statistical methodologies to handle potential biases and confounding with the use of RWD [9], methodologies are discussed in several chapters in Part III of this book. We believe that it’s important for practitioners to understand these approaches, especially sensitivity analysis, to assess the robustness of the findings and apply them appropriately in their RWE projects. We would also like to provide some cautions in methodology development. While many RW study design and/or methodologies have been proposed, some of them might be more of an intellectual interest with less appeal for practical applications. Thus, focusing on those adaptations that are practically feasible will result in the most successful implementations as the research enterprise is collectively gaining experience with this new and evolving field.

3.4 Data Context

Chapter “Assessment of Fit-for-Use Real-World Data Sources and Applications” of this book provides guiding principles for assessing fit-for-use RWD sources in data relevancy and reliability. The authors also illustrate the approach via a hypothetical example. However, further research may still be needed since the actual assessment is very much disease and research question-specific. Further, it may be a good idea for sponsors to engage regulators for discussions on the data source, and rationale and justification on the fit-for-use assessment. As EMA/HMA calls for in [16], it is important to establish and expand catalogues of observational data sources for use in medicines regulation, provide sources of high-quality, validated RWD on the uses for safe and effective medicines, and address specific questions by carrying out high-quality, non-interventional studies, including developing scientific protocols, interrogating relevant data sources, and interpreting and reporting study results. In terms of data sources, technological advancements in health technology and digital wearable devices will become potential sources of RWD. The key to their application in RWE is to ensure that the data generated is of high quality and fit for purpose.

We believe that it will be ideal to establish an industry standard for how an RWD source should be assessed and what criteria constitute a fit-for-use database.

3.5 Governance and Infrastructure Context

In addition to challenges as mentioned previously in this section, there are also additional challenges from resource, logistic, operational, and organizational perspectives. Utilization of RWD and RWE involves cross-functional expertise and collaboration, so building these features into an organization’s processes, systems, and culture is a prerequisite for uptake. An upfront investment in dedicated resources may be needed, such as building or updating processes in clinical development procedures, developing templates of brand development plans and tools, and providing education on RWD sources and RWE methodologies. Change management may be needed to overcome entrenched decision-making processes that are skeptical about the use of RWE.

Especially, we recommend setting up governance to oversee the data acquisition and usage; developing processes and procedures that facilitate the regulators’ requirements for transparency, pre-specification, consistency, reproducibility, and compliance in RWE applications; understanding existing RWD sources and properties and data owner networks; developing data platform to facilitate data flow, data harmonization from diverse sources, and connectivity for research use; and building analytic platform with powerful computational capacity for big data processing and re-usable analytic tools along with centralized coding library to define disease cohorts, exposures, outcome measures, and confounders in a consistent manner.

4 Future Direction and Concluding Remarks

In the past 5 years, we have seen growing international interest among all healthcare stakeholders regarding how to best approach the uptake of RWE and RWD to revolutionize the drug development process. The robust legacy of scientific groundwork as described in this book and regulatory guidance and other literature in recent years has paved the way to the future. What will be the challenges and opportunities for the uptake of RWE over the next 5 years?

In Sect. 3, we discuss opportunities for further advancement in the uptake of RWE from different areas of focus. While the common elements for further advancement have been identified, we expect that the next 5 years will see refinement in the use of specific tools and techniques by regulators around the world. Some agencies may focus on data quality and data platforms, while others may explore novel approaches integrating different sources of RWD for use and further refine guidance documents. Medicine development is a global endeavor. Sponsors therefore will seek a consistent degree of process predictability across target jurisdiction regulatory agencies, and this can come from the use of globally acceptable, standardized, systematic approach to RWE, irrespective of the specific tools and methodologies that each employ in support of its regulatory decisions.

As we have seen more and more collaborations across regions in the world, such as EMA/FDA parallel scientific advice [22] and EUnetHTA21 [26] for an effective and sustainable network for HTA across Europe, a structure process can facilitate work sharing and the potential for joint reviews and improve information sharing with industry partners and other stakeholders. These types of collaborations could also provide a clearer understanding of rationales for different marketing and labeling decisions in different jurisdictions, such as clinical context and the practice of medicine, and alignment of risk management plans. The next 5 years will also see the growing uptake of RWE in regulatory decisions. Whether interacting with regulatory or HTA agencies, establishing a dialogue with the stakeholders early during medicine development can contribute to effective, ongoing communications with a more consistent understanding and implementation of the expectations from each stakeholder.

As RWE becomes the new information currency in healthcare, decision makers will be challenged using these new types of data sources. Over the next 3–5 years for some therapeutic areas, such as oncology or rare diseases, there may be a shift to the use of integrating RWD into phase II or phase III clinical studies. As development progresses, RWE will enhance the understanding of the product’s safety profile and will be used to confirm clinical efficacy and RW effectiveness.

Of course, the use of RWE in drug development will not be without challenges. Great progress has been made on the methodologies to assess the robustness and uncertainty around factors that confound the interpretation of RWE. Further refinement and new methodology development may be called for based on the use cases. RWE collection will need to encompass a global view or, at the least, focus on key markets and jurisdiction experiences. Building a federate model and platform for data and analytic tools sharing may facilitate further leapfrogging in the field. Building high quality RWD and making them widely available may call for standardization of data, such as the use of common data model. Transparency, pre-specification, consistency, documentation, and reproducibility will be the cornerstone to which current and new facilitated regulatory pathways that are designed to accelerate submissions, reviews, and patient access to medicines for serious diseases where there is an unmet medical need will likely be accepted. These new pathways, such as Breakthrough Therapy and Accelerated Approvals in the United States, or Conditional Marketing Authorization in the European Union, may increase the communications and level of commitment between the sponsors and the agencies.

Finally, we want to emphasize that these opportunities to incorporate RWE in drug development should be used with care. The last thing we want to do is to treat opportunities offered haphazardly, which will result in rejected submissions and lead to mistrust in the use. The latter will delay broad acceptance of properly designed and executed studies and submissions incorporating RWE.