Keywords

Background

When discussing with a patient the indication, risk and benefits of an operative or non-operative invasive procedure, the physician must assess multiple factors that may influence the expected outcome and formulate individualized recommendations for the individual patient. This assessment requires a number of clinical decisions as how best to optimize every aspect of care, from presentation to recovery, minimizing the potential for unexpected complications, morbidity and mortality. Patients expect this personalized treatment plan to be based upon the best and most up to date scientific evidence applied to their specific situation.

The problem clinicians face is a scarcity of the highest quality scientific evidence to guide most of these treatment decisions [1]. Healthcare policy makers similarly suffer from a lack of data as they attempt to create systems that produce the most cost-effective, highest quality care [2]. These knowledge gaps lead to decision making that is arbitrary, based on clinician impressions and bias rather than on evidence, and result in variability in practice across clinicians with delivery of suboptimal care and inefficient use of valuable resources [3].

The randomized controlled clinical trial is the gold standard for medical evidence generation. Rigorous traditional clinical trials enroll a homogeneous patient population and attempt to control to the extent possible for variations in clinical practice. As such these trials are considered ‘explanatory’ in that they determine treatment superiority in an idealized setting and form the basis of FDA approval of new therapeutics or label changes in already-marketed drugs. The idealized experimental environment of explanatory clinical trials and the highly selected patient population accounts for the skepticism of clinicians to adopt recommendations based on their findings and accounts for the lag between publication of study results and acceptance by the medical community (the T2 translation gap).

In contrast to explanatory clinical trials, pragmatic studies are designed to inform clinical decision making and typically compare the effectiveness of two or more treatment interventions in settings that more closely reflect usual care. Study selection criteria and procedures mandated by the study protocol are relaxed and lead to enrollment of a more diverse patient population whose treatment more closely resembles that delivered in usual care. Pragmatic clinical trials vary considerably in the extent to which they are integrated into clinic practice, and studies comparing treatment options already in widespread use (comparative effectiveness studies) fit best into the pragmatic framework [4]. It is important to point out that even pragmatic clinical trials can be overly ‘operationalized’ and lose many of the benefits (efficiency, scalability) that this design type offers [4–6].

Widespread adoption of electronic health record systems has made possible a transformational change in pragmatic clinical trial design—the point-of-care (POC) clinical trial. These trials embed clinical trial processes such as subject randomization and ascertainment of outcomes unobtrusively into the electronic health record to the fullest extent possible [7]. The ability to embed clinical trial operation seamlessly into the clinical care ecosystem minimizes the distinctions between clinical care and research and generates more generalizable results that can be rapidly implemented. Other features of POC studies include reduced cost (no need for a separate clinical trial apparatus to treat patients) and greater scalability (from less stringent selection criteria) that allows for rapid iteration of clinical trials and incorporation of findings directly and efficiently into clinical practice as decision support, creating an integrated environment of research-based care that defines a learning healthcare system [8] (Figs. 13.1, 13.2 and 13.3).

Fig. 13.1
figure 1

Panel A. Traditional clinical workflow: choice of intervention is selected by the healthcare provider and the patient is followed for outcomes

Full size image
Fig. 13.2
figure 2

Panel B. Traditional research silo: clinical care and research operate independently. Healthcare provider is in equipoise, and the choice of an intervention is randomized. Defined subset of patients enters the research workflow for structured follow-up. Outcomes often not fed back into clinical workflow

Full size image
Fig. 13.3
figure 3

Panel C. Integrated learning: clinical care and research operate together to create a learning healthcare system. Healthcare provider is in equipoise, and the choice of an intervention is randomized. Patient is randomized but stays in the traditional clinical workflow

Full size image

Definition and Trial Design Characteristics for POC Clinical Trials

POC clinical trials provide a mechanism to perform large, simple and clinically integrated randomized trials to answer a plethora of compelling clinical questions. As stated, the defining feature that distinguishes POC trials is the use of EHR systems to embed the trial in routine clinical care to the maximum extent possible. Ideally, the possibility of randomization should be presented to the provider and patient when a treatment decision needs to be made, with confirmation of eligibility and the informed consent process being the only perturbations from usual care.

Trials best suited to POC methodology address clinician’s uncertainty in the use of common clinical interventions that lack comparative effectiveness data to guide decision making and where healthcare providers do not have a strong treatment preference and are therefore willing to allow randomization (referred to as having clinical equipoise). The interventions studied should be used in an open-label manner consistent with the intervention’s usual and intended practice, and not for new or expanded indications. The safety profile of the treatment, procedure or device should be well known allowing for risk-based monitoring, and exclusion criteria should be minimal allowing for a broad and easily identifiable eligible patient population. Follow-up procedures should follow usual care with minimal or no additional study requirement or visits, and all required data elements should be readily accessible and resident in the electronic health systems databases. Finally, outcomes should be clinically important and to the extent possible ascertained from structured data elements in the EHR [7, 9]. Linkage of multiple health databases (e.g., inpatient and outpatient) can improve capture and confirmation of endpoints and facilitate long-term follow-up [9, 10].

An exemplar point-of-care interventional study is the Thrombus Aspiration during ST-Segment Elevation Myocardial Infarctions in Scandinavia (TASTE) trial [11]. This multicenter open-labeled trial identified and obtained oral informed consent for participants presenting with ST-segment elevation myocardial infarctions (SEMI) to be randomized to manual coronary artery aspiration prior to percutaneous coronary intervention (PCI) or PCI alone. The primary endpoint of the study, all-cause mortality at 30 days, was not reduced by thrombus aspiration. One year follow-up continued to show no difference in mortality [12].

The design and conduct of TASTE demonstrates many of the key features of POC design and conduct. The study addressed a clinically relevant question in which there was equipoise in both the literature and in clinical practice. The outcome was important to patients and providers garnering a willingness to participate. TASTE study procedures were seamlessly integrated into the clinical workflow through minor modifications of the electronic health record system that facilitated study execution. The modifications allowed providers to confirm patient eligibility and document that oral informed consent was obtained from patients. Documentation of these events triggered randomized treatment assignment to thrombus aspiration or usual care study arms. No extra study-specific activities were required, and there was no attempt to blind treatment allocation. Because complications of the study procedures (clot aspiration and PCI) are well established, monitoring of adverse events proceeded as they would for usual care. The outcome of all-cause morality at 30 days was easily captured and confirmed by linkage to national death registries.

Electronic Health Record Systems Requirements

Implementation of POC trials involves a multidisciplinary team, including clinicians, researchers and informatics personnel. Execution of a randomized clinical trial within the clinical care ecosystem is facilitated by an electronic health record system with sufficient flexibility to allow for adaptations required for the study. The best systems are modular and generalizable to allow customizable workflows and data objects.

Essential functionalities include the ability to identify, enroll, randomly assign and implement the study intervention and track all necessary data elements from all participants. Creation of workflows outside of clinicians’ usual clinical care interaction with the electronic health record and use of additional complimentary information systems should be avoided. Not only is development of new software or additional functionality to existing systems resource intensive, the addition of unfamiliar workflows and applications reduces the willingness of clinicians to participate in the program [6, 10].

There are inherent trade-offs that come with using electronic health system-generated databases in point-of-care trials. Data resident in electronic healthcare data systems are easy to access but are primarily collected in nonstandard formats and with varying degrees of accuracy depending on the intention and sophistication of the stakeholder entering the information. Additionally, data aggregated from other sources such as registries and quality assurance databases introduce additional variability in data quality. An understanding of the provenance of all data elements, how and by whom the data elements were collected and some assessment of internal validity is critical to designers of embedded clinical trials and has important ramifications on all aspects of study design such as selection of inclusion and exclusion criteria and definition of study endpoints and adverse events. Outcomes that require data contained only in free text or that require additional adjudication add to the cost and complexity of the trial and should be avoided when possible. Finally, centralized data monitoring systems need to be in place to assure that there is no disruption or change in data availability or structure over the lifetime of the study [9, 10].

Analytical Considerations

The heterogeneity introduced by inclusion of a more diverse study population and real-world implementation in the clinical ecosystem used by POC methodology presents issues when using frequentist analytic approaches. Techniques used to account for this variability result in trials with increased sample sizes, increased time to reach accrual targets, subsequent delays in trial completion and increase in cost.

Use of Bayesian adaptive approaches has been touted as a more efficient statistical method for use in pragmatic comparative effectiveness research [3]. The dynamic features of Bayesian and adaptive approaches tolerate uncertainty and allow for change in trial design as information accumulates during trial conduct. This approach allow for changes such as alteration of the randomization allocations based on information that accumulates during trial conduct, the ability to incorporate new interventional arms and adaptively dropping arms for futility thereby enriching enrollment in surviving options. The capability to produce informative results sooner using smaller samples sizes in a more cost-effective scalable manner has led to increased adoption of Bayesian adaptive approaches by pharmaceutical, device and biotechnology products research and development programs [3].

Decision Support and Creation of a Knowledge Base

Optimally, the electronic health record system used to conduct a POC trial can be adapted to implement the findings of the comparative effectiveness research as decision support. Clinician buy-into transition from pragmatic trial to decision support is facilitated by the nature of the trial—that it was executed within the healthcare system itself and studied the extant patient population. While findings from pragmatic trials may be readily adopted locally (locally selfish research), they may or may not be relevant for other healthcare systems with different patient populations and practice patterns, that is they may lack generalizability [7].

Results from POC trials embedded in clinical care can be combined with relevant background eternal knowledge to create a customized prediction model for individual patients. Creation of such a knowledge base is described by the VA Point of Care Precision Oncology Program [13].

Real-Time Implementation of Trial Results

Pragmatic clinical trials embedded in a healthcare system offer a unique opportunity to close the so-called implementation gap. This is accomplished by a hybrid approach, using frequentist operating characteristic and Bayesian adaption of randomization allocation as proposed in a Department of Veteran Affairs POC trial comparing methods for inpatient insulin administration [7]. The study analysis plan uses adaptive randomization modifying the assignment probability, after accrual of a fixed number of patients, preferentially to the winning therapy using a stopping rule with the acceptable frequentist Type 1 error of an efficiency trial. As a result, if a superior treatment exists by the time the study winner is determined, the majority of patients would have been randomized to the better treatment, thereby having implemented the finding as it was determined. The inferior treatment could then be more easily shut off without significant numbers of patients receiving the inferior treatment. Alternatively if the study fails to reach its efficiency boundary by study termination, then no substantial therapeutic difference exists and other factors such as cost, ease of use or clinician preference come into play in determining the clinical recommendation.

Summary

Point-of-care methodology is well suited for experimental comparative effectiveness research in the conduct of operative or non-operative invasive procedures [14]. Since labeling approval of devices and technology do not require comparative trials (21 CFR860.7(c)2), as required for drug approval, new devices, hardware, robotics, imaging and operational techniques are continuously evolving and can be quickly adopted into practice with little or no comparative evidence showing either improvement in meaningful clinical outcomes or quality of care. POC trials provide a mechanism to compare their use impact on clinically important outcomes such as death, infection or organ failure in an open-label manner within clinical practice. These important outcomes are often routinely captured with some degree of validation in quality assurance and improvement program electronic databases. Bayesian and adaptive designs are also particularly useful in the field as outcomes often occur in a shorter time period following the procedure allowing for adaptive randomization allocation [3] and therefore continuous evaluation. In addition, optimization of peri-procedural management, such as use of anticoagulant, antiplatelet therapy, infection prophylaxis, renal protection from dye load, can be evaluated as new drugs and formulations are adopted into practice.

The principle advantages of using POC trials include lower cost and generation of research results that are more likely to be implemented by the providers who have generated the evidence. This methodology provides a means to institutionalize a process where learning from each patient encounter can help determine the best care for the next patient—an integrated environment of research based care. There are important limitations on the questions that POC trials can address and on the outcomes that can be used as endpoints. Clinical equipoise is a hard requirement, and the questions asked need to be considered important to clinicians and patients. There is an operational dependency on electronic health record systems that are configurable and that have some capacity for incorporation of work flows.

An additional challenge to future expanded use of point-of-care trials methodology is the reexamination of regulatory governance and ethical oversight that has become the norm for human subjects’ research [4, 9]. In particular should the same degree of human subject protection be required for experimental comparative effectiveness research comparing approved treatments, as that used for drugs or devices that are under development? A rethinking of the regulations regarding research consent and engagement, recognizing the different order of human experimentation in comparative effectiveness trials of widely used treatments, is required to facilitate dissemination of POC clinical trials and accelerate the transformation in healthcare that the methodology can provide.