FormalPara Key Points

We prospectively monitored the safety and effectiveness of prasugrel versus clopidogrel in routine care during the first 2 years following the approval of prasugrel.

The prospective, parallel-group, propensity score-matched, new user cohort approach combined with sequential alerting algorithms did not generate any alerts for prasugrel.

This study demonstrates the feasibility of near-real-time monitoring of a new drug in electronic healthcare data.

1 Introduction

Prasugrel is an adenosine diphosphate receptor inhibitor that reduces the risk of thrombotic cardiovascular events in patients with acute coronary syndrome (ACS) who are managed with percutaneous intervention (PCI). The US FDA approved prasugrel in July 2009 based on support from the TRITON-TIMI (Trial to Assess Improvement in Therapeutic Outcomes by Optimizing Platelet Inhibition with Prasugrel-Thrombolysis in Myocardial Infarction)-38 study, which compared prasugrel with clopidogrel in 13,608 patients receiving aspirin [1]. As compared with those who received clopidogrel, patients treated with prasugrel were 19 % less likely to experience the primary efficacy endpoint of death from cardiovascular causes, nonfatal myocardial infarction (MI), or nonfatal stroke (hazard ratio [HR] 0.81; 95 % confidence interval [CI] 0.73–0.90), but were more likely to experience major bleeding (HR 1.32; 95 % CI 1.03–1.68), life-threatening (HR 1.52; 95 % CI 1.08–2.13), and fatal (HR 4.19; 95 % CI 1.58–11.11) bleeding events.

The bleeding risk observed with prasugrel in the well controlled TRITON-TIMI-38 study raises concerns that the drug may further elevate bleeding risk in typical clinical practice [2]. Drug effects observed in trials may not reflect experiences in practice because of differences in patient characteristics between the two settings as well as differences in treatment adherence, duration of follow-up, concomitant drug use, and differences in clinical visit schedules [3]. Patients were excluded from TRITON-TIMI-38 for many reasons, including if they had a history of bleeding or were at an increased risk of bleeding, used daily nonsteroidal anti-inflammatory drugs (NSAIDs), or had any condition associated with poor treatment compliance, such as alcoholism or mental illness [4].

Recognizing the importance of rapidly identifying safety concerns that arise as medical products move from pre-marketing testing and into widespread use in clinical practice [5], Congress, through the 2007 FDA Amendments Act, mandated the FDA to establish a national system to track the safety of regulated medical products. The FDA’s Sentinel System, and others like it [6, 7], will use electronic healthcare data that are collected for large populations and in near real time and that can supply timely comparative effectiveness and safety information on drugs as they are used in actual practice [5]. An important component of these systems is the ability to prospectively analyze the electronic data as they accumulate in order to identify differences in treatment outcomes as early as possible.

We previously developed an approach to active drug safety monitoring and evaluated it in several retrospective applications [7–9]. During the first 2 years of prasugrel’s market availability in the USA, we applied the approach to prospectively monitor the drug’s safety and effectiveness as compared with clopidogrel, with respect to hemorrhagic and ischemic events, in a large electronic data environment that reflects how these drugs are used in clinical practice. In this paper we describe the methodological details of the approach for prospective monitoring and present the results of the prasugrel versus clopidogrel comparison.

2 Methods

2.1 Data Source

We used the HealthCore Integrated Research Database (HIRDSM), which contains longitudinal claims data comprising all filled prescriptions and clinical encounters for approximately 35 million members of 14 commercial health plans across the USA. Characteristics of beneficiaries in the HIRD are representative of commercially insured individuals in the USA. Linked administrative data on prescription fills, hospital discharges, and outpatient visits become available after a short lag period of approximately 6 months, during which claims are adjudicated by the payor and data are prepared for monitoring purposes.

We continuously monitored the safety and effectiveness of prasugrel versus clopidogrel by repeatedly analyzing data, as described below, at pre-defined intervals as they accumulated in the database. We selected clopidogrel as the comparator because it represents the most clinically relevant comparison and because the TRITON-TIMI-38 study results provide a basis against which the prospective monitoring results can be compared. To account for the low but increasing use of prasugrel in the period immediately after its market entry, we defined the first monitoring period as the first 6 months during which prescriptions for prasugrel appeared in the HIRDSM (September 2009 to February 2010) and we defined subsequent periods on a bimonthly basis through the end of August 2011.

The Brigham and Women’s Hospital Institutional Review Board approved this study.

2.2 Study Population

In each monitoring period, we identified all patients 18 years of age or older who filled a prescription for prasugrel or clopidogrel within 14 days following discharge from a hospitalization for acute MI or unstable angina. In our new user design [10], we excluded patients with a dispensation of either study drug in the 180 days prior to the observed exposure [11] and we excluded patients who were enrolled in the database for <180 days prior to the observed exposure to ensure sufficient data availability to measure baseline patient characteristics. We did not impose any other restrictions so that our cohort reflected how prasugrel was used in practice.

2.3 Outcomes and Follow-Up

We assessed the occurrence of five outcomes during follow-up: MI hospitalization, hospitalization for ischemic stroke, hospitalization for hemorrhagic stroke, hospitalization for severe upper gastrointestinal (GI) bleed, and death resulting from any cause. We defined all endpoints with claims-based algorithms. A validation study comparing these algorithms in the HIRD with medical chart reviews found a positive predictive value (PPV) of 88 % for MI (inpatient International Classification of Diseases, 9th Revision, Clinical Modification [ICD-9-CM] code 410.xx, excluding 410.x2), 96 % for ischemic stroke (inpatient ICD-9-CM code 433.x1, 434.x1, 436.xx), and 88 % for severe upper GI bleeds (any of inpatient ICD-9-CM diagnosis codes 531.0x, 531.2x, 531.4x, 531.6x, 532.0x, 532.2x, 532.4x, 532.6x, 533.0x, 533.2x, 533.4x, 533.6x, 534.0x, 534.2x, 534.4x, 534.6x, 578.0x; inpatient ICD-9-CM procedure code 44.43; inpatient Current Procedural Terminology [CPT] code 43255) [12]. We used a validated algorithm for hemorrhagic stroke (inpatient ICD-9-CM code 430.xx or 431.xx) with a PPV of 86 % in a different database [13].

For the analysis of each outcome, we followed patients beginning on the day after initiation of the index drug until the first of (i) occurrence of that outcome, (ii) death, (iii) health plan disenrollment, (iv) initiation of the other study drug, or (v) discontinuation of the index treatment. We defined date of discontinuation as the date of the last prescription dispensation plus the number of days supply of that prescription plus a 14-day gap if no subsequent prescription occurred during that period.

2.4 Patient Characteristics

We ascertained confounders over the 6 months preceding prasugrel or clopidogrel initiation, including the day of initiation, according to demographic data (age and sex), risk factors for ischemic and bleeding events, and health service use variables. We used ICD-9-CM diagnostic and procedure codes and CPT codes to define clinical conditions and potential risk factors related to the index hospitalization, such as whether patients underwent PCI or coronary artery bypass grafting (CABG). Other risk factors included history of diabetes mellitus, angina, GI bleed, hemorrhagic stroke, ischemic stroke, transient ischemic attack, peripheral vascular disease, congestive heart failure, hypertension, and use of various drugs and drug classes that may themselves be associated with the outcomes of interest or used to treat conditions associated with the outcomes. We also measured health service utilization variables, which can serve as proxies for general health status [14], including number of unique drugs used and number of physician visits in the baseline period. In addition, we computed a comorbidity score that combines conditions from the Charlson and Elixhauser indices and that has been shown to perform better than both of these individual scores [15].

2.5 Statistical Analysis

We conducted analyses sequentially as data from each new monitoring period became available throughout the 2-year monitoring timeframe. We used propensity score matching to account for differences in observed characteristics between prasugrel and clopidogrel initiators [16]. A patient’s propensity score is his or her probability of receiving a given treatment versus an alternative conditional on baseline covariates. On average, patients with the same propensity scores will have similar distributions of baseline covariates. We used logistic regression to estimate each patient’s probability of receiving prasugrel versus clopidogrel. We fit new propensity score models using baseline data from all available initiators each time data from a new period became available. We matched prasugrel initiators in each new monitoring period to clopidogrel initiators in the same period and, once matched, followed patients prospectively [5, 8].

In primary analyses we used 1:1 propensity score matching based on a nearest neighbor-matching algorithm with a caliper of 0.05 on the propensity score scale, which ranges from 0 to 1. We included in the propensity score models all baseline variables described above plus calendar time-by-covariate interaction terms between each of these variables and an indicator for the monitoring period in which the patient initiated treatment. We used a high-dimensional propensity score (hd-PS) algorithm [17] that is particularly suited for semi-automated adjustment in scalable drug safety monitoring systems [18, 19] to empirically identify additional baseline variables to include in the propensity score models. The hd-PS algorithm identifies and prioritizes covariates based on parameters that govern their potential to cause confounding, namely their prevalence and associations with exposure and outcome. In this prospective application, we considered only the covariate prevalence and association with exposure. We selected the most prevalent drug, procedure, and diagnosis codes that exhibited the greatest degree of imbalance between treatment groups [20].

After performing propensity score matching at each data update, we estimated incidence rate differences (and 95 % CIs) and incidence rate ratios (and 95 % CIs) comparing prasugrel with clopidogrel on each of the outcomes of interest among all matched patients who had matriculated into the study at that point. Each sequential estimate therefore corresponded to the estimate that would have been obtained if we had waited until that point and performed a retrospective analysis since it was based on all eligible events and person-time that had accrued through the end of that monitoring period. We used SAS® version 9.3 (Cary, NC, USA) for statistical analyses.

2.6 Sequential Alerting

We applied alerting algorithms to determine whether and when sequential estimates suggested an elevated or reduced rate of each outcome among prasugrel initiators compared with matched clopidogrel initiators. Alerting algorithms are statistical monitoring approaches that can be used for sequential analyses in medical product safety and effectiveness monitoring. We used the results of a prior simulation study to select an alerting algorithm for each outcome of interest [9, 21]. The simulation study showed that the relative performance of algorithms, with respect to the accuracy and timeliness of alerting, varies substantially depending on the characteristics of specific scenarios in prospective monitoring. To select algorithms for application to the current assessment, we estimated the expected number of events based on the number of prasugrel initiators in the first monitoring period (assuming that utilization of the drug would increase linearly over time) and on the incidence of each outcome among patients who initiated clopidogrel prior to the market introduction of prasugrel. We then selected simulated scenarios resembling each outcome of this study and chose the top-performing algorithm for each from a total of 93 algorithms tested to optimize sensitivity, specificity, and time to alerting [21].

For MI, the selected algorithm was “generate an alert upon the occurrence of five consecutive period-specific estimates that exceed a given threshold of acceptable risk.” This algorithm was found to have an event-based sensitivity and specificity of 0.69 and 0.95, respectively, among similar scenarios in the prior simulation study [9]. Event-based sensitivity is defined as the proportion of simulated exposed events in scenarios in which a safety issue of interest exists that occurred after the algorithm generated an alert. Event-based specificity is the proportion of simulated exposed events in scenarios in which no safety issue exists that occurred before or in the absence of an alert [9, 21]. We used a null threshold, indicating that we would not tolerate any difference between the two drugs regarding the selected outcomes. Period-specific estimates are based on data from only initiators in that period. For the less frequent severe upper GI bleeds and death from any cause, the best-performing algorithm was “generate an alert when the nominal p value is smaller than 0.01” (event-based sensitivity, 0.51; event-based specificity, 0.95). For both ischemic and hemorrhagic stroke, the selected algorithm was “generate an alert when the nominal period-specific p value is smaller than 0.1” (event-based sensitivity, 0.33; event-based specificity, 0.96).

2.7 Secondary Analysis

At the end of the 2-year monitoring timeframe, we conducted a secondary analysis using variable ratio propensity score matching to maximize the large number of clopidogrel initiators in the comparison group relative to the number of prasugrel users. We allowed prasugrel patients to match to as many clopidogrel patients as possible within a caliper of 0.05 on the propensity score scale; that is, each matched set was allowed to have any number of clopidogrel initiators for each prasugrel initiator. We conditioned rate difference estimates by the number of patients in the matching set using Mantel–Haenszel weights [22].

3 Results

During the first 2 years of prasugrel’s market availability, we identified 1,282 eligible prasugrel initiators and 8,263 eligible clopidogrel initiators (Fig. 1). The monthly number of eligible new users of prasugrel increased over the 2-year monitoring timeframe and the monthly number of clopidogrel initiators decreased (Fig. 2). Prasugrel initiators were younger (mean age for prasugrel vs. clopidogrel: 55.8 vs. 62.4 years), more likely to have undergone PCI during the index hospitalization, and less likely to have undergone CABG (Table 1). Across the ten monitoring periods, we matched 1,255 (98 %) prasugrel initiators to 1,255 clopidogrel initiators. Prasugrel and clopidogrel were well balanced on baseline characteristics in the matched cohorts (Table 1).

Fig. 1
figure 1

Patient flow chart for the two study cohorts. ACS acute coronary syndromes, MI myocardial infarction

Full size image
Fig. 2
figure 2

Eligible new users of prasugrel and clopidogrel in each calendar month and monitoring period, September 2009–August 2011

Full size image
Table 1 Baseline characteristics for all eligible patients and for matched cohorts accumulated across all ten monitoring periods, September 2009 to August 2011
Full size table

We observed 45 MI events among 504 person-years among prasugrel-treated patients and 63 events among 564 person-years contributed by matched clopidogrel users for an overall rate difference of –22.4 events (95 % CI −60.4–15.5) per 1,000 person-years and a corresponding rate ratio of 0.80 (95 % CI 0.54–1.17), during our 2-year monitoring timeframe. The sequential MI rate differences for each of the ten monitoring periods are presented in Fig. 3. The selected algorithms did not generate alerts for any of the outcomes at any point during the pre-specified 2-year monitoring timeframe.

Fig. 3
figure 3

Near-real-time monitoring results for MI among initiators of prasugrel versus clopidogrel. In each of ten periods, the rate difference is plotted based on data available at that time. Rate differences reflect the differences in number of events among 1,000 person-years treated with prasugrel and 1,000 person-years treated with clopidogrel. Positive values indicate a higher event rate among patients treated with prasugrel and negative values indicate a higher event rate among patients treated with clopidogrel. Numbers of events and person-time in the table are cumulative over time. MI myocardial infarction

Full size image

We observed only two hemorrhagic stroke events, both occurring among clopidogrel-treated patients, and nine severe upper GI bleed events—five occurred among prasugrel-treated patients and four among patients in the clopidogrel group. The resulting rate difference for the composite of these events was −0.5 events (95 % CI −12.4–11.3) per 1,000 person-years and the corresponding rate ratio was 0.95 (95 % CI 0.29–3.10; Fig. 4). Three patients in the prasugrel group and five patients in the 1:1 matched clopidogrel group died during follow-up (Fig. 5) for a rate difference of −2.7 deaths (95 % CI −12.7–7.3) per 1,000 person-years and a rate ratio of 0.68 (95 % CI 0.16–2.86).

Fig. 4
figure 4

Near-real-time monitoring results for composite bleeding event (severe upper gastrointestinal bleed and hemorrhagic stroke) among initiators of prasugrel versus clopidogrel. In each of ten periods, the rate difference is plotted based on data available at that time. Rate differences reflect the differences in number of events among 1,000 person-years treated with prasugrel and 1,000 person-years treated with clopidogrel. Positive values indicate a higher event rate among patients treated with prasugrel and negative values indicate a higher event rate among patients treated with clopidogrel. Numbers of events and person-time in the table are cumulative over time

Full size image
Fig. 5
figure 5

Near-real-time monitoring results for all-cause mortality among initiators of prasugrel versus clopidogrel. In each of ten periods, the rate difference is plotted based on data available at that time. Rate differences reflect the differences in number of events among 1,000 person-years treated with prasugrel and 1,000 person-years treated with clopidogrel. Positive values indicate a higher event rate among patients treated with prasugrel and negative values indicate a higher event rate among patients treated with clopidogrel. Numbers of events and person-time in the table are cumulative over time

Full size image

The secondary analysis that increased the number of matched clopidogrel initiators found similar results (Fig. 6). This variable ratio matching process retained 98 % of both prasugrel and clopidogrel initiators and achieved good balance on measured baseline covariates (Table 2).

Fig. 6
figure 6

Forest plot of rate differences (and 95 % confidence intervals) from secondary analysis performed at the end of monitoring (24 months after prasugrel prescriptions first appeared in the database [including 1,255 prasugrel initiators and 8,127 propensity score matched clopidogrel initiators])

Full size image
Table 2 Baseline characteristics for prasugrel initiators and a randomly selected matched clopidogrel initiator in the variable-ratio matched secondary analysis
Full size table

4 Discussion

Well controlled randomized trials are necessary to establish the efficacy of new drugs, but cannot answer all questions about the safety and effectiveness of the drugs once they enter the market and are used by broad populations with varying disease statuses [23]. Sequential observational studies conducted in routinely collected electronic healthcare data may be useful to rapidly identify major safety concerns that manifest in the post-marketing setting and can provide timely estimates of the comparative safety and effectiveness of new drugs as compared with existing treatments [5].

Prasugrel was launched in the USA in 2009. During the first 2 years of its availability, we prospectively monitored its safety and effectiveness relative to that of clopidogrel in a large electronic healthcare data environment using an approach that has been shown to be both sensitive and specific for identifying drug safety signals in historical examples [7, 21]. Our validated approach did not generate any alerts that would indicate significant differences between the drugs with respect to ischemic events, bleeding events, and all-cause mortality. As expected from the TRITON-TIMI-38 study, we observed a modest reduction in rates of hospitalization for MI and ischemic stroke among patients treated with prasugrel versus clopidogrel, but confidence intervals around these estimates were wide. However, we did not find evidence that prasugrel is associated with an increased risk of major bleeding events, providing reassurance about its safety as currently used in clinical practice among a commercially insured population. To our knowledge, this is the first study to provide comparative effectiveness and safety evidence of prasugrel versus clopidogrel among a large population of patients in routine care.

While our MI findings are consistent with results of the TRITON-TIMI-38 study, the discrepant bleeding findings warrant consideration. TRITON-TIMI-38 included patients with moderate-to-high-risk ACS, whereas we studied potentially lower-risk patients discharged from any MI or ACS hospitalization lasting between 3 and 180 days. Indeed, patients in our matched cohort were younger than the trial patients (mean age 56 vs. 61) and major bleeding events occurred less frequently in our cohort. Our bleeding results are similar to those of the TRILOGY ACS (Targeted Platelet Inhibition to Clarify the Optimal Strategy to Medically Manage Acute Coronary Syndromes) trial, which compared prasugrel with clopidogrel among aspirin-treated patients with ACS who did not undergo revascularization [24]. Patients in the TRILOGY ACS trial had much lower bleeding rates than patients in TRITON-TIMI-38. Additionally, our outcome definitions differed from TIMI bleeding events because not all types of bleeds can be reliably identified in claims data. Also, because we were not able to ascertain in-hospital drug use, we included patients who filled an outpatient prescription for a study drug within 2 weeks of discharge; thus, patients must have survived in the outpatient setting long enough to fill their prescription, introducing a short period of immortal time. A non-trivial proportion of patients experienced events in the first 2 weeks of follow-up in the TRITON-TIMI-38 study. Residual confounding due to channeling of patients at higher risk of bleeding to clopidogrel or to potential differences in other unmeasured patient characteristics, such as over-the-counter aspirin and NSAID use, body weight, body mass index, and smoking status, cannot be ruled out. However, we analyzed only clopidogrel patients who were highly similar to prasugrel patients on a large number of variables observed in their claims profiles. Another important limitation of our study is that we observed few events for several outcomes among hd-PS-matched patients (11 total bleeding events and eight deaths among the primary hd-PS-matched subgroup). The variable-ratio matched secondary analysis allowed us to retain many more events among clopidogrel initiators (77 bleeding events and 109 deaths) and resulted in similar effect estimates.

To the extent possible, we designed our prospective safety and effective monitoring approach like a parallel-group randomized controlled trial, but without randomization [25]. In this application, we required that patients be new users of prasugrel and clopidogrel. We ascertained potential confounders prior to treatment initiation and we adjusted for a large number of measured potential confounders or proxies for unmeasured confounders using propensity score matching [17]. As in randomized trials, the propensity score-balanced cohort approach allowed us to study multiple endpoints and, as in many pragmatic trials, we assessed outcomes using hospital discharge data with known operating characteristics [26–28]. Whereas data safety monitoring boards of randomized trials often establish stopping rules to aid decisions about whether to terminate trials because of observed risks that disrupt clinical equipoise, we employed analogous alerting algorithms that have been evaluated in the setting of observational monitoring [8, 9, 21].

The post-marketing drug safety and comparative effectiveness enterprise is undergoing a profound transformation [29]. Rather than relying on spontaneous adverse event reports and ad hoc pharmacoepidemiological studies, prospective sequential monitoring, which relies on large electronic healthcare databases and semi-automated analytics, can inform patients, prescribers, payors, and regulators about relative harms and benefits of new drugs in near real time [30]. Through the Sentinel Initiative and the Mini-Sentinel pilot project, the FDA is currently developing an active surveillance system that will involve a network of distributed databases, including data such as those used in our study, that will comprise information on more than 100 million Americans [31]. Approximately 50 million lives will be available in the database at a given moment [32], which is about four times larger than the population that we studied and which will more quickly lead to high precision around effect estimates. Our application demonstrates on a slightly smaller scale how these data, methods, and analytics can be combined to perform near real-time monitoring that yields timely safety and effectiveness information in the early marketing period.

5 Conclusion

We demonstrated the utility of a prospective, active monitoring system for assessing the comparative effectiveness and safety of newly marketed drugs. The approach closely resembles an ordinary pharmacoepidemiologic study and had been previously validated in only retrospective assessments. While the number of events was small, the sequential alerting approach suggests that, as currently used in practice, prasugrel compared with clopidogrel does not appear to be associated with increased risk of GI and intracranial bleeding. However, continued monitoring for a longer time and in other higher-risk populations is warranted to identify safety issues that may emerge as prescribing continues to evolve.