1 Introduction

Spontaneous reporting of adverse drug reactions (ADRs) remains central to the success of post-marketing surveillance programmes in detecting ADR signals. Monitoring of ADRs in the spontaneous reporting systems has led to changes in safety information on product labels and also, in some cases, the eventual withdrawal of medicines [1, 2]. Under-reporting of adverse events may contribute to a delay in the detection of ADRs [3]. During the period 1975–1999, 81 products in the Physicians’ Desk Reference were issued a black box warning because of newly identified serious adverse events; however, only half of the ADRs were identified within 7 years of marketing of the medicines [4]. The delay in identifying ADRs may place a significant burden on healthcare systems, including excess mortality and morbidity in patients receiving these medicines [5–7]. For example, rofecoxib, a selective cyclo-oxygenase (COX)-2 inhibitor was withdrawn 5 years after marketing (June 1999–September 2004) because of an increased risk of cardiovascular adverse events [8]. It is estimated that there were 88,000–140,000 excess cases of heart disease in the USA due to rofecoxib during that time [9].

Another medicine that had a delayed safety warning issued was rosiglitazone, a medicine used to treat type 2 diabetes mellitus [10]. A black box warning that indicates the medicine is associated with an increased risk of cardiovascular adverse events was issued 7 years after marketing. Rosiglitazone was withdrawn from European countries 10 years after marketing [11]; however, it is still available in Australia and in some other countries with a black box warning in the product information (PI) [12, 13].

Several quantitative methods to detect ADR signals in spontaneous reporting databases have been developed and adopted by pharmaco-surveillance centres, including the proportional reporting ratio (PRR), reporting odds ratio (ROR) and Bayesian techniques [14–16]. Sequence symmetry analysis (SSA) is another quantitative method that has been used to detect ADR signals in administrative claims databases [17–26]. A previous study has shown that SSA has moderate sensitivity (61 %) and high specificity (93 %) to detect ADR signals [27]. Due to its simple algorithm and minimum dataset requirement, SSA is a potential rapid assessment tool that could be used routinely by health regulators with access to computerised claims data, to assist with safety signal detection and potentially confirm suspected signals that arise from spontaneous reports or pre-marketing clinical trials.

In this paper, we present two case studies concerning the evolution of safety evidence and opportunities to utilise quantitative signalling methods for rofecoxib-induced myocardial infarction and rosiglitazone-induced heart failure. In these case studies, we apply three ADR signal detection techniques [PRR, ROR, Bayesian confidence propagation neural network (BCPNN)] to Australia’s national spontaneous reporting database, the ‘Database of Adverse Event Notifications’ (DAEN). We also apply the SSA technique to the Australian Department of Veterans’ Affairs (DVA) administrative claims database. Safety events associated with rofecoxib and rosiglitazone since their first introduction into the Australia market until withdrawal of rofecoxib and the black box warning being issued for rosiglitazone are illustrated. The aim of this study was to determine whether quantitative methods using spontaneous reporting data and administrative claims data could identify signals for rofecoxib-induced myocardial infarction and rosiglitazone-induced heart failure. We also determined the time at which each method detected a significant safety signal after the introduction of the each product to the Australian market.

2 Methods

2.1 Spontaneous Reporting Data Source

The Australian DAEN spontaneous reporting database was used to identify when ADR signals for rofecoxib-induced myocardial infarction and rosiglitazone-induced heart failure were detected. DAEN is available for public access from the Australian Therapeutic Goods Administration (TGA) website [28]. The database contains spontaneous reports of suspected medicine- and vaccine-related adverse events. The database contains more than 250,000 reports provided to the TGA since 1971. The reports are sourced from medicine manufacturers, health professionals and consumers.

All adverse events are coded using MedDRA® (Medical Dictionary for Regulatory Activities). MedDRA® is the international medical terminology developed under the auspices of the International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) [29, 30]. The preferred term under MedDRA® is used to describe the coded adverse events. Information about adverse events associated with a medicine can be searched and identified directly from the website. Information relating to the total number of reports for a particular adverse event is not available on the website but was requested and obtained from the TGA. For the rofecoxib-induced myocardial infarction analysis, we included preferred terms containing ‘myocardial infarction’ and ‘acute myocardial infarction’. For the rosiglitazone-induced heart failure analysis, we included preferred terms containing ‘cardiac failure’, ‘cardiac failure acute’, ‘cardiac failure chronic’, ‘cardiac failure congestive’, ‘cardiac failure high output’, ‘ventricular failure’, ‘right ventricular failure’, ‘left ventricular failure’, ‘right ventricular failure acute’, ‘right ventricular chronic’, ‘left ventricular acute’ and ‘left ventricular chronic’. We grouped all preferred terms under one adverse event, prior to analysis, i.e. heart failure or myocardial infarction. Grouping in this way may not be standard practice for routine signal detection in spontaneous reporting systems; however, our aim was to compare the timeliness of SSA and spontaneous reporting systems, hence outcome events need to be consistent between the two methods. SSA classifies outcomes at the grouped level MedDRA® for the adverse events tested in this study.

2.1.1 Signalling Method

Three pharmaco-surveillance methods, PRR, ROR and Bayesian confidence propagation neural network (BCPNN), were applied to the DAEN data to identify signals for rofecoxib-induced myocardial infarction and rosiglitazone-induced heart failure. These methods have been described in detail previously [14–16, 31]. These methods use disproportionality analyses based on 2 × 2 tables (Table 1) [32]. The information used to support the calculation and the signalling criteria for all three methods is shown in Table 2. The signalling criteria used are those employed by health regulators in the UK, European countries and Australia [14–16]. We used counts of reports as the unit of analysis to calculate the PRR, ROR and the BCPNN because all of the information provided was in counts of reports by the TGA. Cumulative yearly analyses were undertaken starting from the year the medicine was available in Australia.

Table 1 2 × 2 table of the disproportionality analysis of the proportional reporting ratio, reporting odds ratio and Bayesian confidence propagation neural network
Full size table
Table 2 Pharmaco-surveillance methods used by regulatory agencies, information used to generate signals and the threshold for adverse drug reaction signals
Full size table

2.2 Administrative Claims Data Source

Administrative claims data from the Australian Government DVA was used for the SSA. The DVA database contains information on all medicines and healthcare utilisation by veterans, for which the DVA pays a subsidy for a treatment population of 250,000 veterans [33]. This includes data for all medicines dispensed on the Pharmaceutical Benefit Scheme (PBS) and Repatriation Pharmaceutical Benefits Scheme (RPBS), the national schemes for subsidy of medicines in Australia, and hospitalisations. Medicines are coded according to the World Health Organization (WHO) anatomical and therapeutic chemical (ATC) classification [34] and the Schedule of Pharmaceutical Benefits item codes [35]. Hospitalisations are coded according to the WHO International Classification of Disease, 10th revision (ICD-10), Australian modification [36].

2.2.1 Signalling Method

SSA was undertaken to identify ADR signals for rofecoxib-induced myocardial infarction and rosiglitazone-induced heart failure within the DVA database. Medicine dispensing and hospitalisation records for each year starting from the time medicines were first available in the database were analysed cumulatively on a 6-monthly basis (rofecoxib 2001–2004, rosiglitazone 2001–2007). SSA has been described in detail previously [25]. Briefly, SSA determines asymmetry in the sequence of dispensing between two medicines or dispensing of a medicine and hospitalisation event, within a given time window. The sequence consists of one medicine that is suspected to cause the adverse event, also called the index medicine, and a second medicine or hospitalisation event, which serves as an indicator of the adverse event. For myocardial infarction, hospitalisation for myocardial infarction (ICD code: ‘I21’–‘I22’) was used as an indicator. Incident dispensing of frusemide (ATC: C03CA01) and incident heart failure hospitalisation diagnosis (ICD codes: ‘I500’, ‘I501’, ‘I509’) were used as indicators for heart failure. The statistic of interest in SSA is the sequence ratio, which is calculated by dividing the number of people with the indicator for the adverse event after the index medicine by the number of people with the indicator for the adverse event before the index medicine. Because of the within-person design, the sequence ratio is robust to confounders that are stable over time; however, the SSA may be affected by prescribing or event trends over time [17]. For example, if the trend in use of the medicine used to treat the adverse event is increasing, we could expect more people to start this medicine after the index medicine simply by chance. As a result, the sequence ratio will be over-estimated. To control for potential time trends, a null sequence ratio (NSR) is used [25]. The NSR serves as a reference to compare to the crude sequence ratio assuming the sequence of index medicines and the indicators are unrelated with each other [25]. The adjusted sequence ratio (ASR) is then obtained by dividing the crude sequence ratio by the null effect ratio and 99 % confidence intervals are calculated. A signal was considered to be present when the lower limit of the 99 % confidence interval was 1 or more.

2.3 Sources of Data for Objective Information

We also searched for and examined published warning reports of rofecoxib-associated myocardial infarction and rosiglitazone-associated heart failure. We searched the Australian TGA website [37], the US FDA website [38] and the Australian Adverse Drug Reactions Advisory Committee (ADRAC) Bulletin [39]. These sources were chosen due to the likelihood that any adverse event alert will be first notified to and distributed by these agencies. Statistics on subsidised rofecoxib and rosiglitazone dispensings each year from the year the medicine was first subsidised in Australia were extracted from Medicare Australia Statistics website [40]. For subsidised and non-subsidised prescription statistics, data were obtained from the Australian Statistics on Medicines [41]. Both sources of prescription dispensing data represent medicines used by Australians in the community.

3 Rofecoxib-Induced Myocardial Infarction and Rosiglitazone-Induced Heart Failure Case Studies

3.1 The Rofecoxib Story

An illustration of the rofecoxib safety signal detection timeline is shown in Fig. 1. Rofecoxib first became available in Australia in June 1999 for treatment of signs and symptoms of osteoarthritis [42]. The rofecoxib–osteoarthritis pre-marketing trials demonstrated that rofecoxib had similar efficacy to non-selective non-steroidal anti-inflammatory medicines but a significantly reduced gastrointestinal toxicity profile [43, 44]. Rofecoxib was studied in 15 pre-marketing trials in a total of more than 5,000 patients; seven placebo-controlled trials and eight active comparator trials [45]. Eleven of 15 trials were conducted for less than 6 months. Of seven trials that were placebo-controlled trials, five were restricted to 6-week studies [45].

Fig. 1
figure 1

Rofecoxib-induced myocardial infarction timeline in Australia. ADRAC Adverse Drug Reactions Advisory Committee, APPROVe Adenomatous Polyp Prevention on Vioxx, BCPNN Bayesian propagation neural network, DVA Department of Veterans’ Affairs, MI myocardial infarction, PBS Pharmaceutical Benefit Scheme, PRR proportional reporting ratio, ROR reporting odds ratio, SSA sequence symmetry analysis

Full size image

In June 2000, the potential that rofecoxib had cardiac pro-thrombic activities was identified in the VIGOR (Vioxx Gastrointestinal Outcomes Research) trial [46]. The trial was conducted in 8,076 rheumatoid arthritis patients with a mean follow-up of 8 months [46]. While the trial demonstrated that rofecoxib was as effective as naproxen (an anti-inflammatory medicine) for treatment of pain in rheumatoid arthritis and had fewer gastrointestinal adverse events, higher cardiovascular event rates (particularly myocardial infarction) were shown in patients treated with rofecoxib [46]. The findings were considered inconclusive, as it was unclear whether the high incidence of cardiac thromboembolic events in the rofecoxib-treated group was due to a pro-thrombic effect of rofecoxib or whether the lower incidence of cardiac events with naproxen (the control group) was due to a cardio-protective effect of naproxen. Naproxen, a COX-1 inhibitor, has an anti-platelet effect similar to aspirin which is used in the prevention of cardiovascular disease [47]. By the end of 2000, when rofecoxib had been on the private market for 6 months, there were no post-marketing reports of myocardial infarction associated with rofecoxib use in the DAEN spontaneous reporting database.

Safety updates of the VIGOR trial were subsequently submitted to the FDA and reviewed by the Arthritis Advisory Committee in February 2001 [48, 49]. It was considered that the VIGOR safety findings lacked generalisation because the trial used twice the highest recommended dose of rofecoxib (50 mg). In addition, the VIGOR trial compared rofecoxib to the active comparator (naproxen) and excluded patients taking aspirin (patients with high risk of cardiovascular disease). Other evidence of potential rofecoxib-associated cardiovascular events was reviewed, including supplemental application data for rofecoxib indicated for a rheumatoid arthritis trial that was submitted to the FDA [48]. The trial compared rofecoxib with naproxen. The review of the data suggested a higher incidence of myocardial infarction in rofecoxib- than in naproxen-treated patients, consistent with the VIGOR study [50]. Other preliminary data from two ongoing Alzheimer’s disease placebo-controlled trials were analysed and suggested no increased risk of thrombotic cardiovascular adverse events [49]. A pooled analysis of 23 pre-marketing and early post-marketing trials of rofecoxib were conducted and results did not show an excess of cardiovascular thrombotic events in rofecoxib compared with placebo or non-selective anti-inflammatory medicines [51]. However, these trials were not designed to identify thrombotic cardiovascular events. Unresolved issues remained, including whether the thrombotic cardiovascular adverse event rate with rofecoxib was similar to placebo and whether the higher incidence of events seen with rofecoxib was due to a cardio-protective effect of naproxen.

While the potential cardio-thromboembolic events associated with rofecoxib in clinical trials were still debatable, rofecoxib was approved by the Australian Government for subsidised listing in 2001 and rapid uptake of this medicine followed (Fig. 1). In May 2001, the first spontaneous report of myocardial infarction associated with rofecoxib was received by the TGA in Australia [28]. By the end of 2001 (18 months after marketing), there were a total of three spontaneous reports of myocardial infarction associated with rofecoxib in Australia. At this time, there were insufficient reports for PRR, ROR and BCPNN to detect a signal in the DAEN data. In contrast, by the end of 2001, SSA detected a significant signal for myocardial infarction excess risk in new users of rofecoxib within 1 year of PBS listing (ASR 1.89, 99 % CI 1.35–2.66) (Table 3).

Table 3 Rofecoxib-induced myocardial infarction (statistically significant associations are italicised)
Full size table

In 2002, a follow-up analysis on additional data from two previous rofecoxib Alzheimer’s disease placebo-controlled trials and one additional placebo-controlled trial in approximately 3,000 patients conducted in the USA, did not show excess risk of cardiac-thromboembolic events, but showed a non-significant increased trend of cardiovascular thrombotic death in rofecoxib treatment groups [52]. A warning letter was sent to all health professionals in the USA advising caution in prescribing rofecoxib for patients with ischaemic heart disease [53]. The warning messages were not distributed to health professionals in Australia. At this time, dispensing of rofecoxib in Australia continued to increase (Fig. 1). In the same year, three additional post-marketing reports of myocardial infarction associated with rofecoxib were received by the TGA, making a total of six reports. By the end of 2002, PRR, ROR and BCPNN detected the myocardial infarction safety signal (Fig. 1; Table 3). By the end of 2002, all four signalling methods had demonstrated a myocardial infarction signal associated with rofecoxib from two independent data sources (a spontaneous reporting database and a claims database).

In October 2003, the Australian ADRAC issued its first alert concerning cardiovascular risk associated with rofecoxib [54]. By the end of that year, there were a total of 650 rofecoxib reports in DAEN and, of these, 15 were associated with myocardial infarction. Despite the warning, rofecoxib dispensing continued to increase during 2003 until late 2004 (Fig. 1). In September 2004, rofecoxib was withdrawn from the worldwide market after preliminary results from the APPROVe (Adenomatous Polyp Prevention on Vioxx) trial (N = 3,059 patient-years) showed that rofecoxib-treated patients had nearly twice the risk of thromboembolic events (relative risk 1.92, 95 % CI 1.19–3.11), including myocardial infarction, compared with those receiving placebo [55].

3.2 The Rosiglitazone Story

An illustration of the rosiglitazone heart failure signal detection timeline is shown in Fig. 2. Rosiglitazone was approved by the FDA in May 1999 as an adjunct to diet and exercise to improve blood glucose control in patients with type 2 diabetes [56]. At that time, the majority of rosiglitazone pre-marketing trials had less than 6 months’ follow-up and were underpowered to detect any significant difference for heart failure events between groups [57, 58]. Heart failure adverse events were infrequently (<1 %) reported but oedema was two to three times more frequent in rosiglitazone treated groups than in the comparators [57, 58]. At this time, the association of heart failure with rosiglitazone was unclear. Use of rosiglitazone in patients with New York Heart Association (NYHA) Class III and IV heart failure (moderate-to-severe heart failure) was included under the ‘Precautions’ section in the PI because these patients were not investigated in pre-marketing clinical trials [59].

Fig. 2
figure 2

Rosiglitazone-induced heart failure timeline in Australia. BCPNN Bayesian confidence propagation neural network, DVA Department of Veterans Affairs, HF heart failure, NYHA New York Heart Association, PBS Pharmaceutical Benefit Scheme, PI product information, PRR proportional reporting ratio, ROR reporting odds ratio, SSA sequence symmetry analysis

Full size image

Rosiglitazone became available in Australia in July 2000, but was not subsidised on the PBS until October 2003 [60]. Before PBS listing, rosiglitazone was only available on private prescription [61]. The first heart failure case report associated with rosiglitazone was received by the TGA in November 2000 [28]. By the end of 2000 (6 months after marketing), ROR and BCPNN detected the first rosiglitazone-associated heart failure signal based on one report (Table 4). SSA could not identify a heart failure signal associated with rosiglitazone in the same year because the medicine was not yet subsidised. ROR and BCPNN continued to generate signals in 2001 and 2002, when it was still only available in the small private market (Table 4; Fig. 2).

Table 4 Rosiglitazone-induced heart failure (statistically significant associations are italicised)
Full size table

In April 2002, changes were made to the warnings, precautions and adverse events section of the US-approved PI for rosiglitazone to highlight the risk of cardiac adverse events [62]. These changes were made based on the results of three clinical trials cited in the FDA summary safety alert document [63]. The trials were conducted in the USA and found an increased incidence of cardiac failure and other cardiovascular events in patients treated with rosiglitazone compared with placebo [63]. At this time, rosiglitazone use in Australia was contraindicated in patients with NYHA Class III and IV heart failure [64]. No additional rosiglitazone-associated heart failure cases had been reported to the TGA by the end of 2002, despite increasing dispensing of rosiglitazone (Fig. 2).

In April 2003, ADRAC issued a summary of adverse event spontaneous reports relating to rosiglitazone [65]. ADRAC highlighted the cardiac event reports associated with rosiglitazone, including seven reports of oedema. The events were evaluated as possibly related to age and co-morbidities including diabetes, hypertension and ischaemic heart disease. By the end of 2003, 12 reports of rosiglitazone-associated heart failure were held by the TGA, and PRR detected the first signal (Table 4). SSA detected an increase in frusemide dispensing associated with rosiglitazone use but the signal was not statistically significant (Table 4).

In October 2003, rosiglitazone was approved for PBS listing [66, 67] and more rosiglitazone dispensing data became available in the DVA claims database. In 2004, rosiglitazone dispensing data in the DVA claims database represents 9 % of the total subsidised rosiglitazone use and 7.8 % of the total subsidised and non-subsidised rosiglitazone use in Australia (Fig. 2). Two years after PBS listing, SSA detected a heart failure safety signal using frusemide dispensing as an indicator of a heart failure adverse event (ASR 1.50, 99 % CI 1.02–2.22) (Table 4). When using heart failure hospitalisation as an indicator for heart failure, SSA detected a significant signal by the end of 2006 (ASR 2.40, 99 % CI 1.24–4.65) (Table 4). Within 3.5 years of marketing and 2 years of national subsidy, all quantitative signalling methods had detected a rosiglitazone–heart failure signal.

It wasn’t until August 2007 that the FDA issued a black box warning of the potential for rosiglitazone to cause or exacerbate heart failure based on data from four clinical trials [68]. The clinical trials were conducted over periods of between 26 and 52 weeks and showed an increased incidence of heart failure and worsening heart failure symptoms such as oedema and dyspnoea. In the same year, one study published a pooled result of three randomized clinical trials [69–71] that showed a similar risk of heart failure associated with rosiglitazone with the SSA result in 2006 (pooled result; odds ratio 2.1, 95 % CI 1.08–4.08, p = 0.03 vs. SSA result; ASR 2.40, 99 % CI 1.24–4.65) [72]. The Australian rosiglitazone PI was changed by extending the contraindication of the medicine to all grades of heart failure (Class I–IV, mild-to-severe heart failure) and in patients with a history of heart failure [73].

4 Discussion

These case studies highlight the importance of having multiple sources of information and methods to support post-marketing surveillance of new medicines. While there was uncertainty about rofecoxib-induced cardiovascular events [51, 74–77], monitoring using administrative claims data did detect a potential signal. This was subsequently supported by signals from spontaneous reports. The second case study reveals a similar story with the safety signal for rosiglitazone-associated heart failure, first detected with ROR and BCPNN. The signal of the event was subsequently confirmed by the SSA. If SSA was used, it may have provided an opportunity for earlier warning and management such as prescribing restrictions in the heart failure population, so that excess harm may have been prevented.

The case studies also demonstrate that there was a difference in time to issuing warnings to health professionals in Australia (in 2003) about cardiovascular adverse events associated with rofecoxib compared to the USA (in 2002). In 2002, the available evidence about potential cardiovascular adverse events for rofecoxib was not directly generalisable to the Australian population as the majority of the clinical trials for rofecoxib were conducted in the USA [78–80]. Thus, potential safety evidence may need to be evaluated using spontaneous reporting databases and health claims databases. Utilising multiple signalling methods in different data sources may help early verification of safety evidence in a specific population.

Both SSA and disproportionality analysis methods (PRR, ROR, BCPNN) detected adverse event signals 1–3 years after marketing or subsidisation, before results of clinical trials were published that showed an increased risk of serious and potentially life-threatening adverse events. Serious actions were imposed on rofecoxib (withdrawal) and rosiglitazone (black box warning/contraindication warning) in response to these results. By contrast, the trial results and subsequent warning or withdrawal were published 5–7 years after first marketing of these medicines. Quantitative signalling methods could be used to provide more timely evidence of medicines safety to strengthen the evidence of potential signals. Signals generated by these methods should not be interpreted as causal associations because patient’s clinical conditions are not considered. It is important to have a follow-up investigation that incorporates expert clinical reviewers whenever adverse event signals are generated. However, they may serve to provide early alerts or warnings to health regulators and subsequent appropriate interventions can be undertaken.

There are several advantages to using administrative claims data in adverse event signal detection to complement spontaneous report data. Administrative claims data have coverage for an entire population or subset of the population. Hence, data are available earlier and may be more complete. Because the administrative claims data collect complete records of dispensed subsidised medicines for the entire population, data are not subjected to reporting bias. In comparison, spontaneous reporting databases rely on the adverse event actually being reported. In cases where an adverse event is not recognised as being associated with a medicine, no report will be made. SSA could be an additional method to identify adverse event signals using population-based administrative claims records. However, SSA can only be applied to administrative claims data when there is an appropriate indicator of adverse events such as a treatment medicine for the adverse event or a hospitalisation diagnosis that would describe the event.

4.1 Limitations

The present case studies demonstrate the opportunities for quantitative ADR signalling methods to detect known ADRs that received considerable attention before the eventual withdrawal of rofecoxib and issuance of a black box warning for rosiglitazone. The widespread publicity of these medicines may have influenced the level of ADR reporting to the ADR spontaneous reporting system at that time. Future work will investigate how SSA and disproportionality analyses perform when investigating ADRs that received less attention. In this study, we used cumulative yearly analysis in spontaneous reporting database to ensure sufficient cases in the database. It would be possible to use quarterly or monthly analyses in larger datasets for spontaneous reporting data and health claims data which would enable a more rapid signal detection system.

5 Conclusion

SSA using administrative claims data and disproportionality analyses using spontaneous reporting data detected safety signals for rofecoxib-induced myocardial infarction and rosiglitazone-induced heart failure 1 or 2 years after the medicines were subsidised or marketed. As the majority of countries now have electronic dispensing records, there is a potential for SSA to be used in conjunction with quantitative methods from spontaneous reporting reports to assist health regulators in the decision-making process about safety issues for new products in the market.