FormalPara Key Points

Prior to this research, the Therapeutic Goods Administration (TGA) had limited experience of using administrative health data for signal detection.

Using heart failure (HF) as an example adverse event (AE), we applied prescription sequence symmetry analysis (PSSA) to Pharmaceutical Benefits Scheme (PBS) data and found 26 potential new HF signals from 654 tested medicines.

PSSA of PBS data can enhance signal detection. The data source and analysis method should be tailored to the particular medicine and AE being investigated.

1 Introduction

As part of the Department of Health, the Therapeutic Goods Administration (TGA) is Australia’s regulatory body for therapeutic goods. One of its roles is to monitor the safety of therapeutic goods, including medicines, that are available in Australia. TGA’s current post-marketing surveillance signal detection methods rely mostly on analysis of spontaneous adverse event (AE) reports, notifications from sponsors (Marketing Authorisation Holders) or information shared by international agencies. A signal is reported information on a possible causal relationship between an AE and a medicine, the relationship being unknown or incompletely documented previously [1]. The known limitations of current methods and the increasing availability of large administrative health data sets have given rise to greater interest in the use of these data to support pharmacovigilance (PV) [2].

In 2016, the Australian Government Response to the Review of Medicines and Medical Devices Regulation was released. One component of the response was to investigate whether existing datasets held by the Australian Government Department of Health have the potential to contribute to the evidence base the TGA considers, to investigate whether a medicine is associated with an AE. One source of information in Australia is the nationally generated Pharmaceutical Benefit Scheme (PBS) data, a reimbursement dataset of all medicines dispensed that are subsidised under the PBS. These data are useful for PV activities because of the high coverage of the Australian population and complete capture of all subsidised prescriptions dispensed. The PBS subsidises approximately 75% of prescribed medicines in Australia [3]. Utilising administrative data can potentially enhance PV by rapidly and efficiently tracking medicine dispensing and patient outcomes on a large scale, contributing to more timely signal detection and verification [4].

Studies have shown that generation of evidence from administrative data can help to complement spontaneous reporting systems by highlighting new risks associated with old drugs, AEs that have a high background incidence, and AEs that are less pharmacologically predictable and therefore less likely to be reported to spontaneous reporting systems [5, 6]. Methods to generate safety signals have been reviewed [2, 7] and when only dispensing data are available, techniques that exploit dispensing as indicators of AEs can be useful. One such method is Prescription Sequence Symmetry Analysis (PSSA), which has been used by researchers on the PBS data set [8, 9]. PSSA and prescription and event symmetry analysis (PESA) have also been used by researchers in Australia to identify heart failure (HF) as an AE following the use of some medicines, using the Department of Veterans Affairs database [10]. PESA (also referred to as sequence symmetry analysis) utilises hospital separations as the indicators of AEs whereas PSSA utilises medicine dispensing as the AE indicators. Researchers have undertaken PSSA on the New Zealand prescription database to explore its role in post-marketing surveillance [11]. While the method has been used to explore particular safety issues, its utility in the regulatory context in Australia has not been assessed.

The aim of this project was to explore, using HF as an example AE, whether PSSA using PBS data can enhance signal detection by the TGA.

2 Methods

2.1 Tested Medicines

Only single-ingredient medicines identified in the database using the ATC level five classifications and generic drug names were included. Combination products were not examined as initiating multiple products simultaneously makes it difficult to attribute the association between the AE and the individual components. We included all single-ingredient items recorded in the PBS collection except topical and non‐therapeutic agents (such as plasters and creams).

Furosemide initiation was used as a proxy for new-onset HF. All formulations of furosemide available on the PBS were included (oral solutions, tablets and injections). Medicines tested for an association with furosemide initiation are referred to as index medicines. A 1-year run-in period was used to determine incident (or first) dispensing. Only incident dispensing of index medicines and furosemide that occurred within 1 year of each other for the same patient were included in the analysis. This time period was chosen as previous analysis had determined this to be an appropriate time span in which to detect AEs with PESA/PSSA [14].

A total of 766 single-ingredient medicines were identified in the database. Seventy-one medicines did not have incident dispensing of both an index medicine and furosemide; therefore, these medicines were not examined further. Forty-one medicines had incident dispensing of both an index medicine and furosemide; however, the dispensing did not occur within 12 months of each other, therefore these medicines were also removed. After removing these medicines, a total of 654 furosemide–medicine pairs were included in PSSA testing.

A summary of the testing process is shown in Fig. 1.

Fig. 1
figure 1

Flowchart of tested medicines and detected signals. aIncident dispensing: medicine dispensed after 12-month run-in period. PBS Pharmaceutical Benefits Scheme, PI Product Information

Full size image

2.2 Source of Data for Signal Detection

This study used the PBS data collection. PBS claims data is an administrative by-product of the Department of Human Services administration of Australia’s subsidised prescription payment system. This is a national dataset that contains all subsidised scripts for Australian residents who hold a current Medicare card. As of April 2012, this contains all scripts dispensed for medicines listed on the PBS schedule regardless of price. While PBS data can include scripts of overseas visitors from countries with which Australia has a Reciprocal Health Agreement, data from these patients were not included in our study. The PBS does not include dispensings of over-the-counter medications or private prescriptions.

This study used data collected for prescription medicines dispensed under the PBS between 1 July 2012 and 30 June 2016. Medicines are coded according to the Schedule of Pharmaceutical Benefits item codes [13] and mapped to World Health Organization (WHO) anatomical and therapeutic chemical (ATC) classification [12].

The data fields used were medicine item code, supply date of medicine and patient identification key. As the patient identification key was a unique data key rather than a number that can be used to re-identify individuals, the research was considered negligible risk under the National Statement on Ethical Conduct in Human Research. The project was discussed with the Department of Health Human Research Ethics Committee, but formal ethics approval was not required.

2.3 Prescription Sequence Symmetry Analysis

The PSSA method looks for asymmetry in the occurrence of the AE before and after index medicine initiation in a specified time period [15]. In this study, furosemide initiation was used as a proxy for identification of HF. The ratio of sequences of the initiation of two medicines (one medicine being furosemide, the other being the index medicine) was determined [10]. If there is no association between the index medicine and HF, then it would be expected that the number of people initiating furosemide after starting the index medicine would be similar to the number of people initiating furosemide before the index medicine. The crude sequence ratio (CSR) was calculated by dividing the number of people with furosemide dispensed second with the number of people with furosemide dispensed first. To adjust for changes in medicine use over time, a null sequence ratio (NSR) was calculated that determines the sequence ratio that would be expected given the trends in the use of the medicines under study, and assuming there was no association between the proxy and the index medicines [15]. The NSR was then used to adjust the CSR for prescribing trends. This was done by dividing the CSR by the NSR, resulting in an adjusted sequence ratio (ASR). To determine statistical significance when there were insufficient data for a drug (total population of people who used both index and proxy drug was less than 1000), a non-parametric resampling technique was used to create bootstrapped confidence intervals for the ASR. A 95% confidence interval (CI) was generated for the bootstrapped ASR distribution using the bootstrapped-t method.

A signal was considered to be present when the ASR was > 1 and the lower limit of the 95% CI was > 1. In addition to an ASR, temporal associations of interest were graphed.

2.4 Identification of Potential New Heart Failure Signals

To identify new HF signals, the possible signals were first manually screened to exclude those known to be associated with a risk of HF.

The list of signals was limited further by excluding medicines with other clinically relevant associations with HF, such as medicines indicated for HF treatment, or as an adjunct to the treatment of HF; and medicines indicated for the treatment of conditions that are clearly associated with an increased risk of HF (e.g. nicotine addiction and chronic obstructive pulmonary disease). Signal detection with PSSA presumes that there is no underlying association between prescription of the index medicine and furosemide. Positive signals for these medicines are likely to be confounded by the underlying relationship and may not signal a causal relationship. Due to concerns as to whether PSSA could identify incident use of medicines that are used intermittently, signals for anti-infective and analgesic medicines were not considered further.

The TGA-approved Product Information (PI) was the primary reference used to screen the signals. The PIs were extracted for each medicine from the TGA’s internal repository. These were reviewed by TGA evaluators in Adobe Reader using a word search (Ctrl + F) to identify if the preferred terms for the exclusions were listed anywhere in the document, and then evaluated if the reference to the term was a relevant association. The results were tabulated in Microsoft Excel 2010.

3 Results

3.1 Potential New Heart Failure Signals

Of the 654 furosemide–medicine pairs tested, 186 (28%) had an ASR > 1 and were statistically significant (lower limit of the 95% CI for the ASR that was > 1) and therefore were considered potential signals. Of the potential signals, 122 had HF already listed as an AE in the PI, and 20 were medicines indicated for HF or for conditions associated with an increased risk of developing HF (Fig. 1). Fifteen potential signals were for anti-infective medicines and three were for analgesic medicines. The remaining 4% (n = 26) of the tested medicines were considered new HF signals.

The ASR values for the 26 new signals ranged from 1.07 for denosumab to 4.64 for temozolomide. As a class, the two alkylating agents used for cancer treatment (temozolomide and bleomycin sulfate) had the largest ASR values of the 26 medicines with new signals (Table 1).

Table 1 Medicines with potentially new heart failure signals grouped by indication and pharmacological class, ranked by adjusted sequence ratio, (n = 26)
Full size table

For each of the 26 signals, the temporal associations for signals of interest were graphed. By way of example, Fig. 2 shows the temporal analysis of patients who commenced furosemide in each week before and after fluorouracil. A greater number of people commenced furosemide after fluorouracil (n = 1956) than before (n = 1181) (ASR 1.65; 95% CI 1.54–1.78).

Fig. 2
figure 2

Temporal analysis of furosemide prescriptions in relation to fluorouracil initiation, Pharmaceutical Benefits Scheme, Australia, 2012–2016, n = 3137

Full size image

4 Discussion

This study identified 26 medicines on the PBS that may contribute to HF where this was not established in pre-market studies such as clinical trials or through post-marketing surveillance to date. These 26 medicines had statistically significant ASRs, are not indicated for use as an HF treatment (or for diseases with the obvious potential to cause HF) and did not have heart failure or peripheral oedema listed in the Australian PI at the time of the study. Results of this study indicate that the application of PSSA to dispensing data can complement the TGA’s routine signal detection work.

Of the 26 signals detected, there were five signals prioritised for further investigation: temozolomide, fluorouracil, folinic acid, vincristine and vinblastine. Priority was given to signals based on the magnitude of the ASR value, consistency of the signal via the presence of a signal for more than one medicine in the class, the number and proportionality of HF AE reports to the TGA, statistical HF signals in the WHO’s VigiBase database, published literature [16,17,18] and biological plausibility. If there was no information about any cardiac AE in the Australian PI, the potential signals were also considered to have clinical impact. For four signals, further investigation did not result in verification due mainly to likely confounding by other cardio-toxic medicines likely to be co-dispensed with these medicines and/or the likelihood that the reason for use of the medicine itself, for example advanced malignancy, is a more plausible explanation for the signal. Although the pragmatic decision was made to prioritise signals for further investigation in this way, the TGA will continue to monitor the risk of HF for all medicines through routine PV measures. For fluorouracil, the TGA was made aware that concurrent with our study, HF associated with fluorouracil was investigated by the sponsor (Marketing Authorisation Holder) and HF was added to the Australian PI. Corroboration of the PSSA signal through this process was an important result.

In this study, we used PSSA as the method to generate signals. Previous work has demonstrated that PESA, using hospitalisation and medicine dispensing data for AE indicators, has high specificity (around 90%) and moderate sensitivity (61–65%) [14, 19] for detecting known AEs, suggesting that the signals that PSSA detects are likely to be true signals but that some negative results may be missed signals. A recent signal detection study using PSSA identified a large number of known AEs and associations that reflect appropriate clinical behaviour [20]. In that study, 24% of the top 200 drug–drug associations represented known AEs and 30% were explained by mutual indication or reverse causation. In our study, we identified 122 medicines (66% of the 186 possible signals detected) where HF was a known AE, and 20 medicines (11% of the 186 possible signals detected) where the association could be explained by mutual indication. This suggests that, in practice, safety signals generated from data-driven methods, such as PSSA, require filtering to remove known or expected associations. To address this, and to enhance the potential of data-driven methods to enable more efficient, timely and proactive detection of medicine safety signals in the regulatory context, there may be a role for supervised machine learning techniques in automating the filtering of statistical output [21]. Regardless of the method used, clinical input and critical judgement is always required when interpreting the clinical importance of a statistically significant finding [6, 22].

A potential limitation of our study is the small number of medicine pairs as a result of our use of medicines at the chemical substance level, ATC level 5. We did this so that we were able to compare our results to the product information documentation to determine whether the signal was an already known AE of the medicine. Future work should consider the utility of the method at lower levels of the ATC classification categories to potentially increase the statistical power of this approach. Another potential limitation is our use of furosemide initiation as a proxy for new-onset HF. We are not aware of any formal validation of furosemide dispensing as a proxy for HF, or comparison between furosemide dispensing and HF hospitalisations as proxies for HF. In a previous study using PSSA and PESA, 12 medicines with potential HF signals using HF hospitalisations as the outcome were identified [10]. Two medicines in that study, travoprost and latanoprost, were also identified in the present study. Further examination of these two signals in our study via review of spontaneous AE reports to the TGA and internationally, and review of the cardiac safety information that was already present in the Australian PI, did not identify them as high priority signals. There were three other possible signals detected using HF hospitalisations that were also detected in our study but we did not consider them further; the first, bimataprost, has HF mentioned in the PI and the other medicines, tobramycin and paracetamol, are used intermittently and are therefore not good candidates for signal detection using the PSSA method. In the published PSSA and PESA study [10], furosemide dispensing was also used as an indicator of HF. Of the nine potential HF signals detected in that study, our study detected three of the same signals (latanoprost, brinzolomide and ranitidine). Further examination of brinzolomide in our study did not identify it as a high priority signal because of the absence of spontaneous AE reports and lack of biological plausibility. Raniditine was not considered a high priority signal in our study because of possible confounding by non-steroidal anti-inflammatory drug (NSAID) exposure, as concomitant use of ranitidine with NSAIDs to manage NSAID-associated gastrointestinal symptoms is common. Pilocarpine was not included as a signal in our study due to mention of HF in the PI and paracetamol was not included again because of intermittent use. In this study, we used the confidence interval to filter our signals for further evaluation. There are important limitations to this approach and future research should investigate the utility of different filtering approaches and how they impact on the number of signals that are generated for further clinical review [22]. Approaches to filtering signals are described in previous research by Pottegård et al. [22] and should take into account the potential public health impact of the safety issue, which can include consideration of the severity of the adverse event or the prevalence of use of the drug under study. For example, strong but non-significant signals for serious adverse events may be missed and conversely very weak but significant results may occur when the prevalence of use of the medicines is high.

The strength of our study was our use of a national data set that covers the entire Australian population. Compared with linked administrative health data sets, PBS data was also more accessible to the TGA at the time of the study. We used data from July 2012 onwards because since then, all PBS‐listed medicines have been captured in the PBS dataset, regardless of price [23]. PBS data, however, does not include medicine dispensing that occurs outside of the PBS (e.g. unsubsidised private prescribing and over-the-counter usage). This is particularly relevant for examining AEs occurring with newly authorised medicines. In Australia, there is a time lag between authorisation (or registration) and PBS listing (or subsidisation), and some medicines are never PBS-subsidised. This highlights the importance of using multiple complementary approaches for signal detection for the purpose of regulatory PV.

In a regulatory context, the post-market safety of the newer biological medicines is of particular interest due to the difficulties assessing safety during the pre-approval phase (due to the limited predictability of animal studies and a high immunogenicity profile compared with chemically synthetised molecules) [24]. Given their potentially high utilisation, it will be important to proactively monitor the safety of biologic medicines to determine whether regulatory action is required. For ten monoclonal antibody medicines, this study demonstrated an association with HF, which was already known for eight of the medicines (see electronic supplementary material). The ability of PSSA using PBS data to detect a known AE with these medicines suggests it could be a valuable method to detect other AEs with these medicines. If the biological medicine of interest is on the PBS, it may be possible, for example, to use corticosteroid dispensing as a proxy for immune-mediated AEs. Thyroxine and insulin have been investigated as proxies for hypothyroidism and type I diabetes mellitus, respectively, [14, 25] and it may be possible to use these as proxies for immune-mediated endocrinopathies.

An important limitation of any signal detection method is the potential for confounding due to co-prescribing. When signals are detected using spontaneous AE reports, additional information provided by the reporter may allow an assessment of whether co-prescribing is likely to be confounding the signal. Many of the potential signals detected in this study were for medicines used in cancer treatment, which often involves the use of multiple medicines that have known associations with HF. To enhance the feasibility of PSSA in practice, co-prescribed information could be generated or analysis stratified by co-prescribing of other medicines.

In the current study, only medication dispensing data were available for use by the TGA and therefore other critical AEs such as myocardial infarction, renal failure or pancreatitis could not be examined as there are no medicines specific to the management of these conditions. To enhance the applicability of PSSA and PESA for regulatory signal detection, accessing datasets that include hospital separations as well as PBS data would allow a larger number of critical AEs to be investigated. Studies have validated PESA using both hospitalisation and medicine dispensing data for the AE indicators [14, 19]. Using hospital separation data as a proxy for AEs would not, however, overcome the problem of confounding due to co-prescribing, not being able to investigate AEs associated with intermittently used medicines (e.g. antibiotics and analgesics) and AEs associated with combination products. Whether hospitalisation itself influences the use of the index medicine being investigated also needs to be considered in PESA [26, 27]. While the identification of acute AEs is of critical importance, it is also of interest to determine the longer-term effects of medicines, particularly for chronic diseases. PESA and PSSA generally examine medicine usage and outcomes that occur within a short time period, due to the potential impact of time varying confounders when longer exposure periods are used [14]. This means that they are best suited to examining acute-onset drug reactions rather than AEs with a longer latency such as some cancers. Other pharmacoepidemiological methods such as new user cohort and other self-controlled study designs applied to administrative health data warrant further investigation in this setting.

In PV it is important that a complementary approach is taken in which multiple methods are employed and multiple data sources are interrogated [28]. While spontaneous reports are likely to capture the use of newly authorised medicines, the disadvantage of these data are that they require suspicion of the causal link between the AE and the medicine for a report to be made, which may lead to a delay [29]. In contrast, an advantage of using administrative data is that it does not rely on a person making the connection between the medicine and the AE, and then reporting it. This gives proactive data-driven approaches the potential to detect unsuspected and pharmacologically unpredictable AEs [5, 6]. Spontaneous reporting systems and electronic health record-based systems for signal detection can complement each other, with the additional value of one over the other varying dependant on the nature of the adverse event [30]. This study found that on the one hand, AEs that are not obviously attributed to medicines (because they are multifactorial) or that already have a high background incidence may be poorly captured by spontaneous reports. On the other hand, compared with an electronic health record-based system, a spontaneous reporting system may be more cost effective overall [30].

5 Conclusion

In this study, we identified that the PSSA method can be utilised in PBS data to help enhance signal detection. We identified five potential new signals of HF for further examination. With the increasing availability of different administrative health data sources for monitoring medication safety, the strengths and weaknesses of methods used to analyse these data to support rapid and responsive regulatory PV should continue to be evaluated. Future research should examine how multiple complementary approaches to signal detection for the purpose of regulatory PV can be used to enhance the quality and timeliness of medication safety monitoring.