Background

It typically takes many years before an association between a drug and a rare, serious adverse drug reaction (ADR) is established. To ensure patient safety, there is a need to develop robust and rapid methods to identify such associations in a timely manner. Every year prescription drugs cause fatal ADRs in 100 000 individuals in the US, making prescription drugs between the fourth and sixth leading cause of death, after heart disease, cancer, stroke, pulmonary disease and accidents.[1] However, these estimates, as indicated by the authors, must be viewed with circumspection because of the heterogeneity among studies and biases in the samples.

Before a pharmaceutical product is marketed, the manufacturer must prove that the medication is both effective and safe by performing extensive studies in animals and in humans. However, premarketing studies cannot guarantee product safety, as they are limited by the small numbers of patients enrolled (typically between 1000 and 3000 subjects). Importantly, serious ADRs (e.g. agranulocytosis) occur at rates between 1 : 1000 and 1 : 10 000.[2] Moreover, clinical trials typically exclude vulnerable populations such as pregnant women, children, elderly people, those with complicated diseases and those taking other medications. Hence, uncommon adverse effects, delayed effects or consequences of long-term drug administration are not typically observed before the drug has been marketed.[2] While the more common type A ADRs (reactions that are an augmentation of the normal pharmacological actions of the drug) may already have been identified by the time of licensing,[3] type B ADRs (idiosyncratic or bizarre reactions that cannot be predicted from the known pharmacology of the drug) will typically be detected only after licensing, through postmarketing surveillance.[4]

Postmarketing studies are based on collecting spontaneous case reports of ADRs. There are two types of systems used to disseminate voluntary reporting of ADRs. The first is the published medical literature, which is a highly efficient warning system for new adverse reactions, and often recognizes rare events and patients at high risk.[5] These include case reports, cohort and case-control studies. The second consists of national and international ADR monitoring centres, such as the US FDA, which started its voluntary reporting system in the late 1960s, receiving reports from healthcare providers, consumers and pharmaceutical companies. Unlike healthcare providers and consumers, the manufacturers have a regulated duty to report any ADR to the regulatory agencies. The FDA system had been criticized[6] for its delay in sending reports of newly identified ADRs to the Physicians’ Desk Reference. The FDA system receives a large number of ADR reports from all interested parties, and it is impossible to analyze all incoming reports in depth. Moreover, the FDA has not had the regulated authority to mandate drug manufacturers to conduct directed postmarketing surveillance studies,[7] which could help in detecting uncommon serious ADRs.

In this study, we evaluated a detection method developed to allow regulatory agencies and manufacturers to rapidly identify a signal for a drug-ADR association. This method could help in protecting the public from unexpected harmful effects of new drugs. We have used pemoline-associated acute liver failure as a model for the development of this system.

Pemoline Index Case

Pemoline (Cylert™)Footnote 1 is a mild CNS stimulant that was approved in 1975 for the treatment of children with attention deficit hyperactivity disorder (ADHD).[8,9] In 1995, researchers at the Hospital for Sick Children in Toronto, Ontario, Canada, reported the case of a 14-year-old boy diagnosed with ADHD,[10] who had been previously healthy and had received concomitant pemoline 37.5 mg/day for 16 months and methylphenidate 20 mg/day for 2 months to control his symptoms. He was hospitalized because of jaundice, which progressed into acute liver failure. A liver biopsy was suggestive of drug toxicity. He underwent liver transplant, which failed, and the child died. All known causes of liver failure were ruled out, including infection, metabolic disease, tumour and chemical exposure.

The practitioners who were caring for the child found two previous published fatal cases caused by acute liver failure associated with pemoline,[11,12] both from the US. Berkovitch and colleagues wrote that, “the U.S. FDA, and the manufacturer are not aware of additional cases”.[10] They estimated that a child receiving pemoline had a relative risk (RR) of developing liver failure of 45.3 (95% CI 4.1, 510) and urged others to report similar cases. This highly significant association (p < 0.001) suggested causation. This crude estimate was based only on three reported cases, assuming that 80 000 children had been exposed to the drug in North America until that year, and a 1 : 106 rate of fatal paediatric liver failure.[10]

After this report, investigators around the world began to report additional cases of liver failure associated with pemoline. A black box warning was added to the labelling in the US in December 1996, and a ‘Dear Doctor” letter was mailed out by the manufacturer to all US physicians, advising that the drug only be used as a last resort; however, many physicians continued to use pemoline as a first-line therapy.[13]

In September 1999, Health Canada withdrew pemoline from the Canadian market.[14] More cases of pemoline-induced liver toxicity were reported, and pressure on the FDA increased to ban the drug. In May 2005, the manufacturer chose to discontinue sales and marketing of Cylert™ in the US. However, Cylert™ would remain available through pharmacies and wholesalers until supplies were exhausted. In November 2005, pemoline was finally removed from the US market.[15]

From these events one can note that there was a 25-year delay in identifying pemoline-associated acute liver failure, leading to a delay in withdrawing the drug from the market and putting children at risk for fatal drug-related injury.

The objective of this report is to describe a system that is aimed at estimating the potential association between a serious adverse event and a medicinal drug early in its marketing cycle. We hypothesized that this system could predict pemoline-associated acute liver failure several years after its marketing, by using existing data.

Methods

Study Design

This was a postmarketing surveillance study of pemoline-induced hepatotoxicity based on cases reported to either the FDA and/or through the medical literature.

The study population comprised children born in the US and Canada. The number of children who were receiving pemoline was estimated for each calendar year after 1975, when pemoline was approved to be marketed as a treatment for ADHD, as was the number of reported cases of acute liver failure associated with pemoline.

After determining the number of children treated with pemoline per year and the number of children per year who developed acute liver failure while receiving the drug, a comparison was made between these data and the background incidence rate of idiopathic acute liver failure in children in the general population. This comparison was made year by year, to define the earliest year when the rate of serious acute liver failure caused by pemoline was significantly higher than the rate predicted in the general population.

Data Collection

Rate of Idiopathic Acute Liver Failure (ALF) in Children

Data were synthesized from a systematic review of the published literature. We searched PubMed, EMBASE and Scopus for studies published from 1965 to December 2006, using the following keywords: ‘liver failure’, ‘acute’, ‘children’, ‘fulminate’, ‘idiopathic’, ‘unknown reason’, ‘incidence’ and ‘epidemiology’. Only studies calculating the rate of acute liver failure in children and their aetiologies, including idiopathic liver failure, were included.

Annual Number of Children Treated with Pemoline

The information regarding the annual number of children treated with pemoline in Canada was obtained from International Medical Statistics (IMS) Inc., Montreal, Quebec, Canada. IMS Inc. is a holder of statistical medical information that can be accessed by researchers, academics and government. For US data, we gathered the information by synthesizing available published data on the annual number of children treated with pemoline, by a systematic review of MEDLINE, EMBASE and Scopus and obtaining all articles that reported on the number of children prescribed pemoline. For years that did not have reported utilization data, a mean was calculated from the data in the years before and after.

Annual Number of Children Treated with Pemoline in the US and Canada Who Developed ALF

Information on the annual number of children treated with pemoline who developed acute liver failure was obtained from the FDA under the Freedom of Information Act (2002). All pemoline cases reported to the FDA between 1975 and 1999 were analyzed. The criteria for selection of liver injury cases were age between 0 and 18 years and any report of irreversible damage to the liver in children who received the dose schedule recommended for the primary indication.

We excluded cases reporting increased liver enzyme levels that returned to normal once pemoline was discontinued. Some of these cases have also been published in the literature and all articles were obtained. We compared these reports to ensure that no double counting occurred (i.e. including the same case from the FDA and the published literature). We used the same criteria for selection of the literature cases as was used for the FDA cases.

All the published cases were subjected to causality assessment using the Naranjo ADR probability scale.[16] The Naranjo ADR probability scale is a tool widely used to determine the likelihood that an ADR is caused by the implicated medication. Ten questions were answered and assigned a weighted score of +2 to −2. Where there were insufficient data available, the particular question received a score of 0. Based on the Naranjo criteria[16] each case was assigned a likelihood of causing an ADR from ‘doubtful’, ‘possible’, ‘probable’ to ‘highly probable’.

Data Analysis

Analysis of the data started by calculating the yearly RR, which is the ratio between the incidence rate of irreversible acute liver failure among children exposed to pemoline and the incidence rate of idiopathic acute liver failure in children in the general population, using the following equation (equation 1):

$$RR = \frac{{a/(a + b)}} {{c/(c + d)}}$$
((Eq. 1))

where ‘a’ is the number of children treated with pemoline who developed acute liver failure, ‘b’ is the number of children treated with pemoline who did not develop acute liver failure, ‘c’ is the number of children not treated with pemoline who developed idiopathic acute liver failure and ‘d’ is the number of children not treated with pemoline who did not develop acute liver failure.

Fisher’s Exact test was also used to compare the rate of acute liver failure associated with pemoline with the rate in the general population.

Results

Rate of Idiopathic ALF in Children

Based on a recent large comprehensive study of fulminant hepatic failure, approximately 230 children are estimated to be afflicted by this condition each year in the US.[17] The number of children who live in the US has been estimated at 73 043 506,[18] and hence the overall rate of acute liver failure in children is estimated at 1 : 300 000.

All available published studies reporting on the breakdown of aetiologies of acute liver failure in children were reviewed.[17,19–21] Based on these studies, a mean 16% of the cases of acute liver failure were due to unknown (idiopathic) reasons (table I). Hence, we estimated that the rate of idiopathic liver failure in North American children is 1 : 2 000 000.

Table I
figure Tab1

Medical literature reports of idiopathic acute liver failure (ALF) in children

Full size image

Annual Number of Children Treated with Pemoline in the US

We identified five papers that surveyed the prevalence of medication use to treat children with ADHD in the US, and these were used to calculate the annual number of American school children on ADHD medication from 1975 through 1993 (table II) [22-26] For any missing year for which no publication was available, we calculated the mean value from the closest years before and after. The percentage of children receiving treatment with stimulant medication for ADHD ranged between 2.1% and 6%. The percentage of reported pemoline use among ADHD children was 1% in 1975 and increased gradually to 6% in 1987. Between 1987 and 1993, there were no data available on the percentage of pemoline use among ADHD children, so we calculated the mean value based on data from these years. The overall number of school children in the US was obtained from international statistics.[27] Based on these numbers we estimated the number of children taking pemoline in the US (table III).

Table II
figure Tab2

Published reports of US school children treated with attention deficit hyperactivity disorder (ADHD) medications and treated with pemoline therapy from 1975 to 1993

Full size image
Table III
figure Tab3

The calculated number of children treated with pemoline in the US by year

Full size image

Annual Number of Children Treated with Pemoline in Canada

IMS Inc. provided us with the annual number of pemoline prescriptions dispensed by Canadian retail pharmacies from 1987 to 2004. Due to privacy issues, which restrict the company from measuring how many individual patients took the drug, only the number of prescriptions was available. IMS Inc. calculated that the average prescription of pemoline in Canada contained 62.5 tablets, and we assumed that a typical child received one tablet per day for 1 year. Hence (equation 2):

$$No. children treated with pemoline per year = \frac{{No. pemoline prescriptions dispensed \times 62.5}} {{No. days in the year (365)}}$$
(Eq. 2)

Based on these calculations, we estimated that a total of 45 404 Canadian children were treated with pemoline from 1978 to 2004 (table IV). The marketing of pemoline in Canada started only in the 1980s. The Canadian Drug Identification Codes are drug product database books published annually. The first year pemoline was included in these databases was 1981.[28] The first year pemoline was included in the Compendium of Pharmaceuticals and Specialties was 1986.[29]

Table IV
figure Tab4

The calculated number of children treated with pemoline in Canada from 1987 to 2004

Full size image

Annual Number of Children Treated with Pemoline in the US and Canada Who Developed ALF

ALF Cases in Children Receiving Pemoline and Reported to the FDA

Thirty cases of children who were treated with pemoline therapy and who developed irreversible acute liver failure were reported to the FDA. The first case was in 1977. Table V presents the clinical details and the outcome of these 30 cases. Figure 1 shows the annual number of cases reported to the FDA and in the medical literature who developed acute liver failure while receiving pemoline.

Fig. 1
figure 1

Cumulative chart of children developing acute liver failure after receiving pemoline (cases from the US FDA and the medical literature for the period 1975–98).

Full size image
Table V
figure Tab5

Clinical details of 30 cases with pemoline-induced acute liver failure reported to the US FDA from 1975 to 1998

Full size image

Peer Review Reports of Hepatotoxicity and Liver Death Associated with Pemoline

All published articles on pemoline-associated hepatotoxicity meeting our selection criteria were reviewed (table VI).[10–12,30–34] The first case of a hepatotoxic death caused by pemoline occurred in 1977, but was published in 1989.[11] The total number of published cases was 11, and most of them were reported in 1998 (figure 1).

Table VI
figure Tab6

Medical literature reports of acute liver failure and hepatic death ascribed to pemoline

Full size image

Application of the Naranjo ADR probability scale yielded ten cases classified as ‘probable’[10,12,30–34] and one case as ‘possible’[11] (table VII).

Table VII
figure Tab7

Naranjo causality assessment scores of the medical literature reports of acute liver failure ascribed to pemoline

Full size image

Calculating Relative Risk of Pemoline-Associated ALF

Using FDA Reports

The RR of children treated with pemoline developing acute liver failure was high in each year after introducing pemoline, ranging between 9.12 and 24.08. The highest RR was detected in 1978. All values of RR were statistically significant (table VIII).

Table VIII
figure Tab8

The yearly accumulative relative risk (RR) and 95% confidence intervals of children treated with pemoline developing acute liver failure from 1977 to 1993 (cases from the US FDA reports)

Full size image

Using the Published Medical Literature

We repeated the calculation of the annual RR and statistical likelihood of children treated with pemoline developing acute liver failure for the period 1977–93, using only published cases as the source of the cumulative information. We chose the first four published cases because we had the data on the annual number of children receiving pemoline treatment. The paucity of published cases caused large fluctuations in the RR values, as well as in their level of significance (table IX).

Table IX
figure Tab9

Relative risks (RR) and p-values of children treated with pemoline developing acute liver failure from 1977 to 1993 (cases from the medical literature)

Full size image

Discussion

The goal of this research was to develop a system for the early detection of serious ADRs for newly marketed medications. Using pemoline as a model, we were able to show that by using existing data at the time, one could estimate the high incidence rate of a serious ADR early in the marketing cycle. This method has similarities to the recent case-population approach used to estimate an incidence or an absolute risk of particular drug-induced diseases.[35] A similar method is currently used by European regulatory authorities whenever the background incidence rate of the event is known.

Hence, as early as 1978, a significant signal existed indicating that pemoline is associated with acute liver failure, 16 years before the first published suggestion by Berkovitch and colleagues,[10] 22 years before removal of the medication from the Canadian market and 28 years before removal from the US market.

The incidence rate of acute liver failure in children was estimated by us at 1 : 300 000. About 16% of these cases are idiopathic with a resulting annual incidence of one case per 2 million children in the general population. Hence, a rate of 1 : 2 000 000 was used in this study as the background incidence of idiopathic acute liver failure in children in the general population. When a child taking a specific medication develops severe idiopathic acute liver failure, the cause for this adverse event may be the drug itself or yet another unknown cause. The use of our surveillance method can be very effective in demonstrating or refuting an association between a drug and the specific adverse event. The more cases of the specific adverse event that are reported and the lower the background incidence of the idiopathic adverse event, the earlier an association between the drug and the adverse event can be established in the cycle of drug marketing.

Our estimated use of pemoline for ADHD increased from 1% to 6% of all medications taken for ADHD by children since its introduction in 1975 and 1987. This agrees with a study conducted in the US.[36] The treatment prevalence of pemoline among US school children kept increasing despite escalating reports of hepatic toxicity associated with the drug. This was due to the delay in awareness of the association between pemoline and acute liver failure, which should have been highlighted in a black box warning label. The black box warning label regarding risk of acute liver failure with pemoline use was added only in 1996.

In Canada, pemoline was introduced in the 1980s, and continued to increase in popularity year by year until the introduction of the first warning label in 1996, soon after publication of the index case. The publication appears to have been effective, as most Canadian physicians complied with the warning label by reducing use of the drug. This is evidenced by a sudden decline in the number of prescriptions of pemoline issued in 1997 by Canadian physicians (table IV). In subsequent years the number of prescriptions kept declining until Health Canada decided to take the drug off the market in 2000.[14]

The first pemoline-associated hepatotoxic death reported to the FDA occurred in 1977. The event occurred only 2 years after the drug was introduced into the US market. Between 1975 and 1998, we identified, among the numerous cases reported to the FDA, 30 serious cases, all involving children between the ages of 0 and 18 years. These included nine cases of hepatic death and seven cases of liver transplant. A smaller number of serious pemoline hepatotoxicity reports were published in the medical literature than the actual number of cases reported to the FDA.

In order to determine the year in which the association between pemoline and acute liver failure became significant, the RR and p-values were calculated for each year. This analysis showed that in 1978 there was an estimated 24-fold increased risk of acute liver failure caused by the use of pemoline compared with the incidence rate of idiopathic acute liver failure in the general population (RR 24.08; 95% CI 4.67, 124.1; p < 0.05). In the following years, the level of significance kept increasing as more cases were reported.

If the information regarding the unfavourable acute liver failure caused by pemoline had been available, and if a similar analysis had been done as soon as the first cases were reported, many children would not have been put at risk and some children would not have died. The warning label could have been introduced earlier, and the withdrawal of the drug could have been implemented without delay, especially as there were other effective medications for ADHD available.

After the introduction of a new drug to the market, the prevalence of medication use is known to the manufacturer based on sales. Knowing the number of users of a particular medication, along with the number of cases of the specific adverse event, can allow the manufacturer and the regulatory agency to establish or refute associations between the drug and the adverse event by using the method introduced here.

Our study may suffer from potential weaknesses based on the a priori assumptions made and the sources of literature data. In the context of this study, such biases may include the following:

  • • There is no health service system that contains all the data on the number of drug users in specific jurisdictions and detailed information on patient characteristics. This may be a potential limitation to the study if there was an overestimation of the number of users of pemoline (i.e. increase in cell ‘b’ in the 2 × 2 table described in equation 1). As a result, the signal would be diluted and there would be a delay in recognition of the association between the drug and the adverse event, and vice versa.

  • • The voluntary reporting system may result in incomplete information provided to the FDA, which could have led to under-reporting of serious adverse events. The under-reporting (i.e. decrease in cell ‘a’ in the 2 × 2 table) would also have led to false negative results (dilution).

  • • The reference incidence rate of idiopathic liver failure in children was based on systemic review of all studies covering the years from 1982 to 2003. Yet these numbers are relatively small. If the calculated incidence was falsely high, this would have given a high background incidence of idiopathic liver failure in the general population (i.e. decrease in cell ‘d’ in the 2 × 2 table), and the potential signal would be again diluted. The opposite would have happened if the calculated incidence was falsely low.

However, our newly proposed system is not intended to generate an accurate RR but rather to create a signal that should lead to more in depth investigation. In this context, it is very unlikely that an RR of 10 or 15 would be artificially produced by minor inaccuracies. Moreover, most of the potential flaws, if they existed, would have diluted the signal and not augmented it.

This method is useful only if the event has a low background incidence. Unfortunately, in daily practice many events have a rather high background incidence. Hence, the strength of this method is for rare but serious ADRs.

While we present an analysis related to acute liver failure, similar population-based statistics are available for all other adverse events, from agranulocytosis to pulmonary fibrosis.

Conclusion

This method should enable researchers, clinicians, drug companies and regulators to identify uncommon ADRs earlier than they are currently in the course of marketing and thus quantify serious ADRs and identify patient populations at high risk.

Because the data needed to establish the proposed surveillance system are regularly collected by manufacturers and regulatory agencies, this method should be implemented as a prospective, or ‘real time’ tool, enabling more accurate estimates of the total annual usage of a drug, the number of patients treated and dosage rates.