FormalPara Key Points

Spontaneously reported adverse drug events can be a key data source for safety advisories about new adverse effects of licensed therapeutic agents. But in a voluntary reporting system, these safety advisories might also affect the number of events later reported. Health professionals might be less likely to voluntarily report an adverse effect that was already well documented. Conversely, some studies have also reported that certain advisories or media publicity led to increased reporting, sometimes described as “stimulated reporting.”

The objective of this study was to measure changes in reporting of a representative group of US Food and Drug Administration Drug Safety Communications. We compared the number and proportion of reports of the specific adverse effect both before and after the advisory date.

We found no consistent pattern of increased or decreased reporting of an adverse effect following a safety advisory although large increases were seen in a few cases. This study adds to the evidence that while adverse event reports can help establish an association of a suspect drug and adverse effect, these kinds of data may not reliably establish incidence or reporting trends over time.

1 Introduction

Spontaneously reported adverse drug events are one source of data for the US Food and Drug Administration (FDA) and other regulators for detecting signals of new adverse effects of therapeutic drugs after marketing approval [1]. However, only a small fraction of cases that occur are reported in this voluntary system [2, 3]. These reports themselves may also subsequently become the primary scientific evidence for a safety advisory or new warning or restriction in the prescribing information [4]. In addition, adverse event signals detected through other source data such as electronic health records studies, clinical trials or media reports may be evaluated in adverse event databases such as the FDA Adverse Event Reporting System (FAERS) [5].

However, some studies focusing on selected adverse events have reported that media publicity or the safety advisories themselves were associated with an increase in spontaneous reports, a phenomenon characterized as “stimulated reporting” or “notoriety bias” [6,7,8].

On the other hand, other studies have found no consistent effects. Hoffman et al. examined 100 drugs approved from 2001 to 2010 and found a 1% or greater short-term increase in adverse event reporting for 21 drugs that had safety alerts, but also a similar trend in 25 drugs for which the investigators picked a random date for a sham advisory [9]. A study of trends in reporting pathological gambling as an adverse event associated with dopamine agonist drugs [10] concluded that the sustained growth in gambling reports that occurred periodically over a decade sometimes fluctuated with media publicity [11].

In this study, we assessed the effect on spontaneous reporting of FDA Drug Safety Communications from 2010 to 2015 that were the first advisories for a specific safety issue among regulators in the USA, Australia, Canada, and the UK.

2 Methods

The data for this study were the publicly released computer excerpts of 11 million adverse drug event reports submitted to the FDA Adverse Event Reporting System (FAERS) from 1998 to 2019. The quarterly data were further enhanced for research use. The drug names were standardized to chemical ingredient names in the National Library of Medicine RxNorm drug terminology [12]. The event terms were standardized to the Medical Dictionary for Regulatory Activities (MedDRA) [13], the globally used controlled vocabulary for reporting adverse events in clinical trials and spontaneous reports. Only a single copy of each adverse event report (which may be revised, generating duplicates) was selected for analysis using the rule of retaining the earliest initial report date, but the most recent revision. Because the FDA de-identified all personal data prior to public release for research use, we did not seek Institutional Review Board approval.

The safety advisories for this study comprised 16 Drug Safety Communications issued by the FDA from 2010 to 2015 as part of a broader comparative analysis of 1441 safety advisories in Australia, Canada, the USA, and the UK [14]. This multi-factorial study compares the selection and effects of safety advisories by the different regulatory authorities. The safety advisories for this analysis were selected as an adjunct to an ongoing international interrupted time series analysis comparing effects on the drug utilization of 25 safety advisories in the four countries [15]. The criteria for selection of these advisories are described in detail in Fig. 1. To avoid possible confounding by the effects of foreign safety advisories on US adverse event reporting, the subset of 16 were those where the warning occurred first in the USA.

Fig. 1
figure 1

Selection of drug safety advisories for related international study and current study of adverse event reporting. OTC over-the-counter

Full size image

The Drug Safety Communication texts were obtained from the FDA web site. For an adverse event report to be included, the study drug had to be identified as the primary suspect drug in the FAERS report. For each advisory we selected the related Preferred Terms (PTs) using the current authorized MedDRA browser to code the adverse event described in the advisory text. The number of terms coded varied with the specificity of the advisory and its definition of the adverse effect. Only a single PT term (priapism) applied to the advisory for methylphenidate. At the other extreme, we used 2294 different PT terms to capture reports of “cancer” associated with insulin glargine. We limited the study population to adverse event reports that originated in the USA.

Our analysis compared the reports received in the eight quarters preceding the safety alert with those received in the four quarters following the calendar quarter in which the safety alert occurred. The quarter in which the alert occurred was excluded. The FDA’s reporting requirements [16] dictated the calendar quarter as the time unit of analysis. While new serious adverse events have to be reported within 15 days, an adverse event for which an adequate warning is deemed to exist, or which did not have a serious outcome, are required to be reported on a quarterly basis. Adverse events for which the FDA mandated a warning could reasonably be judged to be adequate in subsequent quarters and thus some might be reported on a quarterly basis. The calendar quarter of the warning was excluded because it contained some periods both before and after the warning. The excluded quarter also allowed a time lag between the issue date of the advisory and when reports that might be related to the warning event would begin the flow into the FAERS system. After 3 years, manufacturers may submit periodic reports annually; this pattern was not detected among any of the study drugs.

To assess whether changes had occurred between the pre-warning and post-warning period we calculated a new use of the reporting odds ratio (ROR) for the safety alert event using Fisher’s Exact Test. This self-controlled method compares the proportion of (advisory/non-advisory) events in the pre- and post-periods [(pre-events/pre-non-events)/(post-events/post-non-events)]. Our prospectively set threshold for ruling out the null hypothesis was ROR ≥ 2, p < 0.05, a standard used in other forms of disproportionality analysis of adverse event data [17]. The use of this self-controlled method allowed for comparisons between safety advisories with wide variation in the drug indications, FDA approval dates, patient exposure as well as between warning events that ranged from rare conditions [progressive multifocal leukoencephalopathy (PML)] to highly prevalent conditions (cancer, cardiovascular events).

The data for this study were maintained in a MySQL database (Oracle, 2019) and were analyzed with the open source R statistical program (R Project for Statistical Computing, 2019).

3 Results

The data set included 51,176 adverse events, with 33,737 (65.9%) in the eight quarters prior to the advisory and 17,439 (34.1%) in the four quarters after the advisory. The results are shown in Table 1. The fewest case reports were 726 for olmesartan; the most were 11,928 for fingolimod. The advisories described some events that either occur rarely, are rarely reported, or both. The events of PML, congenital anomaly, and priapism were associated with 13 or fewer reported cases each for the study drugs over the entire 3-year period. At the other extreme, we identified 1414 reported cases of bladder cancer in which pioglitazone was the primary suspect drug.

Table 1 Association of FDA Drug Safety Communications with adverse event reporting of the advisory adverse event
Full size table

We found a statistically significant increase and a ROR ≥ 2.0 for post-risk advisory adverse event reports in 4/16 safety advisories. Three more advisories had RORs that were less than 2.0 but were statistically significant increases. For 7/16 advisories, we did not find a statistically significant change in reporting. We were unable to calculate reporting odds ratios for two safety advisories. The FDA safety advisory for ketoconazole specified that it applied only to the brand name tablet form (Nizoral) of this antifungal agent. However, the FAERS reports for ketoconazole frequently did not distinguish between the oral and topical formulations with either the same brand name or chemical name. The safety advisory for cardiovascular effects of varenicline could not be evaluated because of uninterpretable report dates in thousands of reports from the manufacturer. In one of the pre-warning quarters, the FDA received 26,000 reports of events that had occurred earlier but had not been properly reported to the FAERS system [18].

The assessment of reports for fluconazole and congenital anomaly included a slight change in method. Congenital anomaly is an event outcome (such as death, hospitalization) and was used to identify cases instead of specific MedDRA PTs.

4 Discussion

This study of a substantial group of US safety advisories did not detect a consistent pattern of increased reporting in the four calendar quarters following the initial alert. It confirms a study of safety alerts from 2001 to 2010 that relied on different methods and concluded “a few clear cases occurred” but no overall pattern of increased reporting could be detected [9].

We did observe post hoc that some safety advisories with large increases in target events from pre- to post-periods later resulted in multiple lawsuits for damages, even though case reports specifically tied to litigation were rare in the one-year post-advisory period. For example, pioglitazone became the target of more than 10,000 lawsuits for bladder cancer that were settled in 2015 for $2.4 billion [19]. However, only two adverse event cases in the post-advisory period were coded as being related to litigation. Similarly, canagliflozin was listed in more than 1000 federal lawsuits in 2019 [20], but we could identify no cases in the immediate post-period where the initial reporter was a lawyer.

Following some safety advisories, it is likely that internet and TV advertising for potential clients increases public awareness of the event and its association with the drug, leading to increased reporting. Cases directly related to litigation are typically submitted months-to-years later. However, the emergence of mass torts also signals that the solicitation was successful in identifying substantial numbers of patients who experienced the adverse event with evidence credible enough to support a possible lawsuit. Therefore, mass torts may also reflect more serious and widespread harms.

4.1 Limitations

Our study of Drug Safety Communications was limited to 16 safety advisories where no similar alert that met our inclusion criteria had been previously issued in Australia, Canada, or the UK since 2007. The safety alert itself does not prove that the adverse effect described was in fact caused by the primary suspect drug. For example, the safety alert associating cancer with insulin glargine was issued in 2009 and cited four observational studies “that suggested an increased cancer risk” [21]. However, in January 2011 the FDA issued a second safety advisory saying, “The evidence presented in the studies is inconclusive due to limitations in how the studies were designed and carried out” [22]. Confounding can also occur when multiple safety advisories are issued for different adverse effects of the same primary suspect drug. For example, the canagliflozin safety advisory in 2015 concerned increased risk of bone fracture [23]. However, in 2015–2016 the FDA issued three separate advisories about canagliflozin or its drug class, two about the risk of ketoacidosis, and one about increased risk of leg and foot amputations [24,25,26].

An additional consideration in evaluating changes in adverse event reporting is the suitability of the spontaneous reporting system for capturing different kinds of adverse events. This is reflected in part in the original data sources and likely event prevalence for the different safety advisories. Adverse event data itself was the source data associating fingolimod with PML, a rare, distinctive, and frequently fatal brain infection seldom seen outside immunocompromised patients. On the other hand, the advisories for cancer associated with insulin glargine and bone fractures associated with canagliflozin describe prevalent disorders with multiple causes. The warning source data for the insulin and canagliflozin advisories were observational studies.

Higher prevalence and the presence of multiple alternative potential causes complicates the assessment of causality in spontaneous adverse drug event reports. In addition, individual case reports vary in quality, completeness, and credibility. Although we used standard MedDRA coding practices, our event counts could also have been affected by the selection of MedDRA PTs to code the language in the safety advisories.

5 Conclusions

Changes in reporting following a safety advisory need to be evaluated on a case-by-case basis focusing on the nature of the event described and considering numerous contributing and confounding factors. Furthermore, substantial numbers of reported events that come after a safety alert tend to support the strength and validity of the association, but do not provide reliable evidence of incidence or trend over time.