Background and objectives

Adverse drug reactions (ADR) cause significant mortality and morbidity and present either with symptoms and/or changes in laboratory values. We have shown that the detection rate of ADRs in children and adolescents would almost be doubled by a computerised monitoring system (CMS) analysing laboratory data [12]. Within the last few years computerised methods for detecting adverse drug reactions have been established for adult patients [1, 3, 5, 8]. Recently, Haffner and co-workers [7] have shown that computerised screening of laboratory values is feasible in children using predefined laboratory filter signals. Compared to intensified surveillance, computerised laboratory signals detected mostly minor ADRs in addition to the intensified surveillance system. Using a similar, but methodologically a somewhat different approach, we further tested the value of computerised monitoring system in paediatric patients.

Methods

During a 6-month period, a prospective pharmacoepidmiological survey was carried out on a 22-bed paediatric isolation ward at the Children’s University Hospital, Erlangen-Nuremberg, Germany. The study protocol was approved by the local Ethics Committee. All patients admitted were prospectively monitored with respect to ADRs.

For detection of ADRs, both an intensive chart review system and a computer-based monitoring system were used [1, 4]. All charts were reviewed three times a week with respect to potential ADRs by a pharmacoepidemiological team consisting of a paediatrican and a clinical pharmacist. If any suspicious symptoms, laboratory data or any other unexplained medical findings were documented in the patient’s chart, or were reported to the attending physician, the team analysed the likelihood of an ADR according to the WHO definition. The physician participating in the team was specialised in paediatrics, the two pharmacists in clinical pharmacy and also involved in the drug information service of the University Hospital. The team had previously been involved in similar studies in paediatric and adult patients [4, 5, 12]. Decisions were made by consensus after intensive discussion in the group. All events considered to be ADRs by this review were taken as 100%. Laboratory tests (139 different ones) were only performed according to the clinical needs as requested by the treating physician. The total number of laboratory tests during the study period was 27,434, and for 10,858 predefined rules were available.

The CMS daily received data from the local Hospital Information System automatically, e.g. administrative data and all laboratory test results. Based on predefined study rules it generated a list of automatic laboratory signals (alerts) of possible ADRs including patient data and date of events. Table 1 shows the predefined upper and lower limits and relatives changes (DELTA values) which served as study rules. Alerts were generated either when the test exceeded an upper or lower limit (static change, NEW alert) or when important differences between two measurements were noticed (dynamic change, DELTA alert) (Table 1). The first time a test parameter exceeded an upper or lower limit it was indicated as a NEW alert. Afterwards a DELTA alert was generated, if the difference between two measurements exceeded the predefined limits.

Table 1 Normal ranges (upper and lower limits) and relative changes (DELTA values) of laboratory tests used in the study for generating automatic laboratory signals
Full size table

ADRs were classified by the affected target organ according to WHO–ART system organ classes. Events detected by changes in laboratory values only were termed Lab-ADRs. Thus, two major categories for ADRs were discriminated: “detected by CMS” and “not detected by CMS”. All ADRs were characterised with respect to their probability and severity. The probability was estimated using the Naranjo Score Algorithm, where ten weighted questions had to be answered by the pharmacoepidemiological team. These questions allow the classification of the causal association between drugs and ADRs, e.g. temporal sequence, pattern of response, withdrawal, re-exposure, alternative causes, placebo response, drug levels in body fluids or tissues, dose response relationship, previous patients experience with the drug, and confirmation by objective incidence [9]. Severity was assessed by a weighted score of the following indicators of drug-caused harm (according to the WHO system): if ADR impaired the patients quality of life, caused temporary or permanent inability to work, led to or prolonged hospitalisation, caused temporary or permanent malfunction of an organ system, was dangerous, life threatening or fatal, and if withdrawal of the drug or introduction of a different drug therapy was necessary. A score of 1–4 indicates a mild ADR, 5–8 a moderate and >8 a severe adverse drug reaction. In addition, the physician’s awareness regarding ADRs was monitored. If no evidence was found in the patient record that the responsible physician recognised an ADR, it was categorised as “not recognised”. If relevant chart notes, changes in drug regimen, additional laboratory tests or other diagnostic measures were found in the patient’s chart, the ADR was assumed to be “recognised”.

For determining the precision of CMS, the specificity and sensitivity were estimated by comparing the automatic laboratory signals alerted with the number of ADRs as found by the pharmacoepidemiological team. Sensitivity was defined as the number of ADR positive admissions detected through given type of alerts out of the total number of ADR positive admissions. Specificity was defined as the number of ADR negative admissions without given type of alerts out of the total number of ADR negative admissions.

Results

A total of 396 patients (439 admissions), 214 (54%) male, entered the study. Patients spent an average of 5.2 days (median=3 days, SD±18.5) on the ward. The mean age of the patients was 6.1 years (median=4, SD±6.6). The age distribution of the study population is shown in (Table 2). A total of 1,999 drugs were prescribed (median=4 drugs per admission, min=1, max=28, SD±4.3). According to ATC classification system antiinfectives (n=315) were prescribed in 39.0% of all admissions. (Table 3). A total of 1,282 diagnoses were made in all admissions. (median=2, min=1, max=15, SD±2.025). The most common main diagnosis according to ICD 10 classification system was infectious respiratory diseases.

Table 2 Age groups of study population
Full size table
Table 3 Number of drugs prescribed according to ATC classificationa
Full size table

ADRs (n=73) occurred in 52 patients (62 admissions, min=1, max=4) during the study period, representing an ADR incidence of 13.1%. Patients with ADR were hospitalised for an average of 10.4 days in the hospital. This was significantly longer than patients without ADR. (5.1 days) (p<0.05). Compared to the total number of prescriptions, antineoplastic and immunmodulating agents (34.3%) and antiinfective agents (10.2%) led most often to ADRs (Table 3).

According to WHO–ART system organ classes, gastrointestinal system disorders were the prevailing ADRs (n=18), followed by changes in leukocyte counts (n=10). Out of all ADRs 35 (47.9%) were “Lab-ADRs” without obvious clinical symptoms (Table 4).

Table 4 ADRs and ADRs alerted by CMS according to WHO–ART system organ classes
Full size table

The CMS alerted 31 (42.4%) events at 27 admissions. The treating physicians detected 31 ADRs (42.5%) and 23 ADRs were found only by CMS. All together 74% (n=54) of all ADRs were detected by the CMS and the physicians. Out of all Lab-ADRs (n=35) which were in principle detectable by laboratory parameters 80% (n=28) were alerted by the CMS. About 25% (n=19) of all ADRs escaped both the CMS and the attention of the treating physician. They were only found by intensive chart review. Majority of those ADRs was diarrhoea (n=10), followed by nausea/vomiting (n=2). According to Table 4, hepatotoxicity represented 19.3% of all ADRs detected by CMS. Leukopenia represented 16% of all ADRs, none of which was recognised as ADR by the physicians.

According to the definition applied no “severe” ADR was found in this cohort. However, 25 ADRs were classified as moderate and 48 as mild. The CMS alerted 24 mild and 7 moderate ADRs, 21 out of the 31 ADRs recognised by the physicians were found to be moderate.

Using the Naranjo algorithm, 38 ADRs were regarded as “probable”, 30 (41%) as “possible”, 3 as “definite” and only 2 as “unlikely”. Sixty percent (n=18) of the ADRs alerted by CMS were possible and 40% (n=12) probable whereas physicians recognised 19 probable and 8 possible ADRs.

ADRs detected by the CMS parallelled the frequency of clinical testing. Leucocytes count was the parameter checked most often (n=907). Correspondingly, the CMS alerted leukocytes most frequently (n=500; 55.1 %), of which 76% indicated leukopenia. Serum creatinine and electrolytes (potassium, sodium) were also frequently tested, but only 12.5% (creatinine), 3.5% (sodium) and 10.1% (potassium) led to an alert (Table 5).

Table 5 Number of laboratory tests and number of automatic signals generated by CMS
Full size table

Out of a total number of 27,434 laboratory tests performed routinely, 10,858 applied to predefined study rules. A total of 1,563 were classified as abnormal by the CMS and used as the basis of alerts.

In all, 371 alerts [280 static and 91 dynamic (DELTA) alerts] signalled patients with possible ADRs. Out of them, 62 (16.7%) alerts were directly associated with ADRs. DELTA alerts were significantly more frequent (29.7%) indicators of ADRs than alerts based on static changes (12.5%) (p<0.05) (Table 6).

Table 6 Number of laboratory tests and alerts in ADR positive admissions and in all admissions
Full size table

The sensitivity of the system was 90.3%, since 56 out of 62 admissions with ADRs were alerted by the CMS for potential ADRs. The specificity of the system was, however, only 19.6%. In contrast, when only DELTA alerts were taken into account, specificity increased to 75.9%, i.e. only 91 out of 377 ADR negative admissions were alerted. Vice versa, the sensitivity fell to 50% since only 31 out of 62 admissions with ADR were signalled (Table 7).

Table 7 NEW and DELTA alerts indicating adverse drug reactions (ADRs)
Full size table

The potential of laboratory alerts is demonstrated in a single case: an 11-year-old boy received cefotaxime for treatment of neuroborreliosis (facial nerve palsy). On admission, his eosinophiles count was 3% and rose to 7% on day 6 and went up to 10% on day 10. On day 10, the patient also presented with a generalised maculous papulous exanthema. As a consequence, cefotaxime was stopped on day 10 and replaced by benzylpenicillin. Within the next 2 days, the exanthema improved and the patient was discharged on day 12. The first alert generated by CMS was sent 4 days before the patient showed clinical symptoms of an allergic reaction.

Discussion

This observational analysis confirmed the feasibility and the effectiveness of a computerised monitoring system (CMS) as compared to the routine observation of unwanted drug effects by the practising physician. The quality of this approach was compared to prospective intensive and detailed chart analysis which was assumed to yield all unwanted drug effects (ADRs). The role of the well trained and, in pharmacovigilance, experienced pharmacoepidemiological team in the decision making is of importance for the results of the study [4, 5, 12], as the consensus was found by personal discussion.

One finding of this study is that the CMS can detect ADRs early and alerts physicians to respond to changes in laboratory data before clinical symptoms are apparent. The CMS detected about half of all unwanted drug effects, but did not include all of those which are in principle detectable by monitoring laboratory data.

This is explained by the fact that not all parameters available in principle entered the electronic file (e.g. serum glucose concentration) and hence escaped the control of the CMS. Furthermore, no specific set of routine laboratory values was defined and performed for study reasons. Only laboratory data requested by the treating physician were included in the analysis. Thus, the number of blood tests in children is lower than in adults. In a comparable study performed in our department of internal medicine, a total of 39,819 laboratory tests were performed in 460 admissions (86.8 laboratory tests per admission, average length of hospital stay 9.3 days) [4], whereas in the present study only 27,434 tests were performed (62.5 laboratory test per admission).

A certain number of ADRs causing only clinical symptoms (e.g. diarrhoea) were not detected by the CMS. These clinically overt ADRs are not sensitive to the CMS. Confirming the finding of our prior study, physicians detected 42.5% of all ADRs [12]. Combination of CMS and physicians increased the detection rate to 74% leaving only a minority of rather unimportant effects to be discovered by intensive chart analysis. These findings confirm the results described in our previously published study, where the total detection rate compared with intensive chart review was postulated to be 85% [12].

However, the method applied in this study does not allow us to conclude that changes in laboratory test results caused by drug therapy are generally not recognised by the treating physicians. The majority of ADRs alerted by the CMS were “mild”. It could be assumed that physicians did not pay any attention as at the time of occurrence there was no clinical need to change the therapy or to document the event.

On the other hand, considering physicians workload within their daily routine the presentation of weighted laboratory test results in terms of alerts may be an important support to help discriminate critical laboratory test results and monitor patients more intensively.

The case presentations point to the impact the CMS may have on the early detection of ADRs and may help to prevent clinically important symptoms, e.g. severe allergic drug reactions. The system issued an alert on day 6, whereas clinically symptoms occurred 4 days later. In the future, this could allow for early intervention.

A critical evaluation of the data obtained shows that the sensitivity of the CMS in its present form is high (90.3%), but is clearly influenced by the detection rules on which the system is based [4]. More stringent rules will reduce the sensitivity of the CMS. Furthermore, mostly laboratory changes without significant clinical relevance were detected, but they were complementary to side effects that presented clinically.

However, the specificity in our approach was only 19.6%, but is inline with the work published by Dormann et al. [4]. In contrast, the work of Haffner et al. [7] showed a specificity of 72.8%. This may be explained by the fact that we included more routine laboratory data in our analysis.

The DELTA values which have already proved their efficacy in adult patients [4] may also be of value in paediatric patients. The DELTA values are highly sensitive regarding the early detection of, e.g., trends towards leukopenia or allergy. Using the DELTA approach clearly enhances the specificity of the system. On the other hand, using only the DELTA approach the sensitivity is reduced and would demand on taking much more laboratory assays.

However, the result of implementing a CMS based only on laboratory data on a paediatric ward is not convincing. A positive point is that the CMS increases the awareness of physicians for ADRs. It might be helpful in alerting the physician of unwanted side effects prior to the occurrence of clinical symptoms. A serial analysis of relevant trend values (DELTA alerts) may be helpful. But this is associated with a higher proportion of laboratory values to be ordered. It will take further work to improve laboratory limits for different paediatric patients, clinical conditions or even clinical settings. In addition, it would be of interest to compare the quality and quantity of ADRs found by the treating physicians and alerted by CMS based on DELTA values only.

The implementation of additional knowledge bases and the contextual linkage between the automatic laboratory signals and patients’ individual medication data will improve the effectiveness of the system [2, 6, 7, 10, 11]. On the other hand, presenting only signals linked with established medical knowledge prevents taking advantage of the system’s ability to detect new and unknown unwanted drug effects.

The study demonstrates that using CMS is complementary to the observation by the treating physician as different kinds of mild adverse events were detected. The system presented appears to be sufficiently sensitive, but the specificity is too low to make it acceptable for physicians in daily practice. In children, clinically important ADRs can be detected best by intensified surveillance.