Introduction

Adverse events (AE) in medicine have been studied for several decades, but only recently have they begun to be used as a guide to improve quality. The first studies of AE focused on specific undesirable situations but did not have long-term repercussions [1, 2, 3]. It was not until the mid 1970s that the California Medical Association decided to analyze the importance of AE in a study that reviewed the histories of 20,864 admissions. The review reported a rate of adverse events of 4.6% and evidence of negligence in 0.8% of cases [4].

The first study of surgical AE was performed by Couch et al. [5], who found a rate of avoidable surgical errors of 0.6% in admissions to their university general surgery service. They reported that 55% of these complications led to the death of the patient.

The Harvard Medical Practice Study drew attention to the problem of AE and has become the study of reference in the field; it was a retrospective analysis of 30,121 randomly selected hospital records from in 51 hospitals in New York State in 1984 [6].

Gawande and coworkers [7] found an AE rate of 3% in surgical patients (including births), of which 54% were considered to be preventable. Of these events, 5.6% resulted in the death of the patient. More recently, Calland and coworkers [8] found that 19.3% of deaths in the surgery services at a university tertiary hospital were attributable not to the patient’s primary disease, but to an AE related to the medical intervention. These authors detected errors in 0.24% of the population studied. In a prospective study of 192 general surgical patients in 2000, Wanzel and coworkers [9] found adverse events in 39%, and considered that 18% of the complications were potentially attributable to error.

Finally, Healey and coworkers [10] performed a prospective study to determine the exact percentage of complications in four different specialist services (general surgery, vascular surgery, combined general surgery, and trauma and cardiothoracic surgery) and the proportion of these complications attributable to provider errors. They found complications in 17% of general surgery patients, with a mortality rate of 1.83% and a percentage of avoidable death of 28%. These percentages of complications studied prospectively were between two and four times higher than those identified in the retrospective studies by the Institute of Medicine (IOM). Their data suggest that errors in care contribute to 30% of the mortality.

However, the current literature on adverse events and errors has several shortcomings. Because of the differences in the definitions and methods used to compile data, the measurement and monitoring of events often is imprecise [11]. Furthermore, few studies have controlled the sequelae of these AE. Even more serious, in our view, is the fact that adverse events occurring after patient discharge are not recorded.

This prospective study in a general surgery service was designed to determine the true percentage of adverse events, errors, and sequelae, during the period of 30 days after patient discharge, and to show that continuous collection of data and communication of the results to the health care teams involved will improve the quality of care provided.

Materials and methods

Setting

The study was conducted at a level IIa teaching hospital that is part of the Xarxa Sanitària d’Hospitals d’Utilització Pública of Catalonia (Spain) and has a reference population of 401,204. It has 648 beds, and 19,824 admissions in 2005. In 2005, 24,598 interventions were performed: 8,070 conventional surgeries, 8,768 major outpatient surgeries, and 7,760 minor surgeries. The Hospital’s General Surgery Service sees 2,254 patients annually—1,124 on an outpatient basis—with a case-mix of 1.37, 1,994 interventions are performed annually, and 1,117 ambulatories surgeries.

Staff involved in the study

All the staff at the Surgery Service and all the residents and nursing staff in the surgery wards and operating rooms took part in the study. In the case of patients in the Intensive Care Unit, ICU doctors and nursing staff also participated: in all, more than 20 specialists and 6 residents.

Patients

All patients admitted to the Surgery Service between 15 January 2005 and 30 October 2006 were consecutively included in the study. No hospitalized patients were excluded, but those who underwent minor surgery or major ambulatory surgery were omitted from the study. Minor surgery includes all dermatologic surgery and all ganglionar biopsy. Major ambulatory surgery includes hemorrhoidectomy, groin hernia surgery, and some cases of breast tumorectomy, if the patients accomplished the including criteria for this surgery. The remaining surgeries are included in “conventional surgery.” Patients admitted from the emergency room were included in the study, but those only seen in the emergency room and discharged from there were excluded. Patients referred from another hospital were included in the study and all their AE were recorded from their time of admission. Patients referred to another hospital also were included and all their AE were recorded until 30 days after discharge.

Definitions

Adverse event [6, 12]: unexpected consequence or lesion caused to the patient as a result of treatment rather than the underlying illness. Preventable adverse event: adverse event or event attributable to error [13]. Error of assistance: error produced by mistakes in the planning or execution of diagnosis and treatment.

Study variables

During data compilation, all adverse situations affecting the patient were recorded. The appearance of an AE was assessed by using the six-point scale previously defined by the Harvard Medical Practice Study group.

Classification of adverse event [6]:

  1. (0)

    Minimal or absence of evidence that the AE was caused by the care provided

  2. (1)

    Scarce evidence

  3. (2)

    Unlikely

  4. (3)

    Quite likely

  5. (4)

    Very likely

  6. (5)

    The adverse effect is clearly due to care provided

Classification of sequelae [14]:

  1. (0)

    No sequelae

  2. (1)

    Minimal: Recovery period less than 1 month

  3. (2)

    Moderate: Recovery period between 1 month and 6 months

  4. (3)

    Moderate: Recovery period longer than 6 months

  5. (4)

    Permanent (<50% disability)

  6. (5)

    Permanent (≥50% disability)

  7. (6)

    Death

Classification of error: Medical errors were classified according to the clinical description of the AE and its degree of preventability [15].

  1. (0)

    No medical error

  2. (1)

    Slight medical error

  3. (2)

    Moderate medical error

  4. (3)

    Serious medical error

Method

We designed a password-protected database in ACCESS, which could be consulted by all surgeons on any of the hospital’s computers, including those in the Outpatient and Emergency Units. The design complied with the provisions of the Parliamentary Act of 15/1999, 13 December concerning Personal Data Protection.

Each clinical situation that might be considered an AE was introduced into the database by the person who identified it. At the end of the day, a peer reviewer not directly involved in patient handling assessed whether the event in question was an AE, and, if it was, evaluated the sequelae and the presence of error. The peer reviewer was another member of the Surgery Service.

An event was considered an AE when the mean score of the two reviewers was ≥3, as established in the reference studies. Therefore, situations assessed as minimally likely or unlikely were not included in the study. If there was a major disagreement between the scores of the two reviewers (of 2 points or more), the opinion of a third reviewer was sought.

Once the presence of an AE was established, the associated damage and the degree of disability produced unrelated to the primary disease were evaluated. The method used to assess these sequelae was the same as for the AE. Finally, again using the same method, we evaluated the presence or absence of error.

As a quality control measure, ten clinical histories were randomly selected per week and were reviewed retrospectively to compare and contrast the data compiled in the database during the first 3 months of the study. This review was performed by a specially trained member of the service. Any questions that arose were discussed with the teams involved.

When the research team considered that all the staff were adequately trained, the recording of AE at the Outpatient Service began, on June 1, 2005. The end point for recording an AE was placed at 30 days after discharge.

Finally, monthly and quarterly results from the Service and from each Unit were presented at morbidity and mortality rounds. In this way, the Service received continuous feedback throughout the study.

Statistics

The results are presented as means, medians, and 95% confidence intervals, or as percentages and 95% confidence intervals. Percentages were compared over time using the Mantel-Haenszel test. The concordance was calculated using the weighted Kappa test (quadratic weighting). An alpha of 0.05 was used.

Results

Data were compiled for 3,807 consecutive patients. A total of 28,933 patient-days were analyzed. The mean age of the patients was 60.5 years. We detected 2,193 AE in 1,177 patients (30.9%). Patients with AE had a significantly higher mean age (64.8 years) than those without (58 years): 6.8 years more (95% confidence interval (CI) of the difference, 3.7–9.9 years; P < 0.001). This trend was found in all ages (Table 1). The Mantel-Haenszel test (P < 0.001) showed that the difference was statistically significant. Mean hospital stay also was much longer in patients with AE (13.4 days) than in those without (5 days): a difference of 8.4 (95% CI, 7.8–9) days (P < 0.001).

Table 1 Percentage of patients with adverse events
Full size table

The most frequent adverse events are shown in Table 2, grouped in broad fields. A total of 330 AE caused by errors were recorded, affecting 258 patients (6.9%). Generating the indicator nº of avoidable errors/nº of admissions we obtain 8.7%. Of the adverse events detected, 15.1% were the result of an error during care. The ten adverse events most frequently associated with errors are shown in Table 3. There were no statistically significant differences between the mean age of patients who suffered errors and those who did not (64.3 vs. 64.9 years; P = 0.115).

Table 2 Most frequent adverse events
Full size table
Table 3 Ten adverse events most frequently associated with error
Full size table

Patients who suffered at least one error had a higher number of AE (3.1) per admission. Patients without errors had fewer AE (1.6 per admission). This difference is statistically significant (95% CI, 1.1–1.8; P < 0.001). Thirty-four deaths due to adverse events were detected, representing 0.89% of admissions. In 11 cases mortality was considered to have been avoidable (0.29% of admissions). Over time, the percentages of patients with adverse events remained constant (Table 4).

Table 4 Percentage of patients affected over time
Full size table

From June 2005 onward, when it was considered that all the staff were sufficiently well trained, the data entered were complete, and the feedback to the teams involved was reliable, the percentages of patients with errors decreased significantly (P = 0.005; Table 5).

Table 5 Patients affected over time (only valuable months)
Full size table

The sequelae detected (after discussing disagreements) were: none in 32 cases; minimal in 1,709; moderate in 390; permanent in 20: death in 34. This information was lost in eight cases.

The concordance in the evaluation of AE was high, with a weighted kappa index of 0.98. Table 6 shows the results of the two reviewers’ evaluations of possible AE. Agreement in the evaluation of the sequelae also was high, with a weighted kappa index of 0.92 (Table 7), as was the agreement in the evaluation of the error (weighted kappa index of 0.91). There was a slight tendency on the part of the first reviewer (who was directly responsible for the patient, and who entered the data) to evaluate a specific AE as a non-error, whereas the second reviewer (not directly responsible for patient care) deemed it to be an error. This was the case in 38 evaluations. In 19 cases, the opposite trend was found: the first reviewer (responsible for the patient) considered an error to be present and the second reviewer did not. The kappa index decreased to 0.89 if the error variable was evaluated with a simple yes/no answer (Table 8).

Table 6 Concordance of adverse events
Full size table
Table 7 Concordance of sequelae
Full size table
Table 8 Concordance of error
Full size table

Discussion

Many years ago, Karl Popper said, “We make progress if, and only if, we are prepared to learn from our mistakes.” Today there is no question that surgeons must monitor the adverse events of their actions and must analyze their errors to be able to design mechanisms that minimize them. Thorough, continuous control with regular feedback on the results to the care teams forms the basis of an efficient monitoring system. Our goal was to design a system to promote a culture of improving care at our service and serve as a starting point for more ambitious programs for error detection. As Helmrich says [16], “To err is human; we favor a system which acknowledges that human error will inevitably occur but which incorporates mechanisms that protect both patients and staff.”

Our study is the first part of a project for the continuous improvement of quality of care. After almost 2 years, we have managed to actively involve all the members of the service. Our results represent the longest prospective longitudinal study published to date, with 3,807 patients and 28,933 patient/days controlled.

In previous research, prospective studies report a higher percentage of AE than the retrospective studies and reviews of charts or data from the CMBDAH (the Basic Data Set of Hospital Discharges in Catalonia). Table 9 compares data from previous studies and our own data. In our view, prospective recording of data will always provide higher percentages of AE, especially if the staff are persuaded not to conceal data. We agree entirely with Calland et al. [17] and Wanzel et al. [9] that only prospective studies can guarantee accurate descriptions of AE and, above all, define the cause and effect relationship between an AE and the death of a patient. We are surprised by the results of the study by Marang-van de Mheen et al. [18], which concluded that prospective recording of AE provides no benefit. Our study was not designed to compare prospective and retrospective methods, but the fact that we detected situations, such as skin lesions caused by sticking plaster which were not recorded in the clinical history, corroborates our belief that no retrospective method can improve on prospective recording. We are not alone in this belief: a comparative study of recording methods in the Intensive Care Unit [19] also concluded that only routine prospective recording is able to improve quality of care. We believe that this system works better than the classic M&M round system, in which a large proportion of complications may remain unreported [20]. In any case, we believe that a combination of M&M rounds and the presentation of the adverse events recorded since the last round notably improve this system, both from the educational perspective and in terms of the care provided. At least one study has described a system in which the results of strict monitoring are reported in the rounds [21]: the authors concluded that, in addition to its educational value, this approach will help to improve the quality of care. Our results confirm their conclusions.

Table 9 Previous studies of adverse events
Full size table

Our percentage of AE attributable to error is lower than that recorded in other prospective studies. To an extent, this is because many more AE of lesser importance and with minimal sequelae were recorded in our study, most concerned problems with vein catheters. Nonetheless, because we were evaluating our own practices, we cannot be sure that we were 100% objective in our assessment of errors.

In any case, deaths associated with medical management accounted for 0.32% of admissions. Extrapolation of these data gives an estimate of 14,701 annual deaths (95% CI, between 6,431 and 23,889) in Spain alone due to management errors (using data for hospital discharges in Spain in 2003: 4,594,143).

As in other studies [29, 31], we found more AE in older patients. However, age did not influence the presentation of AE. There is some logic to this: a management error (for example, injecting the wrong amount of serum) has nothing to do with the patient’s age, but an AE may depend on the patient’s intrinsic characteristics (e.g., phlebitis is more likely in an elderly person than an 18-year-old). In addition, the higher the number of AE, the more likely it is that errors will occur.

Our patients with AE were hospitalized for longer than those without AE. Unfortunately, our study design does not let us to calculate how many longer hospitalizations are directly related to the AE. It is not surprising that patients admitted for longer are more likely to have AE, and obviously in patients with AE hospital stay is longer. Other studies have presented conclusive evidence in this regard [29], calculating additional hospital stay in the case of AE to be a mean of 9 days and a median of 4 days. It is likely that these three variables—hospital stay, adverse events, and management errors—are closely related, but our design does not allow us to draw conclusions regarding cause and effect between these variables.

The Hawthorne effect was described during the 1920s in a study performed at a factory in Chicago [32]. The effect refers to the phenomenon that when people know that they are being observed in a study their performance changes, thus bias is introduced in the results. We attribute the reduction in the percentage of AE due to errors over time (from 11.1% to 4.5%) to the Hawthorne effect. At least one other study in the literature has described a similar trend in an emergency room [33] and one more in surgery [21]. All members of our service know that all AE are detected and analyzed and that each month these results are reported at the M&M round. In our routine daily practices, we were particularly careful not to make mistakes, or to minimize them; we do not know whether the Hawthorne effect will continue to exert its positive influence over time.

Clearly, our study has limitations. We can never be entirely sure that data have been faithfully recorded. However, thanks to the controls during the first 3 months when clinical records were reviewed randomly and any anomalies were discussed with the team responsible, and above all the fact that the results were presented at our monthly sessions for each unit, we managed to establish an atmosphere of transparency in which all members accepted that the goal of the system was not to hand out punishments but to achieve continuous improvement in quality.

One of the strong points of the study is the control of AE at the Outpatient Service. Most studies do not control AE after discharge; the few that do note that the percentage is high [25]. A limitation of our study is that we did not use specially trained personnel to evaluate AE, nor did we record late surgical sequelae. This was an issue that the reviewers discussed at length, due to the lack of agreement on many occasions. However, our goal was to improve intrahospital practices and we feel that the assessment of late surgical sequelae would correspond to a study of a different kind.

In this study we did not try to analyze the existence of negligence. We do not share the view that all avoidable AE are due to negligence or error: some AE are avoidable, but do not constitute errors at the moment of their detection. Before judging an error, the current situation in each service must be audited, and this is the goal of this research. We support the system approach to error rather than the person approach [16, 34], and although negligent attitudes may exist, they are found only in a minority of professionals and should not be used as a reference.

Finally, we believe that our study opens up some interesting new lines of research. We are sure that the Hawthorne effect will not be sufficient in the coming months to ensure a continuation of the downward trend in error-induced AE at our service. It is not enough to record adverse events, however thoroughly and efficiently the system is implemented. Our patients expect the design of mechanisms to reduce the percentage of errors, and therefore the design of studies of this kind is a priority. Ensuring high quality of care must be our goal in the future, and a key aspect of this process is identifying areas in which we currently fall behind target.