Background

The investigation of adverse events (AEs) and the related activities for preventing risks and applying safer solutions can significantly contribute to the reduction of the incidence of patient harm and promotes safer health care [1,2,3,4,5,6,7,8,9,10]. In Italy, the rate of AEs is comparable with other countries, with an incidence of 5.2% and a preventability of 56.7% [11].

Organizational characteristics seem to have an important role in the incidence of AEs. In a study carried out in a large Italian hospital [12], the incidence of AEs is demonstrated to be associated with length of stay, type of admission and unit of discharge independently of comorbidity.

For the importance of organizational matters in epidemiology of AEs, some studies attempt to correlate the patient safety indicators or the outcomes of the health-care system with the characteristics of the hospitals.

Thomas and colleagues [13] find that the quality of inpatient medical care varies by type of hospital ownership: patients in minor or non-teaching government hospitals are more likely to suffer several types of preventable AEs when compared with patients admitted in nonprofit or major teaching ones.

It has also been shown that the process of care is better in teaching hospitals than in non-teaching ones [14]. However other studies have failed to show a consistent association between the structural characteristics of hospitals and patient safety [15,16,17].

Therefore, we performed a retrospective study to investigate the association between incidence, preventability and consequences of AEs and the structure of public hospitals in the Tuscany Regional Healthcare Service, characterized by homogeneous political and socioeconomic context.

Methods

Study design and settings

The study is a multicenter, retrospective study. It involves 4 teaching hospitals (THs) and 32 community hospitals (rural or urban) of the Tuscany Regional Healthcare Service.

The hospitals are managed by 16 trusts: the 4 teaching hospitals belong each to a specific trust, and the 32 community hospitals are distributed in 12 local trusts (LTs), which are also in charge of the primary care services within their own district. The community hospitals provide general services without high specializations (e.g., cardiac surgery, transplants, neurosurgery).

The objective of the study is to analyze the incidence, preventability and consequences of AEs in hospitals characterized by different complexity (LTs being less complex than THs).

A random sample of the clinical records of patients admitted to LTs and THs in 2008 was selected from the database of the hospital discharge records of the centers [18, 19]. Data of inpatients of any age admitted for more than 24 h in the hospitals were included. In-hospital admissions coded as “one-day surgery” or “one-day hospital” were excluded.

The method used for analysis consisted of a two-stage retrospective record review (Review Form 1—RF1 and Review Form 2—RF2) of the representative random sample. Clinical records were analyzed with the review tools elaborated by Charles Vincent and colleagues [20]. The method is based on a double stage process.

The first stage of the review process (form RF1) includes the analysis of the representative random sample, 10% of which was double-blindly analyzed. To identify adverse events or complications, we included any records in which at least one of the predefined screening criteria was present.

Records that screened positive were then reviewed in the second stage of the review process (RF2) to identify only adverse events.

The RF2 consisted of the double-blinded review of all selected clinical records with a form addressed to analyze the presence of possible adverse events and their analysis with the Charles Vincent Model [ibidem]; disagreements in terms of identification of an AE, by the two reviewers, were solved by expert opinion (the clinical risk manager of that hospital). The screening criteria for RF1 and the stages of the analysis for RF2 are described in Box 1.

Box 1 Description of Review Form 1 (RF1) and Review Form 2 (rf2) as review tools
Full size table

Adverse events are defined as an unintended and unexpected incident, included in the case record and caused by health-care management rather than a patient’s baseline illness, which results in an injury, disability, extended hospital stay or death [1].

The reviewers for each center were selected according to the profile suggested in previous studies, to control possible bias. For the RF1, seven medical doctors, a nurse, a pharmacist and a biologist with competencies in clinical risk management and analysis of patient records were selected. For the RF2, the reviewers were all physicians. The reviewers were trained with a specific 2-day program.

A concordance evaluation among reviewers was calculated by comparing their opinion on RF1 (in the 10% of cases double-blindly analyzed) and RF2 for each evaluated clinical record.

The reviewers assessed the contributory factors, the preventability and the consequences (i.e., disability, prolongation of the stay and death) of each adverse event. The concordance among reviewers was elaborated centrally by the authors of the study.

Sample size

On the basis of the results of other international studies (incidence of AE 8–12%), the dimension of the study is defined according to an estimate of incidence rate (P 0) that varies between the 5 and 10%. The precision of the requested estimation varies between 1 and 3%, taking into account that the percentage of clinical records passed to the second stage of review is supposed to be around 20%.

Assuming a confidence interval of 95% for P 0, of a scale of d = 0.01, where P 0 is assumed to be equal to 0.05, the final sample dimension was identified in 11,409 clinical records.

Thus, for each center, the number of clinical records to be analyzed is 576, except for the two major teaching hospitals (TH A and TH C) which is 1800, as they also participated in the national study [11]. Since 10% of the clinical records were not considered because of the lack of basic information, the two major teaching hospitals selected 200 additional clinical records, while the other centers selected 50 additional clinical records. Through the definition of a list of casual numbers, a sample of 12,764 patient records was selected. An estimation of the risk can be defined by weighing the percentage of AE of each center with their volume of discharges. The number of the included records looked adequate considering the time necessary for the double review, the available financial resources and the organizational/logistics constraints. The dimension of this study is also consistent with the international studies cited in the review by de Vries et al. [21].

Due to the high variability of the estimation of the incidence of AE, an evaluation of the power of the study was calculated ex post, starting from the level of precision defined in the calculation of the interval of confidence of 95% [22]. A base of 11,409 records allowed an investigation with a power major of 90% with differences between the proportion under null hypothesis (P 0 = 0.05) and the one with alternative hypothesis (P 1) of 1.5%, that is, between 0.0425 and 0.0575.

Statistical analysis

Clinical records included in the two review phases of the study are described using frequency tables. Qualitative variables such as the type of hospital, gender and RF1 and RF2 criteria are described using absolute and relative frequencies (percentage). Quantitative variables such as age are described with mean and standard deviation.

The rate of adverse events was calculated by dividing the number of records with adverse events identified in the RF2 phase by the total number of records reviewed in the RF1 phase. The rate of AEs and relative IC 95% (exact confidence interval for binomial variables) were calculated for all hospitals included in the study.

A multivariate analysis on difference in rate of AE between type of hospital (local vs teaching hospitals) adjusted for gender and age was evaluated by performing Poisson regression models. Crude and adjusted incidence rate ratio (IRR) and relative 95% confidence intervals were reported.

The characteristics of adverse events are described in the frequency tables. Appropriate statistical tests were used to evaluate the eventual statistical difference between the local and teaching hospitals. p values less than 0.05 were considered to be statistically significant.

The association between preventability and consequences of AEs and the RF1 screening criteria were evaluated using multivariate logistic regression models (OR and 95% IC). Separate models (stratified analysis) were estimated for local and teaching hospitals. Gender and age variables were included in the models.

The intra-center level of concordance was evaluated by the percentage of agreement between the two reviewers in both phases of the review process. All analysis was generated with STATA software, version 11 (STATA Corp, College Station, TX, USA).

Results

Among the original random sample of 11,409 clinical records, 116 were discarded because of incomplete data. Of the remaining 11293, 1039 clinical records were positive according to the screening criteria. A total of 354 cases of patients who sustained adverse events were identified and analyzed in RF2. Some patients incurred more than one adverse event, so that the adverse events were in total 423 (see Fig. 1).

Fig. 1
figure 1

Flowchart of the procedure for clinical record selection by Review Form 1 (RF1) and Review Form 2 (RF2)

Full size image

As shown in Tables 1 and 2, among the 11,293 patients included in the analysis, 45.9% were male and the median age was 52.3 years. Table 1 shows a similar number of hospitalizations for males and females, although there is a greater number of males selected for RF1 and adverse events. The diagnosis-related group (DRG) weight almost doubled for the cases with adverse events.

Table 1 Distribution of hospital admission in 2008, Review Form 1 and Review Form 2 records and patients with adverse events based on age, gender and DRG (diagnosis-related group)
Full size table
Table 2 Percentage of records with adverse event and confidence interval (95%)
Full size table

The incidence of AEs is higher in the THs (5.3, 95% CI 4.7–6.1) than in the LTs (1.8, 95% CI 1.5–2.2) and in medical units (40.7%) (see Fig. 2; Tables 3, 4). This difference is connected with a higher DRG weight in THs with the exception of the pediatric teaching hospital (TH2) characterized by a lower complexity of cases.

Fig. 2
figure 2

Territorial variability of the rate of adverse events and confidence interval (95%)

Full size image
Table 3 Crude and adjusted incidence rate ratio of AEs. Poisson univariate and multivariate estimation of incidence rate ratio and 95% confidence interval
Full size table
Table 4 Consequence, Specialties, preventability and contributory factors of detected record of AEs stratified by kind of hospitals
Full size table

There was a significant higher incidence of AEs in males and in the elderly population (age >60 years), AE 4.8 (4.0–5.7); gender male 4.5 (3.9–5.1) vs female 3.1 (2.7–3.5) as well as the DRG.

As shown in Table 4, there was a significant higher number of preventable AEs in LTs compared to THs (56.3 vs 46.5%; p value <0.01), considering the items on the scale from certain to a bit likely. Also, death as a consequence of AE was found more in the LTs than in THs (14.9 vs 7.6%; p value = 0.020). Except for the characteristics of the patients, all the contributory factors seemed to influence in different ways the incidence of AEs in the two types of hospitals. The contributory factors are error-producing conditions and organizational factors affecting the clinical practice [18]. The organization has a major impact on the THs compared to the LTs (65.4 vs 50.0%, p = 0.03), while individual factors (46.9 vs 32.1%, p = 0.03), characteristics of the workplace (22.7 vs 9.3%, p = 0.006) and management affect the LHs more (30.3 vs 15.0%, p = 0.007).

Tables 5 and 6 describe the relationship between the RF1 screening criteria and the characteristics of the AEs in terms of preventability and consequences. Falls, re-interventions and surgical complications have a significant higher probability to be related to an AE than other criteria of RF1.

Table 5 Multivariate logistic regression analysis on preventability and prolongation of adverse events (N = 423) and the presence of screening criteria in Review Form 1 review phase
Full size table
Table 6 Multivariate logistic regression analysis on the disability and death of adverse events (N = 423) and the presence of screening criteria in the RF1 review phase
Full size table

Death is significantly more probable for RF1 criteria related to unexpected cardiac arrest [OR 73.29 (21.99–244.28) LTs, and OR 22.61 (6.65–76.83) THs]. Disability and prolongation of the stay are significantly more probable consequences associated with re-admission in THs (OR 3.3 disability and OR 2.93 prolongation) and to transfer to ICU or HDU in LTs (OR 3.29 disability and OR 4.33 prolongation).

Moreover, falls seem to be more preventable in THs (OR 19.22, 95%, CI 2.45–151.02), while infections are the most preventable ones in LTs (OR 6.22, 95%, CI 1.35–28.67) though the data are not statistically significant. Concerning the consequence of the AE in LTs, death was significantly a more probable consequence associate with unexpected cardiac arrest, while disability and prolongation of the stay are significantly more probable consequences associated with re-admission in THs and to transfer to ICU or HDU in LTs.

The concordance between the two reviewers of RF1 for each clinical record is on average very high (higher than 93% for all the centers except for two (LT F 71% and LT H 75%). Concerning RF2, the concordance between the two reviewers is on average 82.53% with a minimum of 34.7% (LT A) and a maximum of 100% (LT E). Concerning the concordance among centers, the result is 61% in the identification of at least one AE.

Discussion

The study finds a higher incidence of adverse events in the THs compared to the LHs. This is related to a difference in the DRG weight of the treated clinical cases. In fact, the THs are the centers of third level that centralize all the sickest and complex patients, according to the hub and spoke organizational model adopted in the regional health-care service.

Concerning the differences between THs and LHs in the consequences of adverse events, the higher rate in death for the LHs might be related to the presence of some risk factors like older age of patients and the concentration of chronic patients admitted in the spoke units. Instead, there are a significant higher number of cases in THs where the adverse event provokes a prolongation of stay for the patient. In these types of hospitals, the risk of suffering a prolongation of stay or disability after an adverse event is higher when a patient is re-admitted. The data about prolongation might be related to the major complexity of the treated pathologies and consequently a higher number of examinations and consultations for the patient.

With respect to the profiles for THs and LTs, there are important differences in many characteristics of the adverse events. There are a significant higher number of preventable AEs in LTs compared to THs, which might be associated with the different DRG weights of the clinical cases. In the THs with the most difficult cases and their associated complications, it is more challenging to intercept an adverse event, while in LTs the general lower level of experience in managing clinical or therapeutic complications, or in identifying difficult diagnoses in a timely fashion, can additively provoke an adverse event.

The significant number of preventable unexpected cardiac arrests in LTs is an example of how in those small hospitals, it is more difficult to manage complex and unstable patients. In LTs, there are also a significant higher number of deaths following cardiac arrests, compared to the THs. In THs, the higher number of AE related to re-admissions is because patients are more complex, more unstable and more keen to be re-hospitalized.

In relation to the contributory factors for AEs, the organization has a major impact on the THs compared to the LTs, while individual factors, characteristics of the workplace and management have a greater effect on the LHs. This is consistent with the different complexities in the organization and level of assistance in these two types of hospitals.

Concerning the THs, the results are very similar to other international studies in terms of rates, occurrence and preventability of AEs obtained in other countries [22].

Falls, re-interventions and surgical complications have a significantly higher probability to be related to an AE than other screening criteria. This might be related to the fact that, analyzing clinical records, it is easier to identify these factors than others such as infections or neurological deficits. After the study, we planned and realized many different interventions aimed at addressing the different types of adverse events; in particular, we elaborated a regional system of patient safety practices, in accordance with the international guidelines and proposed solutions by the AHRQ and WHO campaigns.

There are no significant differences in the distribution of AE among the specialties.

The variability of results might have been influenced by the following bias [11]:

  • the different interpretations of the definition of AEs by the reviewers;

  • the different levels of experience and competence of the reviewers;

  • the random sample of clinical records that refers to populations of patients characterized by a different exposure to the risk of AE, the different commitment to patient safety of the hospital management and the different patient safety culture among clinicians.

All the methods for the measurement of AEs present limitations and strengths. The review of clinical records represents one of the most reliable methods for the investigation of AE [23]. The variability of the reviewers’ opinion can be limited through the use of a standardized review form, such as the one adopted in this study. The concordance between reviewers was on average very high. The sample selection used in this study is simple and common in this kind of research [1,2,3,4,5,6,7,8,9,10]. 10% of the clinical records that were not considered because of the lack of basic information did not affect the study results, as we added an adequate amount of clinical records to the sample. For future investigations, more complex procedures can be used, especially for the initial selection of the clinical records to be screened. Thus, some results, such as the correlation between AE, age and gender, suggest the opportunity of introducing a stratification of the sample in the study design. The lack of cases with some measure of complexity among hospitals can be considered as a limitation for this and other similar studies. Implications for the theory are to design and run specific qualitative studies in the different contexts in order to understand causes underneath the identified differences in the dynamics of AEs. Implications for the practice are the need for socio-technical interventions in order to improve the management of complex cases and emergencies. It would be useful to work on technical and non-technical skills with the use of simulation and on planning a major level of cooperation between clinicians working in different types of hospitals to better manage the complex cases in the community hospitals.