Clinician’s capsule

What is known about this topic?

ED crowding is associated with poor outcomes, but there is no consensus what the best metrics are for measuring crowding.

What did this study ask?

Which ED input, throughput and output metrics have the strongest association with patient returns to the ED within 72 h?

What did this study find?

The metrics most strongly associated with 72-h re-visits are ED Waiting Time, ED Occupancy, and Boarding Time for admitted inpatients.

Why does this study matter to clinicians?

Clinicians should monitor waiting times, ED Occupancy, and boarding times to identify input, throughput and output bottlenecks in real time.

Introduction

Emergency department (ED) crowding is “a situation in which the demand for emergency services exceeds the ability of physicians and nurses to provide quality care within an appropriate time [1].” It has become the most pervasive challenge facing emergency care providers and one of the most important issues in health care delivery [2]. ED crowding delays the provision of effective care for high-risk conditions, and is associated with adverse patient outcomes [3,4,5,6,7], including mortality [6, 7].

ED crowding is driven by multiple input, throughput and output factors [8]. Input factors relate to the volume and acuity of arriving patients. Throughput factors encompass ED processes of care, including patient assessment, diagnostics, therapeutic interventions, and consultations. Output factors affect patient flow from the ED, most importantly the boarding of admitted inpatients in the ED. Current evidence suggests output factors, particularly boarding inpatients, are the most important contributor to ED crowding [1, 9]. But, the contribution of input, throughput and output factors may vary over time and within and among settings, and their measurement is essential to evaluating and addressing ED crowding.

One of the difficulties with ED crowding is its measurement. A systematic review identified over 700 different crowding metrics that have been used in prior literature [10]. Many crowding metrics have a demonstrated association with patient and quality outcomes, but the comparative strength of these associations has not been assessed. The objective of this study is to compare the strength of association between commonly used input, throughput and output metrics and a patient’s risk of return to the ED within 72 h of discharge. The findings will identify the ED crowding metrics that are most associated with patient outcomes and thus best suited for quantifying crowding in research, quality improvement and healthcare management.

Methods

Setting and participants

We conducted a database analysis in three urban, adult emergency departments in Calgary, Canada during calendar years 2010–2014. At 1.4 million population, Calgary is the fourth-largest city in Canada [11], and the study hospitals have a combined annual census of approximately 230,000 patient encounters. One site is a regional cardiac, stroke, cancer and trauma centre. The other two are full-service community hospitals. All are staffed by certified emergency physicians and rotating trainees, with approximately 105 daily hours of attending physician coverage per site. These three hospitals were the only adult EDs in the city in continuous operation throughout the entire study period.

We included patients presenting to the three participating EDs during the study period. We excluded those who were transferred from other institutions for specialist consultations, those who were dead on arrival, any patient with an ED wait time greater than 24 h, an MD care time, ED length of stay or boarding time greater than 120 h, and any patients with negative or missing values for crowding metrics. In total, 74 patients were removed from the entire data extract.

Data sources

All hospitals in urban Calgary share a common, linked ED information system (EDIS) and electronic medical record that capture patient demographics, electronic time stamps for patient arrival, transfer to care space, physician assessment, diagnostic and therapeutic interventions, consultations, admission orders, discharge decisions and patient disposition. Internal validation has shown these time stamps are accurate within 3–10 min [12, 13]. Crowding and outcome variables were obtained from these electronic systems.

Outcome

The outcome of interest was all scheduled and unscheduled returns to any of the three EDs within 72 h after an index ED discharge. This is a routinely used ED quality measure that may reflect sub-optimal patient experience, incomplete care or potential diagnostic or management error at the index visit [14,15,16]. The vast majority of 72 h re-visits observed in this setting are unplanned, as there is good access to outpatient care for common problems requiring short-term follow-up.

Crowding metrics

ED crowding metrics included in this analysis were identified in the literature as important [10, 14, 17, 18]. We categorized each as an input, throughput or output metric based on recommendations from the Canadian Agency for Drugs and Technology in Health [10].

Input metrics included ED census, EMS offload delay, ED Wait Time, and left without being seen proportion. Throughput metrics included ED occupancy, MD care time and ED length of stay. Output metrics included emergency inpatient counts, emergency inpatient proportion, boarding time for admitted patients and hospital. The metric definitions are shown in Supplemental Appendix 1.

Analysis

Each calendar day in the study period was divided into three eight-hour shifts (i.e., 0001–0800, 0801–1600 h, 1601–2400 h) yielding 5477 shifts. Count metrics (e.g., ED census) were calculated at the beginning of each shift. Cumulative count variables (e.g., patients who left without being seen during a shift) were summed for each shift. For time-based variables (e.g., ED waiting time) means, medians and 90th percentile values were calculated for each shift. Each shift-aggregated metric was then used as the exposure for patients visiting during the same shift during the modelling described below. This approach created a standardized exposure that reflects the crowding experience of a patient cared for on each shift.

The dataset was subdivided according to each patient’s Canadian Triage and Acuity Scale category: CTAS 1 (resuscitation), CTAS 2 (emergent), CTAS 3 (urgent), CTAS 4 (less urgent), and CTAS 5 (non-urgent) [19]. Our primary analysis was performed on the CTAS 2, 3, 4, and 5 groups combined to quantify the associations between crowding metrics and the outcome among the overall population of ED patients, as this analysis is most relevant to guide ED operational decisions. As a pre-planned secondary analysis, we repeated our analytic approach in each CTAS subgroup as the consequences of ED crowding may be experienced differently by patients with differing acuity of presentation. Analyses were not performed on CTAS 1 encounters (n = 2716), as they represent a small proportion of the overall case load, are infrequently discharged, and input processes for these patients are rarely impacted by crowding conditions.

After using the entire dataset to calculate crowding metrics, we conducted our main analysis on patients whose visit ended in discharge from the ED. We constructed mixed effects logistic regression models with 72-h ED re-visit as the dependent variable. In each model, one of the predefined crowding metrics served as the independent variable (i.e., the exposure), we included a standard set of confounding variables in all models (age, sex, CTAS category, month of the year and weekday vs. weekend encounter), and a random effect was incorporated for each patient (See Supplemental Appendix 2).

We used an information-theoretic approach [20] based on Akaike’s Information Criterion (AIC) [21] to identify the input, throughput and output metrics with the strongest association with the outcome. A lower AIC indicates a better-fitting model. From the AIC, Akaike weights (range 0–1) are then calculated to compare the association between 72-h ED re-visits and the different input, throughput and output metrics. Within each group of input, throughput or output metrics, the metric with the highest Akaike weight was considered to have the strongest association with risk of 72-h ED re-visits.

As a validation step, we used a bootstrapping approach [22] to evaluate the consistency of the AIC and Akaike weights for the input, throughput and output metrics.

Ethics

This study was approved by the University of Calgary Conjoint Health Research Ethics Board with a waiver of informed consent. The funding organization had no input in the conduct or reporting of the study. Patients, hospital staff, and the public were not involved in or aware of this research.

Results

Population characteristics

There were 1,149,939 individual ED patient encounters, including 1,116,326 (97.1%) with complete data for the analyzed variables. These encounters were used to calculate the ED crowding metrics.

There were 848,304 ED encounters that ended in discharge for which the associations between crowding metrics and risk of 72-h return visits were calculated. For all main and secondary analyses, 2,716 CTAS 1 encounters were removed yielding 845,588 encounters: 225,078 CTAS 2, 417,176 CTAS 3, 159,348 CTAS 4, and 43,986 CTAS 5 encounters. Demographic and outcome data for ED visits that ended in discharge are shown in Table 1. Mortality among discharged patients was exceedingly rare at 7 (0.05%) and 30 days (0.19%). Crowding metrics for the entire study period are shown in Table 2. Site characteristics are shown in Appendix 2 Table 1.

Table 1 Characteristics of visits ending in discharge from participating EDs
Full size table
Table 2 Crowding metrics across 5774 shifts used quantify crowding exposure to visits ending in discharge
Full size table

Main results

Among all CTAS 2–5 encounters ending in discharge, most crowding metrics had positive association with 72-h ED re-visits, although not all associations were statistically significant (Fig. 1, Appendix Table 2). The input metric with an adjusted OR > 1 and the greatest Akaike weight was median ED wait time. The throughput metric with an adjusted OR > 1 and the greatest Akaike weight was ED occupancy (ratio of patients to funded care spaces) at the start of a shift. The output metric with an adjusted OR > 1 and the greatest Akaike weight was the median boarding time (Table 3).

Fig. 1
figure 1

Adjusted odds ratio (with bars for 95% confidence intervals) and Akaike weights for 72 h ED re-visits of candidate ED crowding metrics (CTAS 2/3/4/5)

Full size image
Table 3 Adjusted OR, Akaike Information Criterion (AIC) and Akaike weight for ED crowding metrics in predicting the outcome of 72 h ED re-visits among visits ending in discharge (CTAS 2/3/4/5)
Full size table

The internal validation of these findings was completed using 15 bootstrap datasets by sampling patients with replacement. The average number of ED encounters in our 15 bootstrap validation datasets was 844,798 (SD = 1944). Of the input and throughput metrics tested, median ED waiting time and ED occupancy, respectively, had the highest Akaike weights across all 15 bootstrap datasets. Of output metrics, median and mean inpatient boarding time metrics had the highest Akaike weights for 67% and 33% of bootstrap datasets, respectively. These bootstrap results agree with the findings of the real data analysis.

The strongest crowding metrics differed by CTAS category (Supplementary Appendix 2 Figs. 1–4 and Supplementary Appendix 2 Table 3). ED waiting times were the input metrics most strongly associated with 72-h return visits among CTAS 2,3 and 4 patients, but there was no clear superior input metric for CTAS 5 patients. Among throughput metrics, ED occupancy had the strongest association with 72-h ED re-visits for CTAS 2 and 3 patients, but among CTAS 4 and 5 patients, other throughput metrics had stronger associations. The output metric most strongly associated with 72-h ED re-visits was inpatient boarding time, although this relationship varied among CTAS strata and was strongest for CTAS 3, 4 and 5 patients.

Discussion

We sought to identify the most meaningful ED crowding metrics by estimating the association of each metric with the risk of a 72-h ED re-visit, a clinically important patient-focused outcome. Among input metrics, median ED waiting time had the strongest association with 72-h ED return visits. Among throughput metrics, ED Occupancy (the ratio of registered patients to funded beds), had the strongest association with 72-h re-visits. Among output metrics, the median admitted patient boarding time (time from admission to patient transfer to inpatient unit) had the strongest association with 72-h ED re-visits. The relationship between crowding metrics to the outcome was generally consistent across CTAS levels, confirming these indicators as the preferred input, throughput and output metrics.

Comparison to prior literature

Our findings are consistent with previous literature, showing important associations between selected ED crowding metrics and the risk of adverse patient outcomes. Crowding has documented adverse impacts on the provision of time-sensitive therapies and on mortality for ED patients [3,4,5,6,7]. Many indicators have been used in the literature to quantify ED crowding [10], yet there is no consensus on the preferred crowding metrics. This is the first study to directly compare crowding metrics to provide evidence as to the superior ED crowding metrics for studying and attempting to remedy ED crowding.

Strengths and limitations

This study was performed in an integrated single-payer health system. This facilitated a population-based analysis that captured all ED encounters and all outcome events in the study region. Patients may have re-presented at EDs outside Calgary; however, that number would be small and unlikely to have influenced the results.

Crowding conditions in the study EDs were similar to other Canadian university-affiliated hospitals but more severe than many non-academic or community hospitals [23]. Median ED waiting times and ED length of stay were longer than observed in United States EDs [24]. Our data encompass the full spectrum of crowding severity experienced in large EDs during a five-year study period. The large sample size across a range of crowding conditions allowed us to identify significant associations between crowding metrics and 72-h re-visits with relative precision.

Our primary outcome, 72-h re-visit rate, is an indicator of incomplete or sub-optimal care [14,15,16], but we could not distinguish between planned and unplanned return ED visits. Most 72-h re-visits in this population are unplanned. Over 85% of residents in Calgary have a regular health care provider [25]. The three hospitals have 24-h computed tomography and 18-h ultrasound imaging availability, and refer any normal next-day imaging results to primary care follow-up. The Calgary Health Zone uses dedicated ED follow-up clinics rather than planned return visits for common problems requiring short-term follow-up (e.g., IV antibiotics). ED return visits within 72 h resulting in admissions and mortality were too infrequent to provide adequate precision as an outcome measure. ED re-visits within 72 h remain accepted as a quality indicator by participants in the ED quality benchmarking summit [18], and we believe this outcome to be a valid quality indicator.

We did not examine composite ED crowding scores. Because these scores combine input, throughput and output metrics, they are not able to identify the component causes of a crowded ED, and are of limited value for identifying and addressing bottlenecks in real-time.

Clinical implications

When monitoring ED crowding in their own department while on shift, clinicians should pay attention to the waiting time, ED occupancy and boarding times to identify bottlenecks that may be addressed in real-time by nimble process and practice changes.

Worsening ED crowding is associated with a small increased risk of an ED re-visit for a discharged patient, but may lead to substantial operational impacts for a busy ED. In our EDs, a patient’s baseline risk of a 72-h ED re-visit is 9.1%. Using the odds ratio for ED waiting time of 1.044, we would predict that a 1-h increase in median waiting time would increase a patient’s risk of return visit to 9.5%. Extrapolating that individual risk to an ED that discharges 60,000 patients/year, a sustained 1-h increase in median boarding times would predict 240 additional ED re-visits per year. A sustained 1-h increase in median inpatient boarding time would increase an individual patient’s risk of 72-h re-visit from 9.1 to 9.3%, but would predict an additional 120 ED re-visits per year for an ED that discharges 60,000 patients/year.

Research implications

We have identified the most meaningful crowding metrics for use in ED crowding research, quality improvement and for the formulation and evaluation of health policy. Median ED waiting time is best suited for measuring impact of ED input processes, ED Occupancy is best suited for measuring impact of throughput processes, and inpatient boarding times are the preferred metrics for measuring impact of output processes.

Conclusion

ED input, throughput and output metrics have important associations with 72-h re-visits, indicating that crowding-related delays during any phase of ED care lead to adverse patient outcomes. Median ED waiting time, ED occupancy and inpatient boarding times should be the preferred measures for clinicians to monitor crowding in real time and for use in research and quality improvement work.