Introduction

The role of anesthesiologists continues to expand throughout the healthcare sector, and it is particularly true for anesthesia cases performed outside of the operating room (OR) [1, 2]. Working closely with nurse management and surgical leaderships, anesthesiologists have been integral to management of the OR environment, a high value resource of any major hospital [3]. In the United States, more than 60% of hospitalized patients undergo some kind of procedure during their stay, which can account for more than 40% of hospital revenues [4,5,6]. Understandably, studies on OR efficiency have identified a number of benchmarks of “well-run” ORs. These include low case cancellation rates, low turnover times (TOTs), high rates of first-case on time starts, and scheduling accuracy [7].

While these benchmarks are well-established for OR environments, there are fewer studies on non-operating room anesthesia (NORA) metrics. In an era where the NORA workload continues to expand, accurate case time estimations and utilization rates are needed to optimize the allocation of anesthesia resources [8,9,10]. Further, the delivery of anesthesia services in the NORA setting can present numerous additional challenges. Some NORA patients may be less medically optimized compared to the general OR population, resources can be much more spread out geographically, and emergent procedures are performed in less suitable environments [11]. Taken together, there may be an assumption that the NORA setting is “less efficient”, but the question is to what extent? Ideally, there would be established and agreed upon efficiency metrics specific for NORA cases.

The standards established in the realm OR studies may not necessarily apply to ongoing investigations of NORA efficiency metrics. Here, we present data on efficiency metrics in a NORA setting compared to metrics used in traditional OR settings. We concentrated on four commonly used efficiency metrics in both settings: turnover time, case cancellation rate, first case start delay, and scheduling error. We hypothesize that there are significant differences in these commonly used benchmarks.

Methods

Data source and analysis

Data were obtained for all anesthetics performed by the Department of Anesthesiology at the University of Vermont Medical Center (UVMMC) from January 1, 2015 to December 31, 2015. A waiver of informed consent was provided by the institutional review board given that this was a retrospective de-identified review of OR and NORA metrics. These data were extracted using WiseOR® (WiseOR Inc., Palo Alto, CA) system and were subsequently imported into RStudio Desktop (version 0.99.896, RStudio, Boston, MA) for analysis. Data were divided into main OR cases and NORA cases. Using RStudio with JAGS (v 4.2.0), along with the BEST and rJAGS packages, Markov chain Monte Carlo (MCMC) estimation using Gibbs sampling was used to analyze means and variances between groups [12]. This Bayesian estimation methodology is useful in comparing means and standard deviations between two groups, as well as the overall normality of the data. Compared to the traditional t test, Bayesian estimation is more robust to outliers and non-parametric data. Particular parameters used are documented in the relevant sections below.

Turnover time

For each workday, TOT was calculated as the amount of time between when one patient left a given room to when the next patient entered the same room for sequentially scheduled cases [13]. TOTs were calculated for all rooms and divided into the main OR and NORA. MCMC estimation was used to compare the mean and standard deviations of the groups. 1000 iterations × 3 chains were used for burn-in and 33,334 iterations × 3 chains were used for sampling.

Cancellation rates

On a daily basis, WiseOR extracts data (e.g. date, actual room in time, actual room out time, anesthesiology staff members, and proceduralist) for each corresponding scheduled case. Cancelled cases without real-time data were validated on OPTUM® (version 8.5, Eden Prairie, WI). We calculated annualized cancellation rates (i.e. cases not staffed / total number of cases scheduled) for the main OR and NORA. We compared cancellation rates between these locations using univariate analyses with odds ratios and an alpha set at 0.05.

First case start delays

From the database, all first cases of the day were identified and their scheduled and actual start times were obtained. Any cases that started early (actual less than scheduled) were removed as well as any case with a delay greater than or equal to 100 min, as examination of the distribution suggested that these instances were data entry errors as opposed to true delays. These values were divided into the main OR and NORA with MCMC estimation being performed using Gibbs sampling. For each comparison, 1000 iterations × 3 chains were used for burn-in and 33,334 iterations × 3 chains were used for sampling.

Scheduling error

Scheduled and actual procedure times were pulled for all cases in the dataset. Any case whose actual time was within a 20% window of scheduled (i.e. greater than 0.8 × scheduled time but less than 1.2 × scheduled time) was deemed accurate and excluded from the dataset. The remaining cases had estimation biases calculated by subtracting that window from the actual time. That is, actual – 0.8 X scheduled (for actual < scheduled) and actual – 1.2 X scheduled (for actual > scheduled). Thus, any negative values indicated the amount of overestimated time outside of the 20% window and positive values indicated underestimated time outside the window. All cases were divided into the main OR and NORA, with MCMC estimation being performed using Gibbs sampling. For calculations, 1000 iterations × 3 chains were used for burn-in and 33,334 iterations × 3 chains were used for sampling.

Results

Table 1 depicts NORA case distribution by specialty. We analyzed a total of 3729 NORA procedures which included 513 procedures performed in the pediatric population. Procedure types included in our analysis were gastrointestinal endoscopy, cardiology, pulmonology, radiology, and interventional pain procedures.

Table 1 NORA case distribution by specialty. All procedures listed were performed in adult patients except for the 513 cases that were performed in the pediatric population
Full size table

Turnover time

During the study period, there were a total of 5851 turnover times. There were 4778 and 1072 distinct turnover times for the main OR and NORA, respectively. Mean TOT for main OR and NORA cases were 37.18 and 16.21 min, respectively. By MCMC estimation, prior distributions were minimally informative (Table 2). Convergence was achieved for all parameters in the dataset. Effective sample size for all parameters was greater than 20,000. Mean main OR TOT was larger than NORA TOT, given that the 95% high density interval (HDI) for each mean did not overlap. Similarly, TOT variances were significantly different, with the main OR variance being less than NORA.

Table 2 MCMC estimations of TOT mean, standard deviation (SD), and median between main OR and NORA cases
Full size table

Cancellation rates

In 2015, there were 18,355 scheduled cases extracted by WiseOR. Out of these cases, 217 were considered cancelled based on missing real-time data (1.18%). In the main OR and NORA sites, 1.35% (164/12,188) and 0.86% (53/6167) of cases were cancelled, respectively (Table 3). A univariate analysis demonstrated that NORA cases were 0.64 times as likely to be cancelled than were main OR cases (p = 0.0031).

Table 3 Univariate analysis of canceled cases between main OR and NORA. Odds ratios (OR) and their 95% confidence intervals (CI) are listed. Significance set at p < 0.05
Full size table

First case start delays

After removal of early start and erroneously entered data, there were a total of 5305 first cases in the dataset. On average, there was a delay of 10.58 min for main OR and 24.45 min for NORA. By MCMC estimation, prior distributions were minimally informative and convergence was achieved for all parameters in the dataset (Table 4). Effective sample size for all parameters was greater than 28,000. Both means and standard deviations between main OR and NORA were significantly different, given that each respective 95% HDI did not overlap. The main OR group had a smaller average of first case start delays as well as a smaller standard deviation.

Table 4 MCMC estimations of first case start delay mean, standard deviations (SD), and median between main OR and NORA cases
Full size table

Scheduling error

In total, there were 14,620 cases in 2015 with scheduling data: 11,483 main OR and 3137 NORA cases. After removal of cases within 20% of their scheduled times, there were 4839 main OR and 1960 NORA cases. By MCMC estimation, prior distributions were minimally informative and convergence was achieved for all parameters (Table 5). Effective sample size for all parameters was greater than 23,000. Comparisons of means and standard deviations had no overlapping 95% HDI, indicating significant and unique values. Both main OR and NORA cases had means less than 0, indicating overestimation of procedure times, but NORA had larger overestimations outside of the scheduling buffer of 20% (4.07 min versus 2.12 min). In addition, the standard deviation of main OR was over twice that of the NORA group, indicating a larger variance in the former.

Table 5 MCMC estimations of scheduling error mean, standard deviations (SD), and median between main OR and NORA cases
Full size table

Discussion

Turnover time

Presumably, efficient ORs have turnover times less than 25 min [7]. While our OR TOTs are above this benchmark, our NORA TOTs are significantly below this benchmark. This results is not surprising.  Previous studies have shown that procedures performed under monitored anesthesia care (MAC) or local anesthetic have lower turnover times when compared to cases done under general anesthesia [14]. A number of NORA logistics may facilitate faster turnover times: technically simpler procedures, less intensive equipment requirements, and differing concerns about sterility [15]. In contrast, the standard deviation for NORA TOTs was larger, indicating less predictability and possibly hinting at the diverse requirements and various geographical locations for various NORA cases. Future analyses should more specifically target NORA workflows in order to find ways of optimizing the work environment for anesthesia health care providers. Specifically, some studies have identified using a dedicated OR team or parallel processing as methods of decreasing turnover times [16,17,18].

Cancellation rates

While prior studies recommend that OR cases have a less than 5% cancellation rate, the application of this benchmark to NORA schedule lists is not known [7]. The overall cancellation rate in our main OR dataset is 1.35%, which is well below these guidelines. Our NORA cancellation rate is even lower at 0.86%, with a comparative odds ratio of 64%. This discrepancy may be due to various patient, provider, or institutional characteristics that were not captured by the data. For example, there may be an inherently higher risk of certain main OR procedures and thus a lower threshold for cancellation of cases. For example, a study demonstrated that patients undergoing cardiac surgery have higher case cancellations rates [19]. Alternatively, a larger percentage of inpatients, who have higher cancellation rates than outpatients, may present for surgery in the main OR [20]. Again, understanding the differences between OR and NORA cancellation rates can help target future improvement strategies. NORA cases could be segmented to see if a particular service line has higher case cancellation rates (such as pediatric MRI in a prior study) [21]. Future studies could also expand the scope for data extraction. Dexter et al. suggest that cases cancelled up to 24 h should be included in data analyses; this study did not attempt to identify cases that had been scheduled and cancelled prior to the day of the procedure. [22].

First case start delays

A number of studies have identified first case on-time starts as an important benchmark of OR efficiency [7, 19, 23, 24]. In this study, NORA cases delays were significantly longer at 24.45 min (compared to average OR delays, 10.58 min). Additionally, the NORA standard deviation was larger, suggesting the possibility that delays are even greater in NORA settings. In a sense, NORA cases may resemble outpatient settings, where gastroenterology and ophthalmology suitestend to have more case delays [19]. Further, these service lines could have a disproportionately large impact on first case delay metrics; the number of cases performed in the decosopy suite constitutes over 45% of the dataset. Both the mean and standard deviation of first case delays in NORA were more than two times greater than that of OR cases, suggesting that there is a significant room for improvement. Successful examples of improving first case on-time starts from the realm of OR management may be beneficial. These include specifying proceduralist and anesthesiologist availability times, patient arrival times, paperwork readiness, and multidisciplinary buy-in [28]. Future studies should breakdown the causes of these delays by the particular subspecialties, the time availability of the proceduralists, anesthesia staffings ratios, and the present of anesthesia and specialty trainees [23, 25,26,27]. By doing so, anesthesiologists will be better able to assess the impact of first case start delays and the potential for over-utilized time in the NORA setting.

Scheduling error

After excluding cases with actual case times that fell within the upper and lower bounds, 42.1% of OR cases and 62.5% of NORA cases had scheduling errors in this analysis. In other words,, while OR cases usually run a little closer to their scheduled times, they do so with greater variability (i.e. things may go very short or long). Although NORA cases consistently ran shorter than scheduled when compared to their OR counterparts, they were more predictable in their variability of case duration. These results serve as an important guide for future scheduling of NORA cases. It should be possible to reduce the amount of time allocated for certain cases, without necessarily worrying that many cases would subsequently run overtime because NORA cases tend to run shorter than scheduled. Prior studies in OR metrics have identified a number of specialties that overestimate their times, as well as describe a methodology of using historical booking times to better allocate anesthesiologist resources [29, 30]. Similar studies should be undertaken in the future for NORA cases to best determine which types of cases and specialties would benefit from this scheduling improvements a caveat, these scheduling changes should be done with continuous data monitoring to understand the impact for NORA case lists.

Conclusions and future directions

A new set of efficiency benchmarks should be tested and validated for NORA settings. There are marked differences between OR and NORA efficiency metrics. For mean TOTs and cancellation rates, anesthesia health care providers working in NORA settings appear to be doing better than their OR counterparts. By contrast, in terms of scheduling error and first case delays, OR metrics appear to be more favorable. While many of our OR efficiency metrcs are well withing the generally accepted standards, the application of these benchmarks to a NORA dataset suggests that different benchmarks may be necessary.

There are a several limitations with this study. First, the analyses encompasses just one rural, academic institution. There can be differences in performance between academic and non-academic institutions, particularly around first case start time or scheduling error, depending on the methodology by which the institution schedules its cases. Future studies should encompass a variety of institutional settings to see if those that have a higher percentage of NORA cases have improved efficiency metrics compared to those that do not have as many NORA cases in order to better establish hospital benchmarks for NORA efficiency.

Second, a myriad of patient, provider, or institutional characteristics could not be controlled for because these data were not available in the database. It is possible that American Society of Anesthesiologist Physical Status (ASA-PS) class, patient age, patient body mass index, provider availability, instrument availability, and procedure could all play important roles in efficiency metrics [26, 31,32,33,34,35]. Case delays and scheduling error could be isolated to certain specialties and surgeries. These differences may make it difficult to apply the same metric to cases performed in the main OR versus a NORA setting. Additionally, these benchmarks could correlate with patient age and ASA-PS class, while service location could just be a confounding variable. By delving into these various characteristics in future studies, future studies should control for potential cofounders and better elucidate the underlying differences between main OR and NORA cases.

Finally, there are a number of other important efficiency metrics, including PACU admission delays, unexpected hospital admissions, excess staffing costs, and contribution margins per OR hour [7, 19]. In an era of increasing healthcare costs, it is imperative to optimize workflows for patient care and allocate the necessary resources. Anesthesiologists working in NORA environments should have a comprehensive view of NORA performance, an understanding of the operational and structural differences, and the potential avenues to improve the delivery of care.