Adverse events and complications can cause health-care facilities to become hazardous environments, with an overall incidence of adverse events estimated at 9.2 % and almost half of these classified as preventable events [1]. Surgical procedures can create considerable risk to patients when complications occur, making surgical safety a major global health issue [2]. In an attempt to reduce and prevent adverse events, checklists have been developed in medicine as well as other high-risk fields, such as aviation, to assign priorities, reduce avoidable errors, minimize risk, and improve patient safety [3].

Modeled after aviation checklists, the World Health Organization (WHO) developed a Surgical Safety Checklist in 2008 as part of the patient safety program, “Safe Surgery Saves Lives,” to improve the safety of surgical care worldwide [4]. The original checklist is divided into three time periods including sign in (before induction of anesthesia), time out (before skin incision), and sign out (before the patient leaves the operating room) and is comprised of a total of 19 safety items [5]. The checklist was initially studied at eight international sites where it demonstrated significant reductions in postoperative morbidity and mortality [6], and now surgical teams worldwide are more aware of this checklist and preventative measures that can be taken to reduce post-operative complications and mortality [7]. In addition to improving patient morbidity and mortality, preoperative checklists have been shown to improve operating room team communication and reduce the number of communication failures [8]. With reduction of errors and improved communication, the use of preoperative checklists could not only improve patient safety but also enhance operating room efficiency.

The field of minimally invasive surgery (MIS) is especially dependent on technology and can be vulnerable to problems introduced by equipment failure and associated potential complications and inefficiencies. In an attempt to improve efficiency, the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) and the Association of Perioperative Registered Nurses (AORN) partnered to create a minimally invasive surgery (MIS) checklist to optimize case flow and minimize errors and frustration. The aim of this project was to evaluate the effectiveness of the SAGES/AORN MIS Surgery Checklist in preventing disruptions that can lead to case delays and determine its ease of use.

Materials and methods

After IRB approval, the SAGES/AORN checklist was implemented across four institutions by members of the SAGES quality, safety, and outcomes committee. These institutions included Carolinas Healthcare System in Charlotte, NC, the Ochsner Clinic in New Orleans, LA, the Naval Hospital Camp in LeJeune, NC, and Inova Fairfax Medical Campus in Falls Church, VA. The checklist was completed by the operating team during any MIS procedures. The checklist could be completed before or after the case so as not to conflict with existing preoperative checklists.

The checklist that was developed via expert consensus of members of the SAGES quality, safety, and outcomes committee and members of AORN, included items to be performed before patient entry, after patient entry, and after patient preparation and draping (Fig. 1). Specific duties were classified as circulating nurse duties and scrub person duties. The checklist was performed and recorded by the surgeon or circulating nurse, and then a checklist assessment was performed by the operating surgeon to assess the effectiveness of the checklist. Specifically, we recorded how often the checklist identified problems and how frequently each of the 45 checklist items was not completed. We also recorded the number of items that were omitted that specifically caused a delay in the case and approximated the time delay. When the checklist was completed at the beginning of the procedure, we assessed the time required to obtain any missing items; when it was completed at the end of the procedure, the surgeon estimated the time wasted during the acquisition of checklist items that had been missed. We noted if any items were wasted during the case because checklist items were not performed.

Fig. 1
figure 1

The SAGES/AORN MIS Safety Checklist

Full size image

The checklist was further evaluated by the time taken to complete it measured in minutes, the frustration related to its completion, its usefulness, and its ease of use. The perceived usefulness (1 = not useful, 5 = very useful), ease of use (1 = very easy, 5 = very difficult), and frustration (1 = no frustration, 5 = extreme frustration) associated with the checklist use were rated on a five-point Likert scale completed by the surgeon. We assessed for any differences dependent on the timing of checklist completion (before or after surgery) as well as differences across the different institutions.

Statistical analysis

Results are reported as means ± standard deviations or medians (range) for continuous and categorical variables. Differences when the checklist was performed before versus after the operation were compared using the Mann–Whitney rank sum test, and differences between institutions using Kruskal–Wallis ANOVA tests. For pairwise comparisons, the Dunn’s method was used. p values <0.05 were considered significant. All data were analyzed using SigmaStat 4.0 Analysis Software (Systat Software, Erkrath - Amtsgericht Wuppertal, Germany).

Results

The checklist was performed during both basic and complex MIS procedures across four facilities (n = 114). The average case duration was 75.6 ± 59.8 min. Sixty-six total items not performed were identified with the use of the checklist, and 27 cases had items that were not performed that caused a case delay (23.7 % of cases). The most frequently missed checklist items included preoperative preference card review (14.0 % of cases), verifying readiness of the carbon dioxide insufflator (8.7 %), and verifying availability of the Veress needle (3.5 %) (Table 1). Checklist items that were never missed during the study include ensuring operating room table orientation and functionality, positioning video monitors appropriately, securing the patient and padding pressure sites, and connecting the light source and CO2. The average case delay due to items not performed on the checklist was 3.9 ± 9.7 min. The total number of items wasted that could have been prevented by the use of the checklist was 17 items (0.15 item per case). The average time to complete the checklist was 3.6 ± 2.7 min. On the Likert scale ranging from 1 to 5, primary surgeons overall rated frustration related to the checklist as 1.3 ± 1.1. Furthermore, they rated the checklist usefulness as 2.6 ± 1.5, and the ease of use was rated as 2.0 ± 1.2 (Table 2).

Table 1 Evaluation of checklist items
Full size table
Table 2 Checklist assessment of all institutions
Full size table

The assessments of the checklist comparing completion before and after the procedure can be found in Table 3. When used before the procedure (n = 36), the checklist identified 33 missing items in 13 cases, and 9 of these items that were not performed caused a delay. The average delay in these cases was 2.8 ± 4.7 min. A total of eight items were wasted when the checklist was performed before the procedure. Frustration related to completing the checklist was rated as 1.3 ± 1.1, its usefulness as 2.8 ± 1.3, and its ease of use as 2.2 ± 1.3.

Table 3 Checklist performed before versus after the procedure
Full size table

When the checklist was performed after the procedure (n = 61), 32 total checklist items were not performed in 18 cases, and 22 of these items caused a delay. Fourteen cases had items that were not performed that caused an average delay of 4.1 ± 11.1 min. Two items were wasted when the checklist was completed after the procedure. When surgeons completed the checklist after the operation, they rated the frustration related to checklist use as 1.3 ± 1.0, its usefulness at 2.2 ± 1.5, and its ease of use as 1.7 ± 0.8. When comparing the surgeon checklist assessments before the procedure to those performed after the procedure, the only difference was that checklist completion before the procedure took a significantly longer time (5.1 ± 2.9 vs 2.7 ± 2.0 min) (p < 0.001).

Differences in checklist assessments across the four institutions are shown in Table 4. Institution #1 had the largest number of total issues identified by the checklist (35 total checklist items) and the largest number of issues that led to preventable delays (12 items with an average case delay of 2.4 ± 4.3 min) (p < 0.001). This institution had the largest proportion of its checklists performed before the operation (47.1 %). The surgeon rated the checklist usefulness the highest (3.6 ± 1.3) (p < 0.001), but also more difficult to use among participating institutions (2.7 ± 1.4) (p < 0.001). Institution #3 had the highest frustration ratings related to the checklist (2.4 ± 1.3; p = 0.016). This institution experienced the most significant delays due to the issues identified on the checklist with an average case delay of 13.6 ± 26 min and had the most preventable waste (8 items in 10 cases). Institution #2 found the checklist not frustrating to use (1.3 ± 0.7) and easy to use (2.0), but found the checklist the least useful (1.8 ± 1.4) among participants. Institution #4 had intermediate values for the checklist assessments including frustration related to the checklist (1.1 ± 1.2), checklist usefulness (2.5 ± 1.5), and checklist ease of use (1.5 ± 1.1).

Table 4 Checklist assessment across institutions
Full size table

We then performed pairwise testing between the participating institutions. When comparing Institution #1 to Institution #2, Institution #1 had significantly more total checklist issues identified (p < 0.05). Additionally at this institution, it took significantly longer to complete the checklist (4.9 ± 2.6 vs. 2.2 ± 0.7 min) (p < 0.05), and the checklist was found to be significantly more useful (3.6 ± 1.3 vs. 1.8 ± 1.4) (p < 0.05). A pairwise test comparing Institution #1 to #3 revealed that the only significant difference in checklist assessments was the time to complete the checklist (4.9 ± 2.6 vs. 0.5 ± 0.5) (p < 0.05). When comparing Institution #1 to Institution #4, Institution #1 had significantly more checklist items identified (35 vs. 14) (p < 0.05), and Institution #1 found the checklist significantly more difficult to use (2.7 ± 1.4 vs. 1.5 ± 1.1) (p < 0.001). Pairwise testing of Institution #2 and Institution #3 found that the only significant difference was frustration during the case. Institution #3 ranked the checklist significantly more frustrating to use (2.7 ± 1.3 vs. 1.8 ± 1.4) (p < 0.05). When assessing the differences between Institution #2 and #4, the only significant difference was Institution #4 took a significantly longer time to complete the checklist (4.9 ± 2.8 vs. 2.2 ± 0.7 min) (p < 0.05). The last pairwise test between Institution #3 and Institution #4 revealed that Institution #4 had significantly longer times required to complete the checklist (4.9 ± 2.8 vs. 0.5 ± 0.5 min) (p < 0.05).

Discussion

Surgical care has the potential to save lives and improve the quality of life, but it carries a considerable risk for complications and even death. Checklists, such as the WHO surgical checklist, have been developed to improve the safety of surgical patients and reduce the rate of major surgical complications [5]. In the pilot study of eight hospitals around the world, implementation of the WHO checklist reduced the rate of death from 1.5 to 0.8 % and decreased the rate of inpatient complications from 11.0 to 7.0 % [6]. Since that time, surgical safety checklists have been implemented and even become mandatory in many hospitals around the world to improve patient safety and outcomes [7].

There have also been surgical checklists created with the aim of improving operating room efficiency, specifically in the field of minimally invasive surgery. The field of minimally invasive surgery requires advanced technology to perform operations laparoscopically or robotically, and the equipment required can be prone to malfunction or failure. This can lead to case delays, prolonged operating room times, and interruptions in case flow, making operating room efficiency extremely important. Other subspecialties have formulated checklists to improve upon operating room efficiency in MIS procedures; for example, an interactive surgical checklist for robotic-assisted gynecologic operations was developed and studied at one institution with results revealing a significant reduction in readmissions without a negative impact on operating room times [9]. Similarly, this study evaluates the effectiveness of the SAGES/AORN checklist in preventing such disruptions and delays in minimally invasive laparoscopic and robotic general surgery cases. It also addresses checklist ease of use, its utility, and the frustration derived from completing the checklist by the surgeon and operating room staff.

In our study, the SAGES/AORN MIS checklist identified problems in 24 % of total cases that led to preventable delays. The checklist overall was easy to complete and not frustrating to perform, indicating that preoperative use of the checklist could improve operative flow. Surgeons in general involved in this study found the checklist only moderately useful; however when comparing institutions, Institution #1 reported spending more time completing the checklist and had significantly more checklist items identified and, in turn, found the checklist the most useful among all participating facilities. This institution also completed the checklist more often preoperatively than the other facilities. It appears, therefore, that the more the time invested in carefully reviewing the checklist items preoperatively, the more benefit may be realized. It is also possible based on our experience that facilities in which the operating room staff may be less consistent or less experienced may find more benefit with the use of the checklist to help adequately prepare for MIS cases. Additionally, operations that are performed at night or on an emergent basis may also benefit from the checklist to improve operating room efficiency and case flow. Preoperative checklist performance was associated with significantly longer times to complete the checklist, but had shorter delays in the case (however, this difference in delays was not statistically significant). This study also identified which checklist items were omitted most often, and this information may help abbreviate the checklist in the future and minimize the frustration and time taken to complete it while maximizing its utility. Abbreviating this checklist and studying the utility of the checklist during both emergent and elective procedures should be further explored in future studies.

Many of the checklist items that were commonly missed items, such as the preoperative preference card review and verifying the readiness of the carbon dioxide insufflator, are items intended to be performed before the patient enters the room by the circulating nurse. Given that nurses usually carry the burden of checklist completion before an operation, to prevent checklist fatigue by the addition of our checklist to the existing perioperative checklists, we encouraged both our nurses and surgeons to complete this checklist. By using this approach, we were still able to collect checklist utility data when nurses were unable or reluctant to use the checklist. We believe our approach was warranted since the literature suggests that to optimize perioperative time-outs, they should be led by the surgeon [10, 11]. Nevertheless, additional nursing input will be sought in future assessments of this checklist.

With the current focus on patient outcomes and quality measures, surgical checklists have the ability not only to improve safety, but to augment teamwork among the operating room teammates. Studies have shown that teamwork in surgery can lead to improved patient outcomes, with higher functioning operating room teams attaining significantly less rates of adverse events [8, 12]. Neily et al. [13] studied surgical team training with the incorporation of surgical checklists and communication strategies and found that there was a significant reduction in surgical mortality. This demonstrates that the integration of surgical checklists can help to improve communication and enhance teamwork, and this has the potential to dramatically affect patient outcomes. With improvements in teamwork and communication, checklists may improve operative case flow and potentially reduce operating times. Similarly, preoperative implementation of the SAGES/AORN MIS checklist may be able to reduce wasted items. With reductions in operating room times and waste, surgical checklists may be able to lower operating room costs.

It has been suggested that for rapid and effective adoption of a surgical safety checklist, the simplicity of the checklist is important [6, 14]. This may limit the adoption of longer checklists including the SURPASS checklist and the checklist utilized in this study. Although the perception of surgical safety and patient care is positive, there have been reported negative perceptions on the use of a surgical safety checklist and efficiency [15, 16]. Barriers to successful checklist implementation include the beliefs that it takes too long to complete or could generate delays [16]. Papaconstantinou et al. studied the implementation of a surgical safety checklist based on the WHO checklist and studied operating room efficiency and did not identify any negative impact with regard to operative time, operating room time, first starts in room on time, and same-day cancellations. Additionally, the study found with implementation of the checklist there was a significant reduction in waste and an overall reduction in direct cost per operation of $68. The authors concluded that a surgical safety checklist does not negatively impact operative efficiency and can reduce overall cost per surgical procedure [17]. With this study, we have identified checklist items that are often omitted as well as the items that cause the most delays. With this information, we can create a more simple checklist including the most frequently missed items unique to minimally invasive procedures to augment its ease of use and aid its implementation. Future studies can be performed to evaluate the use of an updated, simplified checklist and its utility with respect to operating room efficacy, minimizing errors, reducing cost, and impact on patient outcomes.

One inherent limitation of this study is the Hawthorne effect, an inherent improvement in performance secondary to the subjects’ knowledge that they are being observed [18]. However, without the checklist, the operating room staff may overlook more items when they are not observed; hence the checklist might gain even more usefulness when operating room teams are not being observed. Importantly, the use of the SAGES/AORN checklist at the end of the procedure in 53.51 % of our observed cases negated this effect. It is also important to note that our approach for this preliminary evaluation of the checklist is subject to a number of biases such as recall bias and estimation bias when the checklist is completed at the end of a procedure; in addition, bias in the checklist assessments may have also been introduced by having as assessors most of the authors of this study. Nevertheless, the first author who performed the data analysis was not involved in data collection to help minimize this bias. We also were not able to assess the diligence with which each surgeon completed the checklist, which may have impacted its usefulness. The variability in time to complete it across institutions provides evidence that not everyone invested the same amount of effort. Nevertheless, we believe that this pragmatic approach closely resembles reality which is important to consider during implementation of a new checklist. We believe that this checklist has the potential to improve patient care by minimizing disruptions in case flow and preventing waste, error and risk. Future studies may help better delineate such attributes of this checklist.

We must also recognize that there is a potential for checklists to create an environment focused on “checking the box” and to create complacency among OR staff. Furthermore, it is possible that a surgical checklist can be seen as a “rite of passage” and give providers a false sense of security regarding patient safety [19]. Therefore, surgeons must be engaged in patient safety and in assuring that these checklists are completed properly.

Conclusion

The SAGES/AORN checklist was developed for minimally invasive surgery procedures. It identified problems in 24 % of total cases that led to preventable delays. The checklist was found to be easy to complete and not frustrating by the operating surgeon and staff. This suggests that preoperative use of the checklist could improve operative flow, enhance operating room efficiency, and build teamwork, thereby improving patient outcomes. This study also identified which checklist items were most useful which may help abbreviate the checklist, minimizing frustration and time taken to complete it while maximizing its utility. These promising attributes of the SAGES/AORN MIS checklist should be further explored in future larger-scale studies.