Introduction

In 2015 the Lancet Commission on Global Surgery (LCoGS) highlighted disparities in the provision of surgical care in low- and middle-income countries (LMICs) compared to high-income countries (HICs) [1]. There is significant global variation in surgical outcomes, with adults up to three times, and children seven times, more likely to die after emergency abdominal surgery in LMICs compared with HICs [2]. It is estimated that 23 million disability-adjusted life-years are lost each year due to in-hospital adverse events alone and that two-thirds of these occur in LMICs [3].

Quality Improvement is central to improving morbidity and mortality in surgical systems [3, 4]. Most quality improvement research is conducted in HICs and, given the distinct clinical needs and financial constraints in LMICs, research findings from HICs cannot always be extrapolated directly between settings [5]. Surgical site infections (SSI) are the leading cause of post-operative morbidity, and the leading cause of hospital-acquired infections in LMICs. The prevalence of surgical site infections is estimated to be at least twice as high in LMICs compared to HICs [6]. Along with peri-operative mortality, SSI are important targets for surgical quality improvement, as they can result in enormous morbidity, health-care costs, and loss of productivity [6]. The evidence behind the effectiveness of quality improvement interventions targeting mortality and SSI in LMICs has thus far not been well elucidated in the literature [7].

In this systematic review, we aim to assess and quantify the effect of quality improvement processes and structure on mortality, rates of Hospital Acquired Infections (HAI) and SSIs in the peri-operative setting. These findings could be used to help inform the development of evidence-based guidelines that seek to increase the quality, access, and safety of surgical and peri-operative systems in LMICs. The study forms a part of a series of systematic reviews and meta-analyses conducted by the G4 Alliance and International Society of Surgery International Standards and Guidelines for Quality Safe Surgery and Anesthesia (ISG-QSSA) Working Group.

Methods

The G4 Alliance is a 60+member organization representing over 300 international federations, societies, academia, and non-governmental organizations in 160 countries worldwide. In partnership with the International Society of Surgery (ISS), the Alliance formed the ISG-QSSA Working Group, which is comprised of 13 members from surgical, anesthesia, government, and public health specialties with the goal of summarizing the existing evidence base regarding optimal surgical, obstetric, trauma, and anesthesia systems quality improvement interventions in order to arrive at global policy recommendations for LMICs. The working group was charged with identifying relevant research questions and population, intervention, comparator, and outcome (PICO) considerations that formed the basis of this systematic review.

Database search

A systematic review of the literature was performed following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The search included five databases: Medline, CINAHL, SCOPUS, CENTRAL, and EMBASE and incorporated four domains: 1. LMICs, 2. Surgical 3. Interventions 4. Mortality (search terms in Fig. 1). A Grey Literature search was performed using the Open Grey Database, Google Scholar, and WHO regional databases for Africa and Asia. Reference lists of included full-text reports and systematic reviews were cross-checked for relevant records. A date restriction was applied from January 1st, 1989. The search started in February 2020; the date of the last search was 18th August 2020. Results were restricted to English-language full-text articles. Prospero Registration: CRD42020171542.

Fig. 1
figure 1

PRISMA Flowchart

Full size image

Inclusion Criteria

Interventional studies published in English, conducted in countries meeting the World Bank Income Classification, that assessed quality improvement processes with regard to impact on mortality, HAI or SSI were included.

Exclusion criteria

Studies were excluded if they did not involve a surgical or peri-operative system or were not conducted in an LMICs setting. Studies were likewise excluded if they did not include implementation of a particular process, intervention, or structure, or if they did not report outcomes as mortality or morbidity. Studies examining a specific disease process were also excluded. If any of the inclusion or exclusion criteria were unclear, a third reviewer was involved in reaching a consensus. Conference abstracts were not included. Review articles were not included, but all relevant reviews were examined for citations of reports that were not already found in the search.

The retrieved records from the database were imported in Endnote® (Clarivate, Philadelphia PA, USA), and duplicates were removed [8]. According to the inclusion and exclusion criteria, the results were screened by two independent authors (JJ and SA). Data were extracted to a standardised form, which included information on study setting, study population, sample size, method of quality improvement intervention, and the comparison group. The primary endpoint was mortality. The secondary outcomes were the rate of HAI and SSI. The cost-effectiveness of an intervention was also summarised where applicable.

Meta-Analysis

A meta-analysis was conducted when the interventions and outcomes were determined to be combinable. Meta-analysis was performed in R (R Foundation for Statistical Computing, Vienna, Austria) using the "meta" package [9]. The relative risk (RR, 95% confidence interval [CI]) of the primary outcome “mortality” was calculated using original data from the studies by dividing the probability of death given the presence of a quality improvement intervention by the probability of death given the absence of an intervention. The Mantel-Hanzel method was used as the weighing method across studies as it allows better estimates when there are few events. The effect size chosen was the risk ratio. The I2 statistic for each analysis was calculated to estimate the fraction of variation in the effect estimate (i.e., RR of mortality) caused by heterogeneity. Significant heterogeneity was established when the I2 test statistic was greater than the degrees of freedom, the p value was < 0.20, and the I2 was greater than 50%. Random effects were chosen as the analysis moderator if significant heterogeneity among studies was found. Publication bias was assessed using a funnel plot of the effect sizes. No publication bias was noted when the effect sizes were noted to have an even dispersion around the pooled effect estimate. Due to the paucity of randomized controlled studies (RCTs) in certain interventions or in instances where it was not deemed possible to conduct such studies due to the lack of equipoise, non-randomized interventional studies (i.e., before-after studies) were utilized. Although considered critical in the assessment of healthcare evaluations, the risk of bias significantly impacts its findings' applicability. Therefore, a secondary tool, the Risk Of Bias In Non-randomised Studies-of Interventions (ROBINS-I) tool was utilized. ROBINS-I is a new tool that views each study as an attempt to emulate a hypothetical pragmatic RCT and covers seven distinct domains through which bias may be introduced [10].

Consensus

In March 2020, an international group of experts representing the G4 Alliance, ISS, and local Fijian surgeons were hosted by the Fiji Ministry of Health and reviewed the preliminary results. The meeting participants concluded that the results were heterogeneous, and the risk of bias was inherently high due to a large number of uncontrolled before and after observational studies. The panel agreed that the ROBINS-I tool was to be utilized in the final assessment of studies for the GRADE methodology [11].

Results

Initial search results returned 38,273 articles. After duplicate removal, 28,779 were screened, and 28,709 articles were excluded after title and abstract screening. Seventy full-text articles were screened for eligibility. A total of 32 studies were included in the qualitative synthesis. Twenty-nine studies were included in a quantitative synthesis for meta-analysis. Studies were from 20 different countries. A heterogenous group of studies were found, including randomized studies, controlled and uncontrolled before and after studies, and retrospective studies. The PRISMA flowchart is displayed in Fig. 2. The Summary of Interventions and their effects are shown in Table 1. The overall risk of bias summary chart is shown in Fig. 3. The risk of bias for individual studies and domains is displayed in Fig. 1.

Fig. 2
figure 2

Overall Risk of Bias of Included Studies according to ROBINS-I

Full size image
Table 1 Overall Summary of Findings
Full size table
Fig. 3
figure 3

a Meta-analysis of WHO SSC on mortality and b rates of SSI. (1 study by Prakash et al. was removed due to wide confidence interval and small weighing 0.4%.)

Full size image

Effect of the WHO Safe Surgery Checklist on Mortality

In 2008, the World Health Organization introduced the Surgical Safety Checklist (WHO SSC) designed to improve consistency of care [12]. Ten studies have evaluated the effect of the WHO SSC on mortality [13,14,15,16,17,18,19,20,21,22]. Of the ten studies, nine were categorised as uncontrolled before-and-after studies. Only one study by Chaudhary was a randomised controlled trial with 350 patients per arm [13]. A pilot study by Weiser et al. evaluated the effect of the WHO SCC in 8 tertiary centres worldwide [19]. We extracted data pertaining to LMICs to be included in the analysis. The summary of study characteristics is shown in Table 2.

Table 2 Summary of Interventions and Study Characteristics
Full size table

The weighted pooled RR for mortality was 0.68 (0.47 to 0.97) across the analysed studies, representing a 32% reduction in mortality. The confidence interval for the effect size was 0.47 to 0.97, representing a statistically significant reduction. The I 2 was 17%, indicating low heterogeneity. No publication bias was present in the funnel plot (Fig. 3a).

Fig. 4
figure 4

a Meta-analysis of infection bundle interventions on HAI, b hand hygiene interventions on rates of HAI and c antimicrobial stewardship interventions on SSI

Full size image

Effect of bundled interventions to reduce HAI

Eight studies measured the effectiveness of infection control bundles. These studies all utilised combinations of interventions such as the introduction of hand hygiene measures, educational programs that used prophylactic antibiotics, and modular training [23,24,25,26,27,28,29,30]. The outcome measured was any hospital-acquired infection (HAI), including surgical site infection (SSI), central line-associated bacteremia (CLABSI), or ventilator acquired pneumonia (VAP). The summary of study characteristics and interventions are shown in Table 2.

Three studies were cross-sectional studies that measured the incidence of HAI using prevalence surveys before and after an intervention, and five studies used a prospective cohort study approach where an intervention was introduced after baseline values were calculated. All studies used a before-after model to quantify the outcome.

The analysed combined effect size had a RR of 0.39 (0.26 to 0.59) and is shown in Fig. 4a. The studies had considerable heterogeneity with an I2 = 79%. The confidence interval of 0.26 to 0.59 represents a statistically significant decrease in HAI. A funnel plot showed publication bias to be present. Overall, despite the limitations of the study quality and design, there is evidence that bundled intervention programs effectively reduce rates of HAI by 61% in an LMIC environment.

Effect of Hand Hygiene Interventions on rates of HAI

Five studies evaluated the effect of infection control hand hygiene interventions, including implementing an alcohol-based hand rub [31,32,33,34,35]. Three studies were before-after studies, and two studies used an interrupted time-series design reporting the results as a trend. Four studies were included in a meta-analysis. One interrupted time series was not included in the meta-analysis as raw data were not reported. In addition to rates of HAI, all four studies measured outcomes of hand hygiene compliance (Table 2).

There was a non-statistically significant reduction in rates of HAI in populations with hand hygiene interventions (RR = 0.69 (0.46–1.05)) (Fig. 4b). There was significant heterogeneity with an I2 value of 58%, indicating that the effect of the intervention varied depending on location. Although the pooled estimate showed an imprecise effect, two large, well designed, interrupted time series studies showed a significant effect (RR 0.85; (95%CI, 0.79–0.90) and RR of 0.8 P = 0.001) [31,32,33,34,, 33]. These studies were characterised by measurement of the outcome over several years and utilisation of rigorous monitoring of compliance and surveillance of HAI carried out by trained infection control nurses. One study reported that a hand hygiene intervention was cost-effective, with $1074 per HAI prevented and a mean attributable cost of HAI of $1,131 per case [35].

WHO Safe surgical checklist on rates of SSI

Of the ten studies reporting the impact of the WHO SSC on mortality, eight of these also reported the WHO SCC’s impact on SSI. These were therefore included in the analysis twice. Three studies reported SSI only, bringing the total to eleven studies that reported the effect of the implementation of the WHO SCC on rates of SSI [13,14,15, 17,18,19,20,21,22, 36, 37]. One study was a randomised controlled trial while ten studies were observational studies (Table 2).

On average, the implementation of the WHO SCC checklist reduced the rate of SSI by 50%, (RR of 0.50 (0.36 to 0.69)). The effect was statistically significant. I2 was 80%, indicating a substantial dispersion of effects about the mean and no publication bias was present (Fig. 3b).

Effect of Antimicrobial Stewardship Interventions on the rate of SSI

Five studies described the effect of the implementation of an antimicrobial stewardship program [38,39,40,41,42]. These studies aimed to optimize surgical antimicrobial prophylaxis through guidelines aimed at the rational use of antibiotics. The interventions included policies that standardise the use of antibiotics peri-operatively aligned with best practice recommendations.

The overall pooled relative risk was 0.67 (0.48–0.93) (Fig. 4c). Three of five studies showed that while the intervention did not reduce SSI rates, the intervention resulted in significant cost savings. Four studies reported cost-effectiveness, reported as dollar cost of antibiotics per 100 patients. Overall cost reduction ranged from $52 per 100 patients to $253 per 100 patients, with an average cost reduction of 49%.

Other Interventions Not Classified

Five studies reported the effect of the International Quality Improvement Collaboration in congenital heart surgery [43,44,45,46,47]. These studies were not included because they related only to a specific group of surgical patients. These interventions were of a multifaceted collaborative approach, which involved several areas such as empowering nurses, focusing on hand hygiene, utilizing checklists, and resulted in significantly decreased rates of in-hospital mortality, bacterial sepsis, and length of stay. Further studies of note included Enhanced Recovery After Surgery (ERAS) protocols and their effect on mortality and length of stay [48].

Discussion

Mortality and surgical infections remain key issues reflecting quality of surgical care, and advancing surgical quality in LMICs is a major global health concern [49]. An estimated 8.6 million deaths occur each year related to delivery of healthcare. Sixty-percent of these deaths are thought to occur due to poor‐quality healthcare rather than an inadequate access or utilisation of care [50]. A key metric for quality in surgical systems, the peri-operative mortality rate, has remained high in LMICs with the number of deaths following surgery estimated to be at least twice that observed in HIC [51, 52]. Postsurgical infections also contribute to high morbidity and mortality, with SSI affecting up to one third of patients who have undergone a surgical procedure [53]. The GlobalSurg Collaborative found that, following risk factor adjustment, patients in low-HDI countries are at greatest risk of SSI [54]. SSI are the most common HAI in these environments, often with substantial morbidity, mortality and economic impacts and therefore are a priority target for quality improvement and patient safety initiatives. With these disparities in mind, there is great urgency to implement strategies to improve mortality and infection rates in surgical systems LMICs.

This study has shown that the single quality improvement intervention in the peri-operative surgical setting with a quantifiable effect on mortality in LMICs is the WHO SCC. The WHO SCC reduces not only mortality, but also SSI. Antimicrobial stewardship programs reduce SSI, bundled interventions and some hand hygiene interventions, especially those characterised by strong emphases over prolonged periods of time on compliance, reduce HAIs. Our review demonstrates studies aimed at improving mortality and infection outcomes in LMICs are uncommon and this review required examination of 30 years’ worth of work to generate the results. This suggests that there are significant research, policy and investment gaps in LMICs surgical and anaesthesia care.

Our review highlighted evidence that the WHO SCC effectively reduced both mortality and rates of SSI in LMICs. This meta-analysis suggests that the WHO SCC is at least equally effective in LMICs compared to HICs [55]. The majority of the evidence gathered included before-and-after observational studies, and these carry significant risk of confounding due to their non-randomised nature and bias from non-blinding of participants and outcome assessors. Disappointingly, the SSC is reported as being used in only about one-third of patients from LMICs, compared with almost 90 percent in HIC [2]. It may be that the WHO SCC is known and available in LMICs but is not being used nor implemented consistently. One reason for this may be due to a lack of published evidence and experience from LMICs [56]. There may also barriers to implementation including resource constraints [57]. Sustainable methods can help overcome some of the barriers to SSC uptake, including partnerships that emphasise training, knowledge-sharing, and local capacity building [17, 58, 59].

In this meta-analysis, multimodal infection control interventions significantly reduced HAI rates, and there was an effect of hand hygiene interventions, that emphasized compliance, on HAI rates. Multimodal infection control bundles and hand hygiene interventions are the easiest and most effective approach to prevent cross-transmission of multidrug-resistant microorganisms and HAI. LMICs often face unique challenges related to hand hygiene, such as lack of running water, procurement of alcohol-based hand rub, and a lack of awareness of hand hygiene, leading to insufficient prioritisation either at an individual or institutional level [60,61,62]. The challenges of implementing hand hygiene interventions can be addressed through multimodal prevention strategies, which have been demonstrated to be more effective than single interventions to change health care worker behaviour [63]. These multimodal strategies are feasible and sustainable across a range of settings in different countries and lead to a significant increase in compliance in healthcare workers [64,65,66]. As cost remains a barrier in implementation, future studies should investigate the cost-effectiveness of hand hygiene interventions on HAI and SSI [35].

Antimicrobial stewardship interventions that specified prophylactic antibiotics for surgery given as a single dose within 1 h pre-incision reduced rates of SSI [6]. Antimicrobial stewardship was associated with improved antimicrobial utilisation in the peri-operative setting, with corresponding improvements in antimicrobial resistance and adverse events without compromise in short-term outcomes [40,41,42]. Previous reviews have demonstrated that pre-incisional antimicrobial prophylaxis reduces the risk of surgical site infection by 23% compared to post-incisional [67]. Although deemed effective, barriers to uptake for these interventions include a perceived cost barrier, shortage of qualified pharmacists, poor interdisciplinary communication, lack of expertise and infrastructure [68, 69]. The cost-effectiveness of the intervention is significant as cost is an important determinant factor of implementation of these programs. It is important to note that the cost-effectiveness was a secondary measure in these studies. There are other papers that primarily focus on cost-effectiveness, but did not measured rates of SSI and these were not included.

This systematic review is limited by its focus on studies which report outcome of mortality and SSI. Also, studies that describe the implementation of quality improvement interventions which report change in process measures were also not included, and studies that reported primarily the cost-effectiveness of interventions were not included. Due to the significant number of observational studies included, there is significant variation in the quality of included studies and the risk of bias. The meta-analysis was limited by the high heterogeneity of the included studies. The studies were conducted in vastly different environments and the effect of the interventions varied depending on the setting. The included studies are also prone to publication bias.

In order to maximise the number of studies analysed we included studies spanning 30 years. It is likely that the health systems of LMICs have changed considerably over this time, which may have impacted the applicability of the findings. Future studies assessing the effectiveness of quality improvement interventions should consider cluster-randomised controlled trials with a stepped-wedge design to improve the quality of evidence. Further systematic reviews on this topic could solely report the impact of interventions on process outcomes to capture a wide range of interventions. As cost remains a barrier in implementing such interventions, cost-effectiveness should be studied in future systematic reviews.

In conclusion, mortality and SSI in surgical systems in LMICs can be improved by the use of the WHO SCC. Multimodal infection control bundles reduce HAI and hand hygiene interventions that have a strong emphasis on compliance have the potential to improve HAI. Antimicrobial stewardship interventions can improve SSI. Future studies could use this evidence specific to LMICs as a basis for the development of evidence-based guidelines.