Background

While there has been considerable success in reducing under-5 mortality globally, similar success in reducing neonatal mortality remains a challenge [1]. A leading cause of preventable morbidity and mortality in this age group is possible serious bacterial infection (PSBI). The World Health Organization (WHO) and other partners have developed standards of care for diagnosis and treatment of PSBI, including care at primary care facilities when referral to higher levels of care is not feasible [2,3,4]. While the effectiveness of this approach is strong, implementation challenges across the cascade from diagnosis through full treatment have been recognized which threaten the potential impact on reducing neonatal mortality in many settings [5]. These barriers include both access and acceptability of PSBI management, the fidelity of diagnosis and care delivery, and adherence to recommended follow-up care by families [6,7,8]. Understanding the strategies needed to implement this revised strategy for PSBI management remains an important area for researchers and implementers. Never has this been more important than during the COVID-19 pandemic, where interruption of care in general, including for sick newborns, has been well documented across global settings.

Implementation Research (IR) evaluates the use of strategies to integrate evidence-based interventions into real-world settings to improve patient outcomes. There is a growing application of these methods to better evaluate, design, and iterate on work to implement evidence-based interventions into practice, including management of PSBI and in pandemics such as COVID-19 [9]. There is also a growing use of embedded IR, where researchers are part of the implementation team, recognized as important for supporting the transfer of research findings into practice [9,10,11]. Recent studies of PSBI management, when referral is not feasible, resulted in important insights into factors associated with gaps and success in implementation [5,6,7,8, 12]. Common barriers included unreliable supply chains of medicines, which threatened the provision of effective outpatient treatment; health information systems not designed to capture the key steps in diagnosis and treatment of sick newborns; weak referral systems; and limited human resource capacity [11, 13, 14] In contrast, facilitators included: the existing Integrated Management of Childhood and Neonatal Illness (IMNCI) program into which PSBI management could be integrated, trained community healthcare workers, and national leadership recognizing the importance of expanding PSBI management [5, 14]. These studies also identified the need for implementation strategies at multiple levels, including the national, health system, facility, health care worker, and community, recognizing that the integration of strategies into the care pathway for PSBI management requires multiple, sequential steps [5, 10, 15].

In 2020, implementing partners in Ethiopia, Kenya, and India were funded by the Bill & Melinda Gates Foundation to conduct embedded implementation research to identify and test strategies to address barriers and strengthen implementation of PSBI management when referral was not possible during the COVID-19 pandemic. All projects used implementation science frameworks, such as the Consolidated Framework for Implementation Research (CFIR), Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM), and health systems frameworks, which encompass cascades of care and the WHO Health Systems Building Blocks in the formative evaluation phases of their work. The organizations had a long track record of supporting maternal and child health implementation in collaboration with their country’s Ministry of Health. Ethiopia and Kenya completed the project, while India’s work is ongoing. The sub-Saharan African projects offered the opportunity to understand contextual factors between and across the two settings and to compare strategies implemented along the PSBI care treatment cascade throughout the projects’ implementation phases. The emerging knowledge from these two projects is valuable to support work to achieve the goal of reduction of neonatal mortality rates globally, an important Sustainable Development Goal, and needed for effective universal health coverage for the youngest members of any community.

Methods

Study settings

Ethiopia’s health system comprises primary, secondary, and tertiary care levels, with primary care emphasizing preventive and promotive services through the Health Extension Program. Health posts, staffed by two female Health Extension Workers (HEWs), are key in delivering frontline health services.

Kenya’s health system is organized into five tiers, including national referral hospitals, county hospitals, health centers, dispensaries, and community health structures. At the community level, Community Health Volunteers (CHVs) play a vital role in delivering preventive and promotive health services, supervised by Community Health Assistants or Officers, and overseen by Community Health Committees.

In Ethiopia, the project was implemented by JSI Research & Training Institute, Inc. (JSI) between November 2020 and June 2022, in the Dembecha and Lume woredas (i.e., district) of the Amhara and Oromia regions, respectively. Dembecha woreda comprises 31 rural kebeles (i.e., the lowest administrative unit) with limited electricity and road accessibility, with the majority of residents living in rural areas. Lume, situated near Addis Ababa, the country’s capital, and Adama, the zonal capital, has approximately two-thirds of its population residing in rural settings. In contrast, the woreda boasts 35 rural kebeles with better access to health facilities, electricity, and road infrastructure. Community-based newborn care, including PSBI diagnosis and treatment when referral is not feasible, was integrated into Ethiopia’s Health Extension Program (HEP) in 2012. However, coverage of PSBI treatment remained low [16], with HEWs treating only 8% of expected pneumonia cases [17], and an average of 16 sick children per month were brought to HEWs for treatment [18]. This is due to multiple challenges, including inadequate HEP knowledge, supply chain constraints, and conflicting community beliefs around newborn care [19].

In Kenya, Living Goods and Lwala Community Alliance implemented the study in Busia and Migori counties, respectively, in collaboration with the county and sub-county health management teams and Population Council-Kenya as an evaluation partner. Busia and Migori counties are located in western Kenya, sharing borders with Uganda to the west and Tanzania to the south. These are predominantly rural areas that face challenges in healthcare and infrastructure. The management of PSBI, when referral is not feasible, was adopted in Kenya as part of an integrated approach to the IMNCI in 2018. At the community level, a simplified version of IMNCI is incorporated into integrated community case management (iCCM) module that includes the following strategies: postnatal home visits by CHVs within 48 h to assess for danger signs for the newborn and the mother, and if any, refer them to a nearby health facility; CHVs visits on days one, three and seven following birth; and for newborns referred to the health facility, upon their return home CHVs make follow-up visits for PSBI-confirmed cases. Initial implementation of the PSBI guidelines in Kenya experienced challenges, including weak linkages/referrals between community and primary health facilities, lack of awareness about PSBI service availability, and providers’ inadequate skills [19].

The emergence of COVID-19 exacerbated these challenges by straining the healthcare systems further. This strain was manifested through resource diversion, decreased client demand for maternal and child health services, and other constraints, contributing to the ongoing struggle to effectively address PSBI and other healthcare needs in these countries.

Study design

Pragmatic embedded implementation research was conducted in both the Ethiopia and Kenya projects. We defined pragmatic embedded implementation research (PEIR), as an approach of embedded IR using implementation research methods to drive local change and create generalizable knowledge, building on the literature for embedded implementation research and our experience in other projects [20,21,22,23]. In these projects, PEIR is done with and by teams involved in implementation, is designed to identify barriers, and to develop new or adapt existing strategies to overcome gaps in the delivery and uptake of effective interventions, such as treatment for PSBI when referral is not feasible.

Guided by the CFIR, RE-AIM, and health system frameworks that incorporate cascades of care and World Health Organization Health Systems Building Blocks, both projects conducted mixed formative quantitative and exploratory research from April to May 2021, followed by summative evaluations conducted between June and July 2022 to address contextual challenges and improve PSBI management during the COVID-19 pandemic. These evaluations included facility assessments and interviews with community health providers and program managers. Furthermore, in Ethiopia, before-after cross-sectional household surveys were conducted in April 2021 and June 2022 interviewing women who had given birth 2–14 months before data collection. Details of the study design are found elsewhere [24, 25].

Implementation research frameworks and models

A community of practice (CoP) was developed to support the IR, including monthly calls and an in-person meeting in October 2022 in Addis Ababa, Ethiopia. IR frameworks and models were chosen that had been used in low- and middle-income countries (LMICs), including the CFIR 1.0; and the RE-AIM framework to explore implementation challenges and guide measurement and evaluation of implementation outcomes [26, 27]. The group also developed a care cascade for PSBI management when referral is not feasible (Fig. 1) to map the implementation of strategies across the steps from case identification through treatment completion.

Fig. 1
figure 1

Cascade of care for diagnosis and management of PSBI when referral is not possible

Full size image

Reflecting on the importance of the strategies targeting health system strengthening, the WHO Health Systems Building Blocks framework for grouping strategies was also leveraged to develop effective implementation strategies [28]. In addition, the group developed an adapted version of the Implementation Research Logic Model (IRLM) [29] to map cross-cutting contextual factors through the strategies, and proposed mechanisms to achieve targeted implementation outcomes. The IRLM is a compilation of IR frameworks, which supports mapping from contextual factors to strategies through proposed mechanisms to implementation, system, and clinical outcomes.

Data

Both projects developed and implemented a range of activities to capture contextual factors at baseline, including household surveys, facility surveys, and key informant interviews. End-line results were also captured by these two completed projects from project records and surveys conducted by each (Ethiopia November 2020-June 2022; Kenya June-July 2022). Notably, the end-line health facility assessment covered 65 health posts in Ethiopia and 38 primary healthcare facilities in Kenya. The facility surveys were crafted to assess the availability of drugs, equipment, and the implementation fidelity of strategies, including ongoing implementation challenges and adaptations. In Ethiopia, we interviewed 4,262 and 4,082 women who gave live birth 2–14 months before data collection and identified 374 and 264 PSBI cases in April 2021 and June 2022, respectively, to assess changes in implementation outcomes related to care-seeking practices for sick infants [24].

Project teams included an implementation researcher who led work to identify contextual factors, document strategies, and identify targeted implementation outcomes. In addition, the Northwestern University team reviewed protocols, emerging reports, and publications from the two country programs, as well as through a review of emerging results during CoP calls and ad hoc meetings.

Analysis

Results from the formative and end-line data, as well as initial and adapted strategies, targeted implementation outcomes, and results were synthesized from project reports, presentations, and during CoP calls and an in-person meeting in 2022, with follow-up calls held for clarification. The teams collaborated to map strategies across the cascade of PSBI management to review the synthesis of results within and across projects during virtual calls with each project and the in-person meeting. An IRLM was then developed as a method to summarize shared and differing factors, strategies, and outcomes. Results are reported using the Standards for Reporting Implementation Studies (STARI).

Results

Contextual factors

During the formative phase of the projects and throughout, barriers and facilitators were identified across the outer domain (global, including the COVID-19 pandemic, national, community, and individual), the inner domain (health system and facilities), actors including care providers and managers, the intervention (PSBI management when referral was not possible), and systems (i.e., supply chain, transport). Many contextual barriers and facilitators were shared across the two countries (and two sites within each country), although some unique factors were also identified. For example, in Ethiopia, conflict-affected one of the two sites, with supply chain challenges seen at the national level. In Kenya, industrial actions by health care workers (HCWs) were a barrier to implementation not seen in Ethiopia. The barriers and facilitators identified across both projects are summarized in Table 1. These include the existence of global recommendations for PSBI management when referral is not possible, as well as the degree of leadership at the global, funder, and national levels to reduce neonatal mortality rate (NMR).

Table 1 Contextual factors identified by the Ethiopia and Kenya projects for PSBI management when referral is not feasible during the COVID-19 pandemic, mapped based on the CFIR framework
Full size table

Both projects identified existing programs for community-based care as key facilitators, although the design and scope differed between Ethiopia’s HEP (able to provide diagnosis and treatment) and the Women Development Army (outreach) compared to Kenya’s community health worker (CHW) program (outreach and referrals to primary health care (PHC) facilities). Kenya also has an existing electronic community-based information system, while in Ethiopia, this was newly implemented during the project through a separate initiative in one of the two project areas. At the individual and community levels, both projects had to address a lack of understanding and belief in the treatment of sick newborns and address transport barriers to accessing care. While the COVID-19 pandemic was an initial barrier, strategies to reduce risk as well as reduced rates of disease over time, made this less of a threat to implementation outcomes.

Strategies

Despite differences in settings, the projects identified many of the same strategies, although design and implementation sometimes differed to accommodate unique contextual factors (e.g., the war in Ethiopia, CHW scope of practice, health system structure, and COVID-19 rates). These strategies were mapped across the PSBI diagnosis and treatment care cascade and covered many of the WHO Health System Strengthening Building Blocks (Table 2) and across the care pathway for management of PSBI when referral is not feasible.

Table 2 Strategies mapped across the care pathway for management of PSBI when referral is not possible -
Full size table

Strategies across multiple steps of the PSBI management care cascade

Some strategies were developed to address barriers and leverage facilitators mapped across the cascade, including leveraging existing CHWs (Ethiopia and Kenya) to integrate the work into existing primary care. Program monitoring and data use to improve quality across steps was also supported through health care worker audits and feedback, and the spread of learning (learning visits and a local community of practice).

Strategies targeting specific care cascade steps

Identification and screening

All projects implemented strategies to identify newborns, screen for, and diagnose PSBI cases. The Ethiopia and Kenya projects incorporated home visits to identify neonates requiring additional screening and follow-up. Digital health was also used to support data collection by CHWs in Kenya to improve identification and screening of newborns and tracking of care in Ethiopia. In both projects, strategies were developed to overcome barriers in cultural beliefs to accessing care for sick newborns. These included outreach through CHWs and engagement of key stakeholders and opinion leaders, such as in Kenya increasing CHVs involvement in their communities to improve caregiver confidence in treating sick young infants. A state-level sensitization workshop and a technical support unit were established in Ethiopia.

Diagnosis and treatment

Ethiopia and Kenya both implemented training for the healthcare workforce to identify and confirm PSBI diagnosis and to provide treatment according to national guidelines. These were further strengthened through job aids and supportive supervision. While some strategies were successful, challenges remained. For example, in Ethiopia, national gentamicin supply chain gaps were explored at the formative stage and persisted over time, resulting in significant barriers to treatment, which was not the case for Kenya’s project. In response, in Ethiopia, the team implemented local and national supply chain strengthening strategies, including engaging HCWs to adapt medication use, supply sharing through redistribution among woreda-level facilities, and national advocacy to address the gap, resulting in improvement in overall essential supplies across 98% of health posts.

Correct treatment and follow-up care

Both projects in Ethiopia and Kenya implemented a range of strategies, including home visits conducted by CHWs, community education initiatives, the utilization of digital health technologies, integration of PSBI into maternal and child health (MCH) and IMNCI programs, training sessions for healthcare workers, and supportive supervision to reinforce skills and knowledge among health workers and encourage adherence to treatment protocols and follow-up care. In addition, in Ethiopia, peer-learning sessions and primary health care unit (PHCU)--level performance reviews were conducted, accompanied by clinical mentoring meetings. These initiatives aimed to facilitate knowledge sharing among healthcare providers and improve clinical practices.

The IRLM (Fig. 2) was able to show how the strategies targeted many of the contextual factors, providing an understanding of the mechanism by which they were thought to facilitate the targeted outcomes.

Fig. 2
figure 2

Implementation Research Logic Model for Ethiopia and Kenya projects designed to increase PSBI access and delivery during COVID-19 and when referral is not feasible

Full size image

Strategy adaptations

Initial developed strategies were sometimes adapted in response to changing contextual factors (e.g., COVID-19 infection risk levels) or outcomes which were falling short of expectations. For example, to improve CHW-delivered PSBI case identification, referral, and follow-up in Ethiopia, a community health volunteer desk was introduced nationally for monitoring at facility level once it became clear that the community-to-facility linkage needed to be strengthened. This was also supported by the adaptation of data collection mechanisms, for more effective real-time data collection on PSBI by community health volunteers, which was used to provide alerts for follow-up care.

Sustainability and scale

From the start, the projects implemented several strategies to support scale and sustainability locally and nationally. In Ethiopia and Kenya, both groups worked at the national level to strengthen the integration of PSBI management into national MCH/IMNCI strategies rather than treating PSBI programming as a vertical initiative. Both projects implemented deep stakeholder engagement across multiple levels, starting at the planning stage and continuing throughout the project. For example, the team in Ethiopia included a national and district strategy co-design workshop, the introduction of regular kebele-level multisector meetings, a national community of practice, and an assigned facility-level focal person to champion iCCM/PSBI integration. Similar work in the Kenya team included working with county and sub-county stakeholders to sensitize and gain buy-in, and advocacy work at the national level to ensure alignment and agreement as the project was implemented with the integration of emerging lessons into national policy.

Projects recognized that sustainment of the improvement in PSBI management reach and delivery, as well as national scale-up, will require ongoing work, including continued investment to strengthen the following areas: (1) the health workforce to improve capacity, training, knowledge, skills, and decrease workloads; (2) ongoing support of the supply chain to ensure availability of essential drugs and supplies; (3) ongoing strengthening of a network of care from the community-level, primary care, and higher levels across all providers; and (4) community education and engagement to continue to address cultural barriers to accessing PSBI care.

Implementation outcomes

Implementation outcomes were measured according to the RE-AIM and treatment cascade frameworks. Many of the strategies chosen through PEIR were effectively implemented, and implementation outcomes for PSBI management when referral is not feasible were emerging. The strategies were associated with improvement in several steps in the PSBI cascade (Table 3). For example, the Ethiopia site results of household survey showed a significant increase in the proportion of severely ill young infants (i.e., mothers report severe illness signs including infants with difficulty breathing, chest in-drawing, ceasing to feed well, unusually hot or cold, less active than usual, and/or convulsions) that took appropriate antibiotics (gentamicin injection and/or amoxicillin) from 32.6% [95% confidence interval (CI): 25.5–40.7] in April 2021 to 61.8% [95% CI: 51.0-71.6] in June 2022.

Table 3 RE-AIM implementation outcomes from baseline to end-line
Full size table

Despite successful outcomes attributed to implemented strategies, barriers remain that hinder PSBI management, including perceived low quality of service and lack of trust in providers by communities; shortage of health extension workers (Ethiopia) and heavy workloads across health workforce; inadequate knowledge/skills of CHWs; persistent stockouts of essential supplies; and a lack of transport for referrals.

National policy and program impact

The policy impact of the work conducted in both Ethiopia and Kenya is significant and reflects, in part, the strategies and resulting outcomes enabled through the pragmatic embedded implementation research. For example, in Ethiopia, strategies were incorporated into community-level healthcare worker workstreams, weekly and monthly woreda reports, and their annual work plan. At the national level, the IR work informed the MoH’s 2023 and 2024 annual planning and forecasting of iCCM/PSBI commodities.

In Kenya, implementation strategies informed discussions on integrating PSBI indicators into the national health management information system. Based on the Kenya project’s findings, Kenya’s MoH adapted their national tools, including health management information systems (HMIS) and other reporting tools, as well as IMNCI and iCCM protocols, to incorporate PSBI management strategies from this work. In Kenya, PSBI management is now integrated into the community-level healthcare worker curriculum.

Finally, while critical barriers remain, the findings from both projects are already guiding sustainability and national scale-up efforts of PSBI management within each country and may offer important lessons for other LMICs seeking to improve neonatal care.

Discussion

The pragmatic embedded implementation research (PEIR) conducted in Ethiopia and Kenya demonstrated significant achievements in improving access to and quality of treatment for PSBI across the care cascade in neonates during the COVID-19 pandemic. Through a systematic analysis of common factors across settings and the tailored mapping of strategies to address barriers and leverage facilitators, these projects have achieved noteworthy progress in optimizing the integration of PSBI treatment in routine health system functions.

This PEIR drew upon previous PSBI treatment implementation research conducted in resource-constrained settings, leveraging insights gained from past experiences [11, 13, 14, 30,31,32,33,34]. Through this approach, the study identified both similarities and differences in implementation strategies and outcomes compared to previous research efforts. By building upon existing knowledge and adapting approaches to suit the specific contexts of Ethiopia and Kenya, the PEIR was able to identify and capitalize on previously successful strategies while also addressing unique challenges and opportunities present in each setting. This iterative process of learning from previous research ensured a more nuanced and contextually relevant approach to PSBI management in resource-constrained settings, Ethiopia and Kenya. The common barriers identified across projects included fear of COVID-19 infection, poor health referral systems, transport, and communication between higher-level care and PHC, low knowledge, and self-efficacy for treating PSBI among HCWs, weak community engagement, and a sub-optimal support system. While many factors were shared between the two countries, some unique challenges were also observed, such as conflict and shortage of drugs in Ethiopia and industrial actions by healthcare workers in Kenya. Both projects implemented a comprehensive array of strategies to identify newborns, screen for, and diagnose PSBI. Identification and registration of newborns and screening for infection; community engagement and education about newborn infections and COVID-19 preventive measures; stakeholder and opinion leaders engagement throughout the design and implementation phases; training of health workers and community volunteers, coupled with supportive supervision to reinforce motivation and skills; home visitation for identification, screening, and follow-up care; and strengthening referral and communication system were instrumental in overcoming barriers and facilitating the successful implementation of PSBI management interventions across the treatment cascade.

By employing implementation research models and frameworks, the projects were able to map out implementation strategies to address identified barriers and leverage facilitators effectively. Furthermore, they examined causal linkages with outcomes across the cascade and adapted strategies to suit the specific local resources and systems of each setting, ensuring the interventions were contextually relevant and impactful [15, 29]. The Community of Practice (CoP) helped partners coordinate implementation science frameworks, learn from others’ hands-on experience, and gain competencies in implementation science research design, implementation, and monitoring. The teams were actively involved in and shared adaptations of programs and strategies to respond to challenges, which helped them respond effectively to ongoing contextual challenges, including national conflicts and strikes. In addition, the CoP helped implement cross-country teams to identify shared strategies and metrics to support the potential for scale-up.

Evidence from this PEIR suggests increased access to and delivery of PSBI treatment across the care pathway and an improvement in neonatal outcomes attributable to the implemented interventions and strategies. A previous article from the same project also highlights the association of the PEIR approach on enhancing access to and delivery of PSBI treatment [24, 25, 35]. The study conducted an interrupted time series analysis of service statistics data from Ethiopia, revealing a significant increase in the monthly rate of PSBI cases treated by HEWs, thus underscoring the effectiveness of implemented strategies, which exceeded anticipated outcomes if these strategies were not in place. Concurrently, in Kenya, data from service statistics depicted an upward trend in the number of sick young infants receiving treatment at PHC facilities over time, indicating a growing demand for services catering to this demographic within the community during the intervention period [25]. Previous studies have also reported that such implementation strategies improved the treatment of PSBI [36,37,38]. Moreover, the process and results from the PEIR has yielded significant policy impacts in shaping healthcare policies and practices related to PSBI management in both countries, reflecting the integration of implementers into the research and relatively rapid results to inform policy changes. In Ethiopia, the integration of implementation strategies into community-level healthcare worker workflows and national planning processes, such as the Ministry of Health’s annual planning and forecasting, demonstrates the tangible incorporation of research findings into policy decisions. Similarly, in Kenya, the adaptation of national tools and guidelines to incorporate PSBI management strategies, including the integration of PSBI into community-level healthcare worker training curricula, underscores the influence on policy development and implementation.

Our study had several strengths. We were able to compare similarities and variations in the design, implementation, and evaluation processes using the same frameworks. This collaboration across countries helped us to better understand what works in which context and associated implementation outcomes. The use of the IRLM also helped to link the factors to strategies chosen to leverage facilitators or address barriers, which can support the work to generate evidence on strategies to improve PSBI treatment in LMICs. We did, however, have several limitations. The household surveys and facility assessments were structured as pre-post studies, limiting our ability to definitively attribute the observed outcomes solely to the implemented strategies. Also while the projects used the same frameworks, the definitions used for the outcomes differed, so we were not able to combine results. Finally, like many implementation research projects, the timeline was short for measuring sustainment as well as scale.

In conclusion, these methodologies effectively identified weaknesses in the PSBI management cascade of care, understood the underlying factors, and provided a process to select and test mitigating strategies through identified mechanisms and adaptations throughout the project period. The frameworks and methodologies provided an advantage over more traditional baseline and end-line evaluation. In addition, a community of practice strengthened rapid learning and sharing of cross-cutting strategies across different contexts needing to increase reach and delivery and can be scaled up in the respective countries and scaled out to similar settings in the region.