Abstract
The COVID-19 pandemic highlighted the urgent need to strengthen public health systems. In response, the United Nations Disaster Risk Reduction (UNDRR) Public Health System Resilience Scorecard (Scorecard) was applied in workshops across multiple countries. The aim of our research was to explore the workshop findings to develop priority strategies for strengthening public health system resilience. We conducted a workshop from 14 to 16 March 2023, at the UNDRR Global Education and Training Institute in Incheon, Republic of Korea. A sequential modified Delphi method was utilized to develop a set of prioritized resilience strategies. These were drawn from 70 strategies identified from 13 distinct workshops in eight countries. After two surveys, 23 strategies were finalized. Ten received ratings of “High” or “Very High” from 89% of participants. These related to the inclusion of public health risks in emergency plans, integrating multidisciplinary teams into public health, enabling local transport mechanisms, and improving the ability to manage an influx of patients. The Scorecard provides an adaptable framework to identify and prioritize strategies for strengthening public health system resilience. By leveraging this methodology, our study demonstrated how resilience strategies could inform disaster risk reduction funding, policies, and actions.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
A resilient public health system can resist, absorb, adapt to, and recover efficiently and timely from a hazard while preserving and restoring essential services and functions through risk management (Kruk et al. 2015; Haldane et al. 2021; UNDRR 2023; Ryan, Kako, et al. 2023). This includes rapidly adjusting to situations that are dynamic, fluid, and beyond control of the system (Ryan, Kako, et al. 2023). Achieving this results in a return to stability without long-term impacts on community health, well-being, and development (Behrens et al. 2022). However, the COVID-19 (Coronavirus disease 2019) pandemic, as is often the case with many large-scale disasters, revealed the weaknesses in public health and other societal systems in many parts of the world. The difficulties encountered in managing the effects of such events have been exacerbated by longstanding social inequities, unreliable healthcare, and increased urbanization, population growth, and mobility (Rollston and Galea 2020; Assefa et al. 2022). Consistent with other disasters, the people who experienced the greatest impact were the elderly, low-income, self-employed, and people with noncommunicable diseases (NCDs) such as cardiovascular disease, diabetes, and cancer (Rollston and Galea 2020; Hassan et al. 2022; Ryan, Kako, et al. 2023).
During the pandemic, there were surges in demand for care of people with COVID-19, which made it difficult to maintain services (Sagan et al. 2022). In response, public health systems often employed two broad strategies—scaling up, which involved repurposing and redistributing capacity, and adapting or transforming service delivery by implementing alternative patient care pathways (Sagan et al. 2022). Despite these efforts, access to healthcare was impacted in various ways within and among countries. Many patients were hesitant to seek medical care out of concern for contracting the virus, even though public health measures were effective (Splinter et al. 2021; Hassan et al. 2022). Furthermore, resources were often redirected to manage COVID-19 in many countries, thus limiting access to other medical services and affecting people with NCDs and other health conditions and needs (Hogan et al. 2020; Roberton et al. 2020; Abbas 2021; Arsenault et al. 2022; Jain and Dupas 2022). Extensive quarantine measures, while effective for infectious disease control, affected quality of life, particularly for economically vulnerable populations (Ryan et al. 2020; Hassan et al. 2022). Similarly, lockdowns, which helped slow the spread of the virus while societal response capacities were being built, had diverse implications for delivery of essential services, supply chains, and mental health (Prommas et al. 2023).
Every segment of the population was affected differently during the COVID-19 pandemic. For example, school closures were implemented in many locations as a measure to slow the spread of COVID-19, but as a result, students missed out on significant education opportunities with many losing one to two years of competencies (Engzell et al. 2021). The most impacted were students living in disadvantaged households (Vlachos et al. 2021). There was also an estimated USD10 trillion loss in earnings during the pandemic (Global Preparedness Monitoring Board 2020). Many countries lacked the ability to sustain an entire nationwide response operation (Ryan et al. 2020). This experience serves as an opportunity to rejuvenate public health activities across sectors at local, state, national, and international levels to focus on leaving no one behind.
The Severe Acute Respiratory Syndrome (SARS) and Middle Eastern Respiratory Syndrome (MERS) epidemics helped many locations improve preparedness by informing institutional, legal, and physical infrastructure needs (Mohammadpour et al. 2021; Feitelson et al. 2022). This included enhancing disease surveillance, research and development systems, and improving the ability to rapidly identify, implement, and adjust precautionary measures (Algaissi et al. 2020). The locations that best weathered the pandemic applied many of these lessons to create an agile public health system, encourage people to seek early care, and balance community viability with protecting lives and livelihoods (Msemburi et al. 2022; Ryan, Kako, et al. 2023). These strategies highlight the need to consider internal capacities, managerial interventions, and external factors such as socioeconomic determinants before, during, and after an epidemic, pandemic, or disaster (Mohammadpour et al. 2021; Tambo et al. 2021).
To build capacity within and beyond public health systems, collaboration is required across sectors including industry, government, and not-for-profits such as faith-based organizations (Haldane et al. 2021). The World Health Organization (WHO) Health Emergency and Disaster Risk Management Framework (Health EDRM Framework) provides a frame for starting this trajectory by informing an all-hazards risk management approach to reducing the systemic health risks and consequences of all types of emergencies and disasters across the world (WHO 2019; Wright et al. 2020). To respond to the deficiencies of fragmented policies and programs that tended to focus only on preparedness and response to events, the Health EDRM Framework emphasizes that the entire health system and the whole-of-society must work together. This is needed to develop and implement joint actions that reduce hazards, exposures, and vulnerabilities, and build capacities for preparedness, readiness, response, and recovery.
Since all communities have their unique risk contexts, the Health EDRM Framework describes the importance of communities in leading and participating in the design, delivery, and prioritization of programs and services. There is a need to focus attention and resources on those who are disproportionately affected, such as the poor, women, children, people with disabilities, older persons, migrants, refugees and displaced persons, and people with chronic diseases (WHO 2019). The Health EDRM Framework draws on good practice from humanitarian action, epidemic preparedness and response, disaster management, and health systems strengthening. This guidance assists countries and communities in bringing coherence to the implementation of the Sendai Framework, the Sustainable Development Goals, the Paris Agreement on Climate Change, and other local, national, and international frameworks (WHO 2019).
Key to this is measuring public health system resilience and using this to inform the development of priority actions (Ryan, Kako, et al. 2023). The United Nations Office for Disaster Risk Reduction (UNDRR) Public Health System Resilience Scorecard (Scorecard) complements the Health EDRM Framework and serves as a vital tool to achieve this outcome. The Scorecard was developed with multisectoral experts and is available in multiple languages (UNDRR 2020a). Its creation was influenced by application of the Disaster Resilience Scorecard for Cities, which highlighted the need for a specific focus on public health (Ryan, Telford et al. 2023). The Scorecard is freely available online, includes contributions from some of the authors of this article (Ryan, Abrahams, and Bhatia), and was initially launched in July 2018 (Version 1.0), which was followed by an update in April 2020 (Version 2.0) (UNDRR 2020b).
To provide a path towards improving the resilience of public health systems, this study interpreted, ranked, and reached consensus on a set of prioritized strategies identified from workshops in urban settings across multiple countries where the Scorecard had been applied. Perspectives on the application of the Scorecard were also explored to guide future approaches in identifying strategies and actions for improving the resilience of public health systems. By applying this approach, this project has identified generalizable and translatable strategies to build the resilience of public health systems now and into the future.
2 Materials and Methods
The core methodology of this study hinged on a sequential modified Delphi process to develop a set of prioritized resilience strategies based on findings from workshops where the Scorecard had been applied (Hasson et al. 2000). This approach was selected to provide an opportunity to apply a consensus-based process to clarify, rank, and prioritize these strategies. The study was divided into five phases, as represented in Fig. 1. These phases were executed during a three-day workshop, and detailed information about this workshop is provided below. The approach began by exploring the 70 strategies from the 13 Scorecard workshops to create Survey 1. After participants completed Survey 1, its results informed discussion about the strategies to be included in Survey 2. Once Survey 2 was completed, the strategies were ranked and prioritized based on participant feedback. Finally, another workshop discussion was convened to establish consensus on the set of priority strategies for enhancing the resilience of public health systems.
In the initial phase, our plan aimed to incorporate strategies formed during previous workshops where the Scorecard was applied as part of a project funded by the WHO Centre for Human Development (WHO 2023). This included eight workshops conducted in Australia (n = 1), Bangladesh (n = 2), Slovenia (n = 1), Turkey (n = 2), and the United States (n = 2). The scope for this phase was limited due to a combination of funding, project timeline, the COVID-19 pandemic, and professional networks. In an effort to maximize locations participating in this project, we discovered additional sites and opportunities. This allowed for integration of strategies from additional workshops held in Brazil (n = 3) and Chile (n = 1), which were supported by the UNDRR Regional Office for the Americas and Caribbean. Also, an additional workshop was held in Nepal (n = 1) while a facilitator from Australia was in-country. As a result, this study encompassed input from 13 previously conducted workshops, spanning eight countries and five continents with 284 participants. These meetings were conducted in locally acceptable languages from October 2021 to January 2023 (Fig. 2).
This modified Delphi process was selected because it allows participants to deeply explore a topic in different ways and develop generalizable strategies ready for translation into practice (Glaser and Strauss 1967; Ryan et al. 2018). Another benefit is the controlled feedback process and the variety of analysis techniques that can be used to interpret the data (Dalkey et al. 1969). These characteristics are also designed to offset shortcomings of conventional means of pooling opinions obtained from a group interaction (for example, influences of dominant individuals and group pressure for conformity) (Hsu and Sandford 2007). This article built on the analyses of Scorecard workshops conducted in Slovenia, Turkey, and the United States (Ryan, Kako, et al. 2023).
2.1 Workshop Procedure
The workshop was conducted from 14 to 16 March 2023 at the UNDRR Global Education and Training Institute (UNDRR-GETI) in Incheon, Republic of Korea. The workshop was conducted over three days. On the first day participants shared findings from their respective workshops and completed Survey 1. This included aggregated strategies from all previous workshops. This sharing of findings provided a foundation for the subsequent discussions. On day two there were discussions based on Survey 1 results to determine which strategies should be retained for Survey 2. Once these decisions were made, Survey 2 was completed to further refine the strategies. The third and final day centered on discussing the findings from Survey 2 and working towards a consensus on prioritized actions. This collaborative approach allowed for the identification of key strategies for strengthening the resilience of public health systems and communities.
It is worth noting that the recommended number of participants for a Delphi workshop is 10–15, but this workshop saw the participation of 19 individuals due to the extensive application of the Scorecard across different locations and countries (Hsu and Sandford 2007). To ensure the effectiveness of the workshop, it was led by two experienced facilitators, Ryan and Garner, who guided the discussions and ensured that valuable feedback was recorded throughout the workshop.
2.2 Participants
The participant selection process for this study adhered to purposeful and snowball sampling techniques. Purposive sampling was initially used to identify workshop participants through the UNDRR and the researchers’ networks, including those who had previously participated in the Scorecard workshops. Subsequently, participants were encouraged to suggest other individuals who might be interested in participating.
All workshop participants had previous experience in facilitating or participating in workshops where the Scorecard has been applied. However, for the two workshops conducted in Brazil (Ubá and Guidoval), the facilitators were unable to attend our workshop, but their findings were still included in the aggregated strategies. To maintain continuity and familiarity with the workshop findings in Brazil and Chile, a representative from UNDRR Americas and Caribbean was present during the workshop.
2.3 Phase 1—Survey Development
The development of Survey 1 involved a comprehensive process. The survey included questions about demographics, perspectives on the Scorecard method, general comments, and aggregated strategies from workshops where the Scorecard had been previously applied. Input for crafting the survey questions was obtained from the point of contact for each workshop. Additionally, the survey tool took into account an analysis of the Scorecard workshops in Slovenia, Turkey, and the United States (Ryan, Kako, et al. 2023). After rigorous review, removal of duplicates, and refinement, a total of 70 strategies were included in Survey 1. The strategies were listed in alphabetical order to ensure each could be considered on its own merit.
2.4 Phase 2—Survey 1
All participants in the workshop actively engaged in completing Survey 1. This survey was anonymous and served to clarify, rank, and prioritize strategies for inclusion in the subsequent survey (Survey 2). The findings from Survey 1 played an important role in informing the discussions during Phase 3 of the study, where participants deliberated on the results and decided what should be included in Survey 2. The data from Survey 1 underwent a descriptive analysis, and the rankings were analyzed by Fontenot, Ryan, Noel, and Garner immediately following the completion of the survey. Demographic data were collected along with perspectives on application of the Scorecard. This analysis along with grouping the strategies by The Ten Essentials for Making Cities Resilient (Ten Essentials) (UNDRR 2020a) was conducted post-workshop.
2.5 Phase 3—Workshop Discussion about Survey 1 Results
The results obtained from Survey 1 formed the basis for the discussions during Phase 3 of the workshop. Participants deliberated on the strategies identified for clarification or removal, aiming to refine the selection for further investigation in Survey 2. Initially, the recommended cut-off for discussion was set at 10% of the participants selecting remove, this is where a minority can reverse the opinions of a majority (Xie et al. 2011; Ryan et al. 2018). However, due to the large number of participants and time constraints, this threshold was modified to 26% (Ryan et al. 2018). Removal does not mean items are not important; rather, it indicates there are other actions needed first. Strategies that scored over 21% were also considered for discussion. The final decision on removing, clarifying, and adding a strategy was determined through a collaborative consensus-building process (Hsu and Sandford 2007). These finalized strategies were then incorporated into Survey 2.
2.6 Phase 4—Survey 2
Survey 2 was designed for participants to rank the significance of each strategy in terms of strengthening resilience. Survey 2 was anonymous and included a repetition of the demographics-related questions from Survey 1. This approach ensured that demographic information was consistently collected, allowing for comprehensive data analysis. Similar to Survey 1, participants provided their perspective on application of the Scorecard and categorized the rankings for each strategy listed in Survey 2 as “Very Low,” “Low,” “Medium,” “High,” or ‘Very High” in terms of their importance on strengthening resilience. The survey results were used to inform the final discussion of the workshop, which focused on reaching consensus about the priority strategies. Here again, the survey results were analyzed through a descriptive analysis conducted by Fontenot, Ryan, Noel, and Garner immediately following the survey. Demographic data were subsequently analyzed post-workshop along with grouping the strategies by the Ten Essentials (UNDRR 2020a).
2.7 Phase 5—Consensus on Priority Strategies
In line with the consensus approach, strategies were identified as priorities for discussion if there was a general agreement of a substantial majority (Linde et al. 2005). Initially, this agreement was set at 75% or more when considering rankings of “High” or “Very High.” However, after careful consideration of the survey results and extensive discussions with the workshop participants, the agreed-upon cut-off was adjusted to 73% (Linde et al. 2005; Ryan et al. 2018). Strategies that met or exceeded this threshold were subject to thorough discussion, and consensus was achieved. This increased the strategies considered and discussed from 19 to 22. These prioritized strategies aligned with strengthening the resilience of public health systems and reflected the specific context in which the Scorecard was applied. Subsequently, the results were reviewed, interpreted, and evaluated post-workshop by Fontenot, Ryan, Kako, Fink, and Garner. All actively participated in the workshop.
2.8 Ethics
This study was determined by the Baylor University Institutional Review Board (IRB Reference #1792629) to be exempt from review by the institutional review board. Participants were invited to attend the workshop and provided oral consent to participate.
3 Results
This section describes the workshop demographics, perspectives on the Scorecard Approach, survey results, and the prioritized strategies.
3.1 Demographics
There were 19 participants in the Incheon workshop (Table 1). This group consisted of 11 females and eight males. Participants were based in three WHO regions: the Americas (n = 12), Western Pacific (n = 4), and Europe (n = 3). The participants represented a variety of disciplines/roles. This included academia, communications, disaster risk management, environmental health, epidemiologist, education, public health, doctor, international development, linguist, nurse, president, public officer, research assistant, and vice-chancellor. Among these participants, 13 (76.5%) had directly participated in disaster response. Over half of the participants (n = 10, 52.6%) had over 21 years of work experience and 15.8% (n = 3) had 6–10 years of experience. The number of participants with 1–2 years and 16–20 years of experience was the same at 10.5% (n = 2), while 5.3% (n = 1) of participants had either 3–5 years or 11–15 years of experience.
3.2 Perspective on the Scorecard Approach
Participants were asked three questions about the Scorecard use and effectiveness in both surveys using a Likert scale of 0 (lowest) to 5 (highest). The first question focused on the Scorecard’s effectiveness in identifying gaps in the resilience of the public health system with 90.7% (81.3% Survey 1; 100% Survey 2) of the participants rating this as either a 4 (25.0% Survey 1; 35.3% Survey 2) or 5 (56.3% Survey 1; 64.7% Survey 2). Question two related to ease of use, with 68.5% (64.7% Survey 1; 72.2% Survey 2) giving scores of 4 (41.2% Survey 1; 44.4% Survey 2) or 5 (23.5% Survey 1; 27.7% Survey 2). The third question related to effectiveness of the Scorecard in unifying stakeholders across various sectors with 84.8% of participants (81.3% Survey 1; 88.3% Survey 2) giving this a 4 (25.0% Survey 1; 41.2% Survey 2) or 5 (56.3% Survey 1; 47.1% Survey 2).
For Survey 1, one participant put N/A for questions one and three, and data were missing from two surveys. For Survey 2, two respondents indicated N/A for questions one and three, one person indicated 3.5 for question one and 4.5 for question two, these scores were counted as 4 and 5, respectively, and a participant indicated N/A for question two.
The surveys included an opportunity for participants to provide general comments or suggestions. Examples included: “The needs of the ‘left out’ parts of a community, such as the economically disadvantaged, poorly housed, minority groups […] need to be part of scorecard […] these people groups often make up the most heavily affected by disaster”; “Some of the recommendations should be a part of the ICS and ESF that is followed up by PHEP and emergency management teams already”; “Develop communication materials to educate general people”; “these strategies should be extended to the national level”; “[…] other terms should be used included marginalized, underserved, vulnerable and disenfranchised”; “the tool is very useful for local planning. It would include elements of adaptation to climate change with greater emphasis on public health policy decision-making”; and “showing and exchanging information is excellent way of learning.”
3.3 Survey 1
In Survey 1, participants evaluated each of the 70 strategies and decided whether they should be kept, removed, or clarified. The scores are presented in Table 2, and have been grouped by “The Ten Essentials for Making Cities Resilient” (UNDRR 2020a) identified in the Scorecard along with the indicator/question number. Based on the responses, 39 strategies were suggested for removal by at least 26% of the participants. Consistent with the feedback, these strategies were excluded from Survey 2 without further discussion. Seven strategies had removal indications from 21 to 25% of participants. These were put forward for group discussion. A consensus threshold of 10% was established to finalize decisions on these strategies. Voting occurred via a show of hands, and the outcomes were unanimous. Consequently, 24 out of the 70 strategies (34.3%) were retained for Survey 2 without debate, and after group discussion, seven more were added, resulting in 31 strategies for Survey 2.
3.4 Survey 2
Participants completed Survey 2 to prioritize the 31 strategies based on their importance in enhancing resilience. Of these, 11 had rankings for “High” or “Very High” that, when combined, were below 75%. However, during the analysis, it was determined that strategies with a combined “High” and “Very High” ranking of more than 73% should still be considered. As a result of this criterion, eight strategies were removed without further discussion, while three were discussed. After discussing, consensus was reached on a set of 23 strategies, as outlined in Table 3.
Through this process all Ten Essentials were represented, except 4 and 6. The essentials with the most priority actions were 7 and 9, which indicates that these areas require the most development in terms of public health system resilience. Meanwhile, the exclusion of essentials 4 and 6 in the priority strategies demonstrates participants believe that these were the most developed in the context of resilience.
The top 10 strategies with the highest priority are listed below with the strategy reference number listed in parentheses with Table 2 for Survey 1 and Table 3 for Survey 2. Over 89% of the participants selected these as “High” or “Very High” in terms of strengthening public health system resilience.
-
(1)
Municipal emergency plans should consider epidemic outbreaks, pandemics, famine, water shortages, massive accidents such as plane crashes, car accidents on highways, or other disasters (reference 6 in Tables 2 and 3 in Table 3).
-
(2)
Planning for disaster response and recovery actions relating to public health should integrate multidisciplinary teams and include all municipal departments (reference 5 in Table 2 and 4 in Table 3).
-
(3)
Establish a process for continuing services for at risk populations during and after an emergency (reference 58 in Table 2 and 23 in Table 3).
-
(4)
Assess what funding is available to maintain public health services during and after a disaster (reference 12 in Table 2 and 6 in Table 3).
-
(5)
Linking high-risk populations living at home with preexisting conditions with electronic databases used for sharing information and notifications (reference 55 in Table 2 and 24 in Table 3).
-
(6)
Assess to what extent hospitals and emergency care centers can manage a sudden influx of patients (reference 49 in Table 2 and 20 in Table 3).
-
(7)
Establish local public health plans to mitigate the long-term effects of disease outbreaks, pandemics, and disasters. The plans should be integrated with national guidelines and scientific recommendations (reference 68 in Table 2 and 31 in Table 3).
-
(8)
Mapping the entire public health network, including pharmacies and other services beyond clinical treatment and care (reference 4 in Table 2 and 5 in Table 3).
-
(9)
Establish, maintain, strengthen, and restore multisector mechanisms to support local public health services in an emergency situation (reference 57 in Table 2 and 25 in Table 3).
-
(10)
Integrate local transport into the disaster system to supply personal protective equipment and other nonperishable items to remote hospitals, medical centers, and other allied health organizations (reference 61 in Table 2 and 26 in Table 3).
4 Limitations
Caution should be exercised when applying these findings. The prioritization of local strategies must be conducted locally through the leadership and participation of communities and local stakeholders. They will differ across communities and countries due to variations in the local risk profiles, health system architectures, capabilities, political contexts, and many other factors. Additionally, a modified Delphi process is subjective, but to mitigate bias there were two surveys and two rounds of consensus-based discussion. Therefore, all the strategies explored, ranked, and prioritized in this study provide a set of options for decision makers, practitioners, and funders.
A limitation was self-selection bias arising from the use of purposive and snowball sampling methods. For example, there were gaps in the spatial dispersal of participants. However, these sampling techniques were specifically chosen because they allow for targeted selection of individuals who possess deep expertise or firsthand experience in public health systems, disaster management, and resilience. By focusing on these informed participants, the research aimed to capture nuanced insights and complex understandings often missed in broader sampling strategies. Furthermore, although there is an inherent risk of bias, the rich data and unique perspectives derived from such samples often provide invaluable context and depth.
While the workshop set a threshold of 26% representing the participants who rated a strategy as “High” or “Very High” for examining the results from Survey 1, the widely accepted cut-off is 10%. This benchmark indicates a level at which a minority can sway the views of the majority. This cut-off was adjusted due to the number of previous workshops, participants, and time limitations. To address this constraint, participants were given the chance to discuss strategies openly. Additionally, there were opportunities to confirm and validate these strategies during workshop discussions.
The study direction was shaped by the contributions and expertise of the authors. To mitigate potential biases arising from individual perspectives, the authors represented a diverse, multinational, and interdisciplinary team. Specifically, the team of authors comprised professionals from academia, disaster risk reduction, emergency management, environmental health science, health promotion, information technology, medicine, nursing, and public health.
Strategies presented and discussed were identified through the past workshops conducted during the COVID-19 pandemic. This limitation was reduced due to the unique nature of the situation. The COVID-19 pandemic provided an opportunity to understand the impact of an ongoing event along with other hazards that may have emerged.
5 Discussion
Using the Scorecard in conjunction with a modified Delphi process is a versatile method for identifying, ranking, and prioritizing resilience strategies across urban settings in multiple countries (Ryan, Kako, et al. 2023; Ryan, Telford, et al. 2023). This method provides an opportunity to holistically understand the resilience of the public health system by exploring its complexity, subsystem interactions, and the ability to identify strengths and weaknesses affecting multiple factors (Williams and Sands 2020; Dianat et al. 2021). The interactive and participatory nature of the workshops also provided the opportunity to leverage participant knowledge to complete the Scorecard and ultimately inform future public health system preparedness strategies and processes (Ørngreen and Levinsen 2017; Sanchez-Betancourt and Vivier 2019; Shamsuddin et al. 2021; Ryan, Kako, et al. 2023).
Adaptive capacity is vital to managing risks and a wide range of potential events, especially in low-resource settings (Barasa et al. 2018; Forsgren et al. 2022). This would require better collaboration between sectors using an integrated approach and organizational learning (Forsgren et al. 2022). The Scorecard provides a mechanism for this to occur by allowing interested parties to explore key gaps, also setting the scene for public health (Assefa et al. 2021). This is where understanding the structure and functions of systems can support prediction and understanding of risks within subpopulations to identify the number of at-risk individuals in order to inform preventive and protective strategies for the risks associated with a disease outbreak, pandemic, or disaster (Dolley 2018; Assefa et al. 2021). One example of how this could occur is wastewater surveillance of diseases and wastewater-based epidemiology (Langan et al. 2022). Such data would require a multidisciplinary team working across professional boundaries to provide lead indicators of infectious diseases to public health leaders who could adapt and adjust strategies in advance of emerging outbreaks (Langan et al. 2022; Ryan et al. 2022).
The Scorecard has the potential to be applied using in-person or virtual workshops. However, we found that the Scorecard was best applied using in-person workshops. When used in virtual settings, the effectiveness of the Scorecard in encouraging discussion and feedback was compromised. This may be why 68.5% of the participants gave a score of 4 or 5 for ease of use. The Waco and Dallas workshops tested an App version of the Scorecard. It worked well as a test but not all participants could download and/or use the App. This technology could not be tested outside the United States. Consistent with another study that tested a similar App, the participants who could use this technology found it very beneficial, which demonstrates the potential for the approach (Ryan, Telford, et al. 2023). Based on this observation, an easy to use and accessible App will facilitate virtual application of the Scorecard.
The benefits of bringing the results from multiple locations together using a modified Delphi method were demonstrated during the joint workshop. Strategies 15, 34, 57, and 63 in Table 2 were identified as needing discussion from Survey 1, and without this process would not have been included in Survey 2. Consensus was reached after Survey 2 that these strategies, which became numbers 7, 11, 25, and 29 in Table 3, were priorities. After ranking these strategies using scoring from Survey 2, number 57 (Table 2) and later 25 (Table 3) came in the top 10 at number nine. This approach would support progress towards a model for more effective decision making and resource allocation based on the assessed priority needs of health systems and communities, building stronger population-based models for disaster risk management systems (Burkle et al. 2021a; Burkle et al. 2021b).
To build on this work, a validated process is needed to guide the application of the Scorecard through to the development of multiyear action plans. This would incorporate more components of the Health EDRM Framework to help ensure the inclusion of data sharing, open governance, cross-organizational, and inter-platform collaboration (Chan et al. 2022). Such plans would be designed for decision makers to consider when prioritizing policies and funding for disaster risk reduction. For example, the process could go beyond the top 10 priorities generated at our workshop and include the following steps:
-
(1)
Identification of Scorecard leads and workshop sites.
-
(2)
Application of the Scorecard and prioritization of actions during the workshop. This is scalable and could be at local or provincial/state levels.
-
(3)
Workshop report with priority actions presented to decision makers and/or public health leaders. The workshop participants would be provided with the opportunity to review prior to finalization.
-
(4)
If there were multiple applications of the Scorecard, conduct a joint workshop using a modified Delphi process to interpret, rank, and prioritize strategies identified where the Scorecard was applied.
-
(5)
Integration of the findings from steps 2 and 4 to develop multiyear action plans for the priorities identified. This would be tailored to local, provincial/state, and national levels.
This process would provide a pipeline for ongoing improvement of public health systems. For example, it would be aligned with the Ten Essentials, focus on local level transdisciplinary engagement, and help establish, measure, and calibrate baseline resilience, priorities, and actions overtime. Another benefit is that the Scorecard is scalable and translatable to various scenarios and settings due to the level of granularity in identifying strengths and prioritizing areas for improvement (Jones et al. 2021). Application of this approach would help public health leaders, emergency managers, and others involved in disaster risk management consider many aspects of resilience. For example, the vulnerabilities and capabilities of services, organizations, logistics, and ecosystems could be explored individually and collectively and applied to disease outbreaks, pandemics, and disasters. Routine application of the Scorecard would also help sustain gains in resilience across sectors at local, provincial/state, and national levels during a climate of rapidly shifting social, environmental, and fiscal challenges.
6 Conclusion
The Scorecard provides a versatile method for identifying, ranking, and prioritizing a set of strategies for strengthening the resilience of public health systems. This study demonstrated how this tool can be scaled and translated into various scenarios and settings. By applying this process, a range of options were identified. These included mapping the interconnectedness of health facilities and assets, and developing plans to scale and deploy nonhospital staff to complement treatment and care, as well as to expand surge capacity, improve the linking of public and private health systems to mitigate surge risks, and include multidisciplinary teams in public health activities. The integration of local transport and multisector systems is also important for prevention, preparedness, response, and recovery. Routine application of the Scorecard could help establish, measure, and calibrate resilience needs, priorities, and achievements overtime, which is increasingly important due to rapidly shifting social, environmental, and fiscal challenges. Using the method applied, a range of locally led resilience strategies can be identified and developed to prioritize current and future policies, actions, and funding to address disaster risks using a whole-of-society and all-hazards approach.
References
Abbas, J. 2021. Crisis management, transnational healthcare challenges and opportunities: The intersection of COVID-19 pandemic and global mental health. Research in Globalization 3: Article 100037.
Algaissi, A.A., N.K. Alharbi, M. Hassanain, and A.M. Hashem. 2020. Preparedness and response to COVID-19 in Saudi Arabia: Building on MERS experience. Journal of Infection and Public Health 13(6): 834–838.
Arsenault, C., A. Gage, M.K. Kim, N.R. Kapoor, P. Akweongo, F. Amponsah, A. Aryal, and D. Asai et al. 2022. COVID-19 and resilience of healthcare systems in ten countries. Nature Medicine 28(6): 1314–1324.
Assefa, Y., C.F. Gilks, R. van de Pas, S. Reid, D.G. Gete, and W.V. Damme. 2021. Reimagining global health systems for the 21st century: Lessons from the COVID-19 pandemic. BMJ Global Health 6(4): Article e004882.
Assefa, Y., C.F. Gilks, S. Reid, R. van de Pas, D.G. Gete, and W. Van Damme. 2022. Analysis of the COVID-19 pandemic: Lessons towards a more effective response to public health emergencies. Globalization and Health 18(1): Article 10.
Barasa, E., R. Mbau, and L. Gilson. 2018. What is resilience and how can it be nurtured? A systematic review of empirical literature on organizational resilience. International Journal of Health Policy and Management 7(6): Article 491.
Behrens, D.A., M.S. Rauner, and M. Sommersguter-Reichmann. 2022. Why resilience in health care systems is more than coping with disasters: Implications for health care policy. Schmalenbach Journal of Business Research 74(4): 465–495.
Burkle, F.M., D.A. Bradt, J. Green, and B.J. Ryan. 2021. Global public health database support to population-based management of pandemics and global public health crises, part II: The database. Prehospital and Disaster Medicine 36(1): 105–110.
Burkle, F.M., D.A. Bradt, and B.J. Ryan. 2021. Global public health database support to population-based management of pandemics and global public health crises, part I: The concept. Prehospital and Disaster Medicine 36(1): 95–104.
Chan, E.Y.Y., D. Guha-Sapir, C. Dubois, R. Shaw, and C.S. Wong. 2022. Challenges of data availability and use in conducting health-EDRM research in a post-COVID-19 world. International Journal of Environmental Research and Public Health 19(7): Article 3917.
Dalkey, N.C., B.B. Brown, and S. Cochran. 1969. The Delphi method: An experimental study of group opinion. Santa Monica: Rand Corporation.
Dianat, H., S. Wilkinson, P. Williams, and H. Khatibi. 2021. Planning the resilient city: Investigations into using “causal loop diagram” in combination with “UNISDR scorecard” for making cities more resilient. International Journal of Disaster Risk Reduction 65: Article 102561.
Dolley, S. 2018. Big data’s role in precision public health. Frontiers in Public Health 6: Article 68.
Engzell, P., A. Frey, and M.D. Verhagen. 2021. Learning loss due to school closures during the COVID-19 pandemic. Proceedings of the National Academy of Sciences 118(17): Article e2022376118.
Feitelson, E., P. Plaut, E. Salzberger, D. Shmueli, A. Altshuler, S. Amir, and M. Ben-Gal. 2022. Learning from others’ disasters? A comparative study of SARS/MERS and COVID-19 responses in five polities. International Journal of Disaster Risk Reduction 74: Article 102913.
Forsgren, L., F. Tediosi, K. Blanchet, and D.D. Saulnier. 2022. Health systems resilience in practice: A scoping review to identify strategies for building resilience. BMC Health Services Research 22(1): Article 1173.
Glaser, B.G., and A.L. Strauss. 1967. The discovery of grounded theory: Strategies for qualitative research. Hawthorne, NY: Aldine de Gruyter.
Global Preparedness Monitoring Board. 2020. A world in disorder: Global preparedness monitoring board annual report 2020. https://apps.who.int/iris/bitstream/handle/10665/351720/9789240014121-eng.pdf?sequence=1. Accessed 18 Jan 2023.
Haldane, V., C. De Foo, S.M. Abdalla, A.-S. Jung, M. Tan, S. Wu, A. Chua, and M. Verma et al. 2021. Health systems resilience in managing the COVID-19 pandemic: Lessons from 28 countries. Nature Medicine 27(6): 964–980.
Hassan, S.-U.-N., A. Zahra, N. Parveen, F. Khatoon, N.A. Bangi, and H. Hosseinzadeh. 2022. Quality of life and adherence to healthcare services during the COVID-19 pandemic: A cross-sectional analysis. Patient Preference and Adherence 16: 2533–2542.
Hasson, F., S. Keeney, and H. McKenna. 2000. Research guidelines for the Delphi survey technique. Journal of Advanced Nursing 32(4): 1008–1015.
Hogan, A.B., B.L. Jewell, E. Sherrard-Smith, J.F. Vesga, O.J. Watson, C. Whittaker, A. Hamlet, and J.A. Smith et al. 2020. Potential impact of the COVID-19 pandemic on HIV, tuberculosis, and malaria in low-income and middle-income countries: A modelling study. The Lancet Global Health 8(9): e1132–e1141.
Hsu, C.-C., and B.A. Sandford. 2007. The Delphi technique: Making sense of consensus. Practical Assessment, Research & Evaluation 12(10): 1–8.
Jain, R., and P. Dupas. 2022. The effects of India’s COVID-19 lockdown on critical non-COVID health care and outcomes: Evidence from dialysis patients. Social Science & Medicine 296: Article 114762.
Jones, K., F. Pascale, N. Wanigarathna, M. Morga, and S. Sargin. 2021. Critical evaluation of the customisation process of the UNDRR disaster resilience scorecard for cities to earthquake-induced soil liquefaction disaster events. Bulletin of Earthquake Engineering 19(10): 4115–4143.
Kruk, M.E., M. Myers, S.T. Varpilah, and B.T. Dahn. 2015. What is a resilient health system? Lessons from Ebola. The Lancet 385(9980): 1910–1912.
Langan, L.M., M. O’Brien, Z.C. Rundell, J.A. Back, B.J. Ryan, C.K. Chambliss, R.S. Norman, and B.W. Brooks. 2022. Comparative analysis of RNA-extraction approaches and associated influences on RT-qPCR of the SARS-CoV-2 RNA in a university residence hall and quarantine location. ACS ES&T Water 2(11): 1929–1943.
Linde, H. van der, C.J. Hofstad, J. van Limbeek, K. Postema, and J.H. Geertzen. 2005. Use of the Delphi technique for developing national clinical guidelines for prescription of lower-limb prostheses. Journal of Rehabilitation Research and Development 42(5): Article 693.
Mohammadpour, M., E. Zarifinezhad, A. Ghanbarzadegan, K. Naderimanesh, N. Shaarbafchizadeh, and P. Bastani. 2021. Main factors affecting the readiness and responsiveness of healthcare systems during epidemic crises: A scoping review on cases of SARS, MERS, and COVID-19. Iranian Journal of Medical Sciences 46(2): 81–92.
Msemburi, W., A. Karlinsky, V. Knutson, S. Aleshin-Guendel, S. Chatterji, and J. Wakefield. 2022. The WHO estimates of excess mortality associated with the COVID-19 pandemic. Nature 613(7942): 130–137.
Ørngreen, R., and K. Levinsen. 2017. Workshops as a research methodology. Electronic Journal of E-Learning 15(1): 70–81.
Prommas, P., K.S. Lwin, Y.C. Chen, M. Hyakutake, C. Ghaznavi, H. Sakamoto, H. Miyata, and S. Nomura. 2023. The impact of social isolation from COVID-19-related public health measures on cognitive function and mental health among older adults: A systematic review and meta-analysis. Ageing Research Reviews 85: Article 101839.
Roberton, T., E.D. Carter, V.B. Chou, A.R. Stegmuller, B.D. Jackson, Y. Tam, T. Sawadogo-Lewis, and N. Walker. 2020. Early estimates of the indirect effects of the COVID-19 pandemic on maternal and child mortality in low-income and middle-income countries: A modelling study. The Lancet Global Health 8(7): e901–e908.
Rollston, R., and S. Galea. 2020. COVID-19 and the social determinants of health. Los Angeles: SAGE Publications.
Ryan, B.J., R.C. Franklin, F.M. Burkle, E.C. Smith, P. Aitken, K. Watt, and P.A. Leggat. 2018. Ranking and prioritizing strategies for reducing mortality and morbidity from noncommunicable diseases post disaster: An Australian perspective. International Journal of Disaster Risk Reduction 27: 223–238.
Ryan, B.J., D. Coppola, D.V. Canyon, M. Brickhouse, and R. Swienton. 2020. COVID-19 community stabilization and sustainability framework: An integration of the Maslow hierarchy of needs and social determinants of health. Disaster Medicine and Public Health Preparedness 14(5): 623–629.
Ryan, B.J., M.P. Muehlenbein, J. Allen, J. Been, K. Boyd, M. Brickhouse, B.W. Brooks, and M. Burchett et al. 2022. Sustaining university operations during the COVID-19 pandemic. Disaster Medicine and Public Health Preparedness 16(5): 1901–1909.
Ryan, B., M. Kako, R. Fink, P. Şimşek, P. Barach, J. Acosta, S. Bhatia, M. Brickhouse, et al. 2023. Strategies for strengthening the resilience of public health systems for pandemics, disasters, and other emergencies. Disaster Medicine and Public Health Preparedness 17: Article e479.
Ryan, B.J., V. Telford, M. Brickhouse, J. Acosta, C. Allen, S. Bhatia, J. Campbell, and C. Crowe et al. 2023. Strengthening food systems resilience before, during and after disasters and other crises. Journal of Homeland Security and Emergency Management. https://doi.org/10.1515/jhsem-2022-0028.
Sagan, A., S.L. Greer, E. Webb, M. McKee, N.A. Muscat, S. Lessof, I. de la Mata, and J. Figueras. 2022. Strengthening health system resilience in the COVID-19 era. Eurohealth 28(1): 4–8.
Sanchez-Betancourt, D., and E. Vivier. 2019. Action and community-based research: Improving local governance practices through the community scorecard. In Transforming research methods in the social sciences: Case studies from South Africa, ed. S. Laher, A. Fynn, and S. Kramer, 375–392. Johannesburg: Wits University Press.
Shamsuddin, A., A. Sheikh, and R.N. Keers. 2021. Conducting research using online workshops during COVID-19: Lessons for and beyond the pandemic. International Journal of Qualitative Methods. https://doi.org/10.1177/1609406921104374.
Splinter, M.J., P. Velek, M.K. Ikram, B.C.T. Kieboom, R.P. Peeters, P.J.E. Bindels, M.A. Ikram, F.J. Wolters, et al. 2021. Prevalence and determinants of healthcare avoidance during the COVID-19 pandemic: A population-based cross-sectional study. PLoS Medicine 18(11): Article e1003854.
Tambo, E., I.C. Djuikoue, G.K. Tazemda, M.F. Fotsing, and X.-N. Zhou. 2021. Early stage risk communication and community engagement (RCCE) strategies and measures against the coronavirus disease 2019 (COVID-19) pandemic crisis. Global Health Journal 5(1): 44–50.
UNDRR (United Nations Office for Disaster Risk Reduction). 2020a. Disaster resilience scorecard for cities—Public health system resilience addendum. Geneva: United Nations Office for Disaster Risk Reduction. https://mcr2030.undrr.org/public-health-system-resilience-scorecard. Accessed 11 March 2023.
UNDRR (United Nations Office for Disaster Risk Reduction). 2020b. NDRR GETI and WHO Webinar—Resilience of local governments: A multi-sectoral approach to integrate public health and disaster risk management. Geneva: United Nations Office for Disaster Risk Reduction. https://www.undrr.org/event/undrr-geti-and-who-webinar-resilience-local-governments-multi-sectoral-approach-integrate. Accessed 11 Mar 2023.
UNDRR (United Nations Office for Disaster Risk Reduction). 2023. Resilience. https://www.preventionweb.net/terminology/resilience. Accessed 16 Sept 2023.
Vlachos, J., E. Hertegård, and H. Svaleryd. 2021. The effects of school closures on SARS-CoV-2 among parents and teachers. Proceedings of the National Academy of Sciences 118(9): Article e2020834118.
WHO (World Health Organization). 2019. Health emergency and disaster risk management framework. Geneva: World Health Organization.
WHO (World Health Organization). 2023. Systematically identifying and evaluating strategies for strengthening community resilience. Geneva: World Health Organization Centre for Health Development. https://extranet.who.int/kobe_centre/en/project-details/h-edrm_2021_BU. Accessed 17 Aug 2023.
Williams, P., and D. Sands. 2020. UN city disaster resilience scorecard. In Resilience: An edited collection of authored pieces comparing, contrasting, and integrating risk and resilience with an emphasis on ways to measure resilience, ed. M.-V. Florin, and I. Linkov, 248–253. Lausanne, Switzerland: École polytechnique fédérale de Lausanne, International Risk Governance Center (IRGC). https://infoscience.epfl.ch/record/228206?ln=en. Accessed 3 Nov 2023.
Wright, N., L. Fagan, J.M. Lapitan, R. Kayano, J. Abrahams, Q. Huda, and V. Murray. 2020. Health emergency and disaster risk management: Five years into implementation of the Sendai framework. International Journal of Disaster Risk Science 11(2): 206–217.
Xie, J., S. Sreenivasan, G. Korniss, W. Zhang, C. Lim, and B.K. Szymanski. 2011. Social consensus through the influence of committed minorities. Physical Review E 84(1): Article 011130.
Acknowledgments
Thank you to the World Health Organization Centre for Health Development (WHO Kobe Centre), United Nations Office for Disaster Risk Reduction (UNDRR) Global Education and Training Institute, and the UNDRR Regional office for the Americas and Caribbean for supporting this project. The authors also acknowledge workshop participants for their willingness to provide advice and input on the priorities for strengthening public health system resilience. This research was supported by the World Health Organization Centre for Health Development (WHO Kobe Centre—WKC: K21002). The Scientific and Technological Research Institution of Turkey (TÜBİTAK) also provided support for the workshops in Turkey.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Ryan, B.J., Kako, M., Garner, S. et al. Prioritizing Strategies for Building the Resilience of Public Health Systems to Disasters Across Multiple Communities and Countries. Int J Disaster Risk Sci 15, 1–17 (2024). https://doi.org/10.1007/s13753-024-00537-x
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13753-024-00537-x