Background

Rationale for the study

Simulation-based education (SBE) is not a technology, but a learner-centered pedagogical method based on learning theories [1]. The greatest benefit of SBE is that it enables repeated training in a safe environment resembling an actual hospital setting [2]. For example, students can experience cases in which they cannot be directly involved in a clinical setting, such as providing care for a psychiatric patient exhibiting dangerous behaviors or end-of-life care for patients and their families [1]. Moreover, training that requires a more realistic setting, such as dissection, can be performed using immersive virtual reality [3]. As shown here, SBE can be designed with the desired scenario contents based on the learning objectives, and patient information and simulators can be varied to provide different SBE [1].

Simulation-based education helps nursing students to establish their professional identity by experiencing the roles of a nurse in advance [4], and question and-answer sessions and discussions with the instructor during debriefing after the training allows students to engage in self-reflection, through which they can integrate their learned materials and translate them into practice [5]. Due to these benefits, SBE supplements clinical practicum across all topics. Recently, it’s especially advised for situations where students can’t directly interact, like pediatric vaccinations, asthma treatments, and mother-infant cases [1].

As a result of the coronavirus disease 2019 (COVID-19) pandemic that struck the world in 2019, clinical practicum was either suspended or stopped for patient and student safety, and students expressed anxiety about potentially contracting the infection from patients or other students during clinical practicum [6], further highlighting the need for SBE. Moreover, pediatric nursing clinical practicum is very challenging in the Republic of Korea (ROK) compared with other clinical practicums. The ROK is one of the countries with the lowest fertility rates, and it has the most quickly declining cumulative birth rate and total fertility rate among 37 organization for economic cooperation and development (OECD) countries, with an average annual drop of 3.1%. In addition, the number of neonates has dropped dramatically from 490,000 to 2012 to 260,000 in 2021 [7]. Moreover, the number of high-risk neonates vulnerable to infection and injury is on the rise, from 18,232 to 1995 to 30,462 in 2015 [7], which further hinders students from encountering divers even if clinical practicum courses are offered.

A systematic review of studies that conducted a cost analysis for SBE reported that the most common topic—following surgery cases—was pediatrics and obstetrics and gynecology, and that most studies were conducted in low-income countries, with common topics being neonatal and maternal health care, such as “Helping Babies Breathe” (HBB) and “Essential Newborn Care” (ENC) [2]. As shown here, pediatric health is a very important topic of SBE not only in countries with low fertility rates but also in low-income countries. Providing pediatric nursing clinical practicum is very challenging due to the declining number of newborns, increasing incidence of high-risk births, and high cost associated with SBE.

To address these issues, a growing number of studies have evaluated the effects of SBE; however, the types of SBE studied vary widely, and the validity and reliability of scenarios and contents of SBE have not been adequately evaluated. Furthermore, diverse outcome measures have been used and standardized instruments are lacking [4, 8, 9]. The validity of the simulation was described as the degree to which the simulation accurately represented the target task, and the reliability of the simulation was described as the degree to which simulation education was measured using the same method each time the same participants received education under the same conditions [10]. Because simulation is an educational method that enables nursing educators to facilitate and assess learners’ clinical competencies [1], educators must develop valid and reliable scenarios and assess learners using standardized instruments.

There are several types of simulators available, including standard patients, high-fidelity simulators, low-fidelity simulators, and partial task simulators. Instructors choose the type of simulator based on the objectives of SBE. Consequently, the use of an ineffective simulator may curtail the effectiveness of education [1].

As shown here, past systematic reviews of studies on SBE have primarily conducted technical analyses of educational methods and target populations, with a lack of systematic reviews on the contents of SBE. In this context, we conducted a systematic review to examine the characteristics of pediatric simulation-based education (P-SBE) and evaluate the validity and reliability of the development process. The findings of this study will shed light on the direction of future SBE programs and interventions and establish criteria for validity and reliability evaluations of simulation scenarios and programs.

Research questions

This study was a systematic review of past studies that have developed and evaluated the effects of P-SBE. The findings of this study will be used as criteria for evaluating the validity and reliability of future P-SBE. The specific research questions were as follows:

  1. 1.

    Review the characteristics of studies that developed and evaluated the effects of P-SBE.

  2. 2.

    Identify the characteristics of scenarios used in P-SBE.

  3. 3.

    Evaluate the validity and reliability of the process of developing P-SBE.

  4. 4.

    Evaluate the validity and reliability of instruments used to assess the effects of P-SBE.

Methods

Study design

This study conducted a systematic review of P-SBE, specifically examining the general characteristics of the studies, topics of education, simulation methods, reliability and validity of simulation, and dependent variables. The key question selection, literature selection based on inclusion and exclusion criteria, data extraction, setting of scope of literature search and search databases, quality appraisal, and risk of bias assessment were performed in accordance with the Preferred Reporting Items of Systemic Reviews and Meta-Analysis (PRISMA) 2020 statement [11] and 2022 Cochrane Handbook for Systematic Reviews of Interventions version 6.3 [12], and data were analyzed. We classified the characteristics of literature based on typology, referencing the definition of “Simulation Typologies/Modalities” provided by Palaganas et al. in 2020 [13].

Key questions and selection criteria

The key questions of this study were: “What is the construction of P-SBE?” and “What aspects are assessed in P-SBE?”. The specific inclusion criteria were as follows:1) studies that developed a simulation program or scenario, 2) pediatric scenarios, and 3) health and health care-related scenarios (not necessarily in clinical settings, but including events such as traffic accidents, bee stings, bicycle accidents, daily life shocks, etc., these criteria were included in the third round of literature screening). The exclusion criteria were as follows:1) studies on non-human simulations (even if they are related to pediatrics, studies about the development of simulators, etc., were excluded), 2) non-pediatric scenarios, and 3) studies on non-human simulations (even if they are related to pediatrics, studies about the development of simulators, etc., were excluded). The search strategy was established based on the PICO-SD framework for non-Korean databases: “(simulat* or scenario*) and (pediatric or child or children or baby or newborn or infant or kid*) and (valid* or reliab*).

Literature search and selection process

Two researchers independently performed a literature search. The search was conducted from May 23, 2022, to May 28, 2022. The MEDLINE, EMBASE, CINAHL, and Cochrane Library databases were selected according to the PRISMA statement. An advanced search was performed based on the participants, intervention, comparison, outcome, and study design (PICO-SD) framework. In addition, a search was performed using Google Scholar to include as many gray articles as possible.

The criteria for the initial screening were set according to the PICO-SD framework. We did not define a specific participant population and included healthcare providers, nurses, and nursing students. As for the intervention, all P-SBE programs were included. The outcome variables were not specified. For the study design, we included all studies that observed effective outcomes after administering an SBE program, and studies that presented data for the validity and reliability of the scenario and instruments. A total of 1,309 studies were selected during the initial screening and 764 duplicates were excluded. In the second round of screening, the titles and abstracts of 545 studies were independently reviewed by three researchers based on the PCIO-SD criteria. In total, 292 studies were excluded. In the third round of screening, the full texts of the selected studies were obtained, and full texts of 253 studies were available. Of these, 111 studies did not meet the inclusion criteria and were excluded. From the resulting 142 studies, 44 were excluded from the content analysis because they were proceedings and did not show the details of the scenarios. Thus, 98 studies were included in the content analysis. Each researcher independently evaluated the quality of the papers using the Mixed Methods Appraisal Tool (MMAT), 2018 [14]. Only papers with moderate-to high-quality ratings were included in the review. Any disagreements among the researchers during this process were resolved by discussion. If the selected studies did not state the detailed study methodology, the researchers described it upon discussion (Fig. 1).

Fig. 1
figure 1

PRISMA flow diagram

Full size image

Data analysis

The 98 included studies were written as case reports, and qualitative analysis was performed using Excel 2016 software. The case reports contained information about general characteristics (authors, year, title, and country), study characteristics (study design, participant characteristics, simulator type, scenario topic, scenario reliability, and validity), and outcome characteristics (dependent variables, instruments used to measure dependent variables, and reliability and validity of dependent variables).

Study results

Characteristics of the studies

Table 1 presents the general characteristics of the included studies. A total of 142 studies pertinent to P-SBE were identified. Fifteen (10.6%) were published between 2001 and 2010, and 109 (76.8%) were published in the subsequent ten years, showing a more than seven-fold increase. The greatest number of studies were conducted in the United States (n = 62, 43.7%), followed by Canada (n = 17, 12.0%). Experimental studies, including randomized controlled trials (RCTs), were the most common (n = 76, 53.5%), followed by developmental studies, including simulation development (n = 58, 40.85%). According to simulation typology, advanced patient simulation was the most common (n = 92, 64.8%). Most studies used high-fidelity simulation only (n = 75, 52.8%), followed by video-based simulation, and four studies used VR simulation.

Table 1 General Characteristics of the selected studies (n = 142)
Full size table

Analysis of simulation scenario contents

A total of 98 studies were included in the analysis of the P-SBE scenario contents (Table 2). The most common target population of P-SBE was medical staff (n = 44, 44.9%), more specifically, there were 37 (37.8%) studies on medical students, medical residents, or medical fellows and seven (7.1%) studies on medical doctors or medical experts. Of the studies conducted on nursing staff, eight (8.2%) studies were conducted on nursing students, and three (3.1%) studies were conducted on registered nurses or experts. Four (4.1%) studies were conducted on children or students, and three (3.1%) studies were conducted on parents. The most common number of scenarios included in the analysis software was one (n = 49, 50.0%), followed by four (n = 13, 13.3%). The proficiency levels were competency (n = 29, 29.6%), proficient (n = 29, 29.6%), and expert (n = 10, 10.2%). Scenario contents included emergency intervention (n = 59, 60.2%), communication ability and decision-making (n = 19, 19.39%), and protection and safety (n = 17, 17.35%). Specific topics included pediatric rescue (n = 37, 37.8%), neonatal rescue (n = 11, 11.2%), and airway management (n = 8, 8.2%). Among the programs developed for children, two studies developed a simulation to enhance the decision-making ability of children with autism spectrum disorder (ASD) [15, 16], and programs developed for students targeted to train rescue competencies [17] and enhance decision-making ability in relation to cigarette smoking [18]. Seventy-two (73.5%) studies had self-developmental scenarios, and 23 (23.5%) had already been published. In terms of validity and reliability evaluation, 48 studies (49.0%) did not test validity, and 55 studies (56.1%) did not test reliability (Table 2). The most common type of validity tested was content validity (n = 10, 10.2%) and the most common type of reliability tested was inter-rater reliability (n = 10, 10.2%) (Table 3).

Table 2 Characteristics of Simulation program and scenarios (n = 98)
Full size table
Table 3 Specific analysis on the simulation program and scenario
Full size table

Outcome variables of simulation program

Of the studies that used one or more outcome variables, most (n = 65, 66.3%) used the skill category as the outcome variable, namely skills, performance, assessment, and communication skills. Twenty-six (26.5%) studies used the attitude category as the outcome variable, namely attitude, confidence, satisfaction, and stress. Seventeen (17.3%) studies have examined this knowledge. Fifty-six studies (57.1%) used one outcome variable and 31 (31.6%) used more than one outcome variable. Sixty-six (67.3%) studies did not test validity, while 50 (51.0%) did not test reliability (Tables 4 and 5).

Table 4 Outcome Variables of Scenarios (n = 98)
Full size table
Table 5 Specific analysis on the simulation program and scenario
Full size table

Discussion

SBE is recognized as an important field in health education [19], and its technology and field are being advanced and expanded at an astonishing pace [19]. In particular, the need for P-SBE is growing because pediatric patients require highly proficient skills, despite limited access by students in clinical settings [20]. In the present study, we conducted a systematic review to identify the characteristics of the P-SBE programs. We also examined the methods of validity and reliability testing in studies that developed the P-SBE programs. We aim to describe these topics based on the general characteristics of the research for discussion.

Navigating through the vast literature, a total of 142 studies on P-SBE were identified. While research in this field was limited prior to 2004 (n = 3, 2.1%), substantial research has been conducted from 2004 to the present (n = 139, 97.9%). In particular, there has been an increase in up to 20 studies since 2011. Simulations were introduced in medical and nursing education in the 1960s when mannequins that enable training of mouth-to-mouth breathing were developed; owing to advances in state-of-the-art technology and artificial intelligence, types of simulations, fields of application, and simulation scenarios have become increasingly similar to real-world situations, allowing for the achievement of special educational objectives [21]. Moreover, according to the IOM recommendation that education for healthcare providers must comprise evidence-based content and that new technology, such as team-based simulations, should be incorporated into the curriculum to provide safer and more effective treatment [22], SBE strategies are anticipated to be further expanded and advanced in the coming years.

Next, by country, there was the most active research in developed countries, including the United States, with 62 (43.7%) studies published in the United States, 17 (12.0%) studies in Canada, and 10 (7.0%) studies in the United Kingdom. This may be attributable to the fact that while national leaders, organizations, and accreditation bodies have spared no support from educators of healthcare providers in transforming the present and have served a central role in simulation education, SBE has advanced primarily around organizations such as the Society for Simulation in Healthcare (SSH) and International Nursing Association for Clinical Simulation and Learning (INACSL), which mostly includes developed countries [23]. In the future, education systems that provide P-SBE to healthcare providers should be expanded to countries with poor supportive networks.

Based on the study design, the most common type of study design was experimental, including RCTs (n = 33, 23.2%) and quasi-experimental studies (n = 18, 12.7%). The prominence of experimental designs emphasizes the scientific accuracy and commitment of the research community in producing evidence-based results in the field of P-SBE. The focus of current research mainly on the development and evaluation of simulation programs is a positive sign. This trend indicates the academic community values ensuring that P-SBE programs are not only innovative but also effective in delivering essential skills to healthcare providers. Even though such designs have been widely adopted, there is a need to consider mixed methods approaches in the future, capable of offering both quantitative data and deeper qualitative insights into learners’ experiences and perceptions. Additional research is necessary to assess not just the effectiveness but also the feasibility, accessibility, and scalability of P-SBE across diverse environments.

By simulation type, 92 studies used an advanced patient simulator and 52.5% used only a high-fidelity simulator. Next, 30 (32.6%) studies used computer-based training and 17 (20.2%) them used video-based simulations. Ten studies used a standardized patient (SP)/participant, and nine (6.3%) of them used an SP. This is because the key to simulation education for healthcare providers, which is defined as skills training, learning, assessment, testing or system, or platform for gaining an understanding of human behavior in a situation or environment that allows them to experience real-world cases [24], is how well it reflects reality, and high-fidelity simulators provide modifiable, realistic responses to the situation and learners’ input. The current level of technology allows high-fidelity simulators to precisely mimic human body functions and provide realistic responses, such as heart and lung sounds, chest movements, and detectable pulses, enabling learners to be integrated into patient scenarios that require their clinical judgment and practice proficiency [25]. Research utilizing VR or other games is rare. Such technology reflects real-world situations and can detect learners’ real-time responses to changes in the situation, but it is rarely used. In particular, the fact that 17 out of 30 (56.7%) studies on computer-based training used video-based simulations shows that this area requires further development.

The target audience for the scenario’s content could be determined through the analysis of the scenario itself. A total of 98 studies were included in the analysis of the content of P-SBE scenarios in Korea and other countries. Of the 44 studies that developed programs for medical staff, 37 (37.8%) were conducted with students, residents, and fellows. Thirteen (13.3%) studies were conducted on medical staff, nursing staff, and other staff, and 12 (12.2%) studies on other staff, including paramedics, lifeguards, and respiratory therapists. Several studies have developed programs for interdisciplinary teams. The core principle of healthcare providers is “First do not harm” [26]. Nevertheless, it has been reported that at least 44,000 (probably 98,000) patients die each year due to preventable errors by healthcare providers [27]. Simulation training enables the development and maintenance of skills in patient safety and quality management of medical services, and can help to acquire non-technical skills development and knowledge, such as communication skills and critical thinking, and to understand conceptual relationships [28] In addition, developing competencies related to interprofessional practice, including effective communication skills and teamwork, was recognized as essential to maximize patient outcomes and improve patient safety [29], confirming that the program was being developed for the team.

In terms of the five-stage model of skill acquisition [30], the most common stage targeted by SBE programs was competency (n = 29, 29.6%) and proficient (n = 29, 29.6%), followed by advanced beginner (n = 21, 21.4%), expert (n = 10, 10.2%), and novice (n = 9, 9.2%). In the 17th century, Dreyfus brothers developed a five-stage model to describe how individuals acquire skills and how experts master them. In other words, more studies have developed simulations designed to promote mastery among individuals at the competent or proficient level, which requires highly advanced and complex skills and experiences in more complex and challenging situations, as opposed to simulations targeting novices learning simple skills.

Subsequently, the scenarios were categorized based on their content. The most common scenario topic was emergency intervention (n = 59, 60.3%), and of these studies, there were 37 studies on pediatric rescue, 11 on neonatal rescue, and 8 on airway management. Another scenario was communication ability and decision making (n = 19, 22.4%), and the most common topic in this category was critical decisions (n = 13). This is in line with the Institute of Medicine (IOM) recommendations that healthcare providers are required to make accurate and critical decisions within a few seconds, even amid incomplete and inaccurate information; for these reasons, they must keep abreast with technological advances and collaborate with other professionals to rescue patients with complex morbidities [27]. Among programs targeting children, two studies developed a simulation program to enhance the decision-making ability of children with ASD, and programs targeting students included programs on resuscitation and decision-making ability during smoking education. These results show that SBE programs for children aim to improve their decision-making abilities. This is because simulation, an adaptive educational technology, provides an immersive environment in which students can interact with a given patient scenario and make their own decisions, through which they gain insight into their decision-making ability [31].

Regarding the reliability and validity of the scenarios, 30 studies (30.6%) tested the validity and 23 (23.5%) tested the reliability of the scenarios. In other words, there were still many studies that did not validate their findings despite the requirement for studies to be published to include evidence for evaluation or intervention, method of realization, reliability and validity, and educational outcomes to enhance the quality of evidence in medical education [32]. Reliability refers to the degree to which consistent measurements are obtained from the same study population. Validity refers to the degree to which something measures what it intends to measure. These crucial concepts underscore the need for more research to undertake such validation processes and reinforce their results, ensuring their applicability as trustworthy studies in a more effective manner.

Finally, in terms of the outcome variables used in the included studies, skills were the most common (n = 28, 28.6%), followed by performance (n = 24, 24.5%), knowledge (n = 17, 17.3%), and confidence (attitude) (n = 12, 12.2%). These results are in line with the recommendations of the (WHO) recommendations to develop standards and guidelines for simulation-based activities and implement simulation-based activities to accelerate the learning process and provide an opportunity for students and professionals to develop their skills and competencies [33].

In our systematic review, we examined the characteristics and development trends of P-SBE. Research in this domain was limited before 2004 but has witnessed significant growth post-2010. We observed that many P-SBE programs utilize high-fidelity simulators and team-based simulations, with emergency interventions being the primary educational topic to nurture rescue competencies. Most of this research has been conducted in developed countries like the United States, Canada, and the United Kingdom. While our results confirm the considerable advancement in P-SBE, many studies have not critically evaluated their validity and reliability. There’s a pressing need for an international protocol for the development of P-SBE, alongside rigorous validation and reliability testing. Furthermore, incorporating virtual reality technology could enhance the learning experience. It’s noteworthy to mention the limitations of our review: potential publication bias due to the focus on published papers, and the exclusion of scenarios where content specifics were not provided.

Conclusion

SBE has become indispensable owing to strengthened patient rights and the growing importance of patient safety. SBE is an educational method that enables pediatric healthcare staff to effectively improve their proficiency and competencies. It provides an immersive environment in which learners can interact with the given patient case scenario and make decisions, and owing to such benefits, it is actively utilized to train attitude, knowledge, and skills in health care providers and other staff. We hope that studies continue to follow up on these programs and evaluate their validity and reliability. Furthermore, there is a need for instruments that enable the categorization of scenarios and simulations based on the objective and learner’s current level and assess their competencies by level.