Introduction

Trauma is the leading cause of death among Brazilian children and adolescents above 1 year of age and exceeds all other causes of mortality in children combined. [1, 2]. Approximately 5000 children between 1 and 14 years of age die annually due to injury in Brazil. Furthermore, the hospitalization rate due to trauma in Brazil is 261 per 100,000 inhabitants, or approximately 375 children per day [1]. It is also estimated that there are an additional 60 admissions for observation and treatment in emergency rooms for each pediatric trauma death, suggesting over 300,000 pediatric trauma visits per year [3]. As a result, trauma is considered one of the significant health issues for children and adolescents in Brazil.

Pediatric trauma remains a neglected disease worldwide [4, 5]. In many low- and middle-income countries (LMICs), pediatric trauma care is provided at general hospitals, and injured pediatric patients lack access to a well-prepared trauma center specific to pediatrics [5,6,7]. Additionally, providers' confidence to treat children is low among general specialties [8, 9], and there are only a few intervention studies that address how to improve the quality of pediatric trauma care. Even in the United States, which has well-organized pediatric trauma centers, 17.4 million children and adolescents under the age of 15 do not have access to these centers [10]. In Brazil, most trauma assessments occur in hospitals with no trauma team, let alone a pediatric trauma team [5]. Lack of standardization of the Brazilian trauma system and specific protocols for the Brazilian pediatric population leads to inadequate care of this population [5, 6].

One solution to improve pediatric trauma care quality involves investments in strengthening the referral centers’ capacity. A properly functioning trauma center improves outcomes and contributes to an organized trauma prevention program [11,12,13]. However—as crucial as investing in a trauma reference center is—regularly measuring the quality of existing trauma care where it is currently provided is essential.

Measuring the quality of pediatric trauma care is not an easy task. Indicators of quality such as mortality, number of surgeries to treat solid organ injuries, and time to treat femur fracture or brain injury fail to evaluate the quality of care in each trauma assessment phase [14]. Assessing adherence to protocols has been used to determine if interventions to optimize care are efficient, mainly to measure the quality of a specific hospital or a team [15]. It is known that when adherence to trauma protocols is high, time to treatment decreases, as does the number of unnecessary complementary diagnostic exams [6, 16,17,18].

Checklists have been used to improve adherence to trauma protocols and ultimately improve care quality in the United States [10,11,12]. Although checklists have been implemented during capacity-building programs in simulation centers, to the authors’ knowledge, no study evaluated the use of a checklist to improve pediatric trauma care inside hospital practice in the LMIC setting.

This study evaluates health professionals’ adherence rate to standardized pediatric trauma assessment steps before and after implementing a pediatric trauma care checklist.

Material and methods

Approval in ethics committees and informed consent

The project was approved by the ethical review committee/Institutional Review Board Fundacao Hospitalar de Minas Gerais (094B / 2017) and by the Boston Children's Hospital Ethics and Research Committee (P00031402). Informed consent was obtained from every health provider that agreed to participate in this research.

Type of study

Single-arm non-randomized interventional trial.

Study site

This study took place at Hospital João XXIII (HJXXIII) in Minas Gerais, Brazil. HJXXIII is the primary trauma referral center for the region and performs 540 surgeries and 980 hospitalizations for injury per month. Besides being a hospital that serves the local population, the hospital participates in the city’s and state's emergency network system. It receives patients transferred from other health facilities with a catchment area of 20 million people.

Elaboration of the research

Participants

Inclusion criteria
  1. a)

    Children and adolescents from 0 to 14 years old who were victims of trauma (per chapters XIX and XX from International Classification of Diseases);

  1. b)

    Assessment in the trauma bay of the hospital. Classified as emergent (red) or very urgent (orange) by Manchester Protocol [19, 20].

  1. c)

    Participating health professionals must have signed informed consent.

Exclusion criteria
  1. a)

    Patients 15 years of age or older.

  1. b)

    Manchester Protocol classification of blue, green, or yellow.

  1. c)

    Non-traumatic emergencies.

The Manchester Protocol is an international triage system used to classify patients at admission. Several public and private hospitals in Brazil have adopted it to improve their triage. The Manchester Protocol identifies patients that should be prioritized using specific flowcharts for each condition. Depending on vital sign values and trauma mechanisms, the patient is classified under five different colors: red (emergent), orange (very urgent), yellow (urgent), green (slightly urgent), and blue (non-urgent). Red patients should be treated immediately and orange within 10 min.

At HJXXIII, only Manchester protocol red or orange are activated for trauma assessment by a physician in the central trauma bay. Patients with other classifications are referred to a pediatrician’s office [16]. A triage nurse performs initial triage, and the decision of whether to activate the leading trauma team is made following the Manchester protocol.

Calculation of sample number and estimation of collection time

A previous study found an increase of 9.8% of completed tasks of their institution's trauma protocol after introducing a checklist in their assessment [17]. Considering it as a standard with an increase of 9.8%; an alpha error probability of 0.05; a correlation between groups of 0.5; a power test of 0.8 (minimum required to detect a difference), and the hypothesis that the interventions would increase adherence, the sample size was calculated using a matched pair test.

We calculated a minimum sample size of 68 patients using G*Power 3.1.1 statistical software. We planned to enroll an additional 12 patients for a total of 80 patients to account for potential losses to follow-up or exclusions. Medical students were distributed randomly in different shifts (morning, afternoon, and night), so that they could cover 35% of the week. Based on a previous study in which HJXXIII received an average of 1.5 pediatric patients a day with the established inclusion criteria, a 2-month time period would be enough to collect the minimal sample size for each group.

Development and standardization of the assessment protocol

Two protocols are recommended for the care of polytraumatized children and adolescents in the studied hospital: the hospital’s protocol and the Advanced Trauma Life Support (ATLS) protocol [21, 22]. A research protocol was created based on combining the ATLS and the hospital’s existing protocol to know its significant adherence rate.

The criteria were chosen based on critical steps during the primary survey, essential to diagnose or treat lethal conditions such as airway obstruction, c-spine injury, tension pneumothorax, open pneumothorax, massive hemothorax, flail chest, shock, cerebral herniation, and hypothermia (see Box 1).

Researchers also documented if the provider considered the child’s weight, as it is required to accurately assess vital signs and initiate proper treatment.

Other epidemiologic and outcome variables collected are presented in Box 2.

Data collection

Medical students performed data collection after undergoing training by the research team. Training consisted of lectures and guided practice data collection conducted by the study staff members in the trauma bay. A pilot study was performed using data collected by both students and the principal investigator for 2 weeks. The students and the principal investigator collected data simultaneously and after the results were compared. At this moment, students' questions were also answered. The study only started after the principal investigator felt confident that students could adequately collect data during the pilot study. All data and electronic medical records were reviewed after the study period to identify any data entry errors. We defined each assessment step’s completion based on the criteria in Box 1.

The REDCap software was used for data collection [23, 24]. Data were collected at two different time points:

  1. 1)

    Pre-intervention group: we collected the data described before the checklist intervention from July 2019 to August 2019.

  1. 2)

    Post-intervention group: after checklist intervention, from October 2019 to November 2019.

Checklist intervention

We included the proposed pediatric trauma checklist on the professional badges of trauma care providers (Appendix 1) and posted it to the main trauma bay walls at HJXXIII. Two hundred badges were distributed to all pediatricians, general surgeons, and general surgery residents. An in-service consisting of two lectures given by the principal investigator about the checklist intervention and our study goals were held for providers that work in the trauma bay. Finally, electronic information about the checklist and the study was delivered by WhatsApp to all HJXXIII general surgeons, general surgeon residents, and pediatricians. After 1 month of interventions, we collected data again.

Data analysis

We compared the adherence to each letter of the standardized trauma assessment (ABCDE) before and after checklist intervention. We also compared the subjects’ demographic data to assess heterogeneity in the study population during each study phase. The categorical comparisons were performed using the Pearson Chi-square test. For quantitative variables, the Shapiro–Wilk test was used to check for normality. For non-normally distributed variables, a Wilcoxon signed-rank test was employed. Normally, distributed variables were compared using an ANOVA test. Statistical analysis was performed in R version 3.2.5. MINITAB and PASW Statistics—SPSS version 18.

Results

Over 5 months, 80 patients met the inclusion criteria and were included in the study. Of these, 39 patients were observed before the checklist intervention and 41 after the checklist. No patients were excluded. Epidemiologic characteristics of each group are in Table 1. There was no significant difference in demographics in the pre- and post-intervention groups.

Table 1 Epidemiologic characteristics before and after checklist intervention in HJXXIII and provider performing initial assessment
Full size table

The mechanism of trauma was also similar in both groups. Falls were the most common cause of trauma (47.5%), followed by motor vehicle accidents (23.7%).

Unfortunately, it was impossible to calculate trauma scores (PTS and ISS) as a low proportion of vital signs was recorded.

Adherence to protocol rate in each step of the assessment before and after the intervention is presented in Table 2. No improvement in the protocol adherence was observed in any step of the trauma assessment after our intervention.

Table 2 Protocol adherence before and after checklist intervention in HJXXIII
Full size table

There was no difference in the benefit obtained from the intervention between the specialties when we stratified the groups. Finally, there was no difference observed in the outcomes before and after the checklist intervention (Table 3). There was a statistically significant difference in pediatric intensive-care unit admissions, with more admissions in the initial group before the checklist intervention took place (13 patients versus five patients).

Table 3 Outcomes before and after checklist intervention in HJXXIII
Full size table

The average time to initiate the trauma assessment was 6 min with a median of 2 min, without significant difference among groups. No mortality was observed in this study.

Discussion

General assessment and checklist intervention

Our study results highlight the difficulties in improving healthcare delivery through systematic interventions. In our study, an average of 54% trauma assessment steps was performed with only one case where all protocol steps were followed. Although there are few studies about pediatric trauma protocol adherence, which makes it difficult to determine an appropriate standard for protocol adherence, assessment completion was well below what would be consistent with high-quality care [17].

Compared to international studies that used checklist intervention [9, 17, 25], we chose fewer steps, and there were fewer providers per patient in HJXXIII to do the primary survey. These international studies were performed in a high-income country (HIC), highlighting the differences in administering these types of interventions in LMICs where assessment may not follow the same protocolized steps as specialized pediatric centers in HICs. Primary trauma assessment in Brazil is typically performed by a physician and a nurse instead of a dedicated team of providers.

Because previous literature suggested that displaying checklists on walls could improve provider performance in simulated trauma cases [8], we introduced the checklist via wall display and on professional badges. We chose this intervention method in the LMIC setting, as there are fewer providers available to conduct initial assessments; this is a simple method for them to have the protocol at hand. In a busy trauma hospital with a lack of quality improvement programs, keeping interventions as simple as possible is essential. Authors who worked/work in the hospital and were primarily responsible for implementing the interventions were aware that pediatric care interventions would be challenging to implement in a main adult hospital, as they concluded in the previous studies [6, 15, 26]. To intervene as streamlined for the providers, we visited each shift, delivered the badges, and used WhatsApp to spread the checklist information and how it was developed. WhatsApp is a popular tool among Brazilian physicians, as 87% of them routinely use the app [27]. While we cannot know the total number of professionals who received the updated information, by personally distributing the badges in every shift and sending the information via WhatsApp, it is essential to recall that everyone signed the consent form to participate in the study. Also, to further spread the word, a lecture was given at the general surgery meeting and in the pediatricians’ weekly meetings as well.

Our results did not show that introducing a checklist resulted in significant increases in assessment thoroughness. Therefore, it is expected to not have any relevant outcomes difference between the groups. We decided to include outcomes in this work as a secondary aim to understand what would be the clinical impact on the patient if the checklist improved the primary survey. The proposed outcomes were selected based on previous studies that measured the quality and severity of pediatric trauma care in Brazil, mainly when you do not have an adequate vital sign assessment [15].

There was no improvement regarding adherence to protocol and outcomes such as the use of CT scans or hospital admission, except for PICU admission. The first group had more patients admitted to PICU (13 patients versus five patients); however, this is not the first goal of this study. Our sample was not calculated for this comparison, data were collected in different months, and any correlation should be interpreted carefully.

Reasons for checklist failure

There are several possible explanations for the lack of improvement seen in our study. First, checklists may be more effective during simulations as compared to implementation in real-time clinical practice [8, 9, 17]. Second, the lack of a dedicated team and a trauma leader available to help with primary trauma assessment may have impaired checklist use. Unfortunately, the lack of providers and the necessity of multiple assessments needing to be conducted simultaneously is a reality in the LMIC setting [5, 7]. The number of healthcare providers in the emergency team in an HIC is higher than the single physician model with a nurse assistant prevalent in Brazil. It can also be one reason that may explain different results when one compares a trauma assessment in simulation centers and trauma bays in LMICs and HICs.

Finally, a systematic review of the use of the checklist to improve adherence to ATLS was recently published [25]. This study concluded that the current literature failed to show that checklist intervention enhanced patient outcomes, although one study showed promising results.

Low adherence and improving pediatric trauma in LMICs

Efforts are still being made to improve adherence to protocols and pediatric trauma care quality in LMICs. Current research and scientific societies fail to propose effective strategies that genuinely improve pediatric trauma care in LMICs. It is an urgent issue, as pediatric trauma continues to be a significant cause of morbidity and mortality in children worldwide [4, 28]. Increased use of motor vehicles and urbanization are reasons that pediatric trauma will continue to be a major health issue in the coming decades [29]. Because there is an increasing number of households where both parents work, many children now spend a considerable time in daycare centers, typically not well prepared to deal with trauma, mainly falls [30].

In previous works of this group [6, 31], we found some possible reasons for low adherence among providers in HJXXIII hospital. These include the high number of daily cases treated in the referral system (also considering adults); staff burnout; lack of a pediatric trauma leader; lack of pediatric trauma knowledge; fragile referral protocols with over triage of cases referred to trauma centers, even for minor trauma [6].

Weak referral protocols delay pediatric trauma treatment. In Minas Gerais state, it could take 18.5 h for an injured child to arrive at HJXXIII after a referral from another city hospital [26]. Additionally, the Manchester Protocol used for trauma patient triage on arrival is subjective. There is no specific recommendation of which flowchart the triage team should choose for each patient admitted. Depending on the team, they could use the "Protocol for Major Trauma" in some cases or "Worried Parents" in others. Furthermore, in the "Protocol for Major Trauma,” if a provider considers the trauma mechanism significant, the patient will automatically be regarded as severe. It can be subjective and lead to the over triage of patients.

Unfortunately, we also had a low vital sign assessment. Thus, we could not compare the Manchester protocol results with standard trauma prognostic scores such as the Pediatric Trauma Score and the Injury Severity Scale. Indeed, much of our results (low adherence to the steps) were due to the lack of measuring heart rate, respiratory rate, weight, and blood pressure. A qualitative assessment with these providers must be done in future work to understand better why this is being a barrier.

Improving the quality of care in a setting is a challenge, and it has been the focus of this research group in the last years. As we believe that these problems are not exclusive to HJXXIII, we want to share our thoughts regarding other possible solutions to improve pediatric trauma care quality. These solutions include (1) triage protocol optimization to decrease admission of minor trauma in referral centers and in the trauma bay dedicated to severe trauma—this would consist of vital signs better assessment during hospital admission; (2) identification of pediatric trauma leaders in each shift; and (3) continuing medical education regarding differences in regional pediatric trauma epidemiology hospital resources. Finally, checklists should be developed in conjunction with pediatric trauma courses to achieve better results.

Limitations

One of the main limitations of this study is the Hawthorne effect, as providers were observed directly by medical students. It can lead to errors during data collection; however, the use of cameras during the assessment is not allowed by the Brazilian Medical Ethics Council. We opted to choose students to observe the assessments. HJXXIII regularly accepts medical students for clinical observerships, and professionals are used to working with their presence in the trauma bay. The authors did a pilot study, reviewed all the data, and selected trained students to decrease data collection errors.

Future directions

The scientific literature has shown that one of the most critical game-changers in pediatric trauma outcomes is to have providers trained specifically in pediatric trauma, independent of specialty [32]. However, we believe that each institution should analyze which specialty is most prepared to perform pediatric trauma assessments. In our study, although we did not find statistically significant differences regarding specialties' performance, the sample size was not calculated to capture this information.

Unfortunately, a simple intervention was not enough to change behavior in our setting. We did, however, raise awareness by researching pediatric trauma care and sharing the results with the staff. In a recent survey that we performed, 86% of providers told us that a pediatric trauma course could be beneficial [6]. Thus, our next step will be to implement an educationally sound pediatric trauma course to the Brazilian setting. We expect that this course will improve pediatric trauma care in Brazil, and we hypothesize that it could increase checklist adherence. It will also be essential to collect qualitative data from the providers about the studies’ strengths, weaknesses, and future interventions. Finally, self-assessments and pre- and post-tests will also be necessary for the subsequent intervention to evaluate the acquisition and retention of knowledge for a long-term outcome.

Conclusion

Implementation of a checklist intervention at a Brazilian referral center for pediatric trauma primary assessment showed no statistical difference comparing the trauma assessments before and after the checklist was implemented. Initial assessment quality is low for both surgical and non-surgical providers before and after the intervention. Therefore, we do not also observe relevant improvement in clinical outcomes.

These results reinforce the urgent need for a deeper transformation. We propose as further steps to improve the quality of care in our setting: (1) organization of the pediatric trauma system through improved triage, (2) identification of pediatric trauma leaders, and (3) providers’ training for pediatric trauma care.

We have already started to work on these interventions, and we hope sharing the results soon.