FormalPara Clinician’s capsule

What is known about the topic?

Many formal MCI triage tools exist, however there is little data available about how well these tools identify patients requiring lifesaving interventions.

What did this study ask?

Seven triage tools were assessed for their ability to identify patients requiring lifesaving interventions using data from a Canadian provincial trauma registry.

What did this study find?

All triage tools studied had limited effectiveness, with the highest-performing tool reporting a sensitivity of 0.76.

Why does this study matter to clinicians?

Many patients requiring urgent lifesaving interventions will not be identified by MCI triage tools currently in use.

Introduction

Triage is the process of efficiently assessing patients to identify those who are most likely to benefit from receiving limited clinical resources. Since the 1980s, the process of triage during mass casualty incidents (MCIs) has been codified into many formal triage tools [1,2,3,4,5,6,7].

Triage tools are structured as an algorithm to efficiently classify patients into one of a small number of clinical categories. These categories are intended to identify patients who are most likely to benefit from urgent lifesaving interventions (identified as Red, Immediate or Priority 1) and differentiate them from those who are critically ill but more likely to die even if responders intervene (Black, Grey or Expectant) and those who do not require urgent interventions to prevent loss of life (Yellow, Green, Delayed or Priority 2/3).

Clinicians and first responders are often mandated to use one of these formal triage tools when responding to an MCI. Although triage tools are widely employed, however, there is a surprising lack of knowledge about how these tools perform. Many triage tools were developed without any assessment of their performance by their creators [1,2,3,4, 7]. Recent research in the United Kingdom (UK) has demonstrated that many triage tools have poor sensitivity in identifying Immediate patients [8, 9]. This study used data from a Canadian trauma registry to assess the performance of seven triage tools in identifying patients requiring lifesaving interventions.

Methods

Study design

We performed a health records review to assess the accuracy of seven triage tools in identifying Immediate patients using the Alberta Trauma Registry (ATR). The ATR is a web-based trauma registry that captures the prehospital and in-hospital clinical records of eligible trauma patients presenting to one of the province’s ten trauma centres [10]. In order to qualify for enrollment in the registry, patients must have an Injury Severity Score (ISS) of 12 or more and either be admitted to a trauma centre or be pronounced dead in the ED of a trauma centre. All patients admitted to a trauma centre with penetrating trauma are also enrolled in the registry [10]. The ATR is maintained by Alberta Health Services (AHS). Staff at each of Alberta’s trauma centres upload clinical information to the database on a daily basis. AHS staff provided clinical data from the ATR to the study authors in an Excel file [11]. The performance of the triage tools was assessed by comparing the classifications made by each tool against reference standard definitions of triage categorizations [12].

This study design followed the Recommendations for Reporting the Results of Studies of Instruments and Scale Development and Testing [13]. This study was approved by the Research and Ethics Board at the University of Alberta (Pro00114532).

Population

Records were retrieved from the ATR for all patients enrolled in the registry between 1 January 2015 and 31 December 2019. Records were excluded if either the triage tool or reference standard categorizations could not be determined due to missing clinical data. Records that did not have the patient’s age documented were retained in the analysis involving the entire sample but excluded from any age-stratified analyses. Individuals 18 years of age and less were classified as pediatric, and those older than 65 years of age were classified as geriatric.

Triage tools

Clinical data from the ATR were used to determine the triage category that each patient would have been assigned under seven triage tools: START [1], JumpSTART [2], SALT [3], RAMP [4], MPTT [5], BCD [6] and MITT [7]. Each triage tool’s criterion was matched with the earliest and most-appropriate clinical variable documented in the ATR for each patient (either in the prehospital environment, in the ED of a referring hospital or in the ED of the trauma centre).

The following assumptions were made in the application of the triage tools. Due to meeting the eligibility criteria for inclusion in the ATR, all patients were assumed to be non-ambulatory. Patients with a systolic blood pressure of less than 90 mmHg were assumed to not have a palpable peripheral pulse and/or have a capillary refill time of more than 2 s, which is an accepted convention in trauma literature [8, 9, 14]. A sensitivity analysis assessing the impact of this assumption was completed (Supplementary Data Fig. 1). Patients with a Glasgow Coma Score (GCS)-motor of ≤ 5 were assumed to not obey commands, and patients with a GCS-motor of ≤ 3 were assumed to be posturing. Patients with GCS-eyes < 3, GCS-verbal < 5 and GCS-motor < 6 were assumed to not be responsive to voice. Patients who received interventions aimed at airway protection or supporting ventilation (including oral intubation, oral airway placement, nasal airway placement or bag valve mask utilization) were considered to be having “respiratory distress” and requiring airway repositioning.

Fig. 1
figure 1

Visualization of performance measures for triage tools’ ability to identify immediate patients, as defined by the reference standard, stratified by age (upper graph) and mechanism of injury (lower graph). Lines represent a 95% confidence interval

Full size image

Performance measures

The predetermined primary outcome was the sensitivity of each triage tool to assign an Immediate (or equivalent) categorization to patients that met the reference standard definition of an Immediate patient. Specificity, positive predictive value (PPV) and negative predictive value (NPV) were calculated as secondary outcomes.

As the reference standard, we employed the consensus-based definitions of triage categorizations developed by Lerner et al. [9]. Their definition of an Immediate patient is based on the presence of uncontrolled hemorrhage on presentation to an ED or the need for specific lifesaving interventions within the ED. These definitions were translated into criteria utilizing the clinical data available in the ATR (Supplementary Data Table 1).

Table 1 Patient Characteristics
Full size table

Data analysis

Excel functions were applied to determine each patient’s categorization for each triage tool and the reference standard using the earliest recorded prehospital or ED clinical data. Statistical analysis was completed using R version 3.6.3 [15]. Corresponding CIs were calculated using Wilson's score method with a continuity correction.

Results

In total, 9448 records were retrieved. 796 (8.4%) had incomplete clinical data and were excluded leaving 8652 records in the full analytic sample. Patient characteristics are reported in Table 1. Males comprised 71.1% (n = 6149) of the sample. Pediatric patients made up 733 (9.0%) of the records and 2419 (29.0%) records were of geriatric patients. Blunt trauma was experienced by 92.3% (n = 7989) of patients. The median ISS was 18 (Q1 = 16, Q3 = 25). The in-hospital mortality rate was 9.3% (n = 801). Using the reference standard definitions, 2528 (29.2%) patients were classified as Immediate.

The most common reasons for patients to be classified as Immediate under the reference standard were advanced airway protection (61.1%) and chest tube placement (38.3%) (Table 2). Blood product administration was performed in only 417 (16.5%) of Immediate patients. A total of 258 (10.2%) Immediate patients received both advanced airway protection and surgery.

Table 2 Life-saving interventions among immediate patients according to the reference standard definitions
Full size table

There was a wide range in the performance of the triage tools in identifying Immediate patients (Table 3). BCD and MITT performed identically across all performance measures. The most sensitive triage tools were MPTT with a sensitivity (95% CI) of 0.76 (0.75, 0.78) and BCD and MITT with a sensitivity of 0.70 (0.68, 0.71). The remainder of the triage tools had sensitivities below 0.45. JumpSTART had the lowest sensitivity at 0.31 (0.30, 0.33). MPTT had the lowest PPV at 0.41 (0.39, 0.42). The tools all had similar NPVs (ranging from 0.76 to 0.85).

Table 3 Performance of the triage tools in identifying immediate patients as defined by the reference standard
Full size table

Performance measures for the triage tools stratified by age and mechanism of injury are shown in Fig. 1. All tools had a higher sensitivity in identifying pediatric patients and a lower sensitivity in identifying geriatric patients as compared to adult patients. JumpSTART had the lowest sensitivity for identifying pediatric patients, with a sensitivity of only 0.36 (0.30, 0.43). MPTT, BCD and MITT had higher sensitivities for penetrating trauma and burns, but the other four tools were the most sensitive in blunt trauma. All tools had a moderate to high PPV (greater than 0.67) for penetrating trauma.

Table 4 shows the distribution of classifications by each tool for every reference standard triage category. MPTT had the lowest rate of under-triage at 10.1% (9.5%, 10.8%), and BCD and MITT had the second-lowest rate of under-triage at 12.1% (11.4%, 12.8%). JumpSTART had the highest under-triage rate at 21.6% (20.7%, 22.5%). JumpSTART and RAMP had the lowest rates of over-triage at 7.0% (6.5%, 7.6%) and 7.8% (7.2%, 8.3%), respectively. MPTT had the highest rate of over-triage at 30.5% (29.5%, 31.5%). A comparison of the Immediate patients who were appropriately triaged and under-triaged is provided in the Supplementary Data (Table 2).

Table 4 Percentage of patients in each reference standard category placed into each triage category, by triage tool
Full size table

Discussion

Interpretation of findings

When a triage tool undertriages patients, critically injured individuals who are likely to benefit from urgent interventions are at risk of being missed and having a poor outcome. In contrast, when patients are overtriaged limited resources are directed to patients who have less severe injuries, leaving fewer resources for the patients who are critically ill. Previous research has shown that during an MCI, there is a near-linear relationship between the rate of overtriage and mortality [16].

All of the triage tools examined in this study performed sub-optimally in identifying trauma patients who required urgent lifesaving interventions. MPTT had the highest sensitivity at 0.76 but this tool undertriaged 10.1% of patients. The American College of Surgery’s Committee on Trauma has recommended that under normal conditions, trauma systems should aim to have an undertriage rate of less than 1% [17]. It is not surprising that BCD and MITT performed identically since these tools have the same criteria, except for the qualification that all patients aged < 2yo are classified as Immediate under MITT.

Comparison to previous studies

Previous studies have attempted to examine the performance of triage tools using prehospital and hospital data captured during MCIs [18,19,20]. These studies have been limited for several reasons, including insufficient power due to the small number of patients involved in the MCIs. Another factor is that clinical data is often poorly captured during MCIs. Finally, each MCI has unique characteristics that limit the generalizability of the findings. Other studies have assessed the performance of triage tools in predicting mortality or identifying patients with a high ISS [21, 22]. However, neither of these end points represent the function that the triage tools were designed to perform. Mortality is an inappropriate measure because clinical resources during an MCI should be directed to patients who are critically ill but who are also likely to survive, not to those who are likely to die even if they receive treatment. A high ISS has previously been shown to be a poor predictor of the need for lifesaving interventions [23], making this a poor surrogate marker in assessing triage tool performance.

Malik et al. [8, 9] used data from the UK’s Trauma Audit and Research Network (TARN) database to measure the performance of 10 triage tools, including five of the seven tools evaluated in our study. They did not assess SALT (one of the most commonly employed tools in North America) or MITT (which was released in October 2022, after the TARN studies were published). They found that BCD outperformed all other tools, including MPTT [8, 9]. Differences in our findings may be due to the fact that the patient population in our study had experienced more-severe trauma. The average ISS for patients in the adult TARN study was 9, and it was 18 in our patient population. Only 11.3% of patients in the adult TARN study meet the reference standard criteria for immediate classification versus 29.2% of patients in our study [8].

Strength and limitations

Our study provides the first external validation of MCI triage tools using North American data. This research benefits from a large dataset drawn from a quality-controlled registry. The reference standard we employed is from a previously published, consensus-based definition [12]. Our study is the first to assess the performance of the MITT triage tool.

Limitations of our study include the fact that while we were able to determine which patients received potentially lifesaving interventions within a specific timeframe, it is not possible to determine if any specific intervention was actually lifesaving for any particular patient. It is possible that some patients in our dataset received interventions that would have been appropriately withheld if resources had been more limited. We had to make several assumptions when applying the triage tools to the data in the ATR, as well as when applying the criteria for the reference standard categorizations. Patients with blunt trauma made up 92.3% of our dataset. This is representative of the injury patterns observed in many civilian trauma centres but may limit the generalizability of our findings to patients who have experienced other forms of trauma. Our study population had a disproportionally high severity of injuries (average ISS was 18 and all patients were seen at a trauma centre). While this may have impacted the calculation of the specificity and NPV, this is unlikely to have affected the measurement of sensitivity, which was our primary outcome. The clinical data in this study was obtained from a provincial trauma system operating under normal conditions, but the triage tools we studied are intended to be used during MCIs where resources are more limited. The performance of these tools during MCI conditions may differ from our findings.

Clinical implications

Our findings show that all the triage tools we assessed have limited sensitivity. This suggests that clinicians involved in performing triage cannot rely solely on any of the triage tools assessed here to identify critically ill patients during an MCI. START, which is the triage tool most commonly taught and mandated for use in Canada, performed worse than other formal triage tools. Providers should consider transitioning to MPTT, BCD or MITT as they have consistently demonstrate the highest sensitivities and lowest rates of undertriage [8, 9]. MPTT had the highest sensitivity in our analysis. However, BCD and MITT use physiologic cutoffs that are expected to be easier to apply in practice. Our results reflect the performance of the triage tools when they are applied appropriately. In MCIs, however, human factors affect the accuracy with which responders apply triage tools [24]. JumpSTART was created specifically for triaging pediatric patients, but it had the lowest sensitivity in our pediatric subpopulation. This is consistent with the findings from the pediatric TARN study [9]. Providers should consider discontinuing the use of JumpSTART.

Research implications

Future research should focus on developing more-sensitive triage decision rules to overcome the limitations of current triage tools. Potential improvements may include assessing the importance of different physiologic cutoffs and exploring the value of incorporating other types of clinical information into triage algorithms (e.g. age, expected length of extraction time, etc.). Incorporating technology into the triage process would allow for the development of new tools that could be more complex than the current algorithms which are designed to be employed mentally under stressful conditions. It would be valuable to replicate the findings of this study, preferably in a prospective manner.

Conclusion

Our study shows that MCI triage tools currently in use have limited sensitivities for identifying patients who require urgent lifesaving interventions. START, the triage tool most commonly employed in Canada, performs poorly compared to other existing triage tools. MPTT, BCD and MITT were the most sensitive triage tools assessed and had the lowest rates of undertriage. All the triage tools assessed should be employed with caution during MCIs as they may fail to identify a large proportion of patients requiring urgent lifesaving interventions.