Abstract
Background
Delivery of major trauma care is complex and often fast paced. Clear and comprehensive documentation is paramount to support effective communication during complex clinical care episodes, and to allow collection of data for audit, research and continuous improvement. Clinical events are typically recorded on paper-based records that are developed for individual centres or systems. As one of the priorities laid out by the Scottish Trauma Network project was to develop an electronic data collection system, the TraumaApp was created as a data collection tool for major trauma that could be adopted worldwide.
Methods
The study was performed as a service evaluation based at the Queen Elizabeth University Hospital Emergency Department. Fifty staff members were recruited in pairs and listened to five paired major trauma standby and handover recordings. Participants were randomised to input data to the TraumaApp and one into the existing paper proforma. The time taken to input data add into was measured, along with time for clarifications and any errors made. Those using the app completed a System Usability Score.
Results
No statistically significant difference was demonstrated between times taken for data entry for the digital and paper documentation, apart from the Case 5 Handover (p < 0.05). Case 1 showed a significantly higher time for clarifications and number of errors with digital data collection (p = 0.01 and p = 1.79E-05 respectively). There were no other differences between data for the app and the proforma. The mean System Usability score for this cohort was 75 out of 100, with a standard deviation of 17 (rounded to nearest integer).
Conclusion
Digital real-time recording of clinical events using a tool such as the TraumaApp is comparable to completion of paper proforma. The System Usability Score for the TraumaApp was above the internationally validated standard of acceptable usability. There was no evidence of improvement in use over time or familiarity, most likely due to the brevity of the assessments and the refined user interface. This would benefit from further research, exploring data completeness and a potential mixed methods approach to explore training requirements for use of the TraumaApp.
Similar content being viewed by others
Background
Trauma contributes to 8% of all deaths worldwide [1]. It is the leading cause of death in Western countries for those under 40 years old [2]. Inclusive trauma systems have been shown to improve outcomes in major trauma [3]. The Scottish Trauma Network (STN) was established to coordinate care between the Scottish Ambulance Service, local trauma units with the creation four newly created of four Major Trauma Centres (MTCs) [2]. Development of the network has provided the opportunity to standardise processes with the need for granular data to monitor impact of this investment on patient outcomes. Clear and comprehensive documentation is paramount to enable effective communication between teams [4], and to allow collection of data for audit, enabling systems to systematically improve clinical care [5].
The National Institute for Health and Care Excellence guidelines lay out consensus criteria for complete trauma documentation, including Standby and Handover information, A to E assessment (complete primary assessment of a trauma patient in critical order as per Advanced Trauma Life Support guidance) [6] and interventions recorded contemporaneously by a designated member of the trauma team [7].
Similar to the majority of centres worldwide, the Queen Elizabeth University Hospital (QEUH) in Glasgow trauma case management is currently documented on a paper proforma. While documents across trauma systems capture similar key data there is considerable variation and compliance with completion is limited [8]. One of the priorities laid out by the Scottish Trauma Network (STN0) in 2019/20 was to develop an electronic data collection system, allowing easier and more accurate data collection [5].
Electronic documentation has consistently been shown to improve the quality and utility of documentation by reducing errors, omissions [9, 10] and length of stay in the Emergency Department (ED) [11, 12]. Baumann et al. performed a systematic review which showed that, initially, documentation time was slower following implementation of an electronic documentation system. However, as familiarity increased the documentation time decreased again [13]. The TraumaApp was developed as a collaboration between the EmQuire research group, Scottish Trauma Network and Daysix, a digital transformation company. The work was funded by InnovateUK (Project 104,540) to create a data collection tool for major trauma that could be adopted worldwide. During the two-year project, extensive co-design involving a range of trauma clinicians enabled rapid wireframing and prototyping to refine the tool. The data dictionary was based on current Scottish Trauma Audit Group and Trauma Audit and Research Network (TARN) data reporting and was aligned to international reporting standards [5, 14, 15].
Methods
The aim was to compare the feasibility of using a digital data collection tool (TraumaApp) against the paper trauma proforma used in the ED at the QEUH specifically,
-
The time taken to input information and the number of mistakes or clarifications required.
-
Assess usability of the TraumaApp
Secondary aims included assessment of improvement in digital tool use over time, suggesting familiarity, and gaining qualitative feedback on app use.
The study was performed as a service evaluation based at the Queen Elizabeth University Hospital ED. This tertiary ED sees approximately 110,000 > 16-year-old presentations per year and will be officially designated a MTC in August, 2021. The department is currently staffed by 30 consultants, 55 training grade doctors and 122 nurses. All staff types may be expected to use the TraumaApp including new roles such as trauma co-ordinators.
Fifty ED Staff members were recruited in pairs (Table 1). They were consented and, in each pair, one participant entered data into the TraumaApp and one into the existing paper proforma. Participants listened to five paired Standby and Handover recordings (Additional file 1) in sequence, with the cases in random order (Table 2) using a random number generator. Staff were familiar with the paper documentation from previous clinical exposure and departmental educational sessions. Participants were timed to assess how long it took to input the data after each recording was completed. Additionally, the time taken for clarifications was also recorded. Finally, the proforma and the app were assessed against a “template” and any mistakes and omissions were counted.
Prior to assessment, the participants who used the TraumaApp were given five minutes to familiarise themselves with the ‘Standby’ and ‘Handover’ screens (Fig. 1) on departmental iPads®. Following completion of the five cases they were asked to complete a System Usability Scale (SUS) [16]. This 10-question scale was selected as the most used and reliable measure of usability [12]. Free text feedback was also collected for both the app and the paper proforma.
Data were collected on an Apple iPad® using development version 1.0.6 of the TraumaApp based on iOS (DaySix Edinburgh, UK). Data were analysed by un-paired t-test using Microsoft Excel (Version 2103 Office 365).
Results
Demographics
Of 50 participants, two were called away during the initial case recording and were excluded from the results. Of the 48 included participants, there were 30 Junior Doctors (62.5%), seven Consultants (14.6%), seven Nurses (14.6%), two Major Trauma Coordinators, one Acute Care Nurse Practitioner and one Student Nurse. Table 1 describes the characteristics of the participants including their familiarity with Apple operating system and personal use of digital device.
Primary analysis
Time taken to input data for each recording is displayed in Fig. 2 for digital versus paper documentation. The length of the recordings is also shown.
There was no statistically significant difference between times taken for data entry for the digital and paper documentation, apart from the Case 5 Handover (p < 0.05)(Table 3).
Figure 3 shows data for mean time needed for clarifications after listening to the recordings, together with the mean number of errors noted.
Case 1 showed a significantly higher time for clarifications and number of errors with digital data collection (p = 0.01 and p = 1.79E-05 respectively). There were no other differences between data for the app and the proforma.
SUS Score
All participants using the TraumaApp (n = 24) completed a System Usability Score (SUS) questionnaire after completing the data entry. The mean SUS score for this cohort was 75 out of 100, with a standard deviation of 17 (rounded to nearest integer), as shown in Fig. 4.
Improvement in app use/ familiarity over time
The recorded data input times were standardised as percentages of recording duration and then re-ordered in the sequence that they were played to the participants. The mean values were compared in groups Digital and Paper, as shown in Fig. 5. There was no significant decrease in time taken over repeated use of the app or the paper proforma.
Discussion
Early identification and management of injuries following major trauma is critical in reducing morbidity and mortality [7]. This relies on clear communication, synchronous assessment and point of care investigations. Accurate documentation allows the clinical team to have a shared understanding of the case and enhances team situational awareness, to provide effective, safe care [17]. Contemporaneous documentation is often complicated by the rapid and high volume of information-rich data in often busy chaotic environment. This is most notable during standby calls, where documented information allows for appropriate trauma call preparation, and handover at arrival to hospital, where patient condition and pre-hospital treatment are relayed to direct ongoing management [18].
This study shows equivalence between digital and paper documentation time of standby and handover. The results show that both digital data collection and use of paper proforma result in some loss of information and errors during Standby and Handover, which has been identified in previous studies [19]. The significantly higher clarification time and volume of errors in the digital cohort seen in Case 1 was deemed to be due to a high volume of information to be entered. Further work is required around training to enhance familiarity and to optimise the user interface to facilitate data entry. The System Usability Scale was developed in the 1980s to allow objective assessment of perceived usability [20]. A mean score of 68 has been internationally validated as the standard of acceptable usability [12], and the 25 participants who completed the questionnaire gave a mean score of 75.
A secondary objective to examine familiarity to data recording over time was more difficult to assess, due to the differing lengths and complexities of the recorded cases. The case orders were randomised to combat this, and the time taken to record data was calculated as a percentage of each case’s recording length, as shown in Fig. 5a and b. A general downward trend over time, seen in Fig. 5a and b, suggests development of familiarity in both digital and paper documentation. No true assessment could be made about improvement in use of the Trauma App over time, as each participant only used it for 20–25 min. Previous studies of digital system use suggests that development of familiarity with the icons and images used in an application is required to improve operating efficiency [21]. This indicates that users may benefit from instruction on the TraumaApp but the addition of iconography would potentially increase the training demand. Twenty of the 24 app users reported that they became more familiar with the app as they used it, although this is not supported by the data in Fig. 5.
Limitations
This was a single-centre study involving a small cohort of participants. An increased number of participants may have given more power to allow identification of significant differences between digital and paper documentation. The TraumaApp continues to be refined prior to live release, so data may not be truly representative of the version used in practice. Demographic of participants is not representative of those who are likely to be scribing for trauma cases.
Conclusion
No significant difference was identified overall in the utility of digital documentation using the TraumaApp compared to the existing paper proforma in this study. The System Usability Scale for the TraumaApp was above the internationally validated standard of acceptable usability. There was no evidence of improvement in use over time or familiarity, most likely due to the brevity of the assessment. This would benefit from further research, exploring data completeness and a potential mixed methods approach to gain further user feedback and assessment of performance and utility.
Availability of data and materials
The datasets generated and/ or analysed during the current study are available from the corresponding author on reasonable request.
Abbreviations
- STN:
-
Scottish Trauma Network
- MTCs:
-
Major Trauma Centres
- QEUH:
-
Queen Elizabeth University Hospital
- STAG:
-
Scottish Trauma Audit Group
- ED:
-
Emergency Department
- SUS:
-
System Usability Scale
References
World Health Organisation. Injuries and Violence. [Internet] 2021 [cited 19/05/2021] Available from https://www.who.int/news-room/fact-sheets/detail/injuries-and-violence
Scottish Trauma Network. Introduction and background. [Internet] 2021 [cited 19/05/21] Available from https://www.scottishtraumanetwork.com/about/
Utter GH, Maier RV. Inclusive trauma systems: do they improve triage or outcomes of the severely injured? J Trauma. 2006;60(3):529–37.
Collins SA, Cato K, Albers D, et al. Relationship between nursing documentation and patients’ mortality. Am J Crit Care. 2013;22(4):306–13.
Scottish Trauma Audit Group. STAG Priorities [Internet] 2020 [Cited 31/3/2021.] Available from https://www.stag.scot.nhs.uk/new.htm.
van Olden GDJ, Meeuwis JD, Bolhuis HW, Boxma H, Goris RJA. Clinical impact of advanced trauma life support. Am J Emerg Med. 2004;22:522–5.
National Clinical Guideline Centre. Major trauma: assessment and initial management. 2016. https://www.nice.org.uk/guidance/ng39/evidence/full-guideline-2308122833 Accessed 19/05/21
O’Connor TL, Raposo AE, Heller-Wescott T. Improving trauma documentation in the emergency department. J Trauma Nurs. 2014;21(5):238–43.
Coffey CA, Wurster LA, Groner J, Hoffman J, Hendren V, Nuss K, Haley K, Gerberick J, Malehorn B, Covert J. A comparison of paper documentation to electronic documentation for trauma resuscitations at a level I paediatric trauma center. J Emerg Nurs. 2015;41(1):52–6.
Bilyeu P, Eastes L. Use of the electronic medical record for trauma resuscitations: how does this impact documentation completeness? J Trauma Nurs. 2013;20(3):166–8.
Zikos D, Diomidous M, Mpletsa V. The effect of an electronic documentation system on the trauma patient;s length of stay in an emergency department. J Emerg Nurs. 2014;40(5):469–75.
Bloom BM, Pott J, Thomas S, Gaunt DR, Hughes TC. Usability of electronic health record systems in UK EDs. Emerg Med J Epub. doi:10.1136/ emermed-2020–210401 Available from https://emj.bmj.com/content/38/6/410
Baumann LA, Baker J, Elshaug AG. The impact of electronic health record systems on clinical documentation time: a systematic review. Health Pol. 2018;122(8):827–36.
The Trauma Audit and Research Network. Resources. [Internet] 2021[cited 19/05/21] Available from https://www.tarn.ac.uk/Content.aspx?ca=4
Palmer CS, Davey TM, et al. Standardised Trauma Monitoring: The development of a minimum dataset for trauma registries in Australia and New Zealand. Injury Int J Care Injured. 2013;44:834–41.
J, Brooke. SUS: A "quick and dirty" usability scale. Usability evaluation in industry. s.l. : CRC Press, 1996.
Lowe DJ, Ireland AJ, Ross A, et al. Exploring situational awareness in emergency medicine: developing a shared mental model to enhance training and assessment. Postgraduate Medical Journal. 2016;92:653–8.
Evans SM, Murray A, Patrick I, Fitzgerald M, Smith S, Cameron P. Clinical handover in the trauma setting: a qualitative study of paramedics and trauma team members. Qual Saf Health Care. 2010;19: e57. https://doi.org/10.1136/qshc.2009.039073.
Carter AJE, Davis KA, Evans LE, Cone DC. Information loss in emergency medical services handover of trauma patients. Prehosp Emerg Care. 2009;13(3):280–5.
Lewis JR. The system usability scale: past, present, and future. International journal of human-computer interaction. 2018;34(7):577–90.
Shen Z, Zhang L, Xiao X, et al. Icon familiarity affects the performance of complex cognitive tasks. I-Perception. 2020;11(2):1–18.
Acknowledgements
Dr Alexandra Mather and Dr Joe Campbell for support in the early development of the App. The WoS Innovation hub for supporting the project team developing the app. The Scottish Trauma Network and Digital Health Scotland for clinical and technical expertise from inception to delivery. InnovateUK for providing grant funding Project 104540.
Funding
Funded by InnovateUK Project 104540.
Author information
Authors and Affiliations
Contributions
DJL and JB conceived and designed the study. JB, EW and LL carried out participant recruitment and conducted the study. JB and EW performed data analysis and interpretation. JB drafted the manuscript and all authors provided editorial comments. The author(s) read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
There was no requirement for ethics approval and each participant was given a copy of the study protocol and verbally consented.
Consent for publication
Not applicable.
Competing Interests
The authors declare that they have no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Additional file 1.
Transcripts of five cases.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Butler, J., Wright, E., Longbottom, L. et al. Usability of novel major TraumaApp for digital data collection. BMC Emerg Med 22, 39 (2022). https://doi.org/10.1186/s12873-022-00578-9
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12873-022-00578-9