Introduction

Non-enhanced cranial computed tomograph (CT) scan is a crucial diagnostic tool in emergency departments as it could be used to diagnose patients with head trauma and serious illnesses [1]. According to the estimation of the National Center for Injury Prevention and Control, in any given year, 1.7 million Americans suffer from traumatic brain injury and almost 1.3 million people require treatment in emergency departments [2]. In order to have an instant decision-making and manage patients appropriately without losing the golden time, a time-critical interpretation of CT scans is inevitable. Since many of these CT scans are performed after the regular working hours of attending radiologists, emergency physicians and residents are required to interpret CT scans and make therapeutic decisions accordingly. The accuracy of this interpretation has been the subject of some studies. The disagreement between these groups was 20–38.7 %, some of which did not result in a major change of management [3, 4].

In this study, the impact of incorrect non-enhanced brain CT scan interpretations by emergency medicine team on patients’ primary and secondary outcome was evaluated and the attending neuroradiologist’s reports were considered as the gold standard.

Materials and methods

This was a prospective study and all adult patients who had undergone a cranial CT scan at the emergency department of two major referring hospitals and trauma centers during the clinical shifts of the research team, enrolled in this study and were followed for 28 days in order to assess the primary and secondary outcomes. All brain CT scans were filed and viewed in the picture archiving and communication system (PACS) of the hospitals. The exclusion criteria were as follows: (1) Low quality cranial CT scans that could not be interpreted. (2) Follow-up CT scans with radiologist reports. (3) CT scans of children under 18 years old. (4) CT scans performed in other hospitals or with documented neuroradiologist reports on admission. The primary outcome was considered as mortality within 28 days after admission. The secondary outcome was the spectrum of morbidity, from minimal life style change with minimal neurologic deficit to major life style change with severe neurological deficit.

Since in both hospitals, the neuroradiologists were not routinely available for reporting the scans 24 hours a day and 7 days a week, radiologists’ reports for all emergency room brain CT scans were available in PACS within two working days after the scans had been done. As a result, emergency medicine interpretation was considered final for patient management, unless the emergency physicians requested an emergent neuroradiologist consult for reporting a suspicious scan, and those scans were excluded from the study.

All patients’ emergency room medical template records were reviewed by the panel of experts in order to compare emergency physician’s interpretation with radiologists’ report and its impact on patient’s management.

Conducting this study and accessing medical records were approved by the hospital’s board of management and ethics. Informed written consents from patients, their legally authorized representatives or closed families on their behalf were collected and filed. A representative of the board of clinical governance committee of both hospitals attended all panel sessions.

During each clinical shift, emergency medicine residents and the attending emergency medicine physician interpreted the cranial CT scans which they had requested and recorded their interpretations on a template sheet. Their interpretation and management were considered as final in each hospital as long as the patients were managed in the emergency room and not transferred to other wards, ICU, or discharged. If the emergency team requested a neuroradiologist consult for reporting patient’s scan while managing the patient, that patient was excluded from the study. Based on the emergency team interpretation, patient management was done. At the end of each shift, all interpretations were collected and filed. Patients were followed up by the research assistant at the end of the first 24 hours and on the 28th day after admission by making phone calls to the patient or attending physicians. Neuroradiologist reports were considered as the gold standard, and the emergency teams’ interpretations were compared to their reports. After each comparison, the results were then divided into two main groups, the agreement group in which the reports and the emergency team’s interpretation were exactly the same and the disagreement group, in which the reports were not the same as emergency team’s interpretation.

All medical records in the disagreement group were discussed in the expert panel every working day. The panel consisted of an attending neuroradiologist, an attending emergency medicine physician, a neurologist, and a neurosurgeon. The panel discussed further classification of the disagreement group based on the management protocol and the possible impact of the emergency team’s interpretation on patient’s outcome.

The patients in the disagreement group were then subdivided into two major categories: clinically significant and insignificant. The clinically insignificant group consisted of any disagreement that did not have a clinical impact on the management of the patient. For instance, congenital anomalies were in this category. However, in the clinically significant group, the disagreements had a spectrum of clinical impact on patients’ management and outcome.

The clinically significant disagreement was further subcategorized into two major subcategories:

In the first subcategory, all patients were only managed by the emergency medicine team without any further consult and had been discharged home; however, patients in the second subcategory were managed by the emergency team while consulting a neurologist or a neurosurgeon. However, emergency medicine team’s decision was considered final.

Each subcategory was then divided into three main branches:

  • Branch one: no definite change on outcome. Despite the misinterpretation and possible minimal mismanagement, according to the decision of the expert panel, it did not have an impact on patient outcome. In 28-day follow-up, there was not any added risk of mortality or morbidity. Patients whose managements were changed by consultant neurologist or neurosurgeon were also categorized in this branch.

  • Branch two: possible change on outcome, unnecessary management was done. The misinterpretation has resulted in unnecessary medication, procedures, increase length of stay in ER or ward, or unnecessary imaging and increased cost. In 28-day follow-up, there was not any added risk of mortality or morbidity.

  • Branch three: definite changes on outcome, necessary management was not provided. The misinterpretation resulted in mismanagement and misdiagnosis which put patients’ safety at risk and definitely jeopardized patients’ outcome. In 28-day follow-up, there was an added risk of mortality or morbidity.

    The calculated sample size was 240. After all data were collected, the results were analyzed with kappa correlation using SPSS version 11 [5].

Results

In the 3-month period of this study, 450 from 1840 non-enhanced cranial CT scans were enrolled in the study. Of the 450 patients, 190 patients (42.2 %) were women. The mean age of the patients was 51.5 ± 8.7 years. The major clinical indications for cranial CT scan were trauma (37.4 %), stroke (25.5 %), decreased level of consciousness (19 %), confusional state (11 %), and headache (7.1 %).

The most common findings in radiologists’ reports were normal (56.9 %), brain atrophy (6 %), ischemic infarct (4.7 %), brain edema (4 %), and lacunar infarct (3.6 %).

Three hundred eighty-nine interpretations (86.4 %) of emergency medicine team were in the agreement group, and 61 reports (13.6 %) were in the disagreement group (Fig. 1). The inter-rater reliability between emergency medicine team and neuroradiologists was substantial (kappa = 0.68) and statistically significant (p < 0.0001). The detailed kappa correlation, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for each pathological finding have been summarized in Table 1. When considering each pathology individually, the lowest kappa correlation between emergency team and neuroradiologist report was for intracerebral hemorrhage (kappa = 0.34) and pneumocephalus (kappa = 0.4) (Table 1).

Fig. 1
figure 1

Algorithm of patients based on the CT scan reports compared to the gold standard

Full size image
Table 1 Comparing the analysis of each pathologic report between emergency medicine team and the gold standard (radiologists’ report)
Full size table

In the disagreement group, one disagreement was clinically insignificant (arachnoid cyst). The medical records of two patients with the radiologist’s report of scalp hematoma and the other one with small subdural hematoma were missed during follow-up, and their outcome was unclear as they had left the hospital against medical advice. These two patients have been excluded from the study. Fifty-eight patients were placed in the category of clinically significant (Fig. 1). These 58 patients were followed for 28 days, and their medical records were discussed in the panel. When considering these patients in the panel, 41 patients (70.6 %) were placed in the no-definite change in outcome category (branch one), 12 patients (20.7 %) had received unnecessary treatment (branch two); however, in five patients (8.6 %), the necessary treatment had not been provided (branch three) (Fig. 1).

The 28-day mortality rate was 0.2 % (one case). When considering morbidity in the definite change subcategory (branch 3), only three patients had a slight headache.

The primary and secondary outcomes and radiologist reports of the patients who were placed in the “necessary treatment not provided” and “unnecessary treatment” group are summarized in Tables 2 and 3, respectively.

Table 2 Primary and secondary outcome of patients in whom necessary management was not provided
Full size table
Table 3 Primary and secondary outcomes of patients in whom unnecessary management was done
Full size table

When considering unnecessary treatment group (Table 3), most patients had normal reported neuroradiologist CT scans; however, most cases were reported as cortical contusions or cortical ischemic infarcts by the emergency team and consulting teams.

Discussion

In this study, 450 patients’ cranial CT scans were interpreted and reported by emergency medicine team and attending neuroradiologists. The agreement rate between neuroradiologist reports and emergency medicine team was 86.4 %. Apart from one case of arachnoid cyst, the rest of patients in disagreement group (58 patients) had clinically important diagnosis in their brain CT scans. However, among these 58 patients, only five patients did not receive the necessary management (Table 2). Four of them were discharged from the emergency room and during the follow-up had minimal problems without lifestyle change. However, the expert panel believed that they should have been sent to other wards for further investigation and management. The other patient in this group, who died in emergency room, had chronic bilateral subdural hematoma (cSDH). However, according to the panel, this patient’s death was because of hepatic encephalopathy and not related to the missed CT scan finding. There were 12 patients in the unnecessary management subgroup (Table 3). In most cases, the unnecessary management resulted in increase length of stay at emergency department, unnecessary extra imaging, and seizure prophylactic medicine. According to the expert panel, in only one patient (patient M) (Table 3), lumbar puncture (LP) could have harmed the patient, but eventually, this procedure did not lead to any morbidity and CSF analysis was normal. As nearly half of the patients in this group had a history of head trauma and they had symptomatic mild traumatic brain injury at the time of brain CT scan interpretation by the caregiving team, this might have an impact on their interpretation, reporting most of them as cortical contusions. Due to the expert panel’s idea in this group, an increase in length of stay at the ER was inevitable because patients were symptomatic and they needed medication and observation; however, requesting extra imaging and invasive procedure were unnecessary and the final decision could have been postponed till the next day when the neuroradiologist report was available on PACS. During the 28-day follow-up, the mortality rate was 0.2 % (one case), and due to expert panel idea, this was not because of the emergency team missed interpretation. The inter-rater reliability between emergency medicine team and attending radiologists was substantial (kappa = 0.68) and statistically significant (p < 0.0001).

Similar studies have been done comparing emergency medicine residents with neuroradiologists. In a study done by Arendts et al., the overall concordance rate was 85.2 % (kappa = 0.69) which is close to the findings of our study. The discordant rate was 14.8 and 41.1 % of the discordant reports had a clinical consequence for patients [6]. In another study, the overall discordance was 37.1 %, and from 555 cases, only three cases were managed inappropriately without any major adverse effects [3]. In our study, 17 patients suffered from either unnecessary treatment or not receiving the required treatment in whom outcome was not jeopardized directly by misinterpretation. When comparing five emergency department medical staffs with two radiologists interpreting cranial CT scans, the overall agreement was 86.6 % and the false negative rate was 4.2 %, and radiologically, all false negative cases had a subtle change without the need to be immediately transferred to the neurosurgery ward. No finding that could result in a different overall management was missed [4].

While comparing the performance of emergency medicine residents and physicians with radiologists, a study showed that the performance of emergency medicine residents and physicians was significantly discordant with radiologists (p < 0.0001), and the emergency team made significant mistakes in interpretation [7].

Other studies have compared the performance of radiology residents with neuroradiologists. In one study, 1324 head CT scans of patients in the emergency department were interpreted by radiology residents and then a neuroradiologist. The agreement rate was 91 %, and the significant disagreement rate was 2 %. The level of training of the residents had a significant impact on their reports. The main missed pathologies were fractures and chronic ischemia [8]. In another study, 22,590 cranial CT scans were interpreted by radiology residents and neuroradiologists, and the main discrepancy was intracranial hemorrhage (13.6 % of total discrepancy) [9]. Another study showed that, significant misinterpretation occurred in 12 (0.09 %) cases and a significant change in patient management occurred in 12 cases and the potential change in patient outcome was rare [1].

All the above studies have had similar results to our study.

Conclusion

We conclude that although the overall agreement in this study was 86.4 %, 58 cases were in the clinically significant disagreement group with neuroradiologist report of which five cases were not given the urgent necessary treatment and 12 cases underwent unnecessary urgent management leading to financial burden for both the patient and the emergency department. Most of these patients eventually were discharged home and referred to outpatient clinics without major problem. The direct mortality and morbidity imposed by this mismanagement was unremarkable due to the expert panel idea and the 28-day follow-up. We believe that the brain CT scan interpretation by emergency medicine team and its disagreement rate could result in nonsignificant morbidity and mortality. As patient safety and financial burden could be major concerns in the overcrowded emergency room, we recommend that in mild to moderate symptomatic patients with minor traumatic brain injury and suspicious primary brain CT scan and when invasive procedures like lumbar puncture could be postponed till next day, emergency medicine team might be able to observe these patients until the neuroradiologists’ reports of non-enhanced CT scans are ready. We also suggest that in such cases, emergency team put a priority alert for radiologist to report these scans on their earliest time possible as this might result in better and more confident patient management by the emergency medicine team. However, the emergency medicine team performance in this study showed that they are capable of interpreting the brain CT scans with life-threatening diagnosis. Depending on the health and insurance policy of centers around the world, this liability varies significantly. In medical settings in which instant neuroradiologists reports are not available, background emergency physicians’ training on how to interpret brain CT scans is of paramount importance. Arranging training workshops on brain CT scans interpretation for emergency physicians by neuroradiologists might be highly beneficial. In our study, all emergency medicine physicians had been trained by neuroradiologists on interpreting CT scan of the brain, spine, chest, abdomen, and pelvic as this is mandatory in the emergency medicine residency curriculum.

In this prospective study, the interpretation of CT scans occurred bedside at emergency department under stressful conditions. As a result, our results were equal to the true settings of emergency departments. Further studies could be done to evaluate the effect of education on emergency team’s performance. Moreover, patients in the agreement group were not part of the follow-up process. We recommend this follow-up in future studies.