Introduction

Mechanical thrombectomy has become a standard of care for patients presenting with anterior circulation emergent large vessel occlusion (ELVO) [1,2,3]. The main inclusion criteria for most mechanical thrombectomy trials were a limited infarct core on parenchymal imaging and evidence of ELVO on angiographic imaging. Several trials were using additional imaging modalities to improve patient selection for mechanical thrombectomy, such as perfusion imaging to identify salvageable brain tissue or dynamic CT angiography (CTA) to assess the leptomeningeal collateral status [4,5,6,7,8,9].

There is no consensus on the optimal combination of imaging modalities for the decision on mechanical thrombectomy. Two recent trials on an extended time window used CT perfusion (CTP) for patient selection, and there is a current debate whether CTP should be part of the standard of care acute stroke imaging protocol, especially for patients who present beyond the 6-h time window [6, 7].

We aimed to investigate the inter-rater agreement in decision-making on mechanical thrombectomy for ELVO based on five commonly used acute stroke CT-imaging protocols and to assess the added value of CTP and delayed-phase CTA to standard non-contrast computerized tomography (NCCT) and arterial-phase CTA.

Materials and Methods

For this study, we selected patients from a cohort of consecutive patients who presented between January 2015 and November 2016 to our center with acute focal neurological deficits (“code strokes”). The default acute stroke CT scan protocol consisted of concomitant NCCT, CTA with an arterial and delayed-phase (aCTA and dCTA), and CTP. Patients were included in the study when they fulfilled the following criteria: (1) acute focal neurological deficits related to an arterial occlusion on CTA; (2) imaging performed within 12 h after symptom onset; (3) concomitant NCCT, aCTA, dCTA, and CTP as the first line of imaging investigations. We obtained approval from the Research Ethics Board of our institution. The board waived the need for patient consent because of the retrospective nature of the study.

Imaging acquisition

Images were obtained on a 320 multi-detector row CT system (Toshiba Aquilion ONE; Toshiba Medical Systems Corporation, Japan). All patients underwent standard NCCT with 3-mm section thickness covered from 1 cm below base of the skull to the vertex followed by 4D CTP covered from 1 cm below base of the skull to the vertex, and then helical aCTA and dCTA of the head and neck covered from the aortic arch to the vertex. For CTP, a total of 50–60 mL of Ultravist 370 contrast agent was injected at a rate of 6 mL/s. The mask scan was done at time 0. Dynamic volume scanning began after a 5.5-s delay after contrast material injection. The total scan time was 75.5 s with 80 kVp for mask and dynamic volume, 300 mA for mask, and 150 mA for dynamic volume. The images were acquired with a 5-mm section thickness. The dynamic unsubtracted volumes were then transferred automatically to the RAPID Server, which then sent processed images to PACS. For helical CTA of head and neck, a total of 50 mL of the same contrast agent was injected at a rate of 5 mL/s followed by a saline chase of 20 mL. After a 5-s delay, scanning started at 150 HU on the aortic arch or manual trigger when contrast seen in the arch. The second phase was acquired after a delay of 5.7 s. Images were acquired with a 1 to 2-mm section thickness. The exposure for helical scan was 120 kVp and 300 mA. The axial images were then reconstructed at 2-mm overlapping sections. Multi-planar reconstructions for axial, coronal, and sagittal images of the Circle of Willis were performed with 2-mm thickness at 2-mm intervals for both aCTA and dCTA. Thick-section maximum intensity projections at 30-mm thickness and 2-mm intervals for axial images and 10-mm thickness and 2-mm intervals for coronal and sagittal images of both aCTA and dCTA were also reconstructed.

Imaging review

The imaging studies were grouped into five commonly used protocols: (1) NCCT and aCTA; (2) NCCT, aCTA, and CTP; (3) NCCT, aCTA, and dCTA; (4) NCCT, aCTA, dCTA, and CTP; and (5) NCCT and dCTA. Two interventional neuro-radiologists independently reviewed the five imaging protocols for each patient. To avoid recall bias, images were evaluated at minimum 6 months after the patient had presented to our center; their images were de-identified and presented in random order. The raters reviewed one imaging protocol on each occasion with at least a 1-month interval between the different imaging protocols. Patients were presented in random order to further avoid recall bias. Clinical information including gender, age, neurological deficits including the NIH Stroke Scale score, and time interval between symptom and onset to the image study, were provided. The raters decided on the indication for mechanical thrombectomy while being blinded to the treatment decision made in clinical practice. They indicated how confident they felt about their decision, categorized into “uncertain,” “low confidence,” “confidence,” and “high confidence.” The raters were asked to indicate which factors impacted their decision: tissue at risk, infarct core, collateral status, location of occlusion, procedural risk, age and premorbid status of the patient, and image quality.

For discrepant treatment decisions or when the rater was indecisive, the raters reached consensus in a separate session.

Statistical analysis

Inter-rater agreement, dichotomized level of confidence (“uncertain/low confidence” vs “certain/high confidence”), and treatment decisions were compared for the five imaging protocols. We assessed whether addition of CTP and/or dCTA to NCCT and aCTA resulted in different treatment decisions.

We used descriptive statistics for patient characteristics and Cohen’s kappa for the inter-rater agreement on treatment decisions. Kappa values were interpreted as follows: 0.00–0.20, slight agreement, 0.21–0.40, fair agreement, 0.41–0.60, moderate agreement, 0.61–0.80, substantial agreement, and 0.81–1.00, almost perfect agreement [10].

We used the Cochran’s Q-test to compare treatment decisions and the rater’s level of confidence for the five different scan protocols. McNemar testing for marginal homogeneity was used to assess changes in treatment decisions when CTP, dCTA, or both were added to NCCT and aCTA. Data analysis was performed using IBM SPSS Statistics software, version 24.0 (IBM Corporation, Armonk, NY, USA).

Results

The total number of patients presenting as a code stroke who underwent NCCT, CTA, and CTP was 532. From this cohort, 459 patients were excluded because they had no symptomatic intracranial occlusion (n = 232), intracranial hemorrhage (n = 47), a stroke mimic (n = 124), or they had no delayed-phase CTA (n = 56). So, the total number of included patients was 73. Their baseline characteristics are summarized in Table 1.

Table 1 Baseline characteristics
Full size table

Inter-rater agreement

The inter-rater agreement on treatment decisions was moderate for the protocols consisting of two sequences and substantial for protocols consisting of three or more sequences. The inter-rater agreement was highest for NCCT in combination with aCTA and CTP (Table 2).

Table 2 Inter-rater agreement
Full size table

Inter technique agreement

The decision to perform mechanical thrombectomy did not significantly differ between the five protocols (p = 0.05). For the majority of cases, the raters had a high level of confidence in their decision. Rater 1 had a lower level of confidence in decision-making based on NCCT in combination with aCTA alone compared with the other imaging protocols (p < 0.001), and rater 2 had a lower level of confidence in decision-making based on NCCT and dCTA alone compared with the other imaging protocols (p < 0.001).

Adding CTP to NCCT and aCTA increased the inter-rater agreement with κ 0.198. Nine more patients would be offered mechanical thrombectomy (Fig. 1), and two patients would be withheld from mechanical thrombectomy due to the presence of established infarcts involving functional regions. Adding dCTA to NCCT and aCTA increased the inter-rater agreement with κ 0.120, and three more patients would be treated. Adding both CTP and dCTA to NCCT and aCTA resulted in five more patients being offered mechanical thrombectomy. These differences in treatment decisions did not reach statistical significance (Table 3). We repeated all analyses for patients with anterior circulation occlusions only and obtained similar results.

Fig. 1
figure 1

CTP changes decision-making on mechanical thrombectomy. a NCCT, right frontal established infarct (arrow). b aCTA, right M1 occlusion (arrow head), poor collaterals in the region of the established infarct. c dCTA, poor collaterals in the region of the established infarct. d CTP, perfusion mismatch (pink areas represent infarct core; green areas represent infarct core with penumbra). Based on NCCT, aCTA, and dCTA, the decision was not to proceed with mechanical thrombectomy due to the presence of established infarct with poor collaterals in the right frontal lobe and no parenchymal change with good collaterals to maintain normal blood supply in the other regions supplied by the right MCA. When CTP was added to NCCT, aCTA, and dCTA, the decision was made to proceed with mechanical thrombectomy due to the presence of a larger than expected area of perfusion mismatch

Full size image
Table 3 Added value of CTP and delayed-phase CTA to NCCT with arterial-phase CTA alone
Full size table

The most common reason why the raters would not advise mechanical thrombectomy was the extent of the infarct core. This varied from 68% of patients excluded from thrombectomy based on NCCT and aCTA up to 94% of excluded cases based on NCCT, aCTA, and CTP.

Discussion

In this study, we found that the inter-rater agreement on mechanical thrombectomy increased from moderate to substantial when a third CT modality was added, and that the raters’ confidence level was higher for three-modality than for two-modality CT protocols. This did not result in significant differences in decision-making on mechanical thrombectomy. Interestingly though, there was a tendency toward a lower threshold for mechanical thrombectomy when CTP and/or dCTA were added to NCCT and aCTA.

The trials that demonstrated mechanical thrombectomy to be superior to standard of care, each used different imaging protocols and different criteria for patient inclusion. The MRCLEAN trial included patients with an intracranial occlusion in the anterior circulation seen on a single phase CTA, regardless of the ASPECTS and of the collateral score. [11] Patients were eligible for the ESCAPE trial if they had a proximal intracranial occlusion in the anterior circulation seen on (preferably multiphase) CTA, moderate-to-good collaterals, and ASPECTS higher than five [8]. Apart from demonstrating a proximal intracranial occlusion on CTA, the EXTEND-IA and SWIFT PRIME trials selected patients based on mismatch between ischemic penumbra and ischemic core on CTP [4, 9]. REVASCAT used a threshold of ASPECTS higher than six on NCCT when patients presented within 4.5 h after last seen well, and for patients who presented later, the ASPECTS needed to be assessed on CTA source images, CBV maps of CTP, or on DWI-MR [5]. DAWN and DEFUSE-3 used CTP core volume to select patients presenting beyond 6 h after last seen well [6, 7].

The main goal of imaging in acute stroke is to directly or indirectly estimate the amount of salvageable tissue in proportion to the amount of irreversibly damaged tissue. For this study, we focused on CT modalities. NCCT is widely available and allows us to estimate the infarct core using ASPECTS. The inter-rater agreement for ASPECTS is moderate though, and it can be a challenge to reliably assess the infarct core [12]. CTA source images may assist in a more accurate interpretation of the infarct core, which appears as an area of hypo attenuation due to decreased blood volume, but it remains challenging to assess the infarct core [13]. Compared with CTP and CTA source images, NCCT has been demonstrated to have the lowest interobserver agreement for individual ASPECTS regions [14].

ASPECTS in combination with information on the collateral circulation will help predict the likelihood of infarct core expansion. Several studies showed that a robust collateral status is associated with reduced infarct expansion and more favorable functional outcomes [15,16,17,18]. Collaterals may already be seen in the arterial-phase of a CTA but one or more delayed-phases allow evaluation of slower filling collaterals [19, 20]. Also, the thrombus length can be better assessed than on an arterial-phase CTA [21].

CTP delineates the estimated infarct core from the surrounding penumbra, and penumbra from the surrounding unaffected tissue. RAPID post-processing software uses thresholds for the infarct core of more than 30% reduced cerebral blood flow and for the penumbra of more than 6 s delay in time to peak [22]. So CTP is an indirect measure to estimate reversible and irreversible tissue damage and is therefore still a surrogate biomarker [23]. Infarct core is dependent not only on the magnitude but also on the duration of decreased cerebral blood flow. In the extended time window, leptomeningeal collaterals may result in partial reperfusion and underestimate the infarct core. Conversely, the infarct core may be overestimated in case of early presentation and reperfusion. Therefore, to assess infarct core and penumbra, CTP findings should always be related to the NCCT and CTA results [24].

It is unknown which mismatch threshold should be used to determine if mechanical thrombectomy will be effective. An observational study found a cut-off point of a 20-mL final infarct volume that differentiated between good and poor clinical outcome [25]. The DAWN trial and DEFUSE-3 trial were designed to select patients for mechanical thrombectomy who present beyond 6 h after last seen well. They used clinical-imaging mismatch based on CTP and allowed a larger core, up to 50–70 cc, as long as the patient was under the age of 80 and had severe neurological deficits. Both trials were overwhelmingly positive [6, 7].

The drawback of CTP is its limited availability and variability in diagnostic accuracy among different software packages [24, 26,27,28,29,30]. There is a concern that CTP may overestimate core and prevents patients from being offered mechanical thrombectomy, but our results indicate that CTP rather rules in than it would rule out patients from being offered mechanical thrombectomy. An observational study did not find a difference in the accuracy of ASPECTS on NCCT and core volume on CTP in the determination of infarct volume on DWI [28], but two meta-analyses found a higher diagnostic accuracy in the detection of infarct core on CTP compared with NCCT [26, 29]. Another study compared several post-processing software packages and concluded that RAPID-software has the highest accuracy in the prediction of final infarct volume [30]. Also, for the SWIFT PRIME cohort, CTP with RAPID-software predicted 27 h infarct volume in patients treated with mechanical thrombectomy [22, 31].

The main strength of this study is that it concerns a relatively large cohort of patients who had all the CT-imaging modalities. Given that all patients with acute focal neurological deficits are being offered the complete imaging protocol by default, regardless of the likelihood of a large vessel occlusion, we believe that selection bias is limited. In contrast to a previous study, we chose to be pragmatic and left it at the discretion of the observer to provide a decision on mechanical thrombectomy without defining imaging criteria since no validated thresholds for ASPECTS and core-penumbra mismatch are available at this point [20, 24]. We found the highest level of agreement and largest number of patients being offered mechanical thrombectomy when using NCCT, aCTA, and CTP, but this study was potentially underpowered to show statistically significant differences in treatment decisions between the scan protocols. A second limitation is the risk of recall bias; although we aimed to avoid this by introducing a time delay between patient presentation to our center and the imaging assessments as well as between the different imaging protocols. Third, we merely compared decisions between scan protocols not being able to conclude whether these decisions were the most optimal for that patient. This was not feasible since it concerned a retrospective study in a single cohort. Finally, decision-making in acute stroke is rapidly changing over time because of the availability of new data which may impact the clinical relevance of this study. Yet, there is no consensus on the most appropriate scan protocol. Replication studies would be valuable to determine external validity.

There is a trade-off between the benefit of obtaining additional information from multimodal CT imaging versus availability, costs, scan duration, additional contrast administration, and radiation exposure. Our results do not underline a necessity to include CTP because the treatment decisions did not significantly differ, but we would advocate for usage of at least a three-modality CT protocol if available and feasible. The higher level of decision-maker agreement and confidence may lead to faster decision-making.

Conclusions

There is substantial agreement and a higher level of confidence in decision-making on mechanical thrombectomy in acute stroke when CTP and/or a delayed-phase CTA is added to NCCT and arterial-phase CTA. Using at least three CT modalities creates a tendency towards a lower threshold for mechanical thrombectomy.