Introduction

Guidelines and algorithms are invaluable aids when selecting patients for treating acute ischemic stroke. Nevertheless, simultaneously assessing the multiple clinical and radiological factors to reach a therapeutic decision can be complex. Furthermore, advances in imaging technologies like multimodal CT including CT perfusion (CTP) can bring added value to the decision-making process on whether to perform mechanical thrombectomy (MT), especially considering its widespread use in patient selection at stroke-centers, yet it is underrepresented in the current guidelines [1, 2].

The decision-making process in real-life case selection for MT is sometimes less than straightforward, especially when patient characteristics are not fully represented in the guidelines [3]. There may sometimes be discrepancies between the clinical and/or radiological presentation of acute stroke, leading to debate over both sides of the question, namely whether to propose or withhold treatment using MT, with cogent arguments existing for both these options.

On the other hand, notwithstanding the guidelines, variability in decisions to perform or not iv thrombolysis by neurologist has been highlighted [4,5,6]. As a consequence, variability in decision-making as to whether to perform MT in different clinical and radiological settings needs to be evaluated to provide feedback allowing further discussion with a view to reaching consensus.

The aim of this study was to exploratively evaluate the assessment of indications for MT in 12 selected cases by 50 international colleagues with experience in MT using an interview-administered questionnaire. Additionally, the reasons for excluding intervention and variability in the decisions for each case were analyzed.

Materials and methods

Ethical approval by the local institutional board was obtained. An informed consent was not waived.

Case selection

Twelve real-life cases were selected for inclusion on the questionnaire. These patients were evaluated during 2017–2018 by different interventional neuroradiologists from our department, who decided how to proceed. The specific clinical and/or radiological characteristics of the cases contributed to inter and intra-disciplinary discussion about the indications for or against MT. Taking into account the limitation on the number of cases to be included on the questionnaire, we decided by consensus not to include tandem occlusions and strokes of the posterior circulation, since they could deserve their own questionnaires. The final selected cases focused on occlusions in the anterior circulation, namely the middle cerebral artery (MCA), the internal carotid artery (ICA), and the distal artery occlusions. As uncertain aspects in the decision-making process for MT are to be addressed, we agreed on the need to study and cover the following spectrum: (a) MCA occlusions: cases with different degrees of mismatch/infarct. Influence of age on decision; (b) ICA occlusions: cases with acute non-stenotic occlusion of the external and/or internal circulation, with and without intracranial occlusion, including the combination of ICA occlusion with “normal” perfusion. Cases with large core infarcts vs time window (unknown and short time window); (c) distal vessel occlusions: cases with different degrees of mismatch/clinical symptoms, proximal M2 vs distal M2 segment occlusions, occlusion of the A2 segment of the anterior cerebral artery.

There was also a first example case (case 0). One case (case 6) having typical clinical and radiological characteristics favoring a decision to perform MT was included for control purposes. Summarized descriptive details of the individual cases included on the original questionnaire are available as supplementary material (Figures: I to XIII).

Questionnaire design and procedure

The questionnaire was implemented as a Microsoft Access 2016 database and was administered by an interviewer at an advanced neuro-interventional congress (2018) attended by experienced international neurointerventionalists. Two of the authors were in charge of helping raters with questionnaire workflow and ensured that raters answered all questions alone (no other potential participants could take part at the same time), without interruptions, and after appropriately analyzing all the information furnished. They encouraged raters to answer concerning the indications for or against mechanical thrombectomy as if the patients (cases) were already in the CT room at their own hospitals. Retrospectively changing answers was not allowed. All participants were instructed not to discuss the cases with other potential participants. The questionnaire interview was conducted and recorded on a laptop suitably set up for that purpose. The workflow of the questionnaire was the same for all participants.

Demographic information on the raters included country, age, sex, hospital, specialty, years of training as assistant operator, years of experience as first operator, and number of MTs performed yearly, as well as the presence of a stroke unit, the availability of MT and 24/7 on-call MT, the total number of MTs performed yearly, and the number of team members at their hospitals. The type of stroke imaging on admission and the first-choice technique for MT used were also recorded. In addition, all raters provided an estimated percentage team agreement with the therapeutic decision.

The structure and number of images provided were the same for all 12 cases and for the example case (case 0), namely eight representative images per case at two different brain levels. The images for each case were selected by one of the authors (AVG) and re-evaluated by an intern for suitability before starting the questionnaire. Further information about imaging acquisition is presented as supplemental material. A pair of non-enhanced CT (NECT), CBF map, CBV map, and CT angiography (CTA) images that best revealed the most relevant characteristics of each case were selected from the multimodal CT on admission to help the raters to quickly and easily identify: (a) infarct demarcation on the NECT scan; (b) the location of the occlusion in the CTA and the degree of collaterals; and (c) qualitative analysis of the degree of mismatch at two brain levels. Additionally, for each case, the raters were allowed to decide whether or not the images were sufficient to provide an answer, and this information was also recorded. A brief clinical information report was included as well. For some of the cases, the NIHSS score or time window was a primary unknown (primary lack of information just as it was for the patients arriving at our hospital). That is, all the same clinical data that were not previously available were also omitted from the questionnaire.

Statistical analysis

The sample size calculation performed to estimate the total number of cases and the number of raters needed to reach a sufficient degree of precision in estimating the percentage decision-making concerning MT yielded 12 cases and 50 raters. Details are given in the supplemental material.

Statistical analyses were performed using SPSS version 24 (IBM Corp. Released 2016; IBM SPSS Statistics for Windows, Version 24.0, Armonk, NY; IBM Corp.). The continuous variables in the descriptive analysis were reported as the mean, standard deviation, median, 25% percentile, 75% percentile, interquartile range (IQR), minimum, and maximum. The categorical variables were represented as the absolute and relative frequencies. The Mann-Whitney U test or the Kruskal-Wallis test and Fisher’s exact test were used to compare baseline continuous and categorical variables between the groups. Spearman regression coefficients were calculated. P values ≤ 0.05 were deemed to indicate a statistically significant difference. All reported p values are two-sided.

The Simpson index was applied to the observed relative frequencies for the decisions concerning MT to assess the diversity of answers [7, 8].

The influence of rater and patient characteristics on the decision whether to employ MT was analyzed using logistic regression (LR) models. For case-based analyses, classical LR analysis was used. For a pooled analysis of all cases, the data structure was modeled using generalized estimation equations (GEEs) for binary outcomes. An unstructured working correlation matrix was assumed. If numerical problems were encountered, an exchangeable working correlation was used as a fallback procedure. Univariate and multivariate analyses were performed. Odds ratios (ORs) including 95% confidence intervals (CI) were reported. Influencing factors were tested by the Wald tests within the respective models. Further information on the statistical analysis appears in the supplemental material.

Results

Demographics

Fifty international raters answered the questionnaire, evaluating the 12 cases. The median age of the raters was 42 years (IQR, 38–47), and 12% of them were women (n = 6). All participants performed MT: the median number of years worked as first operator was 7 (IQR, 4–22), and the time they trained as an assistant operator was 3 years (IQR, 2–5). The demographic characteristics of the raters and hospitals are available as supplementary material (Table I). The median number of MTs per hospital/year was 100 (IQR, 50–141). Interventional teams were composed of 3 or more doctors in 74% of cases (median IQR, 4 [2–4]), and the median number of MTs performed as first operator/year was 39 (IQR, 25–60). There was a significant positive correlation between the number of MTs/hospital and the number of people per team (R = 0.301; p < 0.001). Multimodal CT (NECT, CTA, and CTP) was the most frequently used imaging modality on admission (58%, 29/50), more than NECT+CTA (32%, 16/50) and multimodal MRI (12%, 6/50).

Seventy-four percent of participants used stent retrievers (SR) as the first-choice technique for MT (37/50) and aspiration alone in 26% (13/50). Overall, the use of a balloon guide catheter was reported in 64% of cases (32/50). The technique selected for MT was independent of rater age (p = 0.935), years of experience as first operator (p = 0.510), and number of MTs performed per rater (p = 0.127) or per hospital (p = 0.243). Sixty-two percent of raters estimated their team agreement for therapy decisions at 100–90% (31/50), 43% at 90–80% (17/50), and 4% at 70–50% (2/50).

Per-case rater agreement

Table 1 presents a summary description of the cases, the therapeutic decision, and the main reasons for excluding MT. The relationship between the Simpson index (agreement) and the percentage selection of MT for treatment is depicted in Fig. 1. Two cases (cases 11 and 7) showed higher agreement among raters to exclude MT, and, in contrast, another two cases (cases 6 and 9) displayed higher agreement to perform MT. Case 12, followed by cases 1 and 4, was located at the bottom of the curve, with the lowest Simpson scores.

Table 1 Radiological description of the decision concerning mechanical thrombectomy by case
Full size table
Fig. 1
figure 1

Relationship between the Simpson index (diversity) and percentage recommendation to perform MT (% yes MT). Each case is represented by a circle whose area depicts the number of recommendations to exclude MT

Full size image

Agreement to exclude MT

Case 11: occlusion of the internal carotid artery (ICA) + M1, CT demarcation and match involving the MCA territory, and the ipsilateral anterior cerebral artery (ACA) territory. Ninety-four percent of the raters (46/49) ruled out MT for this patient. A total of 135 reasons for not performing MT were given, namely small CBV/CBF mismatch (34%, 46/135), size of infarct core shown by CBV (32.6%, 44/135), NECT infarct size (29.6%, 40/135), unknown time of onset (3%, 4/135), and patient age (< 1%, 1/135).

Case 7 (male; 68 years; NIHSS score, 4; time of onset, 4 h) presented an extracranial occlusion of the ICA with recanalization in the cavernous segment of the ICA. NECT and CTP scans were normal. Imaging was not clear enough to take a decision for six raters: four recommended an MRI to detect small watershed or embolic infarcts and the other two a diagnostic angiography to evaluate collaterals. Of the rest, 75% of raters (33/44) considered that MT was not indicated. There were 55 reasons given for not performing MT: no or minimal perfusion changes (45%, 25/55), low NIHSS score (42%, 23/55), and sufficient collaterals on CTA (13%, 7/55). According to the logistic regression, operators with more MTs/year proposed MT less frequently in this case than in the others (OR, 0.957; 95% CI, 0.921–0.992; p = 0.0254).

Agreement to perform MT

Predominant indications for MT were found in cases 6 (female; 79 years; NIHSS score, 8; proximal M2 occlusion), with 100% of answers reporting indication for MT, and case 9 (male; 50 years; NIHSS score, 13; distal M1 occlusion), with 92% of answers reporting indications for MT. In both, there were no or minimal signs of infarct demarcation on the NECT scan with a significant mismatch between CBV-CBF.

High diversity of opinion

The greatest diversity in the answers was recorded for case 12 (0.503), case 1 (0.517), and case 4 (0.520). Case 12 (female; 84 years; ischemic stroke after mastectomy; NIHSS score, 15; time of onset, 90 min) with occlusion of the right ICA+M1 presented early signs of infarct on NECT and an extended CBV infarct core with a little mismatch. In this case, two raters recommended an additional MRI for decision-making, 54% of raters (26/48) recommended MT, and 46% (21/48) ruled out intervention. A total of 58 reasons for excluding MT were given, including low CBV/CBF mismatch (40%, 23/58), size of the infarct core on the CBV map (38%, 22/58), infarct size on NECT (15%, 9/58), and patient age (7%, 4/58).

Cases 1 and 4 both involved distal arterial occlusion. Case 1 (female; 88 years; NIHSS score, 18; time of onset, 3 h) presented occlusion of the left A2 segment with an aneurysm of the anterior communicating artery. For one rater, the imaging was insufficient to take a decision, while 41% of raters (20 of 49) excluded MT and 59% (29 of 49) recommended MT. In the logistic regression, raters that used SR techniques with a distal access catheter recommended MT more frequently in this case (OR, 4.604; 95% CI, 1.103–19.220; p = 0.0362).

Case 4 (male; 73 years; NIHSS score, 0; time of onset, 2 h) presented occlusion of the right distal M2 segment with a mismatch. Sixty percent of raters excluded MT and a total of 30 reasons for not performing MT were given: low NIHSS score (80%, 24/30), the distal location of the occlusion (10%, 3/30), and sufficient collaterals on the CTA scan (10%, 3/30). In the logistic regression, raters that used NECT+CTA as the imaging protocol more frequently decided in favor of MT in this case (OR, 0.231; 95% CI, 0.56–0.959; p = 0.044). In this regard, some of the raters commented that the perfusion changes were so small that probably, based only on the NECT and CTA scans, without perfusion, they would indicate MT for this patient.

Internal carotid occlusions

Case 7 (NIHSS score, 4; see the “Agreement to exclude MT” section) and case 8 had occlusion of the ICA with permeable MCA and ACA territories but different clinical presentations. Case 8 presented an NIHSS score of 18 and occlusion of the left intracranial ICA. Perfusion changes were subtle and involved the territory of the anterior choroidal artery. When compared, recommendations for MT were greater in case 8 (80%, 40/50) than in case 7 (25%, 11/43), but overall, case 7 had the highest number of raters recommending an MRI for decision-making purposes (12%, 6/50).

Distal vessel occlusions

Cases 1 and 4 (the greatest diversity of answers) and case 6 (upward trend favoring MT) presented distal vessel occlusions and have already been mentioned above. Case 3 (male; 64 years; NIHSS score, 7; time of onset, 5 h) presented proximal M2 occlusion with infarct demarcation on the NECT scan and a match rate of > 50%. Raters did not recommend MT in 21.3% (10/47), and three participants found that the imaging was not sufficient for a decision. The reasons given for excluding MT were infarct size on the NECT scan (32%, 8/25), little CBF/CBV mismatch (32%, 8/25), infarct core size on the CBV map (28%, 7/25), and sufficient collaterals on the CTA scan (8%, 2/25).

Case 5 (female; 72 years; NIHSS score, 13; time of onset, 3 h) presented distal M2 occlusion without demarcation on the NECT scan but a perfusion match of > 50%. MT was recommended by 64% of raters (32/50) and was disadvised by 36% (18/50). The reasons for excluding intervention were the distal location of the occlusion (44.4%, 12/27), low CBV/CBF mismatch (37%, 10/27), and infarct core size on the CBV map (18.5%, 5/27). A few participants recommended iv thrombolysis as the first-choice therapy.

Group analysis

The potential effect of the demographics for raters and their hospitals on the recommendation for MT was analyzed. No single factor was found to have an impact. In contrast, all the clinical and radiological characteristics of the cases were found to exert a significant influence on the decision concerning MT (Table 2). Regarding the group of cases with time window unknown, MT recommendations increased with the number of MTs performed per hospital and year (OR, 1.491 per 100 MTs/year; 95% CI, 1.105–2.009; p = 0.0146). For other subgroups, according to the analysis based on the number of MTs performed per hospital/year, MT recommendations did not vary significantly with either patient age or NIHSS score.

Table 2 Effect of the clinical and radiological characteristics of cases on the decision concerning mechanical thrombectomy
Full size table

Discussion

This is the first study exploring differences in decision-making for MT in cases for difficult clinical and radiological constellations. All included “decision-makers” performed MT and had a median of 7 years of experience as first operator. Diversity in the decisions was high and revealed the current divergence of opinion regarding therapeutic decisions in difficult cases. MT was significantly more likely to be recommended in the clinical and radiological groups with presumably better prognoses.

Overall, none of the demographic characteristics of the raters, e.g., age, number of MTs performed, or technique for MT used, influenced the decision with regard to therapy. In contrast, all clinical and radiological characteristics of the cases markedly influenced the decision regarding MT. The global interpretation was positive, i.e., the raters’ decisions were based on patient characteristics. However, evaluation of the results and their variability per case indicated the difficulties of raters in assessing and prioritizing the different clinical and radiological information by its importance, which could have influenced the outcomes in the different directions [3, 9]. In agreement with Brún et al (decision-making process for performing iv thrombolysis), responders were more likely to propose or exclude MT in cases in which they perceived the evidence base for this decision to be robust (cases 0, 6, and 11) [5]. MT was significantly more likely to be recommended for the clinical groups with presumably better prognoses: age ≤ 80, time window ≤ 3 h, and NIHSS score ≤ 6 [1, 10,11,12,13]. In keeping with this same reasoning, a recommendation to perform MT was more likely in cases without or with only minor demarcation on the NECT scan and also in cases with mismatch > 50% [10, 14, 15].

Contrary to the guidelines, raters dismissed thrombectomy more frequently based on the perfusion imaging characteristics than on the infarct size on NECT scans [1, 2]. We assume that early signs of infarct are not always easy to identify on NECT scans and experience is required, whereas color changes on the volume maps may be more readily detected [16]. We believe that NECT and CTA, as standard stroke imaging, simplify the amount of data to be considered and interpreted. In fact, individuals are limited in their information processing capacity and so will tend to simplify the decision-making requirements of the decision-making process as a form of shortcut for sifting through the relevant items of information for taking a fast decision [3, 9, 17, 18]. In this sense, Shamy et al described almost 87% uncertainty in interpretations of perfusion imaging by neurologists [19]. Thus, adding CTP to our protocols entails, on the one hand, an increase in the difficulty of interpretation but, on the other, increased sensitivity in detecting ischemic infarction, thereby helping with patient selection for invasive treatments [14, 20,21,22,23].

Raters were more likely to offer MT in M1 occlusions than in distal occlusions across the questionnaire. Characteristically, M2 occlusions should exhibit less severe neurological deficits on presentation, smaller extent of ischemic injury, and smaller perfusion deficits than M1 occlusions [24]. Since the natural evolution of distal occlusions seems to be more benign than that of proximal occlusions, decisions may also be influenced by the type of neurological symptoms and their correlation with the imaging results. In this sense, we need to be trained to avoid misunderstandings concerning the anatomical-clinical correlation, because, in the end, the question is not only how much tissue can be saved but also which symptoms could be reversed by recanalization.

Finally, we think that MRI could add valuable information regarding the true extent of irreversible changes compared with multimodal CT and furthermore could aid in detecting watershed or embolic infarcts to help decide whether endovascular treatment is justified [25, 26]. Thus, we agree with the 12% of raters who recommended an MRI in case 7 (embolic ICA occlusion; NIHSS score, 4; and normal perfusion), and in case 12 (short onset time but already extensive changes in the CBV map) to more accurately delimit the real extent of infarct before excluding the patient from endovascular treatment [27, 28].

Limitations

The results of this questionnaire cannot be generalized to a different population. To make the questionnaire feasible, there were obvious limitations on the number of cases and the number of images presented per case. Nevertheless, the 8 representative multimodal CT images are just that, representative: they were carefully selected by one of our senior neuroradiologists, an interventional neuroradiologist, and re-evaluated by an intern for the purpose of readily depicting the major characteristics of each case, since the objective of this study was not to reach a diagnosis but a decision concerning therapy.

Nevertheless, even if a fine measurement of uncertainty in the final decision was not possible, raters always had the option to state that the available imaging was not sufficient to take a decision. A potential influence on the therapeutic decision of full imaging information vs selected slice-imaging or the selection of other imaging modalities (multimodal MRI vs multimodal CT) could not be ruled out.

The order of the cases presented during the interview was the same for all the participants (a random order of cases was not technically feasible). A potential risk of bias from past answers influencing subsequent decisions could be neither ruled out nor proven. This potential bias could have influenced the results of this explorative study, but it is highly unlikely that arranging the cases in random order could have delivered greater agreement. Moreover, past answers/cases may be more likely to exert an influence if raters were aware of the different outcomes of patients, which were indeed blinded.

Time of decision as a measure of the difficulty in reaching a decision for each case was not evaluated. As explained in the discussion of the demographic characteristics of the raters, they were mostly experienced senior interventionalists for mechanical thrombectomy, meaning participants were used to taking such decisions quickly. Moreover, the interviews were conducted reminding all participants to decide just the same as if the patients were already in the CT room at their own hospitals. A major limitation is related to the nature of the study: the relationship between what responders say and they do may not always be strong. In addition, each rater decided how to proceed individually, without feedback from a neurologist. In real-life clinical practice, at any rate, team discussions are not always possible when on call. These interactions could have modified the evaluations and thus the final decision, perhaps changing the results of the study [10].

Summary

Decisions for or against MT were based on the clinical and radiological characteristics of the cases, but even so, a consensus was reached in only a minority of the cases, revealing the current divergence of opinion regarding therapeutic decisions in difficult cases. The aim of the study was to highlight the current divergence of opinions about the management of patients with borderline clinical/radiological characteristics.

We cannot presume to know the best treatment option for each of the 12 cases. Our aim was to reveal the diverse opinions of the raters and highlight the need for new specific research centering on this group of patients to allow more debate, exchange of views, and ultimately more scientifically well-founded decisions.