Introduction

Atherosclerotic steno-occlusive disease of the extracranial internal carotid artery (ICA) accounts for 20 % of acute ischemic strokes [1, 2]. Early carotid endarterectomy has been established as evidence-based practice for secondary, and in selected cases, even primary prevention of ischemic stroke in patients with extracranial ICA stenosis [3–6]. The degree of stenosis remains the critical factor for decision making in revascularization candidates, therefore requiring accurate quantification. Although digital subtraction angiography is considered the reference standard for the assessment of ICA disease, its utilization is limited by invasiveness, costs, and potential complications [7]. In fact, duplex ultrasonography (DUS) has become the first-line screening study for detection and quantification of extracranial ICA steno-occlusive disease [8]. In the decision making upon revascularization therapy, a combined approach of both DUS and computed tomography angiography (CTA) has been widely established, especially when DUS yields equivocal or otherwise non-conclusive results [8, 9]. However, only a few studies investigated the diagnostic agreement between DUS and CTA in ICA steno-occlusive disease, and most of these studies showed heterogeneous results with discordant classification of the degree of stenosis in a relevant number of cases [10–12].

Numerous and often discrepant diagnostic criteria were introduced for non-invasive assessment of steno-occlusive disease of the ICA [13–16]. In an attempt to achieve more uniformity in interpretation of DUS, the German Society of Ultrasound in Medicine (known by its acronym DEGUM) recently proposed a novel multi-parametric ultrasound approach for comprehensive and accurate assessment of extracranial ICA steno-occlusive disease based on extensive experience of experts [17, 18]. These criteria have been adopted by the Neurosonology Research Group of the World Federation of Neurology for ultrasound grading of carotid stenosis [19].

Any diagnostic criteria adopted by an ultrasound laboratory have to undergo local validation as part of accreditation process [20]. The novel DEGUM multi-parametric criteria have not been validated by laboratories adopting them. Therefore, the aim of the present study was to determine the diagnostic agreement between the multi-parametric DEGUM ultrasound criteria and CTA for assessment of extracranial ICA steno-occlusive disease in patients with acute cerebral ischemia.

Methods

Study population

We retrospectively evaluated consecutive patients with acute cerebral ischemia who were admitted to our tertiary stroke center from January 2012 to December 2012. Patients were eligible for our study if their diagnostic workup included DUS and CTA performed within 5 days of each other. Patients who underwent acute revascularization therapy of the extracranial ICA prior to completion of both diagnostic studies were excluded from our analysis. Demographic characteristics and baseline stroke severity with the National Institutes Health Stroke Scale (NIHSS) score were collected during hospitalization. The local Institutional Review Board/Ethics Committee (no. 111032014) approved this study.

Ultrasonography assessment of internal carotid artery disease

Shortly after its publication in 2010, we implemented the multi-parametric DEGUM ultrasound criteria as routine standard for diagnostic assessment of the extracranial ICA at our institution. The DEGUM criteria predict the North American Symptomatic Carotid Endarterectomy Trial (NASCET) degrees of ICA stenoses (i.e., measurements based on the distal ICA diameter narrowing) [5]. By applying a set of local hemodynamic (i.e., the intrastenotic peak systolic and end-diastolic velocities) as well as upstream and downstream criteria (i.e., pre- and post-stenotic spectral Doppler waveforms, presence of intracranial and periorbital collateral channels), the DEGUM criteria grade ICA stenosis in 10 % increments and allow differentiation of severe stenosis from complete occlusion (Fig. 1) [17–19].

Fig. 1
figure 1

The DEGUM ultrasound approach (modified from Arning et al.) [15]. The numbers for the stenosis degree relate to a 10 % range (±5 %). NASCET North American Symptomatic Carotid Endarterectomy Trial, PSV peak systolic velocity, ACA anterior cerebral artery, CCA common carotid artery, EDV end-diastolic velocity, ICA internal carotid artery

Full size image

Duplex ultrasonography (Toshiba Aplio MX SSA-780a System®, Toshiba Medical Systems, Germany) with a 7.5–10-MHz linear array transducer was used for examinations of the extracranial carotid arteries. Transcranial (2 MHz) and continuous-wave (4–8 MHz) Doppler ultrasound (EZ-Dop® or Multi-Dop®, DWL, Germany) were used for assessment of the intracranial and periorbital arteries. All examinations were performed as part of routine workup by a certified vascular technologist or a physician certified in ultrasonography who was unaware of the purposes of this study. All findings were interpreted according to the multi-parametric DEGUM criteria by a vascular neurology specialist with training in cerebrovascular ultrasound who documented the final degree of stenosis in the written reports as standard of care.

CT angiography assessment of internal carotid artery disease

Computed tomography angiography was performed using a Siemens Sensation 64 scanner. The multi-slice CT acquisition was performed with isotropically resolved contrast media-enhanced angiographic imaging of the extracranial vessels. In order to achieve optimal timing of arterial contrast bolus tracking, a region of interest within the aortic arch and a threshold set to 120 HU was used. The procedure utilized 80-cc intravenous contrast of Solutrast® 370 (Bracco Imaging) or Ultravist® 370 (Bayer Schering Pharma) with an injection rate of 3–4 cc/s followed by 50-cc sodium chloride injection. Other parameters were as followed: 100 KV, effective 160 mAs, rotation time 0.5 s, detector collimation 0.6 mm, reconstructed slice thickness 0.75 mm, pitch 1.2, kernel H20, and image acquisition order caudal-cranial.

The raw CTA data was initially inspected for overall quality, carotid artery occlusion, and the presence of bifurcation calcifications. Subsequently, 3D mapping was applied using maximum intensity projection (MIP) to allow the quantitative analysis of vascular data by increasing the artery-to-tissue contrast to define the presence and location of the ICA stenosis. To allow the precise measurement of the extent of the stenosis, the data was reformatted in multiplanar reconstruction (MPR) in order to create consecutive and freely adjusted planes in respect to the ICA orientation. While two MPR planes were set along the principal artery axis, the third was adjusted orthogonally to both planes and adapted in the presence of irregular stenosis. The standard center and window (c/w) Hounsfield scale settings were 250/600. In the presence of calcifications, c/w was adjusted individually to allow optimal differentiation between the calcified plaques and the endoluminal contrast media.

Finally, the carotid stenosis grade was measured using the NASCET approach [5]. This grading provides a ratio of the maximum stenotic narrowing (A) and the diameter of the far distal ICA beyond the stenosis and post-stenotic dilation (C), calculated by [1 − A/C × 100] (Fig. 2). All data were prospectively treated and analyzed by a physician with expertise in cerebrovascular imaging and blinded to the ultrasonography as well as clinical findings.

Fig. 2
figure 2

a CTA-MIP showing severe narrowing of proximal internal carotid artery (arrow). b Sagittal multiplanar reformatted (MPR) images along with the axial mages perpendicular to the longitudinal axis. On axial images, luminal diameter was measured at the narrowest portion of carotid bulb (arrow) and related to the distal internal carotid artery (ICA) where the walls are parallel as per NASCET. c According to the multi-parametric DEGUM approach, duplex ultrasound (DUS) yielded 70–80 % stenosis of the proximal ICA

Full size image

Statistical analysis

Continuous and non-continuous variables are presented as mean ± standard deviation (SD), median (interquartile range, IQR), and percentage as appropriate. Spearman’s correlation coefficient for non-normally distributed data was computed to assess the relationship between DUS and CTA for identification of ICA steno-occlusive disease. Agreement between DUS and CTA was evaluated using Bland-Altman method with calculation of the mean difference (i.e., bias) and the 95 % limits of agreement (i.e. mean ± 1.96 SD) [21]. When DUS stenosis measurements resulted in a range (e.g., 70–80 %), its average value (i.e., 75 %) was used for analysis.

We also tested our interrater and intrarater reliabilities of NASCET-type measurements using Pearson’s product-moment correlation (for normally distributed data) as previously described [22]. For this purpose, 25 randomly selected carotid arteries of varying degrees of the disease were independently reassessed by the same rater (T.F.) after 3 months, and the initial 25 results were compared with the assessments of a blinded expert neuroradiologist (H.K.). Significance level was set at 0.05. Statistical analyses were performed with STATA software (version 12.1, StataCorp., College Station, TX).

Results

During the study period, a total of 346 patients with acute ischemic stroke (n = 284) or transient ischemic attack (n = 62) were admitted to our stroke center and underwent both CTA and DUS. Of these patients, 43 (12 %) were not eligible for the final analysis due to the following reasons: ultrasonographic assessment after acute revascularization therapy, n = 13; elapsed time between DUS and CTA >5 days, n = 23; and only one vascular imaging modality assessable (e.g., streak artifacts from dental implants on CTA), n = 7.

The final study population consisted of 303 patients with 593 DUS and CTA carotid artery pairs available for comparison; mean age was 72 ± 12 years, 58 % were men, and median baseline NIHSS score was 4 (IQR, 7) points. The median elapsed time between DUS and CTA was 1 (IQR, 2) day. Significant stenosis (≥50 %) or occlusion was detected by DUS in 50/593 (8.4 %) and by CTA in 57/593 (9.6 %) internal carotid arteries (Table 1).

Table 1 Steno-occlusive disease of the internal carotid artery as identified by computed tomography angiography (CTA) and duplex ultrasonography (DUS)
Full size table

Diagnostic agreement between DUS and CTA

Spearman’s correlation showed a positive relationship (r s  = 0.783, p < 0.001) between DUS and CTA for identification of ICA disease. The Bland-Altman analysis showed that the mean difference among the two data sets was small (mean difference in degree of stenosis 3.57 %), indicating no clinically relevant bias between DUS and CTA. However, the 95 % limits of agreement for the corresponding pairs of measurements were meaningfully wide (−22.42 to 29.55 %) to over the entire range of steno-occlusive disease. The Bland-Altman plot is shown in Fig. 3.

Fig. 3
figure 3

The Bland-Altman plot shows the difference between duplex carotid ultrasound and CT angiography for the degree of internal carotid artery stenosis plotted against the mean result of both modalities. The mean difference between DUS and CTA for quantifying the degree of ICA stenosis was 3.57 %. A total of 92.1 % (546/593) of differences between the two modalities were within the 95 % (1.96 SD) limits of agreement (−22.42 to 29.55 %). The size of the dots represents the number of measurements. CTA computed tomography angiography, DUS duplex ultrasound, ICA internal carotid artery, SD standard deviation

Full size image

Intrarater and interrater reliabilities for NASCET-type measurements on CTA

There was a strong positive correlation between the repeated assessments of one expert reader (r = 0.997, p < 0.0001) and between the assessments of two expert readers (r = 0.974, p < 0.0001), suggesting excellent intrarater and interrater reliabilities for manual NASCET-type measurements on CTA.

Discussion

Our study showed that although novel multi-parametric criteria may provide overall good agreement between DUS and CTA, significant discrepancies may arise and lead to false DUS screening results in moderate and severe ICA stenosis ranges at our laboratory. Of note, DUS tended to undercall the stenosis in mild ranges and overcall in moderate to severe ranges. Our results underscore the need for local validation of diagnostic criteria adopted for use in clinical routine.

Our results are in agreement with prior studies. A recent systematic review covering the years 2000 to 2009 identified only 4 out of 12 studies (n = 244; 431 arteries) that directly compared DUS with CTA and fulfilled a minimum of methodological standards (e.g., NASCET grading of the carotid arteries) [10]. In these studies, disagreement between DUS and CTA was substantial with one third of all arteries being misclassified by each method as either “medical” when they were actually “surgical” and vice versa. These findings are in line with our results as illustrated by the Bland-Altman plot: (1) the average discrepancy between the two methods tended to become larger as the mean degree of stenosis increased (trend line not shown) and (2) the 95 % limits of agreement were wide enough to be clinically important. Furthermore, considering only clinically relevant categories of the disease in our study, almost two thirds of arteries classified as moderate or severe stenosis by DUS were categorized differently by the comparator modality.

Given the substantial number of arteries with discrepant findings, our observations have potential clinical implications. Previous studies have highlighted the role of a referee imaging modality following discrepant findings to identify those patients that eventually may be suitable candidates for interventional revascularization procedures [23, 24]. Consequently, our findings could be interpreted as reassurance for clinicians that additional confirmatory neuroimaging studies should be considered in case of clinically relevant discrepant findings among DUS and CTA. However, recent meta-analyses pointed out that sensitivities and specificities of non-invasive imaging modalities for diagnosing 70–99 % carotid stenosis are acceptable but substantially less accurate for 50–69 % carotid stenosis [25, 26]. Thus, it remains questionable whether diagnostic discrepancies can eventually be resolved by a third non-invasive test or rather need clarification by digital subtraction angiography, especially when making decisions on revascularization therapies in patients with moderate degrees of the disease. The issue with combining non-invasive tests (e.g., DUS and CTA) as opposed to their single use (e.g., DUS or CTA) for diagnosis of carotid artery disease is that specificity increases at the expense of sensitivity, and vice versa, and therefore does not necessarily improve overall diagnostic accuracy [26]. Hence, accurate criteria for non-invasive tests are needed that allow clinicians to reliably identify patients with carotid artery lesions amenable to revascularization therapy independently of other tests.

There is uncertainty concerning the most reliable and valid ultrasound criteria for quantification of carotid artery disease [27]. While most criteria routinely used by clinicians mainly rely on intrastenotic peak velocity measurements, the multi-parametric DEGUM ultrasound criteria additionally consider post-stenotic flow patterns as well as collateral and transcranial ultrasound findings for stenosis grading [17–19]. Several correlation studies have shown that the exclusive use of intrastenotic hemodynamic parameter does not correlate well with the degree of stenosis as measured by digital subtraction angiography and the consideration of upstream and downstream measurements may improve the diagnostic accuracy [17, 28, 29]. However, even though intrastenotic velocity cutoffs as specified by the DEGUM criteria have been validated against digital subtraction angiography, the approach as a whole has not been validated yet and was rather created by consensus instead of a clinical study [17]. Pinpoint measurements (i.e., steps of 10 %) as provided by the DEGUM approach instead of range estimations (i.e., 50–69 %) are desirable in clinical practice where progression of the disease (e.g., conversion of asymptomatic moderate to severe stenosis) can have significant therapeutic implications [30]. However, the more precise estimate of the degree of stenosis comes at the expense of an increased number of criteria, which in turn may compromise the test performance. Prospective validation studies are needed to further evaluate the DEGUM ultrasound criteria utilizing digital subtraction angiography as the reference standard and to compare these with other widely used ultrasound criteria such as those proposed by the US Society of Radiologists in Ultrasound Consensus Conference [15].

Our study has limitations. First, the single-center design and the relatively low prevalence of clinically relevant carotid artery disease may have affected generalizability of our results. Second, our findings are only conveyable to carotid stenosis grading using the multi-parametric DEGUM approach and preclude any conclusions regarding other ultrasound criteria. Also, our results do not provide insights whether either of both modalities overestimates or underestimates the true degree of stenosis. Third, since ultrasound studies were conducted as part of routine diagnostic workup, we cannot exclude that sonographers were inadvertently influenced by clinical information and imaging findings. Fourth, inherent limitations associated with manual NASCET-type measurements on CTA (e.g., caliper positioning for residual and reference lumen measurements, subjectively optimized c/w settings) may have harbored false positive or negative results. Nonetheless, both intrarater and interrater reliabilities were excellent in our study and other groups have shown superiority of manual stenosis measurements on CTA compared with other techniques [13]. Finally, we allowed a maximum time gap of 5 days between ultrasound and CTA studies bearing the potential that the degree of stenosis may have changed by the time the comparator imaging modality was performed. However, on average, only 1 day elapsed between CTA and DUS that complies with common practice in most stroke centers worldwide.

The strength of our study includes the sample representative of consecutive stroke patient admissions, standardized carotid artery assessments with DUS and CTA, and the first comparison of the novel multi-parametric DEGUM ultrasound criteria with CTA.

Conclusions

Although the novel DEGUM criteria showed overall good agreement between DUS and CTA across all stenosis ranges, potential for wide incongruence with CTA underscores the need for local laboratory validation to avoid false screening results.