Introduction

Atherosclerosis is a chronic disease that affects medium and large arteries. The resultant plaque can obstruct lumen or disseminate material into blood stream and could be a cause of myocardial infarction, stroke and peripheral vascular disease [1].

In particular, carotid atherosclerosis is very important in the pathogenesis of cerebral ischemia [2]. Although advances in the understanding and treatment of these lesions have been performed, thrombotic complications of atherosclerosis remain one of the most important causes of morbidity and mortality in Western society [3].

It should be important to have a noninvasive imaging technique capable to identify, not only the presence, but also the stage of intra-plaque hemorrhage. Currently, the predominant noninvasive imaging modalities investigated for this purpose are (a) ultrasonography (US) and (b) magnetic resonance imaging (MRI) techniques [4].

In particular, US and MRI are highly sophisticated imaging modalities for an accurate and timely diagnosis [2, 5, 6]. MRI utilizes powerful static magnetic fields to align the magnetic spins of the protons in water molecules and to drive the Larmor precession which provides the signal used to construct the image [7], whereas the principle of ecography is similar to that of sonar or radar; in essence, following an US pulse transmission, echoes from the medium being interrogated are detected and used to form an image [8].

These US waves are transmitted from probe to body, and the signal is reflected wherever there is an interface between two tissues with different acoustic impedances. In this context, color or power Doppler US is a simple and noninvasive method used to characterize carotid atherosclerotic lesions, using echogenicity, even if it depends on operator and patient.

For this reason, there are circumstances where an alternative carotid arteries MRI imaging modality has been reported to characterize plaque components of lipid cores, intra-plaque hemorrhage, fibrosis, and calcification [9,10,11,12]. It should be possible to differentiate soft and unstable plaques with liquid constituents from solid, hard, and stable plaques.

These diagnostic modalities are based on acquisition and correct evaluation of images. Underhill [13], Crowe [14] and Mani [15] were the first who investigated the relation between carotid MRI and US.

In any case, it is important to guarantee the maintenance of the consistent image quality of the radiological equipments. For this reason, in the clinical imaging study, it is important to have accurate confirmation of several physical characteristics of the medical imaging device. In this context, quality controls (QC) have an important role because QC enables a complete evaluation of system status and image quality [16,17,18] and permit the identification of image quality degradation and source of possible equipment malfunction [5, 19]. In fact, the purpose of QC testing is to detect changes that may result in a clinically significant degradation in image quality [20] which reduces the ability to detect and correctly interpret abnormal findings.

For this reason, the optimization of image reproducibility motivates the calibration of acquisition protocols that use phantoms with defined dimensions, inserts etc.

Phantoms are usually used in order to guarantee the maintenance of consistent image quality over lifetime of the diagnostic equipment.

In this context, to geometrically characterize carotid atherosclerotic lesions, diameter stenosis percentage and area stenosis percentage have to be determined. Therefore, accuracy of stenosis size represents a significant parameter that should be estimated during QC, to correctly evaluate area stenosis percentage.

However, most US and MRI scanners adopt specific automated procedures that require the use of dedicated phantoms. Usually, these phantoms are different for the two modalities and require specific protocols depending also on specific diagnostic device [21,22,23].

Recently, we realized a single polymethyl methacrylate (PMMA) phantom that was suitable to perform quality inspection both on CT and on MRI that may be used to provide a complete quality inspection of a CT and MRI equipment [24]. On the basis of this experience, we propose a novel dedicated phantom easy to implement which can be used on both US and MRI scanners. In this work, we performed a US and MRI examination, by using the novel phantom, for a qualitative and quantitative stenosis evaluation.

The reliability of this study was evaluated by a detailed statistical analysis. In particular, Fisher exact test (F test) and concordance correlation coefficient (CCC) were used: the F test was useful to assess whether a significant difference existed between data sets that were obtained following different procedures, while the CCC test was important in order to assess agreement and concordance between the MRI and US methods.

Materials and methods

Phantom

This novel dedicated phantom consists of a PMMA empty box (18.0 cm × 28.5 cm × 18.0 cm) diagonally crossed by a PMMA hollow cylinder, oriented at 26.84° across the beam width, simulating a blood vessel. The parameters that characterize the blood vessel are: diameter 5 mm and thickness 1 mm. In order to mimic the US and MRI characteristics of a stenosis, we inserted a thin semirigid plastic wall. This wall is a hollow tube (3.0 mm × 0.8 mm), inside the hollow cylinder (Fig. 1), which mimics an area stenosis percentage of 36%. The choice of PMMA and semirigid plastic tubing was related to the necessity of using no ferromagnetic elements in order to perform geometric stenosis measurements both on an MRI scanner and on an US device. In previous works, stenosis models with different diameter and/or area reductions have been manufactured in order to study the effect of the degree of stenosis and flow rate in large vessels in CT, MRI and US [21,22,23], but no comparison among different diagnostic devices was performed. In particular, models which mimicked area stenosis in large vessels ranging from 18% up to 90% were made.

Fig. 1
figure 1

Upper and lateral view of the phantom

Full size image

The PMMA box was filled with distillate water, while the hollow cylinder was filled with a liquid solution, mimicking human blood, and containing: water, glycerol, orgasol, detergens, citric acid and Acnibio OCS (Dansk Fantom Service). Orgasol, characterized by nylon particles 5 µm in diameter, has been used in blood mimicking fluids when natural buoyancy of the particles has been achieved, whereas glycerol avoids refraction [25,26,27]. The presence of other elements guarantees long-time preserving of the fluid. The solution was diluted with distillate water (1:9 in volume), following the manufacturer’s indication, in order to guarantee the same characteristics (e.g., viscosity, acoustic properties) of human blood, as indicated in Table 1. All the geometric parameters that characterize the novel developed phantom are reported in Table 2.

Table 1 Physical and acoustic properties of blood mimicking fluid compared with the human blood
Full size table
Table 2 Parameters that characterize the blood vessel and simulate stenosis
Full size table

US and MRI data acquisition

The common carotid artery (CCA) US data were obtained as longitudinal cross sections using two different US devices: a Philips iU22 US and a Philips iE33 system. In both US systems, a broadband linear L9-3 probe, operating in 3–9 MHz frequency range, was used. The L9-3 probe can be utilized for vascular (carotid, arterial and venous) and superficial imaging applications.

In this case, vascular carotid preset on the machine was used (Vasc Car preset, persistence low, XRES and SONOCT on) and the gain was optimized by the operator who is an experienced vascular sonographer.

The best images are still captured and stored in the Digital Imaging and Communications in Medicine (DICOM) file format for further evaluation, and the selected ones were qualitatively and quantitatively analyzed using the owner software of the US devices (Fig. 2).

Fig. 2
figure 2

US images of the novel dedicated phantom. a Longitudinal cross section; b longitudinal cross section and measurements performed

Full size image

The MRI data were obtained using two different scanners that are characterized by a different static magnetic field values, 1.5 T and 3.0 T, respectively (Fig. 3). In both cases, the sequences we considered were a T1-weighted turbo spin echo (T1w TSE) sequence and a T2-weighted turbo spin echo (T2w TSE) one. In fact, the T2w TSE and T1w TSE weighting was essential for plaque assessment and identification of calcification. In addition, a T2 spectral presaturation with inversion recovery (T2 SPIR) sequence was also considered in order to suppress fat and reconstruct carotid artery wall [28,29,30]. All the scan parameters are indicated in Table 3.

Fig. 3
figure 3

MRI images of novel dedicated phantom. a sagittal; b coronal; c axial

Full size image
Table 3 MRI (1.5 T and 3.0 T) scan parameters
Full size table

Also in this case, the MRI DICOM images were acquired and directly analyzed using the owner software of the MRI devices.

All the US and MRI images were acquired in static conditions, i.e., the effect of the blood flow was negligible.

In both cases, the parameters that we consider for measurements are: filter vessel, diameter vessel, filter stenosis, diameter and angle stenosis.

Image measurements and statistical analysis

Three raters, neurovascular expert with more than 3 years of clinical experience analyzed US and MRI images using the owner software of the devices. The raters were blinded from information of phantom. Blood vessel diameter, blood vessel thickness, simulate stenosis thickness, simulate stenosis diameter and angle were manually segmented.

Accuracy measurements, using Eq. (1), were performed to assess whether the developed phantom could be used on both MRI and US devices.

$$ {\text{Accuracy (\% )}} = \left| {\frac{{x_{\text{meas}} - x_{\text{ref}} }}{{x_{\text{ref}} }}} \right| \times 100 $$
(1)

where xmeas represents the measured values and xref indicates the reference ones.

Later, a parametric analysis was carried out because the results of the Shapiro–Wilk normality test indicated that most of the target variables were normally distributed. Analysis of variance, using F test was performed, to assess whether a significant difference existed among the different data sets. The F test is highly used when the aim of the study is to evaluate a precision of a measurement technique. In fact, analysis of variance consists of factorization of the total variance into a set of partial variances, which correspond to different and estimated variations. For continuous data, the CCC was used, to assess the agreement between two methods (MRI 1.5 T and 3.0 T; US iU22 and iE33) and to assess the concordance between MRI and US methods. The concordance correlation coefficient combines measures of precision and accuracy, to determine how the observed data deviate from the line of perfect concordance (line at 45° on a square scatter plot). Intra-rater analysis was used to determine test rest-reliability, comparing the different scores marked by the same rater on the same phantom with different methods.

Finally, inter-rater agreement was assessed between the various measurements obtained with the different techniques in order to test the performance stability with different raters. Inter-rater agreement was estimated through intra-class correlation coefficients (ICC), one-way analysis of variance with random effects. The maximum ICC value is 1.00, and it represents the stronger reliability. In general, ICC values above 0.75 indicate a good reliability. Analyses were performed using an open-source R3.0 software package (http://www.r-project.org). A 95% of confidence level was set with a 5% alpha error. Statistical significance was set at p < 0.05.

Results

Accuracy measurements

In Tables 4 and 5, the results of the measurements, respectively, conducted on US and MRI devices are reported. All the parameters that characterized the novel phantom have been evaluated.

Table 4 Accuracy measurements conducted on US devices
Full size table
Table 5 Accuracy measurements conducted on MRI devices
Full size table

Fisher’s exact test

In Tables 6 and 7, continuous variables were expressed as mean ± standard deviation. The F test was used to verify whether data sets were comparable. For iU22 and iE33 US data, we did not obtain statistically significant difference between the two methodologies (Table 6). For 1.5 T and 3.0 T MRI, we did not obtain statistically significant difference between the two methodologies (Table 7).

Table 6 US images results
Full size table
Table 7 MRI images results
Full size table

Concordance correlation coefficient

The CCC showed a moderate agreement between some methods (Table 8). In US (iU22 and iE33), we obtained a moderate/high agreement for blood vessel CCC = 0.84 (− 0.42 to 0.99) and for simulate stenosis CCC = 0.94 (0.40; 0.99) diameter; in 1.5 T and iU22 for blood vessel diameter CCC = 0.97 (0.35; 0.99); in 1.5 T and iE33 for blood vessel diameter CCC = 0.86 (0.006–0.99) and for simulate stenosis diameter CCC = 0.94 (0.40; 0.99). Agreement between 3.0 T and other methods results poor, and this could be due to the fact that the 3.0 T has a better resolution compared to US and MR 1.5 T.

Table 8 Concordance correlation coefficient values (95% confidence intervals)
Full size table

Inter-rater agreement

For inter/RR, ICC was calculated to evaluate the consistency of parameters considering different raters. In Table 9, we reported the resulting ICCs for the three comparison conditions for the different techniques. The results obtained by intra-rater agreement to compare two techniques are significant. We obtained a moderate/high agreement in 1.5T, iU22 and iE33. For the 3.0T MRI measurements, we observed that high agreement in all ICC values is > 0.90.

Table 9 Inter-class correlation coefficient values (95% confidence intervals)
Full size table

Discussion

Qualitative classification of vessel stenosis, which highly depends on medical staff capability, and its quantitative characterization are crucial in the diagnosis and treatment planning. In this study, we assess the diagnostic ability of US and MRI techniques to identify and measure carotid stenosis.

In order to choose a method for investigation, different parameters should be considered: availability of modality, level of optimization and the capability of patient and operator. For neurological research, MRI is usually preferred. US is the cheapest and safest modality, but it is operator-dependent. MRI is an expensive technique and it is not present in most of the hospitals.

It is important to consider that US is more available than MRI. Thus, it is used more frequently than MRI as a screening modality. On the other hand, by using MRI it is possible to acquire tridimensional images with the high contrast resolution.

Other authors have conducted studies about the use of dedicated phantoms to evaluate stenosis parameter [21,22,23], but no comparison between diagnostic devices was performed. This is a first study that reports a comparison among two different US modalities and two different MRI strength fields (1.5 and 3 T) with a novel phantom approach.

In this context, we developed a novel phantom, simulating a vessel stenosis that can be used both on MRI and on US devices. The phantom mimics the anatomical characteristics of the obstructed vessel, as closely as possible, but remains simply to use and provides reproducible data. Its preparation requires only the filling of the hollow cylinder and the PMMA box, using, respectively, a solution, simulating human blood and distillate water.

The accuracy obtained using the 3 T device is significantly higher than that obtained using the other diagnostic scanners.

However, statistical analysis showed that no significant difference exist between US and MRI techniques. In fact, US and MRI have shown similar diagnostic efficacies for the evaluation of vessel stenosis, using the novel phantom. The obtained results showed that there was concordance between the two different diagnostic techniques; however, we found significant results in inter-rater agreement. In particular, high diagnostic agreement and good triage of rater agreement were found for MRI 3.0 T. This could be due to the fact that the 3.0 T magnet has the capability to provide a better image quality as the base for improved diagnostic performance, because doubling the field strength (almost) doubles signal-to-noise ratio, that is, the quantity of signal made available from the patient in order to build MRI images.

The proposed method employs a new universal phantom, which can be used on any US and MRI device, in a quick manner, to evaluate stenosis vessel. In addition, this developed methodology could represent a simple, versatile and reproducible instrument to evaluate new physical parameters for implementing routinely QC programs.