Carotid atherosclerosis is a major cause of ischemic stroke accounting for approximately 18–25% of all strokes [1]. Disease severity and risk of stroke are traditionally determined by luminal stenosis and surface irregularities. However, many recent studies have suggested the critical importance of identifying vulnerable plaques by determining their compositions [2, 3], due to their correlation with higher risks of cerebral infarction. Multi-contrast MR imaging (MRI) has been established as an effective method for plaque characterization. Major relevant applications include the diagnosis of intraplaque hemorrhage (IPH) [4] and lipid-rich necrotic core (LRNC), both of which are high-risk features associated with cerebrovascular events. Despite its reliability and noninvasiveness, MR plaque imaging is complex and requires expertise in assessing various plaque components which is a qualitative and subjective task in nature. Radiomics is a computing process of extracting and analyzing large number of quantitative features from medical images. The success of radiomics rests on the assumption that medical images contain additional information which may not be obvious under visual inspection [5]. It has proven to be a valuable tool in oncology, including diagnosis, staging, and grading of cancer; assessment of treatment response; and prediction of clinical outcomes [6,7,8,9]. Recently CT- or ultrasound-based texture analysis has been employed to discriminate symptomatic carotid plaques with promising results [10, 11]. Compared with CT and ultrasound, high-resolution MR plaque imaging provides inherently superior soft tissue contrasts, and a combination of multiple contrast weightings potentially confers more valuable information. Additional advantages of MRI include its lack of radiation and operator-independence. However, up until today, there is no data available regarding MR radiomics on carotid plaque, except for one initial report on MR texture analysis of basilar artery plaque [12].

The purpose of this study was to build a high-risk plaque MRI-based model (HRPMM) using a radiomics approach combined with machine learning and to evaluate its performance in differentiating symptomatic from asymptomatic carotid plaques by comparing it with traditional MR features.

Materials and methods

Patients

We retrospectively collected 217 patients from January 2015 to June 2019, who had carotid stenosis of > 30% diagnosed by ultrasound or angiography (NASCET criteria [13]). These patients underwent both carotid plaque MRI and head MRI within the same day. The study was approved by the ethics committee of our institution and informed consent was obtained from all patients.

All patients were divided into symptomatic and asymptomatic groups according to whether they had clinical symptoms within 2 weeks before the MRI examination and/or whether their head MRI showed acute/subacute stroke [14, 15]. The clinical symptoms included classic TIA (transient ischemic attack) and ischemic stroke in the anterior circulation (carotid territory) [16, 17], as well as monocular symptoms (amaurosis fugax or retinal artery occlusion) ipsilateral to the carotid plaque [18]. Classic TIA was defined as an abnormal focal neurologic deficit lasting less than 24 h. A complete ischemic stroke presented with an abrupt onset of a focal neurologic deficit lasting > 24 h [17]. Clinical characteristics were recorded including gender, age, and atherosclerotic risk factors (hypertension, diabetes, hyperlipidemia, and cigarette smoking).

The exclusion criteria were as follows: (a) evidence of cardiogenic stroke; (b) bilateral TIA/stroke, brainstem involvement only or undetermined hemispheric involvement; (c) primary intracranial diseases; (d) radiotherapy-induced carotid stenosis; (e) contraindications to MRI (such as pacemaker or severe claustrophobia) or to gadolinium. In total, 55 patients were excluded and 162 patients were finally included in this study. Eligible patients were randomly divided into a training cohort (n = 121) and an independent test cohort (n = 41) in a ratio of 3:1. From all these enrolled patients with carotid stenosis, if a carotid plaque was found in one with clinical symptoms or positive head MRI findings, it was classified as a symptomatic plaque; otherwise, it was asymptomatic.

Magnetic resonance imaging

All patients were scanned on a 3-T MR scanner (MAGNETOM Verio, Siemens Healthineers) with a 16-channel head coil and an 8-channel carotid coil. The following carotid MR pulse sequences were applied to obtain the cross-sectional images centered at the carotid bifurcation: (a) 3D time-of-flight (TOF), (b) T1W turbo spin-echo (TSE), (c) T2W TSE, (d) dynamic contrast-enhanced 3D volume-interpolated breath-hold examination (DCE 3D VIBE), (e) post-contrast delayed T1W TSE.

Gadopentetate dimeglumine (Magnevist, Bayer) was used for DCE plaque imaging with a dosage of 0.1 mmol/kg body weight followed by a 20-ml saline flush through an 18-G venous catheter positioned in an antecubital vein. The contrast material was injected by a power injector at an injection rate of 2 ml/s. The DCE acquisition was repeated 16 times continuously with each acquisition of 18 s. The 2nd acquisition was initiated synchronously with the start of the contrast administration. Post-contrast delayed imaging was done immediately after DCE acquisition. Detailed imaging parameters are in Table 1. The total scan time was approximately 22 min.

Table 1 Summery of imaging parameters for the multi-contrast MRI protocol
Full size table

Image analysis and segmentation

Traditional measurements and analysis were performed on a workstation (Syngo; Siemens Healthineers) by a radiologist with 4 years of experience in plaque imaging. The slice with the largest plaque area was chosen to be measured. The inner lumen and outer wall boundaries were manually outlined on T1WI to measure the plaque burden (PB), minimal luminal area (MLA), remodeling index (RI), and contrast enhancement ratio (CER). The related calculation formulas were as follows:

$$ {\displaystyle \begin{array}{c} PB=\left(1\hbox{--} \frac{\ lumen\ area}{vessel\ area}\right)\times 100\%;\\ {} RI=\left(\frac{vessel\ area\ at\ the\ point\ of\ maximum\ stenosis}{reference\ vessel\ area\ at\ the\ distal\ portion}\right);\\ {} CER=\left(\frac{\left( signal\ of\ plaque\ \left[ post- contrast\ delayed\right]\right)}{\left( signal\ of\ plaque\ \left[ pre- contrast\ T1 WI\right]\right)}\hbox{--} 1\right)\times 100\%.\end{array}} $$

The degree of stenosis was measured on T1WI according to the NASCET criteria [13]. Plaque major components including IPH and LRNC were identified and recorded too [19, 20]. Disrupted luminal surface was identified as a deficit in the fibrous cap or discontinuous surface of the plaque [17].

Plaque segmentation for radiomics analysis was performed with an open-source software ITK-SNAP (version 3.8.0, www.itk-snap.org). After choosing the slice showing the largest plaque on different sequences including T1W, T2W, DCE, and post-contrast delayed T1W, regions of interest (ROIs) were manually drawn by the same radiologist along the margin of the plaques. For the DCE sequence, we drew ROI on the 5th phase image due to its highest signal intensity in the carotid lumen at that phase. Sample patient images were shown in Fig. 1.

Fig. 1
figure 1

The procedure of radiomics segmentation. a In one patient with the regions of interest (ROIs) outlining the carotid plaque on T1W, T2W, DCE (dynamic contrast-enhanced), and POST-contrast delayed images. b The segmentations extracted from the ROIs corresponding to the images on a

Full size image

Feature extraction, selection, and model development

Of all the radiological variables from 162 patients, we chose those with p < 0.05 in univariate tests to perform multivariable logistic regression analysis and calculated the odds ratios (ORs) with 95% confidence intervals (CIs). The variables with p < 0.05 in multivariate analysis were finally used to build a linear traditional model.

Before radiomics feature extraction, all MRI images were subjected to imaging normalization (the intensity of the image was scaled to 0–100) and resampled to the same resolution (2 mm × 2 mm × 2 mm) to avoid data heterogeneity bias. Radiomics features were then extracted on PyRadiomics platform [21, 22]. For each ROI on each sequence, a wavelet filter was applied.

To build a predictive radiomics model for identifying high-risk plaques, the following steps were done. Firstly, the method of normalization to z distribution ([value – mean value]/standard deviation) was applied for each extracted feature. Secondly, t tests were used to select the features with p < 0.05 in each sequence, and then the features with significant differences and an area under the curve (AUC) > 0.65 by receiver operating characteristic (ROC) were selected. Thirdly, the least absolute shrinkage and selection operator (LASSO) algorithm was applied for further feature reduction. The most significant features with the smallest deviance were then selected using the LASSO algorithm for the final features. The LASSO algorithm is a penalized regression method which has been successfully applied to some oncologic researches [8, 9, 23, 24]. The LASSO algorithm can estimate the regression coefficients by maximizing the log-likelihood function (or the sum of squared residuals) with the constraint, reduce the coefficients of indistinctive covariates to zero, and enable the non-zero features to be combined into a radiomics model [23]. Using these final features after the LASSO algorithm, we constructed a linear radiomics model called high-risk plaque MRI-based model (HRPMM) in the training cohort and then evaluated its performance in the test cohort. With this model, the risk score for each patient was calculated by a formula weighted by regression coefficients: Risk score = constant + coefficients × features. To determine the optimal cutoff value to classify the plaques into low- and high-risk, the R software was used to calculate the Youden index for all possible cutoff values (c) (Youden index = maxc {Sensitivity + Specificity − 1 }), and the value of c that achieves the maximized index will be considered the optimal one [25].

Finally, a combined model was built based on the multivariate logistic regression analysis in the R software combining both the traditional and radiomics features (flow chart in Fig. 2).

Fig. 2
figure 2

Flow chart for building the models in the identification of high-risk plaques

Full size image

The inter-observer agreement

To evaluate the reproducibility of the radiological and radiomics analysis, another radiologist with 7 years’ experience in plaque imaging independently completed the plaque analysis in 40 patients who were randomly selected from the study population.

Statistical analysis

All statistical analyses were performed using the R software (version 3.5.0) and IBM SPSS Statistics 23.0. Univariate analysis was performed to assess the relationship between each variable and symptomatic status, t test was used for continuous variables, and the Chi-squared test or Fisher’s exact test was for the categorical variables. Variables with statistical significance in the univariate analysis were used in the multivariate logistic analysis. The clinical characteristics of the training and test cohorts were compared appropriately using t test, Fisher’s exact test, or Chi-squared test. A two-tailed p value of < 0.05 was considered statistically significant.

A nomogram of the combined model was established by the R software. For each model, AUC values were calculated using ROC analysis to assess the predictive ability. Accuracy, sensitivity, specificity, negative predictive value (NPV), positive predictive value (PPV), positive diagnostic likelihood ratio (PDLR), and negative diagnostic likelihood ratio (NDLR) were also measured using the defined optimal cutoff values. The Delong method was used for comparing the AUCs of all these models [9].

To correct overfitting bias, corrected AUCs were calculated using bootstrapping validation (1000 bootstrap resamples) in the training cohort. Intraclass coefficients (ICCs) were calculated for the inter-observer agreement [9]: ICC < 0.40, poor agreement; 0.41–0.60, moderate; 0.61–0.80, substantial; and > 0.80, excellent.

Results

Patient characteristics

Of all the 162 enrolled patients, there were 108 symptomatic and 54 asymptomatic patients. The clinical and radiological characteristics are listed in Table 2. These characteristics in the training and test cohorts show no significant difference in Table 3.

Table 2 Clinical and radiological characteristics of the carotid plaques
Full size table
Table 3 Clinical and radiological characteristics in the training and test cohort
Full size table

Traditional assessment of the carotid plaques

Univariate analysis showed that hyperlipidemia, IPH, LRNC, disrupted surface, MLA, degree of stenosis, PB, CER, and RI were associated with symptomatic plaques (p < 0.05, Table 2). Multivariate logistic regression analysis showed that IPH (OR = 7.568; 95% CI, 1.933–26.627) and LRNC (OR = 3.835; 95% CI, 1.409–10.441) were independent predictors of clinical symptoms. In the training cohort, the AUC values of IPH and LRNC were 0.697 and 0.715 respectively; the AUC value was improved to 0.825 when combining IPH and LRNC. In the test cohort, the above values were 0.668, 0.690, and 0.804 (Table 4).

Table 4 Predictive ability of all models
Full size table

Radiomics analysis of the carotid plaques

From each ROI on each sequence, we extracted 788 radiomics features on both the original and filtered images, including 18 features of first-order statistics, 14 features of shape, 68 features of texture (22 gray-level co-occurrence matrix (GLCM) features, 16 gray-level run-length matrix (GLRLM) features, 16 gray-level size-zone matrix (GLSZM) features, 14 gray-level dependence matrix (GLDM) features), and 688 features of wavelet. In total, 510,624 radiomics features were extracted from all ROIs, and we selected 387 features with significant association with clinical symptoms, including 85, 68, 72, and 162 radiomics features from T1W, T2W, DCE, and post-contrast delayed images respectively. After the LASSO algorithm was applied, 33 features were finally retained and used to build HRPMM (details are shown in the Appendix).

In the training cohort of HRPMM, radiomics features on T1W, T2W, DCE, and post-contrast delayed images had AUC values of 0.861, 0.869, 0.907, and 0.890, while AUC values of the test cohort were 0.894, 0.897, 0.965, and 0.978. When combining four sequences, in the training cohort, the AUC value reached 0.988 with a sensitivity of 92.54%, specificity of 96.77%, and accuracy of 93.88%; in the test cohort, the AUC was 0.984 with a sensitivity of 81.48%, specificity of 100%, and accuracy of 90.49% (Table 4).

A combined model was finally constructed and shown as a nomogram (Fig. 3). This combined model exhibited an AUC value of 0.989 in the training cohort and 0.986 in the test cohort (Table 4). The ROC curves of the HRPMM and the combined model outperformed those of the traditional model in the training cohort (p = 0.000, p = 0.000) and test cohort (p = 0.021, p = 0.020). However, the combined model showed no significant difference with the HRPMM in the two cohorts (p = 0.798, p = 1.000) (Fig. 4).

Fig. 3
figure 3

A nomogram integrates the radiomics scores and traditional features of the training sets. The probability scoring of the carotid plaque being symptomatic is marked on each axis and increases from the left to right. The following is an example of how to calculate the total points of a plaque on T1W and predict its risk: a carotid plaque with LRNC but no IPH receives 7.5 + 0 = 7.5 points from traditional features. A radiomics score of 2 corresponds to 90. Therefore, this plaque is scored 97.5 on the nomogram, which indicates a risk of over 95% of exhibiting symptoms. IPH, intraplaque hemorrhage; LRNC, lipid-rich necrotic core; DCE, dynamic contrast-enhanced sequence; POST, post-contrast delayed sequence; All, all features including T1WI, T2WI, DCE, and POST

Full size image
Fig. 4
figure 4

Receiver operating characteristic (ROC) curves of all the models (traditional model, radiomics model—high-risk plaque MRI-based model (HRPMM) and combined model) in the training and test cohorts respectively. The ROC curves of the radiomics model and combined model outperform those of the traditional model in both the training (AUC, 0.988/0.989 vs. 0.825) and test (AUC, 0.984/0.986 vs. 0.804) cohorts. All, all features of each model; DCE, dynamic contrast-enhanced sequence; POST, post-contrast delayed sequence; IPH, intraplaque hemorrhage; LRNC, lipid-rich necrotic core

Full size image

The corrected AUCs from resamples in the training cohort were as follows: T1WI (0.860), T2WI (0.863), DCE (0.906), post-contrast delayed (0.890), and all sequences (0.988), which were close to those of the radiomics model in Table 4.

The inter-observer agreement

The ICCs for measuring the radiological and radiomics features ranged from 0.705 to 0.951 and were summarized in the Appendix.

Discussion

With the risk of ipsilateral cerebrovascular events caused by carotid atherosclerosis, a considerable number of MRI studies on carotid plaque have been performed to search for relevant plaque components or characteristics. While previous studies were mostly focused on the traditional evaluation of the plaques, in this study, we used a radiomics approach to extract radiomics features from conventional plaque MRI and used the HRPMM to differentiate symptomatic carotid plaques from asymptomatic ones. The model consists of 33 radiomics features after optimization with LASSO and exhibits excellent diagnostic performance. With HRPMM, the risk score of each plaque could be computed, and the high-risk plaques could be identified. Additionally, the radiomics model shows an improved diagnostic performance compared with the traditional model, especially in specificity.

We found that independent radiomics features from 4 sequences were different, because different signal characteristics in various sequences were determined by different pathophysiological features. For example, the high signal on pre-contrast T1W possibly represents IPH, while the high signal on post-contrast T1W may be attributed to the contrast uptake by neovascularization and inflammation. In the 33 final features, only sphericity, i.e., a shape feature, appeared in the four sequences’ final screening results (see Appendix). It implies that sphericity may be the most important feature to delineate the morphology of the plaques. In both the training and test cohorts, the contrast-enhanced images from either the DCE or post-contrast delayed sequence demonstrated the highest AUC values. After combining all the radiomics features from four sequences, the AUC values were further improved in both cohorts. However, the diagnostic performance of the combined model was not significantly improved compared to the radiomics model alone. This could be explained by the relative weights of the radiomics versus traditional model, and the combined model was weighted heavily towards the radiomics features which produced better performances (Fig. 3). This comparison suggests that the traditional qualitative indicators might be covered, if not replaced, by quantitative descriptions by radiomics features. It further points to the potential of a more standardized diagnostic process based on objective quantitation. The reliability of our results was validated by the corrected AUCs after randomly sampling the training and test cohorts.

A number of studies have found certain carotid plaque features, such as IPH and LRNC, that were associated with an increased risk of plaque rupture and distal brain embolization. Measurements of stenosis, plaque burden, and enhancement ratio may also be predictive of future ipsilateral carotid symptoms. However, radiomics can provide reader-independent quantitative data [5] that are not readily apparent or are too numerous for an individual radiologist to assess visually. Radiomics contains richer information than traditional features; for example, IPH is a marker of high-risk carotid plaques, but most previous studies were qualitative identification in which the quantitative information about signal intensity, volume/shape, and the complicated distribution of IPH was lacking. That is why our HRPMM model was superior to traditional features in the classification of the high-risk carotid plaques.

To our knowledge, this study is the first investigation of high-risk carotid plaques with multi-contrast MRI using the radiomics approach and machine learning. A previous CT study [11] used discrete wavelet transform and texture features to classify carotid plaques with an accuracy of 88%, sensitivity of 90.2%, and specificity of 86.5%. Texture analysis was also successfully applied in ultrasound studies to improve the identification of symptomatic carotid plaques [10, 26]. Recently, a study [12] using radiomics texture analysis on MRI accurately distinguished symptomatic basilar artery plaques with an excellent AUC of 0.936, while clinical imaging features alone had an AUC of 0.833. Based on these prior studies supporting the use of radiomics analysis to improve risk estimates, we further developed a MRI-based radiomics model to classify carotid plaques. Compared with previous reports, our radiomics model contained not only texture features but also other first-, second-, and higher-order statistics, all together contributing to the higher diagnostic accuracy of this model.

This study has the following limitations. The first was its relatively small sample size; especially in the test cohort, a larger data set is needed to evaluate this prediction model in the future. Secondly, ROIs were manually segmented, not yet fully automated, despite the excellent reproducibility. Thirdly, our traditional model only included two final radiological characteristics (IPH and LRNC), because all the relevant imaging features in relation to stroke risk were too mutually correlated so that they might be eliminated by multivariate analysis [18]. Fourthly, because of the indistinct boundary of the plaque on TOF, we did not perform radiomics analysis on it. Fifthly, the radiomics analysis was done on the slice with the largest plaque area. 3D analysis from whole image slices was not used because of the time restriction.

Conclusions

Radiomics analysis and machine learning of carotid plaques on MRI can accurately distinguish symptomatic from asymptomatic plaques. The radiomics model is better in the detection of high-risk plaques than the traditional model. Radiomics analysis has the greatest benefit of providing quantitative variables from the already-acquired imaging data to improve diagnostic performance beyond conventional evaluations. Prospective studies are needed to further examine the ability of HRPMM for predicting stroke risks.