1 Introduction

Carotid artery disease is one of the foremost dangerous diseases that target blood vessels. It occurs due to the accumulation of deposits or fatty substances inside the artery, causing the blockage of these blood vessels that transport blood to the brain and head (Sousa et al. 2014). CCA supplies blood to the front part of the brain that performs functions of thinking, speech, character formation, and sensory ability. Therefore, the CCA narrowing and stricture exposes the patient to the risk of stroke and cardiovascular disease (Members et al. 2014). The number of deaths from these cardiovascular diseases exceeds the number of deaths from any other cause of death. The World Health Organization estimated 17.7 million people died from cardiovascular diseases every year (Mozaffarian et al. 2015).

Automatic diagnosis (localization, segmentation, classification, and IMT measurement) of the CCA is very important to determine the degree of blockage of the artery and its classification if it is healthy or stiff to avoid cardiovascular disease and stroke that may cause permanent disability (Liu et al. 2020).

The types of medical imaging of the CCA differ, but ultrasound (US) imaging of the CCA is a sustained and painless way in which sound waves are used for checking the blood flow through the carotid arteries. Carotid US images test the narrowing or stricture of the carotid arteries. Doctors recommend using the carotid US imaging if you have chronic diseases or certain types of stroke. The ultrasound imaging is performed by recording the echo while the waves bound off the artery tissues. The computer transposes the these waves into an image on the screen (Latha et al. 2020).

The novelty and main contributions of this article are represented in several dimensions. First, a complete CAD diagnosis system for carotid artery disease that combines a four-phase is implemented and evaluated. The state-of-the-art in this field does not include similar research to our developed research, i.e., with these specific phases. Second, improving each phase in our integrated diagnosis system to have its main contributions. Third, integrating CCA segmentation, localization, IMT measurement, and classification is a great step towards real-time deployment. Fourth, the used dataset from the signal processing (SP) lab dataset, 100 longitudinal images dataset, and Cyprus dataset is extracted and manually annotated, and this dataset will be available as a standard benchmark for other researchers who conduct research on CCA. Fifth, the conducted experimental works have proven the superiority of the developed CAD system in terms of CCA segmentation, localization, IMT measurement, and classification. We have been proposed an automatic diagnostic system for CCA disease that includes four phases as segmentation, localization, IMT measurement, and classification of the CCA longitudinal sections as normal and abnormal depending on the thickness of the IMT. We have been applied an automated technique for CCA localization in the transverse section which is based on using the Faster R-CNN to localize the CCA bounding box and then localize the artery. Also, an automated technique has been introduced to segment the CCA in longitudinal mode using active contour. We classify the CCA longitudinal images for different datasets as normal and abnormal images. We have been measured the IMT of the CCA in a longitudinal section for healthy and unhealthy people and compare our measurements with the clinicians’ values. The main contributions of the four phases can be summarized as follows: (1) A robust and fully automated localization system and it's initialization method to localize the CCA in the transverse section achieving the highest similarity compared with the clinicians' results; (2) Segmentation of a large number of carotid longitudinal section images including healthy and atherosclerosis images; (3) Classification of CCA longitudinal images using the transfer learning of AlexNet achieving 100% accuracy, specificity, and sensitivity for our collected dataset; (4) Measuring the IMT of the CCA achieving a great match with the clinicians’ values that work as the ground truth.

The residue of the paper is arranged as follows: Sect. 2 presents the literature. The methodology is described in Sect. 3. The testing environmental is explained in Sect. 4. Section 5 presents the results and discussion. Conclusion and future work are provided in Sect. 6.

2 Literature

Our diagnostic system contains various methods (localization, segmentation, classification, and IMT measurement) that have been applied to the CCA images. So, due to the lack of papers that contain all of our methods in one paper, we have been divided the literature as follows.

2.1 Localization of the CCA in transverse section

Until recently, a number of studies have been done for the localization of the CCA in transverse mode images. Hamou et al. (2007), proposed a powerful programming contouring method for CCA segmentation that utilizes a restrained Canny detection and a function that permits a user to make a Priori knowledge about a the specified object to be recognized. They used 91 images of carotid artery in ultrasound using a SONOS System. They achieved accuracy from 0.866 to 0.903 and their segmentation results validated depending on an independent clinical radiologist.

Yang et al. (2011), suggested a morphology method to segment the carotid images which is based on self-adaptive histogram equalization, non-linear filtering and canny edge detector. Their dataset contains three patients with 180 Ultrasound images and their segmentation results compared with manual contours delineated by an experienced sonographer which achieved 0.705 index similarity. Yang et al. proposed a hybrid method comprising mathematical morphology and gradient-vector-field snake (Yang et al. 2012). The authors were provided with 110 CCA images in transverse mode from Robarts Research Institute (RRI) at The University of Western Ontario (UWO), Ontario, Canada. The algorithm accuracy was evaluated by the comparison to the physician-drawn contours with 90.3% ± 3.5% Dice similarity.

Tang et al. (2013), were used a semi-automated technique based on level sets steered by regional intensity and gradient information for CCA lumen segmentation. The authors used 56 images of carotid artery from a publicly available database on the cls2009 challenge. They obtained a Dice similarity coefficient of 90.2%. Luo et al. (2019) were proposed a segmentation technique for CCA semi-automatically on 283 images from a study of Chinese Atherosclerosis Risk Evaluation. The average Dice index for these images was 0.87 ± 0.09.

Groves et al. (2020), applied segmentation of the transverse images of the carotid artery using Mask RCNN and U-Net models. Their two models were evaluated by applying fourfold cross-validation. Their dataset images were collected using an Ultrasonic scanner (Sonic Touch, USA) obtaining 15 subjects. They obtained a dice score of 71% by comparing their results with manual segmentation of the carotid applied by an experienced student in carotid imaging. Ukwatta (2013), were proposed a semiautomated system to segment both CCA boundaries as the Media adventitia and Lumen intima depending on the modified level of the sparse field. Their system required user interface at each segmented image to define the anchor points that covered the carotid artery. They used 21 CCA US images as narrowed and healthy arteries from Ontario university of Research Ethics. Their algorithm achieved a Jaccard similarity of 84.85 ± 5.6 and 90.4 ± 4.3 for the carotid MAB and LIB.

Benes et al. (2013), were proposed a genetic programmed approach to localize the carotid images in transverse image. Their dataset consists of 16 normally ultrasound sequences for volunteers aged 23–47 from the SP lab and they divided it into training and validation. They obtained a detection accuracy of 82.7% and with this result, they outperformed those before them by a difference of 4%. Lorza et al. (2013), were suggested a semiautomated system to segment the normal and abnormal carotid artery images by applying cuts of the graph surface. Their system needs user interface to put the starting points that works as the initialization points to perform the carotid segmentation. Their dataset images consist of twenty subjects of atherosclerotic and non-atherosclerotic carotid images that obtained by using the free hand and external US software. They achieved two Jaccard similarity values for non-atherosclerotic and atherosclerotic as 84% and 66.7%.

Yeom et al. (2014), were introduced three semiautomated techniques to segment the carotid artery images by using the correlation map and a drawing ellipse to produce of the blood boundary and the initial boundary was selected manually. Their segmentation techniques have various shapes and sizes of the carotid lumen. Their dataset consists of 12 CCA subjects obtained from Visual sonics US imaging system in Canada. Their three system results were measured as the ratio between their expanded and contracted area as follows 59%, 61.3%, and 62.3 for the first, second, and third technique.

Narayan and Marziliano (2015), were proposed a semi-automated technique to detect and segment the CCA depending on its echogenicity and the US inferior formation. They applied hypoechoic anatomical of each dataset images to find the region of interest that delineate the CCA. Their 41 images dataset and the manual segmentation of each image was obtained by the Otorhinolaryngology hospital. Their segmentation algorithm achieved a JSC of 87.4% while comparing with the clinicians manually extracted regions.

Smistad and Lindseth (2015), were applied a real time and automated system to segment the carotid in transverse section by using the Kalman filter to delineate the artery and then perform the segmentation using mesh algorithm. Their dataset consists of two sequences of B mode US images (Analogic, USA). Their algorithm achieved an average registration error of 12.4 mm and an average JSC of 86%. Narayan et al. (2015), were introduced an automated algorithm to segment the carotid images using supervised learning and asymmetry of features. They used three datasets as follows 34 images acquired from unhealthy people and volunteers, 18 images from Elizabeth Hospital in Singapore, and 24 images used for training and testing the supervised learning network. Their similarity with the clinicians was calculated by finding the average JSC as 87.9%.

Dorazil et al. (2019), proposed a technique to track the CCA boundary in US images depending on the intensity difference and a circle exhaustive select algorithm. They used 145 US images acquired by Ultrasonic medical system in Canada. Their system achieved a mean error of 3.12 px and a midsection rate of 1.38%. Then they applied a different algorithm to track the carotid artery using Bayesian and optical tracking algorithm (Dorazil et al. 2020). Their system performance was evaluated using the RMSE of 28.58 and SD of 6.56.

Singh and Sahani (2020), proposed three various machine learning algorithms to localize the CCA boundary as support vector machine, Adaboost, and logistic regression. They used 100 images of the ARTSENS dataset and divided it into 60% and 40% for training and testing. The Adaboost algorithm obtained the highest accuracy in comparable with the other algorithms as 91%. Hossain et al. (2014), applied a semiautomated method to segment the CCA LIB using edge level technique on five patients achieving a JSC of 89.82%. Then they also applied another semiautomated system for localization using a regularized edge level and energy function with a JSC of 88.31% (Hossain et al. 2015).

2.2 Segmentation of the CCA in Longitudinal section

A Few studies had been proposed for the segmentation of the CCA in longitudinal section and classification of its images as normal and abnormal. Golemati et al. (2007), were proposed a technique for segmenting the CCA in longitudinal section images based on Hough transformation. They used four nonstenotic carotid artery images recorded from Ultramark Systems. Their average segmentation accuracy was 0.95%. Benes et al. (2011), were proposed a RANSAC method to localize the CCA which based on classification of pixels depending on image information with localization accuracy of 96.66%. They used 30 B-mode ultrasound images to validate their technique which captured by Sonix OP of different volunteers.

Kumar et al. (2019), proposed an algorithm to segment 84 longitudinal CCA images by applying active oblongs obtaining a Jaccard index of 0.87. Rocha et al. (2012), were applied the fuzzy edge detection and the fuzzy classification algorithm to segment the carotid longitudinal lumen. Their dataset consists of 50 longitudinal US gray scale images of non-stenotic people.

Bastida-Jumilla et al. (2013), were segmented 16 normal US images of longitudinal carotid images by inserting frequency implementation in the active contour criteria. Santos et al. (2013), had automatically identify the bifurcation of the carotid artery in longitudinal mode images after applying anisotropic filter for removing speckle noise. Then they were applied chan vese process to segment the artery for posterior contouring of the artery boundary. Their dataset consists of 5 grayscale normal ultrasound people acquired from a GE Healthcare system.

Rouco and Campilho (2013), were proposed a phase symmetry process to segment the carotid longitudinal images depending on a monogenic function and dynamic programming. They were applied their algorithm on 50 non stenosis longitudinal US images obtained from 25 various volunteers. Dhupia et al. (2020), were segmented the intima lumen boundary of the longitudinal carotid images using numerous processes using median filter, operations of pixel offset, curves fittings and oblongs convergence. Their algorithms were applied on 84 healthy US carotid arteries of the SP laboratory.

2.3 Classification of the CCA in Longitudinal section

Bandyk et al. (1985) were classified the internal common carotid artery stenosis using 243 images and 82% of them are correctly classified. Chaudhry et al. (2013) proposed a technique to classify 100 longitudinal of the CCA as normal and abnormal images which obtained from Shifa International Hospital, Islamabad, Pakistan using SVM. Chaudhry et al. (2016), applied classification technique using SVM based on measuring the IMT of the segmented image. They tested their algorithm on 300 longitudinal ultrasound images and yields an accuracy of 98.8%.

Samiappan and Chakrapani (2016) were classified the carotid ultrasound images into two steps. First, they applied segmentation of the carotid IMT using gradient filtering, active contour, and thresholding. Then, they classified the segmented IMT as normal and abnormal using an ANN classifier. Their system was evaluated using 361 images understudy and achieved an accuracy of 89.34%. Varma et al. (2020) applied CNN to classify 100 carotid images as normal or clot acquired from the hospital of SRM with an accuracy of 79.75%.

Chatterjee et al. (2020) were classified the CCA images as atherosclerotic or normal by using basic CNN and then applied transfer learning of MobileNet to perform the classification process. Their dataset consists of 92 cases acquired from authenticated sources achieving an average accuracy of 95%. Saxena et al. (2020) have applied intense dynamic thermography to classify 27 carotid US images as healthy or unhealthy with an accuracy of 90%, specificity of 95%, and sensitivity of 88%.

2.4 IMT measurement for plaque detection

A number of studies had proposed to measure the IMT of the CCA in longitudinal section for early detection of plaque. Loizou et al. (2009) investigated a segmentation method to measure the IMT for 100 longitudinal CCA images which recorded from ATL HDL ultrasound scanner with a linear probe and a frequency of 7 MHz using snake segmentation. Petroudi et al. (2011) presented an algorithm to segment the IMT for 10 longitudinal images of CCA based on chan-vese model for active contouring. Petroudi et al. (2012), were introduced a technique to segment and measure the IMT of 100 images based on active contour criteria using random initialization. Christodoulou et al. (2012) discussed an algorithm to segment the lumen and the IMT of the CCA images to measure the stenosis in the artery for 20 normal asymptomatic subjects obtained from ATL HDL ultrasound scanner with 4 MHz linear transducer using adaptive snake contour algorithm.

3 Methodology

Actually, machine learning methods get research interest which gives the ability to diagnose a large number of ultrasound images with high efficiency. Our proposed system aims to diagnose the common carotid artery (CCA) disease. The presented system has four stages for CCA localization, segmentation, classification, and IMT measurement as shown in Fig. 1. The first stage is used for CCA localization in transverse mode images. The second stage has been applied to segment the CCA in longitudinal mode. The third stage is used to measure the CCA intima-media thickness (IMT) in longitudinal mode images. The fourth phase is used to classify the CCA into normal and abnormal in longitudinal mode. The whole system has been evaluated using ultrasound images from different databases. The four phases are divided as follows.

Fig. 1
figure 1

A flowchart of the proposed system for Common carotid artery (CCA) diagnosis. The flowchart shows the four phases and highlights the different types of machine learning techniques used for each system phase. The deep learning was used in the localization and classification phase

Full size image

3.1 Localization of the CCA in transverse mode

We introduce an automated method to localize the CCA in transverse section ultrasound images as shown in Fig. 1. Our algorithm consists of two main steps. First, we generate a rectangular box around the CCA using Faster R-CNN (Ren et al. 2016). This Faster R-CNN architecture consists of a Convolutional neural network (CNN), Region Proposal Network (RPN), and Object Detection Network. We used 23 layers AlexNet (Shin et al. 2016) as our CNN and pre-trained it on the CCA images in Dataset 1. Then, we localize the CCA by converting the bounding box (detected) around the artery to a circle that contoured the CCA. Our circle localization technique has been applied by getting the yellow mask which represents the detected bounding box and automatically filled it as first. Then, we find the centroid of the bounding box and its equivalent circular diameter. Finally, we draw the circle and display the red contour centered over our yellow bounding box. The CCA localization consists of two main stages.

3.1.1 Faster R-CNN for object detection

Faster R-CNN is an architecture network suggested by Ross Girshick et al. (Ren et al. 2015) and it is an improved iteration of the Fast R-CNN (Girshick 2015). We used the Faster R-CNN to localize the CCA in the transverse section images. The Faster R-CNN takes the ultrasonic image as an input and generates a rectangular box around the object of interest (CCA) as an output. The Faster R-CNN utilizes a region proposal generation procedure (RPN) (Nagaoka et al. 2017) instead of using the selective search that used in the Faster R-CNN.

The input images are given to the AlexNet (Shin et al. 2016) which getting the convolutional feature map. We pre-trained the AlexNet on images of dataset1. The convolutional feature map goes through the RPN which is trained with the ground truth data to give the Region Proposals. The RPN used to predict whether there is a CCA in the image or not and also predict the bounding box around the artery. The RPN is fine-tuned for the region proposal task which initialized by the pre-train image classifier. For every window, the network generates the outmost variety of k- anchor boxes. We tend to use two cases to assign a positive label to associate anchor: if the Intersection-over-Union (IoU) (Cao et al. 2019) between the anchors and the ground truth is greater than 0.7, it classified as a positive label. It considered as a negative label if the IOU overlap is < 0.3.

The loss function (Ren et al. 2016) is characterized in:

$$L\left({P}_{n},{t}_{n}\right)=\frac{1}{{N}_{cls}}\sum_{n}{L}_{cls}\left({P}_{n},{{P}_{n}}^{*}\right)+\lambda \frac{1}{{N}_{reg}}\sum_{n}{{{P}_{n}}^{*}L}_{reg}\left({t}_{n},{{t}_{n}}^{*}\right)$$
(1)

where \({\mathrm{P}}_{\mathrm{n}}\) shows the predicted probability of the n-th anchor. \({{\mathrm{P}}_{\mathrm{n}}}^{*}\) is the ground truth and it is 1 if the anchor is positive.

There are four parameterized coordinates saved in \({\mathrm{t}}_{\mathrm{n}}\) for the predicted bounding box. The overlapping between the ground truth and a positive anchor referred to\({{\mathrm{t}}_{\mathrm{n}}}^{*}\). \({\mathrm{L}}_{\mathrm{cls}}\) \({,\mathrm{L}}_{\mathrm{reg}}\) are the binary and smoothed loss. The mini-batch size (Ncls = 256) and the reg term is quantified by the anchor location (Nreg ∼ 2, 400) and λ = 10. The four coordinates (Meng et al. 2018) for the bounding box regression:

$${t}_{m}=(m-{m}_{a})/{f}_{a}$$
(2)
$${t}_{n}=(n-{n}_{a})/{g}_{a}$$
(3)
$${t}_{f}=\mathit{log}\left(\frac{f}{{f}_{a}}\right)$$
(4)
$${t}_{g}=log(g/{g}_{a})$$
(5)
$${{t}_{m}}^{*}=\frac{ {m}^{*}-{m}_{a}}{{f}_{a}}$$
(6)
$${{t}_{n}}^{*}=({n}^{*}-{n}_{a})/{g}_{a}$$
(7)
$${{t}_{f}}^{*}=log({f}^{*}/{f}_{a})$$
(8)
$${{t}_{g}}^{*}=log({g}^{*}/{g}_{a})$$
(9)

where (m, n) is the bounding box center. The bounding box width and height are represented in f and g. The anchor box is \({m}_{a}\), and m* is the ground truth box.

3.1.2 Circle localization

We localize the common carotid artery after detecting the ROI around the CCA using Faster R-CNN and it is an important step in the automated diagnosis of atherosclerosis. We outline the CCA by a circle that contoured the CCA. The CCA localization consists of three stages as shown in Fig. 1. The workflow of the circle localization is shown in Algorithm 1. First, we get the yellow mask which represents the detected bounding box by extracting the color channels in the image and filled it. Then, we find the centroid of the bounding box and its equivalent circular diameter. Finally, we draw the circle and display the red contour centered over our yellow bounding box.

figure a

3.2 Segmentation of the CCA lumen in longitudinal ultrasound images

Segmentation of the CCA is significantly important to diagnose atherosclerosis. We proposed an automatic technique to contour the CCA in B-mode longitudinal section. The workflow of the CCA B-mode longitudinal section segmentation is shown in Algorithm 2. A complete outline of our proposed lumen segmentation is shown in Fig. 1.

figure b

3.2.1 Preprocessing

The median filter was used because of the inferior quality of US images; thus, it can eliminate the speckle noise significantly. Median filter is a nonlinear filter that preserves edges while removing noise and replaces each pixel value with the median of the pixel values in the local neighborhood (Blanchet and Charbit 2006). The median filter splits the image into small squares called pixels. Each of these squares holds a number that symbolizes a specific color. The Median filter groups these numbers into a specific circumference and arrange them to take the number which is in the middle (Shah et al. 2020).

3.2.2 Morphological operations

We have been applied morphological operations on the filtered ultrasound images. Dilation was performed with a flat disk-shaped structuring element of radius 10 pixels. With the assistance of this structuring element just it decides how much the image is to be widened (Chudasama et al. 2015). It helps to vanish small dark spots and enormous the small lucid spots at the places of the US images where their intensity changes quickly. Assume X be a series of input image coordinates and Y be a series of structuring element coordinates and Yb is an interpretation of B with the goal that its beginning is at b. consequently dilation of X by Y is a combination of all points of b to such that the intersection of Yb with X is not null. The dilation of X by Y is characterized as:

$$X \oplus Y=\left\{ b \right| {\widehat{Y}}_{b} \cap A \ne \varnothing \}$$
(10)

3.2.3 Snake segmentation

The active contour without edges (snake segmentation) (Bruntha et al. 2019; Chan and Vese 2001) has been used to segment the lumen of the CCA in longitudinal B-mode images which have been achieved based on curve evolution, Mumford–Shah functional for segmentation, and level set methods. Level set methods (Osher and Sethian 1988) present a very accurate and robust method in order to track the interfaces. The efflorescence of the curve is adjusted by properties of the region in the image Uo (x, y) surrounded by the curve. The model seeks to divide the image into regions depend on pixel intensities and introduces the following energy functional (Li et al. 2020):

$$\mathrm{F }\left({\mathrm{c}}_{1},{\mathrm{c}}_{2},\mathrm{C}\right)=\upmu .\mathrm{Length}\left(\mathrm{C}\right)+\mathrm{v }.\mathrm{Area}\left(\mathrm{inside}\left(\mathrm{C}\right)\right)+ {\uplambda }_{1}{\int }_{\mathrm{inside}\left(\mathrm{C}\right) } {{[\mathrm{U}}_{\mathrm{o}} (\mathrm{x},\mathrm{ y}) - {\mathrm{c}}_{1} ]}^{2}\mathrm{dxdy }+ {\uplambda }_{2} {\int }_{\mathrm{outside}\left(\mathrm{C}\right) }[{{\mathrm{U}}_{\mathrm{o}} (\mathrm{x},\mathrm{ y}) - {\mathrm{c}}_{2} ]}^{2}\mathrm{ dxdy}$$
(11)

where µ ≥ 0, v ≥ 0, λ1, λ2 < 0, are fixed parameters.

3.3 Intima media thickness (IMT) measurements of the CCA

A detailed outline of the proposed IMT measurement is provided in Fig. 1. The algorithm developed to measure the IMT in longitudinal ultrasound images for healthy and unhealthy people includes the following steps:

3.3.1 ROI extraction

In this step, we cropped the ROI that contains the IMT manually as a rectangular area.

3.3.2 Filtering

We used low pass Gaussian filter with kernel size 5 X 5 pixels to remove speckle noise. The Gaussian filter (Kumar and Sodhi 2020) result can be expressed as:

$$I\left(m\right)=\frac{1}{2\pi {\delta }^{2}} . {e}^{-\frac{{m}^{2}+{n}^{2}}{2{\delta }^{2}}} . I\left(n\right)$$
(12)

where m and n are the horizontal and vertical distance from the origin, and δ is that the standard deviation of the Gaussian distribution.

3.3.3 Morphological operations

We have been used morphological closing (Setyawan et al. 2018) to merge any small openings of the image by applying a disk-shaped structuring element of disk 15. The closing morphology has been applied by repeating the dilation more than once and only once erosion. The closing of an image X by a structuring element Y is a dilation followed by erosion (Said et al. 2016) as follows.

$$X. Y=\left(X \oplus Y\right)\ominus Y$$
(13)

3.3.4 Edge detection

We has been used canny edge detection (Yuan et al. 2020) to find the cropped image edges. The Canny detector searches for the local maximum of the image function gradient which calculated using the derivative of the Gaussian filter. The Canny detector uses two thresholds to find strong and weak edges and connected them. The local gradient (Shrivakshan and Chandrasekar 2012) is given by:

$$h\left(a, b\right)=[Ha^2+Hb^2]^{1/2}$$
(14)

and the direction angle (Shrivakshan and Chandrasekar 2012) is given by:

$$\alpha \left(a, b\right)=arctan(Ha/Hb)$$
(15)

3.3.5 IMT measurement

We have been measured the IMT to detect if it is normal, thickening, or plaque. The IMT is less than 1 mm or 0.1 cm for normal people or 1 mm < IMT < 1.3 mm means that there is thickening in the CCA or IMT is greater than 1.3 mm or 0.13 cm means that there is an obvious plaque and there must be medical intervention. Different measurements of the IMT have been performed on longitudinal CCA ultrasound images.

3.4 Classification of CCA in longitudinal B mode

In this paper, we proposed a classification system using a pre-trained Deep CNN to classify the CCA artery as normal or abnormal. A detailed outline of the proposed classification approach is provided in Fig. 1. We used the transfer learning approach utilizing AlexNet (Krizhevsky et al. 2017). The weights of AlexNet is used as initialization to be fine-tuned using the Dataset 3. A very small learning rate of 10–4 was used to update the weights of the convolutional layers. We have changed the final three layers in AlexNet with a two nodes fully connected layer, Softmax layer, and classification layer to fine-tune the AlexNet. When training the network on the CCA images, it gives us an impressive result and make the diagnostic system stronger compared to the other papers mentioned in the literature.

4 Testing environment

4.1 Machine tool

All computations were performed on an Intel® Core™ i3 CPU M380 @2.53GHZ Processor with 4 GB RAM Laptop with MATLAB (2018b).

4.2 Dataset

In this paper, four datasets have been used for CCA diagnosis as shown in the Table 1. The first dataset is used in the CCA localization in transverse mode, the second dataset is used in the IMT measurements, the third dataset is used for the classification and the other last one is used for CCA segmentation.

Table 1 Dataset distribution
Full size table

4.2.1 Dataset 1

This dataset has been used for the localization of the CCA in transverse sections. It is an online dataset provided by the signal processing (SP) lab, Brno University of Technology (K. M. ŘÍHA). It consists of 283 transverse CCA images acquired by Ultrasonic scanner.

4.2.2 Dataset 2

This dataset has been used for the IMT measurements of the CCA in the longitudinal section. It is an open-access dataset of 100 longitudinal images of the CCA that were provided by the Cyprus Institute of neurology and genetics (Loizou et al. 2014). These grayscale images were recorded at the size of 768 × 576 pixels from 42 females and 58 males (from 26 to 95 years old) that obtained from healthy and atheromatous people.

4.2.3 Dataset 3

This dataset has been used for the classification of the CCA as normal and abnormal. It consists of 196 images of the CCA in the longitudinal section. The 196 images were collected as 100 images from the Cyprus dataset (Loizou et al. 2014), 84 images from SP lab dataset obtained from healthy volunteers (mean age 27.5 ± 3.5 years) scanner, and 12 abnormal images collected from different patients that have obvious atherosclerosis.

4.2.4 Dataset 4

This dataset has been used for the segmentation of the CCA in the longitudinal section. It consists of the same 196 images of the previous dataset but all of its images used for testing.

5 Results and discussion

5.1 Evaluation metrics

The localization performance has been quantified using the Jaccard Similarity Coefficient (JSC) to measure the degree of similarity between our contour and the experts' contour as follows in Eq. (4), respectively (Kabir et al. 2020).

$$JSC\left(M,P\right)= \frac{\left| M\cap P\right|}{\left| M\cup P\right|}$$
(16)

where M and P denote the region of the manual and proposed method contours, respectively.

The classification performance is being evaluated using Sensitivity, Specificity, and Accuracy that have been calculated as in Eqs. 1–3, respectively (Hosny et al. 2020).

$$Sensitivity\, (SV)=\frac{\beta }{\left(\beta +\delta \right)}$$
(17)
$$Specificity \, (SP)= \frac{\gamma }{\left(\gamma +\rho \right)}$$
(18)
$$Accuracy\, (Acc)= \frac{\beta +\gamma }{\beta +\gamma +\delta +\rho }$$
(19)

where β, γ, ρ, and \(\updelta \) refers to true positive of positive cases that are correctly classified, false-negative of negative cases that are correctly classified, false-positive of positive cases that are incorrectly classified, and false negative of negative cases of that are incorrectly classified.

The IMT measurement performance is being evaluated using the mean distance \({d}_{|m|}\).we have been obtained the mean value (Petroudi et al. 2012) of the Euclidean distance differences between the U (our segmented IMT) boundary and the V (ground truth) boundary. It is defined as follows:

$${d}_{|m|}\left(U,V\right)= \frac{1}{X}{\sum }_{RR=1}^{n}|U\left(RR\right)-V(RR)|$$
(20)

where RR is an index corresponding to the image columns and n is the total number of columns in the outlined IMT.

5.2 Results

5.2.1 Localization of the CCA in transverse mode

The localization technique has been achieved a CCA detection accuracy of 97.5%. Figure 2 shows the performance of these system on different samples of the test images from dataset 1. We compare our CCA contour with the manual contouring reported by Brno university’s’ signal processing (SP) which serves as the ground-truth for performance evaluation yielded a JSC of 90.86%. Table 2 shows the outperformance of the proposed system over the literature.

Fig. 2
figure 2

Localization of the CCA in transverse section. The first column provides the original image. The second column provides detected bounding box around the CCA using Faster R-CNN. The third column provides circle localization of the CCA over the yellow mask on the original image

Full size image
Table 2 Comparative analysis of various algorithms for CCA localization in transverse mode
Full size table

5.2.2 Segmentation of the CCA lumen in longitudinal ultrasound images

Our segmentation algorithm has been applied using dataset 4 including Speckle noise filtering using median filter, morphological dilation with a structuring element of radius 10 pixels, and active contouring of the CCA lumen in a longitudinal section. A total of 196 images have been perfectly contoured and segmented of the CCA lumen as shown in Fig. 3. Table 3 shows the outperformance of the proposed system over the literature.

Fig. 3
figure 3

Segmentation of the lumen of the CCA in longitudinal ultrasound images. The first column provides the original image. The second column provides the segmented binary Lumen of the CCA in Longitudinal B-mode section. The third column provides the Active contour of the CCA lumen

Full size image
Table 3 Comparative analysis of various algorithms for CCA segmentation in longitudinal mode
Full size table

5.2.3 Intima media thickness (IMT) measurements of the CCA

Our proposed IMT measurements have been applied in dataset 2 to detect the IMT and measure its thickness. Figure 4 shows the performance of the system on different samples of the test images. Linear regression of the estimated IMT from our proposed method and the expert’s manual delineations is illustrated in Fig. 5. The performance of the technique is evaluated by comparing our measurements with the clinician's measurements and compute the mean and SD for both. The mean ± standard deviation (SD) of our method and the experts for IMT measurements are 0.7573 ± 0.52 mm and 0.7604 ± 0.52. Table 4 provides a comparison between the proposed system evaluated IMT and the expert’s evaluation. Table 5 shows the outperformance of the proposed system over the literature.

Fig. 4
figure 4

IMT measurements of the CCA. The first row provides Cropped ROI contains normal, thickening and plaque of IMT in the CCA image. The second row provides De-speckle filtering. The third row provides morphological operations. The fourth row provides canny edge detection

Full size image
Fig. 5
figure 5

Linear regression of the estimated IMT from our proposed method and the expert's manual delineations

Full size image
Table 4 Comparison between our evaluated IMT and the expert’s evaluation
Full size table
Table 5 Comparative analysis of various algorithms for calculating (IMT mean ± SD) in longitudinal CCA sections
Full size table

5.2.4 Classification of CCA in longitudinal B mode

The proposed classification system has been used the transfer learning of deep convolutional neural network which used the pre-trained AlexNet to classify the CCA artery as normal or abnormal. Different evaluation metrics have been used to evaluate the classification stage which accomplished 100% for accuracy, sensitivity, and specificity. The result shows the superiority of the proposed system over the literature as shown in Table 6. Figure 6 presents the accuracy and loss curves for the training and the validation.

Table 6 Comparative analysis of various methods for CCA classification as normal and abnormal images
Full size table
Fig. 6
figure 6

Training progress contain the accuracy in the upper graph and the loss in the lower graph which provide different curves of the training accuracy, validation accuracy, training loss, and validation loss

Full size image

5.3 Discussion

In this article, we have presented a novel complete diagnostic system to detect and identify CCA hardening, and this is very important because carotid artery stricture is a slow and regular disease that may occur in childhood. Therefore, quick diagnosis aids in stopping atherosclerosis from becoming more dangerous and consequently stopping danger to a stroke.

The novelty of this paper is represented in the proposed complete CAD system and its four phases included in it as the carotid transverse mode localization, the carotid longitudinal lumen segmentation, classification of the CCA in the longitudinal section as normal and abnormal, and measure its IMT.

A complete system of CCA diagnosis is not yet available in the literature. In addition, integrating localization, segmentation, classification, and measure the IMT within a single CAD system is still an open challenge. Having the classification phase in the system leads to discard the normal images from measuring the IMT. This increase the chance of the real-time deployment to our automatic CCA diagnosis system that saves computing power and time. Processing time is relevant as the plan is to use the CAD system in real time deployments. We do believe that CCA diagnosis in real time could be useful for radiologists and for physicians to offer faster and better diagnosis accuracy. Therefore, we have considered a short processing time as an evaluation parameter of the developed CCA CAD System. Although the training time was not considered, the average testing time per image was measured at 1.03 s.

Our novel Complete CAD diagnosis system can perform a vital role in the early prediction and diagnosis of carotid artery atherosclerosis and narrowing. It can work as a beneficial tool during the patients’ examination using US screening. Doctors can use our complete system to check the ultrasound image in the beginning before applying any processing to it and classify the US image as normal or abnormal. Then, when classified as abnormal, the doctors can identify and segment the artery in both sections to see the size of the blockage and its effect on the blood flow. In addition, Our CAD system can predict the probability of a person suffering a stroke by accurately measuring the thickness of the artery wall. These measurements are considered as powerful evidence and confirmation of any slight variation that happens to the artery or the start of the buildup of fatty deposits or calcium and consequently taking the required medical actions without having medical difficulties. So, our complete diagnostic approach can be used to examine the CCA totally and accurately, whether in the longitudinal or transverse section.

Our complete system is not yet available in the literature. Unlike the other researchers that discuss only one method in their diagnosing of the CCA stiffness and narrowing. For example, Luo et al. (2019), only proposed a semi-automated technique to localize the CCA in the transverse section. Golemati et al. (2007) were proposed a technique to segment the CCA in longitudinal sections. Chaudhry et al. (2016) proposed a model to classify the longitudinal images of the CCA as normal and abnormal. Loizou et al. (2009) proposed a method to measure the IMT for 100 longitudinal CCA images. But our diagnostic system is distinguished as an integrated system that uses all these methods, which gives a very accurate and early diagnosis of any slight change that occurs to the carotid artery.

The localization system has been implemented using two main steps: the CCA bounding detection using Faster RCNN and the circle localization method. The Faster RCNN consists mainly of the AlexNet and the RPN. The 23 layers of pre-trained AlexNet have fine-tuned to learn how to classify the CCA by extracting its features from our CCA surrounding bounding boxes. The AlexNet has been used because it has less classification error, limit overfitting, and train rapidly. The RPN is responsible for detecting the yellow bounding box around the CCA by applying the training process of the location of the CCA and compare its resulted box with a validated ground truth table.

The main contributions of the localization system are represented in the robust and fully automated initialization method, robust and fully automated localization system, and the highest similarity when compared to the clinicians' results. Our first phase system has been achieved the goal of designing a fully automated and robust localization system. Our localization system assists the doctors obviously during the examination monitoring of the carotid dynamic and static parameters (Rocha et al. 2011) such as the artery stricture and lumen diameter (Sudha et al. 2018). Unlike Narayan and Marziliano (2015), introduced a semi-automated procedure to localize the CCA rely on its echogenicity. Hossain et al. used a semiautomated approach to segment the carotid using the edge level method (Hossain et al. 2014, 2015).

Our Localization system has been provided a fully automated initialization method to apply the robust and strong localization of the artery using Faster RCNN. There are numerous manually or semi-automatically or simple principle initialization techniques that are not appliable and reliable. For example, Ukwatta (2013) employed a user interface to specify the anchor locations that comprised the carotid artery. Andres et al. (Lorza et al. 2013) proposed a carotid semiautomated segmentation system that needs the offset points to achieve the carotid segmentation. Eunseop et al. (Yeom et al. 2014) select the bounding box that covered the CCA manually segment the carotid artery. Yang et al. (2012) proposed a semi-automated GVF model to segment the CCA using manually initialization values by user. Lian et al. (Luo et al. 2019) introduced an algorithm to contour the weak boundary carotid images with the help of the user interface.

A semiautomated deep learning system used to segment the carotid artery was introduced by Zhou et al. (2019a) to recognize the patches generated by a specific window. The carotid segmentation is done by combining ROI carotid images to a U-Net design to train for pixel classification. Tang et al. (2013) suggested a system requiring expert interference to outline the carotid exterior boundaries. Some researchers have applied segmentation of surface interfaces of the carotid images (Ukwatta et al. 2011; Seabra et al. 2009) however they did not discuss the manual tuning point to produce reasonable results. In this article, we applied the faster RCNN to produce the initial bounding box automatically by applying the AlexNet fine tuning to classify the CCA position.

Our automated initialization algorithm is a strong point that shows the efficiency and strength of this paper than others. The initialization method can be used to determine the plaque in the longitudinal and transverse sections of the carotid artery. There are many papers that segment the plaque using deep learning or other algorithms but the initial step was manual or semi-automatic as the plaque is Ultrasound images has an irregular and nonunique shape. For example, Nirvedh (Meshram et al. 2020) introduced a plaque segmentation procedure that requires a manually initial anchor by an expert to provide satisfying segmentation results on patients with an abundant plaque and associated shadowing artifacts. Destrempes et al. (2011) implemented a system in which manual segmentation of plaque must apply by a sonographer depending on the Bayesian model to approximate plaque edges. McCormick et al. (2012) have segmented the occlusion manually depending on the segmented atherosclerotic region observed over the cardiac period and Bayesian regularization. Zhou et al. (2019b) applied the U-Net model to segment the plaque in the transverse section by providing data of the initial outline of the lumen intima and media-adventitia interface boundary.

We intend to use the initialization procedure of our localization system to segment the carotid plaque in transverse and longitudinal mode. Where carotid occlusion is characterized by a non-unique and complex appearance in the US images. Some deep learning networks such as U-Net (Groves et al. 2020) have resulted in imprecise plaque splitting. Since the size of the blockage can vary greatly, a stricter selection algorithm is required after the processing. Faster R-CNN is going to be more proper than U-Net for this application type, as it used the RPN to select the most overlapped bounding box with the correct one, which is necessary for applying segmentation. So, we tend to collect a large dataset that contains carotid plaque areas in the transverse and longitudinal section to segment the occluded area in it.

Our localization technique has been yielded much better outcomes than other studies. we have been obtained results very adjacent to the standard doctors' results. Our localization system presented a fully automated way to localize the CCA in the transverse mode. We accomplished a JSC of 90.86% with a detection accuracy of 97.5% and this is the highest percentage compared to the other researches as shown in Table 2. The CCA localization has been validated on the Brno University of Technology, SP lab publicly available carotid artery database and achieved a great match with the results of the experts.

The main contribution in our segmentation system is evident in the segmentation of a greater number of images in the longitudinal section that contains normal and atherosclerosis images compared to other studies as presented in Table 3. Also, our segmentation technique eliminates the weaknesses of comparable systems such as unreliable segmentation of non-horizontally and curved longitudinal arteries. However, the lumen of the CCA in the longitudinal section has contoured and segmented perfectly as shown in Fig. 3.

Many articles discussing the difficulty of CCA segmentation in the longitudinal section have been conveyed on a small number of images (Santos et al. 2013; Bastida-Jumilla et al. 2013; Golemati et al. 2007). Plausibly, the lack of annotated data is the cause for non-reporting of segmentation results/papers on carotid artery using US longitudinal section images. To overcome this problem partially, we have manually collected three datasets consisting of 196 images of healthy and atherosclerosis longitudinal images of the CCA. We attend to provide the annotated data for our collected datasets to validate our segmentation results by specialist doctors.

The main contribution of our IMT measurement technique is the great match of our dataset measurements and the experts’ estimations that work as the ground truth as illustrated in Table 4. Accurate measurements of the IMT are deeply requested by specialists because the recent studies proved the high correlation between the IMT thickness and the CVD. We measured the IMT of 100 longitudinal images of the CCA. The mean ± standard deviation (SD) of our proposed method and the expert’s measurements for the IMT are 0.7573 ± 0.52 mm and 0.7604 ± 0.52 and they are almost identical and better than others as shown in Table 5.

The main contribution of our classification system represented in using transfer learning of AlextNet to classify the carotid entire image as normal or abnormal which is not a similar study in the US state-of-the-art. Also, our classification model achieved 100% for specificity, sensitivity, and accuracy after 160 iterations which outperformed than other states of the arts as shown in Table 6. There are not many researchers who have classified the carotid artery as normal and abnormal due to the lack of a database containing images of the atherosclerotic carotid artery and contains plaque. Therefore, we tend to collect a standard database that contains images of the CCA in the longitudinal section and also contains plaque.

6 Conclusion and future work

An automated CAD system for CCA diagnosis that contains tasks of localization, segmentation, classification, and IMT measurement is proposed in this paper. The localization task of the CCA images in the transverse section is performed using Faster R-CNN and circle localization techniques. The segmentation task of the CCA images in a longitudinal B-mode section is performed using active contour without edges to contour and segment the lumen of the artery. The classification task is performed to classify the CCA images dataset as normal and abnormal using transfer learning for the pre-trained AlexNet. The IMT measurement is performed to measure its thickness that is considered a very important indicator of healthy or atherosclerotic CCA. The proposed CAD system results are compared with the clinician's measurements and the state-of-the-art. Its performance has a superiority over all known methods in the literature in each stage. In the future, our goal is to generate a larger standard dataset containing a large number of CCA plaque images. Also, we aim to improve the CCA images classification using the optimized deep neural networks.