Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Despite more than five decades of active research since the Framingham study first identified risk factors for atherosclerosis, it is still the second leading cause of death in industrialized nations. Because most measurements of atherosclerosis currently in use provide indirect measurements of the disease, they may be inadequate for use in modern patient care and management and to test new treatments. New sensitive and specific measurements may provide critically needed and more specific risk management tools and more sensitive ways to assess new patient treatments.

Atherosclerosis and its sequelae, stroke, and heart attack are still the leading causes of mortality and morbidity in the developed world, and are expected to take a growing toll in the developing world. Stroke is the most common, serious neurological problem globally and the third leading cause of death among North American adults. Clearly, stroke represents a staggering mortality, morbidity, and economic cost. Improved methods to identify patients at increased risk for stroke, and better techniques to treat and monitor them will have an enormous impact.

Many new molecular therapies and phenotypic therapeutic targets for atherosclerosis are currently being investigated and developed. Parallel to the development of new therapies, an equally important requirement is the development of treatment - specific measurement tools or biomarkers. As the numbers of new treatments for atherosclerosis are expected to increase rapidly, it is critical to continue to develop and validate cost-effective, noninvasive, and direct measurements or phenotypes of carotid atherosclerosis to use as efficacy targets for treatment and for monitoring therapy.

The utility of ultrasound (US) imaging in diagnosis and monitoring of carotid atherosclerosis is well established. Because it is noninvasive and technological advances are continually improving image quality, US imaging is achieving a greater role by providing direct views of the vessel wall and plaque. Thus, improved and well-characterized US imaging has the potential to be used for quantification of plaque surface morphology, volume, and composition, and therefore become the primary carotid atherosclerosis monitoring modality in patients at risk of stroke, who are being treated with plaque stabilization strategies.

2 3D Carotid Ultrasound Imaging

2.1 Benefits of 3D Ultrasound Imaging

Conventional 2D US imaging systems are used extensively to image the carotid arteries and diagnose carotid disease. These US systems are highly flexible, allowing users to manipulate hand-held US transducers freely over the neck to generate real-time 2D images of the carotid arteries and blood flow within them. However, conventional 2D US systems suffer from the following disadvantages for use in carotid disease monitoring:

  • The use of conventional 2D US imaging requires users to mentally integrate many 2D images to form an impression of three-dimensional (3D) pathology in the carotid arteries. Although this approach can be effective, at times, it leads to longer procedures and variability in diagnosis and monitoring changes in the vessels.

  • Monitoring progression and regression of carotid atherosclerosis in response to nonsurgical therapy requires that the same view of the carotid arteries be images over time periods that can be as long as a few years. Since the 2D US imaging transducer is held and controlled manually, it is difficult to relocate of the 2D US image at the exact location and orientation in the body when imaging a patient. Thus, 2D US imaging is suboptimal as an accurate carotid disease monitoring technique.

Over the past two decades, many researchers and commercial companies have been developing efficient 3D US imaging systems for use in a wide variety of applications [1–7]. Because 3D US systems require significant computational speed for reconstructing and viewing of 3D US information in real- or near real-time on inexpensive systems, progress has been slow. However, advances in low-cost computer and visualization technology have now made 3D US imaging a viable technology that can be used in a wide range of applications. Thus, 3D US imaging capability is now widely available using commercial systems and ones developed in research laboratories. The challenge now is to develop efficient and informative 3D image analysis tools for use in monitoring carotid atherosclerosis in the large number of patients with the disease.

Before describing methods for segmentation and analysis of carotid atherosclerosis, we briefly review methods used for acquiring 3D carotid US images and the tools required for visualizing and measuring carotid disease. Further technical and computational aspects of 3D US can be found in recent review articles and books on the subject [8–15].

Because analysis of the carotid arteries requires at least a 4 cm length, use of real-time 3D systems (i.e., 4D systems with 2D arrays) with small transducer apertures cannot be used effectively. Thus, all current 3D US systems used to acquire images of the carotid arteries make use of conventional 1D transducer arrays that produce 2D US images. To produce a 3D image of the carotid arteries, the conventional US transducer must be moved over arteries to collect 2D images necessary to reconstruct the 3D US image. Thus, a method to track the position and orientation of the transducer must be used so that the acquired 2D images can be placed in their correct position and orientation relative to each other in the 3D image. Over the past decade, two methods have been developed to image the carotid arteries: mechanical linear scanners and magnetically tracked free-hand scanners.

2.2 Methods Used to Acquire 3D Carotid US Images

2.2.1 Mechanical Linear 3D Carotid Ultrasound Imaging

Linear mechanical scanners use a motorized mechanism to translate a conventional transducer linearly along the neck over the carotid arteries, as shown in Fig. 1, and two examples of linearly scanned 3D ultrasound images of carotid arteries with complex plaques are shown in Fig. 2. As the transducer is moved over the carotid arteries, transverse 2D images of the carotid arteries are acquired at regular spatial intervals. Since a motor is used to move the transducer linearly at a constant velocity, each acquired 2D image is spaced equally and all images are parallel to each other. This acquisition geometry makes 3D reconstruction easy and possible in real-time. Since mechanical mechanisms can be constructed with different lengths, the length of the scan can be made to be 4–6 cm. Typically, we acquire 2D US images every 0.2 mm. If the 2D US images are acquired at 30 fps, a 4 cm length will require 200 2D US images, which can be collected in 6.7 s without cardiac gating.

Fig. 1
figure 1

(a) Photograph of a mechanical linear scanning mechanism used to acquire 3D carotid US images. The transducer is translated along the arteries, while conventional 2D US images are acquired by a computer and reconstructed into a 3D image in real-time. (b) Photograph of the system being used to scan the carotid arteries

Full size image
Fig. 2
figure 2

Two 3D carotid ultrasound views of two different patients with complex and ulcerated carotid plaques. For each patient, a transverse (a) and (c) and longitudinal (b) and (d) views are shown side-by-side

Full size image

Because the 3D carotid US image is produced from a series of equally spaced conventional 2D US images, the resolution in the 3D image will not be isotropic. The spatial resolution of the image transverse to the vessel axis will depend on the resolution in the acquired 2D image plane. However, the spatial resolution in the 3D scanning direction (i.e., along the artery) will depend on the elevational resolution of the transducer as well as the spacing between the acquired images. The spacing between acquired image planes can be optimized by varying the translating speed and 2D US acquisition rate. Thus, the acquisition rate can be matched to the frame rate of the ultrasound machine, and the velocity can be adjusted so that the sampling interval is no more than half the elevational resolution of the transducer [16]. In this way, the spacing between acquired images is typically not limiting the spatial resolution. Since the elevational resolution is worse than the in-plane resolution of the 2D US images, the resolution of the 3D US image will be the poorest in the 3D scanning direction. Therefore, a transducer with good elevational resolution should be used to obtain optimal results [13].

2.2.2 Free-Hand Scanning with Position Sensing

Since mechanical 3D US scanning mechanisms used for imaging the carotid arteries tend to be bulkier than conventional 2D US transducers, 3D US scanning techniques have been developed that do not require mechanical scanning devices. To use a conventional 2D US transducer for 3D US imaging, the position and orientation of the transducer must be tracked as it is moved over the carotid arteries. This can be accomplished by mounting a sensor on the transducer to allow measurement of the transducer’s position and orientation as the transducer is moved over the body. In this way, the 2D US images and their relative location and orientation recorded and used to reconstruct the 3D US image [17]. Since the locations and orientations of the acquired 2D images are not predefined, an operator must move the transducer over the anatomy at an appropriate speed to ensure that the spatial sampling is proper and to ensure that the set of 2D images does not have any significant gaps. Several approaches for free-hand scanning have been developed, but the approach most commonly used for carotid imaging makes use of magnetic field sensing technology: Ascension–Bird sensor [18]; Polhemus–Fastrak sensor [19]; and Northern Digital–Aurora sensor [20].

The magnetic field sensors approach makes use of a time-varying 3D magnetic field transmitter, which is placed near the patient, and a small receiver containing three orthogonal coils (with six degrees-of-freedom) mounted on the probe to sense the strength of the magnetic field in three orthogonal direction. The position and orientation of the transducer are calculated by measuring the strength of the three components of the local magnetic field as each 2D US image is acquired. This information is then used to reconstruct the 3D US image.

The magnetic field sensor 3D scanning approach allows free-hand scanning with a small and unobtrusive device that allows the transducer to be tracked. However, electromagnetic interference can compromise the accuracy of the tracking, and geometric distortions in the 3D US image can occur if ferrous (or highly conductive) metals are located nearby. Current magnetic field sensors produce excellent images and are less susceptible to sources of error, particularly ones that use a magnetic transmitter placed between the bed and the patient. Nonetheless, to minimize these potential sources of error, the position of the sensor relative to the transmitter must be calibrated accurately and precisely [21–29].

3 Carotid US Imaging Phenotypes

In this subsection, we briefly review the carotid US imaging phenotypes that have been used recently. As discussed in Sect. 1, carotid imaging techniques are increasingly becoming popular as tools to quantify surrogate end points of carotid atherosclerosis due to their capability of direct measurement of plaque as opposed to indirect measurements such as blood serum levels [30]. Such carotid imaging phenotypes allow assessment of atherosclerotic disease progression, and reduce the need for studying the effect of interventions on hard clinical end points, such as stroke or myocardial infarction (MI) [31].

There have been numerous US phenotypes of carotid atherosclerosis used in the past, which have been investigated as surrogate end points of vascular outcomes for monitoring the progression and regression of atherosclerotic plaque burden in patients during medical interventions. For example, Doppler ultrasound-based measurements of stenosis [32, 33], intima–media thickness (IMT) [34, 35], total plaque area (TPA) [36], total-plaque-volume (TPV) [37–39], vessel-wall-volume (VWV) [40, 41], and vessel-wall-plus-plaque thickness (VWT) maps [42–44]. Although Doppler US is a widely used tool for assessing stenosis severity [45, 46], it does not provide information on plaque morphology, plaque vulnerability, and artery remodeling. 2D US IMT measurement, a one-dimensional measurement of arterial thickness, is a reproducible measurement that has been shown to correlate significantly with the risk factors of stroke [47]. However, it is not sensitive to the changes in plaque burden, which is a stronger predictor of cardiovascular events [48, 49]. In addition, change in IMT occurs at a rate of fraction of a millimeter per year [50], resulting in long follow-up times in the monitoring study designs as well as algorithms with high precision to detect significant changes in IMT.

3.1 3D Carotid US Imaging Phenotypes

Recently, there has been an increased interest in the use of area and volumetric measurements of carotid plaque burden, because they are more reflective and sensitive to the changes of the plaque burden [36, 51–53]. With the availability of such measurements, it may be possible to show an effect of therapy with fewer numbers of patients in shorter clinical trials with imaging end points as opposed to longer clinical trials with hard end points involving large number of subjects. For example, the results from the 6-year follow-up of the Tromsø study [54] involving more than 6,000 subjects showed that in a general population, carotid TPA was a stronger predictor of myocardial infarction than IMT. With a 10-year follow-up, the study also reported that the TPA is a strong predictor of first-ever ischemic stroke than IMT [55].

With the availability of the 3D carotid ultrasound images, several 3D volumetric measurements have been used in clinical studies such as, 3D US TPV [37], 3D US VWV [40], and 3D US VWT maps [42, 53].

3D US TPV is a direct volumetric measurement of plaque burden in the left and right carotid arteries [38]. TPV measurements require an expert observer to delineate plaque–lumen and plaque–outer wall boundaries as shown in Fig. 3. TPV measurements encompass more of the complex plaque morphologies and geometries than TPA measurements that are performed only in longitudinal views. Therefore, volumetric measurements are expected to be even more sensitive than the two-dimensional measurements, because carotid plaque progression occurs in three dimensions. Thus, measurements of plaque volume have the potential to be more sensitive to change than do measurements of plaque area, IMT, and carotid stenosis [38]. Therefore, sample sizes required to test the effects of new therapies might be smaller for measurements of plaque volume than for 1D and 2D measurements [38].

Fig. 3
figure 3

Transverse and longitudinal views of the 3D US image are shown with overlaid manual segmentations for the generation of 3D US VWV. (a) Transverse and longitudinal view of a 3D carotid US image. (b) Manual delineations of the lumen and wall boundaries that are performed on transverse slices that are 1 mm apart. (c) and (d) The surfaces generated by interpolation of the 2D contours for the lumen and vessel wall boundaries using the manual segmentations

Full size image

The 3D US VWV is the volume enclosed between the outer wall and the lumen boundaries for the common carotid artery (CCA), internal carotid artery (ICA), and less commonly for external carotid artery (ECA). The measurement of 3D US VWV requires an observer to manually outline the lumen–intima/plaque and media–adventitia boundaries as shown in Fig. 4. The VWV measurements have a higher reproducibility than TPV measurements [40], due to the fact that lumen–intima/plaque and media–adventitia boundaries are more straightforward to interpret than plaque–lumen and wall boundaries in 3D US images. In addition, VWV boundaries measurements are more regular and circular, which may simplify the development of semiautomated segmentation techniques.

Fig. 4
figure 4

Transverse and longitudinal views of the 3D US image are shown with overlaid manual segmentations for the generation of 3D US TPV. (a) Manual delineations of plaque boundaries are shown in red, which were performed on transverse views. (b) Transverse cross section of plaque surface. (c) Longitudinal and transverse view of the plaque surface. (d) The 3D plaque surface generated from interpolating the 2D transverse contours that are outlined manually

Full size image

The 3D US phenotypes discussed so far are global measurements of plaque progression and/or wall thickening, which provide a single value of measurement such as an area or volume. While such phenotypes provide valuable quantitative information about global plaque burden, it does not identify the locations in the vessel, where volumetric changes are occurring. However, identification of the locations of change in plaque burden will enable more precise evaluation of changes in plaque burden during medical interventions. Chiu et~al. [44] proposed 3D US VWT maps (see Fig. 5) and 3D US VWT-change maps (see Fig. 6) to quantify and visualize local changes in plaque morphology on a point-by-point basis using the segmented surfaces for the lumen and outer wall of the carotid arteries. To facilitate the visualization and interpretation of these maps for clinicians, Chiu et~al. [43] proposed a technique to flatten the 3D VWT maps and VWT-change maps to 2D (see Fig. 6a, b).

Fig. 5
figure 5

(a) and (b) The 3D US VWT map color-coded and superimposed on the vessel wall, and the lumen of one patient. The red region on the VWT map (i.e., the region with maximum VWT) corresponds to a deep depression on the lumen surface (i.e., the location of a large plaque) [44]

Full size image
Fig. 6
figure 6

The VWT maps at the first and second scan, and the VWT-change maps for a single patient. (a) The VWT map at baseline, (b) the VWT map at 3 months after baseline, and (c) the VWT-change map for the same patient [43]

Full size image

4 Segmentation Algorithms for 3D Carotid US Images

Image segmentation is a way of partitioning an image into two or more regions, where often the segmented region is an object of interest. To generate the volumetric measurements of carotid plaque burden, the outer wall, lumen, and plaque boundaries are required to be segmented from 3D US images. In particular, for VWV and VWT measurements, the lumen and wall boundaries need to be segmented, whereas for TPV measurement the plaque boundaries need to be segmented. However, obtaining the lumen and outer wall boundaries prior to plaque segmentation may help to locate the plaque boundaries more efficiently.

4.1 Requirement for Segmentation Algorithms

Unfortunately, although 3D US VWV, TPV, and VWT maps provide many advantages compared to measurements of arterial wall thickening or TPA alone, they are still not widely used in research or clinical practice because of the inordinate amount of time required to train observers and to generate those 3D US phenotypes. In this regard, semiautomated methods for segmentation of the carotid outer wall, lumen, and plaque boundaries would greatly improve the time to train observers and for them to generate 3D US phenotypes with high reproducibility. In the following sections, the segmentation algorithms for carotid wall, lumen, and plaque boundaries will be described.

4.2 Segmentation of Carotid Wall and Lumen Boundaries

Currently, carotid outer wall and lumen segmentations are achieved either via manual segmentation or semiautomated segmentation techniques. Although, fully automated segmentation techniques are effective in some segmentation tasks of medical imaging, there are no techniques yet developed for 3D US carotid imaging. So far, manual segmentation is by far the most widely used technique for segmenting the carotid wall and lumen boundaries from 3D carotid US images [40, 41, 56–59]. However, there has been recent progress in the development of semiautomated techniques for 3D US segmentation, which are described in Sect. 4.2.2.

4.2.1 Manual Segmentation

Figure 4 shows manual segmentations of the outer wall and the lumen boundaries overlaid on a 3D US image and the surfaces of the outer wall and lumen for computing 3D US VWV. Typically, 15–20 mm portion of the artery is segmented including 10/15 mm section of the CCA from the carotid bifurcation and 5 mm section of the ICA from the carotid bifurcation. Manual segmentations of the carotid arteries from 3D US images are performed in transverse views. The observer first defines the bifurcation point and the axis of the CCA and ICA. This defined axis is used to obtain approximately orthogonal transverse slices to the artery for the segmentation. The observer is then shown the transverse and the longitudinal views of the artery simultaneously using multi-planar reformatting [13]. The observer first outlines the outer wall in transverse slices that are 1 mm apart, along the section of the artery that is to be outlined. The observer then outlines the lumen in the transverse slices. When the segmentation of both boundaries is completed for the CCA, the same segmentation steps are repeated for the ICA.

4.2.2 Semiautomated Segmentation Algorithms

Although, manual segmentation is still widely used for the carotid wall and lumen segmentation, semiautomated segmentation algorithms are preferred over the manual segmentation because manual techniques are tedious, time-consuming, and may be subject to high operator variability. In addition, manual segmentation demands more training and expertise from the observers. Therefore, developing semiautomated techniques for segmentation and analysis is important for translating the use of 3D carotid US phenotypes to clinical research and ultimately to clinical care. However, due to the low image contrast of the boundaries and less distinguishable intensity histograms, it is challenging for algorithms to segment 3D carotid US images robustly (Fig. 7).

A summary of the segmentation techniques for delineating the carotid outer wall and lumen from 3D US images is shown in Table 1. Most recent segmentation tasks are performed using optimization-based approaches, such as, active contours [60], level sets [61], and active shape models [62]. In such approaches, an objective function is defined, which describes the goodness of the segmentation. The objective function is then minimized to locate the correct boundary. Such objective functions typically contain two types of energy terms: external energy term, derived from the (pre-processed) image data, and internal energy terms, derived from high-level knowledge about the expected shape of the boundaries. A parameter is used to determine the relative importance of each term for the segmentation. Depending on the strength of the data term, this parameter(s) is be tuned for the segmentation task in hand. For example, if the data term is strong, more weight can be assigned to the data term.

Fig. 7
figure 7

This figure illustrates the degree of overlap in the intensity histograms for the lumen, wall, and the background. (a) Transverse view of a 3D US image with manual delineations of the lumen and the outer wall. (b) The corresponding intensity histogram for the lumen, outer wall, and the background region

Full size image
Table 1 Previous papers describing carotid lumen and/or wall segmentations from 3D US B-mode images
Full size table

As shown in Table 1, the algorithms developed for the segmentation of the carotid artery lumen and wall can be broadly categorized into two classes, which are based on the measurement of interest: methods that segment only the lumen boundary and ones that segment the lumen and the outer wall boundary. In this chapter, we briefly review all such algorithms, but focus more on the segmentation methods that are specifically designed for segmenting both boundaries for the purpose of generating volumetric phenotypes of plaque burden.

4.2.2.1 2D and 3D Methods that Segment Only a Single Boundary

Some methods have been developed to delineate only the lumen boundary [63–67]. For example, Gill et~al. [64] proposed a semiautomated method based on dynamic balloon model to segment only the lumen boundary from 3D US images, but validated using a single image. After placing a seed mesh within the lumen, it is inflated to find the approximate boundary using an inflation force. The mesh is then refined using edge-based forces proximal to the lumen boundary. The algorithm yielded a submillimeter value for the mean separation between the manual and algorithm surface. However, the algorithm needed to be validated using multiple images to adequately evaluate its robustness and usefulness in practice. Zahalka and Fenster [67] proposed a lumen only segmentation method using a deformable model with a stopping term based on the image gradient, which was validated using only with a phantom. Solovey [66] proposed a level set-based method to segment the lumen by incorporating a region-based term that minimizes the probability distribution overlap and a weak geometric prior, which encourages convexity of the boundary in a level set framework.

4.2.2.2 2D Methods that Segment Both Boundaries

Ukwatta et~al. [68, 69] proposed a combined 2D segmentation method to segment the outer wall and the lumen boundaries of the carotid arteries from 3D US images using a modified sparse field level set method [70]. Their method is the first semiautomated technique that was proposed to segment both the lumen and outer wall boundaries from 3D US images for the purpose of computing volumetric measurements of plaque burden. Therefore, the rest of this section is devoted to describing their segmentation method.

The segmentation pipeline of their method is shown in Fig. 8. Their method initializes the boundary close to the actual boundary by choosing four or more anchor points x iA , where i = 1 … N on each boundary on each transverse slice to be segmented. After initialization, two level set segmentations with different energies were used to segment the outer wall and lumen boundaries. The outer wall was segmented initially, where the segmented outer wall contour was used to constrain the lumen segmentation from leaking in regions with low image contrast.

Fig. 8
figure 8

Block diagram of the segmentation pipeline [68]

Full size image

Here, the level set equations [61] are reviewed before describing the particulars of the carotid segmentation methods [68]. Let the image be defined as \( I:\varOmega \to \mathbb{R} \), and x ∈ Ω is a spatial variable representing a single point within the image domain \( \varOmega \subset {\mathbb{R}}^2 \). The level set is an implicit representation of a boundary defined by the function \( \phi \left(\mathbf{x}\right):\varOmega \to \mathbb{R} \). The level set evolution equation is then given by:

$$ \frac{\partial \phi \left(\mathbf{x}\right)}{\partial t}+\nabla E\;\left|\nabla \phi \left(\mathbf{x}\right)\right|=0, $$
(1)

where E is the objective function that defines the goodness of the segmentation and ∇E denotes the speed of the evolution. A typical Chan–Vese (CV) energy term [71] that models an object and background by their mean image intensities is given by:

$$ E=\underset{\mathrm{S}\mathrm{moothness}\ \mathrm{term}\ \left({E}_{\mathrm{S}}\right)}{\underbrace{{\displaystyle {\int}_{\varOmega }}\kern0.24em \left|\nabla H\left(\phi \left(\mathbf{x}\right)\right)\right|\;d\mathbf{x}}}+\underset{\mathrm{G}\mathrm{lobal}\ \mathrm{CV}\ \mathrm{term}\ \left({E}_{\mathrm{G}}\right)}{\underbrace{\lambda {\displaystyle {\int}_{\varOmega }}H\left(\phi \left(\mathbf{x}\right)\right)\;{\left(I\left(\mathbf{x}\right)-u\right)}^2+\left(1-H\left(\phi \left(\mathbf{x}\right)\right)\right)\;{\left(I\left(\mathbf{x}\right)-v\right)}^2d\mathbf{x},}} \quad$$
(2)

where H(x) is the regularized Heaviside function that represent the interior region of the boundary, the exterior region of the boundary is given by (1 − H(ϕ(x)) and u and v are the mean image intensities of the inside and outside, respectively.

The speed equation is then obtained from the first variation of the energy function E as follows [71]:

$$ \frac{\partial \phi \left(\mathbf{x}\right)}{\partial t}=\delta \left(\phi \left(\mathbf{x}\right)\right)\left\{\mathrm{div}\;\left(\frac{\nabla \phi \left(\mathbf{x}\right)}{\left|\nabla \phi \left(\mathbf{x}\right)\right|}\right)+\lambda\;\left(\;{\left(I\left(\mathbf{x}\right)-u\right)}^2-{\left(I\left(\mathbf{x}\right)-v\right)}^2\right)\right\} $$
(3)

where δ(ϕ(x)) is the Dirac function, which is the derivative of the Heaviside function. λ > 0 is a parameter that determines the relative importance of the data or the smoothness term.

Outer wall segmentation: Since the outer wall has neither a strong edge nor consistent regional features for segmentation, a combination of various criteria was used for the outer wall segmentation. For example, the outer wall has high image contrast at orthogonal locations to the US beam direction, whereas the parallel locations of the boundary would generally have low image contrast and the image intensities of the interior and exterior of the outer wall are heterogeneous. Therefore, an objective function combining local region-based, edge-based energies, and user interaction are used for the outer wall segmentation. Edge-based forces drive the segmentation successfully in the presence of strong image gradients, whereas the local region-based forces localize the boundary in the local neighborhood.

Let \( \widehat{\mathbf{x}}\in \varOmega \) be another independent spatial variable. The objective function E WB for the outer wall segmentation is defined as:

$$ \begin{array}{lll}{E}_{\mathrm{WB}}\hfill & =\hfill & \underset{\mathrm{L}\mathrm{ocal}\ \mathrm{region}\hbox{-} \mathrm{based}\ \mathrm{energy}\ \left({E}_{\mathrm{L}}\right)}{\underbrace{\alpha_1{\displaystyle {\int}_{\varOmega }}\delta \left(\phi \left(\mathbf{x}\right)\right)\kern0.24em {\displaystyle {\int}_{\varOmega }}{B}_{\mathrm{L}}\left(\mathbf{x},\widehat{\mathbf{x}}\right)\kern0.24em \left[H\left(\phi \left(\widehat{\mathbf{x}}\right)\right)\;{\left(I\left(\widehat{\mathbf{x}}\right)-{u}_{\mathrm{x}}\right)}^2+\left(1-H\left(\phi \left(\widehat{\mathbf{x}}\right)\right)\right){\left(I\left(\widehat{\mathbf{x}}\right)-{v}_{\mathrm{x}}\right)}^2\right]\;d\widehat{\mathbf{x}}\;d\mathbf{x}}}\hfill \\ {}\hfill & \hfill & +\underset{\mathrm{S}\mathrm{moothness}\ \mathrm{term}\kern0.5em \left({E}_{\mathrm{S}}\right)}{\underbrace{{\displaystyle {\int}_{\varOmega }}\kern0.24em \left|\nabla H\left(\phi \left(\mathbf{x}\right)\right)\right|\;d\mathbf{x}}}+\underset{\mathrm{A}\mathrm{nchor}\hbox{-} \mathrm{point}\ \mathrm{based}\ \mathrm{energy}\ \left({E}_{\mathrm{A}}\right)}{\underbrace{\alpha_2{\displaystyle \sum_{i=1}^{N_{\mathrm{P}}}}\kern0.24em {\displaystyle {\int}_{\varOmega }}\;\delta \left(\phi \left(\mathbf{x}\right)\right)\;{B}_{\mathrm{A}}^i\left(\mathbf{x}\right)\;{\left(\phi \left(\mathbf{x}\right)-\phi \left({\mathbf{x}}_{\mathrm{A}}^i\right)\right)}^2\kern0.24em d\mathbf{x}}}\hfill \\ {}\hfill & \hfill & -\underset{\mathrm{E}\mathrm{dge}\hbox{-} \mathrm{based}\ \mathrm{energy}\ \left({E}_{\mathrm{E}}\right)}{\underbrace{\alpha_3{\displaystyle {\int}_{\varOmega }}\kern0.24em \delta \left(\phi \left(\mathbf{x}\right)\right)\;\frac{B_{\mathrm{E}}\left(\mathbf{x}\right)}{\varepsilon +\left|\phi \left(\mathbf{x}\right)-\phi \left({X}_{\mathrm{E}}\left(\mathbf{x}\right)\right)\right|}d\mathbf{x},}}\hfill \end{array} $$
(4)

where the parameters α 1 − 3 > 0. The local region-based energy (E L), which was originally proposed by Lankton and Tannenbaum [72] assumes that at each point on the object boundary, the local interior and exterior neighborhoods can be modeled by their mean intensities u x and v x within the circular region with radius r L defined by \( {B}_{\mathrm{L}}\left(\mathbf{x},\widehat{\mathbf{x}}\right)=\left\{\begin{array}{ll}1\hfill & \mathrm{if}\;\left\Vert \mathbf{x}-\widehat{\mathbf{x}}\right\Vert <{r}_{\mathrm{L}}\hfill \\ {}0\hfill & \mathrm{otherwise}\hfill \end{array}\right. \).

The anchor-point energy encourages the contour to pass through each anchor point x iA , placed by the observer, when segmentation is within a distance r A to an anchor point, which is defined by \( {B}_{\mathrm{A}}^i\left(\mathbf{x}\right)=\left\{\begin{array}{ll}1\hfill & \mathrm{if}\;\left\Vert \mathbf{x}-{\mathbf{x}}_{\mathrm{A}}^i\right\Vert <{r}_{\mathrm{A}}\hfill \\ {}0\hfill & \mathrm{otherwise}\hfill \end{array}\right. \). When the contour points are away from the anchor point by more than r A there is no influence by this energy term.

Similar to edge-based geometric active contours proposed by Caselles et~al. [73] and Malladi et~al. [74], an edge-based energy is used, which utilizes the edge map M(x) generated during the pre-processing stage, where \( {B}_{\mathrm{E}}\left(\mathbf{x}\right)=\left\{\begin{array}{ll}1\hfill & \mathrm{if}\kern0.5em 0<\left\Vert \mathbf{x}-{X}_{\mathrm{E}}\left(\mathbf{x}\right)\right\Vert <{r}_{\mathrm{E}}\hfill \\ {}0\hfill & \mathrm{otherwise}\hfill \end{array}\right. \), is the circular-shaped region around the point x with a radius r E and ε is a small positive constant. \( {X}_{\mathrm{E}}\left(\mathbf{x}\right)=\underset{\widehat{\mathbf{x}}\in s}{ \arg \min}\left|\right|\mathbf{x}-\widehat{\mathbf{x}}\left|\right|,\kern0.5em s=\left\{\widehat{\mathbf{x}};M\left(\widehat{\mathbf{x}}\right)=1\right\} \) is the nearest edge point from point x. The edge force is nonzero when the contour is within a distance r E to the closest edge point.

By taking the first variation of the E WB (4) with respect to ϕ we obtain the evolving equation

$$ \begin{array}{lll}\frac{\partial \phi \left(\mathbf{x}\right)}{\partial t}\hfill & =\hfill & \delta \left(\phi \left(\mathbf{x}\right)\right)\left\{{\alpha}_{\mathrm{L}}{\displaystyle {\int}_{\varOmega }{B}_{\mathrm{L}}}\left(\mathbf{x},\widehat{\mathbf{x}}\right)\delta \left(\phi \left(\widehat{\mathbf{x}}\right)\right)\right.\;\left[{\left(I\left(\widehat{\mathbf{x}}\right)-{u}_{\mathrm{x}}\right)}^2-{\left(I\left(\widehat{\mathbf{x}}\right)-{v}_{\mathrm{x}}\right)}^2\right]d\widehat{\mathbf{x}}\hfill \\ {}\hfill & \hfill & +{\alpha}_{\mathrm{S}}\;\mathrm{div}\left(\frac{\nabla \phi \left(\mathbf{x}\right)}{\left|\nabla \phi \left(\mathbf{x}\right)\right|}\right)+{\alpha}_{\mathrm{A}}\;{\displaystyle \sum_{i=1}^{N_{\mathrm{P}}}}\;{B}_{\mathrm{A}}^i\left(\mathbf{x}\right)\;\left(\phi \left(\mathbf{x}\right)-\phi \left({\mathbf{x}}_{\mathrm{A}}^i\right)\right)\hfill \\ {}\hfill & \hfill & \left.+{\alpha}_{\mathrm{E}}\;{B}_{\mathrm{E}}\left(\mathbf{x}\right)\frac{\operatorname{sgn}\left(\phi \left(\mathbf{x}\right)-\phi \left({X}_{\mathrm{E}}\left(\mathbf{x}\right)\right)\right)}{{\left(\varepsilon +\left|\phi \left(\mathbf{x}\right)-\phi \left({X}_{\mathrm{E}}\left(\mathbf{x}\right)\right)\right|\right)}^2}\right\},\hfill \end{array} $$
(5)

for the outer wall segmentation, where sgn(ϕ(x)) is the signum function.

Lumen segmentation: Similar to the outer wall segmentation, a combination of numerous criteria is used for the lumen segmentation to utilize its characteristics. For example, the lumen boundary has an irregular shape due to the presence of plaque and does not typically contain sharp, high contrast edges. In addition, the intensity distribution of the interior of the lumen is fairly homogeneous. Therefore, a combination of local region-based and global region-based forces is employed to segment the lumen. To stop the leaking at the low-contrast boundaries, the previously segmented outer wall boundary is used to constrain the current lumen segmentation. The objective function E LB used to segment the LIB is given as

$$ \begin{array}{lll}{E}_{\mathrm{L}\mathrm{B}}\hfill & =\hfill & \underset{\mathrm{L}\mathrm{ocal}\ \mathrm{CV}\kern0.5em \mathrm{term}\ \left({E}_{\mathrm{L}}\right)}{\underbrace{\lambda_1{\displaystyle {\int}_{\varOmega }}\delta \left(\phi \left(\mathbf{x}\right)\right)\kern0.24em {\displaystyle {\int}_{\varOmega }}{B}_{\mathrm{L}}\left(\mathbf{x},\widehat{\mathbf{x}}\right)\kern0.24em \left[H\left(\phi \left(\widehat{\mathbf{x}}\right)\right)\;{\left(I\left(\widehat{\mathbf{x}}\right)-{u}_{\mathrm{x}}\right)}^2+\left(1-H\left(\phi \left(\widehat{\mathbf{x}}\right)\right)\right){\left(I\left(\widehat{\mathbf{x}}\right)-{v}_{\mathrm{x}}\right)}^2\right]\;d\widehat{\mathbf{x}}\;d\mathbf{x}}}\hfill \\ {}\hfill & \hfill & +{\displaystyle {\int}_{\varOmega }}\underset{\mathrm{S}\mathrm{moothness}\ \mathrm{term}\ \left({E}_{\mathrm{S}}\right)}{\underbrace{\kern0.24em \left|\nabla H\left(\phi \left(\mathbf{x}\right)\right)\right|\;d\mathbf{x}}}+\underset{\mathrm{A}\mathrm{nchor}\hbox{-} \mathrm{point}\ \mathrm{based}\ \mathrm{energy}\kern0.5em \left({E}_{\mathrm{A}}\right)}{\underbrace{\lambda_2{\displaystyle \sum_{i=1}^{N_{\mathrm{P}}}}\kern0.24em {\displaystyle {\int}_{\varOmega }}\;\delta \left(\phi \left(\mathbf{x}\right)\right)\;{B}_{\mathrm{A}}^i\left(\mathbf{x}\right)\;{\left(\phi \left(\mathbf{x}\right)-\phi \left({\mathbf{x}}_{\mathrm{A}}^i\right)\right)}^2\;d\mathbf{x}}}\hfill \\ {}\hfill & \hfill & +\underset{\mathrm{G}\mathrm{lobal}\ \mathrm{CV}\ \mathrm{term}\ \left({E}_{\mathrm{G}}\right)}{\underbrace{\lambda_3\;{\displaystyle {\int}_{\varOmega }}H\left(\phi \left(\mathbf{x}\right)\right)\;{\left(I\left(\mathbf{x}\right)-u\right)}^2+\left(1-H\left(\phi \left(\mathbf{x}\right)\right)\right)\;{\left(I\left(\mathbf{x}\right)-v\right)}^2\;d\mathbf{x}}}+\underset{\mathrm{Boundary}\ \mathrm{separation}\ \mathrm{term}\ \left({E}_{\mathrm{T}}\right)}{\underbrace{\lambda_4{\displaystyle {\int}_{\varOmega }}\kern0.24em \delta \left(\phi \left(\mathbf{x}\right)\right)\;{B}_{\mathrm{T}}\left(\mathbf{x}\right)\;d\mathbf{x},}}\hfill \end{array} $$
(6)

where the parameters λ 1 − 4 > 0. Similarly to the outer wall segmentation, the smoothness energy, local region-based energy, and the anchor point-based energies are used in the lumen segmentation. The global CV term (E G) (2) is used here to utilize the homogeneous intensity of the lumen region.

Because the outer wall and lumen are separated by the carotid media layer, they are enforced to have a greater or equal separation distance (d T) of 0.5 mm from each other [75]. The boundary separation-based energy E T is given by where \( {B}_{\mathrm{T}}\left(\mathbf{x}\right)=\left\{\begin{array}{ll}1\hfill & \mathrm{if}\kern0.5em \underset{\widehat{\mathbf{x}}}{ \min}\kern0.5em D\left(\mathbf{x},\widehat{\mathbf{x}}\right)<{d}_{\mathrm{T}}\hfill \\ {}0\hfill & \mathrm{otherwise}\hfill \end{array}\right. \), and \( D\left(\mathbf{x},\widehat{\mathbf{x}}\right)=\left|\right|\mathbf{x}-\widehat{\mathbf{x}}\left|\right|,\nabla H\left({\phi}_{\mathrm{MAB}}\left(\widehat{\mathbf{x}}\right)\right)=~1 \). Therefore, the term E T avoids the intersection of the outer wall and lumen.

By taking the first variation of the E LB (6) with respect to ϕ we obtain the evolving equation for the lumen segmentation.

$$ \begin{array}{lll}\frac{\partial \phi \left(\mathbf{x}\right)}{\partial t}\hfill & {=}\hfill & \delta \left(\phi \left(\mathbf{x}\right)\right)\left\{{\lambda}_{\mathrm{L}}\;{\alpha}_{\mathrm{L}}\;{\displaystyle {\int}_{\varOmega }}{B}_{\mathrm{L}}\left(\mathbf{x},\widehat{\mathbf{x}}\right)\;\delta \left(\phi \left(\widehat{\mathbf{x}}\right)\right)\left[{\left(I\left(\widehat{\mathbf{x}}\right){-}{u}_{\mathrm{x}}\right)}^2{-}{\left(I\left(\widehat{\mathbf{x}}\right){-}{v}_{\mathrm{x}}\right)}^2\right]\;d\widehat{\mathbf{x}}\right.\hfill \\ {}\hfill & \hfill & +{\lambda}_{\mathrm{S}}\;\mathrm{div}\;\left(\frac{\nabla \phi \left(\mathbf{x}\right)}{\left|\nabla \phi \left(\mathbf{x}\right)\right|}\right)+{\lambda}_{\mathrm{G}}\;\left(\;{\left(I\left(\mathbf{x}\right)-u\right)}^2-{\left(I\left(\mathbf{x}\right)-v\right)}^2\right)\hfill \\ {}\hfill & \hfill & \left.+{\lambda}_{\mathrm{A}}\;{\displaystyle \sum_{i=1}^{N_{\mathrm{P}}}}\kern0.24em \mathrm{sgn}\;\left(\phi \left({x}_{\mathrm{A}}^i\right)\right)\kern0.24em {B}_{\mathrm{A}}^i\left(\mathbf{x}\right)\;\left(\phi \left(\mathbf{x}\right)-\phi \left({x}_{\mathrm{A}}^i\right)\right)+{\lambda}_{\mathrm{T}}{B}_{\mathrm{T}}\left(\mathbf{x}\right)\right\}\hfill \end{array} $$
(7)

This algorithm was evaluated with 231 2D carotid US slices extracted from 21 3D US images. The algorithm segmentations were compared to the manual segmentations on a slice-by-slice basis as shown in Fig. 9. The algorithm yielded high Dice coefficients (DC) (>92 %), submillimeter errors for the mean absolute distance (MAD) and maximum absolute distance (MAXD) (see Table 2 for more details), and a comparable coefficient of variation (COV) (5.1 vs. 3.9 %) to manual segmentations for the outer wall and lumen boundaries, respectively. The minimum detectable difference of the algorithm in computing the VWV was comparable (64.2 vs. 50.3 mm3) to manual segmentations. Using this method, the total segmentation time is reduced by 5.5 min in comparison to the manual segmentation (2.8 vs. 8.3 min). Out of the mean time of 2.8 min, 1.2 min is required for the computation and the 1.6 min for the initialization by the observer. The authors also reported on a coupled level set segmentation method to segment both boundaries simultaneously [76] for a more robust segmentation.

Fig. 9
figure 9

Sample results of the lumen and wall segmentations for three patients using four anchor points chosen from the initial eight anchor points (dashed outer contour = manual wall, dashed inner contour = manual lumen, dotted outer contour = algorithm-generated wall, dotted inner contour = algorithm-generated lumen). 1, 5, 7, and 10 mm indicate the distance to the BF from each slice [68]

Full size image

This study was validated using the global metrics such as DC, MAD, MAXD, and volume errors, which are sufficient for evaluating for 3D US VWV. However, when the algorithm is required to be used for generating VWT maps, the algorithm must be validated using localized techniques. Chiu et~al. [77] conducted a study, which evaluated the algorithm using standardized carotid maps for the population.

4.2.2.3 3D Methods that Segment Both Boundaries

One of the main drawbacks of the previous method proposed by Ukwatta et~al. [68] is that it still requires a considerable amount of user interaction on each slice to obtain a good segmentation. If the user decides to segment each and every slice of the 3D image, a 2D segmentation technique would take a long time.

Ukwatta et~al. [78] described a direct 3D segmentation algorithm to delineate the outer wall and lumen of the carotid CCA from 3D US images. The segmentation pipeline of the algorithm is shown in Fig. 10. The algorithm uses a similar initialization approach used previously for the 2D segmentation, but requires initialization only on a smaller subset of transverse slices than used in the 2D method [68]. A direct 3D segmentation method has the potential for reducing user interaction over a 2D method, while increasing the robustness of the segmentation by integrating out-of-plane image information. The 3D method used very similar energy functions to the 2D approach and an extension to the 2D sparse field level set method.

Table 2 Overall CCA segmentation results for the segmentation algorithm [68] for 231 2D US images extracted from 21 3D US images
Full size table
Fig. 10
figure 10

Steps involved in 3D segmentation algorithm pipeline [78], which is similar to the previous 2D segmentation method proposed by Ukwatta et~al. [68]

Full size image

The algorithm-generated accuracy and intra-observer variability results are comparable to the previous methods, but with fewer user interactions. For example, for the ISD of 3 mm, the algorithm yielded an average Dice coefficient of 94.4 ± 2.2 % and 90.6 ± 5.0 % for the MAB and LIB and the COV of 6.8 % for computing the VWV of the CCA, while requiring only 1.72 min (vs. 8.3 min for manual segmentation) for a 3D US image.

4.3 Segmentation Algorithms of Carotid Plaque from 3D US Images

The segmentation of plaque boundaries is an even more challenging task than the lumen and outer wall segmentations due to the fuzzy boundaries of the plaque. Unlike the measurement of 3D US TPV, which requires observers to distinguish plaque–lumen and plaque–outer vessel wall boundaries, the measurement of 3D US VWV requires an observer to manually outline the lumen–intima/plaque and media–adventitia boundaries—similarly to the measurement of IMT. These boundaries are more straightforward to interpret than plaque–lumen and plaque–wall boundaries in 3D US images. Furthermore, the measurement of TPV from 3D US images requires trained observers who are experts in 3D US image interpretation and in distinguishing vessel wall from plaque in 3D US images.

4.3.1 Manual Segmentation

Currently, TPV measurements are obtained by manual segmentation of the plaque [38, 39, 79–81]. Initially, the observer defines the medial axis of the artery in longitudinal view. After familiarizing with the orientation and geometry of the plaque using a multi-planer reformatting software, the observer outlines the plaque boundary on transverse slices with inter-slice distance (ISD) of 1 mm. After outlining the complete plaque, the software generates a 3D surface from these contours. As an illustration, Fig. 3a shows some example manual segmentations of plaque overlaid on a 3D US image, and Fig. 3d shows the reconstructed surface of plaque for the computation of TPV. Limitations of this approach include image interpretation and measurement differences within and between observers, long training times for observers, and long durations to perform manual segmentations.

4.3.2 Semiautomated Segmentation Algorithms

Although, there are several methods that have been developed for 2D segmentation of plaque from either 2D or 3D US images, there exists only one study that describes a semiautomated tool to generate the TPV measurement. Buchanan et~al. [82] reported on a semiautomated plaque estimated method to outline the plaque boundaries from 3D US images thereby computing TPVs. The workflow of their method is shown in Fig. 11. For the initialization, the observer defines the beginning and end points of the plaque in the long axis view, followed by two contours in the transverse view to identify the regions of greatest change in shape. The algorithm was evaluated with 22 plaques from 10 3D US images. An example comparison of the manually and algorithm-generated plaque surfaces are shown in Fig. 12. The algorithm plaque volume and the manual plaque volumes were significantly correlated (r = 0.99, p < 0.001) while maintaining a mean COV of ≈12.0 %. One of the main drawbacks of this method is that it currently does not use any image information by the algorithm. After the algorithm segmentation is obtained, the ability to adjust the segmented boundary by the user is highly desirable.

Fig. 11
figure 11

User input for semiautomated TPV measurement. In the longitudinal view (and with assistance from the axial view, not shown) the user identifies the maximum and minimum z-values representing the end points of the plaque (Min Z and Max Z). The user identifies the mid-point of the plaque (C1) and finally C2 and C3 are identified and generated in the axial view. Uniform plaque geometry between C2 and C3 is assumed and a final volume is generated [82]

Full size image
Fig. 12
figure 12

Representative manual and semiautomated segmentation boundaries. (a) Axial view, (b) longitudinal view. Manual (red) algorithm contours (yellow). Yellow bar = 10 mm [82]

Full size image

5 Future Direction

As discussed in this chapter, 3D US imaging is increasingly being used as a surrogate for monitoring carotid plaque burden in patients in clinical trials during medical interventions. For example, Shai et~al. [59] used 3D US VWV measurements to assess the effect of dietary interventions for reducing the carotid plaque burden. The study found that diet-mediated weight loss over a 2-year period induced significant regression of 3D US carotid VWV, even though the effect is similar in low-fat, Mediterranean, or low-carbohydrate diets. Krasinski et~al. [41] used 3D US VWV measurement to assess the effect of atorvastatin in reducing the carotid plaque burden. The clinical trial had 35 subjects with carotid stenosis >60 %, where subjects were randomly assigned to either the statin drug or placebo in a three month trial. The results showed an increase of 70 ± 140 mm3 in 3D US VWV in the placebo group and reduction of 30 ± 130 mm3 in the atorvastatin group. Similarly, Ainsworth et~al. [83] conducted a study to evaluate the effect of atorvastatin using 3D US TPV. They also observed significant difference of the two groups, such that placebo group had a progression of 16.81 ± 74.10 mm3 and the atorvastatin group had a regression of 90.25 ± 85.12 mm3. Since the number of patients suffering from carotid atherosclerosis and subsequently the number of novel treatments for carotid atherosclerosis is expected to grow rapidly, it is important to continue to develop and validate noninvasive, cost-effective, and direct measurements of atherosclerosis for more accurately stratifying risk of stroke.

With improved strategies to treat atherosclerosis nonsurgically and cost effectively, sensitive and noninvasive 3D US techniques that allow quantification of plaque burden are becoming more important in the management of patients who are suffering from carotid atherosclerosis. However, for translating 3D US imaging to clinical research and subsequently to clinical care, development and validation of semiautomated algorithms for segmentation that reduce the overall observer time and observer variability are vital. Especially for large multicenter clinical trials, the inter-observer variabilities of the algorithm in computing 3D US phenotypes should be small enough to detect a significant change.

Although, algorithms have been developed that are either fully automated or that require minimal user interaction for the segmentation of carotid computed tomography angiography (CTA) [84], carotid black-blood MR images [85, 86], segmentation of carotid arteries from 3D US images is a more challenging problem, that still requires a considerable amount of user interaction. State of the art global segmentation algorithms, such as continuous max-flow-based algorithms that have recently been introduced to medical imaging, have been successfully applied for segmentation of the lumen and the outer wall from black-blood MR images. Such global optimization-based techniques that can be easily parallelized may address the limitations of the existing segmentation methods.

Even though 3D US volumetric measurements described in this chapter are directly correlated with the risk of stroke, they are not suggestive of plaque vulnerability for rupture. However, plaque composition measurements and inflammation are strong indicators of plaque vulnerability [87]. A plaque could remodel from a more vulnerable plaque to a stable plaque, without essential change in size during medical interventions [88]. Therefore, volumetric measurements alone are not adequate to stratify plaque vulnerability to rupture and cause ischemic stroke. Although the ability of MR imaging for plaque composition [89, 90] and inflammation analysis [91] have been validated, the ability of 3D US to identify individual plaque components such as fibrous, necrotic core, loose connective tissue, and calcifications is still unclear. Therefore, future work lies in investigating the capability of 3D US imaging to identify plaque components, which could be validated using digital histopathology images of endarterectomy specimens and in vivo MR imaging.