Introduction

Coronary computed tomography angiography (CCTA) has emerged as a robust non-invasive tool for detection of coronary artery disease (CAD) [1, 2]. CCTA allows for the assessment of coronary atherosclerotic plaque characteristics as well as luminal narrowing [3, 4]. Several invasive and non-invasive studies have previously demonstrated incremental value of high-risk plaque characteristics and overall plaque burden for predicting future cardiac events, in addition to those provided by luminal stenosis [5–7]. Moreover, aggregate plaque volume has been shown to improve identification of ischemia-causing coronary stenoses based on fractional flow reserve (FFR) as compared to stenosis alone [8]. Therefore, plaque quantification and characterization appear inherently essential for detailed risk stratification [9]. However, despite its overall importance, plaque quantification is not routinely used in clinical practice. This is partially related to inherent limitations of CCTA such as artefacts related to motion or calcium blooming, which may hinder accurate identification of atherosclerotic plaque from surrounding epicardial fat based on obscurity of boundary conditions, although recent advances in CT technology have overcome some of these limitations [10, 11]. Furthermore, current plaque quantification often requires significant effort, including manual drawing of inner and outer lumen contours for accurate plaque quantification, which is time-consuming and heavily dependent on visual inspection by an experienced CT reader [8].

Recently developed quantitative computed tomography coronary atherosclerotic plaque analysis software (QCT), has been shown to accurately extract centrelines, an essential prerequisite to plaque quantification, as well as demonstrate excellent agreement for 2D and 3D parameters with virtual histology intravascular ultrasound (IVUS-VH) [12–14]. Although prior studies have demonstrated excellent correlation with IVUS-VH, its correlation with iLab intravascular ultrasound (IVUS-iLab) has not been previously evaluated. Furthermore, importantly clinical feasibility of fully automated analysis, as well as inter-observer agreement using the software based on clinical experience, has not been previously reported.

In this study, we evaluated the overall agreement between IVUS-iLab and QCT analyses regarding 2D and 3D parameters, as well as the performance of QCT analyses between expert and non-expert CCTA readers and the fully automatic method.

Materials and methods

Patient population

From December 2008 to December 2011, 151 consecutive patients with suspected coronary artery disease who underwent CCTA and invasive coronary angiography (ICA) with iLab intravascular ultrasound (IVUS-iLab) within 60 days at four hospitals (Severance Cardiovascular Hospital, Seoul, Korea, Gangnam Severance Hospital, Seoul, Korea, Wonju Severance Christian Hospital, Wonju, Korea, and National Health Insurance Corporation Ilsan Hospital, Goyang, Korea) were retrospectively enrolled. A total of 161 coronary artery segments with the ICA and IVUS-iLab confirmed culprit lesions were selected.

CCTA data acquisition

CCTA images were acquired using a 64-slice multidetector-row computed tomography system (Somatom Sensation 64; Siemens Medical Solutions, Forchheim, Germany) utilizing prospective or retrospective electrocardiographic gating. All patients with a heart rate of 65 beats per minute or higher prior to CT received 50 mg of metoprolol orally, unless contraindicated. In addition, all patients received a 0.3-mg sublingual dose of nitroglycerin immediately prior to scanning, unless contraindicated. Bolus tracking method was used for contrast injection. The scan parameters were as follows: 64 × 0.6 mm section collimation, 330 ms rotation time, 100 kVp or 120 kVp tube voltage (depending on body habitus), and 350–800 mAs tube current (depending on body habitus). The estimated radiation dose for CTs ranged between 2 and 10 mSv.

IVUS data acquisition

We used a commercially available IVUS system (iLab, Boston Scientific SciMed, Maple Grove, MN, USA). Motorized transducer pullback permitted cross-sectional area (CSA) measurements at 0.5 mm/s and 30 frames/s throughout the length of the lesion. All IVUS studies were performed after intracoronary administration of 200 mcg of nitroglycerin.

CCTA and IVUS data analysis and co-registration

An experienced IVUS reader independently analyzed target segments between bifurcations proximal and distal to the culprit lesion in a blinded fashion using offline software (QCU-CMS v4.69, LKEB, Leiden University, The Netherlands. This is the research version of QIvus, Medis medical imaging systems bv, Leiden, The Netherlands) [15].

QAngio CT Research Edition (v2.02; Medis medical imaging systems bv, Leiden, The Netherlands) was used for the semi-automatic and fully automatic quantitative CT analysis. The QCT plaque analysis process starts with an automatic centreline extraction. Based on these centrelines, straightened multiplanar reformatted (MPR) volumes were reconstructed for the segmentation and quantification. Longitudinal inner lumen and outer vessel wall contours were detected using an automatic algorithm. However, manual editing of both inner lumen and outer vessel wall delineations is possible when needed. Thereafter, cross-sectional lumen and vessel wall detection guided by attraction points derived from the longitudinal contours was performed. The lumen and vessel walls were segmented automatically in the transversal images. Similar to the longitudinal contours, manual editing of the transversal contours is also allowed by the program. However, it should be noted that editing the transversal contours can be a time-consuming process given the possible large number of images. Two experts (level III-experienced CT readers) and two non-experts (5 years of experience as CT technicians) performed analyses on the CCTA data. While both expert and non-expert groups used automatic centreline extraction, the experts edited the inner lumen and the outer vessel wall contours manually, primarily sectional images with 0.5-mm slice thickness, whereas the non-expert reader used the longitudinal contours for manual manipulation, with minimal cross-sectional editing. Lastly, analyses were performed on the same segments using the fully automatic contour detection algorithm, with no manual editing.

The CCTA images were subsequently co-registered and compared with IVUS images, as follows. During IVUS analysis, the slices corresponding to bifurcation proximal and distal to each lesion were recorded. Next, the extracted MPR stack from the CCTA data was used to record the slice location of the corresponding bifurcations. The slice numbers from IVUS and CCTA were then used to determine the correspondence between lesion length, and length differences (which would result in volume measurement differences) between two modalities (possibly due to cardiac motion or pullback speed variations) were corrected by dividing the measured volume parameters in CCTA by the measured lesion length in CCTA, and multiplying that by the lesion length measured in IVUS (Fig. 1). Thereafter, we evaluated the agreement of plaque quantification by the expert reader, non-expert reader, and fully automatic method compared to IVUS, defined as the gold standard. For 2D parameters, minimal lumen area (MLA), maximal lumen area stenosis percentage (%AS), and mean plaque burden percentage (%PB) in each segment were evaluated; and for 3D parameters, plaque volume, defined as vessel volume minus lumen volume, was evaluated. Interobserver variability for all segments was analyzed with regard to plaque burden and plaque volume in a blinded manner between two expert readers and two non-expert readers; and intra-observer variability for randomly selected 20 % of segments was analyzed with regard to plaque burden and plaque volume in a blinded manner by one expert reader and one non-expert reader, respectively, more than 30 days apart.

Fig. 1
figure 1

Corresponding images of IVUS and QCT (expert/non-expert/automatic) plaque analysis at the same coronary artery segment. a IVUS measurements of lumen volume, vessel volume, and plaque volume were 60.21 mm3, 140.39 mm3, and 80.18 mm3, respectively. b Expert QCT measurements of lumen volume, vessel volume, and plaque volume were 54.43 mm3, 147.06 mm3, and 92.63 mm3, respectively. c Non-expert QCT measurements of lumen volume, vessel volume, and plaque volume were 29.38 mm3, 86.63 mm3, and 57.25 mm3, respectively. d Automatic QCT measurements of lumen volume, vessel volume, and plaque volume were 52.06 mm3, 136.83 mm3, and 84.77 mm3, respectively. Compared to IVUS measurements, expert QCT reader slightly overestimated the plaque volume (80.18 vs. 92.63 mm3), while non-expert QCT reader underestimated (80.18 vs. 57.25 mm3). However, automatic analysis showed good result (80.18 vs. 84.77 mm3). IVUS = intravascular ultrasound, QCT = quantitative computed tomography coronary atherosclerotic plaque analysis

Full size image

Statistical analysis

Continuous variables were expressed as mean ± standard deviation when normally distributed, or as median (interquartile range [IQR]) if non-normally distributed. Categorical variables were expressed as absolute numbers or percentages. 2D and 3D parameters were compared between QCT (expert/non-expert/automatic) and IVUS by means of the Pearson correlation coefficient using two-sided p values, with a p-value < 0.05 considered statistically significant. The Fisher z-transformation statistic was applied for comparing two correlations. Bland–Altman plots with 95 % confidence intervals were calculated for correlation. The paired t test was used to analyze differences between IVUS and QCT (expert/non-expert/automatic). Inter- or intra-observer variability was assessed by intraclass correlation coefficients (ICC) for absolute agreement of single measures between consistent raters. Statistical analyses were performed using MedCalc statistical software (version 12.7.5, MedCalc Software bvba, Mariakerke, Belgium).

Results

Selected segments and baseline characteristics

We excluded 11 segments (7 %) from nine patients because of insufficient IVUS (four segments) or CT image quality (seven segments) due to severe calcification or motion artefacts. In total, 150 segments from 142 patients were analyzed. Baseline characteristics of the study population are described in Table 1. The median time interval between CT and IVUS was 24 mins (interquartile range [IQR], 11.0–33.2). The mean age was 62 ± 7.4 years, and 68.3 % of patients were male. Among 150 segments, 8 (5.3 %) were left main artery, 112 (74.7 %) were left anterior descending artery, 9 (6 %) were left circumflex artery, and 21 (14 %) were right coronary artery. The most common plaque characteristics were mixed plaque (51 %), followed by non-calcified plaque (31 %) and calcified plaque (18 %), and the median calcium score was 177.0 (IQR, 11.0–526.5). By IVUS, the average lesion length and %AS were 22.42 ± 9.41 mm and 51.3 ± 13.1 %, and 83 (55.3 %) segments were shown to be obstructive stenosis (>50 % AS). The average analysis time for the expert, non-expert, and automatic QCT method was 34.3 ± 17.8, 27.7 ± 15.2, and 5.8 ± 1.8 m per vessel, respectively.

Table 1 Baseline Patients Characteristics (n = 142)
Full size table

2D parameters: MLA, %AS, and %PB

The mean MLA as determined by non-expert QCT readers was significantly lower than IVUS measurement (2.49 ± 1.36 vs. 2.74 ± 1.42 mm2, p < 0.001); however, there was no significant difference in the mean MLA measurements by expert readers or fully automatic QCT analysis compared with IVUS (Table 2). Correlation coefficients (r) of MLA measured by IVUS compared with expert, non-expert, and the automatic method were 0.89 (95 % confidence interval [CI] 0.86–0.92), 0.82 (95 % CI 0.76–0.87), and 0.80 (95 % CI 0.73–0.85), respectively, showing good to excellent correlation (Table 3). Bland-Altman analysis showed a mean bias of −0.03 mm3 with 95 % limits of agreement extending from −1.34 to 1.28 mm3 between IVUS and expert QCT measurements; when IVUS was compared to non-expert QCT and the automatic method, the mean bias values were 0.25 mm3 and −0.14 mm3, and 95 % limits of agreement were −1.39 to 1.88 mm3 and −1.95 to 1.66 mm3, respectively (Appendix Figs. 4 and 5). For %AS, there was no significant difference of means in non-expert and automatic QCT measurements compared with IVUS, although expert QCT overestimated %AS compared with IVUS (53.0 ± 13.0 vs. 51.3 ± 13.1, p = 0.01) (Table 2) and the correlation coefficients (r) between IVUS and automatic QCT were comparable to expert QCT (0.82 vs. 0.84, p = 0.58) and non-expert QCT (0.82 vs. 0.80, p = 0.62) (Table 3). Bland–Altman analyses showed a mean bias of −1.6 % with 95 % limits of agreement −16.3 to 13.0 % between IVUS and expert QCT; a mean bias of -0.9 % with 95 % limits of agreement −17.1 to 15.3 % between IVUS and non-expert QCT; and mean bias of −1.1 % with 95 % limits of agreement −16.3 to 14.2 % between IVUS and automatic QCT (Appendix Figs. 4 and 5). Compared with IVUS, %PB by non-expert QCT was significantly higher (59.7 ± 10.4 % vs. 57.5 ± 9.7 %, p < 0.001), while the automatic method was significantly lower (55.1 ± 9.4 % vs. 57.5 ± 9.7 %, p < 0.001). On the other hand, there was no significant difference between expert QCT and IVUS (Table 2), and the correlation coefficient (r) between IVUS and automatic QCT showed superior performance to non-expert QCT (0.84 vs. 0.73, p = 0.01) and inferior performance to expert QCT (0.84 vs. 0.91, p = 0.01) (Table 3 and Fig. 2). Bland–Altman analyses showed a mean bias of -0.0 % with 95 % limits of agreement extending from −8.0 to 8.0 % between IVUS and expert QCT; with IVUS compared to non-expert and automatic QCT, mean bias values were −2.2 % and 2.4 %, and 95 % limits of agreement were −16.6 to 12.2 % and −8.3 to 13.1 %, respectively (Fig. 2). Maximal diameter stenosis percentage (%DS) analysis is described in Appendix Tables 5 and 6.

Table 2 Mean and SD measurements of IVUS and QCT (Expert/Non-expert/Automatic)
Full size table
Table 3 Correlation coefficient (r) between IVUS and QCT (Expert/Non-expert/Automatic)
Full size table
Fig. 2
figure 2

Linear regression and Bland–Altman analyses of mean plaque burden percentage between IVUS and QCT (expert (a, b) /non-expert (c, d) /automatic (e, f)). Correlation coefficients (r) of expert (a), non-expert (c), and automatic (e) were 0.91, 0.73, and 0.84, respectively. Abbreviations as in Fig. 1

Full size image

3D parameter: plaque volume

There was no significant difference in the means of plaque volume between IVUS and expert or non-expert QCT, except that it was significantly underestimated by automatic QCT (152.20 ± 87.20 vs. 129.92 ± 75.26, p < 0.001) (Table 2), However, the correlation coefficient (r) between IVUS and automatic QCT showed comparable performance to non-expert QCT (0.84 vs. 0.79, p = 0.20) and inferior performance versus expert QCT (0.84 vs. 0.94, p < 0.001) (Table 3 and Fig. 3). While non-expert QCT underestimated the mean lumen volume (91.14 ± 45.05 vs. 105.12 ± 48.52, p < 0.001) and vessel volume (242.89 ± 122.69 vs. 258.47 ± 127.21, p = 0.01) compared to IVUS, and no significant difference was observed in the mean plaque volume, automatic QCT showed no difference in the mean lumen volume with IVUS (107.78 ± 48.03 vs. 105.12 ± 48.52, p = 0.21) and underestimated the vessel volume (237.70 ± 115.60 vs. 258.47 ± 127.21, p < 0.001), Hence, plaque volume was measured lower than IVUS (Appendix Table 5). However, the correlation coefficient (r) of lumen and vessel volume between IVUS and automatic QCT showed comparable performance to non-expert QCT lumen (0.90 vs. 0.85, p = 0.06) and vessel volume (0.85 vs. 0.83, p = 0.56) in correlation with IVUS (Appendix Table 6).

Fig. 3
figure 3

Linear regression and Bland–Altman analyses of plaque volume between IVUS and QCT (expert (a, b) /non-expert (c, d) /automatic (e, f)). Correlation coefficients (r) of expert (a), non-expert (c), and automatic (e) were 0.94, 0.79, and 0.84, respectively. Abbreviations as in Fig. 1

Full size image

Performance of QCT by plaque subtype

For calcified plaque, there was no significant difference in the mean plaque volume of IVUS compared with expert or automatic QCT measurements; however, non-expert QCT overestimated plaque volume (difference of mean ± SD = 23.5 ± 55.50, p = 0.028), and the correlation coefficient (r) between IVUS and automatic QCT showed superior performance to non-expert QCT (0.90 vs. 0.83, p = 0.02) and inferior performance to expert QCT (0.90 vs. 0.96, p < 0.001) (Table 4). Automatic QCT underestimated mixed plaque volume (difference of mean ± SD = 24.51 ± 5.72, p < 0.001) and non-calcified plaque volume (difference of mean ± SD = 27.93 ± 48.17, p < 0.001) compared with IVUS. On the other hand, expert and non-expert QCT showed no significant difference in the means of plaque volume compared with IVUS. However, the correlation coefficient (r) between IVUS and automatic QCT was comparable to non-expert QCT in non-calcified plaque (0.77 vs. 0.77), and even superior in mixed plaque (0.82 vs. 0.72, p = 0.03).

Table 4 Plaque subtype analysis - mean and SD and correlation coefficient (r) between IVUS and QCT (Expert/Non-expert/Automatic)
Full size table

Inter- and intra-observer variability of expert and non-expert QCT

Interobserver variability of expert QCT for all segments for plaque measurements were excellent, as follows: plaque burden: ICC = 0.93, p < 0.001; plaque volume: ICC = 0.96, p < 0.001. However, the interobserver variability of non-expert QCT was at a slightly lower correlation: plaque burden: ICC = 0.80, p < 0.001; plaque volume: ICC = 0.82, p < 0.001. Intra-observer variability of expert QCT for a randomly selected 20 % of segments measured more than 30 days apart showed excellent correlation, as follows: plaque burden: ICC = 0.98, p < 0.001; plaque volume: ICC = 0.97, p < 0.001. Similarly, intra-observer variability of non-expert QCT for a randomly selected 20 % of segments measured more than 30 days apart showed a slightly lower correlation: plaque burden: ICC = 0.86, p < 0.001; plaque volume: ICC = 0.89, p < 0.001.

Discussion

In this study, we were able to demonstrate high agreement between expert QCT analyses and IVUS measurements for all 2D and 3D parameters analyzed, and although fully automatic QCT analyses showed inferior performance to expert QCT in general, there was no significant difference in means of MLA, %AS, and lumen volume measurements compared to IVUS, showing comparable performance to non-expert QCT analyses. For %PB and plaque volume measurements, however, automatic QCT showed underestimated results and inferior performance to expert QCT in correlations with IVUS, but was still comparable to non-expert QCT (Tables 2 and 3). Furthermore, depending on the plaque subtype, the performance of QCT differed: calcified plaque was overestimated by non-expert QCT, and mixed or non-calcified plaque was underestimated by automatic QCT. Regarding the correlation with IVUS in calcified and mixed plaque, automatic QCT showed superior performance to non-expert QCT (Table 4). Therefore, we conclude that there is significant user variability in semi-automatic plaque quantification based on user level of experience. Furthermore, we were able to show that the automatic QCT analysis, which is time-saving and not user dependent, has excellent ability to detect the lumen and good ability to detect plaque, especially for calcified or mixed plaque, in comparison to IVUS as the gold standard.

In a prior study comparing QAngio CT with IVUS, the correlation coefficients for automatic plaque quantification were 0.75 for the MLA and 0.64 for the mean plaque burden on a lesion basis, which are lower than our automatic analysis results of 0.80 and 0.84, respectively (Table 3) [13]. A subsequent study using QAngio CT demonstrated an improved correlation coefficient of 0.84 for MLA when using automated analysis as compared with IVUS-VH. This is similar to our current study, although their correlation coefficients for lumen volume (0.92), vessel volume (0.96), and plaque volume (0.93) were slightly higher than our current automatic QCT analyses of 0.90, 0.85, and 0.84, respectively (Table 3) [14]. Previous studies have demonstrated that the automatic analysis tends to underestimate the lumen volume and overestimate the vessel volume, leading to overestimation of plaque volume [14, 16]. This is somewhat contrary to our current study, in which the automatic QCT lumen volume analysis was similar to IVUS but the vessel volume analysis was significantly lower, leading to underestimation of plaque volume compared to IVUS measurements (Table 3, Fig. 3). This might be due to our use of the latest version of QCT software (QAngio CT RE, v2.02) and iLab-IVUS in this study, whereas an older version of QCT software (QAngio CT RE, v1.1) and VH-IVUS were used in previous studies [13, 14]

The ROMICAT (Rule Out Myocardial Infarction using Computer Assisted Tomography) study demonstrated the beneficial effects of using CCTA in patients with acute chest pain who presented to the emergency department, where the use of CCTA was associated with lower hospital costs and shortened hospital stays [17]. However, based on the complexity of CCTA interpretations due to artefacts such as motion or blooming related to calcified plaques, CT interpretations often require experienced CT readers, which may result in underutilization in the emergency setting [10, 11]. This problem is of paramount importance during off-hours such as nights or weekends, when an expert reader may not be available, leading to underutilization and causing significant delays in diagnosis. These issues highlight the need for fast and automated software enabling a more rapid diagnosis, especially when an expert reader might not be available. Several recent studies have evaluated the feasibility of an automated software-guided detection of coronary artery disease by CT [18–20]. In the absence of available qualified CT readers, one might expect that a reliable second reader could play a crucial role, especially in emergency settings. Based on our current study, we cautiously suggest that automatic QCT analysis might be used in the near future as a reliable second reader, especially for lumen detection.

CCTA can visualize not only the luminal stenosis, but also the surrounding plaque [3, 4]. Recent studies showed that plaque burden ≥ 70 % and MLA ≤ 4 mm2 had significant clinical meaning and were closely related to future acute coronary events [7]. As such, an accurate method for quantification of plaque burden and MLA could be of use for further risk stratification. In our study, we were able to show good correlation coefficients for minimal luminal area (0.80) and mean plaque burden (0.84), indicating that an automatic QCT analysis may be a reliable tool for prediction of future coronary events, without the need for a reader and the inherent user-dependent variability (Table 4). Our study had limitations. This was a retrospective study, and we evaluated certain lesions that were also assessed by IVUS. Therefore, we were subject to selection bias on a per-patient as well as per-lesion basis. In addition, we only included CT or IVUS images of sufficient quality for this analysis. Finally, we analyzed various kinds of plaques, including heavily calcified lesions, which were thus subject to artefacts, and this may have affected our values.

In conclusion, the current study demonstrated the clinical utility and feasibility of a fully automatic QCT analysis compared with IVUS measurements, as well as compelling performance when compared with semiautomatic expert or non-expert analysis. Although there was user-dependent variability in correlation and quantitative agreement between QCT and IVUS, automatic QCT analysis—which is not dependent on a reader, and hence free from subjective bias, requiring one-sixth the time of expert analysis—showed excellent capability for lumen and plaque detection.