Introduction

Invasive fractional flow reserve (FFR) is a well-established clinical standard for identifying ischemia-causing lesions to guide revascularization for better prognosis [1,2,3]. Recently, the application of computational fluid dynamics (CFD) to coronary computed tomography angiography (CCTA) allowed for CT-FFR (FFRCT of HeartFlow and cFFR of Siemens) to be calculated noninvasively, achieving satisfactory diagnostic performance, using invasive FFR as the reference standard [4,5,6,7,8,9]. However, the CFD-based model, especially the three-dimensional FFRCT, has high computational demands, which limits on-site clinical use [10]. Recently, a new version of cFFR based on machine learning was introduced after being trained on a large database generated from CCTA anatomies and CFD-base computation [11]. The trained model provides computational FFR values at all locations along the coronary arteries, correlating excellently (0.9994, p < 0.001) with CFD-based cFFR and well (0.729, p < 0.001) with invasive FFR. Most importantly, the machine-learning model reduces the execution time by more than 80 times compared with the CFD-based method, allowing for almost real-time assessment of cFFR [11].

Before machine-learning training, the coronary stenosis parameters are considered as follows: (1) 0–3 stenosis on main branches and 0–2 stenosis on side branches, (2) the maximum degree of radius stenosis, (3) the total length, (4) the location of the stenosis center, (5) the length of the segment with minimum radius, (6) the overall degree of tapering along the stenosis, and (7) bifurcation stenosis. By applying these parameters, 12,000 coronary geometries are generated, covering many common characteristics of pathological situations encountered in clinical practice, except for rare coronary anomalies and aneurysm [11]. The cFFR value (target value in the training step) for the 12,000 coronary geometries is finally computed throughout the entire coronary artery tree using a reduced-order computational blood-flow model [12].

The machine-learning model maps the coronary anatomy to the computed value of cFFR. It is trained using a deep neural network with four hidden layers. The input layer has 28 neurons corresponding to the different features computed from the coronary anatomy, including local geometric features and upstream and downstream geometric features. The four hidden layers contain 256, 64, 16, and 4 neurons, respectively, and use the sigmoidal activation function. Finally, the output layer has a single neuron with the linear activation function. The entire network is optimized using a mean-squared loss function with a stochastic gradient descent algorithm. The generated 12,000 coronary anatomies and corresponding cFFR values are used to train the model. After the model is trained, it can be used to compute the cFFR value along the coronary tree from the coronary anatomy according to CCTA images [11].

The validation of ML-based cFFR was carried out on synthetic-anatomical and patient-specific models. As a result, the ML-based cFFR was confirmed to have excellent correlation to CFD-based FFR in both models (r = 0.9998, p < 0.001, and p = 0.9994, p < 0.001). At the same time, the ML-based cFFR proved to have good diagnostic accuracy at 83.2% against the invasive FFR measurement [11]. Limited independently-verified data have been reported so far. Here, a group of 105 cases with invasive FFR measurement was employed to assess the diagnostic performance of the ML-based cFFR.

On the other hand, several previous studies have found that the performance of identifying lesion-specific ischemia could be improved by considering the amount of myocardium at risk based on invasive catheter angiography (ICA) and CCTA data [13,14,15,16]. Similarly, the subtended myocardial mass defined by CCTA has been confirmed to enhance the ability of intravenous ultrasound or qualitative coronary angiography to identify lesions that need revascularization [17,18,19]. The coronary stenosis and myocardium at risk are considered the two most important factors in predicting FFR. Therefore, the diagnostic performance of ML-based cFFR is compared to that of the combination of DS and myocardium at risk.

Materials and methods

Study population

Between September 2013 and December 2017, 156 consecutive patients were reviewed retrospectively from the electronic patient-record system of the hospital. These patients had undergone both CCTA and invasive FFR measurements within 60 days (mean 15 ± 13 days). The study included intermediate stenosis (40–80% DS on visual estimation) of the coronary artery. 51 patients with the following conditions were excluded: (1) > 30% stenosis in left main coronary artery lesions (n = 12), (2) occlusion in any major coronary artery (n = 7), or > 80% stenosis in FFR-measured coronary arteries (n = 16); (3) significant motion or blurring artifact in CCTA (n = 5), (4) prior coronary stenting in any major coronary artery (n = 11). No coronary arteries with anomalous origin and coronary aneurysm were included. Finally, a total of 105 patients with 117 intermediate coronary lesions were included in the study analysis. This retrospective observational study was approved by the Institutional Review Board of the hospital, and the requirement for written informed consent was waived.

CCTA imaging and diameter-stenosis estimation

CCTA was performed using a second-generation dual-source 128-slice CT scanner (SOMATOM Definition FLASH, Siemens Healthineers, Forchheim, Germany). Patients with heart rate > 70 bpm were treated with oral beta-blockers. Sublingual sprayed nitroglycerin was routinely administered before contrast-material injection. A 60- to 70-ml bolus of iodinated contrast (370 mgI/ml, Ultravist, Bayer Schering Pharma, Berlin, Germany) was injected at a rate of 4.0- to 4.5-ml/s, followed by 20 ml saline chaser. Prospectively ECG-triggered axial acquisition was used in patients with regular heart rate (HR) between 65 and 90 bpm. Retrospectively ECG-gated spiral acquisition was applied when HR was higher than 90 bpm or the HR was irregular with a standard deviation ≥ 5 bpm. Prospectively ECG-triggered high-spiral acquisition was applied in patients with regular HR ≤ 65 bpm. The tube voltage and tube current were set based on the body mass index (BMI) of the patients: BMI ≤ 22.5 kg/m2, 80 kV/400 ref. mAs; BMI > 22.5 kg/m2 or < 27.5 kg/m2, 100 kV/370 ref. mAs; BMI ≥ 27.5 kg/m2, 120 kV/370 ref. mAs, using automatic tube current modulation. All CT images were reconstructed with slice thickness of 0.75 mm and increment of 0.5 mm, using a medium/soft kernel of B26. Two radiologists with 13 and 5 years of experience, respectively, visually estimated the DS of all coronary arteries. At first, they judged the DS independently. Then, they discussed differences in the results and reached a consensus. The Agatston scores of the diseased and total coronary arteries were recorded to assess the coronary calcification load.

Calculation of the myocardium jeopardy index (MJI)

The MJI was introduced initially based on the coronary size and the location in ICA images [14, 20] to estimate the percentage of the left ventricular myocardium subtended to a coronary lesion. It was calculated based on high-quality CCTA images in this study according to the following rules. A score of 1–3 was assigned to all visible coronary arteries in CCTA images, including the left anterior descending artery (LAD), left circumflex (LCX), right coronary artery (RCA), ramus, diagonal artery (D), obtuse marginal (OM), posterior descending artery, and posterolateral artery. In visual assessment on curved planar reformat and volume rendering images, a score of 3 represented a large artery with a length greater than two-thirds of the distance between the cardiac base and the apex. A small vessel, less than one-third the distance, was assigned a score of 1. A middle artery with a length of one-third to two-thirds the distance was assigned a score of 2. The total score of septal branches was 3. The MJI was the ratio of the sum of the artery scores distal to the target lesion to the overall sum of all artery scores. The two radiologists who assessed coronary-DS defined the scores by consensus.

On-site ML-based cFFR measurement

Measurement of cFFR was performed using a workstation-based software (cFFR prototype version 3.0 on syngo.via Frontier, Siemens Healthineers) by two independent radiologists with 5 and 2 years, respectively, of experience in coronary diagnosis. They were informed of the exact position of the measured invasive FFR in the coronary arteries but were blinded to the FFR values. After loading the diastolic CCTA data, the centerlines of each coronary artery, coronary artery lumen, and endocardial and epicardial myocardial borders were generated semi-automatically. Then, manual adjustment was performed to confirm the correct contour. The cFFR values were computed automatically afterward and were color coded along the coronary arteries. The specific cFFR values appeared when moving the mouse on the coronary artery. All cFFR values were tabbed at the exact position of measured invasive FFR. In addition, the cFFR values were also tabbed when color coding changed significantly along the coronary trees. The time needed for image processing and cFFR calculation and tabbing was recorded.

Statistical analysis

Continuous variables were described using mean ± SD, and counts and percentages were used for categorical variables. The cFFR and invasive FFR values were not normally distributed (as assessed with the Shapiro–Wilk test). Therefore, Spearman’s correlation was used to assess the correlations of cFFR and invasive FFR (0.00–0.19 very weak, 0.20–0.39 weak, 0.40–0.59 moderate, 0.60–0.79 strong, 0.80–1.0 very strong). Univariate regression analysis was used to judge if the calcification affected the diagnostic performance of cFFR.

The area under the curve (AUC) was determined using a receiver operator characteristic curve with 95% confident interval (CI) to detect an FFR of ≤ 0.8 in univariate and multivariate analyses. The results of these analyses were used to identify the optimal cutoff point of the scoring scheme to determine the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) with 95% CI. P ≤ 0.05 was considered to suggest statistical significance. Bonferroni correction was used to adjust the p value in multiple comparisons. All analyses were performed using R software, version 3.3.2 (R Foundation for Statistical Computing, Vienna, Austria).

Results

Characteristics of patients and lesions

The mean age of the 105 patients was 62 (44–80) years, including 32 female patients with an average age of 61 (48–80) years. The interval time between the CCTA and invasive FFR was 0–60 days, with an average of 15 ± 13 days and a median of 10 days. Among the 117 coronary arteries, 36 vessels (30.8%) were identified as hemodynamically significant with invasive FFR of ≤ 0.8. Among these 117 vessels, 88 (75.2%) LAD were found, in which four vessels were accompanied by left main lesions (≤ 30% stenosis). Other vessels consisted of 11 RCA, 12 LCX, five diagonal arteries, and one OM. The total coronary Agatston score of the 105 patients was 116 ± 194. The Agatston score of the 117 coronary arteries was 72 ± 118. Patient baseline characteristics are summarized in Table 1.

Table 1 Baseline characteristics of the study patients (n = 105)
Full size table

Among all 105 patients, 64 were scanned with the prospectively ECG-triggered sequence mode with an estimated effective dose of 3.12 ± 0.84 mSv and 29 were scanned with the retrospectively ECG-gated spiral mode with a dose of 6.23 ± 2.89 mSv. The remaining 12 patients were scanned with the prospectively ECG-triggered high-spiral mode with a dose of 0.93 ± 0.33 mSv. The overall average effective radiation dose of CCTA was 3.65 ± 2.18 mSv. The mean time of cFFR assessment was 18 ± 7 (12–30) min including the 3-dimensional model of coronary tree generation and cFFR calculation for each patient.

Diagnostic performance of cFFR

When cFFR ≤ 0.8 was used to diagnose invasive FFR ≤ 0.8, the sensitivity, specificity, PPV, and NPV were 61% (95% CI 44–78%), 91% (95% CI 85–98%), 76% (95% CI 63–91%), and 84% (95% CI 79–90%), respectively (Table 2). The diagnostic accuracy was 82% (95% CI 75–88%), with an AUC of 0.864 (95% CI 0.788–0.932). The diagnostic performance was much better than that for DS, with an 11.5% increase in AUC from 0.775 (p = 0.013, Table 2; Fig. 1). The average calculated cFFR was 0.82 ± 0.086, while the average invasive FFR was 0.82 ± 0.096. Spearman’s correlation analysis showed that cFFR values correlated well to invasive FFR with a coefficient of 0.665 (p < 0.01).

Table 2 Per-vessel diagnostic performance of CCTA for identifying invasive FFR ≤ 0.8
Full size table
Fig. 1
figure 1

Comparison of receiver operating characteristic curves of cFFR, DS, myocardial jeopardy index (MJI), and DS + MJI. The AUC of cFFR, which was the largest, was not significantly different from the AUC of MJI + DS

Full size image

Compared to invasive FFR values, when cFFR ≤ 0.80, 22/29 (75.86%) lesions were true positive (Fig. 2). Eight lesions were false negative within 22 cFFR of 0.81–0.83. While cFFR ≥ 0.84, only 6/68 lesions were confirmed as false negatives. Among all 117 cFFR measurements, there were 14 false negative values, including four 0.81 and four 0.82. On the other hand, there were seven false positive cFFR values, including two at 0.80 (Table 3). The characteristics of lesions or causes of false results are summarized in Table 3. The most common lesion feature was diffuse or long lesions (11/21), (Figs. 3, 4). Other features included calcification (8/21), ostial lesion (3/21), myocardial bridge (2/21), and impaired image quality (2/21). However, univariate regression analysis showed that the calcification load was not the significant factor of measurement accuracy for cFFR.

Fig. 2
figure 2

A true positive cFFR of left anterior descending coronary artery (LAD) with proximal lesion. The curved planar reconstruction image of the LAD showed that the DS was estimated at 55% (a) with an invasive FFR of 0.73 (b). The measurement of cFFR value was 0.72 (c)

Full size image
Table 3 The causes of false positives and false negatives
Full size table
Fig. 3
figure 3

A false negative cFFR of left anterior descending coronary artery (LAD) with diffuse lesion, no calcification. In the curved planar reconstruction image of the LAD (a), with the DS estimated at 40%. Invasive FFR of 0.69 (b), and the measured cFFR value was 0.82 (c)

Full size image
Fig. 4
figure 4

A false positive cFFR of left anterior descending coronary artery (LAD) with a 5-cm long lesion with multiple calcification. In the curved planar reconstruction image (a), the DS was estimated at 50% at the maximum, with a distal invasive FFR value of 0.85 (b), the measured cFFR value before the distal mixed plaques was 0.85, but it was reduced to 0.72 afterward (c)

Full size image

Diagnostic ability comparison between cFFR and combined DS and MJI

When using MJI to diagnose invasive FFR ≤ 0.8, the shortcoming was low specificity at 56% and unsatisfactory accuracy at 60% (Table 2). However, the combined diagnostic accuracy of DS and MJI was increased significantly to 79%, with an AUC of 0.846 (95% CI 0.755–0.922). It was significantly increased from AUCs of DS (0.775 vs. 0.846, p = 0.028), and MJI (0.609 vs. 0.846, p = 0.013). This integrated diagnostic performance was at the same level with that of cFFR (AUC 0.846 vs. AUC 0.864, p = 0.572) (Table 2).

DS and MJI did not provide additional information to enhance the diagnostic ability of cFFR. The AUC of cFFR + DS was 0.882 (95% CI 0.804–0.941) (0.882 vs. 0.864, p = 0.457), and the AUC of cFFR + MJI was 0.868 (95% CI 0.789–0.931) (0.868 vs. 0.864, p = 0.671). The combination of cFFR and DS + MJI achieved a higher AUC of 0.916 (95% CI 0.844–0.967), but with no significant increase compared to that of cFFR alone (0.916 vs. 0.864, p = 0.117) (Fig. 5).

Fig. 5
figure 5

DS and the MJI did not help enhance the diagnostic ability of cFFR. The area under the curve values were as follows: cFFR, 0.864; cFFR + DS, 0.882 (vs. cFFR, p = 0.457), cFFR + MJI, 0.868 (vs. cFFR, p = 0.671), cFFR + DS + MJI, 0.916 (vs. cFFR, p = 0.117)

Full size image

Discussion

After a decade of clinical use of CCTA to evaluate the anatomic severity of coronary lesions, new methods of computed FFR from computed fluid dynamics based on CCTA images are being developed. A three-dimensional CT-FFR modeling technique (HeartFlow, FFRCT) requiring off-site supercomputer analysis has good correlations to invasive FFR ranging between 0.63 and 0.82, with AUCs ranging from 0.81 to 0.90 [4,5,6]. More recently, a one-dimensional computational analysis technique (Siemens cFFR) was developed and can be performed on on-site workstations. The correlation of cFFR and invasive FFR varies from 0.59 to 0.75. The AUC of cFFR in identifying ischemic-causing lesions ranges from 0.83 to 0.92 [7,8,9]. The ML-based cFFR is the latest advanced version of the one-dimensional CFD-based cFFR. This algorithm learns the output of a computed fluid dynamic model through the methods of feature extraction and model training using CCTA images. Although almost perfect results were obtained from its verification test [11], further studies are required for extensive validation of clinical use. In our study, this on-site and rapid computing method displayed a comparative performance with almost the same accuracy at 82% and slightly lower correlation at 0.665 with invasive FFR. Our results are not on par with those of a recent new study, in which only third-generation dual-source CCTA images were used [21]. Previous studies have shown that CFD-based cFFR had different diagnostic performance in different studies [7,8,9, 22]. This can be explained by diverse samples and different CT scanners. The higher image quality shortened calculation time and achieved better diagnostic performance. Therefore, it can be affirmative that ML-based cFFR has comparable accuracy with CFD-based cFFR in identifying ischemic-causing coronary stenosis.

DS is the main criterion in present clinical practice using CCTA because it has excellent performance in assessing coronary stenosis. Although CCTA-based assessments of anatomy stenosis correlate well with ICA assessments [23, 24], intermediate stenosis indicated by CCTA does not necessarily cause real myocardial ischemia [25, 26]. Therefore, a non-invasive method is required to judge the hemodynamic significance of coronary lesions. Compared to DS in our study, ML-based cFFR gained more than 10% accuracy increase in diagnostic ability. Of note, it improved specificity from 78 to 91%, which can reduce the false-positive rate and potentially avoid unnecessary ICA.

Besides DS, the other important factor affecting FFR is myocardium at risk subtended to coronary stenosis, which can be estimated by CCTA [19, 22]. MJI showed a modest correlation to FFR in an invasive coronary angiography study [27]. In addition, cardiac MR imaging verified that MJI accurately estimated the amount of threatened myocardium [27]. In this study, we also proved that the diagnostic accuracy was increased from 75 to 79% with an AUC increase from 0.775 to 0.846 when adding MJI and DS. The diagnostic accuracy of the combination was not significantly different from that of ML-based cFFR alone.

On the other hand, the results of our study showed that neither DS nor MJI helped increase the diagnostic performance of ML-based cFFR. When designing the ML-based cFFR, the designer included comprehensive anatomical information of the stenotic region and the entire coronary arterial tree. During the training phase of the ML algorithm, the computational blood-flow model was simulated to match anatomical features [11]. The endocardial and epicardial contours were depicted to estimate left ventricle (LV) myocardial volume. Only the myocardium at risk was not considered in the cFFR computation. We surmise that the negative result of the MJI effect on cFFR in this study derives from sampling error or disturbance of the whole LV myocardial volume.

Based on detailed analysis of FP and FN cases, it was found that the most common causes of misinterpretations are diffuse or long lesions (11/21) and calcification (8/21). This may be partially explained by the fact that although many features of the stenosis was used in the ML-based cFFR model, some complex configurations of diffuse or tandem diseases were not included [11]. In terms of coronary calcification, previous studies have suggested that three-dimensional FFRCT maintains high diagnostic compatibility in patients with high calcification [5, 28,29,30]. A recently reported ML-based cFFR study also showed good performance in calcified lesions [21]. In this study, the results also revealed that the performance of ML-based cFFR was not affected by coronary calcification.

Finally, some limitations of this study should be mentioned. Because of the retrospective design, selection bias may have occurred, as FFR measurements were performed depending on cardiologist judgment. Several patients did not accept to undergo invasive FFR measurement after the detection of suspicious lesions in CCTA. As a result, only 105 patients with 117 vessels were included in the study, despite the long-term study period from 2013 to 2017, and most of the included diseased arteries were LAD. Moreover, because of the low-level calcification load in this group (diseased coronary-artery Agatston score 72 ± 118), the results of this study cannot be generalized to the diagnostic performance of cFFR in patients with high coronary-calcification load. Finally, because all our patients underwent CCTA with a second-generation dual-source CT scanner, the results of this study may not be directly applied to other CT scanners.

Conclusion

ML-based cFFR ≤ 0.80 computed from CCTA shows better diagnostic performance for detecting ischemia-producing coronary lesions than coronary DS. Its performance is comparative to the ability of combination of DS and MJI. This on-site and near real-time method is looking forward to further clinical investigation.