Introduction

In stable coronary artery disease (CAD), mortality and morbidity benefits of revascularization by percutaneous coronary intervention (PCI) have not been fully realized in clinical trials.1,2 One hypothesis is revascularization, although visually “successful”, does not improve myocardial perfusion because incorrect lesions and/or vessel(s) are targeted. This is very plausible especially in cases with multivessel disease or serial stenoses. Typically, fluoroscopic angiography (FA) is performed independently of functional data such as myocardial perfusion imaging (MPI) and therefore, the image datasets are clinically segregated. Hypothetically, individualized fusion of FA and MPI datasets could assist and improve revascularization decisions if anatomic and functional abnormalities could be accurately aligned.

To test this hypothesis several processes must be developed. First, 2D FA datasets must be accurately converted into 3D datasets while maintaining anatomic precision. Second, extraction of left ventricular (LV) epicardial surface from MPI datasets must be accurate. Third, and most importantly, fusion of the 3D FA datasets with LV MPI datasets must be accurate. Fourth, the conversion, extraction, and fusion processes must be fast enough such workflow is not compromised and revascularization is not delayed.

Several fusion techniques, landmark-based3,4 and rigid iterative closest points (ICP),5,6 were developed and validated over the past decade. They require a pair of size-matched 3D artery anatomy from fluoroscopy angiograms and LV surface from single-photon emission computed tomography (SPECT) images. These conditions are challenging because of cardiac motion and thus image acquisitions at different cardiac frames impacting the accuracy of the 3D fusion between artery anatomy and LV surface. In order to match the time points, several studies used principal component analysis-based6 or visual estimation-based4 methods to select and fuse the end-diastolic fluoroscopy angiograms and end-diastolic SPECT images. However, all these estimation-based methods cannot guarantee a pair of size-matched artery anatomy and LV surface. A deformable registration algorithm is needed to improve the accuracy of the 3D fusion.

The objective of this study was to develop a deformable 3D fusion approach to integrate 3D coronary artery anatomy from fluoroscopy angiograms with LV epicardial surface from SPECT-MPI to guide revascularization decision-making.

Materials and Methods

Patient Data

Thirty patients (21 males, and age = 63.0 ± 8.68 years) were retrospectively enrolled. All patients had either stable or exertional angina before they underwent SPECT-MPI, CT angiography, and FA. The severity of perfusion defects varied among the 30 patients: 14 had normal perfusion and 16 had fixed or reversible defects. The description of coronary lesions can be found in Table 1.

Table 1 Coronary lesions among the thirty patients
Full size table

SPECT scan was performed using a dual-headed scanner (CardioMD, Philips Medical Systems, Milpitas, California) with a standard Tc-99 m-based resting and stress protocol and the acquired SPECT images have a voxel size of 6.4 × 6.4 ×  6.4 mm3. CT angiography was performed using a 64-slice scanner (Somatom Definition, Siemens Medical Solutions, Forchheim, Germany) with a standard protocol and the acquired CT angiograms have a voxel size of 0.32 × 0.32 × 0.75 mm3. Fluoroscopic coronary arteriography was performed using an interventional angiography system (AXIOM-Artis, Siemens Medical Solutions, Munich, Germany) with a standard protocol and the acquired fluoroscopy angiograms have a pixel size of 0.34 × 0.34 mm2. It is noted that 19 of the 30 patients did not show right coronary artery (RCA) abnormality so each of them only took one RCA angiogram. This study was approved by the ethics committee of The First Affiliated Hospital of Nanjing Medical University.

The Workflow of this Study

First, 3D arterial anatomy was reconstructed from fluoroscopy angiograms. Second, LV epicardial surface was extracted from SPECT-MPI images. Third, the 3D artery anatomy was registered with the LV epicardial surface. A computer simulation was executed to technically evaluate the accuracy of the 3D fusion approach. Real patient data was used to evaluate the clinical feasibility of the 3D fusion approach.

Fluoroscopy Image Processing

Reconstruction of 3D artery anatomy from angiograms

The reconstruction of 3D artery anatomy includes three steps: artery extraction from fluoroscopy angiograms, imaging geometry calibration, and vessel point correspondences and 3D vessel reconstruction.

Artery extraction

A deep learning model with an improved U-Net++ architecture7,8 was used to extract the coronary arteries on fluoroscopy angiograms (Figure 1B and F). The model completed artery extraction using an encoder-decoder architecture with nested connections and feature pyramids. Based on the extracted artery contours, a morphology thinning-based algorithm9,10 was used to skeletonize the extracted artery tree and then an edge-linking algorithm11 was applied to link the separate skeleton pixel points, where adjacent skeleton pixel points were linked together to form vessel segments till encountering edge junctions or endpoints. The vessel segments were selected via a developed interactive tool to construct complete centerlines of major arteries. Manually drawn segments were involved when the deep learning model performed poorly and therefore failed artery skeletonization. Accordingly, the centerlines on the primary and secondary projection views were extracted and paired (Figure 1C and G). The topology of the artery anatomy was then established and the bifurcations between the arteries were automatically identified. The radii of vessels were obtained by computing the distance between the centerlines and the outer contour of corresponding arteries.

Figure 1
figure 1

Reconstruction of 3D fluoroscopy artery anatomy. (A) and (E) Selected angiograms from the primary and secondary projection views; (B) and (F) extracted arteries; (C) and (G) artery skeletonization and generation of topology landmarks (red stars); (D) and (H) reconstructed 3D artery anatomy

Full size image
Imaging geometry calibration

Imaging geometry parameters are key factors for 3D artery reconstruction. However, the parameters obtained from DICOM header may not be able to accurately define the imaging geometry because of several uncertainties, such as unknown image skew parameters, table translation between image acquisitions, and device assembly tolerances. A calibration algorithm was developed to optimize these parameters, as explained in steps i and ii.

  1. (i)

    A mathematical model was first developed. As shown in Figure 2, a spatial artery bifurcation point \( Q_{i} \) is projected at an intersection point \( q_{1,i} \) on the primary projection plane and at an intersection point \( q_{2,i} \) on the secondary projection plane. Based on the principles of X-ray angiography and pinhole camera models,12 projection matrix mapping spatial point \( Q_{i} \) to projection points (\( q_{1,i} \), \( q_{2,i} \)) was derived. In the coordinate system of primary view, projection matrix \( P_{1} \) can be expressed as in Eq. 1 , where \( SID \) is the distance between X-ray source and center of detector, s is the skew parameter in radial direction, \( u_{c} \) and \( v_{c} \) are the center coordinates of detector. Since the transformation from the primary to the secondary projection systems can be defined as a rotation \( R \) and translation\( t \), the projection matrix \( P_{2} \) can be formulated as in Eq. 2. With a preset skew parameter s, all the geometry parameters in the equations can be initialized, though they may be not precise, by the parameters from DICOM header. Therefore, given two projection points\( q_{1,i} \left( {u_{1} ,v_{1} } \right) \) and \( q_{2,i} \left( {u_{2} ,v_{2} } \right) \), the spatial point \( Q_{i} \left( {x_{i} ,y_{i} ,z_{i} } \right) \) can be obtained by solving an over-determined equation created by the combination of Eqs. 1 and 2 , as shown in Eq. 3, where \( p_{a}^{bT} \) is the \( bth \) row of the projection matrix \( P_{1} \) or \( P_{2} \), \( a \)= [1, 2], \( b \) = [1, 2, 3].

    Figure 2
    figure 2

    Mathematical model of the fluoroscopy angiography system. \( q_{1,i} \) and \( q_{2,i} \) are the projection points of a 3D arterial bifurcation on the primary and secondary planes. Rotation \( R \) and translation \( t \) establish the relationship of the primary and secondary coordinate systems. Eqs. 1, 2, and 3 denote the mathematic model

    Full size image
  2. (ii)

    An objective function was then proposed and optimized to calibrate the geometry parameters. The objective function containing 15 geometry parameters were created to minimize the following mismatch errors: (1) Euclidean distance between the artery bifurcations and the corresponding back projections of reconstructed 3D bifurcations on each image; (2) difference between the directional vectors defined by artery bifurcations and the corresponding back projections of reconstructed 3D bifurcations on each image. A nonlinear optimization algorithm, Levenberg-Marquardt,13 was used to optimize the objective function to obtain the calibrated geometry parameters.

Vessel points correspondence and 3D reconstruction

With the calibrated parameters, an epipolar constraint-based method14 was used to pair the vessel centerline points on the primary and secondary images. Given a point on one of the images, there should be at least one corresponding point lying on the epipolar line on the other image. To avoid multiple correspondences, a dynamic programming-based method was used to find the optimal corresponding point. This method minimizes the error defined by the distance of the corresponding point from epipolar line. After establishing the correspondence of the centerline points on the primary and secondary images, 3D artery centerlines were then reconstructed using the mathematical model mentioned above, and then the artery surface was meshed with quadrangles to reconstruct 3D artery anatomy, as shown in Figure 1 D&H.

Evaluation of Artery Reconstruction

A computer simulation was implemented to evaluate the accuracy of the artery reconstruction algorithm. Fluoroscopy angiograms were simulated using GATE simulator15 and X-CAT phantom.16 A digital system of fluoroscopy angiography was created in GATE based on Philips system for human body. Fluoroscopy angiograms of left coronary arteries (LCA) from two regular views, LAO45°&CRA30° and RAO30°&CAU35°, were simulated. Two regular views of RCA, LAO1°&CRA29° and RAO33°&CAU5°, were simulated. All the simulated images have a pixel size of 0.34 × 0.34 mm2 and a resolution of 512 × 512.

With the simulated angiograms (Figure 3A, B, E, F), LCA and RCA centerlines were reconstructed using the proposed reconstruction algorithm. The 3D LCA and RCA centerlines from X-CAT phantoms were extracted using a 3D thinning algorithm.17 To evaluate the accuracy of 3D artery reconstruction, the mean distances between reconstructed centerlines and the corresponding phantom centerlines (ground truth) were paired and computed (Figure 3C, D, G, H).

Figure 3
figure 3

Computer simulation of fluoroscopy angiography (A, B, E, F) and comparison between reconstructed centerlines and the ground truth centerlines extracted from X-CAT phantom (C, D, G, H). (A) and (B) are simulated LCA fluoroscopy angiograms; (C) and (D) are the comparisons between reconstructed LCA artery centerlines (red lines) and ground truth(yellow lines); (E) and (F) are simulated RCA angiograms; (G) and (H) are the comparisons between RCA centerlines (red lines) and ground truth(yellow lines)

Full size image

SPECT Image Processing

LV epicardial surface extraction from SPECT images

A graphical user interface was developed to identify LV parameters including LV center, apex, base, anterior and inferior grooves (Figure 4A). Once the parameters were determined, a dynamic programming-based (DP) algorithm18 was used to extract LV epicardial surface from SPECT images. This algorithm first transformed long-axis SPECT images from Cartesian to polar coordinates and then calculated the gradients of the polar image by the differences in horizontal and vertical directions. LV epicardial contour in the polar image was identified via searching for the maximal gradients with a preset constraint using a dynamic programming algorithm and thereafter transformed back to Cartesian coordinates. LV epicardial contour points were triangulated and then smoothed using a triangulation mesh smoothing algorithm.19 The surface was then rendered with myocardial perfusion data, as shown in Figure 4B. After extracting the LV epicardial surface, the anterior and inferior grooves were generated and used as landmarks for the initial alignment of 3D artery anatomy and LV epicardial surface.

Figure 4
figure 4

User interface to identify LV parameters and landmarks. (A) Identification of LV parameters. LV center, apex, and base were used to extract LV epicardial surface. Anterior and inferior grooves were used to generate landmarks for 3D fusion. (B) Extracted LV epicardial surface from SPECT images

Full size image

Evaluation of LV Surface Extraction

The accuracy of LV surface extraction was evaluated via a computer simulation. A digital Siemens ECAT system20,21 for the human body was built in GATE, and a heart phantom generated with X-CAT was loaded into the simulation system. Standard physics processes for nuclear imaging were included. With the simulated coincidence data, nuclear images were reconstructed with a voxel size of 3.2 × 3.2 × 3.2 mm3, as shown in Figure 5A.

Figure 5
figure 5

Simulated nuclear images and LV epicardial surface evaluation. (A) Simulated nuclear images using GATE; (B) manually extracted LV epicardial surface (ground truth); (C) LV epicardial surface extracted with DP-based approach; (D) sampling points comparison between (B) and (C)

Full size image

The nuclear images were then processed using our DP-based approach to extract the LV epicardial surface (DP-LV surface), as shown in Figure 5C. For comparison, an experienced operator who was blinded from DP-LV surface manually extracted the LV epicardial surface (ME-LV surface) in Figure 5B. The sampling points of DP-LV and ME-LV surfaces were paired (Figure 5D). The mean distance of DP-LV surface from ME-LV (ground truth) surface was computed to evaluate the accuracy of the surface extraction.

Image Fusion

Fusion between 3D artery anatomy and SPECT LV epicardial surface

Three steps were implemented to complete the 3D fusion: (1) landmark-based initial alignment, (2) fine registration using S-ICP, and (3) vessel-surface overlay.

Landmark-based initial alignment

According to the characteristics of coronary anatomy, left anterior descending artery (LAD) travels in the anterior interventricular groove, proximal left circumflex artery (LCX) travels in the left atrioventricular groove, and posterior descending artery (PDA) travels in the inferior interventricular groove. The grooves obtained from SPECT images (“LV epicardial surface extraction from SPECT images” section) were used as landmarks (as the white arrow shown in Figure 6A) to complete rough alignment of arteries and LV surface. A cost function was created by minimizing the sum of squared distance between the following three curve pairs: (a) between LAD and anterior interventricular groove, (b) between proximal LCX and LV base, (c) between PDA and inferior interventricular groove.

Figure 6
figure 6

Fusion of artery anatomy and LV epicardial surface. (A) Rough registration by landmarks (green line as illustrated as white arrow); (B) fine registration using S-ICP; (C) vessel overlay and vessel contour rendering

Full size image
S-ICP fine registration

Based on the initial alignment, scaling iterative closest points algorithm (S-ICP)22 was implemented to refine the alignment. It introduced a scaling factor into standard ICP to form a quadric constraint optimization problem concerning a transformation with respect to scale S, rotation R, and translation t. Two steps were iteratively executed to solve this optimization problem. The first step was to create correspondences between LV epicardial sampling points and artery centerline points in current status, which is completed by a Delaunay triangulation-based algorithm.23 The second step was to optimize an objective function that minimizes the distance between artery centerline points and the corresponding epicardial sampling points. A singular value decomposition (SVD)-based method was used to optimize the objective function. Therefore, the transformation parameters were obtained until the iteration reaches a preset threshold. Figure 6B shows the result of S-ICP fine registration.

Vessel-surface overlay

After the fine registration by the S-ICP algorithm, all the arteries were overlaid onto the SPECT LV epicardial surface using a vessel-surface overlay algorithm,5 and then artery contours were created using quadrangles as shown in Figure 6C.

Evaluation of the 3D fusion

The accuracy of the 3D fusion was evaluated using both computer simulation and real patient data.

Computer simulation

The artery anatomies from simulated angiograms (“Evaluation of Artery Reconstruction” section) and DP-LV surface (“Evaluation of LV Surface Extraction” section) were fused using the 3D fusion approach. The LV epicardial surface extracted from the X-CAT phantom was manually registered with the DP-LV surface, and then the phantom arteries were overlaid onto the DP-LV surface. The mean distances of fluoroscopy arteries from phantom arteries (ground truth) were computed to technically evaluate the accuracy of the 3D fusion, as shown in Figure 7.

Figure 7
figure 7

Comparison between simulated fluoroscopy and phantom centerlines. (A) and (B) are artery centerlines and LV&RV epicardial surfaces extracted from X-CAT phantom. (C) and (D) are the simulated fluoroscopy artery centerlines (red lines) and ground truth from X-CAT phantom (yellow lines) overlaid on SPECT epicardial surface. Abbreviations are shown in Table 4

Full size image
Real patient data

Fluoroscopy angiograms and SPECT images in 30 patients were integrated using the 3D fusion approach. Their CT angiograms were manually processed by experienced operators who were blinded from the fluoroscopy angiograms and SPECT images. They manually extracted major arteries and LV&RV epicardial surfaces on the CT angiograms using an open-source tool (3D slicer),24 and then registered the CT LV epicardial surface with SPECT epicardial surface via aligning the landmarks (LV base, frontier, and inferior grooves) on both epicardial surfaces. The transformation parameters of registration were also applied to the extracted CT arteries which travel on the LV epicardial surface, and therefore the CT arteries were closely aligned to the SPECT epicardial surface. The aligned CT arteries were overlaid onto the SPECT epicardial surface and regarded as the ground truth to evaluate the accuracy of the 3D fusion. Figure 8 is an example illustrating the comparison of the fluoroscopy and aligned CT arteries.

Figure 8
figure 8

Comparison of fluoroscopy and CT artery anatomy. (A) Coronary arteries (yellow lines) on the CT LV epicardial surface; (B) coronary arteries from CT and fluoroscopy angiograms (red lines) overlaid on the SPECT LV epicardial surface. The mean distance of fluoroscopy and CT arteries on the SPECT epicardial surface, as illustrated by the blue arrow, was used to evaluate the accuracy of the 3D fusion. Abbreviations are shown in Table 4

Full size image
Metrics to Evaluate the Accuracy of 3D Fusion

Two metrics were used to evaluate the accuracy of 3D fusion. The first one is distance-based mismatch error between CT and fluoroscopy arteries on the SPECT LV epicardial surface. It stands for the mean distance between paired CT and fluoroscopy artery points, as the black line illustrated by the blue arrow in Figure 8. If the fluoroscopy and CT artery lengths were different, the distances were only computed for the paired points. The second metric is a segment-based Kappa agreement rate using the AHA 17-segment model. The segments that both fluoroscopy and artery arteries present were recorded and used to compute the Kappa agreement rate.

Results

Accuracy of Artery Reconstruction

Table 2 lists the mean distances between simulated fluoroscopy and phantom arteries (ground truth). In LCA system, a total of 368 fluoroscopy-phantom artery point pairs were evaluated with an overall mismatch error of 1.67 ± 1.07 mm (min: 0.13, max: 4.85). In RCA system, a total of 392 point pairs were evaluated with an overall mismatch error of 1.22 ± 1.06 mm (min: 0, max: 8.12).

Table 2 Mean distances between simulated fluoroscopy arteries and phantom arteries (ground truth)
Full size table

Accuracy of LV Surface Extraction

A total of 4200 DP-ME-LV sampling point pairs were evaluated. The mean distance between DP-LV and ME-LV surfaces (ground truth) was 2.82 ± 1.53 mm (min: 0.02, max: 14.5), which is smaller than the pixel size of the simulated nuclear image (3.2 mm).

Accuracy of 3D Fusion

In the technical evaluation with computer simulation, Table 3 lists the distance-based mismatch errors between simulated fluoroscopy and phantom arteries (ground truth) after registered and overlaid on the DP-LV surface. In the LCA system, the overall mismatch error is 1.86 ± 1.43 mm (min: 0.04, max: 6.87). In the RCA system, due to the absence of posterolateral branch (PLB), only PDA was evaluated. PDA had a mismatch error of 2.21 ± 2.25 mm (min: 0.05, max: 10.74).

Table 3 Distance-based mismatch errors between simulated fluoroscopy arteries and phantom arteries (ground truth) on the simulation LV surface
Full size table

In the clinical validation, Table 4 lists the distance-based mismatch errors between fluoroscopy and CT arteries on the SPECT surface in 30 patients’ data. In the LCA system, the overall mismatch error was 3.84 ± 3.15 mm (min: 0, max: 20.46). In the RCA system, the overall mismatch error was 5.55 ± 3.64 mm (min: 0.11, max: 24.25). Moreover, Table 5 lists the segment-based mismatch error between fluoroscopy and CT arteries on the SPECT surface. In the LCA system, the overall Kappa agreement rate was 0.91 (CI 0.89-0.93). In the RCA system, the overall Kappa agreement rate was 0.80 (CI 0.67-0.92).

Table 4 Distance-based mismatch errors between fluoroscopy arteries and CT arteries (ground truth) on SPECT LV epicardial surface
Full size table
Table 5 Segment-based mismatch errors between the fluoroscopy arteries and CT arteries (ground truth) on the SPECT LV epicardial surface
Full size table

Processing Time

All the images were processed with a personal computer: Core I5 CPU (2.8 GHz), 8 GB memory, and Microsoft Windows 10 operating system. In fluoroscopy angiogram processing, the artery extraction required ~ 5 s for each image after loading the deep learning model into the system memory and the 3D artery reconstruction consumed approximately 13 ± 4 s. In the SPECT image processing, the LV epicardial surface extraction required 6 ± 3 s. The 3D fusion between them consumed 7 ± 2 s.

The interactive identification of artery centerlines on the fluoroscopy angiograms was approximately 4 minutes, which is the time barrier.

Discussions

The primary objective of this study was to develop and validate an approach that integrates 3D fluoroscopy artery anatomy with SPECT LV epicardial surface to guide PCI decision-making. The 3D fusion approach showed high accuracy in technical evaluation with computer simulation. Besides, in clinical evaluation with 30 real patients’ data, 3D fusion had mismatch errors of 3.84 ± 3.15 mm for LCA and 5.55 ± 3.64 mm for RCA, which is much smaller than the segment size of the AHA 17-segment model (~ 30 × 30 mm2); the Kappa test showed good agreement rates of the fluoroscopy and CT artery locations on the SPECT epicardial surface: 0.91 for LCA, 0.80 for RCA. Accordingly, the 3D fusion approach showed clinical feasibility to integrate fluoroscopy angiography and SPECT-MPI for guiding revascularization decision-making.

Clinical Significance of 3D Fusion

SPECT-MPI stress testing is considered a “gatekeeper” prior to invasive angiography and/or PCI in patients with stable CAD. Commonly, revascularization is determined based on visual assessment of a coronary vessel taken in context with perfusion abnormalities described in a written report. Several studies have demonstrated that SPECT-guided PCI improves morbidity compared to anatomic assessment or medical therapy alone.25,26

However, SPECT-MPI guided revascularization without fusion is suboptimal. First, the specificity of SPECT-MPI is limited by attenuation artifacts. Second, standard polar map distorts the size, shape, and locations of perfusion defects.27 Third, vascular territories often overlap and do not necessarily follow standard ascribed distributions. Fourth, although human coronary anatomy is generally similar, each patient’s coronary tree is unique with variations of branch vessels and dominance. These limitations lead to 50-60% mismatches between standard segment-based myocardial perfusion territories and the distribution of patient-specific anatomic coronary trees.28 Finally, in patients with multivessel disease, SPECT-MPI may not demonstrate perfusion abnormalities in each significant vessel. All these factors decrease the diagnostic sensitivity and specificity and in turn, reduce the utility of SPECT-guided revascularization in clinical practice.

Despite the challenges with SPECT-MPI, our data clearly demonstrates the feasibility of real-time 3D fusion of SPECT-MPI and fluoroscopic coronary angiography, which offers an opportunity to improve revascularization decisions and outcomes. Further studies will be required to gauge the potential impact of fusion vs. standard of care for clinical decisions.

Fusion Techniques of Coronary Vessel Anatomy and LV Epicardial Surface

Over the past decade, several fusion techniques for coronary vessels (arteries or veins) and LV surface were developed and validated. These techniques are in three categories: (1) Landmark-based methods. Zhou et al.4 and Faber et al.3 proposed landmark-based methods to integrate LV epicardial surface with 3D coronary vessel anatomy. In both studies, the landmark-based method can only align the major landmark points, however, the branches and extensions of the major vessels may not be accurately aligned. (2) Rigid ICP methods. Babic et al.29 and Toth et al.6 used standard ICP or Go-ICP to fuse LV epicardial surface with coronary vessel trees. Although these two studies completed the fusion by taking advantage of all the vessel points rather than only landmark points, these fusions are rigid transformation and may fail when two models have scale mismatches caused by the separate image acquisitions at different time points of cardiac beating. Therefore, a non-rigid registration has important advantages. (3) Deep learning-based methods. Toth et al.30 used the imitation learning method to register 2D coronary vessels with 2D projection of CT epicardial surface. Due to the complex overlaps of vessels on 2D coronary angiograms, doctors prefer a 3D artery anatomy fusion with LV surface to better exhibit the stenosis of arteries from any views.

The S-ICP algorithm in our study non-rigidly registered 3D coronary artery anatomy with SPECT epicardial surface when scale mismatches existed between them. S-ICP adjusted the scale of 3D artery anatomy up to or down to the optimal scale and then registered it with the SPECT epicardial surface for higher fusion accuracy, which enhances the clinical applicability of 3D fusion. The small distance-based mismatch error and high Kappa agreement rate between fluoroscopy and CT arteries affirmed the accuracy of the 3D fusion approach.

Clinical Applicability

Two essential factors may affect the applicability of the 3D fusion technique. First, for 3D artery reconstruction, the spatial angle gap between the primary and secondary projection views preferably ranges from 45 to 145°. LCA and RCA angiography usually meets this condition from standard views. Second, clear interventricular groove landmarks on the short-axis image (Figure 4A) are needed for the initial registration of 3D fusion. Fortunately, these landmarks constantly exist and can be identified for most of the enrolled 30 CAD patients.

Two interactive operations, artery centerline identification on fluoroscopy angiograms and landmark selection on short-axis images, may affect the reproducibility of the 3D fusion approach differently. In the first operation, based on the artery contour from the deep learning model, the extracted centerline segments are usually clear except those with overlaps on angiograms. Manual selection for the clear segments maintains relatively high consistency, which barely impacts the reproducibility. For the centerline segments with complicated overlaps, manually drawn segments for correction vary among operators; however, the reproducibility can still be well guaranteed because (1) overlap is relatively limited compared to the entire artery tree, (2) experienced operators can distinguish the overlaps through observing dynamic cine of coronary arteries from different views, and (3) centerline points from the primary and secondary views that meet epipolar geometry constraints are paired in the 3D artery reconstruction, whereas the incorrectly drawn centerline points by the operators will not be paired. In the operation of landmark selection, a small number of CAD patients who show blurry interventricular grooves on short-axis images, the identification of landmarks among operators may be different. In the 3D fusion approach, the landmarks are used to initialize the S-ICP registration. As mentioned above, S-ICP registers all artery centerlines rather than landmarks with LV surface based on their morphological features. Therefore, the variation in landmarks identification among operators barely impacts final registration.

The clinical validation with the 30 patients confirmed the applicability of the 3D fusion technique. The overall small distance-based mismatch error and high Kappa agreement rate ensure the accuracy of the 3D fusion. The average processing time of 5 mins is short compared to the procedural time of PCI (approximately 60 minutes), which guarantees the feasibility of this technique.

Limitations

First, the technical accuracy and clinical feasibility of the 3D fusion approach were tested in a relatively small sample size. Prospective validation in a large population with a control group is needed to establish the clinical usefulness of the technique. Second, the diameters of reconstructed arteries were not quantitatively evaluated. A quantitative assessment is needed to further evaluate the artery diameters. Third, the interactive operations affect the reproducibility of the 3D fusion approach, especially the arteries with complicated overlaps. An improved semantic artery extraction is needed to enhance the reproducibility for broader clinical applications. Finally, stress testing (not necessarily SPECT-MPI) prior to FA is a Class IIa indication in patients with symptomatic suspected CAD. As such, if patients have not had a SPECT-MPI prior to FA, an additional SPECT with associated additional radiation and cost would be necessary for fusion.

New Knowledge Gained

A novel technique, which maintains anatomic accuracy, for reconstructing 2D fluoroscopic angiography data into 3D datasets has been developed. The 3D angiographic datasets can accurately and quickly be fused with MPI datasets.

Conclusions

The developed fusion approach is technically accurate to guide revascularization decision-making and clinically feasible to be used in the catheterization laboratory. Future studies are necessary to determine if fusion improves PCI-related outcomes.