Introduction

Ischemic stroke treatment has recently evolved thanks to mechanical thrombectomy [1]. It consists of removing the blood clot by inserting catheters into the femoral artery that are endovascularly navigated under 2D fluoroscopy up to the clot to catch it. At each bifurcation, the navigation is slow down due to difficulties in pointing into one artery rather than another. The first bifurcation appears at the top of the aortic arch where the physicians have to enter a selected carotid artery. Patient selection for mechanical thrombectomy is done using magnetic resonance angiography (MRA) or computed tomography angiography (CTA).

Mechanical thrombectomy presents many advantages in comparison with thrombolysis. Nevertheless, endovascular navigation can be difficult [2]. Physicians can be supported by roadmap injection but it is only used once the carotids are reached as the volume of aortic arch is important. Currently, the navigation in the aortic arch is done with no contrast dye fluoroscopy. Recent work investigated the potential interest of fluoroscopy fusion with pre-operative imaging [3]. To this end, the registration approach was based on anatomical landmarks selected manually. This study showed that further improvements of the registration technique are required to be compatible with interventional workflow. To our knowledge, no image fusion approach solved the problem of multimodal (MRA/CTA and 2D fluoroscopy) registration with no contrast dye fluoroscopy. In this paper, we address the issue of 3D/2D registration for navigation guidance from aortic arch to carotids.

The 3D/2D registration methods can be classified as feature-based or intensity-based approaches [4]. Intensity-based approaches assume that the same anatomical structures of interest are visible in the both modalities. Feature-based approaches require segmentation of features in both images. The aortic arch or carotids can be segmented in both MRA and CTA and could serve to perform the registration. Nevertheless, the aortic arch is not injected during the intervention and is not visible on the fluoroscopy. Few works investigated the fusion of image between 3D pre-operative imaging and 2D fluoroscopy with no contrast dye. In [5], Ambrosini et al. proposed to segment the catheter on the fluoroscopy and register it with the 3D centerline of the segmented vascular structures from the MRA. This method, applied on arteries, relies on the hypothesis that small vessels heavily constrain the position of the catheter. In our case, the aortic arch does not constrain enough the catheter to be close to the centerline. Another issue is how to search for the optimal transformation. Usually, the search relies on an optimization process. These methods can converge to a local optimum instead of a global optimum. Another strategy is to perform an exhaustive search by sampling the parameter space of the transformation and retaining the transformation that provided the optimal result. Nevertheless, the trade-off between computation time and registration accuracy can be difficult to find. An exhaustive search approach, consisting in starting the process at a lower level of details and progressively refining the result by increasing the level of details of the images, is a possible solution to overcome this difficulty as used in [6].

In this work, we propose a 3D/2D multimodal catheter-based registration between 3D MRA or CTA and 2D live no contrast dye fluoroscopy for image fusion to support catheter navigation from the aortic arch to the carotids. The proposed method relies on a similarity measure exploiting the projection of the segmented aortic arch from the pre-operative imaging and the successive catheter positions from the fluoroscopy. The search is based on a multiresolution exhaustive search. This paper is structured as follows: Section “Method” provides a global view of the proposed approach then describes the different algorithmic components. Section “Results” presents the results on a physical phantom and on one patient. Section “Discussion” presents the discussion.

Method

This section presents the proposed registration method. It includes the definition of the similarity measure, the implementation of the geometrical setup and the search for optimal transformation.

Overview of the method

An overview of the proposed method is presented in Fig. 1. Aortic arch region is segmented from pre-operative imaging to define the moving image. The pseudo-digitally reconstructed radiography (pDRR) is computed as the conic projection of the moving image without X-ray attenuation. Catheter positions are segmented frame by frame from the fluoroscopy and are overlaid to define the fixed image. The registration process then relies on a multiresolution exhaustive search of the 3D/2D transformation. The final 3D/2D transformation is obtained at the maximum of the similarity measure comparing the fixed image with the pDRR.

Fig. 1
figure 1

Overview of the registration method

Full size image

Similarity measure

The similarity measure is used to compare the projection of the segmented 3D aortic arch with the combination of the successive segmented 2D catheters.

From the pre-operative imaging (MRA or CTA), aortic arch lumen is segmented to obtain the binary moving image \({I}_{\mathrm{moving}}\). The pDRR is computed given the fluoroscopic imaging device (C-arm) parameters. It consists in the 2D binary projection of \({I}_{\mathrm{moving}}\) with no X-ray attenuation.

Catheter is segmented on the fluoroscopic frames while it navigates in the aortic arch from time \(t={t}_{0}\) up to \(t={t}_{n}\) that leads to \(\left({t}_{n}-{t}_{0}\right)\times {f}_{\mathrm{Carm}}\) different segmented catheters images (with \({f}_{\mathrm{Carm}}\) the acquisition frequency of the C-arm) that are overlaid on a new image to define the fixed image \({I}_{\mathrm{fixed}}\). It is expressed as follows:

$$ I_{{{\text{fixed}}}} = \bigcup\limits_{k = 0}^{n} {S_{k} } $$

with \({S}_{k}\) the image of the segmented catheter from the fluoroscopic frame at the time \(t={t}_{k}\).

The similarity measure is applied between the pDRR and the binary image \({I}_{\mathrm{fixed}}\). It is defined as the square of normalized cross-correlation:

$$ C\left( {f,m} \right) = \frac{{ \langle f - \overline{f} ,m - \overline{m} \rangle^{2} }}{{\left| {f - \overline{f} } \right|^{2} \left| {m - \overline{m} } \right|^{2} }} $$

in which \(f\) and \(m\) are the vectors representing the image \({I}_{\mathrm{fixed}}\) and the pDRR, \(\overline{f }\) and \(\overline{m }\) are the mean value of f and m. \(\langle,\rangle\) denotes the inner product, and \(\mid \cdot \mid \) denotes the \({L}^{2}\) norm.

Geometrical setup and search for optimal transformation

The implementation of a 3D/2D registration requires the expression of the transformation allowing to bring the pre-operative data in the intra-operative reference frame. This transformation can be decomposed into three transformations: a first 3D transformation to represent the pose of the C-arm, a second to position the pre-operative image in space and a projective transformation (pDRR).

The pose of the C-arm is determined by the source–detector system. In order to correctly position the pre-operative volume in space, a transformation A is defined between the pre-operative coordinate system \({S}_{v}\) and the world coordinate system \({S}_{w}\). The transformation P describes the inherent projection of the fluoroscopic acquisition system. To express the 3D rigid transformation related to the pose of the C-arm, a coordinate system \({S}_{a}\) is defined so that the axis Oy is aligned with the main axis of the C-arm, and the plane Oxz is parallel to the plane of the 2D image. The transformation A is then expressed as follows:

$$ A = T_{{{\text{Vol}}\:c}} \cdot T_{xyz} \cdot R_{y} \cdot R_{z} \cdot R_{x} \cdot T_{{{\text{Vol}}\:c}}^{ - 1} $$

where \({T}_{\mathrm{Vol}} c\) is the translation defined by the position of the center of rotation of the volume, \({T}_{xyz}\) is the translation along the axes of \({S}_{a}\) and \({R}_{y}\), \({R}_{z}\) and \({R}_{x}\) are the rotations around the axes of \({S}_{a}\).

The determination of the optimal transformation is based on an exhaustive search. It consists in sampling the space of the parameters of the transformation, in generating a pDRR and compute a similarity measure for each combination of parameters and finally, keeping the parameter combination that provided the optimal similarity measure. This strategy has advantages in terms of robustness, notably by being less sensitive to the risk of non-convergence inherent to classical optimization strategies.

Using pDRR instead of DRR, it is still relevant to reduce the number of pDRRs computed. To this end, the transformation \(A\) is decomposed into two transformations to estimate the similarity measure of several successive transformations from a single pDRR. The idea is to exploit the properties of in-plane and out-plane as shown in the equations below. A 3D transformation A3D describes the out-of-plane rotations (parameters \({\omega }_{x}\) and \({\omega }_{z}\)) that are applied to the 3D moving image before generating the pDRR. A 2D transformation A2D is applied directly to the pDRR and describes the remaining degrees of freedom: a rotation (parameter \({\omega }_{y}\)), a scaling (parameter \({t}_{y}\)) and a translation (parameters \({t}_{x}\) and \({t}_{z}\)). At a given resolution level, the sampling of the parameter space performed by defining a discretized range of variation per parameter and creating a set of 6n parameters leading to n transformations such that:

$$ G^{i} = A_{2D}^{i} \circ P \circ A_{3D}^{i} ; i = 1, 2, \ldots ,n $$
$$ \left\{ {\begin{array}{*{20}l} {A_{3D}^{i} = A\left( {t_{x}^{{{\text{ini}}}} ,t_{y}^{{{\text{ini}}}} ,t_{z}^{{{\text{ini}}}} ,\omega_{x}^{i} ,\omega_{y}^{{{\text{ini}}}} ,\omega_{z}^{i} } \right)} \hfill \\ {A_{2D}^{i} = A\left( {t_{x}^{i} - t_{x}^{{{\text{ini}}}} ,t_{y}^{i} - t_{y}^{{{\text{ini}}}} ,t_{z}^{i} - t_{z}^{{{\text{ini}}}} ,0,\omega_{y}^{i} - \omega_{y}^{{{\text{ini}}}} ,0} \right)} \hfill \\ \end{array} } \right. $$

where \({t}_{x}^{i}\), \({t}_{y}^{i}\), \({t}_{z}^{i}\), \({\upomega }_{y}^{i}\), \({\upomega }_{x}^{i}\) and \({\upomega }_{z}^{i}\) are values from the ranges of variation of the parameters of G, and the notation \(\mathrm{ini}\) is used to indicate the initial parameters values. Transformations using the same \({A}_{3D}^{i}\) transformation only need one pDRR generation.

Out-plane coarse initialization exploits parameters of the C-arm and information on its location with respect to the patient: the source-to-image distance, the source-to-object distance, the craniocaudal angle and the right/left anterior angle. Therefore, the transformation representing the pose of the C-arm is included in A through initialization parameters. To study the feasibility of our method, the in-plane initialization is interactive. It consists in roughly identifying one feature point on the first generated pDRR and the corresponding point on the intra-operative image. The initial rough transformation \({A}^{\mathrm{ini}}\) is thus obtained by computing the 2D translation (\({t}_{x}\) and \({t}_{z}\)) between these points.

To fasten the computation, images are processed in a multiresolution scheme. Beginning with a resolution of 64 × 64, image resolution is doubled at each new level. The parameters of the multiresolution exhaustive search are presented in Table 1.

Table 1 Parameters setting for the multiresolution exhaustive search
Full size table

Evaluation method

Ground truth registration for patients is not accessible since aortic arch is not visible on the fluoroscopy. The choice has been made to evaluate the proposed method using a silicon physical phantom on which structures are visible on fluoroscopy. From a CT acquisition of the phantom, 3D volume of lumen is segmented using level set segmentation to define the moving image \({I}_{\mathrm{moving}}\) containing the ascending aorta, the aortic arch and approximatively two-thirds of the descending aorta. The endovascular navigation has been performed in the phantom vessels by an interventional neuroradiologist at the Rennes University Hospital (Rennes, France). Two diagnostic catheters, in combination with the Neuron MAX 6F delivery catheter and the Terumo 0.35 guidewire, have been used: the JB2 and the Simmons. The phantom has been placed on the patient table, and the navigation has been guided by fluoroscopy on which the catheter was seen. The navigation has been performed 3 times from the beginning of the aortic arch up to the right carotid for each catheter that resulted in six recorded navigations.

The segmentation of the catheter on the fluoroscopy was performed with edge filtering using MeVisLab. Examples of the resulting segmentation are shown in Fig. 2 that also shows the resulting fixed image from the superposition of segmented catheter images between \({t}_{0}\) and \({t}_{n}\). The time \({t}_{0}\) is choosen such that the catheter is inserted with the tip at the frontier between the ascending aorta and the aortic arch, and the time tn is chosen such that the catheter reaches the carotid at the time \({t}_{n}\)+1 s.

Fig. 2
figure 2

Overview of the computation of the fixed image from the physical phantom fluoroscopy. Catheter positions are segmented on successive frames of the fluoroscopy, the fixed image is formed by the overlay of these binary segmented images

Full size image

To evaluate registration accuracy of the proposed method, comparison is made between vascular structures visible in fluoroscopic images and projected structures. To compare them, the projected structure should represent the walls of the phantom instead of the lumen. For this reason, another segmentation \({I}_{\mathrm{walls}(3D)}\) of the walls of the phantom is computed from the CT for evaluation. Once registration is performed using \({I}_{\mathrm{moving}}\), registering parameters are used to project \({I}_{\mathrm{walls}(3D)}\) for evaluation. Aortic arch phantom is segmented on the fluoroscopy using MeVisLab to define \({I}_{\mathrm{walls}(2D)}\). The method is evaluated through computation of dice score and computation of mean landmark error (MLE). Dice score is computed between projected \({I}_{\mathrm{walls}(3D)}\) and \({I}_{\mathrm{walls}(2D)}\). Fifteen landmarks are defined to evaluate the registration accuracy for types I, II and III aortic arch. We make the assumption that considered successive catheter positions within aorta do not depend of the aortic arch type. From the original type I aortic arch phantom segmentation, supra-aortic vessels are removed. Points TI, TII and TIII are defined to represent different origins of the brachiocephalic artery corresponding to aortic arch of types I, II and III, respectively, as shown in Fig. 3. For each point TN (with N = I, II and III), four other points are derived: TN−2, TN−1, TN1 and TN2. They are defined such that TN−2 and TN−1 are 0.3 and 0.15 times the diameter of common carotid artery (CCA) lower than TN.. TN1 and TN2 are defined such that they are 0.15 and 0.3 times the diameter of CCA higher than TN. A reference transformation has been semi-automatically obtained by considering the fluoroscopic image on which the phantom is visible. MLE is computed between 2D projection of the fifteen landmarks obtained using the reference transformation and using the registration method.

Fig. 3
figure 3

Definition of landmarks TI, TII and TIII

Full size image

Results

The proposed method has been evaluated on a physical phantom with CT as pre-operative imaging and fluoroscopy as intra-operative imaging and has been retrospectively applied on one patient who had a mechanical thrombectomy as treatment for ischemic stroke with an MRA and fluoroscopy as pre-operative and intra-operative imaging.

Phantom evaluation

Registration performance is investigated upon three criteria: parameters settings, aortic arch type and 2D fluoroscopy sequence duration. Registration accuracy is evaluated through dice score and MLE.

Parameters setting

For evaluation, the same in-plane translation parameters (\({t}_{x}=-118 \mathrm{mm}\) and \({t}_{y}=-104 \mathrm{mm}\)) were used to initialize every registration. These parameters were chosen to initialize the algorithm with a voluntarily imprecise transformation that is supposed to be worse than manually initialized transformation. Given a pair of fixed and moving images, it leaded to a dice score of 0.36 between the fixed image and the pDRR. For comparison, on the same pair of images, the same operator initialized roughly 10 manual transformations. They leaded to a mean dice score of 0.53 with a standard deviation of 0.01.

The registration results using the search parameters setting (Table 1) were compared to the results using a reference setting reported in Table 2. Starting from an accurate initialization, the reference setting has high sampling rates in order to get as close as possible as the ground truth registration. The registration performed using the fluoroscopic image of the first navigation with the Simmons catheter leads to a final similarity measure of 0.459 while the registration with the proposed setting leads to a final similarity measure of 0.454 being 10 times faster. This highlights the good balance between registration accuracy and registration time of the proposed parameters setting. An example of registration using this setting is given in Fig. 4.

Table 2 Parameters of the reference setting
Full size table
Fig. 4
figure 4

Registration result for the first navigation (Simmons catheter). Initialization showing pDRR (in black) superposed on the fixed image (in white) (a). Projection, from registration, of the considered moving image on the fixed image (b). Projection, from registration, of the whole 3D phantom walls on the fluoroscopy (c)

Full size image

Aortic arch type

For each of the six recorded navigations, registration is performed and evaluated through dice score and MLE. The influence of the aortic arch type on the registration result, reported in Table 3, uses the three different sets of points defined in 2.4.

Table 3 Results of registration, using MLE (mean and std) and dice score, regarding three different sets of points (TI, TII and TIII)
Full size table

For every registration, the dice score is higher than 93%. The evaluation through MLE gives an error in mm on the 2D projection plan. The carotid diameter of 8–10 mm and the inter-carotid distance of 8–10 mm are doubled when projected onto the 2D plane (for these specific C-arm parameters) leading to a distance of 16–20 mm. We consider the registration successful since the MLE is smaller than the quarter of the carotid diameter (\(\mathrm{MLE}<4\mathrm{mm}\)). Registration is considered accurate for the different types of aortic arch.

Two-dimensional fluoroscopy sequence duration

The influence of the duration of fluoroscopic navigation sequence to compute the fixed image on the registration is explored using three different durations \({d}_{\mathrm{eval}}^{k}\) (with k = 1, 2 and 3) such that the fixed image is issued from the superposition of catheter segmentations on the fluoroscopy from time \(t={t}_{0}\) up to \(t={t}_{n}^{k}\). For every navigation, duration of catheter navigation in the fluoroscopic images to compute the fixed image is reported in Table 4. For one of the navigations, the three different computed fixed images given the three different durations \({d}_{\mathrm{eval}}^{k}\) (with k = 1, 2 and 3) are shown in Fig. 5.

Table 4 Durations of catheter navigation in the fluoroscopic images to compute the fixed images. The durations \(d_{{{\text{eval}}}}^{1}\) are defined by \(t_{n} - t_{0}\). The durations \(d_{{{\text{eval}}}}^{2}\) and \(d_{{{\text{eval}}}}^{3}\) are defined by \(d_{{{\text{eval}}}}^{2} = t_{n}^{2} {-}t_{0}\) and \(d_{{{\text{eval}}}}^{3} = t_{n}^{3} {-}t_{0}\) with \(t_{n}^{2}\) and \(t_{n}^{3}\) such that \(d_{{{\text{eval}}}}^{2} = \frac{2}{3}d_{{{\text{eval}}}}^{1}\) and \(d_{{{\text{eval}}}}^{3} = \frac{1}{3}d_{{{\text{eval}}}}^{1}\)
Full size table
Fig. 5
figure 5

Fixed images computed from fluoroscopic images with three durations for the first navigation (Simmons). Fixed images with long a, medium b and short c duration

Full size image

The registration was computed for each of the six recordings using the three associated fixed images resulting in a total of eighteen registrations. Registration results are reported in Table 5. Durations are between 2 and 22 s, and MLE (mean and std computed on the 15 points defined in 2.4) is still lower than 4 mm. These results show that only few seconds of navigation are required for the registration to succeed.

Table 5 Results of registration, using MLE (mean and std) and dice score, obtained from the six recordings using durations \({d}_{\mathrm{eval}}^{k}\) with k = 1, 2 and 3 to compute the fixed image
Full size table

Patient evaluation

Data of one patient, with a type II aortic arch, who underwent a mechanical thrombectomy as a treatment for ischemic stroke (right proximal M1 occlusion) at the University Hospital of Rennes were retrospectively collected. The data consist of a pre-operative MRA and an intra-operative fluoroscopy. Navigation has been performed using the select SIM diagnostic catheter, the AXS Infinity delivery catheter and the Terumo 0.35 guidewire.

Segmentation of vascular structures in pre-operative imaging as well as segmentation of catheter positions in intra-operative imaging have been done semi-automatically using MeVisLab. The qualitative results are presented in Fig. 6. Also, catheter inclusion into the projected volume on the total fluoroscopic sequence, including images of the insertion of the catheter into the carotid, has been computed and is equal to 100%.

Fig. 6
figure 6

Registration result on a real patient using the proposed method. Projection on two different frames

Full size image

Discussion

Image fusion is a crucial step to support endovascular navigation. It requires registration between pre-operative and intra-operative imaging. To this end, we presented a new registration method. The originality of the proposed approach relies on the consideration of fluoroscopy without contrast dye as intra-operative imaging. Successive catheter positions on the fluoroscopy are used to compute the fixed image. The transformation research is done thanks to a multiresolution exhaustive search to have a computational complexity compatible with clinical application.

The proposed method has been evaluated on a physical phantom of an aortic arch and carotids. Algorithm behavior has been evaluated through parameters settings, aortic arch type and 2D fluoroscopy sequence duration. Results shown that the method is compatible with clinical routine practice in terms of computational complexity and in terms of fluoroscopic sequence duration to perform the registration.

Although the results obtained are satisfactory, the proposed method presents several assumptions and limitations. The phantom does not reproduce cardio-respiratory motion or motion resulting from the movement of the head. The motion due to the head movement should not be problematic as a head positioner is used most of the time and is easy to managed. The deformation due to cardio-respiratory motions has been quantified [7, 8], and mean left-to-right translation is smaller than 1.3 mm, mean posterior-to-anterior translation is smaller than 3.9 mm and mean inferior-to-superior translation is smaller than 2.8 mm. Given the inter-carotid distance and the diameter of carotids of 8–10 mm, the algorithm should perform well on patients.

Evaluation on the phantom gives good results. The next step would be to perform further evaluation on patients, first on a larger set of retrospective data then directly during the intervention. For exploitation during the intervention, the segmentation of catheters on the fluoroscopy should be automatized. It would be feasible using deep learning approaches [9]. Initialization could be automatized [10] as well as the segmentation of pre-operative imaging [11, 12].

Finally, the way to project the structure to guide the navigation should be investigated. We worked on the registration method but the projection of structures on the fluoroscopy for guidance has still to be considered. It should be investigated how to project segmented vascular structures (surface, DRR, etc.). Existing solutions of endovascular guidance for similar yet different applications [13, 14] may help to decide what is best.

Conclusion

Mechanical thrombectomy is an effective approach to treat ischemic stroke despite endovascular navigation difficulties. Today, no image fusion guidance through automatic registration is available to support physician for the navigation from the aortic arch up to the carotids. In this work, we proposed a 3D/2D registration method for matching a pre-operative MRA or CTA with intra-operative fluoroscopy without contrast dye. First evaluation on a physical phantom demonstrated the feasibility and accuracy of the method.