4D Cardiac Motion Modeling Using Pair-Wise Mesh Registration

Abstract

In this paper, we present a novel method for the real-time cardiac motion compensation. Our method generates interpolated cardiac motion using segmented mesh models from preoperative 3D+T computed tomography angiography (CTA). We propose a pair-wise mesh registration technique for building correspondence and interpolating the control points over a cardiac cycle. The key contribution of this work is a rapid creation of a deformation field through a concise mathematical formulation while maintaining desired properties. These are $C^2$ continuity, invertibility, incompressibility of cardiac structure and capability to handling large deformation. And we evaluated the proposed method using different conditions, such as deformation resolution, temporal sampling rates, and template model selection.

This work was supported by Industrial Strategic Technology Development Programs (10052980, 10077502) funded By the Korea Ministry of Trade, Industry and Energy (MI, Korea).

1 Introduction

Cardiovascular diseases (CVDs) are a leading cause of death in developed countries [3]. Not only CVDs is one of the most mortality but also it is reported as a major cost burden on health-care system in North America [14]. Health-care cost of CVDs is continuously growing as developed countries become an aging society [14]. In practical treatment of CVDs, cardiovascular imaging systems and techniques became a gold standard for diagnosis and functional analysis [4, 12]. Analyzing a cardiac motion over cycle, it helps to reduce morbidity and mortality induced from cardiovascular disease [11].

A number of studies for cardiac motion have been conducted using spatiotemporal images, such that ECG-gated CT angiography [8] and tagged MRI [7]. In order to estimate a cardiac motions, registration method is crucial to building a correspondence between anatomic land-marks and to computing a deformation fields between different cardiac phases [6, 9, 13]. Although many of authors have studied motion modeling of using registration methods, these are still suffered from the heavy computation of displacement field and lack of correspondence between scenes. Also, a large amount of data is required for statistical population-based approach to cover a variability of shapes.

In this study, we propose a novel 4D cardiac motion modeling method using cage deformation scheme. For efficient computation, we formulate a representation of an object using control points and constant weight values. Thus deformation is only governed by the position of control points. In order to estimate a cardiac motion, shape interpolation method is addressed with pair-wise mesh registration. Proposed motion modeling method resolves lack of correspondence and heavy calculation of displacement field, therefore, it is suitable for real-time use.

2 Related Work

We categorize the studies of cardiac motion analysis methods into their (1) motion modeling method and (2) applications.

Motion Modeling Methods. In cardiac motion modeling, the one of the desired property is spatiotemporal continuity. In the aspect of temporal continuity, group-wise registration method defines a cost function accumulatively or globally. This approaches register multiple objects simultaneously, therefore it improves consistency and continuity between scenes [9, 13, 15]. In spatially, the spatial transformation model governs the way of regional displacement between scenes. Basis functions and parametric models use a basis function or parametric geometries, such as splines, curves, and surfaces, to represent the regional movement of the anatomical object, smoothly. Also, data-driven methods use a plenty of data to model a deformation of the anatomical object [1]. The variability of shape can be analyzed by a statistical method.

Applications. The main application of cardiac motion modeling is image guided intervention. Recently, Yabe et al. report the motion compensation and real-time visualization of coronary arteries significantly reduce the contrast volume and fluoroscopic time while percutaneous coronary intervention [17]. In ablation therapy, Wilson et al. studied an electrophysiological (EP) mapping method that is essential for arrhythmia treatment [16], they introduced the patient-specific 4D cardiac mapping method to improve EP mapping under the dynamic situation. In the perspective of diagnosis and monitoring, Gilbert et al. proposed a rapid quantification method for biventricular function using standard MRI [5].

Creating Motion Model for Complex Structure. Tiny and complex structures of the heart, such as coronary arteries, are very challenging to model dynamically. Due to the motion artifacts and topological/geometrical variations, Baka et al. shifted the problem of motion modeling from coronary arteries to the attached muscle [1]. Representation of lumen changes is limited because they focus on the centerline of the coronary artery.

In this context, we propose a novel 4D motion modeling method for either convex/non-convex structures using pair-wise registration and shape interpolation. Our contribution is to simplify a shape deformation, therefore it is possible to real-time usage.

3 Method

Proposed 4D shape modeling method mainly consists of pair-wise registration and shape blending method. In order to build the correspondence between sampled cardiac phases from 4D CT, pair-wise registration is conducted between segmented meshes from 4D CT. After pair-wise registration, control points governing the shape deformation are interpolated for the whole cardiac phase (Fig. 1). The result of interpolation creates a 4D shape model.

Fig. 1.

Generating a 4D motion model for the cardiac structure periodically changing according to the motion of the heart (left). Transforming the source object into target object by optimizing the cost function (right).

3.1 Shape Representation for Cardiac Structures

The shape of the cardiac structure is obtained by segmentation from volumetric images, such as CT, 3D ultrasound, and MRI. In this paper, we use a mesh representation for shape in order to model a specific cardiac structure, such as a coronary artery, atrium, or ventricle. A shape V is defined as a mesh that consists of vertices, edges and faces. Therefore, $V=\{v_1,\ldots {},v_{I_{V}}\}$ and $E=V\times {}V$, where $v_k\subset R^d$, $I_{V}$ is number of vertices, and d is a embedding dimension of an object. Let us consider a sequence of objects from spatiotemporal image, such as 4D CT, then a sequence is $V_s=\{V_1,\ldots {},V_n\}$ where $V_i$ is a shape.

3.2 Landmark Base Registration

The purpose of registration is to find the best alignment from source to target, source is referred as moving mesh by transformation and target is destination. Now, let mesh $V_s \in V$ be a source, and mesh $V_t\in V$ be a target, respectively. The goodness of alignment is measured by cost function. Cost function C is defined as

$$\begin{aligned} C:=\sum d(T(x)\circ V_s),V_t) \end{aligned}$$

(1)

where d is a disparity measure between meshes $V_s$ and $V_t$. For simplicity, we use the Euclidean distance. T is a transformation model, and x is a transformation parameter. The best transformation parameter $x^*$ is optimized by minimizing cost function,

$$\begin{aligned} x^* =\arg \min _x C=\arg \min _x \sum d(T(x)\circ V_s,V_t) \end{aligned}$$

(2)

3.3 Shape Deformation

In order to formulate a shape deformation, we assign a displacement vector, $u_i \in R^d$, to each of vertices. Displacements of vertices represent a shape deformation as

$$\begin{aligned} T:V \rightarrow V+U(x) \end{aligned}$$

(3)

where U(x) is movements of vertices about transformation parameter x. Shape deformation U(x) can be decomposed by affine transformation $A\in {} R^{d\times d}$ and non-linear deformation u(x),

$$\begin{aligned} T:V \rightarrow V+U(x)=A(x_{a})\circ V+u(x) \end{aligned}$$

(4)

where $x_{a}$ is affine parameter and $u(x)=(V-A(x_a) \circ V)+U(x)$ is a local deformation after global affine transformation.

3.4 Finite Element Mapping for Deformation Modeling

In order to represent the object V using control points, let domain $\varOmega$ be $\varOmega =[min(V),max(V)]$, where $\varOmega$ defines a bounding box of object V. $\varOmega$ is partitioned by $m^d$ number of evenly spaced grid and ${(m+1)}^d$ control points.

A cage P consist of control points $V_P=\{p_i\}_{i\in I_P }\subset R^d$, where $I_P$ is number of control points and $p_i$ is a control point. If a point $v \in R^d$ is inside cage P, such that $v \subset \varOmega _P=[min(P),max(P)]$, then given vertex $v \in R^d$ can be represented by control points as below,

$$\begin{aligned} v=F(v;P)=\sum _{k\in I_P} \varphi _k (v) p_k \end{aligned}$$

(5)

where $F(\cdot ,\cdot )$ is a finite element mapping and $\varphi _k (\cdot )$ is a weighting function with respect to given vertex v. For example, a 3D weighting functions are $\varphi _1 (x,y,z)=\frac{(1-x)(1-y)(1-z)}{8}$, ..., and $\varphi _8 (x,y,z)=\frac{(1-x)(1+y)(1+z)}{8}$. After weighting functions are determined, the movement of vertex v only depends on control points,

$$\begin{aligned} v'= F(v;P')=\sum _{k\in I_P} \varphi _k (v) p_k' \end{aligned}$$

(6)

where $P'$ represents the changed cage P, such that $V_{P'}=\{p'_k\}_{k\in I_P }\subset R^d$. The moved vertex $v'$ in (6) can be rewritten using (4) as below,

$$\begin{aligned} v'=A(x_a) \circ v+u(x) =\sum _{k\in I_P} \varphi _k (v) (A(x_a) \circ p_k+u_k(x)) \end{aligned}$$

(7)

3.5 Optimization

As we mentioned in Sect. 3.2, registration problem is rewritten as optimization of cost function. We define dissimilarity measurement, as below

$$\begin{aligned} d(V_s,V_t,x)={\sum _{i\in I_{V_s}}\min _{j\in I_{V_t}} |T(x)\circ v_{i}-v_{j}|^2} \end{aligned}$$

(8)

where $v_i \in V_s$ and $v_j \in V_t$.

The regularization supports that the deformation became a diffeomorphism, therefore deformation is invertible and smooth. We are inspired from [2]. The hyper-elasticity regularization is easily extended to proposed deformation setting. The hyper-elasticity regularization is defined as below

$$\begin{aligned} S^{hyper} (x)=\int \alpha _1 \varphi _{vol} (x)+ \alpha _2 \varphi _{sur} (x)+ \alpha _3 \varphi _{len} (x) d\varOmega \end{aligned}$$

(9)

where $\alpha _i$ are balancing parameter and $\varphi _{vol}$, $\varphi _{sur}$, $\varphi _{len}$ are cost functions that penalizing a change of volume, area, length. Control points of cage P act like corner points of hexahedron of [2]. By introducing the hyper-elasticity regularization, cost function has an infinity value for non-diffeomorphic transformation.

The cost function is combination of dissimilarity measurement and regularization as below,

$$\begin{aligned} C(V_s,V_t,x)= d(V_s,V_t,x)+S^{hyper}(x) \end{aligned}$$

(10)

We decouple optimization process into two steps, (1) finding global affine transform ($x^*_a$) and (2) finding non-linear transformation ($x^*$) with regularization. In step (1), we neglect regularization energy. Updating control points is simply iterative closest point (ICP). In step (2), we use Levenberg-Marquardt (LM) method to optimize a non-linear least square problem. In order to use gradient decent optimization, the derivative of dissimilarity measurement is given as below,

$$\begin{aligned} \frac{\partial }{\partial x} d(V_s,V_t,x)= 2 \sum _{i\in I_{V_s}} \sum _{k\in I_{P}} {\varphi _k(v_i)(v^T_i-v^T_j)} \end{aligned}$$

(11)

where $v_i \in V_s$ and $v_j \in V_t$.

3.6 Interpolation of Shapes

Let a vector $P_k=\{p_{k1},p_{k2},\ldots {},p_{k{((m+1)^3-1)}},p_{k(m+1)^3}\}$ be set of control points at k-th object, where $p_i\in R^{d}$, d is embedding dimension. By registering a template shape to the k-th sequence, the control points are updated $P_{(k)}=A_k P_k+U_k$, where $A_k$ is an affine transformation and displacement of control points is $U_k=\{\Delta p_{k1},\Delta p_{k2},\ldots {}, \Delta p_{k{((m+1)^3-1)}}, \Delta p_{k(m+1)^3} \}$.

From sets of control points, $S(t)=\{s_1 (t), \ldots {}, s_{(m+1)^3} (t)\}$ is interpolated, where $s_i (t)$ is a spline function of corresponding control points. In our implementation, closed cubic spline is used. Therefore, spline vector S(t) has $C^2$ continuity with respect to t. A vector function S(t) is a interpolated control point of time, which maps $S:R\rightarrow R^{(m+1)^3\times d}$. Then shape is function of time as below

$$\begin{aligned} v(t)=F(v;S(t))=\sum _{i\in I_V}{\phi _i (v)s_i (t)} \end{aligned}$$

(12)

4 Evaluation and Discussion

In order to evaluate the performance of 4D modeling, we used anonymous patient data. A patient data was evenly sampled from 4D CT within R-R peak (5% sampling interval), therefore, 20 volumes ($M_{0\%}$, $M_{5\%}$, ..., $M_{95\%}$) were available. The coronary artery models were manually segmented from CT volumes using ITK-Snap [18]. Through the evaluation, we observed (1) the effect of cage partitioning, (2) the effect of the phase sampling, and (3) the effect of the template model. We define the discrepancy of modeling by an average Euclidean distance of bijective closest point pairs as below

$$\begin{aligned} d=\frac{1}{I_{Bi}}{\sum _{i\in I_{V_s}}\min _{j\in I_{V_t}} |T(x)\circ v_{i}-v_{j}|^2} \end{aligned}$$

(13)

where $I_{Bi}$ is the number of bijective pairs.

4.1 Effect of Cage Partitioning

In this section, we observed the effect of cage partitioning. Cage partitioning governs the degree of freedom for deformation. The spatial partitioning is defined by the number of cage control points. For example, let a region $\varOmega _S$ be the bounding box of a source model $M_{source}$, then a cage partitioning produces the cages that $\varOmega _S=\varOmega _{\{0,0,0\}}\cup \varOmega _{\{1,0,0\}}\cup \dots \cup \varOmega _{\{i,j,k\}}$, where i, j, k are the number grid partitioning in 3D. We compared different partitioning resolutions [i, j, k] as 4 cases that are [2, 2, 2], [4, 4, 4], [8, 8, 8] and [12, 12, 12]. We measured the discrepancy between the registered mesh model $M_{i\%}'$ and manually segmented model $M_{i\%}$. As Fig. 2, the resolution of cage partitioning affect the accuracy of registration, the higher resolution provide the lower error. Time consumption for creating 4D model was 15 min for [2, 2, 2], [4, 4, 4], 20 min for [8, 8, 8], and 30 min [12, 12, 12] using Intel i7-7700K CPU.

Fig. 2.

Comparison of different partitioning resolutions [i, j, k] as 4 cases that are [2, 2, 2], [4, 4, 4], [8, 8, 8] and [12, 12, 12].

4.2 Effect of the Phase Sampling

In order to observe the effect of sampling, the proposed 4D model was compared to manually segmented model under different sampling conditions. Our data contain 20 volumes per each patient, we were able to select some volumes to create the 4D model. For example, if a sampling step is 10%, then $M_{0\%}$, $M_{10\%}$, ..., $M_{90\%}$ are selected. We selected the 75% phase as the template model. The missed models are generated from 4D model. For example, if a sampling step is 10%, then missed models are $M_{5\%}'$, $M_{15\%}'$, ..., $M_{95\%}'$). We calculated the discrepancy between the interpolated model $M_{i\%}'$ and manually segmented model $M_{i\%}$. As Fig. 3, the temporal sampling largely influences the accuracy of 4D modeling, the higher temporal sampling rate provides the lower error.

Fig. 3.

The effect of sampling was observed by comparing manually segmented models and interpolated models under different conditions (5 samples $\{15,35,55,75,95\}$, 10 samples $\{5,15,25,35,45,55,65,75,85,95\}$ and 20 samples). We use [8, 8, 8] partitioning and 75% phase as template

4.3 Effect of Template Model

Because of the very complex movement of the cardiac structure, quality of the cardiac image depends on the selection of phase [10]. This variation of image quality affects the visibility of cardiac structures, therefore, segmentation results are non-consist. We observed the select of template model section, we selected the 35% and 75% phases as the template models, that are known as the least movement phase [10]. Although the template models are different, trend of discrepancy is similar (Figs. 4 and 5).

Fig. 4.

The discrepancy between the registered mesh models and manually segmented mesh models under different template model selection (35% and 75%). We use [8, 8, 8] partitioning.

Fig. 5.

The colored coronary arteries and cages show the amount of displacement from the template model (75% cardiac phase) (Color figure online)

5 Conclusion

In this study, we present a pair-wise registration and shape interpolation method for 4D motion modeling. In order to formulate the shape deformation concisely, we use a finite element mapping. A shape is represented by cages that consist of control points and their weighting constants. Therefore, the deformation only depends on the change of control point. Also, the deformation is decomposed by global affine transform and local non-linear deformation to cover large deformation.

However, local non-linear deformation can’t ensure to be diffeomorphic deformation. By using hyperelastic regularization of [2], the non-diffeomorphic deformation is penalized. Thus, our deformation model is able to cover large deformation while preserving diffeomorphism. One of desired property of cardiac modeling is the incompressibility, this is provided by regularization term and decomposition of motion. The correspondence of control points is given by pair-wise registration between different objects, then the corresponding control points are interpolated by spline function over the time domain. The spline interpolation of control points is smooth and invertible, because of the inherited property from spline function.

Despite we use a regularization term for the method, this regularization term doesn’t ensure that the volume preservation while interpolating shapes. The motion of the right coronary artery is very dynamic within the cardiac interval of 0% to 20%, therefore, it is hard to accurately segment. These non-consist segmentation results produce the large error at that interval.

Combination of pair-wise registration and control points interpolation reconstructs the cardiac model at any time point. The proposed method enables to simplify the shape deformation by introducing finite element mapping. Although [12, 12, 12] partitioning spend 30 min to create 4D motion modeling, it is able to real-time rendering (60 fps) during the cardiac intervention. Because shape deformation is simply represented by control points that change over time.