Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Registration of brain volumes with tumors is important to track the changes between two instances in order to assess the progression of the tumor and the treatment response. The first step is a rigid/affine registration between volumes. Although this is a challenging problem due to the changes caused by the tumor, various approaches on the problem reported successful results in the literature [4, 13, 15]. The total deformation caused by the tumor growth can be taught as the combination of infiltration to the healthy tissue and mass effect components. Our aim in this work is to separate the mass effect and infiltration components, so that malignancy and the reversibility of the destruction can be determined. The healthy brain tissue in one of the images can be warped onto the other ignoring the tumor tissue regions, as the latter may contain uncertainty due to highly complex tumor growth and therapy processes. Hence, matching only the healthy tissues in baseline and follow-up tumor images provides an estimation of the intracranial pressure caused by the tumor growth plus the mass effect.

A similar problem arises in deformable registration of the brain with tumors to a healthy population atlas. The main difference to the intra-subject registration problem is that the deformation also includes inter-subject variations. Hence, a general strategy to solve this problem is to iterate the forward model by simulating the tumor growth on the atlas and refining the parameters of the simulation model by comparing it to the tumor image [6]. This requires strong models, which rely on realistic models of tumor growth and deformations due to the mass effect [2, 6, 8, 14]. The main problem with those approaches is that the growth of the tumor is mostly affected by the uncontrolled parameters such as treatment and requires sophisticated tumor growth models even without treatment. In “Geometric Metamorphosis” paper, Niethammer et al., proposed an interesting approach to the problem using a weak model by separating the foreground, hence the tumor growth, and the background changes [11].

Fig. 1
figure 1

A sample axial slice from baseline (on the right) and follow-up (on the left) MRI

Full size image

For the problem of intra-subject registration, our approach is based on the assumption that the deformations except around the tumor volume are caused by the mass effect of the tumor, hence obey the bio-mechanical rules. This is different from the atlas matching problem, since there is also inter-subject variations between the images. Sample baseline and follow-up MRI slices are shown in Figs. 1 and 2. Deformation of the ventricles and sulci due to the mass effect of the growing tumor can be clearly observed on the right hemisphere. Although a mapping of the tumor tissue between the baseline and the follow-up is not well defined due to the uncertain growth pattern and therapy effects, a mapping between the healthy tissues can be estimated. Therefore, our aim is to find a mapping between the healthy tissues of the brains, which obeys the nonlinear elastic finite deformation models.

Fig. 2
figure 2

A sample coronal slice from baseline (on the right) and follow-up (on the left) MRI

Full size image

The results of the experimental studies on animals suggest to model the brain with a homogeneous hyper-viscoelastic non-isotropic material [9]. In the image analysis literature, simplified hyper-elastic or linear models are used: Neo-Hookean [7], Ogden type [10, 16], Saint-Venant Kirchhoff model [18], linear elastic [3], linear viscoelastic type [2]. In their work, comparing viscoelastic, hyper-elastic and linear elastic models on brain simulations, Wittek et al. report no significant difference on the results obtained [17]. In “Nonlinear Elastic Registration with Unbiased Regularization in Three Dimensions”, Saint-Venant Kirchhoff model was used as a regularizer in the registration of serial magnetic resonance images [18].

Our novel contributions in this paper are: (a) Matching healthy tissue to healthy tissue of the brain using a dedicated image data term; (b) Using the hyper-elastic Neo-Hookean strain energy density as a regularizer in deformable registration framework; (c) Derivation of displacement field update equations based on the Neo-Hookean model.

2 Methods

2.1 Background

The mechanical properties of a hyper-elastic material are characterized completely by a scalar strain-energy density function W. Specifying the strain energy density W as a function of deformation gradient tensor F: W = W(F) ensures that the material is perfectly elastic. The general form of the strain energy density is guided by experiment [1].

Let us define the displacement field from the un-deformed to deformed configuration as \(u : \Omega \rightarrow {\mathbb{R}}^{3}\) where \(\Omega \in {\mathbb{R}}^{3}\). In addition to the strain energy density W = W(x, ∇ u), let f = f(x, u(x)) denote the external energy, then the equilibrium configurations can be determined as the minima of the total energy functional:

$$\displaystyle{ I(u) =\int _{\Omega }W(x,\nabla u)\mathrm{d}x -\int _{\Omega }f(x,u)\mathrm{d}x }$$
(1)

The Euler–Lagrange system associated with the functional I is precisely the equilibrium equations:

$$\displaystyle{ \mathrm{div}\left (\frac{\partial W} {\partial F} (x,\nabla u(x))\right ) + \frac{\partial f} {\partial u}(x,u(x)) = 0 }$$
(2)

where \(F_{ij} =\sigma _{ij} + \frac{\partial u_{i}} {\partial x_{j}}\) and \({\frac{\partial W} {\partial F} }^{T}\) is called the First Piola–Kirchhoff tensor. Common constitutive hyper-elastic strain energy density models include [12]:

  • St Venant–Kirchhoff material:

    $$\displaystyle{ W(F) =\alpha {(trE)}^{2} +\beta tr({E}^{2})\ \ \mathrm{where}\ \ E = \frac{1} {2}({F}^{T}F - I) }$$
    (3)

  • Neo-Hookean material:

    $$\displaystyle{ W(F) = a\|{F\|}^{2} + g(detF)\ \ \mathrm{where}\ \ a > 0 }$$
    (4)

2.2 Hyper-Elastic Registration for Tumor Follow-Up

Let R(x) and T(x) denote the brain tissue maps (White Matter + Gray Matter + Tumor) of reference (undeformed) and target (deformed) volumes, respectively. We are assuming the mechanical properties of white matter and gray matter are similar. The problem is to find the displacement u from the volume R to T, which minimizes the given functional:

$$\displaystyle{ {u}^{{\ast}} = \mathrm{arg\,min}_{ u}\int _{\Omega }f(R(x),T(x - u(x))) +\alpha W(x,\nabla u)\mathrm{d}x }$$
(5)

where f is the external energy density term calculated as a similarity measure between the two volumes and α is the weighting parameter of the regularizer term W.

Distance maps to the binary segmentations are used, instead of common image similarity measures, such as mutual information. In this way, state-of-the art segmentation methods specific to the problem could be used to obtain reliable image forces. Reliability of the image term is especially important for the proposed regularizer dominant approach.

2.3 Volumetric Data Term

Since a mapping between the two solid body is needed, all the displacements from the reference should point towards inside of the target body. This is ensured by the following external energy density term:

$$\displaystyle{ f_{V }(x,u(x)) =\chi _{R}(x)D_{T}(x + u(x)) }$$
(6)

where χ R is the indicator function of the reference body, with the value of 1 for the points inside and 0 for the outside. D T is the distance function to the target body, where D T is zero inside the target body and takes the distance values outside the target body. The goal is that the displacement vectors can move freely inside the target volume. To derive the Euler–Lagrange condition for this energy density, the derivative of the functional in Eq. (6) is written as:

$$\displaystyle{ \frac{\partial } {\partial u}f_{V }(x,u(x)) =\chi _{R}(x) \frac{\partial } {\partial u}D_{T}(x + u(x)) }$$
(7)

This is simply the gradient of the distance function of the target body:

$$\displaystyle{ \frac{\partial } {\partial u}f_{V }(x,u(x)) =\chi _{R}(x)\nabla D_{T}(x + u(x)) }$$
(8)

2.4 Boundary Data Term

In addition, a solution that matches the outside surfaces of the two bodies is required. Therefore, another external energy density is added which penalizes the distance from the surface of the reference body to the surface of the target. We note that displacement vectors can still move on the target surface freely. This is ensured by the following energy term:

$$\displaystyle{ f_{B}(x,u(x)) =\chi _{\partial R}(x)D_{\partial T}(x + u(x)) }$$
(9)

where χ ∂R is the indicator function of the boundary of the reference body, having the value of 1 for the points on the boundary and 0 elsewhere. D ∂T is the distance function to the boundary of the target body. The Euler–Lagrange condition for this energy density is:

$$\displaystyle{ \frac{\partial } {\partial u}f_{B}(x,u(x)) =\chi _{\partial R}(x) \frac{\partial } {\partial u}D_{\partial T}(x + u(x)) =\chi _{\partial R}(x)\nabla D_{\partial T}(x + u(x)) }$$
(10)

2.5 Hyper-Elastic Regularizer

For simplicity, assuming nonlinear hyper-elastic model in Ogden form as in [10]:

$$\displaystyle{ W ={ \frac{2\mu } {\alpha }^{2}} (\bar{\lambda {_{1}}}^{\alpha } +\bar{\lambda { _{ 2}}}^{\alpha } +\bar{\lambda { _{ 3}}}^{\alpha } - 3) + \frac{1} {D_{1}}{(J - 1)}^{2} }$$
(11)

where principal strains \(\bar{\lambda _{i}} =\lambda _{i}/{J}^{1/3}\), \(\lambda _{i} = e_{i}^{1/2}\), e i ’s are eigenvalues of B = FF T, J is the determinant of the deformation J = det(F) and deformation gradient tensor \(F_{ij} = \frac{\partial u_{i}} {\partial x_{j}} +\delta _{ij}\). In [9], α parameter of the model for the brain tissue is determined as \(\alpha = -4.7\). For simplicity, in this work we will use α = 2, which is known as the Neo-Hookean model. By replacing \(Tr(B) =\lambda _{ 1}^{2} +\lambda _{ 2}^{2} +\lambda _{ 3}^{2}\), the strain energy density function becomes:

$$\displaystyle\begin{array}{rcl} W = \frac{\mu } {2}\left (\frac{Tr(B)} {{J}^{2/3}} - 3\right ) + \frac{1} {D_{1}}{(J - 1)}^{2}& &\end{array}$$
(12)

Let us derive the Euler–Lagrange condition on the displacement field u for minimizing the given strain energy functional in terms of the trace and determinant. The derivative of the energy density W with respect to u i is given by:

$$\displaystyle\begin{array}{rcl} \nabla _{u_{i}}W = \frac{\partial W} {\partial u_{i}} -\sum _{j} \frac{\partial } {\partial x_{j}} \frac{\partial W} {\partial (\partial u_{i}/\partial x_{j})}& &\end{array}$$
(13)

The first term drops, as the energy density functional W is not dependent on the u but its derivatives. The derivative with respect to ∂u i  ∕ ∂x j is identical to the derivative with respect to F ij , therefore, rewriting Eq. (13) results in:

$$\displaystyle\begin{array}{rcl} \nabla _{u_{i}}W = -\sum _{j} \frac{\partial } {\partial x_{j}} \frac{\partial W} {\partial F_{ij}}& &\end{array}$$
(14)

Now, we need the derivatives \(\frac{\partial W} {\partial F_{ij}}\), which is also known as the 1st Piola–Kirchhoff tensor in mechanics literature. After a set of manipulations, the following derivative is obtained:

$$\displaystyle\begin{array}{rcl} \nabla _{u_{i}}W& =& - \frac{\mu } {{J}^{2/3}}\sum _{j}\left ( \frac{\partial F} {\partial x_{j}}\right )_{ij} + \frac{2\mu } {3{J}^{2/3}}\sum _{j}F_{ij}Tr\left ({F}^{-1} \frac{\partial F} {\partial x_{j}}\right ) \\ & & -\left (\frac{\mu Tr(B)} {3{J}^{2/3}} -\frac{2J(J - 1)} {D1} \right )\sum _{j}\left ({F}^{-1} \frac{\partial F} {\partial x_{j}}{F}^{-1}\right )_{ ji} \\ & & -\left (\frac{2\mu Tr(B)} {9{J}^{2/3}} + \frac{2J(2J - 1)} {D1} \right )\sum _{j}({F}^{-1})_{ ji}Tr\left ({F}^{-1} \frac{\partial F} {\partial x_{j}}\right ) \\ & & + \frac{2\mu } {3{J}^{2/3}}\sum _{j}({F}^{-1})_{ ji}Tr\left ( \frac{\partial F} {\partial x_{j}}{F}^{T}\right ) \end{array}$$
(15)

2.6 Implementation Details

A multi-resolution approach is implemented to increase the convergence speed. Volumes are down-sampled using trilinear interpolation and the obtained displacement field is interpolated to the higher resolution at the end of each stage.

The update equation for the displacement field u is obtained by the gradient descent method as:

$$\displaystyle{ \frac{\partial u} {\partial t} = -(\mathrm{Eq}.\,\mbox{ (8)} + \mathrm{Eq}.\,\mbox{ (10)} + \mathrm{Eq}.\,\mbox{ (15)}) }$$
(16)

If the maximum change due to the regularizer is greater than 1, the update of the displacement is normalized by dividing to the maximum update:

$$\displaystyle{ \nabla _{u}W = \left \{\begin{array}{ll} \frac{\nabla _{u}W} {\max (\nabla _{u}W)} & \text{if }\nabla _{u}W > 1 \\ \nabla _{u}W &\mathrm{otherwise}\\ \end{array} \right. }$$
(17)

Scaling of the strain energy density function of the regularizer term is arbitrary. Therefore, instead of having two independent parameters μ and D 1 for the energy functional in Eq. (12), the algorithm is affected mainly by the ratio \(\frac{\mu }{1/D_{ 1}} =\mu D_{1}\).

The effect of a μD 1 at a higher limit is shown on the sub-figure at the center of Fig. 4, which corresponds to the first term in Eq. (12), whereas the effect of a zero μD 1 is shown on the right sub-figure of Fig. 4, which corresponds to the second term. In this work, our aim is to estimate the cause (tumor deformation) by observing the result (displacement at the boundaries). Therefore, we assure the reversibility by enforcing a highly incompressible behavior to prevent the energy to be stored in the material, which we cannot observe by MRI. This is achieved by penalizing the local volume changes more by setting a low μD 1. We also assume that the total volume increase of the brain parenchyma is mostly caused by the increase of the tumor volume. Therefore, by setting the tumor region as a hole for the regularizer, which does not contribute to the external energy, and using an incompressible strain energy density, the observed local volume changes at the boundaries of the parenchyma are carried to the tumor area. We also note that, by decreasing the μ, the dependence of the strain energy density to the model parameter α in Eq. (11) is decreased, which strengthen our simplification approach of using Neo-Hookean model (α = 2) instead of Ogden form with \(\alpha = -4.7\).

Fig. 4
figure 4

Left: A closer view of the output of the regularizer test on phantom. Middle: The result obtained on phantom by increasing μ in the strain energy density model in Eq. (12). Right: The result obtained on phantom by setting the μ as zero in the strain energy density model in Eq. (12)

Full size image

Starting with an initial time step δt for the regularizer, the value is halved if the strain energy is not decreased by updating the displacement field.

The first derivatives are calculated by using central difference finite differencing scheme as:

$$\displaystyle{ V _{x}(x,y,z) = \frac{V (x + h,y,z) - V (x - h,y,z)} {2h} }$$
(18)

The derivative \(\frac{\partial F} {\partial x_{j}}\) is given by the following matrix:

$$\displaystyle{ \frac{\partial F} {\partial x_{j}} = \left (\begin{array}{ccc} \frac{{\partial }^{2}u_{ 1}} {\partial x_{1}\partial x_{j}} & \frac{{\partial }^{2}u_{ 1}} {\partial x_{2}\partial x_{j}} & \frac{{\partial }^{2}u_{ 1}} {\partial x_{3}\partial x_{j}} \\ \frac{{\partial }^{2}u_{ 2}} {\partial x_{1}\partial x_{j}} & \frac{{\partial }^{2}u_{ 2}} {\partial x_{2}\partial x_{j}} & \frac{{\partial }^{2}u_{ 2}} {\partial x_{3}\partial x_{j}} \\ \frac{{\partial }^{2}u_{ 3}} {\partial x_{1}\partial x_{j}} & \frac{{\partial }^{2}u_{ 3}} {\partial x_{2}\partial x_{j}} & \frac{{\partial }^{2}u_{ 3}} {\partial x_{3}\partial x_{j}}\\ \end{array} \right )}$$

Its components are calculated by second order finite difference discretization as:

$$\displaystyle\begin{array}{rcl} & & V _{xx}(x,y,z) = \frac{V (x - h,y,z) - 2V (x,y,z) + V (x + h,y,z)} {{h}^{2}} \\ & & V _{xy}(x,y,z) = \\ & & \quad \times \frac{V (x + h,y + h,z) - V (x + h,y - h,z) - V (x - h,y + h,z) + V (x - h,y - h,z)} {4{h}^{2}} \end{array}$$
(19)

Other derivative components are calculated similarly. At the boundaries, one-sided differences are used, for both the first and the second order derivatives.

3 Experiments and Results

3.1 Regularizer Test

To test the regularizer initially, the algorithm is run on 10 ×10 ×10 mask on a 20 ×20 ×20 lattice with a single constant displacement vector (0.0001,0.0001,0.0) and zero boundary conditions. The central xy-plane of the input and the result obtained with μ = 0. 8 and D 1 = 1. 0 is given in Fig. 3 with a closer look at Fig. 4. The effect of setting the parameters of the model at the lower and higher limits is shown on the middle and the right sub-figures of Fig. 4. Although incompressibility is a necessity for our problem as discussed in the method, setting a nonzero value to μ helps to increase the stability of the solution.

Fig. 3
figure 3

Left: Input phantom for the regularizer test. Right: Output of the regularizer test

Full size image

3.2 Phantom Results

To test the algorithm, two synthetic volumes are created as a sphere and an ellipsoid. Firstly, the method is run to determine the displacement field from sphere to ellipsoid by considering only the image terms, without the regularization. The results obtained with these external energies alone are shown in Fig. 5.

Fig. 5
figure 5

The central slices of the results obtained on synthetic volumes by applying the volumetric data term only (on left) and boundary data term only (on right) without regularization

Full size image

The central xy-plane of the result obtained by considering both image terms and the regularizer with μ = 0. 4 and D 1 = 1. 0 is given in Fig. 6. Since the target volume is larger than the reference, the incompressibility, enforced by a low μ, results curled field to minimize the change of the volume. Shortfall appearance of the vectors is due to the 2D slice visualization of the 3D vector field.

Fig. 6
figure 6

The result obtained on the synthetic volumes. Reference spherical volume is labeled with white color, the blue contour represents the boundary of the target ellipsoid, and the displacement field from sphere to ellipsoid is indicated with arrows in red color

Full size image

3.3 Experiments on MRI Brain Tumor Follow-up Volumes

3.3.1 The Data

MR images of a brain tumor patient (Glioblastoma Multiforme), obtained by 1.5T MRI scanner at high resolution (\(\thickapprox\) 0.5 ×0.5 ×1.0 mm) contiguous axial T1 weighted 3D SPGR (TE/TR = 3.16s/8.17s, FA = 25 ∘ ) sequence acquired after IV injection of 10cc 0.5M Multihance Gd, is used for validation study. The sample slices of the baseline and the follow-up volume, obtained 35 days after, are given in Figs. 1 and 2.

3.3.2 Preprocessing MRI Volumes

Statistical Parametric Mapping (SPM) software,Footnote 1 distributed by Wellcome Trust Centre for Neuroimaging, London, which gives accurate results in brain volumes with tumors, is used for standard operations such as: rigid registration, segmentation, and smoothing. Specifically, the following preprocessing operations are applied to the data before the execution of the deformable registration:

  • Follow-up volume is registered to the baseline volume using co-register function of SPM8.

  • White matter (WM) and gray matter (GM) segmentations (P > 0. 5) are obtained for both volumes using SPM8.

  • Tumors in both volumes are segmented using the Tumor-cut algorithm [5].

  • For each of the volumes, Tumor, WM and GM segmentations are combined using: \(\Omega _{\mathrm{Tumor}} \cup \Omega _{\mathrm{WM}} \cup \Omega _{\mathrm{GM}}\).

  • Each combined binary volume is smoothed by smooth function of the SPM8 with a Gaussian kernel having 8 ×8 ×8 mm full width at half maximum.

  • Finally, binary maps are converted to isotropic voxels (1 ×1 ×1 mm).

3.3.3 Results

Sample slices of the result obtained on tumor patient data are shown in Fig. 7. Intense displacement on the hemisphere with tumor, due to the mass effect, can be observed. At the bottom of the sub-figure on the right, displacement of the sulci due to the increased cerebrospinal fluid (CSF) pressure can be seen. Also, the increase in the CSF pressure results slight displacements at the ventricles of the healthy hemisphere, on the top. Displacement field, overlayed on a sample axial slice of the reference MRI in high resolution, is given in Fig. 8. The mass effect around the tumor is clearly observed. A closer look to the ventricle at the hemisphere without tumor depicts the displacement due to the expansion of the ventricle. When we focus on the displacement around the tumor as in Fig. 9, the displacement caused by the mass-effect (at the top) and the tumor growth (on the right) can be observed. The vectors at the bottom-left of the tumor explains the local shrinkage of the tumor as a tissue displacement.

Fig. 7
figure 7

Sample slices of the result obtained on tumor patient data in 3 ×3 ×3 mm voxel size. Binarized brain tissue of the reference volume is labeled in white color, the blue contour indicates the boundary of the target volume, and the displacement field is indicated with arrows in red

Full size image
Fig. 8
figure 8

Displacement field (in red) overlayed on a sample MRI slice of the reference volume, with the boundary of the target volume indicated with the blue contour. A closer look to the ventricle of the hemisphere without tumor

Full size image
Fig. 9
figure 9

A closer look to the tumor where the baseline tumor volume is labeled with white color, the follow-up tumor boundary is indicated with blue contour and the displacement field is overlayed with red arrows

Full size image

4 Discussion and Conclusions

A method to register the brain tissues in baseline and follow-up MRI volumes using finite deformation models is presented. Implementations on synthetic and patient data, with only minimal user interaction, provide promising results, which might have relevant use in clinical problems. Publicly available state-of-the-art algorithms for the rigid registration and tissue/tumor segmentation are able to provide highly accurate outputs, which is a necessity to obtain an accurate displacement field. Accuracy of the method could be increased by improving the data term, such as by adding vessel correspondences or manual landmarks.