A Novel Image Reconstruction Method in Three Dimensions

Abstract

The reconstruction of three-dimensional biological structures from magnetic resonance image and computed tomography data remains challenging because of the limitations of existing numerical techniques and substantial computer resources required. The work renders the structure as the zero-level contour of a level set function, which converges to the model when an objective functional, the sum of a fitting energy term used to extract the local intensity and a diffusion term acting as a regularization contributor, is minimized. In addition, a reaction-diffusion method developed to replace the original time-difference algorithm by finite element analysis. Numerical examples illustrate that correspondingly clear and smooth structures of a woodpecker’s skull can be obtained. Compared with the model reconstructed in a commercial tool, Mimics, our results presents substantially clearer and smoother interfaces without any isolated part.

1 Introduction

It is highly desirable to build effective and reliable three-dimensional (3D) models from tomographic images collected from computed tomography (CT) and magnetic resonance (MR) because it will provide practicians with comprehensive and intuitive structural information in clinic. The 3D reconstruction largely relies on image segmentation techniques [1, 2] to separate the raw data into non-overlapped geometric regions in line with their intensity information. However, there are still some difficulties, such as inhomogeneity of tissues, artefacts of images, a lack of computer resources, and irregularities of noise hinder the development of reconstruction of complex biological structures.

Recently, the global fitting energy in Chan-Vese model, which is a popular piecewise smoothed model and applies the specific energy functional as sum of the contour perimeter and fitting term has been replaced by a term of local data fitting energy, aiming to measure the image intensities on both sides of the contour. Its minimization allows the local intensity to participate in directing the motion of the boundary [18]. In addition, a regularization term can be included into the energy functional to increase the robustness of updating algorithm for the conventional Hamilton-Jacobi equation. Nevertheless, there are still two issues associated with this method: (1) small time step is compulsory when updating the governing equation with time-difference method; and (2) the level set function should be re-initialized frequently to remain the regularity [19].

In this study, we develop the reaction-diffusion approach to expediting the updating process of Hamilton-Jacobi equation, which involves both a reaction process to express local interaction between intensities and a diffusion process to allow the intensities homogenized in a spatial context [20]. The original idea of the reaction-diffusion method is to reveal the propagation of a gene population [21], and then it has been successfully introduced to structural topology optimization [22] to make the objective functional get convergent within fewer iteration steps. Another attractive character of reaction-diffusion method is the dispensability of the re-initialization process because the smoothness of the level set function is enhanced mathematically by a diffusion term in the energy functional. The regularity of the contour can be fully retained by a diffusion energy, thus the perimeter length term can be removed from the reaction-diffusion model. Furthermore, the finite element method used in renewing the Hamilton-Jacobi equation is able to allow a much larger time step than time difference method. Finally, we use this reaction-diffusion based level set method to reconstruct the skull of a woodpecker from the raw data, and hence a well-constructed structure is obtained within few iteration steps. Compared with the model reconstructed in a commercial tool, Mimics [23], our results presents substantially clearer and smoother interfaces without any isolated part.

The rest of paper is organized as follows. In Sect. 2, we describe the image reconstruction methodology. Section 3 presents some numerical implementations and examples. Conclusion is drawn in Sect. 4.

2 Image Reconstruction Methodology

• 1. Level set method

Level set method distinguishes the individual parts from the raw data by tracing the zero-level contour of a high-dimensional level set functional [13]. Mathematically, it is represented as:

$$ \left\{ {\begin{array}{*{20}l} {\varphi (x) > 0} \hfill & {\forall x \in\Omega _{1} } \hfill \\ {\varphi (x) = 0} \hfill & {\forall x \in\Gamma } \hfill \\ {\varphi (x) < 0} \hfill & {\forall x \in\Omega _{2} } \hfill \\ \end{array} } \right. $$

(1)

where the interface is represented as Г, which commonly separates the solid region Ω₁ and void region Ω₂ in domain Ω, and x denotes the spatial variable. The evolution of the boundaries with respect to the fictitious time t is tracked by the well-known Hamilton-Jacobi equation, formulated as:

$$ \frac{\partial \varphi (x,t)}{\partial t} - V_{N} (x,t)\left| {\nabla \varphi (x,t)} \right| = 0\quad {\text{in}}\,\,\Omega $$

(2)

where V_N is the velocity along the normal direction, which is normally determined as the negative variational of an objective functional J with respect to the level set function as:

$$ V_{N} = - \frac{\partial J(\varphi (x,t))}{\partial \varphi (x,t)} $$

(3)

where the negative sign allows the functional being minimized. By introducing the time difference method, Eq. (2) becomes:

$$ \varphi^{n + 1} = \varphi^{n} + \Delta tV_{N} \left| {\nabla \varphi^{n} } \right| = 0 $$

(4)

where the finite time-step ∆t needs to satisfy the Courant-Friedrichs-Lewy (CFL) stability condition [24]:

$$ V_{\hbox{max} }^{n} \cdot\Delta t \le \, min \, \left( {h_{x} ,h_{y} ,h_{z} } \right) $$

(5)

Thereinto, h_x, h_y and h_z denote the grid space in x, y and z directions, respectively.

Obviously, the size-dependent time step could result in very slow convergence for the raw data with extremely high resolution in the level-set based image separation. Conversely, large time increment may cause the level set function unstable and diverge largely from the value of signed distance quickly in the evolution. Another reason of low efficiency in updating the level set function is the compulsory re-initialization process in each or every a few steps to guarantee the level set function remaining as a distance function from the interface and therefore avoiding the ill-conditioning (e.g. small gradient of level set function) when numerically locating the interface [25].

• 2. Reaction-diffusion equation

We herein propose to adopt the reaction-diffusion equation for improving the efficiency of updating the level set function. Since the Hamilton-Jacobi equation is solved using the finite element analysis in this algorithm, the constraint to the time step resulted from CFL condition is released, and thus the re-initialization process becomes unnecessary. Without specific indication, a general objective function J is employed without showing its physical meanings in the following derivative. In the next Section, this function will be replaced by a sum of two energy terms used for image segmentation.

The optimization problem is mathematically expressed as:

$$ \mathop { \inf }\limits_{\varphi } \quad F(\varphi ) = J(\varphi ) + \frac{1}{2}\int_{\Omega } {\tau \left| {\nabla \varphi } \right|^{2} d\Omega } + \lambda \left( {\int_{\Omega } {d\Omega - V_{0} } } \right) $$

(6)

in which λ represents the Lagrangian multiplier for the volume constraint V₀. The second term in Eq. (6) represents the interface energy, which is widely used in image processing and other physical model like the Cahn-Hilliard equation [26]. For the sake of controlling the regularization effect, it is weighted by a small positive factor τ > 0. This energy term serves as number of roles, such as a regulator to the optimization problem, a smoother of the level set function, and a stabilizer of the algorithm. Due to the competences between regularity and the smoothness, the re-initialization process becomes unnecessary [22].

In accordance with the Karush-Kuhn-Tucker conditions [27], by introducing a fictitious time t and assuming that the level set function φ is an implicit function of time t, the structural changes in domain Ω are naturally and flexibly implemented. In the level set-based optimization method, the object functions are updated by solving time evolutional equation as Eq. (2). Here, a standard algorithm to minimize function F is to find the steady state solution of the gradient follow equation which assumes that the variation of the level set function φ with respect to time t is equal to $\partial F/\partial \varphi$ which is the Gâeaux derivative [28] of the function F, as:

$$ \frac{\partial \varphi }{\partial t} = - \frac{\partial F}{\partial \varphi } $$

(7)

Substituting Eq. (6) into Eq. (7), the following equation is obtained:

$$ \frac{\partial \varphi }{\partial t}{ = } - (\frac{\partial J(\varphi )}{\partial \varphi } - \tau \Delta \varphi ) \, $$

(8)

where ∆ is the Laplace operator. Based upon the definition that the boundary is composed of Dirichlet boundary on the non-design boundary and Neumann boundary on the rest, the time evolutionary equation related to boundary conditions is expressed as:

$$ \left\{ {\begin{array}{*{20}l} {\frac{\partial \varphi }{\partial t} = - \frac{\partial F}{\partial \varphi }{ = } - (C\frac{\partial J(\varphi )}{\partial \varphi } - \tau\Delta \varphi )} \hfill & {{\text{in }}\Omega } \hfill \\ {\frac{\partial \varphi }{\partial n} = 0} \hfill & {{\text{on }}\partial {\Omega \backslash }\partial\Gamma _{N} } \hfill \\ {\varphi = 1} \hfill & {{\text{on }}\partial\Gamma _{N} } \hfill \\ \end{array} } \right. $$

(9)

In the reaction-diffusion method, τ∆φ is considered to be the diffusion term other than the regularization term in the time-difference algorithm, while the derivative of the cost function dJ(φ)/dφ acts as a reaction term to account for the change in the curve (2D) or surface (3D) which is weighted by a factor C for normalization, defined as [22]:

$$ C = \frac{{c\int_{\Omega } {d\Omega _{1} } }}{{\int_{\Omega } {\left| {\frac{\partial J(\varphi )}{\partial \varphi }} \right|d\Omega _{1} } }} $$

(10)

Rearrange Eq. (9) in a weak form for implementing discretization by finite element method, as follows:

$$ \left\{ {\begin{array}{*{20}l} {\int_{\varOmega } {\frac{{\varphi (t +\Delta t)}}{{\Delta t}}} \varphi d\varOmega + \int_{\varOmega } {{\nabla }^{T} \varphi (t + \Delta t)} (\tau {\nabla }\varphi )d\Omega } \hfill & {} \hfill \\ { = \int_{\Omega } {(C\frac{\partial J(\varphi )}{\partial \varphi } + \frac{\varphi (t)}{\Delta t})\varphi d\Omega } } \hfill & {{\text{for}}\,\forall \varphi \in \, \widetilde{\varphi }} \hfill \\ {\varphi = 1} \hfill & {{\text{on }}{\partial \varGamma }_{N} } \hfill \\ \end{array} } \right. $$

(11)

where $\widetilde{\varphi }$ is the Sobolev functional space of level set function.

For the time-related discretization, the implicit scheme is applied and the domain Ω is discretized based on finite element method. Thus, the discretized evolution equation is formulated as:

$$ \left\{ {\begin{array}{*{20}l} {\left( {\frac{1}{\Delta t}{\mathbf{T}}_{1} + \tau {\mathbf{T}}_{2} } \right){\varvec{\upzeta}}(t + \Delta t) = \frac{\varphi (t)}{{\Delta t}} + C\frac{\partial J(\varphi )}{\partial \varphi }} \hfill & {{\text{in}}\,\,\Omega } \hfill \\ {\varphi = 1} \hfill & {{\text{on}}\,\,{\partial \varGamma }_{N} } \hfill \\ \end{array} } \right. $$

(12)

where ζ(t) is the nodal value vector of the level set function at time t, T₁ and T₂ can be described as follows:

$$ \left\{ {\begin{array}{*{20}l} {{\mathbf{T}}_{1} = \bigcup\limits_{j = i}^{e} {\int_{{V_{e} }} {{\mathbf{N}}^{T} {\mathbf{N}}dV_{e} } } } \hfill \\ {{\mathbf{T}}_{2} = \bigcup\limits_{j = i}^{e} {\int_{{V_{e} }} {\nabla {\mathbf{N}}^{T} \nabla {\mathbf{N}}dV_{e} } } } \hfill \\ \end{array} } \right. $$

(13)

where e is the number of element. j is the number of the elements and N is the interpolation function of the level set function.

Through application of the reaction-diffusion equation to update the level set function, the compulsory re-initialization process in the level set method and the typical weaknesses from time difference method can be avoided. Therefore, introducing reaction-diffusion equation into the level set method is able to establish a more efficient, powerful and re-initialization-free algorithm for 3D reconstruction.

3 Numerical Implementation and Examples

• 1. Numerical implementation

Compared with current popular energy formulation in dealing with image segmentation schemes, here we only apply a data fitting energy [18] and a diffusion term which replaces the sum of two necessary terms in others, including an arc length term and a regularization term.

$$ F(\varphi,\, f_{1},\, f_{2} ) = \sum\limits_{i = 1}^{2} {\lambda_{i} \int {(\int {K_{\sigma } (x - y)\left| {{\mathbf{I}}(y) - f_{i} (x)} \right|^{2} M_{{_{i} }}^{\varepsilon } (\varphi (y))dy} )dx} } + \tau \int {\left( {\left| {\nabla \varphi (x)} \right|} \right)}^{2} dx $$

(14)

where M₁(φ) = H(φ) and M₂(φ) =1 − H(φ), I denotes an image, λ₁ and λ₂ are positive constants, and f₁(x) and f₂(x) are the two values that approximate the weighted averages of the image intensities in Ω_in and Ω_out which are inside and outside a close contour in the design domain Ω, respectively. The intensities I(y) effectively involved in the above fitting energy are in a local region, whose size can be controlled by the Gaussian kernel K_σ which is shown as follows:

$$ K_{\sigma } ({\mathbf{I}}) = \frac{1}{{(2\pi )^{{{n \mathord{\left/ {\vphantom {n 2}} \right. \kern-0pt} 2}}} \sigma^{n} }}e^{{ - \left| {\mathbf{I}} \right|^{2} /2\sigma^{2} }} $$

(15)

where the scale parameter σ > 0.

The Heaviside function used in the above energy functional is defined by

$$ H(x) = \frac{1}{2}\left[ {1 + \frac{2}{\pi }\arctan (\frac{x}{\varepsilon })} \right] $$

(16)

The derivative of H is

$$ \delta (x) = H^{\prime}(x) = \frac{1}{\pi }\frac{\varepsilon }{{\varepsilon^{2} + x^{2} }} $$

(17)

The functional F(φ, f₁, f₂) is the minimized postulate of a fixed level set function φ. By applying the calculus of variations, the functions f₁(x) and f₂(x) need to satisfy the Euler-Lagrange equations, as follows:

$$ \int {K_{\sigma } (x - y)\left| {{\mathbf{I}}(y) - f_{i} (x)} \right|^{2} M_{{_{i} }}^{\varepsilon } (\varphi (y))dy} = 0\,\,\,\,i = 1 , 2 $$

(18)

From Eq. (18), we obtain

$$ f_{i} (x) = \frac{{K_{\sigma } (x) * [M_{i}^{\varepsilon } (\varphi (x)){\mathbf{I}}(x)]}}{{K_{\sigma } (x) * M_{i}^{\varepsilon } (\varphi (x))}}\,\, i = 1,2 $$

(19)

where the symbol * represents a convolution operator. Keeping f₁ and f₂ be fixed, the derivative of energy functional F(φ, f₁, f₂) with respect to t is:

$$ \frac{\partial \varphi }{\partial t} = - \delta (\varphi )(\lambda_{1} c_{1} - \lambda_{2} c_{2} ) + \tau \nabla^{2} \varphi $$

(20)

where δ is Delta function and c₁ and c₂ is given by:

$$ c_{i} (x) = \int {K_{\sigma } (x - y)\left| {{\mathbf{I}}(x) - f_{i} (y)} \right|^{2} dy} \,\,\,\,i = 1,2 \, $$

(21)

Next, the updating process is carried out by using the reaction-diffusion scheme with respect to Eq. (12). The approach in this section will be implemented through the following numerical examples.

• 2. Numerical example

In this section, we reconstruct three numerical models for a woodpecker from the raw CT data to assess the validity and utility of our proposed method. In all examples, the value of ε in Eq. (16) is set to be 0.1 and parameter K_σ = 1.0 and the value of λ₁ and λ₂ in Eq. (20) are set to be λ₁ = 1.0 and λ₂ = 20, respectively.

Due to huge amount of invalid information in CT images, the intensity distribution (see Fig. 1) shows that the useful data related to the bone of a woodpecker is mainly located in the region where intensity is greater than 500. Thus, in these given case scenarios, only the part where conforms to intensity T > 500 is considered.

Fig. 1.

Image intensity distribution

• A. Case 1

In the first case, the diffusion parameter τ is set to be τ = 5.0 × 10⁻⁶, parameter σ = 6.0 and the time increment ∆t = 0.01. Figure 2 illustrates the segmentation process using the proposed method, the value of m stands for the number of iteration steps. As shown in the 500^th step, a smooth and clear model for a woodpecker’s skull is obtained by using the proposed method.

Fig. 2.

Configurational transformation in case 1

As mentioned before, the major novelty of this new method lies in applying the reaction-diffusion equation to update of the level set function. To demonstrate the superiority of the proposed method to traditional algorithms, an example obtained from time difference method was considered, which needs to add an extra perimeter length term into the energy functional for maintaining the smoothness of level set contour. Herein, the arc length parameter in perimeter length term µ = 1.0 and all other sets are not changed for reasonable comparison.

The two sub-figures in Fig. 3 specifically show that the surface obtained by the proposed method (Fig. 3(a)) is clearer, smoother and more accurate than that in the conventional time difference method (Fig. 3(b)). In the next stage of our research, we will investigate the impact-resistance of the woodpecker’s skull. Thus, clarity, smooth and accuracy of the model are primary considerations. Some preliminary studies have shown that the asperous result in Fig. 3(b) is not suitable for carrying out impact simulation.

Fig. 3.

Comparison of the configurational performance between (a) the reaction diffusion algorithm and (b) the time difference algorithm

• B. Case 2

In the second case, diffusion parameter τ is set to be τ = 5.0 × 10⁻⁵, parameter σ = 3.0 and time step ∆t = 0.001. Instead of variable values with regard to Eq. (10), parameter C is assigned to be the fixed value of C = 0.1. The segmentation process in this case is displayed in Fig. 4.

Fig. 4.

Configurational transformation in case 2

Further algorithmic comparison between the proposed method and time difference method are shown in Figs. 5 and 6, which includes the performance of configurations and numerical analysis. Note that the parameter µ in the length term for the conventional model is set to be µ = 1.0 by remaining the other parameters unchanged.

Fig. 5.

Comparison of the configurational performance between (a) the proposed method and (b) the time difference method

Fig. 6.

The comparison of energy variation between the proposed method and time difference method, including (a) total energy, (b) fitting energy, (c) regularity-related energy, and (d) perimeter length energy

Differing from the comparison in Fig. 3, both the results in Fig. 5(a) and (b) are fairly similar. It is difficult to distinguish which provides better configuration by eyes. Thus, an in-depth comparison between the models with the reaction-diffusion equation and time difference method is needed.

In these four line charts in Fig. 6, the blue line presents the energy variation in the model using the proposed method, while the red line shows the transformation using the conventional finite difference method. In order to check the convergence of energy $J(\phi, \, f_{1},\, f_{2} )$, we use the following criterion: if $\left( {J_{{{\text{n}} + {1}}} - J_{\text{n}} } \right)/J_{\text{n}} \, \leqslant \,{1}0^{{ - {3}}}$, the nth iteration step attains convergence. Herein, $J_{\text{n}} \left( {\phi, \,f_{1}, \,f_{2} } \right)$ represents the energy value in the nth iteration step. It is evident that both models are able to converge quickly after 2 iterations. It means that the proposed model can not only generate more accurate results, but also achieve a goal of immediate image segmentation by adjusting a certain parameter.

Note that Sections A and B confirmed that the proposed method has potential to reconstruct complex three-dimensional raw data into biological structures with well-defined boundaries. In comparison with the conventional time difference method, the proposed method exhibits certain superiority in 3D reconstruction.

• C. Parameter investigations

Note that the scale parameter σ plays an important role in determining the width of the Gaussian kernel. The alteration of σ leads to the changes in K_σ, f₁ and f₂. In addition to σ, diffusion parameter τ that is used to control the effect of regularization also plays a significant role in running the proposed method. As two of the significant parameters, the effects of σ and τ are investigated further in this section. Before discussing the effects of parameters, a case scenario is presented in Fig. 7. Except that the diffusion parameter τ is changed to τ = 7.0 × 10⁻⁶ and scale parameter σ is set to σ = 3.0, the other parameters are remained the same as case 1.

Fig. 7.

Configurational transformation in case 3

Compared with the result of case 1 shown in Fig. 2, while the configurations in the both cases are of almost the same level of smoothness, huge differences occur in the other aspects. Therefore, it proves that the change of τ and σ play significant roles in the transformation of model.

Figure 8 compares the performances of the obtained configurations under different values of σ. In terms of clarity and smoothness, the surfaces on all results are nearly same, but increasing σ increases the volume of final configurations. Nevertheless, the complexity of calculation is also raised numerically. It means that more computational resource and cost are needed to complete the entire modelling process. Therefore, taking the efficiency into considerations, setting σ as big as possible does not mean the best choice.

Fig. 8.

Comparison of the configurational performance when σ is equal to (a) 3.0, (b) 6.0 and (c) 9.0

Next, the effects from regularization parameter τ are analysed as shown in Fig. 9 in which the results at the 500^th iteration are compared for the different values of τ. Owing to the functions of controlling the effects of the regularization and adjusting length of level set contours, the rise of parameter τ leads to length shortening in the configurations (see Fig. 8). It is estimated that the case (b) and (c) in Figs. 8 require more iteration steps to achieve a similar configuration to case (a). Namely, the change of parameter τ makes the segmentation process longer. To some degree, an appropriate reduction in τ is beneficial to improving the efficiency. Nevertheless, specifying τ at a smaller value could potentially make the proposed method unable to run.

Fig. 9.

Comparison of the configurational performance in 500th iteration step when τ is set to (a) 7.0 × 10⁻⁶, (b) 2.0 × 10⁻⁵, (c) 7.0 × 10⁻⁵

Based on the discussion in this section, it is found that the parameters σ and τ has significant effects on the reconstruction results. Prescription of reasonable parameters would be helpful for dealing with different 3D structures.

• D. Reconstruction in Mimics

At present, several commercial codes are able to conduct image processing, such as Mimics, Simpleware [29] and 3D-doctor [30]. As one of the most popular reconstruction programs, Mimics uses 2D cross-sectional images, e.g. obtained from computer tomography and magnetic resonance image, to reconstruct 3D models. In order to make a comparison between both the results generated from Mimics and the proposed method, the woodpecker’s CT data is imported into Mimics for 3D modelling.

The modelling in Mimics is relying on defining a certain range of grey scale for identifying the eligible sections in each CT image. These distinguished parts are used to make up a new 3D model. However, due to proximate greyscales for skull, spine and weasand, it is difficult to segment an accurate scope for reconstructing the woodpecker’s skull. Further, as presented in Fig. 10, the marginal area in the materialized model is fuzzy with a huge number of isolated regions. Thus, the simulation of a woodpecker’s impacting process is hard to be conducted by directly using Mimics results unless further tunning is taken.

Fig. 10.

Reconstruction result in Mimics

4 Conclusions

This study proposed a novel level set method by incorporating the reaction-diffusion equation, and applied it to reconstruct the three-dimensional structures from the high resolution raw data. We defined the energy functional with a fitting energy, which extracted the initial contour toward desired boundaries by utilizing local intensity and a diffusion term to regularize the level set function and smooth the interface contour.

The proposed method is capable of reconstruction images with high inhomogeneity of intensity, and provide desirable reconstructed model for complicated scanning images with blurred boundaries. With the diffusion term in the reaction-diffusion equation, the regularity of the level set function and the smoothness of the contour can be intrinsically maintained to ensure accurate computation, and thus the expensive re-initialization process and perimeter length term which are compulsory in conventional level set algorithms can be eliminated.

In addition, the finite element method replaces the time difference method to allow a greater time step when updating the level set function. For this reason, the efficiency to a certain degree is improved. The numerical example showcased the desired performance of the proposed method for CT image based reconstruction of a woodpecker’s skull. Compared with the model reconstructed using commercial code Mimics, the proposed method is able to provide a better result with clearer and smoother interfaces without isolated parts.