Geometric Transformation Image Registration Based on Discrete Feature Point Fitting

Abstract

Image registration is the premise of image fusion, it is important to find a high-precision registration method to make the multi-modal medical images align between the coordinate points of the same particle in the space. Previous researchers studied multi-modal medical image registration, which is based on the three-dimensional contour points and the largest common area of the images. Although the final experimental results are acceptable, part of the original image data is lost. Instead of intercepting the maximum common region, all layers of the two modes are involved in the calculation of the registration algorithm in this paper. The feature points of each layer are extracted, and the feature axis is obtained by linear fitting. The rotation operator is constructed based on this characteristic axis, then the 3D models of the two modes are registered in space.

1 Introduction

Medical images can be roughly divided into two types according to the different description of human body information: one is the anatomical image used to describe tissue structure and anatomical information, the other is to describe the metabolic information in the body. Cranial image registration technology has many advantages in clinical medicine [1].

Image registration has been born very early, and has a long history in the field of medicine. In foreign countries, it was Anuta P.E. et al. That was the first to study a fast Fourier transform (FFT) method to register different images [2]. Then, the multi-mode image registration based on feature neighborhood mutual information method is proposed by P.A. Legg et al. [3]; The registration method of ICP was proposed by N. Senin et al. [4]. The domestic research on medical image registration started relatively late, but it has also made remarkable achievements. Shuqian Luo et al. Proposed multimodal medical image registration based on maximum mutual information [5]. In recent years, some experts have applied Clifford algebra [6] in the field of information processing and computer graphics. Hui Liu et al. Used Clifford differential and Clifford gradient to calculate the gradient norm of a pixel in a multispectral image to judge whether the pixel is an edge point [7]. Liang Hua et al. applied Clifford algebra to the registration of 3D multimodal medical images, and transferred the classical ICP algorithm to Clifford algebra space for calculation [8].

In this paper, the geometric invariants of each set of images are found and established in the geometric algebra and analytic geometry space by combining the characteristics of the skull contour, the overall structure and the differences of the human posture in the actual image acquisition. High precision translation, rotation, scaling and other operations are implemented for the images to be registered, and the images with fixed modes are registered and proofread.

2 Mathematical Theory and Calculation Method

2.1 Mass Moment of Inertia

Mass moment of inertia refers to the measure of inertia of a rigid body when it rotates around a fixed axis (a unique property, that is, a rotating object keeps its uniform circular motion or static), and is represented by the letter I or J. In classical mechanics, the unit of mass moment of inertia is $\mathrm{Kg}\cdot {\mathrm{m}}^{2}$. For a particle, its mass moment of inertia is $\mathrm{I}={\mathrm{mr}}^{2}$, where m is the mass of the particle and R is the vertical distance from the particle to the rotation axis.

The expression of mass moment of inertia is as follows:

$$ {\text{I}} = \sum\nolimits_{{\text{i}}} {{\text{m}}_{{\text{i}}} {\text{r}}_{{\text{i}}}^{2} .} $$

(1)

If the mass of a rigid body is continuously distributed, the formula for calculating the mass moment of inertia can be written as follows:

$$ {\text{I}} = \int {\int_{{\text{V}}} {{\text{r}}^{2} {\text{dm}}} = \int {\int_{{\text{V}}} {{\text{r}}^{2} \rho {\text{dV}}.} } } $$

(2)

2.2 Geometric Algebra

Clifford algebra, also known as geometric algebra [6], integrates two operations of inner product and outer product.

1) Geometric product

The geometric product is the sum of outer product and inner product. For any two vectors a and b, the geometric product can be expressed as follows:

$$ {\text{ab}} = {\text{a}} \cdot {\text{b}} + {\text{a}} \wedge {\text{b}}. $$

(3)

Where, the dot product $\mathrm{a}\cdot \mathrm{b}$ is a scalar and the outer product $\mathrm{a}\wedge \mathrm{b}$ is a double vector, then ab is a multiple vector (Fig. 1).

4) Reflection

The reflection vector a' of vector a in plane B can be expressed as (Fig. 2):

$$ {\rm{a^{\prime}}} = {\text{a}}_{{ \bot {\text{B}}}} - {\text{a}}_{{\left\| {\text{B}} \right.}} = -\upeta {\text{a}}\upeta ^{ - 1} . $$

(4)

Note: when $\upeta$ is a unit vector, ${\upeta }^{-1}=\upeta$.

Fig. 1.

Reflection of vector on plane

Fig. 2.

A rotation to b

5) Rotation

Rotate unit vector a at any angle $\uptheta$ to get unit vector b, make the unit angle bisector $\upeta$ of vectors a and b in the plane $\mathrm{a}\wedge \mathrm{b}$, and make the vertical plane $\uppi 1$ of the vector $\upeta$, then make a reflection vector about the plane $\uppi 1$ for vector a, the reflection vector c of a can be expressed as:

$$ {\text{c}} = -\upeta {\text{a}}\upeta . $$

(5)

Then making the vertical plane $\uppi 2$ of vector b, we can get that vectors b and c are perpendicular to the plane $\uppi 2$, Then b can be expressed as:

$$ {\text{b}} = - {\text{bcb}} = - {\text{b}}\left( { -\upeta {\text{a}}\upeta } \right){\text{b}} = {\text{b}}\upeta {\text{a}}\upeta {\text{b}} = {\text{RaR}}^{ - 1} . $$

(6)

The bisection vector of unit angle is $\upeta =\frac{\mathrm{a}+\mathrm{b}}{\Vert \mathrm{a}+\mathrm{b}\Vert }$, and the rotation operator is $\mathrm{R}=\mathrm{b\eta }$.

3 Image Preprocessing

3.1 Two Dimensional Image Edge Extraction

First of all, Canny operator is used to extract the edge of the two-dimensional image. Finally, only the outermost contour of the image is obtained, and then the outermost contour of each layer is extracted by horizontal and vertical scanning method. So far, the edge of the image has been extracted to prepare for the three-dimensional modeling of the image (Fig. 3).

Fig. 3.

Contour extraction of two-dimensional slice

3.2 3D Modeling of Contour

From a physical point of view, due to the protection of the skull, the soft tissue inside the skull is not easy to deform due to internal and external factors, thus maintaining the clarity of the contour, so we can treat the skull image as a rigid body (Fig. 4).

The edge points are stacked in the Z direction to get the three-dimensional contour of the skull. Get the three-dimensional outline of the skull, as shown in Fig. 5.

Fig. 4.

Three dimensional outline of skull

4 Discrete Feature Points Extraction and Linear Fitting

After unifying the spatial resolution of the image groups of the two modes, Canny operator is used to extract contour points, the images of the two modes are stacked in the Z direction to obtain the 3D model of the image group, the centroids of the three-dimensional models of the two modes are translated to the center of the plane with the average height of the left and right ear tips as the height.

The characteristic points of each layer are obtained by using the principle of minimum mass moment of inertia, the formula of mass moment of inertia for discrete points is as follows:

$$ {\text{I = }}\sum\nolimits_{{\text{i}}} {{\text{m}}_{{\text{i}}} {\text{r}}_{{\text{i}}}^{2} } . $$

(7)

For the convenience of calculation, the mass of discrete points is recorded as unit 1, let the characteristic points of each layer be ${\mathrm{O}}_{\mathrm{z}}$, where Z is the height of each slice in the coordinate system. The contour point of the registration mode 3D model whose ordinate is Z = 0 is taken as an example: firstly, all the contour points in Z = 0 layer of the registered modal 3D model are extracted ${\mathrm{P}}_{\mathrm{z}=0}$, let the characteristic point of this layer be ${\mathrm{O}}_{\mathrm{z}=0}$ (x,y,0); then the sum of mass moment of inertia of each contour point around this point is calculated; finally, when the sum of mass moment of inertia of all contour points to this point is minimum, the corresponding values of X and y are obtained, the obtained points ${\mathrm{O}}_{\mathrm{z}=0}$ is the feature points of this layer. The specific calculation process is as follows:

Firstly, the mass moment of inertia of each contour point around this point is calculated:

$$ {\text{I}} = {\text{m}}_{{\text{i}}} {\text{r}}_{{\text{i}}}^{2} = ({\text{x}}_{{\text{i}}} - {\text{x}})^{2} + ({\text{y}}_{{\text{i}}} - {\text{y}})^{2} . $$

(8)

Then the sum of mass moment of inertia of all contour points around this point in this layer is:

$$ {\text{I}} = \sum\nolimits_{{\text{i}}} {{\text{m}}_{{\text{i}}} {\text{r}}_{{\text{i}}}^{2} } = \sum\nolimits_{{\text{i}}} {\left[ {({\text{x}}_{{\text{i}}} - {\text{x}})^{2} + ({\text{y}}_{{\text{i}}} - {\text{y}})^{2} } \right]} . $$

(9)

The partial derivatives of x and y are obtained from the above formula:

$$ \left\{ {\begin{array}{*{20}c} {\frac{{\partial {\text{I}}}}{{\partial {\text{x}}}} = \sum\nolimits_{{\text{i}}} {(2{\text{x}} - 2{\text{x}}_{{\text{i}}} ) = 0} } \\ {\frac{{\partial {\text{I}}}}{{\partial {\text{y}}}} = \sum\nolimits_{{\text{i}}} {(2{\text{y}} - 2{\text{y}}_{{\text{i}}} ) = 0} } \\ \end{array} } \right.. $$

(10)

By solving the partial derivative equation, the values of x and y are obtained, then the feature points of the registered modal 3D model in Z = 0 layer is obtained, and the sum of the mass moment of inertia from all the contour points to this point is minimum. The feature points of other layers of the registration mode are solved by the same method, then the eigenvectors of registration modes are obtained by linear fitting of these discrete points ${\mathrm{h}}_{\mathrm{f}}$, The process of linear fitting is as follows:

Suppose that the linear equation of the feature discrete points of the registered modal three-dimensional model is as follows:

$$ \frac{{{\text{x}} - {\text{x}}_{{\text{f}}} }}{{\text{X}}} = \frac{{{\text{x}} - {\text{y}}_{{\text{f}}} }}{{\text{Y}}} = \frac{{{\text{x}} - {\text{z}}_{{\text{f}}} }}{{\text{Z}}}. $$

(11)

The equations of x, y with respect to Z are obtained:

$$ \left\{ {\begin{array}{*{20}c} {{\text{x}} = \frac{{\text{x}}}{{\text{z}}}({\text{z}} - {\text{z}}_{{\text{f}}} ) + {\text{x}}_{0} = {\text{az}} + {\text{b}}} \\ {{\text{y}} = \frac{{\text{y}}}{{\text{z}}}({\text{z}} - {\text{z}}_{0} + {\text{y}}_{0} = {\text{cz}} + {\text{d}}} \\ \end{array} .} \right. $$

(12)

The least square method is used to list the equations:

$$ \left\{ {\begin{array}{*{20}c} {{\text{Q}}_{{\text{x}}} = \sum\nolimits_{{{\text{f}} = 1}}^{{\text{m}}} {[{\text{x}}_{{\text{f}}} - ({\text{az}}_{{\text{f}}} + {\text{b}})]^{2} } } \\ {{\text{Q}}_{{\text{y}}} = \sum\nolimits_{{{\text{f}} = 1}}^{{\text{m}}} {[{\text{y}}_{{\text{f}}} - ({\text{cz}}_{{\text{f}}} } + {\text{d}})]^{2} } \\ \end{array} } \right.. $$

(13)

The partial derivatives of equations for a, b, c and d are obtained as follows:

$$ \left\{ {\begin{array}{*{20}c} {{\text{b}}\sum\nolimits_{{{\text{f}} = 1}}^{{\text{m}}} {{\text{z}}_{{\text{f}}} + {\text{a}}\sum\nolimits_{{{\text{f}} = 1}}^{{\text{m}}} {{\text{z}}_{{\text{f}}}^{2} = \sum\nolimits_{{{\text{f}} = 1}}^{{\text{m}}} {{\text{x}}_{{\text{f}}} {\text{z}}_{{\text{f}}} } } } } \\ {{\text{bm}} + {\text{a}}\sum\nolimits_{{{\text{f}} = 1}}^{{\text{m}}} {{\text{z}}_{{\text{f}}} = \sum\nolimits_{{{\text{f}} = 1}}^{{\text{m}}} {{\text{x}}_{{\text{f}}} } } } \\ {{\text{d}}\sum\nolimits_{{{\text{f}} = 1}}^{{\text{m}}} {{\text{z}}_{{\text{f}}} + {\text{c}}\sum\nolimits_{{{\text{f}} = 1}}^{{\text{m}}} {{\text{z}}_{{\text{f}}}^{2} = \sum\nolimits_{{{\text{f}} = 1}}^{{\text{m}}} {{\text{y}}_{{\text{f}}} {\text{z}}_{{\text{f}}} } } } } \\ {{\text{dm}} + {\text{c}}\sum\nolimits_{{{\text{f}} = 1}}^{{\text{m}}} {{\text{z}}_{{\text{f}}} = \sum\nolimits_{{{\text{f}} = 1}}^{{\text{m}}} {{\text{y}}_{{\text{f}}} } } } \\ \end{array} } \right.. $$

(14)

If $\mathrm{F}=\left[\begin{array}{cccc}{\mathrm{z}}_{1}& {\mathrm{z}}_{2}& \cdots & {\mathrm{z}}_{\mathrm{m}}\\ 1& 1& \cdots & 1\end{array}\right]$, $\mathrm{A}=[\begin{array}{cc}\mathrm{a}& \mathrm{b}\end{array}]\mathrm{^{\prime}}$, $\mathrm{X}=[\begin{array}{ccc}{\mathrm{x}}_{1}& \cdots & {\mathrm{x}}_{\mathrm{m}}\end{array}]\mathrm{^{\prime}}$, $\mathrm{B}=[\begin{array}{cc}\mathrm{c}& \mathrm{d}\end{array}]\mathrm{^{\prime}}$, $\mathrm{Y}=[\begin{array}{ccc}{\mathrm{y}}_{1}& \cdots & {\mathrm{y}}_{\mathrm{m}}\end{array}]\mathrm{^{\prime}}$, then the solution of the above equations (8) can be transformed into the solution of the following matrix:

$$ \left\{ {\begin{array}{*{20}c} {{\rm{FF^{\prime}}}{\text{A}} = {\text{FX}}} \\ {{\rm{FF^{\prime}}}{\text{B}} = {\text{FY}}} \\ \end{array} .} \right. $$

(15)

The solutions of vectors A and B can be obtained by taking all of the m discrete characteristic points into the above matrix equation, the solutions of a, b, c and d are obtained, then the equation of the line where the discrete feature points are located is obtained.

The eigenvector ${\mathrm{h}}_{\mathrm{f}}$ is the unit vector in the direction of (X, Y, Z). Using the same method, the characteristic points of each layer of the fixed mode are obtained, when the sum of the mass moment of inertia of the discrete points of the layer contour to this characteristic point is minimum. The eigenvectors of the fixed modes are obtained by linear fitting of these discrete eigenpoints ${h}_{r}$, the eigenvectors ${h}_{f}$ and ${h}_{r}$ of two different modes are used to construct the rotation operator, the eigenvector of the registration mode ${\mathrm{h}}_{\mathrm{f}}$ is rotated to the position coincident with the reference mode according to the rotation operator, the rotation operator R is as follow:

$$ R = h_{r} \frac{{h_{f} + h_{r} }}{{\left\| {h_{f} + h_{r} } \right\|}}. $$

(16)

5 Experimental Results and Analysis

The medical images used in this experiment are from the “retrospective image registration evaluation” project library of Vanderbilt University in the United States, and this study was based on patient _ 001, the fixed CT image and PD image group are selected for registration. The 3D model of CT and PD image group is established before registration, the three-dimensional model is translated to the center of the plane with the average height of the left and right ear tips as the height as the origin. Figure 5 shows the multimodal 3D modeling before registration, Fig. 6 shows the 3D model of multimode after registration.

Fig. 5.

3D modeling before registration

Fig. 6.

3D modeling after registration

The experimental registration parameters are shown in the following table (Table 1):

Table 1. Registration parameters

The evaluation results are shown in Table 2:

Table 2. Error between experimental results and gold standard

The fusion effect is shown in Figs. 7:

Fig. 7.

Cross section parallel to xoy plane

The effectiveness of the method in this chapter can be well verified by three-dimensional model, gold standard and fusion renderings, the experimental algorithm also achieves good results in other image groups of patient 1 and other image groups of patients. It is proved that the method in this chapter also has strong robustness.

The error of this algorithm is compared with the previous methods of other scholars in this field. The error comparison of various algorithms is shown in Table 3.

Table 3. Error comparison of various algorithms

In this paper, all the contour points are involved in the calculation, and combined with the geometric invariance of mass moment of inertia, the feature points are constructed, the line where the feature points are fitted linearly is taken as the feature vector, and the rotation operator is constructed. This method has the advantages of intuitive geometric meaning, simple calculation and reliable experimental results.

Operation efficiency is also an important index to judge the experimental method, this experiment is carried out on MATLAB r2017a, and the computer configuration is CPU: Intel CeleronG5400 3.7 GHz, Memory: 8.00 GB, operating system: Windows 10 64 bit.

Comparing the data in Table 4, we can find that, the average time of this algorithm is much smaller than that of other algorithms. These algorithms are ICP registration methods proposed by N. Senin et al., Improved ICP image registration algorithm proposed by Weimin Li et al., and the mutual information maximization method proposed by Viola P. Because the geometric invariants based on Clifford algebraic space reduce the operation scale and complexity to a certain extent, so the time of registration is greatly shortened.

Table 4. Comparison of registration speed

6 Conclusion

This experiment is based on the skull contour points, the accuracy of contour points extraction will directly affect the accuracy of the experiment, although the experimental error is within the acceptable range. Therefore, to find more accurate and optimized edge extraction method will be a new breakthrough in this paper. And due to the protection of the head, human brain tissue is generally fixed shape and not easy to deform, so the research of registration is rigid registration, in order to better grasp the medical image registration methods, flexible image registration is also an important research topic, and will become a mainstream direction of image registration.