Image feature detection algorithm based on the spread of Hessian source

Abstract

Image feature detection can be obtained from many methods including the feature point detection. This paper adopts the image feature point detection method based on second-order characteristics of point and the image feature detection algorithm based on the Hessian matrix to detect more feature points. By combining the gray-scale-based image-matching technology with the feature-based image feature detection technology, we propose a Hessian algorithm to obtain more matching points, which can search for matching more quickly. The proposed algorithm overcomes the traditional matching methods that have Ergodic properties of the search strategy. Experiments demonstrate the speed and accuracy of the proposed algorithm, and we use the correct detected feature points to realize image registration, image fusion and image stitching.

1 Introduction

An image is the main source of that human accesses and exchanges information, so that people pay more and more attention to image processing [1–6, 21–27]. In these processes, Image feature detection is a key problem of image processing. Although its development is still very short, it brings the remarkable achievements which had already been widely used in the almost all image-related areas, such as industrial inspection, traffic management, and remote sensing, and it has been a technology that is closely related with our life. The image feature detection is to find the spatial relationship between the two different images [1]. Image feature detection includes image point feature, line feature, curve feature, semantic profile feature, regional feature, and so on. All the features can be used for image matching and mosaicing, image searching and recognition, image data compression, image data mining and information hiding, and object tracking and detection. What’s more, image feature detection is a fundamental technology in image processing and computer vision, e.g., through image point feature detection we can realize image matching and mosaicing with the characteristics of rigid, affine, perspective and irregular transformation. In addition, the point feature is the basic feature unit for others. We can get other specific features by using the top algorithms if many enough point features can be found in image.

With the widely spread developments of image feature detection, new requirements and applications gradually come into being. The research on image feature detection algorithms also gradually deepens. Requirements for image processing technology are improved as well, in order to pursue a more rapid, accurate, and efficient image feature detection algorithm. The traditional methods have their advantages and disadvantages. When applying them, we should select the suitable registration method according to the actual situation as they cannot detect many enough feature points in some indistinct images. These images will be detected well with our algorithm.

Hessian matrix is an analysis tool, and is widely used in engineering. Hessian matrix, whose independent variable is vector, is a block-diagonal matrix, consisting of second-order partial derivative of a real function. It is a second-order partial derivative matrix. If R represents a real, and Rn represents n dimensional real space, then f is a second-order differentiable function from X to Rn, where X is a subset of R. Then, Hessian can be described as the element lying in the ith row and jth column across partial derivatives of the ith parameters and jth parameters of f [2]. In a word, Hessian matrix is used to judge whether a point is an extreme point or not, with its concrete methods to shuffle up a matrix from second-order partial derivative of multivariate function, instead of a matrix obtained by derivation. In fact, Taylor is the result of Hessian matrix expanded, but we change the form by putting it in the matrix form. We take the maximum for negative definite, while minimizing for positive definite.

In this paper, the main contributions are as follows:

• (i) Employing the properties and usages of Hessian matrix;

• (ii) Employing the thermal dissipation technology and construction method;

• (iii) Utilizing the Hessian matrix determinant and trace to construct a new thermal dissipation method;

• (iv) Utilizing the new Hessian diffusion structures to construct a feature detection algorithm;

• (v) Utilizing the peripheral reciprocal inhibition model to construct a feature detection algorithm.

2 Related work

2.1 Hessian technology

Hessian is a lightweight remoting on http tool, although it uses a simple method but realizes RMI functionality. Compared with Web service, Hessian is simpler and faster. Hessian uses the binary RPC protocol, which is suitable for sending binary data.

In mathematics, Hessian matrix is a block-diagonal matrix which makes up for the second-order partial derivative of the real-valued function, whose independent variables is vector. This function is as follows:

$$ f(x_{1},x_{2}, \ldots ,x_{n}) . $$

(2.1)

If the whole second derivatives of f exist, then Hessian matrix of f is:

$$ H(f)_{ij}(x) = D_{i}D_{j}f(x) . $$

(2.2)

Here, x = x (x ₁, x ₂,…, x _n ) is:

$$ H(f) = \left[ \begin{gathered} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \frac{{\partial^{2} f}}{{\partial x_{1}^{2} }}{\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \frac{{\partial^{2} f}}{\partial x_{1}\partial x_{2}}{\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \cdots {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \frac{{\partial^{2} f}}{\partial x_{1}\partial x_{n}} \hfill \\ \frac{{\partial^{2} f}}{\partial x_{2}\partial x_{1}}{\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \frac{{\partial^{2} f}}{{\partial x_{2}^{2} }}{\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \cdots {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \frac{{\partial^{2} f}}{\partial x_{2}\partial x_{n}} \hfill \\ {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \vdots {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \vdots {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \ddots {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \vdots {\kern 1pt} \hfill \\ \frac{{\partial^{2} f}}{\partial x_{n}\partial x_{1}}{\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \frac{{\partial^{2} f}}{\partial x_{n}\partial x_{2}}{\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \cdots {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \frac{{\partial^{2} f}}{{\partial x_{n}^{2} }} \hfill \\ \end{gathered} \right]. $$

(2.3)

So, the second derivative of multivariate function is a Hessian matrix.

Mixed partial derivatives of Hessian matrix are the elements on the non-main diagonal of Hessian matrix. If they are continuous, there are no differences between the sequences of the derivations, then:

$$ \frac{\partial }{\partial x}\left( {\frac{\partial f}{\partial y}} \right) = \frac{\partial }{\partial y}\left( {\frac{\partial f}{\partial x}} \right). $$

(2.4)

The formula above can also be written as

$$ f_{xy} = f_{yx} $$

(2.5)

Formally, if f function is continuous in the region D and the second derivative exists everywhere, then the Hessian matrix of f in the region D is the symmetric matrix.

Given a continuous second derivative function f: R ² → R, the Hessian matrix determinant can be used to identify the critical point that belongs to the saddle point or extreme point [3].

If the critical point of f is (x ₀, y ₀), then $\frac{{\partial f(x_{0} ,y_{0} )}}{\partial x} = \frac{{\partial f(x_{0} ,y_{0} )}}{\partial y} = 0$. But according to its first derivative,

$$ H = \left| \begin{gathered} {\kern 1pt} {\kern 1pt} \frac{{\partial^{2} f}}{{\partial x^{2} }}{\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \frac{{\partial^{2} f}}{\partial x\partial y} \hfill \\ \frac{{\partial^{2} f}}{\partial y\partial x}{\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \frac{{\partial^{2} f}}{{\partial y^{2} }}{\kern 1pt} \hfill \\ \end{gathered} \right| = \frac{{\partial^{2} f}}{{\partial x^{2} }}\frac{{\partial^{2} f}}{{\partial y^{2} }} - \left(\frac{{\partial^{2} f}}{\partial y\partial x}{\kern 1pt}\right)^{2} . $$

(2.6)

We can know that it is a saddle point, local maximum point or local minimum point by using Hessian matrix.

• (i) H > 0: If $\frac{{\partial^{2} f}}{{\partial x^{2} }} > 0$, then (x ₀, y ₀) is a local minimum point; if $\frac{{\partial^{2} f}}{{\partial x^{2} }} < 0$, then it is a local maximum point.

• (ii) H < 0: (x ₀, y ₀) is a saddle point.

• (iii) H < 0: The second derivative cannot tell the properties of the critical point, wanting for a higher order derivative relied on the Taylor formulation [4].

It can be concluded that when an image is seen as a rational set with discrete gray values, to be specific, when taking image data as matrix to analyze, all the image information is hiding in the mathematical space of image matrix. According to the image original mechanism, image information includes the relevant information from one or all of the local regional information, the multi-regional coupling information, and the image whole area information. Therefore, the image feature detection can be transferred into mathematical and physical methods that analyze on the above information. Furthermore, that analysis results directly depend on the tools, algorithms and procedure of analysis. While the analyzing tool is the precondition of the later two, and Hessian is just a perfect analyzing tool.

When analyzing the image with mathematical and physical methods, two means can be adopted. One is to compute the closed solution by using the equation transform method on the image local or whole information, and the other is to conduct the direct analysis based on templates or operators. In terms of the target of rapid analysis, the former cannot meet the actual need of rapid calculation and communication, while the latter exhibits obvious advantages on analyzing time and visualization. Furthermore, the image point feature detection requires to build an operator which can exhibit the image local and deep information. On multi-scale condition, the deeper information represents the evolve speed of image structure, and we can hold this speed in multi-angle analysis from static to dynamic. In addition, the accelerated speed information can describe the morphological changes of image structure, while Hessian is just an operator which takes the acceleration as the indicators. What’s more, the Hessian has the change capability of the local area within an accurate location of 4 × 4. Through this capacity we can make sure the relationship between a specific pixel and its neighbors. It can also conduct similar detection on a bigger size matrix based on matrix transformation and is more suitable for the multi-scale image point detection.

2.2 Optimization of image feature detection algorithm

Currently, the research based on image feature detection can be seen as an interface or a channel of information where users access image database. A typical system consists of five parts, namely query, description, matching, extraction, and verification.

The image search methods mainly focus on the low-level physical properties, such as color, shape, contour, and texture extraction, and high-level semantic retrieval property of objects which can be divided into two categories: one for a single object or the description and identification of objects, such as color, shape, structure, texture; the other for describing objects’ composition or 3D information of objects, such as distance, angle, etc. [5].

We extract feature points by image feature detection. Feature point is a class of local point features, i.e., image gray in the X-axis direction and Y-axis direction, which have both enormous changes, including corner, inflection and T-intersection, etc. Among them the image corner is the most important, because it is a pixel point that the gray transforms sharply and has obvious difference with the adjacent vertex around. It has a great effect on the field of object description and recognition, image matching and motion estimation, target tracking, and so on [6]. There are many manners for a corner detection algorithm, but their effects are not the same in different types of corner detection. Corner detection algorithm mainly includes two categories. One is the corner detection algorithm in the image based on image edge, which needs image segmentation and edge extraction. Because of absolute challenges in image segmentation and edge extraction, there are considerable difficulty and computational complexity. Once the edge line is interrupted, it will profoundly affect the corner extraction results; the other is corner detection based on image gray, considering the state of gray change about pixel, instead of the entire target corner detection, in order to avoid the above drawbacks, such as Moravec Corner Detection Operator, Harris Corner Detection Operator, and SIFT Method [7].

This paper involves the optimization of the image feature detection algorithm, extraction and matching of applying Hessian diffusion-based image feature detection algorithm for image features [8]. Based on the previous image, we first use feature-based image feature detection technology, and then run the SIFT algorithm to further improve in detection technology. Therefore, it can get more matching points by the Hessian algorithm that could be more quickly matched.

Image scale and affine invariant feature detection is an important and difficult problem in image processing. It is easy to conduct the feature point detection under rigid and affine transform condition. However, there exists deficiency and lack of control on the number, stability and representativeness of feature points, which result in a need and space for improvement in feature analyzing, matching and mosaicing for some range images. Hessian can make up for the insufficient of existing methods [18]. More important, there is no method to deal with the invariant feature in perspective, uneven transformation, and local occlusion. The Hessian multi-scale method can mine these feature points through the multi-scale computation on point, local area and the whole area, it especially exhibits great advantages on number, stability and controllability.

2.3 Image mosaics application

Image mosaics technology is an imaging technology that has some overlapping part images that are large, seamless and with high resolution. These images may be gained in different ways, i.e., perspectives or sensors [9]. Among them, image registration and image fusion are the cores of the image mosaic. Generally, the computational complexity of the image registration algorithm is considerable. The development of the image mosaics technology primarily depends on improving the image registration technology. As same as the former, it mainly uses the point matching method. These methods are slow and have low resolution, and they often need manual selection of the initial matching point, which is unable to adapt to a lot of data fusion. There are various image mosaics methods, in which the process is similar to a certain extent, as shown below: (see Fig. 1 ).

Fig. 1

Basic flow chart of mosaics applications

• (i) Image acquisition mode

The image mosaics technology relies on the overlapping part of an image which converges and strives for a panoramic image of high resolution. These overlapping images are generally obtained from the two ways: one is to fix camera axis, and rotate pictures about the axis; the other is to fix to the camera’s lens pictures of the horizontal interlaced image jitter in optical center. The former is mainly used for vision or remote sensing image acquisition, while the latter is mainly used for microscopic image acquisition. They have in common the collecting of the two-dimensional overlapping image.

• (ii) Image pretreatment

Geometrical distortion will cause serious problems [10]. For example, in the two original images, the same objects are distorted, which will lead up to failing to match and thus cause the image registration error. Therefore, solving the problem of geometric distortion is particularly important. The basic idea of image correction is: to establish mathematical model, to extract the required information, and to restore the real feature of the distorted image according to the opposite process. Actual recovery process is to design a filter [11], which can obtain the estimation of the real image by calculating and following the predetermined error criterion to farthest approximate lifelike images.

Image noise can be defined as the deviation between a real signal and an idea signal. In other words, it is understood as various factors that hinder human visual perception or interfere with the system sensor on analysis of the information received. In general, noise is a random signal and also unpredictable, usually using the probability statistical method for analysis. Noise is essential on the image processing, involving all aspects of the image processing, especially in inhibiting the image input and acquisition of noise, and it is a very important issue. According to the noise source, it can approximately be divided into external noise and internal noise; at the same time, from the statistical point of view in the noise defined, it can also be divided into stationary noise and non-stationary noise. If the noise inputted is accompanied with a larger sound, it is bound to affect the whole process of image mosaic and leads to reduce the image’s accurate rate. Various types of noise images on the screen can be divided into two categories: one is generally salt and paper noise. Its amplitude is essentially the same, but the position of noise is random; the other is contrary. The noise presents in every point, but the amplitude of the noise is randomly distributed. Density function from the size distribution of the noise amplitude is the Rayleigh type and Gaussian type, which are, respectively, called Rayleigh noise and Gaussian noise. Another example is White noise whose spectrum is of the uniform distribution.

Image registration is defined as a process of optional matching for two or more images obtained from different sensors, times or angles. Briefly, image registration is aligned with the position between the images. For the task of clearer image registration, the image registration problem uses more precise mathematical language to describe. Registration can be used as follows: Set the same scene from different angles or at different times to obtain the similarity measure S (I₁, I₂) between two images I₁, I_2, and then searching the corresponding points from I₁, I₂ to determine the images optimal transformation T between images so that S (T (I₁), I₂) reaches a maximum.

Image registration always aims at multiple images. In the actual process, it often chooses one of images termed the reference graph as the reference of registration, while the others are called as the search graph. The operation steps that we take in image registration are generally to choose an image sub-block centered on target point in reference graph firstly, which is called printing form of image registration. What’s more, we use the printing form which moves constantly in an orderly manner within the search graph, and compare it with the corresponding position within the search every time until reaching the track of the registration position. If there are two identical target images in painting form, then the image registration will be easy to complete. However, in fact, the same objects in the two images for image registration are often obtained according to different conditions. If the environment, such as different imaging position, different imaging time and even different imaging system, adds a variety of noise role, then it will make the same objects unable to be exactly same; in other words, two images can only be similar to a certain degree. So the image registration is actually a fairly complex technology project.

• (iii) Feature-based registration

The idea of the ratio matching method is to use a pixel ratio which is a portion of two columns in the image as a painting form, that is to say, to obtain a portion of the pixel ratio from the overlap region in the reference graph T, then to calculate its ratio as painting form, and finally to find the optimal marching from the search graph S. Matching process is to calculate the corresponding pixel value one by one in the process of search from left to right, i.e., the distance between two identical things. Then, these ratios are compared with the painting form and the minimum differentiation between the corresponding rows is the optimal matching. In contrast, only a set of data that uses a two-column pixel and regional information is applied.

Ratio matching method has the advantages:

• The idea of the algorithm is clear, simple and easy to be understood and implemented.

• A few calculations and high speed.

Ratio matching has its disadvantages:

• Lack of using the image feature information. It only applies two vertical–parallel characteristic line pixels, without taking the most portion of image feature information into consideration. Although it is mentioned that calculating the corresponding pixel value one by one in the search from left to right, it can compare for the painting form, and it seems to use the most portion of image feature information within the search graph S. But in the reference image T, two characteristic lines are arbitrarily chosen, instead of making the most of overlap region’s characteristics.

• Image acquisition requirement is high. In order to sole the smaller shifting down, the algorithm compares the photo in vertical firstly, and then compares it in the horizontal. However, sometimes there is a shallow rotation of the camera lens, so that the algorithm where the pictures are taken in a rotation degree cannot be resolved. Without obvious characteristics, the images all have the overlap region, such as sea background and sky background, which results in a big problem when we choose features of the painting form. Because there are the largely same textures in the picture, which have small differences with the painting form, it will easily lead to matching errors.

• Difficult to determine the distance between two characteristic lines. Without selection conditions, the algorithm is to choose a portion of pixels from two portions within the overlap region of the reference image T. However, after the stitching algorithm realizing automatic stitching, if the characteristic line is not appropriately selected, it will lose its true value as a painting form. Meanwhile, if setting separation distance is too large, the image information of overlap region cannot be made full use of. On the contrary, the calculation is a lot.

Feature point matching method mainly refers to the obvious points in an image, such as housing corners, circle point, etc. Obviously, the ratio matching method only uses relatively rare and limited image features, and it is easy to lead to error when a shallow rotation happens. For such problems, the matching method based on feature points can be better resolved. The operator used for interest point detection is called as the beneficial operator or interest operator. Since the 1970s, a series of varied and characteristic interest operators have been generated, famously such as Moravec operator, Harris operator and Foistner, etc. [12].

This paper, based on the SIFT framework, uses Harris adding Hessian operator for interest point detection. In this paper, we first describe the development environment required to use in the development process, and then describe its associated technologies, such as Matlab language and SIFT framework. Finally, the key issues encountered and solutions are discussed, including the optimization of the feature detection algorithm, quickly finding out feature points more and stably, and image stitching applications as well.

3 Feature detection algorithm based on Hessian diffusion

This section describes in detail the various specific algorithms and their implementation steps. We also introduce new image extraction, smooth processing, Gaussian and Gaussian difference scale-space calculation, image matching, image mosaics algorithm, and so on [13].

3.1 Overview of image feature detection

Image features are remarkable basic features or called characteristics in the image. Characteristics of the image can be divided into three categories: point, line, and surface. Point feature is the most common image features, including corner, edge, crossing point, etc. [14]. Generally, the point detection methods of interest are:

• (i) Interest operator method

Interest points are pixels, which are characterized by having a unique performance in the corresponding region, and therefore it can be extracted quickly. Famous interest point extraction operators include Moravec operator, Harris operator, Forster operator, and so on.

• (ii) Edge point extraction method based on wavelet

By applying to scale analysis theory based on wavelet, edge point extraction methods first use the image for wavelet and calculate modulus values of wavelet transform, and then make the local maximum be the edge points of the image.

• (iii) Corner detection

The corner detection method is based on the gray detection method, and calculates the gradient and curvature [15] to detect the various corners of the image by considering the pixel gray changes in various fields. Traditional edge detection is based on the remarkable feature where the gray in image edge changes the most greatly. By using the differential substitution derivative operation, each pixel of the image performs differential or second-order differential to capture the edge pixels. When it comes to the simple first-order derivative operation of images, because it is fixed directivity, merely the edge of a specific direction can be detected, thus leading to be universal. Line feature is evident in the image, such as the edges of buildings, the edges of roads, and so on. Firstly, line feature extraction is used to extract some clear line information in the image, and moreover employed to screen qualifying line segments based on limited conditions as line features. The common line extraction operator has the pixel differentiating operator, including gradient operator, differential operator, directional differential operator, Roberts operator, Sobel operator, and Prewitt operator. In addition, the operator using second-order differential includes directional second-order differential operation, Gauss–Laplace operator, Laplace operator, Wavelet operator, Canny operator, and so on.

3.2 Hessian algorithm

This paper introduces the optimization of image feature detection algorithm, which applies the Hessian diffusion to study the image feature extraction and matching. Based on the previous image research, we first perform the SIFT algorithm and then offer to Hessian algorithm to get more matching points, so that the matching can be searched more quickly. Through the experiments, we confirm the rapidity and accurate of algorithm, because the Hessian matrix obtained by using the SIFT image local value is more stable in the MATLAB environment. Based on the SIFT algorithm, the picture threshold is set for the Hessian implementation, so that feature point matching found in images is relatively stable [16].

First of all, we find the original image, and calculate the H value for each pixel to form a new image; what is more, we calculate the H value for each pixel in Gaussian to form a corresponding new image; finally, we find DOG images, and calculate the H value for each pixel in every DOG to form a corresponding new image. The steps discussed above will produce different image results after combining.

We can obtain I = Ixx × Iyy − Ixy × Ixy based on Hessian matrix

$$ H = \left| \begin{gathered} {\kern 1pt} {\kern 1pt} \frac{{\partial^{2} f}}{{\partial x^{2} }}{\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \frac{{\partial^{2} f}}{\partial x\partial y} \hfill \\ \frac{{\partial^{2} f}}{\partial y\partial x}{\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \frac{{\partial^{2} f}}{{\partial y^{2} }}{\kern 1pt} \hfill \\ \end{gathered} \right| = \frac{{\partial^{2} f}}{{\partial x^{2} }}\frac{{\partial^{2} f}}{{\partial y^{2} }} - \left(\frac{{\partial^{2} f}}{\partial y\partial x}{\kern 1pt} \right)^{2} . $$

(3.1)

Then according to Harris-Hessian, we further obtain Eq. (3.2):

$$ I = Ixx\; \times Ixx \times (Iyy\; \times Iyy) - (Ixy \times Iyx) \times (Ixy \times Iyx) - \alpha \times (Ixx \times Ixx + Iyy \times Iyy). $$

(3.2)

The variable α is better controlled between 0.1 and 0.5.

3.3 Heat conduction equation

3.3.1 Physical model

In the three-dimensional space, we take uniform and isotropic objects into consideration, assuming it is an internal heat source or sink, and has heat exchange with the medium surrounding. It offers a way to study the temperature distribution inside the object.

3.3.2 Fourier theory

The heat dQ, where objects along the normal direction n flow into a infinitesimal area dS in the infinitesimal period Δt, is proportional to the directional derivative $\frac{\partial u}{\partial n}$ where the object’s temperature goes along the normal directional of the surface dS (see Fig. 2).

Fig. 2

Schematic diagram

Note that the minus is because the heat always flows from the high temperature to the low temperature, which results in the opposite sign between dQ and $\frac{\partial u}{\partial n}$.

Take any a closed surface inside object to be G, the area surrounded denotes as Γ, and the area surrounded denotes as Ω.

The heat flows into a closed surface Γ from time t ₁ to time t ₂ is:

$$ Q = \int_{{t_{1} }}^{{t_{2} }} {\left\{ {\int\limits_{\varGamma } {\int {k(x,y,z)\frac{\partial u}{\partial n}} } dS} \right\}} dt . $$

(3.3)

In the time interval [t ₁, t ₂], the temperature changes from u(x, y, t ₁) to u(x, y, t ₂), its heat absorbed is:

$$ \iiint\limits_{\varOmega } {c(x,y,z)\rho (x,y,z)[u(x,y,t_{2} ) - u(x,y,t_{1} )]}dxdydz. $$

(3.4)

Here, cq represents the specific heat; and ρ represents the density.

3.3.3 No heat source object internal equation

According to the heat conservation law, when the object internal is no heat source, we can get the following equation [17]:

$$ {\rm Heat} - {\rm heat} = \inf {\rm low} $$

$$ t = t_{2} \,t = t_{1} \,\,t_{1} \le t \le t_{2} $$

(3.5)

$$ \begin{gathered} \int_{{t_{1} }}^{{t_{2} }} {\iint\limits_{\varGamma } {k(x,y,z)\frac{\partial u}{\partial n}}} dSdt \hfill \\ = \iiint\limits_{\varOmega } {c(x,y,z)\rho (x,y,z)[u(x,y,t_{2} ) - u(x,y,t_{1} )]}dxdydz \hfill \\ \Rightarrow \hfill \\ \int_{{t_{1} }}^{{t_{2} }} {\iiint\limits_{\varOmega } {\left[ {\frac{\partial }{\partial x}\left( {k\frac{\partial u}{\partial x}} \right) + \frac{\partial }{\partial y}\left( {k\frac{\partial u}{\partial y}} \right) + \frac{\partial }{\partial z}\left( {k\frac{\partial u}{\partial z}} \right)} \right]}} dxdydt \hfill \\ = \iiint\limits_{\varOmega } {c(x,y,z)\rho (x,y,z)\int_{{t_{1} }}^{{t_{2} }} {\frac{\partial u}{\partial t}} }dtdxdydz \hfill \\ \end{gathered} . $$

(3.6)

Exchange integral order:

$$ \int_{{t_{1} }}^{{t_{2} }} {\iiint\limits_{\varOmega } {\left[ {c\rho \frac{\partial u}{\partial t} - \frac{\partial }{\partial x}\left( {k\frac{\partial u}{\partial x}} \right) - \frac{\partial }{\partial y}\left( {k\frac{\partial u}{\partial y}} \right) - \frac{\partial }{\partial z}\left( {k\frac{\partial u}{\partial z}} \right)} \right]}} dxdydz = 0. $$

(3.7)

Here, all t ₁, t ₂ and Ω are arbitrary, and the homogeneous equation of the heat conduction is:

$$ c\rho \frac{\partial u}{\partial t} - \frac{\partial }{\partial x}\left( {k\frac{\partial u}{\partial x}} \right) - \frac{\partial }{\partial y}\left( {k\frac{\partial u}{\partial y}} \right) - \frac{\partial }{\partial z}\left( {k\frac{\partial u}{\partial z}} \right) = 0. $$

(3.8)

3.3.4 Heat source object internal equation

If an object internal has heat source, then we can define as follows:

$$ {\rm Heat} - {\rm heat} = \inf {\rm low} + {\rm formation}\,\,\text{amount} .\,\,t = t_{2} \,\,t = t_{1} \,\,t_{1} \le t \le t_{2} \,\,t_{1} \le t \le t_{2} $$

(3.9)

So the non-homogeneous equation of heat conduction is:

$$ c\rho \frac{\partial u}{\partial t} - \frac{\partial }{\partial x}\left( {k\frac{\partial u}{\partial x}} \right) - \frac{\partial }{\partial y}\left( {k\frac{\partial u}{\partial y}} \right) - \frac{\partial }{\partial z}\left( {k\frac{\partial u}{\partial z}} \right) = f(x,y,z,t). $$

(3.10)

3.3.5 Mathematical model in uniform object

If the object is uniform and isotropic, the mathematical model is written as

$$ \frac{\partial u}{\partial t} = \alpha^{2} \vartriangle u + f(x,y,z,t), $$

(3.11)

where

$$ \alpha^{2} = k/C\rho ,f\left(x,y,z,t = f_{0} /C,\vartriangle = \frac{{\partial^{2} }}{{\partial x^{2} }} + \frac{{\partial^{2} }}{{\partial y^{2} }} + \frac{{\partial^{2} }}{{\partial z^{2} }}\right). $$

(3.12)

Especially, one-dimensional case is:

$$ \frac{\partial u}{\partial t} = \alpha^{2} \left( {\frac{{\partial^{2} u}}{{\partial x^{2} }} + \frac{{\partial^{2} u}}{{\partial y^{2} }}} \right) + f(x,y,t). $$

(3.13)

And two-dimensional case is:

$$ \frac{\partial u}{\partial t} = \alpha^{2} \frac{{\partial^{2} u}}{{\partial x^{2} }} + f(x,t). $$

(3.14)

3.4 Diffusion equation

In fact, diffusion is the theory support of using Hessian and the similar matrix, which can be applied to the Gaussian scale-space [19, 20]; otherwise, it will lead that Hessian matrix cannot filter each layer of Gaussian function or DOG function.

Then, why diffusion is sufficient to support it? In simple terms, the diffusion can make the image project into a scale-space. Gaussian function gradually changes in image convolution, thereby generating a series of images, which mimics the diffusion theory, that is to say, it mimics the thermal diffusion theory. Then the image is equivalent to a diathermal medium without putting heat source which can release heat, and if the medium has a non-uniform initial temperature, then it will transfer heat each other in different places at different temperatures because of the temperature difference, which will form a thermal diffusion. Further, the diffusion has the diffusion scale-space. If you want to transfer no heat source into heat source, then you can add a heart source. For example, that Hessian filters the Gaussian scale-space of SIFT algorithm is equivalent to add a heat source of Hessian type in the process without heat source which Gaussian scale-space represents, thus leading to a new generation of scale-space. This property is very different from the original Gaussian space, and they even have the essential difference.

The most basic diffusion question about the image is Gaussian filtering. For the two-dimensional image u(x, y), the classic method is to consider u(x, y) as a homogeneous isotropic (the diffusion coefficient of every point in the image is the same) in the linear diffusion process.

3.4.1 Physical model

Taking a uniform and isotropic object within three-dimensional space into account, by assuming that its internal has diffusion source, we can study the distribution about the concentration of object’s intrinsic molecular at the time t.

3.4.2 Mathematical model

$$ \frac{\partial u}{\partial t} = D\vartriangle u + f(x,y,z,t). $$

(3.15)

Here, u(x, y, z, t) represents the molecular concentration in (x, y, z) at the moment t; f(x, y, z, t) represents population produced every unit volume with every unit time; and D is the diffusion coefficient.

3.5 Procedures and steps of the proposed algorithm

3.5.1 Getting new pictures

Based on SIFT, we use the Hessian matrix equation to extract the stable point, and then generate a new image. Here is a user-defined LZ function, aiming at completing the effects of the transformation image, and producing new images.

3.5.2 Image smoothing

The smoothing technique is used for the noise in the smoothing image, which is designed to eliminate or minimize the effects of noise, thus improving the image quality. Smoothing noise can be used in the space domain, the common methods of image smoothing include multi-image averaging method, neighborhood averaging method, and median filtering method, etc. We use the neighborhood averaging method. Threshold value in the neighborhood averaging method is a gray T. If the gray of a pixel is greater than the average of that in neighborhood and exceeds the threshold value, then we can obtain an average gray override the pixel gray, and its mathematical expression is written as

$$ \text{g} \left( {i,j} \right) = \left\{ {\begin{array}{*{20}c} {f\left( {i,j} \right)_{\text{eq}}\quad if(f\left( {i,j} \right) - f\left( {i,j} \right)_{\text{eq}} > T)} \\ {f\left( {i,j} \right) \quad others} \\ \end{array} } \right. $$

(3.16)

Specific steps are as follows:

• (i) Obtain the image size and the data area, and copy the data area into the buffer;

• (ii) Loop to obtain each point pixel values;

• (iii) Obtain the average value of eight pixels around;

• (iv) Determine whether the point is greater than the average pixel value. If yes, then the average value will be assigned to this point; otherwise, we retain the original pixel gray values.

• (v) The data changed in the buffer are copied to the original data area.

3.5.3 Gaussian difference scale-space

DOG Gaussian differential is to achieve the characteristics for the extraction of object recognition and to calculate differential of different scales-space images, thus obtaining the eigen values and describing the object [4].

3.5.4 Image matching

Image matching is defined to identify the corresponding points from two or more images by a matching algorithm, for example, 2D image matches by comparing to the correlation coefficient which is about the same size window correlation coefficient of the target zone and the search area and takes the central point of the window of the maximum correlation coefficient in the search area as corresponding points. Essentially, when there are similar primitives, it is to take the best matching search criteria.

Image matching method is very rich, it can generally be divided into two categories: gray-based matching method and feature-based matching method. General feature matching primitives include points, lines, areas, and notable feature images. So the matching method based on image feature has the following four categories: point matching, edge matching, closed contour matching, and the use of advanced feature matching. This paper uses feature-based matching method. First of all, we extract features in the original image, and then create the corresponding relationship of matching between two images.

3.5.5 Image mosaics

Image mosaics technology is a technology that splices a number of overlapping images into a large, seamless and high-resolution image, and these images may be obtained from different times, different perspectives or different sensors. There are mainly three steps in it: image preprocessing, image registration, image fusion and edge smoothing.

4 Experimental results and analysis

This section mainly offers to get more matching points by Hessian algorithm, and the more quickly matching search. This algorithm views feature points obtained as a search basis, by changing the search strategy in ergodic properties of the traditional matching method. Through the experiments, it is confirmed that this algorithm is fast and accurate. Finally, we achieve the image mosaic by the correct matching feature points on the realization of image registration and image fusion.

4.1 Experimental results

The experimental environment is a PC machine running in the Windows XP system, where we use MATLAB 7.11.0. Here are two photos called as tower 1 and tower 2 in this experiment. It is the same building taken from different perspectives (see Fig. 3).

Fig. 3

Original chart tower 1 (left) and original chart tower 2 (right)

4.1.1 Image feature detection result of the original SIFT algorithm

To detect feature points, namely stable points, we use the corner detection of image feature detection methods which to find the key points representing the characteristics of the image, but it can be seen to only find some unrepresentative corners as shown in Fig. 4.

Fig. 4

Corner detection

Then to match the two images, Fig. 5 displays the related data specifically matched, the number of matching points found, and so on.

Fig. 5

Matching results from command window

We run the experiments, and match the corners found according to the steps in Sect. 3.5. The similar characteristics are connected. It can be seen in the Fig. 6 that the matching is limited and inexact. Therefore, it is needed to improve as follows.

Fig. 6

Image matching

4.1.2 Image feature detection results of original SIFT algorithm after improving

Figures 7 and 8, respectively, calculate the Gaussian differential scale-space about the two images with the changes of parameter values. The display effects of the images in Figs. 7 and 8 are also different.

Fig. 7

Calculation of Gaussian difference scale-space about figure tower 1

Fig. 8

Calculation of Gaussian difference scale-space about figure tower 2

Figure 9 aims at identifying the frame features. As we see from the image, it obtains much more features than the results obtained after changing the algorithm.

Fig. 9

SIFT frame features selected

In Figs. 10 and 11, the red dot represents the feature points found, while the green circle represents the characteristics domain detected. It can clearly be seen that the characteristics found and got are accurate and abundant.

Fig. 10

Finding the characteristics domain and the feature point of figure tower 1

Fig. 11

Finding the characteristics domain and the feature point of figure tower 2

Figure 12 shows the results after matching. Compared with the previous results, it is more satisfactory, and the matching points are more and more accurate.

Fig. 12

Key point matching

4.1.3 Image mosaics

Based on the Hessian algorithm above, we achieve the panoramic image mosaics. Panoramic image refers to a process where the fixed object images obtained at different times from different perspectives or different sensors are aligned together seamlessly, thus resulting in a broad view and a high resolution image. The perspective of tower 1 in the left of Fig. 4 is lower than that in the right. After the image mosaics technology, they can be synthetic to be an image, thus obtaining the following results (see Fig. 13).

Fig. 13

Image mosaics

4.2 Experimental analysis

From the experimental results above, we can draw the conclusion that if only feature points detected by a single SIFT, then corner is limited and inaccurate, which will lead to serious consequences of subsequent image matching, image registration and image fusion, and even the failure of successfully achieving the basic matching operation. The proposed algorithm is effective immediately after improving with obvious salient points. After adding the Hessian matrix, it will find more stable points faster and more accurately.

5 Conclusions and future work

This paper introduces the Hessian matrix to further optimize the image feature detection technology, where the properties of Hessian matrix and the thermal dissipation technology are employed. We utilize the Hessian matrix determinant and trace to construct the new thermal dissipation method and employ the new Hessian diffusion structures to construct feature detection algorithm. Finally, we conduct the experiments of the new tectonic feature detection algorithm in the applications of image feature detection.

Almost all of the image feature detection techniques have problems. Image feature detection is still an open research topic. In the image feature detection, inspired by the research results of human visual cognitive science, similarity measure which is consistent with human visual perception should be learned. It is desirable to obtain a feature description, which is dimensionless and insensitive to absolute brightness and contrast, and reflects the property of the image. In a word, the image processing and analysis by using phase information for image feature detection is a new research direction, and the theories and algorithms of these technologies need to be further refined and improved.