1 Introduction

Image matching is the process of establishing point-to-point correspondence between two images of the same scene (Tang 2014), which is necessary to ensure coherence among the fused images. It is commonly used in IR imaging systems, such as electronic stabilization of sensors, image fusion, multi-frame super-resolution and nonuniformity correction in infrared focal-plane arrays (IRFPA) (Zuo et al. 2012; Liu et al. 2017).

One of the most intuitive types of feature point in images is the corner. Corners carry abundant information on the structure of images and they can be invariant under various geometric transforms. Corner detectors play important roles in many image processing tasks, for example, image matching, motion tracking and objection recognition (Xiao et al. 2016; Clady et al. 2015). There are mainly two categories of corners detectors, one is based on edge feature and the other is based on gray value difference. The latter directly deals with the pixel gray value, avoiding the defects including the edge detection error and the complex curvature calculation (Nachar et al. 2015; Arif et al. 2017). The common detectors include Moravec, Harris and SUSAN etc.

Moravec detector (Moravec 1977) is a simple and fast corner detection algorithm proposed early. However, it is particularly sensitive for edges, isolated points and noise points, which may bring about a high false detecting rate (Lu et al. 2011). Harris detector (Harris and Satephens 1988) is developed from Moravec algorithm, which has advantages including simple calculation, reasonable and stable corner extraction. But several parameters of this algorithm need to be determined manually. Besides, the corner positioning accuracy is unsatisfactory (Panchala et al. 2016). SUSAN detector (Smith and Brady 1997) is proposed by Smith and Brady with a circular mask. This algorithm has a strong anti-noise ability while it is proved to be simple, positioning accurate and fast. However, the threshold of gray value difference is fixed, and the problems of misjudgment and missing detection also need to be solved (Ku et al. 2013).

IR images are taken by IR sensor to record the thermal radiation of different objects. The characteristics of IR images would reduce the influence of external environment, such as light, smog and other condition factors. However, the edges of objects in IR images are blurred because of the low resolution. Moreover, the ambient temperature interference and imperfect thermal imaging system may produce much noise. In this paper, an improved algorithm for IR image matching is proposed based on SUSAN corner detection, against the above problems of traditional SUSAN algorithm. Experimental results illustrate that this method makes the matching accuracy higher and the matching speed faster.

2 SUSAN algorithm and its improvements

2.1 Traditional SUSAN algorithm

SUSAN algorithm is a low-level image processing method proposed by Smith and Brady. The SUSAN operator has a circular mask with a centre pixel which shall be known as the “nucleus”. Pixels within a mask that have the same (or similar) gray value with that mask’s nucleus compose an area of the mask, which is called the “USAN”. As shown in Fig. 1, the USAN area is at a maximum when the nucleus lies in a flat region of the image, it falls to half of this maximum very near a straight edge, and falls even further when inside a corner.

Fig. 1
figure 1

Change of USAN area size

Full size image

The circular mask is placed at each pixel in the image and the gray value of each pixel within the mask is compared with that of the nucleus. The comparison is determined as follows:

$${\text{C}}\left( {{\text{x}}_{0} ,{\text{y}}_{0} } \right) = \left\{ {\begin{array}{*{20}l} {1,} \hfill & {\left| {{\text{I}}\left( {{\text{x}},{\text{y}}} \right) - {\text{I}}\left( {{\text{x}}_{0} ,{\text{y}}_{0} } \right)} \right| \le T} \hfill \\ {0,} \hfill & {\left| {{\text{I}}\left( {{\text{x}},{\text{y}}} \right) - {\text{I}}\left( {{\text{x}}_{0} ,{\text{y}}_{0} } \right)} \right| > T} \hfill \\ \end{array} } \right.$$
(1)

where \({\text{I}}\left( {{\text{x}},{\text{y}}} \right)\) is the gray value of the mask nucleus, \({\text{I}}\left( {{\text{x}}_{0} ,{\text{y}}_{0} } \right)\) is the gray value of any pixel in the mask, T is the threshold of gray difference, C is the output of the comparison. In practice, a more stable and effective formula is commonly adopted instead, that is:

$${\text{C}}\left( {{\text{x}}_{0} ,{\text{y}}_{0} } \right) = {\text{e}}^{{ - \,\left( {\frac{{{\text{I}}\left( {{\text{x}},{\text{y}}} \right) - {\text{I}}\left( {{\text{x}}_{0} ,{\text{y}}_{0} } \right)}}{\text{T}}} \right)^{6} }}$$
(2)

The USAN area of the pixel \(\left( {{\text{x}}_{0} ,{\text{y}}_{0} } \right)\) is calculated with the following formula:

$${\text{n}}\left( {{\text{x}}_{0} ,{\text{y}}_{0} } \right) = \mathop \sum \limits_{{{\text{i}} = 1}}^{\text{num}} {\text{C}}\left( {{\text{x}}_{\text{i}} ,{\text{y}}_{\text{i}} } \right)$$
(3)

where num is the number of pixels contained in the circular mask, and \(\left( {{\text{x}}_{\text{i}} ,{\text{y}}_{\text{i}} } \right)\) is the pixel in the circular mask with \(\left( {{\text{x}}_{0} ,{\text{y}}_{0} } \right)\), which is the nucleus.

Then, the initial response of the corner is calculated as follows:

$${\text{R}}\left( {{\text{x}}_{0} ,{\text{y}}_{0} } \right) = \left\{ {\begin{array}{*{20}l} {{\text{T}}_{\text{g}} - {\text{n}}\left( {{\text{x}}_{0} ,{\text{y}}_{0} } \right),} \hfill & {{\text{n}}\left( {{\text{x}}_{0} ,{\text{y}}_{0} } \right) < {\text{T}}_{\text{g}} } \hfill \\ {0,} \hfill & {\text{others}} \hfill \\ \end{array} } \right.$$
(4)

where \({\text{T}}_{\text{g}}\) is the fixed threshold value (\({\text{T}}_{\text{g}} = \frac{1}{2}{\text{nmax}}\)), nmax is the maximum of the USAN area of the pixel.

At last, the corner points are determined by non-maximal suppression (Wang 2008) in the neighborhood of the candidate corners.

This algorithm is proved to be simple, positioning accurate, fast and has a good anti-noise performance as it does not require gradient operation. However, the SUSAN algorithm still has problems including the fixed gray level difference threshold, wrong detection and missed detection. In this paper, an improved method for image matching is proposed based on the analysis and comparison of the existing SUSAN algorithms.

2.2 Existing improvements

The threshold of gray value difference is an important factor that affects the detection results. In the original algorithm, the threshold generally has a value of 25 and can be reduced to 7 in low contrast ratio images. A single fixed threshold does not apply to the whole image with complex contrast changes, and so many researchers have proposed different methods to adaptively determine the threshold.

Liu and Zhong (2006) got the histogram of the gray value difference of the mask by calculating the gray value differences between each point and the center point in the mask, took the mean value of the gray value difference as the iteration initial value, and then figured out the gray-scale difference threshold in the mask by iteration. Zhang (2009) calculated the adaptive threshold by extracting the gray scale maximum and minimum values of the image. Wang and Zhao (2011) proposed a method of adaptive local threshold based on the contrast between local and global gray standard deviation. Here, we call this method Wang method. The first two methods only use the gray scale variety of the local image to determine the threshold, without taking the global gray-scale distribution into account, which leads to lots of pseudo corners. The last method, although taking the global gray distribution into account, still cannot effectively detect the symmetry corners, and as a result, too many invalid corners greatly affect the matching speed.

On account of the detection insufficiency of complex corners of traditional SUSAN algorithm, Yang et al. (2011) judged complex corners with the number of the gray value saltuses of the pixels on the ring mask. If the USAN area is half of the circular mask, a ring mask is attached to the SUSAN operator. If the brightness of the ring mask changes twice, the pixel is an edge point; if the brightness changes more than twice, the pixel is a corner. This method explains the validity of the complex corner detection in theory, but the improved algorithm cannot locate the pixel with the USAN area of 18 and calculate the saltus number of the ring mask accurately because of the existence of the noise (Wang 2015; Liu 2016).

2.3 Improvements of this paper

Traditional USAN refers to the region of which the gray value is the same as the nuclear pixel grey value in the response circle. Its distribution is not taken into consideration, resulting in misjudgment and missing detection. The improved method is to replace it with the region in the response circle of which the gray value is the same as the nuclear pixel grey value and is adjacent to the nuclear pixel.

In order to avoid the cumbersome calculation, the whole picture is taken as the adaptive region to calculate the adaptive threshold. Grey values of pixels in each column of the image are compared and the min and max n gray values of each column are taken to compute the threshold, which is determined according to the following formula:

$${\text{T}} = {\text{k}} \times \left| {\frac{1}{{\text{s}}} \times \frac{1}{{\text{n}}} \times \left( {\mathop \sum \limits_{{{\text{j}} = 1}}^{{\text{n}}} \mathop \sum \limits_{{{\text{i}} = 1}}^{{\text{s}}} {\text{I}}_{{{\text{i}}_{{\text{max}} } }} - \mathop \sum \limits_{{{\text{j}} = 1}}^{{\text{n}}} \mathop \sum \limits_{{{\text{i}} = 1}}^{{\text{s}}} {\text{I}}_{{{\text{i}}_{ {\text{min}} } }} } \right)} \right|$$
(5)

where T is the threshold, \(\Delta {\text{I}} = \mathop \sum \limits_{{{\text{i}} = 1}}^{\text{n}} \left( {{\text{I}}_{{{\text{i}}_{ \text{max} } }} } \right. - \left. {{\text{I}}_{{{\text{i}}_{ \text{min} } }} } \right)\), representing the absolute contrast of the image, and \({\text{k}} \in \left[ {0.15,0.3} \right],\;{\text{n}} \in \left[ {5,10} \right]\). In consideration of the low contrast of IR image, k can be appropriately reduced. In this paper, k takes 0.1, s takes 10. When the value of T occupies 15–30% of ΔI, the corners can be better extracted.

It is found that the adaptive thresholds of the two images to be matched will detect many non-matching corners and lead to a high false matching ratio, especially the two images with large differences in the gray value range. However, with the smaller adaptive threshold extracted from one of the images to be matched, more valid corners can be detected while the matching efficiency and accuracy are improved.

In addition, a double ring mask, reflecting the local structure of the nucleus better and judging the corner more accurately, is attached to the traditional SUSAN circular mask, as shown in Fig. 2. Firstly, traverse the image with the circular mask and calculate the USAN area \({\text{n}}\left( {{\text{x}}_{0} ,{\text{y}}_{0} } \right)\) for each pixel. Secondly, find the saltus number of pixel gray values on the double ring mask, when \({\text{n}}\left( {{\text{x}}_{0} ,{\text{y}}_{0} } \right)\) is equal to the geometric threshold value \({\text{T}}_{\text{g}}\). If the saltus number is greater than 2, mark the nucleus as a candidate corner and recalculate its corner response.

Fig. 2
figure 2

Double ring mask. a Inner ring mask, b outer ring mask

Full size image

Calculate the USAN area approximately with the following formula:

$${\text{S}}_{\text{USAN}} = {\text{S}} \times \frac{{{\text{H}}_{\text{USAN}} }}{\text{H}}$$
(6)

where \({\text{S}}_{\text{USAN}}\) denotes the area of USAN obtained with the ring mask, S denotes the circular mask area, H denotes the total length of the double ring mask, and \({\text{H}}_{\text{USAN}}\) denotes the number of pixels in the double ring mask in the USAN area. Then put \({\text{S}}_{\text{USAN}}\) into the formula (4) to obtain the corner response value.

After obtaining the initial response value of each pixel, it is easy to generate pseudo corners by using the non-maximum suppression method in the neighborhood to judge whether the point is the corner point. So the principle of gravity is used to suppress pseudo corners.

The center of gravity \(\left( {{\text{x}}_{\text{g}} ,{\text{y}}_{\text{g}} } \right)\) of the USAN at \(\left( {{\text{x}}_{0} ,{\text{y}}_{0} } \right)\) in image I is:

$${\text{x}}_{{\text{g}}} = \frac{{\mathop \sum \nolimits_{{{\text{i}} = 1}}^{{{\text{num}}}} {\text{x}}_{{\text{i}}} \times {\text{C}}\left( {{\text{x}}_{{\text{i}}} ,{\text{y}}_{{\text{i}}} } \right)}}{{{\text{n}}\left( {{\text{x}}_{0} ,{\text{y}}_{0} } \right)}},\;{\text{y}}_{{\text{g}}} = \frac{{\mathop \sum \nolimits_{{{\text{i}} = 1}}^{{{\text{num}}}} {\text{y}}_{{\text{i}}} \times {\text{C}}\left( {{\text{x}}_{{\text{i}}} ,{\text{y}}_{{\text{i}}} } \right)}}{{{\text{n}}\left( {{\text{x}}_{0} ,{\text{y}}_{0} } \right)}}$$
(7)

where num is the total number of pixels contained in the circular mask. The Euclidean distance between the center of gravity and the nucleus and the specified threshold \({\text{T}}_{1}\) are compared. If the result is in line with the following formula, the initial response value is calculated.

$$\left( {{\text{x}}_{\text{g}} - {\text{x}}_{0} } \right)^{2} + \left( {{\text{y}}_{\text{g}} - {\text{y}}_{0} } \right)^{2} > {\text{T}}_{1}$$
(8)

Within the neighborhood centered on \(\left( {{\text{x}}_{\text{i}} ,{\text{y}}_{\text{i}} } \right)\), where \({\text{i}} \in \left[ {{\text{d}} - 1,{\text{N}} - {\text{d}}} \right]\), if \({\text{R}}\left( {{\text{x}}_{\text{i}} ,{\text{y}}_{\text{i}} } \right)\) is the maximum, \(\left( {{\text{x}}_{\text{i}} ,{\text{y}}_{\text{i}} } \right)\) will be the corner.

In this paper, the USAN neighborhood interconnection criterion and the adaptive threshold are combined to improve the accuracy and adaptability. For the complex corner, a double ring mask is added and the approximate calculation of the corresponding corner response is proposed. Finally, the center of gravity principle is used to suppress pseudo corners. The corner detection performance of this improved method is better than the SUSAN algorithm proposed by Tang et al. (2014), which combines the single ring mask with the iterative method to extract the adaptive threshold. Here, we call

3 Comparison and analysis of experimental results of image matching

After corner extraction, the corners need to be matched. In this paper, the similar measure NCC (Normalized Cross Correlation) is chosen to carry out rough corner matching. Assuming that a window in the image I1 is w(x, y) and the corresponding region in the image I2 is T(x, y), the similar measure between the two input images can be expressed by the formula:

$${\text{S}} = \frac{{\iint {\left\{ {{\text{I}}_{2} \left[ {{\text{T}}\left( {{\text{x}},{{\text{y}}}} \right)} \right] - {\bar{{\text{I}}}}_{2} } \right\}\left\{ {{\text{I}}_{1} \left[ {{\text{T}}\left( {{\text{x}},{{\text{y}}}} \right)} \right] - {\bar{{\text{I}}}}_{1} } \right\}{{\text{w}}}\left( {{\text{x}},{{\text{y}}}} \right){{\text{d}}}_{{\text{x}}} {{\text{d}}}_{{\text{y}}} }}}{{\sqrt {\iint {\left\{ {{\text{I}}_{2} \left[ {{\text{T}}\left( {{\text{x}},{{\text{y}}}} \right)} \right] - {\bar{{\text{I}}}}_{2} } \right\}{{\text{d}}}_{{\text{x}}} {{\text{d}}}_{{\text{y}}} }} \sqrt {\left\{ {{\text{I}}_{1} \left[ {{\text{T}}\left( {{\text{x}},{{\text{y}}}} \right)} \right] - {\bar{{\text{I}}}}_{1} } \right\}{{\text{w}}}\left( {{\text{x}},{{\text{y}}}} \right){{\text{d}}}_{{\text{x}}} {{\text{d}}}_{{\text{y}}} } }}$$
(9)

Then the RANSAC (Random Sample Consensus) algorithm is used to filtrate and optimize the matching point pairs to complete the image matching.

The test object is an IR image sequence, in which the relative position of the vehicle changes and the remaining objects remain static. Image size in pixels is 320 × 256. Test computer configuration: CPU of AMD A6 2.1 GHz, memory of 4.0 GB.

3.1 Image matching based on different corner detection methods

Figure 3 shows the image matching based on different corner detection methods, where (a) shows the image matching based on the Harris method (Zhang et al. 2017), (b) shows the image matching based on the proposed method. Table 1 compares the results of the image matching based on the different corner detection methods.

Fig. 3
figure 3

Image matching based on different corner detection methods. a Harris method, b the proposed method

Full size image
Table 1 Image matching results based on the different corner detection methods
Full size table

The experiments show that corners detected with the proposed method are more accurate, the number of homonymy corners from image matching is bigger, and the matching accuracy rate is higher than which with Harris method. The detection time of Harris method is less than which of the proposed method while the matching time is longer. Therefore, as far as the total image matching time is concerned, the proposed method is better than Harris method. Above all, it is verified that the SUSAN corner detection algorithm has a great advantage in IR image matching.

3.2 Image matching based on different SUSAN methods

Figure 4 shows the image matching results based on different SUSAN methods, where (a) is the image matching based on traditional SUSAN method, (b) is the matching based on Wang method, (c) is the image matching based on Tang method, (d) is the image matching based on the proposed method. Table 2 compares the results of image matching based on different SUSAN methods.

Fig. 4
figure 4

Image matching based on different SUSAN methods. a Traditional SUSAN method, b Wang method, c Tang method, d the proposed method

Full size image
Table 2 Image matching results based on different SUSAN methods
Full size table

It can be seen from the experimental results that the accuracy rates of image matching based on the proposed method is the highest. Though the number of homonymy corners is the largest based on Wang method, the matching accuracy is unsatisfactory, owing to the detection of excessive invalid corners.

In these five methods, the detection time of the proposed method is the longest, but makes little difference with the others. And the total matching time of the proposed method ranks second only to that of the original SUSAN algorithm on account of the fast corner matching. The results show that the image matching performance of this improved algorithm is better than other SUSAN algorithms.

4 Conclusion

In this paper, a variety of SUSAN corner detection algorithms are comparatively analyzed. In allusion to the problems of SUSAN algorithm such as low accuracy of corner detection, fixed gray threshold, false detection and leakage detection, an adaptive improvement is proposed for image matching requirements:

  1. (1)

    The traditional USAN is replaced with the region that has the same gray value with the nuclear pixel and is adjacent to it in the response circle, and the principle of gravity is used to suppress false corners.

  2. (2)

    The total graph is taken as the adaptive region, the pixel gray levels of each column are compared, and the maximum and minimum is taken to calculate the adaptive gray value difference threshold. The corners of two images are separately detected with the minimum of the adaptive thresholds extracted from two images. The adaptability and accuracy of image matching are improved.

  3. (3)

    When the USAN area is equal to the geometric threshold, an attached double ring mask is used to judge whether the pixel is an edge point or a corner. After the pixel being judged as a candidate corner, the USAN area is recalculated approximately to calculate the corner response.

  4. (4)

    The principle of gravity is used to suppress false corners.

On the basis of improvement, two groups of contrast experiments are accomplished, which are the image matching based on different corner detection methods and the image matching based on different SUSAN methods. Experiments show that the proposed method improves the matching accuracy and matching speed effectively.