Keywords

1 Introduction

As a basic feature of the image, linear feature contains a wealth of information and is more stable. So, compared with the traditional gray level matching and point feature matching, linear feature matching has considerable advantages in stability and universality, which makes it become a new research hotspot in the field of image matching. Linear and circular features that can be expressed by mathematics analytic formula were first utilized at early stage [1], while the FFLFs such as coastlines and roads are less applied. This is because such features are composed of a series of continuous irregular edge points, which are difficult to be expressed through mathematical analytic model and difficult to be directly applied in the processing [2].

Compared to point matching, little progress has been made in curve matching (including linear matching) in recent years. Only a few methods for curve matching are reported in literature until now. Xiao et al. [3] reported a novel feature called edge-corner that announced corners as the intersection of two or more edges. The initial matching method was done by comparing patches around the edge corners, using Sum of Squared Differences (SSD) and followed up by an optimization process that modeled a more precise affine model for the matching. Zuliani [4] presented a new physically motivated curve/region descriptor based on the solution of Helmholtz’s equation which satisfies the six principles set by MPEG-7 and can be generalized in order to take into account also the intensity content of the image region defined by the curve. Wu et al. [5] proposed a feature vector field for images, which is built by the inner products and exterior products of image gradients. The feature vector field can effectively represents image edges and feature points including corners and edge points with big curvature. Experiments show that the descriptors have a good adaptability to small image affine transformation. Kingsbury [6] proposed a matching system that took both appearance and spatial constraints into consideration. SIFT descriptor and a pair-wise relationship descriptor was used to measure the similarity between them. Wu [7] presented an integrated point and edge matching method, which incorporates the edge matching with point matching in the same dynamic matching propagation process. It takes advantage of the edge-constrained Delaunay triangulations with the capability of generating point and edge matches preserving the actual textural features. Zhang et al. [8] proposed a novel affine invariant curve descriptor based on membrane vibration model. They mainly focused on Jardon curve and open curve; the experimental results show the proposed method outperforms the existing Fourier descriptor and Zernike moment descriptor. Saeedi and Mao [9] introduced a novel feature-2EC feature and a unique matching technique, the proposed feature utilizes two straight lines and their intersections. It can establish match correspondences between two oblique aerial images under a large projective transformation.

As a conclusion, these existing FFLF matching methods can be generally divided into two types: one is to extract part of the information of the linear features for matching, such as the shape parameter, center of gravity and rotation invariant moment and other features, which have better stability for the change of imaging conditions but will abandon large amount of the feature information, so as to lack high matching accuracy. Another is to try to use all the coordinate information of the edge points [2] to realize image matching by the chain code edge descriptor matching, although this makes full use of the rich information of the edge, the high dimensionality of character description vector and large amount of calculation make the stability depend on the stability of the edge detection to a large extent, and the edge fracture problems are often hard to overcome; additionally, high-precision initial conditions are needed.

In order to better resolve the conflict between the full use and effective description of the rich linear feature information in the study of FFLF matching, this paper proposed a remote sensing image matching method using the hierarchical matching. First, the FFLFs of the image were extracted, from which a variety of stable features were extracted for coarse matching; and finally, sub-pixel edge points with different sampling rates were orderly used to realize the accurate matching.

2 Research Methods

FFLFs matching method based on hierarchical matching mainly consists of such steps as extraction of FFLFs, extraction of features to be matched, coarse matching and accurate matching. Its basic principle is as follows: first carry out sub-pixel edge detection on the image, and then track and refine the edge points, so as to obtain FFLFs with better continuity; then extract the CLFs, LFIs and LFCs and other stable features for coarse matching; eliminate the false matching by using the geometry information and the parameter distribution features of the model determined by the conjugate features combination and then determine the initial value of accurate matching; finally based on ICP, apply sub-pixel edge points with the sampling rate from low to high for accurate matching. The following is the introduction of main principle of the proposed method.

2.1 Extraction of Free-Form Sub-pixel Linear Features

Extraction of FFLFs consists of two main steps, the edge detection and the edge tracking & refining. Because the edge detection precision directly affects the final precision of the matching method based on edge features, with the application of computer vision technology in every field in recent years, people expect to obtain the accurate location and size of the target based on existing image data. Thus, sub-pixel edge detection has received the extensive attention of the researchers [10, 11]. In order to ensure a higher matching accuracy, the method proposed in 11 of the references, sub-pixel edge detection using extremal gradients, is applied to detect the edge of the image; with good universality, this method can carry out better detection on the edges of various types, especially having higher positioning accuracy for the edge of the corner; the principle is detailed as follows.

The extremal gradients include positive gradient and negative gradient, which respectively represents the maximum increase gradient and minimum decrease in the grey level of each image point; the magnitude is determined by the maximum differential of grey level between current image point and its eight neighborhood image points, and the direction is from current image point towards the neighborhood image point corresponding to the maximum difference. By calculating the extremal gradient of each point, the initial edge consisting of two types of image points which has partially maximum grey increase and maximum grey decrease can be obtained; finally sub-pixel localization models with different types of edges are established according to the features of the initial edge.

The results obtained from all edge detection algorithms are the discrete edge points, which need to be tracked before the application. In general, free-form edges have complex forms and intersect with each other; additionally, the influence of the factors like noise also makes some extracted edges fracture and unsmooth; however, most of the current tracking algorithms have not fully analyzed and discussed these conditions, which causes that the edge features cannot be fully and effectively extracted, restricting the application of FFLFs to a certain extent.

In view of the above problems, reference [12] proposed a sub-pixel edge detection method, which can, based on high positioning accuracy of edge detection, effectively extract the smooth FFLFs with better continuity; thus it is applied to provide tracking & refining of the detected edge. The basic principle of this method is: to propose a tracking strategy of sub-pixel edge points according to the actual distribution of the edge points, so as to record the free-form edges as much as possible; in order to restore the continuity of the edge, trying to avoid the connection of uncorrelated edges and apply an expansion connection method of double-angle and length control to connect the edge; finally using the smoothing algorithm based on curve inner expression to smooth the two-dimensional FFLFs.

2.2 Extraction of the Features for Matching

First of all, this method extracts the CLFs and LFIs and LFCs from FFLFs to realize coarse matching; in order to balance the matching efficiency and precision, extraction of sub-pixel edge points with different sampling rates is applied to gradually realize accurate matching by the edge points with the sampling rate from low to high. So this section mainly introduces the extraction methods and matching entities of the features applied in the two matching processes, as well as the similarity measurement.

  • (1) Closed Linear Features (CLF)

    The CLFs can be determined by identifying the conjugate linear features of the starting point and ending point. This paper selects the CLFs with the area meeting the requirement of threshold for the matching, and 7 invariant moments not changing with rotation, translation and scaling are taken as its matching entities. Let M i (p m ) and M j (q n ) (i, j = 1, 2,…,7; m, n = 1, 2,…) represent 7 invariant moments of various CLFs of left and right images; due to large difference of invariant moments, the normalized correlation algorithm shown in Formula (1) is applied to measure the similarity between the CLFs. Without ambiguity, p m , q n are used to represent the CLFs and also represent the center of gravity of the surrounding area.

    $$ R(p_{m} ,q_{n} ) = \sum\limits_{i = 1}^{7} {\frac{{\left| {M_{i} (p_{m} ) - M_{i} (q_{n} )} \right|}}{{\left| {M_{i} (p_{m} )} \right| + \left| {M_{i} (q_{n} )} \right|}}} $$
    (1)

  • (2) Linear Feature Intersection (LFI)

    The tracking method mentioned in Sect. 2.1 made a complete extraction of the linear features, so the edge points with the number of recording not less than three times in the linear features are the LFIs. Since the extraction of LFI with more than three linear feature branches can be easily affected by the noise and there is small quantity, only the intersections containing three branches and stable direction are selected for matching and the included angles between the branches are taken as its matching entities.

    In order to guarantee the uniqueness of the matching entities for LFIs, the following rules should be followed to arrange three included angles: clockwise arranging and the maximum angle are placed at the starting position. Clockwise arrange the direction vector of each branch and calculate the included angle between the adjacent direction vectors; clockwise record the included angles successively from the maximum angle. Through analysis, arbitrary arrangement of three vectors with the corresponding starting point only includes two types, the clockwise and counterclockwise arrangement, and order exchange of the latter two vectors can realize the exchange of two types (Number shown in Fig. 1 represents the serial number of each branch); thus, after the direction vector is determined, the arrangement is identified by comparing the horizontal and vertical components of each vector and then adjusted appropriately to realize the clockwise arrangement.

    Fig. 1.
    figure 1

    Two types of arrangements of three direction vectors

    Full size image

  • (3) Linear Feature Corner (LFC)

    Douglas-Peucker (DP) method is to determine the feature points of the curve from the overall to local, which has invariance to translation and rotation, as well as high computational efficiency. But its threshold is not easy to be determined: a larger threshold is easy to cause excessive compression; a smaller threshold is sensitive to the “thrusting” caused by the noise and is easy to get wrong feature points. The intersection extracting method based on the difference of absolute chain code sum proposed by Li et al. [13] has good information compression ability and anti-interference ability. Therefore, this paper applies this method to test the reliability and accuracy of the corners extracted by DP method, with its basic principle shown in Fig. 2.

    Fig. 2.
    figure 2

    Basic principle of correction of the corner location

    Full size image

    Set O as the corner to be selected (that is the feature point determined by DP method) and consider the O and two edge points on both sides A i, B i (i = 1, 2): first identify if the difference of absolute chain code sum of O and A 1 , B 1 meet the requirements of threshold; if only O can meet the requirement, it should be the corner; if one of point A 1 and B 1 can meet the requirement, its reliability should be compared with that of point O and the point with the greatest reliability is taken as the corner; if all the above three points cannot meet the requirement, then A 2 and B 2 should be further identified. When at least one point meets the requirement, comparing it with the point O, the one with greater reliability should be taken as the corner; otherwise the point O will be eliminated as a false corner.

    The reliability of point which is selected as corner may be calculated by the model shown in Formula (2).

    $$ C = {1 \mathord{\left/ {\vphantom {1 {(\sum\nolimits_{i = 1}^{2} {\sum\nolimits_{j = 1}^{h} {d_{ij} } } )}}} \right. \kern-0pt} {(\sum\nolimits_{i = 1}^{2} {\sum\nolimits_{j = 1}^{h} {d_{ij} } } )}} $$
    (2)

    In which: d 1j and d 2j respectively represent the distance from P j and Q j to the straight line S 1 and S 2; S i (i = 1, 2) is the connection (i.e. OAʹ, OBʹ) between the current point (point O as shown in Fig. 3) and the edge point (i.e. point Aʹ, Bʹ) which has h number of points interval with its some linear feature branch; P j , Q j (j = 1, 2,…, h) are respectively the edge point of No. j on both sides of the branch.

    Fig. 3.
    figure 3

    The principle of elimination of false matching by using the geometric information

    Full size image

    It should be noted that the method proposed in reference [13] aimed at pixel-level edge curve, while the linear features extracted in Sect. 2.1 and processed by DP method are sub-pixel edge curves; thus each sub-pixel edge point is replaced by the center of the image point (Coordinate as an integer) in the chain code related calculation and the sub-pixel coordinates should be still applied for the reliability calculation.

    In order to obtain the corners with uniform distribution and a reasonable number, only the corners which have stable linear feature branch direction and the interval is greater than a certain threshold value should be applied. The angle between the direction vectors of the branches on both sides of the corner is taken as its matching entity. The direction of the angle bisector is taken as the main direction.

  • (4) Selection of the Features for Accurate Matching

    ICP method is considered as three-dimensional object matching algorithm based on pure geometric model, which can directly process the data of 3D object not related to the surface, with no need to assume and divide the object’s features. Thus it has been widely used to realize 3D registration. ICP method can obtain higher matching accuracy when the point set has high precision, but it is vulnerable to the interference of noise point, with poor robustness; and there is low efficiency when the point set quantity is large. Based on the above analysis, in the selection of edge points for accurate matching, the following two strategies are taken to reduce the influence of noise points and improve the efficiency of the algorithm:

    1. (a)

      smoothing it with the linear features of corner: considering that the ground object especially the artificial ground object has smooth edge curve and in order to keep the corner with higher accuracy and better stability in the edge detection results, in the edge smoothing of Sect. 2.1, the corners extracted in (3) are used and only the edge curves between the corners are smoothed.

    2. (b)

      using the edge points with different sampling rates in various stages of the iteration: the matching results of two sampling rates show that under the condition of edge points having the same precision, the higher the sampling rate is, more likely the more accurate matching precision will be obtained.

      Therefore, in the process of matching, in order to ensure the calculation efficiency, the results of reducing the sampling rate were used for the first few iterations; then all the edge points were used; when there was a certain iteration and the model parameters were stable, the interpolation results were used for the matching in order to improve the matching accuracy. It should be noted that the corners extracted in (3) are still retained while reducing the sampling, which means only reducing the sampling rate to the edge points between the corners.

2.3 Coarse Matching by Using Multiple Linear Features

In remote sensing image processing, two-dimensional or three-dimensional similarity transformation can approximately replace many 2D or 3D transformations; because there is less the transformation parameter, it can independently make the solving and widely used [2]. This paper takes 2D similarity transformation (formula (3)) as the model for coarse matching, in order to determine the conjugate CLFs, intersection and corner and provide the initial value for accurate matching.

$$ \left( {\begin{array}{*{20}c} {x_{2} } \\ {y_{2} } \\ \end{array} } \right) = s \cdot \left( {\begin{array}{*{20}c} {\cos \alpha } & { - \sin \alpha } \\ {\sin \alpha } & {\cos \alpha } \\ \end{array} } \right)\left( {\begin{array}{*{20}c} {x_{1} } \\ {y_{1} } \\ \end{array} } \right) + \left( {\begin{array}{*{20}c} {t_{x} } \\ {t_{y} } \\ \end{array} } \right) $$
(3)

In the Formula: (x 1, y 1) and (x 2, y 2) are respectively the image points of the conjugate features on the left and right images, similarly hereinafter; s, α and (t x , t y ) are respectively the scaling, rotation and translation parameters between the images.

  • In the coarse matching by a variety of relatively stable features extracted in Sect. 2.2, first identify the conjugate features to be selected of each feature of the left and right images according to their similarity measure and then make the combination of every two to get a number of groups of combinations of the conjugate features to be selected (hereinafter referred to as “combination to be selected”); then gradually eliminate the combination with false matching through the following two methods (that is to say, at least one pair of the conjugate features to be selected is not the correct conjugate feature, hereinafter referred to as “false matching combination”).

  • (1) Elimination of False Matching Combination Using the Geometry Information

  • Since the 2D similarity transformation is taken as the transformation model, only the scaling, rotation and translation exist between the features through the model transformation, that is to say, the magnitude of various features and the distance between the features show the changes in the same proportion, while the angle between the features has no such changes; based on the above characteristics, the false matching combination can be eliminated by using the area of features to be matched and the distance & angle between the features.

  • The following is the introduction of the basic principle taking the CLFs as an example: the CLFs of the two images are shown in Fig. 3: set N 1(p 2, q 1), N 2 (p 2, q 2), N 3 (p 3, q 3) and N 4 (p 4, q 4) as four pairs of conjugate features to be selected determined according to the similarity measure, and make the combination of every two to get 6 combinations: N 1 N 2, N 1 N 3, N 1 N 4, N 2 N 3, N 2 N 4, N 3 N 4; then orderly calculate the maximum difference scaling of each combination as shown in Formula (4) and identify whether they meet the ratio threshold T s ; if yes, then keep the combination; if no, it should be considered as the false matching and eliminated.

  • $$ S = \mathop {\hbox{max} }\limits_{i,\;j = 1,\,2,\,3;\,\,i \ne j} \left| {S_{i} - S_{j} } \right| \le T_{s} $$
    (4)

  • In which: \( S_{1} = {{L_{{p_{2} p_{3} }} } \mathord{\left/ {\vphantom {{L_{{p_{2} p_{3} }} } {L_{{q_{1} q_{3} }} }}} \right. \kern-0pt} {L_{{q_{1} q_{3} }} }} \), \( S_{2} = \sqrt {{{A_{{p_{2} }} } \mathord{\left/ {\vphantom {{A_{{p_{2} }} } {A_{{p_{3} }} }}} \right. \kern-0pt} {A_{{p_{3} }} }}} \), \( S_{3} = \sqrt {{{A_{{q_{1} }} } \mathord{\left/ {\vphantom {{A_{{q_{1} }} } {A_{{q_{3} }} }}} \right. \kern-0pt} {A_{{q_{3} }} }}} \), L represents the length of connection of center of gravity conjugate to two CLFs, S represents the area of the conjugate CLFs.

  • Through the above method, the combination containing N 1 - the conjugate features to be selected can be easily eliminated. The principle of eliminating the combination of LFI and LFC is similar and the difference is that the angle between the feature point main directions and the angle between the feature points connection; it will not be detailed here.

  • (2) Elimination of False Matching Combination Based on Space Distribution of the Transformation Parameters

  • After preliminary elimination of false matching combination, each pair kept is used to determine a set of transformation parameters; the analysis shows that the parameter determined by the combination composed of two pairs of correct conjugate features will be very close to the real value, i.e. the relatively concentrated distribution; however, there is large difference between the parameters determined by false matching combination, so the transformation parameters determined by each combination are considered as the points of high dimensional space (R 4); and K-D tree and K-means clustering are successively applied to further eliminate the false matching combination.

  • After the establishment of K-D tree by use of the space points, the closest point of each point (that is the transform parameter with minimum difference) should be searched and the combination of the space points with the distance from the closest point more than the distance threshold T d should be eliminated; this method can eliminate most of the false matching combinations. And then K-means is applied to divide the remaining points into two types; if the distance of center of two types of points on the various dimensions is less than the threshold T l , it should be considered small difference of the remaining points and the center of the remaining points is taken as the transform parameter determined by coarse matching, so as to determine the conjugate features; otherwise the one with more points should be kept and K-means method is used to make the classification, until the distance between the center of two types of points meets the threshold requirement or the number of remaining points is less than 3; for the latter case, it should be considered that the conjugate features were not found.

  • The experiment and analysis show that the CLFs have moderate quantity and stable performance. In order to improve the efficiency, the CLFs should be first matched in the coarse matching; when there are more than three pairs of correct conjugate features realizing the matching, they will be taken as prior reference information to determine the conjugate intersection and corner; otherwise, the above method is further applied to match the intersection and corner extracted, in order to determine the conjugate intersection and corner of two images.

2.4 Accurate Matching Based on Multi-level Two-Dimensional ICP

Based on the principle of the ICP method, this paper proposed an accurate matching method using multi-level two-dimensional ICP. The initial value of the transformation model is calculated according to the conjugate features identified by the coarse matching; the sub-pixel edge points with different sampling rates are used successively for matching. Firstly, the sub-pixel edge points with the lowest sampling rate are used to participate in matching. Each edge points in left image are transformed according to the initial parameters, and the edge points closest to the transformed point are searched in the right image by using the K-D tree. These closest point pairs are regarded as conjugate points to correct the transformation parameters. The above iterative process should be repeated until it meets the convergence condition. Then, edge points with higher sampling rate are used to participate in matching until the edge points with highest sampling rate complete matching to obtain the transformation parameter. The calculation model is introduced briefly as follows.

This paper chooses the projection transformation model shown in Formula (5) for accurate matching to express the transformation relationship between two images; this model is generally applied in the case of complex or unknown transformation model between two images.

$$ \left\{ {\begin{array}{*{20}c} {x_{2} = \frac{{L_{1} x_{1} + L_{2} y_{1} + L_{3} }}{{L_{7} x_{1} + L_{8} y_{1} + 1}}} \\ {y_{2} = \frac{{L_{4} x_{1} + L_{5} y_{1} + L_{6} }}{{L_{7} x_{1} + L_{8} y_{1} + 1}}} \\ \end{array} } \right. $$
(5)

In which: L = (L 1 L 2 L 3 L 4 L 5 L 6 L 7 L 8)T is the projection transformation parameter.

Both sides of the equation are multiplied by the denominator at the same time can obtain the linear form as follows:

$$ \left\{ {\begin{array}{*{20}c} {F_{dx} = (L_{7} x_{1} + L_{8} y_{1} + 1) \cdot x_{2} - L_{1} x_{1} + L_{2} y_{1} + L_{3} } \\ {F_{dy} = (L_{7} x_{1} + L_{8} y_{1} + 1) \cdot y_{2} - L_{4} x_{1} + L_{5} y_{1} + L_{6} } \\ \end{array} } \right. $$
(6)

Error equation and normal equation are shown in Formula (7) and (8) respectively.

$$ V = AL - l{\text{ P}} $$
(7)

In which: \( A = \left[ {\begin{array}{*{20}c} { - x_{1} } & { - y_{1} } & { - 1} & 0 & 0 & 0 & {x_{1} x_{2} } & {y_{1} x_{2} } \\ 0 & 0 & 0 & { - x_{1} } & { - y_{1} } & { - 1} & {x_{1} y_{2} } & {y_{1} y_{2} } \\ \end{array} } \right];\,l = \left[ {\begin{array}{*{20}c} {x_{2} } \\ {y_{2} } \\ \end{array} } \right] \); P is weight matrix, P i is determined by the distance between the conjugate points.

$$ L = (A^{T} PA)^{ - 1} \times (A^{T} Pl) $$
(8)

Through the above adjustment model, the transformation parameter is constantly corrected until the change of the parameter meets the demand of threshold.

3 Experimental Results and Analysis

In order to validate the performance of this proposed method, the following three groups of experiments were designed.

  • Experiment 1: using the simulated data to validate the stability and precision of coarse matching

  • In order to validate the stability of various features for coarse matching and the reliability and accuracy of coarse matching method, two groups of matching experiments of the images under different rotation angles and scaling factors were designed. Part of an urban area aerial image of 339 × 353 pixels (Fig. 4(a)) was selected as reference image. Simulated images shown in Fig. 4(b) and (c) were obtained by using the similar transformation parameters in Table 1, they respectively contain a small and large rotation angle.

    Fig. 4.
    figure 4

    Reference image and simulated images generated by the similarity transformation

    Full size image
    Table 1. Comparison between the actual transformation parameters and the transformation parameters determined by coarse matching
    Full size table

  • Two simulated images were provided with the coarse matching to the reference image based on the proposed method; the features extracted and the matching results are shown in Table 2. The first column represents the feature number extracted from the reference image. The three columns below simulated images represent the feature number extracted from simulated image, the true conjugate features number, and the conjugate features number determined by coarse matching and the correct conjugate features which is shown in parentheses. The parameters determined by coarse matching are shown in Table 1.

    Table 2. The number of the features which used for coarse matching
    Full size table

  • For quantitative evaluation of the matching precision, checkpoints are picked up from the reference image interval of 10 pixels both in row and column direction (a total of 1,155 checkpoints). Then the actual parameters and the parameters determined by coarse matching were used to compute the theoretical coordinates and matching coordinates of the checkpoints in the simulated image. Matching precision is evaluated through the difference between these coordinates. These checkpoints were distributed evenly over the entire image, therefore, they have good performance to evaluate the precision. Quantitative evaluation results (root-mean-square error, RMSE) in column direction (X), row direction(Y) and overall (XY) on these two group experiment data are shown in Table 3.

    Table 3. Quantitative evaluation results on the two groups experiment data (pixel)
    Full size table

  • As shown in Table 2, the number of CLFs is moderate, CLFs have the largest number. And these two features have good stability for rotation, scaling transformation and noise. The matching results show that all these three kinds of features have good matching performance, high matching precision, low leakage matching rate and without mismatching. Tables 1 and 3 show that the parameters determined by coarse matching are very close to the actual parameters; the error of the checkpoints in row and column direction are both small, the total RMSE is less than 1 pixel.

  • The experiment results show that the extracted features have stable performance and the coarse matching can effectively eliminate the false matching to ensure high precision and provide more accurate matching initial value for accurate matching.

  • Experiment 2: using the simulated data generated by affine transformation to validate the accuracy of this method

  • This experiment utilizes the affine transformation to generate the simulated images; by matching it with the reference image and comparing the matching accuracy between it and SIFT, ASIFT, MSER and least square matching (LSM) based on SIFT. This experiment is designed to validate the matching accuracy of this method.

  • The simulated image obtained from the affine transformation (transformation parameter as shown in Table 4) of the image shown in Fig. 4(a) is shown in Fig. 5(a); the features used for coarse matching are shown in Fig. 5(b) and (c), in which the white area and the asterisk respectively represent the CLF and its center of gravity; blue dot and line respectively represent the LFI and the direction of its branches; red star and red line with arrowhead respectively represent the LFC and its main direction, similarly hereinafter. The results of coarse and accurate matching are respectively shown in Fig. 5(d) and (e). Because there were too many conjugate points determined by the accurate matching (a total of 7,921 pairs), the connection between the conjugate points was not made.

    Table 4. Comparison of the matching results between different methods
    Full size table
    Fig. 5.
    figure 5

    Matching between the images with the affine transformation relationship

    Full size image

  • The parameters determined by various matching methods are shown in Table 4. Quantitatively evaluate the accuracy of each matching method use the same way with Experiment 1, with the results shown in Table 5 below.

    Table 5. Quantitative evaluation results of the checkpoints on various matching methods (pixel)
    Full size table

  • The experiment result shows that the extracted features used for coarse matching evenly distributed, based on which the coarse matching has higher accuracy; because a few steps were taken to eliminate the false matching in the process of the coarse matching, there was higher correct rate of matching; in the process of accurate matching, there were a lot of sub-pixel edge points to participate in the matching, which can reach the sub-pixel accuracy equal to that of least square matching. In addition, the experiment result also shows that the proposed method has certain adaptability to the affine transform. When there was not severe affine deformation between the images, it still could realize high accuracy matching.

  • Experiment 3: Remote sensing stereo images matching

  • This experiment utilized the IKONOS images of United States of California Santa Barbara (UCSB) campus, which consists of such ground object types as building, sea and lake, with the image size of 1,024 × 1,024 pixels and 1,064 × 1,172 pixels respectively; the experimental images, the features for coarse matching, the results of coarse matching and accurate matching are shown in Fig. 6.

    Fig. 6.
    figure 6

    Results of aerial stereo image matching

    Full size image

  • Theory analysis and the results of the former two experiments indicate that the LSM matching has very high matching precision and can reach 1/10 to 1/100 pixel. Therefore, the parameters determined by SIFT-LSM are regard as actual parameters in this experiment. The matching precision was compared by the quantitative evaluation method of aforementioned groups of experiments. The checkpoints error statistics are shown in Table 6.

    Table 6. Quantitative evaluation results of the checkpoints on IKONOS Image (pixel)
    Full size table

  • Figure 6 and Table 6 show that this method can achieve precision matching between high-resolution remote sensing images which only partially overlapping. Coarse matching has comparative accuracy with classical point feature matching methods such as SIFT, ASIFT, MSER, and accurate matching has comparative accuracy with LSM method. Experiment results also fully demonstrate that the proposed method has stable performance of features matching and the matching strategy is feasible to achieve higher matching accuracy and precision.

4 Conclusions

In the study of FFLF matching, full use and effective description of information in the features constitute a conflict, which restricts the development of this kind of method to a certain extent. In addition, because only the linear features are utilized, this method is more sensitive to the error of the linear feature extraction, of which the reliability and accuracy, to a great extent, depend on the reliability and accuracy of the linear feature extraction. In order to better solve the above problems, this paper proposed a remote sensing image matching method using the strategy of hierarchical matching. First the edge features of the image were detected and tracked, in order to extract the free-form sub-pixel linear features with better continuity; then the CLFs, LFI and corner were extracted for coarse matching and the false matching was gradually eliminated based on the area, angle and other geometry information as well as the parameter distribution of the model determined the conjugate features combination to be selected; finally, based on multi-level two-dimensional ICP, sub-pixel edge points were orderly used with the sampling rate from low to high for accurate matching. Experimental results show that this method has stable performance for the coarse matching; high accuracy and precision of the coarse matching can provide the initial matching parameters of high precision for accurate matching; accurate matching can reach the sub-pixel level precision equal to that of the least square matching.

The features extracted for coarse matching in this paper have good stability under the smaller affine condition and can better realize the matching between the images with smaller affine deformation. Further study on FFLF is still needed. The features that can remain the same even under the severe affine deformation may be extracted in future, so as to improve the affine resistance of FFLF matching.