Keywords

5.1 Introduction

The shape matching of planar curves that are subjected to certain occlusion and viewing transformations is motivated by a board of problems arising from different applications in many fields such as robot navigation [1, 2], medical image matching [3, 4], face recognition [5, 6], and object tracking [7, 8].

The research on the shape matching is recognized as a classical field and well-studied in the literature. However, there are other challenges that still remain open until now such as noise, distortion, and deformation. They are caused by two main reasons: First, images taken at different moments and from different angles of the same object suffer from perspective distortions [9]. This transformation can be approximated by a two-dimensional affine transformation [10] if the object is planar and far enough from the image plane. The second reason, the curve sometimes can present a little part of the shape [11].

However, it is not obvious to define an approach that satisfies these different criteria, so an appropriate trade-off is necessary. In the last decades, various methods for shape matching have been developed. The best-known researches include affine-invariant Fourier descriptors (AIFD) method [12, 13], shape context (SC) methods [14], the inner distance shape context (IDSC) [15], the height function (HF) descriptor [16], and so on. However, they suppose that the complete shape can be extracted from the images [17]. While in reality, the extracted shape may be occluded by many distortions [18] like noise, articulations and missed contour portions, as well as affine transformation. In [19], Chen et al. introduce a partial shape matching method based on the Smith–Waterman algorithm. Also, Latecki et al. [20] define an elastic partial shape matching to model distortion and occlusion of shapes. In [21], the author proposes a matching algorithm based on dynamic time warping. Therefore, only some methods treat both affine and open curve matching [9, 22]. In [17], Mai and al. define an affine-invariant partial shape matching approach where each contour is segmented into affine-invariant segments by the application the local maxima of curvature scale space (CSS). Then, the different parts are matched using the Smith–Waterman algorithm. However, the Smith–Waterman (SW) algorithm is sensitive to position jitter of a point sequence as indicated in [9]. Moreover, Huijing Fu and al [9] present an affine planar shape matching and exploit it for partial object recognition where an affine-invariant curve descriptor (AICD) using the affine-invariant signature is defined. Then, a partial curve matching algorithm is developed by combining AICD with a curve segmentation strategy based on inflexion points. Yet, their method has limited accuracy in the noise condition case caused by the high number of derivation [23]. In this paper, we are interested in part-to-part affine shape matching. We can present the partial shape matching problem by giving as an input two partial shape, often called source and target, the goal is to recover the affine transformation that optimized the pairing between the source and the target. So to achieve this pairing, we should define the same parameterization for each curve. The underlying idea here is to divide each shape into ordered affine-invariant segments inspired by the expression of affine curvature scale space [24]. Then to handle the matching problem, we estimate the affine transformation parameters using affine curve matching (ACM) [25] algorithm. This algorithm minimizes the approximation of L2 distance between pairs by the computation of the pseudo-inverse matrix. In experimentation, we will evaluate the performance of proposed approach in the context of shape retrieval and we will compare it with different affine curves matching methods presented in the state of the art.

The remainder of the paper is organized as follows: In Sect. 5.2, the detailed descriptions of suggested curve matching algorithm will be presented. Section 5.3 will investigate the effectiveness of the proposed approach through experiments and analyses. Finally, the last section submits the conclusion.

5.2 Affine Curvature Scale Space Curve Matching Algorithm

Here, we define our new partial contour matching based on curvature scale space matching algorithm. Therefore, this section will be devoted firstly to recall the affine re-parametrization based on the affine curvature scale space (ACSS) [24]. After that, we recall the newly developed affine curve matching (ACM) algorithm [25] and apply the pseudo-inverse matrix to estimate the translation vector B and the special linear transformation A existing between the two curves up to a special affine transformation.

5.2.1 Affine Curvature Scale Space Descriptor (ACSS)

As given Ω1 and Ω2 two planar shapes represented by closed or open continuous curves. We extract two contours of each shape which are represented by two parameterizations, respectively, as: \( \Omega _{1} = [f(t) = (f^{x} (t) f^{y} (t))] \, \left( {t = 1;2; \ldots ;N} \right) \) and \( \Omega _{2} = [h(t^{\prime}) = (h^{x} (t^{\prime})h^{y} (t^{\prime}))] \, (t^{\prime} = 1;2; \ldots ;N) \), where their relation is defined by:

$$ h\left( {t^{\prime}} \right) = Af\left( t \right) + B $$
(1)

With B is a translation vector and A is a special linear transformation. It is obvious that each curve can be represented by different parameterizations. Therefore, we cannot consider that the two contours have the same parameterization and we compare the different viewpoints of planar shape. To handle this problem, we must assure that the parameterization is independent from transformations and distortions. As a result, we need to re-parameterize the points of the contour. In [24], it is proved that the locations of the contour local maxima in the affine curvature scale space (ACSS) image is invariant under an affine transformation. Moreover, they are robust to noise as indicated in [17]. The underlying idea is to do an affine curves re-parameterization by applying an ACSS descriptor. Then, we describe the following main steps of the ACSS method as indicated in [24]. Firstly, we re-parameterize the contour using the affine length function l(t) defined by

$$ l = \frac{1}{L}\mathop \int \limits_{0}^{t} \left| {{ \det }(\dot{f}(u),\ddot{f}(u) )} \right|^{{\frac{1}{3}}} $$
(2)

where the total affine length L of the considered curve is presented by:

$$ L = \mathop \int \limits_{0}^{T} \left| {{ \det }(\dot{f}(u),\ddot{f}(u)) } \right|^{{\frac{1}{3}}} {\text{d}}u $$
(3)

With \( \dot{f} \) and \( \ddot{f} \) denote, respectively, first and second derivatives of f and T are a positive real. As a result of re-parameterization by this affine length, the relation between the two curves becomes:

$$ h^{*} (l) = Af^{*} (l) + B $$
(4)

With \( h^{*} \) and \( f^{*} \) denotes a re-parameterization by this affine length. In the rest of paper, we will replace \( f^{*} \) by f and \( h^{*} \) by h to simplify the notation. Then, we compute the curvature function \( k(l) \) expressed by:

$$ k(l) = \frac{{\dot{f}^{x} (l)\ddot{f}^{y} (l) - \ddot{f}^{x} (l)\dot{f}^{y} (l)}}{{\left( {\dot{f}^{{x^{2} }} (l) + \dot{f}^{{y^{2} }} (l)} \right)^{{\frac{3}{2}}} }} $$
(5)

where \( \dot{f}^{x} (l) \), \( \dot{f}^{y} (l) \) and \( \ddot{f}^{y} (l) \), \( \ddot{f}^{x} (l) \) are the first and second derivatives. If g(l, σ), a 1-D Gaussian kernel of width σ, is convolved with each component of the curve, then \( f_{\sigma }^{x} \left( {l,\sigma } \right) \) and \( f_{\sigma }^{y} \left( {l,\sigma } \right) \) represent the components of the resulting curve, \( f_{\sigma } \):

$$ f_{\sigma }^{x} \left( {l,\sigma } \right) = f^{x} { \circledast }g\left( {l,\sigma } \right)\quad f_{\sigma }^{y} \left( {l,\sigma } \right) = f^{y} { \circledast }g\left( {l,\sigma } \right) $$

where ⊛ is a convolutional operator. The curvature of \( f_{\sigma } \) is given by:

$$ k\left( {l,\sigma } \right) = \frac{{\dot{f}^{x} \left( {l,\sigma } \right)\ddot{f}^{y} \left( {l,\sigma } \right) - \ddot{f}^{x} \left( {l,\sigma } \right)\dot{f}^{y} \left( {l,\sigma } \right)}}{{\left( {\dot{f}^{{x^{2} }} \left( {l,\sigma } \right) + \dot{f}^{{y^{2} }} \left( {l,\sigma } \right)} \right)^{{\frac{3}{2}}} }} $$

5.2.2 Affine Curve Matching (ACM) Algorithm

To solve the matching problem, we must find A and B to estimate the relation and motion between the different contours as indicated in [25]. The re-parametrization of two curves by the ACSS gives the following rectangular system formed by 2*N equations and 6 unknown variables:

$$ \left\{ {\begin{array}{*{20}c} {h_{\sigma }^{x} \left( {l_{1} } \right) = f_{\sigma }^{x} \left( {l_{1} } \right)a_{11} + f_{\sigma }^{y} \left( {l_{1} } \right)a_{12} + B^{x} } \\ {h_{\sigma }^{y} \left( {l_{1} } \right) = f_{\sigma }^{x} \left( {l_{1} } \right)a_{21} + f_{\sigma }^{y} \left( {l_{1} } \right)a_{22} + B^{y} } \\ . \\ . \\ . \\ {h_{\sigma }^{x} \left( {l_{N} } \right) = f_{\sigma }^{x} \left( {l_{N} } \right)a_{11} + f_{\sigma }^{y} \left( {l_{N} } \right)a_{12} + B^{x} } \\ {h_{\sigma }^{y} \left( {l_{N} } \right) = f_{\sigma }^{x} \left( {l_{N} } \right)a_{21} + f_{\sigma }^{y} \left( {l_{N} } \right)a_{22} + B^{y} } \\ \end{array} } \right\} $$

with f(l) and h(l) as the re-parametrization, respectively, of two contours f(t) and \( h\left( {t^{\prime}} \right) \), B = (\( B^{x} \); \( B^{y} \)) and \( A = \left( {a_{ij} } \right)_{{1 = < \left( {i,j} \right) = < 2}} \). Our goal is to minimize the error e between the two contours by the estimation of A and B which will be defined by:

$$ \min _{{\left( {A,B} \right)}} \left| {\left| {Af_{\sigma } (l) + B - h_{\sigma } (l)} \right|} \right|^{2} \cong e $$

This system can be written in matrix notation:

$$ H = DU $$

With \( U = \left[ {a_{11} \;a_{12} \;a_{21} \;a_{22} \;B^{x} \;B^{y} } \right]^{t} \); \( H = \left[ {h_{\sigma }^{x} \left( {l_{1} } \right),h_{\sigma }^{y} \left( {l_{1} } \right) \ldots h_{\sigma }^{x} \left( {l_{N} } \right),h_{\sigma }^{y} \left( {l_{N} } \right)} \right] \) and

$$ D = \left\{ {\begin{array}{*{20}c} {f_{\sigma }^{x} \left( {l_{1} } \right)} & {f_{\sigma }^{y} \left( {l_{1} } \right)} & 0 & 0 & 1 & 0 \\ 0 & 0 & {f_{\sigma }^{x} \left( {l_{1} } \right)} & {f_{\sigma }^{y} \left( {l_{1} } \right)} & 0 & 1 \\ {f_{\sigma }^{x} \left( {l_{2} } \right)} & {f_{\sigma }^{y} \left( {l_{2} } \right)} & 0 & 0 & 1 & 0 \\ 0 & 0 & {f_{\sigma }^{x} \left( {l_{2} } \right)} & {f_{\sigma }^{y} \left( {l_{2} } \right)} & 0 & 1 \\ . & . & . & . & . & . \\ . & . & . & . & . & . \\ {f_{\sigma }^{x} (l_{N} )} & {f_{\sigma }^{y} (l_{N} )} & 0 & 0 & 1 & 0 \\ 0 & 0 & {f_{\sigma }^{x} \left( {l_{N} } \right)} & {f_{\sigma }^{y} \left( {l_{N} } \right)} & 0 & 1 \\ \end{array} } \right\} $$

In [25], the author applies the least squares method to solve the over determined system of linear equations when the numbers of equations are more than unknown variables. Thus, the resolution of this rectangular system can be done by minimizing the error via inverting the system by using pseudo-inverse of the matrix D as indicated in [25].

$$ U = \left( {D^{t} D} \right)^{ - 1} D^{t} H $$

Then, we calculate the normal matrix \( \left( {D^{t} D} \right) \) which has the following expression:

$$ D^{t} D = N^{6} \left( {\begin{array}{*{20}c} {\overline{{X^{2} }} } & {\overline{XY} } & 0 & 0 & {\overline{X} } & 0 \\ {\overline{XY} } & {\overline{{Y^{2} }} } & 0 & 0 & {\overline{Y} } & 0 \\ 0 & 0 & {\overline{{X^{2} }} } & {\overline{XY} } & 0 & {\overline{X} } \\ 0 & 0 & {\overline{XY} } & {\overline{{Y^{2} }} } & 0 & {\overline{Y} } \\ {\overline{X} } & {\overline{Y} } & 0 & 0 & 1 & 0 \\ 0 & 0 & {\overline{X} } & {\overline{Y} } & 0 & 1 \\ \end{array} } \right) $$

\( \overline{X} = \frac{1}{N}\mathop \sum \nolimits_{k = 1}^{N} (f_{\sigma }^{x} \left( {l_{k} } \right)) \) and \( \overline{Y} = \frac{1}{N}\mathop \sum \nolimits_{k = 1}^{N} (f_{\sigma }^{y} \left( {l_{k} } \right)) \)

The reader can find more details in [24, 25].

5.3 Experiments

In this section, we provide the recognition rates of the suggested approach and compare it with the exiting shape matching methods. The experiments are carried on multi-view curve dataset (MCD). First, we evaluate the performance of our approach in the shape retrieval. Then, we prove the performance of the proposed algorithm on shape registration. Finally, we will analyze the algorithm complexity of our approach.

5.3.1 Retrieval Accuracy

To perform our algorithm in shape retrieval task, we calculate the bulls-eye score as defined in [26, 27] on multi-view curve dataset (MCD) [28], since it contains forms that undergo an affine transformation. So to calculate the bulls-eye score, we compare each curve to the whole MCD dataset curve (including itself). Then, the contour number of the same class that are midst the 2 * Nc most similar is recovered, with Nc is the sample number per class. The bulls-eye score is the ratio of the number of correct results and the highest possible number of correct results [9, 27].

The MCD contains 40 shape classes taken from MPEG-7 datasets. Each class presents 14 curve samples that correspond to different perspective distortions of the same shape. Samples of contours from MCD datasets are shown in Fig. 5.1.

Fig. 5.1
figure 1

Different contours images from the MCD database, two images from each class [28]

Full size image

Table 5.1 compares the retrieval bulls-eye score to the first 10 contours of the MCD database using our approach with some existing methods. In terms of the average rate performance, the proposed approach performs reasonably well as compared to many other techniques such as affine curve matching (ACM) [25] algorithm with 94% of rate and especially the curvature scale space Smith–Waterman (CSS-SW) [17] approach which prove that our algorithm is more efficient in terms of registration since we are based on ACSS in the re-parameterization step.

Table 5.1 Retrieval results on the entire MCD Dataset
Full size table

5.3.2 Shape Registration

Shape registration is a crucial applications of the proposed algorithm [29]. Most shapes of MCD datasets are represented by closed curves. So to evaluate the proposed algorithm in partial occlusion and deformable registration, we take off some parts of the contour to make it open as indicated in [17]. Figure 5.2 shows our method in full-to-full and part-to-part registration case.

Fig. 5.2
figure 2

Proposed method: a and b are the initial shape part; c and d show the original shape overlaid with the registered shape

Full size image

5.3.3 Algorithm Complexity

Here, we compare our algorithm complexity with the CSS-SW. The calculation of the shape descriptors and shape matching are two independent steps for the proposed approach. Therefore, the complexity of the proposed method will be evaluated separately here. We consider that N is the number of sample points of the contour. For each curve, we should start by the re-parametrization step based on ACSS descriptor which is a common step between the two methods. In the matching step, the proposed algorithm can do the matching with O(N) complexity which underlines the speed of pseudo-inverse. However, Smith–Waterman complexity is O(\( N^{3} \)) [9], since it requires SVD calculation.

5.4 Conclusion

In this paper, we propose a new matching method which can deal with both occlusion and affine transformations. Firstly, the contour is divided into affine-invariant segment by applying the ACSS descriptor. Then, we estimate the affine transformation using the ACM algorithm. As a result, an affine curve matching is achieved. Experiment results show that the proposed algorithm is simple and can cope partially occlusion and affine transformation. In our future work, we will apply our algorithm in different application domains as remote sensing and robotic recognition.