Keywords

1 Introduction

Over recent years, thanks to the development of sensor technology and mobile computing, trajectory-based object motion analysis has gained significant interest from researchers. It is now possible to accurately collect location data of moving objects with less expensive devices. Thus, applications for sign language and gesture recognition, global position system (GPS), car navigation system (CNS), animal mobility experiments, sports video trajectory analysis and automatic video surveillance have been implemented with new devices and algorithms. The major interest of trajectory-based object motion analysis is the motion trajectory recognition. The motion trajectory recognition is generally achieved by a matching algorithm that compares new input trajectory with pre-determined motion trajectories in a database.

The first generation of matching algorithms only used raw data to calculate the distance between two trajectories, which is ineffective. Raw data of similar motions will appear differently because of various varying factors such as scale and rotation. To overcome this problem, local features of trajectory, called signature, were defined for motion recognition [1–3]. This signature performs better in flexibility than other shape descriptors, such as B-spline, NURBS, wavelet transformation, and Fourier descriptor. Trajectories represented by the signature and the descriptors are invariant in spatial transformation. However, computing the distances between trajectories using this signature is not enough for accurate recognition of 3D motion. To improve the performance, some matching approaches were used to ignore similar local shapes of different motion trajectories or to ignore outliers and noise.

‘Matching’ is an important process in motion recognition and classification, which have been studied for years and widely used in many fields. It is achieved by alignment algorithm, and the famous and efficient ones in motion recognition are dynamic time warping (DTW), continuous dynamic time warping (CDTW), and longest common sub-sequence (LCSS) [4–6].

Recent researches use the full length of trajectory data for motion recognition [1–3]. However, many points of the trajectory have similar signatures because they lie on a straight line, thus computing task for signatures can be useless. To eliminate this drawback, we developed a new method that computes the signatures only from the region of ‘elbow points’ to gain advantage of computing speed. Besides, we also present a set of descriptors and normalization process for invariant motion recognition.

2 Preprocessing Method

Due to the system noise, measurement noise or both, trajectory data may not be accurate. ‘Smoothing’ process is an important task because it enhances the signature’s computational stability by reducing the noise and vibration of motion. However, trajectory shape may be affected by the smoothing process. To cope with the effect of noise, the derivatives of a smooth version of data using a smoothing kernel \( \phi \) are considered, i.e. \( x^{(j)} (t) = (x(t) * \phi (t))^{(j)} \). By the derivative theorem of convolution, we can have \( x^{(j)} (t) = x(t) * \phi^{(j)} (t) \). For this paper, a B-spline \( B(t) \) is taken to be the smoothing kernel \( \phi (t) \). An odd degree central B-spline of degree \( 2h - 1 \) with the integer knots \( - h,\; - h + 1,\;..,0,\;h - 1,\;h \) is given by

$$ B(s) = \frac{1}{(2h - 1)!}\sum\limits_{l = - h}^{h - 1} {( - 1)^{l + h} \;\left( {\begin{array}{*{20}c} {2h} \\ {l + h} \\ \end{array} } \right)} \;(s - l)_{ + }^{2h - 1} $$

where the notation \( f_{ + } (s) \) mean \( f(s) \) if \( f(s) \ge 0 \) and 0 otherwise. For a quantic B-spline, \( h = 3 \) [7].

Next, we normalize the location and the scale of a 3D trajectory so that all trajectories are transformed to a common domain. Trajectory normalization makes scale, rotation and translation invariant, which can produce better performance for the following processes. We applied the continuous principal component analysis (PCA) [8] to the trajectory points, where we assume that three distinct nonzero eigenvectors can be computed from the 3D trajectory coordinates. The continuous PCA ensures the invariance of the translation, the rotation, the reflection, and the scale.

3 Signature as a Trajectory Descriptor

For trajectory matching, we need a descriptor that can well describe the shape of the trajectory. In our study, we use the signature for the descriptor. For a point t, the signature S(t) is defined by five values: \( \kappa (t) \), \( \tau (t) \), \( h(t) \), \( e(t) \) and \( c(t) \). \( \kappa (t) \) is the ‘curvature’ that is a measurement for the turning amount of the contour, and \( \tau (t) \) is the ‘torsion’ that presents its twist amount out of the tangent-normal plane. Other three values \( h(t) \), \( e(t) \) and \( c(t) \) are the Euclidian distances from the point t to the start-point, the end-point, and the center-point of the trajectory, respectively. Note that the center-point is computed by the continuous PCA which is performed at the normalization process. Thus, for a motion trajectory in 3D space with N points\( \Upgamma = \{ x(t),y(t),z(t)|t \in [1,N]\} \), the signature set \( D^{*} \) for the entire trajectory is defined in the following form

$$ D^{*} = \{ \kappa (t),\;\tau (t),\;h(t),\;e(t),\;c(t)|t \in [1,N]\} $$

where

$$ \kappa (t) = \left\| {\dot{\Upgamma }(t) \times \ddot{\Upgamma }(t)} \right\|/\left\| {\dot{\Upgamma }(t)} \right\|^{3} $$
$$ \tau (t) = (\dot{\Upgamma }(t) \times \ddot{\Upgamma }(t)) \cdot \dddot \Upgamma (t)/\left\| {\dot{\Upgamma }(t) \times \ddot{\Upgamma }(t)} \right\|^{2} $$
$$ h(t) = \left\| {\Upgamma (t) - \Upgamma (1)} \right\| $$
$$ e(t) = \left\| {\Upgamma (t) - \Upgamma (N)} \right\| $$
$$ c(t) = \left\| {\Upgamma (t)} \right\| $$

An elbow point is a point on the trajectory which have the curvature value \( \kappa (t) \)larger than a threshold \( \phi \). If we know the coordinates of the elbow points and their sequential order, we can rebuild an approximated trajectory by connecting the elbow points with the straight lines of points. Consequently, information about the elbow points is good enough to align two trajectories for matching task. Therefore, a new literal set of signature only with the elbow points can be described as

$$ D^{\prime} = \{ \kappa (t),\;\tau (t),\;h(t),\;e(t),\;c(t)|t \in [1,\;N],\tau (t) > \phi \} $$

This new set of the signature only with the elbow points has four or five times less number of elements (signatures, points) than D*. As a result, computational burden for matching two trajectories can be dramatically reduced by using \( D^{\prime} \) rather than D*. An illustration for the elbow points (block dots) and the three distances are shown in the Fig. 1.

Fig. 1
figure 1

Illustration of elbow points (block dots) and 3 Euclidian distances

Full size image

4 Signature Alignment

As mentioned in previous section, each trajectory is represented by a set of signature. Note that in our proposed method, we only compute the signatures at the elbow points. For each elbow point, as mentioned above, five signature elements are obtained: \( \kappa (t) \), \( \tau (t) \), \( h(t) \), \( e(t) \) and \( c(t) \). In order to match two trajectories, two corresponding signatures should be correctly paired. Since there are many noisy factors such as different number of signatures in two trajectories, a matching approach should consider methods to handle the noisy factors. There are many approaches to match two set of sequence data such as LCSS and DTW [4]. The LCSS is more adaptive and appropriate distance measurement for trajectory data than DTW [1]. We therefore choose LCSS for matching process in our study.

Given an integer δ and a real number \( 0 < \varepsilon < 1 \), we define the \( LCSS_{\delta ,\varepsilon } (A,B) \) as follows:

$$ \left\{ \begin{gathered} 0\;if\;A\;or\;B\;is\;empty \hfill \\ \begin{array}{*{20}c} {1\; + \;LCSS_{\delta ,\varepsilon } \;(Head\;(A),\;Head(B)),} \\ {\begin{array}{*{20}c} {if|a_{{x,{\kern 1pt} n}} - b_{{x,{\kern 1pt} m}} |\; < \varepsilon } \\ {and|a_{{y,{\kern 1pt} n}} - b_{{y,{\kern 1pt} m}} |\; < \varepsilon } \\ {and|a_{{z,{\kern 1pt} n}} - b_{{y,{\kern 1pt} m}} |\; < \varepsilon } \\ {and|n - m|\; \le \delta } \\ \end{array} } \\ \end{array} \hfill \\ \hbox{max} (LCSS_{\delta ,\varepsilon } (Head(A),B),\;LCSS_{\delta ,\varepsilon } (A,Head(B))),\;otherwise \hfill \\ \end{gathered} \right. $$

The constant δ controls how far in time we can go in order to match a given point from one trajectory to a point in the other trajectory. The constant ε is the matching threshold. The similarity function S between two trajectories A and B, given δ and ε, is defined as follows:

$$ S(\delta ,\varepsilon ,A,B) = \frac{{LCSS_{\delta ,\varepsilon } (A,B)}}{\hbox{min} \,(n,m)} $$

This LCSS model allows stretching and displacement in time, so we can detect similarities in movements that happen at different speeds, or at different times.

5 Experimental Results

In order to implement and evaluate the proposed method for matching the 3D motion trajectories, we have used trajectories information of the Australian Sign Language (ASL) data set obtained from University of California at Irvine’s Knowledge Discovery in Databases archive [9]. The ASL trajectory dataset consists of 95 sign classes (words), and 27 samples were captured for each sign word. The coordinates x, y and z are extracted from the sign’s feature sets to calculate the trajectory signature. The length of the samples is not fixed. The details for the experimental setup are exactly the same as that described in [10], where the data set consists of sign words ‘Norway’, ‘alive’, and ‘crazy’. Each sign-word category has 69 trajectories.

Haft trajectories from each category were used for training, and the remains were used for testing. A test sample is classified by the nearest neighbor rule (k = 5). The experiment was repeated 40 times (each time with a randomly selected training and test datasets). The average result of recognition was 84.76 %. We also performed the experiment with pose normalization method [1]. Our proposed method was compared with other methods included PCA-based Gaussian mixture models (GMM) and global Gaussian mixture models [10], and the comparison result is reported in the Table 1.

Table 1 Sign-word recognition results using object motion-based trajectory
Full size table

Note that our proposed method used only a subset of trajectory data while other methods used the full length trajectory data. Even though the recognition result of our proposed method does not outperform the PCA-based GMM method, the number of data points for recognition process is much smaller, which implies less computational complexity. Therefore, our proposed method is more advantageous than PCA-based GMM in term of recognition speed.

6 Conclusion

In this paper, we proposed a new method for matching the 3D motion trajectories, and demonstrated experiments to show its effectiveness. It used only the features, named in signature, obtained from ‘elbow points’ which are the points that have the curvature value larger than a specific threshold.

In the first step, all trajectories are smoothed and then normalized by continuous PCA. By using continuous PCA, all trajectories are invariant in translation, rotation and scale. Once all the trajectories are normalized, a set of signature which contained both local features and global features of trajectory is computed from only the elbow points. LCSS matching algorithm was used to match the signatures from the elbow points in two trajectories. Comparison of one trajectory and another trajectory in a database, actually one set of signatures and another set of signatures in a database, is quite complicated if the database size is big and the length of the trajectory is long. Therefore, using only subset of full trajectory points is simple and fast in trajectory matching process.

Even though our method uses less information of the trajectory for matching, sign word recognition results showed that our proposed method can still maintains the recognition rate compared to the existing methods. This implies that the features from the elbow points are good enough to include the information for matching two trajectories. However, further works should include investment for the sensitivity of the threshold value to recognition results, which affects the number of elbow points. Also, practical application study should be performed with large number of sigh-words.