1 Introduction

The broad proliferation of information technology has aroused the real need for a reliable and easy automatic mechanism to identify individuals. Conventional means based on passwords and/or magnetic cards present a some limitations, as they may be stolen or compromised. In contrast, biometrics has been emerging and developing as a new and effective identification technology counterpassing these inconveniences. By human biometrics, we mean the use of physiological characteristic of human body parts and their appearances to identify individual human beings in the course of their daily activities. The appearances of body parts, especially in imaged data, have a large variability and are influenced by their shapes, colors, illumination environment, presence of other parts, and so on. Therefore, the biometrics researchers have focused on body parts and images that try to minimize this variability within class (subjects) and maximize it across classes. Although several modalities, such as face [47], iris [34], fingerprint [20], palmprint or hand geometry and the behavioral features like signature [52], handwriting [7, 8], voice [41], walk, speech or gait. Palmprint has drawn considerable attention of many researchers thanks to their merits high accuracy, high user-friendliness, low cost, high level of security and usability. It contains rich, stable and unique features like principal lines, wrinkles, ridges, minutiae points, singular points and texture. Some of these features, such as the minutiae, ridges and the singular points, need the high resolution palmprint images; at least 400 dpi. They are suitable for Medicolegal applications as the identification of criminals where the acquisition is done by the ink or the use of the latent prints for offline recognition. On the other hand, low resolution Palmprint images, less than 150 dpi, are suitable for civil and commercial applications like access control and real-time authentication where the recognition is done online. Although the minutiae, ridges and the singular points can not be revealed in low resolution images, principal lines, wrinkles and texture are easily extracted from these images. A reliable palmprint identification system is able to recognize the persons using the majority of the available information.

1.1 Related work

In this section, we provide an overview of the related work methods for palmprint identification systems. The later can be considered as the current and future key for solving security problems due to their several advantages, such as the low-resolution imaging with the low-cost capturing devices (cheaper devices), the higher level of security, difficulty to forge and user friendly interface, etc.

So far, these already existing proposed systems can be subdivided into three popular categories mainly on the basis of extracted features.

  1. (1)

    Texture-pattern-based-approaches are the global approaches which have been widely-studied. The task of recognizing the individuals by the texture of their palmprint has been approached in several ways, which leads to successful varying levels. Several approaches have been suggested to describe the texture of palmprint in recent years. Among these, we can mention those of which Hammami et al. [21] proposed an approach based on dividing the whole images into sub-images and applying the Local Binary Pattern (LBP) operator to analyze the texture for each sub-image. Then, these authors investigate the Sequential Forward Floating Selection (SFFS) algorithm in order to select the pertinent features of the palmprint texture. In order to determine the identity of a person, a chi-square X 2 statistic is used based on calculating the Chi-square distance between the image test and all the images. They evaluated their approach on two datasets with several challenges in terms of variations of orientation, position and illumination in capturing palmprint images. In [48], wang et al. described a novel algorithm for palmprint texture recognition based on combining the 2D Gabor wavelets and Pulse Coupled Neural Network (PCNN). These authors decomposed the palmprint images by 2D Gabor wavelets. Then, they employed the PCNN to decompose each resulting Gabor sub-band into a set of binary images, followed by a Support Vector Machines (SVM) classifier for classification. A roughly similar idea is proposed by khalifa et al. [29] that used the Gabor filters for feature texture extraction, treating the SVM as a classifier for palmprint identification. Moreover, the authors described two other methods to analyze the palmprint texture like the Co-occurrence Matrix (CM) and the Discrete Wavelet Transform (DWT) and used yet the SVM in classification phase. Charfi et al. [9] presented a new identification system focusing on the extraction of the palmprint texture using the Scale Invariant Feature Transform (SIFT). In [27], Jawsal et al. employed 2D Gabor filter to describe the palmprint texture information and used subspace methods for dimension reduction like the Principal Component Analysis (PCA) and the Linear Discriminant Analysis (LDA). Then, the Euclidean Distance (ED) is used for the matching phase. In [16], Fei et al. suggested an approach based coding theory including the Double-Orientation Code (DOC) scheme to show the palmprint orientation feature. Then, the authors chosen a nonlinear angular matching score metric to provide a assessment of the similarity between the DOC.

    Likewise, the fractal geometry theory is one of the most efficient achievements in mathematics in the 20th century as it contains several methods which are among the earliest successful methods used to analyze the image texture in the pattern recognition and the image processing fields [42]. In fact, such methods have been reliably applied to extract the behavioral features like handwriting [7] and signature [52] in the literature. These obtained results are very promising and have given high performance in the behavioral biometric recognition systems.

  2. (2)

    Line-based-approaches group the structural approaches which are based specifically on principal lines [49], wrinkles [10], ridge and feature point [15]. Numerous approaches have been proposed to extract the principal lines of the palmprint. In particular, Tunkpien et al. [46] proposed an approach to extract the principal lines of palmprint based on an average filter and magnitude gradient. Then, they used the morphological operators for smoothing the principal line contours. In [5], the principal line detection is performed by six steps which are normalization, median filtering, average filters, grayscale bottom-hat filtering, combinations of bottom-hat operations, binarization and post processing. Khan et al. [30] proposed another algorithm which uses niblack binarization, morphological operation and thinning to extract the principal lines. Then, they detected the endpoints of the lines and created a connection between them. Finally, the authors exploited the quantitative features from the triangle of Delaunay Triangulation (DT) in order to identify the persons. In [24], Huang et al. suggested a palmprint verification method based on extracting the principal lines using the Modified Finite Radon Transform (MFRAT). Then, they are used the pixel to-area comparison as matching metric to calculate the similarity between the palmprints. Han et al. [22] detected the palm lines by the Sobel and morphological operations. Wu et al. [49] extracted principal lines using canny edge algorithm. The feature vector is determined by the magnitude and orientation of the gradient for each edge point. Then, the fuzzy energy of each direction was calculated in order to generate this feature vector.

    Unfortunately, the previously proposed works could not explicitly and clearly extract the principal lines and achieve high recognition rates. Besides, there has not been any suggestion to determine the shape for each detected line with the existing challenge, such as illumination, direction and position variations which include the missing part, contraction and stretching of the principal lines, during the palmprint collection.

  3. (3)

    Intra-modal-based-approaches are the recently proposed approaches which are based on the information fusion technologies that have shown promising results in palmprint identification. Several approaches utilize the fusion technique based on multi-descriptors to define the multiple features that are eventually used in identification. In fact, Kumar et al. [33] provided an approach based on combining the various representations which represent the multi-features such as Gabor-based texture feature, line feature and PCA feature. Similarly, Krichneswari et al. [31] proposed an intra-modal feature fusion system based on extracting multiple features like Gabor texture and Line features for palmprint recognition. Then, the authors used the Particle Swarm Optimization (PSO) and PCA to reduce the feature vector and matched them with stored template using the Nearest-Neighbor (NN) classifier. Likewise, Luo et al. [35] used the LBP-structure descriptors and the Local Line Directional Pattern (LLDP) with different parameters to extract the texture and the line space, respectively. In [51], the authors proposed a new hierarchical method for palmprint recognition based on generating the major principal lines with a probability distribution template and applying the Dual Tree Complex Wavelet (DTCW) to extract the texture feature of palmprint image. In [32], the authors investigated a novel method using nonlinear rank level combination of multiple palmprint feature representations which are principal lines, Eigen Palm and Gabor filter texture. Hu et al. [23] proposed a new algorithm based on fusing three important features of palmprint, such as texture feature, principal lines and appearance feature applying 2D Gabor filter, Modified Finite Radon Transform (MFRAT) and Orthogonal Discriminant Locality Preserving Projections (ODLPP), respectively.

    The intra-modal based approaches conclude that using multi-features yields better results than using a single feature for a single biometric modality (unimodal).

1.2 Contribution of the work

As described in the state-of-the-art approaches, the combination of the various palmprint features represents a reliable way to achieve a higher level of performance in palmprint identification [31, 32, 51].

Thus, in this paper we propose to fuse principal line features and texture features for efficient palmprint identification. In summary, the main contributions of this paper are:

  • Principal line extraction from palmprint and elastic shape analysis of resulting curves. Actually, the previous principal lines based approaches did not investigate the elasticity of these lines. Moreover, we investigate intrinsic mean calculation of principal curve and geodesic distance calculation. This distance is invariant to the scale, the pose and the re-parameterization of these curves. Thus, only the shape of the principal lines are evaluated.

  • A fractal based analysis of the texture image of the palmprint. The fractal dimension is calculated for all sub-regions of Region of Interest (ROI) of palmprint using the Box Counting method (BC) in order to describe the texture features of the palmprint ROI. This BC method focuses on dividing each sub-region into a number of boxes for various scales 𝜖 and computing the number of the boxes containing the texture information N(𝜖) for each scale.

  • Fusion of fractal-based feature vector and shape analysis-based one and efficient palmprint classification. A feature level fusion is applied, the features emanating from shape analysis of principal lines and fractal issued features are concatenated to form the feature vector and the classification is based on the multi-class version of Random Forest algorithm.

  • An extensive experimental analysis involving three different datasets which are PolyU-Palmprint dataset, CASIA-Palmprint dataset and IIT-Delhi-Palmprint-V1 dataset.

The remainder of this paper is organized as follows. The fractal analysis of the texture and the shape analysis of the principal lines are detailed in Section 2. The fusion of the texture and principal lines is presented at the end of the same Section. The experimental settings, the dataset used and also the report results in terms of accuracy of palmprint recognition in comparison with state of the art solutions are described in Section 3. Finally, in Section 4 the concluding remarks are drawn and future research directions discussed.

2 Proposed approach

In this paper, we investigate an intra-model palmprint identification system. The technique consists in efficiently combining the principal line shape analysis and the texture pattern analysis to make the system more robust. The overview of the shape analysis approach and the fractal based method are depicted in Fig. 1a and b respectively. The principal lines are extracted as three types of curves ( C 1, C 2 and C 3) based on Steerable filter and Hysteresis thresholding as illustrated in Fig. 1a The shape analysis framework is then applied between the extracted curves. Representatives for each person P e r j are calculated using training data for each type of curve ( C 1, C 2 and C 3), where j={1,2,...,k}. These representatives are the intrinsic means provided by the shape analysis framework, of all curves belonging to the same person P e r j and same curve type C i . The feature vector for a given curve β of type C i is calculated based on the geodesic distance between β and all the means corresponding to the same type C i . A probe image is considered as a set of three curves of different types.

Fig. 1
figure 1

Overview of the proposed method a The different steps of Shape analysis: in the left, the extracted principal lines for each person in the gallery data; in the middle, the calculated Means ( μ 1, μ 2 and μ 3) for each person; in the right, the recognition of the given image based on the Geodesic distance ( d S ) between the given curve and the obtained Means b The different steps of Fractal analysis: ROI Preprocessing (Binarization and edge detection of ROI), the ROI division into (r × r) sub-regions and the fractal dimension for each sub-region is calculated by the box counting method

The texture information is next investigated based on fractal analysis as illustrated in Fig. 1b. The texture pattern information is extracted based on the fractal approach on calculating the fractal dimension obtained by the box counting method. Furthermore, we combine the relevant information of the different obtained features from the principal line shape and the texture pattern to increase the system accuracy. Finally, the palmprint recognition is performed by Random Forest Classifier after fusion of shape analysis-based feature and fractal-based one.

2.1 Texture analysis: Fractal theory

2.1.1 Background on fractal analysis

The fractal theory is an efficient technique which was developed by Mandelbrot [17] to design the objects which have a very irregular, interrupted or fragmented (geometrically complicated) shape. A fractal object is a mathematical object which comes from an iterative process and has a self-similarity character, i.e. its shape is repeated at different scales. Each object is then characterized by the Fractal Dimension that is often denoted by FD which was obtained by several methods. As will be shown in the following equation, the division of dimension D of the fractal object by a scale factor (1/𝜖) gives a number of objects: N.

$$ N=(1/\epsilon)^{D} $$
(1)

The following equation represents the application of the logarithm function:

$$ log(N)=D\log(1/\epsilon) $$
(2)

Thus, Fractal Dimension FD is determined by the following equation:

$$ FD=\lim \limits_{\epsilon \to \ 0}{log (N(\epsilon)) \over log(1/\epsilon)} $$
(3)

Where, N(𝜖) is the number of specimen of the initial object, (1/𝜖) is the scale factor and FD is obtained by a least squares regression method.

Nowadays, the concepts resulting from this fractal theory are used in various scientific fields like computer science, medicine and specifically in the analysis of the image texture pattern. Therefore, the texture analysis by fractal geometry is the major field in the image retrieval process. It is obtained by calculating the fractal dimension. Several methods are granted to estimate this fractal dimension of a texture pattern such as the Box Counting Method (BC), the Differential Box Counting Method (DBC), the Box Counting Density (BCD), etc. The Box Counting (BC) method is widely used for estimating the FD thanks to its effectiveness to measure the complexity and irregularity of the image texture pattern. This method is basically interested in partitioning the image space into a number of boxes (fixed-grid of square boxes) for different scales 𝜖 and in calculating the number of the boxes containing the information N(𝜖) for each scale.

2.1.2 Texture feature vector extraction

The features extraction is a substantial step of the palmprint biometric recognition. In this paper, we investigate the applicability of the fractal dimension to extract the pertinent texture features of the palmprint. Firstly, we apply a low pass filter for reducing the noise of the ROI. Then, we extract the ROI edge and convert each one from a grayscale image to a binary image. Afterward, we split the preprocessing ROI palmprint, successively, into r × r sub-regions equivalent square: S r e g 1, S r e g 2,..., S r e g (r × r). Therefore, we have obtained r × r sub-regions for each ROI, each of them has a normalized side S/r, where S is the size of palmprint ROI. Secondly, we calculated the fractal dimension for each sub-region (5) using the Box Counting method (BC) to describe the texture features of the palmprint. This BC method focuses on dividing each sub-region into a number of boxes for various scales 𝜖 and computing the number of the boxes containing the texture information N(𝜖) for each scale. The Fractal Dimension FD is obtained by a least squares regression method or the slope of the linear regression line (curve) that we plot its Y-axis which consists of the value of l o g(N(𝜖)) against its X-axis which consists of the l o g(1/𝜖).

$$ FD= slop {log (N(\epsilon)) \over log(1/\epsilon)} $$
(4)

Finally, the concatenation of these fractal dimensions will be stored in a vector V texture that represents our texture features vector for an palmprint ROI image, as described in the following equation:

$$ \textbf{V}_{\textbf{texture}}= FD_{1}, FD_{2},..., FD_{(r\times r)} $$
(5)

2.2 Shape analysis of principal lines

The principal lines convey several information which can uniquely identify an individual. There are usually three principal lines that are more noticeable, remarkable and vary somewhat over time such as life line, head line and heat line (see Fig. 2). These principal lines have different orientation, depth and shape. Therefore, they are irregular even for the same palm of a person, which implies the variability intra-class and thus the difficulty of their extraction.

Fig. 2
figure 2

The three principal lines of the palmprint

The commercial palmprint recognition system still has several challenge problems, such as intra-class variations (e.g., texture affected by ageing, contraction and stretching during the palmprint pose collection for the same person leads to structure changes and elastic shape of palmprint principal lines) and noise in data (e.g., illumination effect to palmprint image pixels leads to shape deformations of palmprint principal lines).

The previous approaches are interested on the structure similarity of the principal lines among the same people but ignored the elastic and deformation shape producing by the different challenges.

Generally speaking, the MFRAT is also a powerful program to extract the principal lines with different widths [23, 24]. However, this method is not suitable in the proposed approach. Actually, we propose to analyze the shape of principal lines using a shape analysis framework. The shape of these curves is investigated independently of their width, the proposed shape analysis allows elastic analysis of principal line curves. Moreover, it allows intrinsic mean calculation of principal curve and geodesic distance calculation. This distance is invariant to the scale, the pose and the re-parameterization of these curves. We are convinced that the principal lines extraction with a one-pixel width of is more suitable for our approach. Thus, only the shapes of these principal lines are evaluated. Therefore, we need to use a simple filter as ”Steerable Filter” to extract the three principal line curves for each person. Then, we applied a shape analysis framework to analyze the shape of these curves. Outcome of this principal lines shape analysis are itemized in the following subsections.

2.2.1 Lines extraction

In this section of the paper, we present a new method to extract the principal lines of the palmprint based on the feature detection that contained two efficient descriptors like the Steerable Filter and Hysteresis Thresholding which are suggested by the elegant work of Freeman and Adelson [18].

Steerable filter

This filter of steerablity is a robust descriptor for the noise and illumination changes. It is successfully widely-applied in a great deal of image processing and pattern recognition areas such as the image denoising [43], color image segmentation [6], texture analysis [11] and the edge detection and ridge detection [19, 28], etc. In this paper, we propose steering the ridge filter to compute the ridge orientations i.e. filter with response in orientation. Thus, we follow a generic approach for ridge detector design from a class of the different functions of Steerable filter which are already observed by Jacob and Unser [26]Footnote 1. We use the second order detector of feature template which can be implemented as an Eigen-decomposition of the Hessian matrix [26] that can be interpreted in terms of Steerable filters for ridge detection [18].

Generally, the steerable filter plugin contains several parameters which are indicated as follows:

Input :

The original palmprint ROI image.

M or Order :

The order of the feature template Order up to 5 are supported in the current implementation i.e. M = (1,2,3,4,5). Where, for edge detection M= 1, 3 or 5 and for ridge detection M= 2 or 4.

sigma ( σ ) :

The standard deviation of the Gaussian on which the steerable templates are based. This controls the feature width.

set :

Set number in case there are multiple configurations for order M.

mu ( μ ) :

Smoothness parameters used during the filter optimization.

Filter Weights :

Weight vector: Gxx, Gyy, Gxy, etc. The array of weights that control the linear combination of Gaussian derivatives up to order M which constitute the feature template.

In our experiment, the different parameters used are M = 2, set1 with (μ = 0.0), σ = 3.0 and filter Weights Gxx = 3.0 and Gyy = 1.0 (The second derivation of Gaussian), as illustrated in Fig. 3.

Fig. 3
figure 3

The different parameters of Steerable filter: The order of the feature template, The smoothness parameters used during the filter optimization, The standard deviation of the Gaussian on which the steerable templates are based and the Weights vector

The obtained results mentioned in Fig. 4b, c and d correspond to the output of the ridge detection (principal lines and some strong wrinkles) based on the Steerable filter descriptor.

Fig. 4
figure 4

Steerable Filter, Non-maximum suppression and Hysteresis Thresholding a The original palmprint ROI b The 2nd Order Response Image or the Gradient Norm Image c Gradient Orientation Image d The Ridges Detection by Non-maximum suppression and gradient parameters: I m a g e R i d g e e The final extracted principal lines: I m a g e PL i n e s (The Hysteresis Thresholding)

The Steerable filter is able to extract the ridges which are contain the three principal lines and some strong wrinkles in some images. These strong wrinkles exist in some images owing to their variability inter-class and intra-class proved according to change of illumination. Next, we are interested in extracting only the principal line and thus eliminating these strong wrinkles which present the small lines.

Hysteresis thresholding

Since the variation of illumination during the collection of the palmprint image in the dataset, the resulting extracted ridge in some images include both principal and some strong wrinkles lines. To resolve this problem, we improve the detected ridges using Non-maximum suppression and the Hysteresis filter Thresholding which are able to reduce the probability of false ridges and fragmented lines through the following stages.

  1. 1.

    The input image: original palmprint ROI image (See Fig. 4a);

  2. 2.

    Select the obtained 2nd order response image or Gradient image (Fig. 4b);

  3. 3.

    Select the obtained Gradient Orientation image (Fig. 4c);

  4. 4.

    Apply the Non-maximum suppression to locate the lines of ridges: create new image local maxima where only local maxima should be marked as line and this image takes the same size of the original ROI palmprint image (Fig. 4d).

  5. 5.

    Apply the Hysteresis filter for thresholding to eliminate spurious responses.

    It can be seen from Fig. 4d that the resulting extracted image, I m a g e R i d g e , includes three principal lines and some strong wrinkles. Therefore, it is necessary to determine the potential ridges and to choose two thresholds which are T H i g h and T l o w . Consequently, we obtained the final image (see Fig. 4e) which is called I m a g e PL i n e s . The pseudo-code of Hysteresis Thresholding algorithm is summarized in algorithm 1. Hysteresis ThresholdingInput: I m a g e R i d g e : the image of ridge detection T H i g h and T l o w :The two thresholds Output: I m a g e PL i n e s (x,y): the final principal lines image Begin If I m a g e R i d g e (x,y)>T H i g h Then A pixel (x,y) is called strong; Else if I m a g e R i d g e (x,y)<=T l o w Then A pixel (x,y) is called weak; Else All other pixels are called candidate pixels: I m a g e PL i n e s (x,y) End If 1- In each position of (x,y): If pixel (x,y) is weak Then Discard this pixel End If If pixel (x,y) is strong Then Take the pixel as output. End If 2- If the pixel (x,y) is a candidate Then Follow the chain of connected local maxima in both directions along the line, as long as I m a g e R i d g e (x,y)>T l o w . End If 3- If the starting candidate pixel (x,y) is connected to a strong pixel Then Take this candidate pixel as output. Else Not take the candidate pixel as output. End If End

  6. 6.

    The final principal lines are determined using Hysteresis Thresholding based on suppressing all the ridges that are strong wrinkles and the small unconnected lines.

figure d

2.2.2 Background on the shapes of curves

Let \(\beta :I \to \mathbb {R}^{2}\), represent a parameterized curve representing a principal line, where I = [0,1]. To analyze the shape of β, we shall represent it mathematically using the square-root velocity function (SRVF) [44], denoted by q(t), according to: \(q(t) = {\dot {\beta }(t) \over \sqrt { \|\dot {\beta }(t)\|} } \); q(t) is a special function of β that simplifies computations under elastic metric.

Actually, under \(\mathbb {L}^{2}\)-metric, the re-parametrization group acts by isometries on the manifold of q functions, which is not the case for the original curve β. To elaborate on the last point, let q be the SRVF of a curve β. Then, the SRVF of a re-parameterized curve βγ is given by \(\sqrt {\dot {\gamma }} (q \circ \gamma )\). Here γ:II is a re-parameterization function and let Γ be the set of all such functions.

Define the preshape space of such curves:

\(\mathcal {C} = \{q: I \to \mathbb {R}^{2}| \|q\| = 1 \}\ \subset \ \mathbb {L}^{2}(I,\mathbb {R}^{2})\), where ∥⋅∥ implies the \(\mathbb {L}^{2}\) norm. With the \(\mathbb {L}^{2}\) metric on its tangent spaces, \(\mathcal {C}\) becomes a Riemannian manifold. Also, since the elements of \(\mathcal {C}\) have a unit \(\mathbb {L}^{2}\) norm, \(\mathcal {C}\) is a hypersphere in the Hilbert space \(\mathbb {L}^{2}(I,\mathbb {R}^{2} )\). The geodesic path between any two points \(q_{1}, q_{2} \in \mathcal {C}\) is given by the great circle, \(\psi : [0,1] \to \mathcal {C}\), where

$$ \psi(\tau) = {1 \over \sin(\theta)} \left(\sin( (1 - \tau )\theta) q_{1} + \sin(\theta \tau) q_{2} \right), $$
(6)

and the geodesic length is 𝜃 = d c (q 1,q 2)=c o s −1(〈q 1 q 2〉).

In order to study shapes of curves, one identifies all rotations and re-parameterizations of a curve as an equivalence class. Define the equivalent class of q as: \([q] = \text {closure}\{ \sqrt {\dot {\gamma }(t)} O. q(\gamma (t)) ,\ \ \gamma \in {\Gamma }\}\), where OS O(2) is a rotation matrix in \(\mathbb {R}^{2}\). The set of such equivalence classes, denoted by \(\mathcal {S} \doteq \{ [q]| q \in \mathcal {C}\}\) is called the shape space of open curves in \(\mathbb {R}^{2}\). As described in [44], \(\mathcal {S}\) inherits a Riemannian metric from the larger space \(\mathcal {C}\) due to the quotient structure. To obtain geodesics and geodesic distances between elements of \(\mathcal {S}\), one needs to solve the optimization problem:

$$ (O^{*},\gamma^{*}) = argmin_{\gamma \in {\Gamma}, O \in SO(2)} d_{c}( q_{1}, \sqrt{\dot{\gamma}} O.(q_{2} \circ \gamma)). $$
(7)

The optimization over Γ is done using the dynamic programming algorithm for a fixed O in S O(2), and the optimization over S O(2) is performed using SVD for a fixed γ∈Γ. The solution is reached by iterating over the two previous steps. Let \(q_{2}^{*}(t) = \sqrt {\dot {\gamma ^{*}(t)}}O^{*}.q_{2}(\gamma ^{*}(t)))\) be the optimal element of [q 2], associated with the optimal re-parameterization γ of the second curve and the optimal rotation O , then the geodesic distance between [q 1] and [q 2] in \(\mathcal {S}\) is \(d_{s}([q_{1}],[q_{2}]) \doteq d_{c}(q_{1}, q_{2}^{*})\) and the geodesic is given by (6), with q 2 replaced by \(q_{2}^{*}\). This representation was previously investigated for biometric [1, 1214] and soft-biometric applications [2, 50] based on the face shape.

2.2.3 Mean calculation of principal lines

One advantage of a shape analysis framework of the principal lines is that one has the actual deformations in addition to distances. In particular, we have a geodesic path in \(\mathcal {S}\) between the two curves β 1 and β 2 in \(\mathbb {R}^{2}\). This geodesic corresponds to the optimal elastic deformations of two curves. We can use our framework to calculate some statistics as the ”mean” of curves in the space of the shape of the curves lying in \(\mathbb {R}^{2}\).

The Riemannian structure defined on the manifold of shape of the curves in \(\mathcal {S}\) enables us to perform such statistical analysis for computing principal lines means and variance. The Karcher mean utilizes the intrinsic geometry of the manifold to define and compute a mean on that manifold. It is defined as follows: let d s (β i,β j) denote the length of the geodesic from β i to β j in \(\mathcal {S}\). To calculate the Karcher mean of curves {β 1,...,β n} in \(\mathcal {S}\), define the variance function:

$$ \mathcal{V} : \mathcal{S} \rightarrow \mathbb{R}, \mathcal{V} (\beta)= \sum\limits_{i=1}^{n} d_{s}(\beta,\beta^{i})^{2} $$
(8)

The Karcher mean is then defined by:

$$ \mu=\arg \min_{\eta \in \mathcal{S}}\mathcal{V}(\eta) $$
(9)

The intrinsic mean may not be unique, i.e. there may be a set of points in \(\mathcal {S}\) for which the minimizer of \(\mathcal {V}\) is obtained. To interpret geometrically, \(\overline {\beta }\) is an element of \(\mathcal {S}\), that has the smallest total deformation from all given curves. We present a commonly used algorithm for finding Karcher mean for a given set of line curves. This approach, presented in algorithm 2, uses the gradient of \(\mathcal {V}\), in the space \(T_{\mu }(\mathcal {S})\), to iteratively update the current mean μ.

figure e

Since this is a gradient approach, it only ensures a local minimizer of the variance function \(\mathcal {V}\).

The motivation for using statistical means of shapes in a biometric application is to represent several principal lines belonging to the same class by an intrinsic mean shape. The feature vector for a curve classification is based on the calculated means.

2.2.4 Shape feature vector

The feature vector is built using the distances to the means of the principal line curves belonging to the same persons, calculated on train data as follow. Given train sets \(T_{1} = \{{\beta _{1}^{1}},...,{\beta _{1}^{n}}\} \in {\mathbb {R}^{2 \times n}}\), \(T_{2} = \{{\beta _{2}^{1}},...,{\beta _{2}^{n}}\} \in {\mathbb {R}^{{2 \times n}}}\), and \(T_{3} = \{{\beta _{3}^{1}},...,{\beta _{3}^{n}}\} \in {\mathbb {R}^{{2 \times n}}}\) holding respectively C 1, C 2 and C 3 curves, belonging to k different persons. Where n is the curve number of same type C i in the train data. Each curve corresponds to a person l a b e l j ∈{P e r 1,...,P e r k }. We first calculate, using algorithm 2, the means \({\mu _{1}^{j}}\), \({\mu _{2}^{j}}\) and \({\mu _{3}^{j}}\) for each class (person). Next, we calculate the geodesic distance d S between a given curve β and the mean curves of the same type ( C 1,C 2 or C 3). Thus a vector of distance of size k is provided as feature vector to classify the curve β.

In order to classify the principal lines provided by a palmprint, we concatenate the feature vectors for the corresponding β 1, β 2 and β 3 of types C 1,C 2 and C 3 respectively. the palmprint resulting feature vector, denoted V shape Lines , includes the distances between β 1 and the curves of type C 1, β 2 and the curves of type C 2 and β 3 and the curves of type C 3; the size of the feature vector is k × 3.

$$ \begin{array}{lllll} \textbf{V}_{\textbf{shape Lines}}=(d_{S}(C_{1},{\mu_{1}^{1}}), d_{S}(C_{1},{\mu_{1}^{2}}),..., d_{S}(C_{1},{\mu_{1}^{k}})),\\ (d_{S}(C_{2},{\mu_{2}^{1}}),d_{S}(C_{2},{\mu_{2}^{2}})..., d_{S}(C_{2},{\mu_{2}^{k}})),\\ (d_{S}(C_{3},{\mu_{3}^{1}}), d_{S}(C_{3},{\mu_{3}^{2}})...,d_{S}(C_{3},{\mu_{3}^{k}}))). \end{array} $$
(10)

2.3 Fusion of shape and texture for palmprints recognition

2.3.1 Feature vector building

Feature fusion is a good way to improve the performance and increase the system accuracy. In this paper, the features are issued from the same modality: thus they are called intra-modal features.

Several levels of fusion have been presented in literature to reach the robustness of system such as, (1) fusion at feature level, (2) fusion at feature score level and (3) fusion at feature decision level. In this paper, we focus on the feature level fusion that concatenates the features vectors: V texture and V shape Lines of a palmprint which are obtained using Fractal as well as Shape Analysis, respectively.

2.3.2 Classification

For the classification phase, we used the Multi-class version of Random Forest algorithm. This algorithm was proposed by Leo Breiman in [4] and defined as a meta-learner comprised of many individual trees. It was designed to quickly operate over large datasets and more importantly to be diverse by using random samples to build each tree in the forest. Diversity is obtained by randomly choosing attributes at each node of the tree and then using the attribute that provides the highest level of learning. Once trained, Random Forest classifies a new person from an input feature vector by putting it down each of the trees in the forest. Each tree gives a classification decision by voting for this class. Then, the forest chooses the classification having the most votes (over all the trees in the forest). In our experiments, we used Weka Multi-class implementation of Random Forest algorithm by considering 250 trees. A study of effect of the number of trees is mentioned later in the experimental result part.

3 Experimental results

This section recapitulates our empirical results and provides an analysis of the performance of our method compared to other state-of-the-art results using three large datasets which are ”PolyU-Palmprint” dataset [40], ”CASIA-Palmprint” dataset [3] and ”IIT-Delhi-Palmprint” dataset [25].

Our experiments were evaluated on an AMD Dual Core Processor system and all simulations were performed using MATLAB 2010. The main experimental results of these datasets are reported in the following sub sections.

3.1 Data pre-processing

The palmprint data pre-processing step is a prerequisite and nontrivial in the biometric system. It consists in extracting the region of interest (ROI) of the palmprint using several operations [3638]. As illustrated in Fig. 5, this data preprocessing step includes the following items:

  • Binarization: The grayscale original image is first converted into binary image by using the Otsu’s method [39], based on the segmentation of the input image into foreground and background.

  • Morphology Operations: Sometimes, the users put their hands with rings, which can create a problem of identification. In order to solve and avoid this problem, the morphology operations: the closing and the opening operations, are used.

  • Edge Detection: the edge of the image is detected and a smoothing filter (low pass filter) is applied to eliminate the noise and remove the holes curve.

  • Coordinate System Stabilization: the key points between the fingers (specifically the two finger-webs: P w1 and P w2 as indicated in Fig. 5) are extracted in order to stabilize the coordinate system which is able to successfully locate the region of interest ( F1F2F3F4) in the form of a square. Then, a mask having the same coordinate of this resulting square on the original image is applied to correctly extract the ROI from the rest of the hand.

  • ROI Normalization: this step consists of the following phases:

    Firstly, the palmprint images must be resized to a standard dimension since they may have different sizes due to variability from one hand to another. Hence, they must fix the size of the located ROI to T × T with T = 150 pixels. Secondly, the extracted ROI should be rotated to a vertical position as the palmprint images were taken with different orientations. Finally, a low pass filter is applied to reduce the noise in order to improve the quality of the extracted ROI.

Fig. 5
figure 5

Flow chart of the palmprint preprocessing step: There are various pipelines for this step. Namely, the pipelines are (1) Binarization, (2) Morphology Operations: the closing and the opening operations, (3) Edge Detection, (4) Coordinate System Stabilization and ROI extraction, and (5) ROI Normalization: The resizing and rotating of ROI

3.2 Comparative evaluation on the PolyU-Palmprint dataset

3.2.1 Dataset description and experimental setting

The Hong Kong Polytechnic University PolyU-Palmprint dataset was acquired by collecting 7752 palmprint images captured from 386 different palms. Each palm is represented by twenty samples acquired in two sessions (10 samples by session). The average time interval between the two sessions is two month. The capture system captures the palmprint images using a real time CCD capture device. All palmprint images of this dataset are 8 bits gray–level, BMP format and the size of each of them is 384 × 284 pixels. These images collected in the same position and direction in the two sessions which implies that all users have the freedom to hold their hands on the capture system with any restriction. Furthermore, there is a variation and a change of the distributed illumination between the first and the second sessions which brings the difficulty of the person’s identification.

Figure 6 shows the used capture system, some examples of these palmprint images.

Fig. 6
figure 6

Palmprint Image Scan of “PolyU-Palmprint” dataset a The used capture system b Some examples of the palmprints

Figure 7 depicts four palmprint ROI images of PolyU database, which were captured from a same person but in different sessions. This figure presents the first two images which were captured in the 1st session and the last two images which were captured in the second session. It can be noticed that there are great changes of illumination between these images captured in different sessions.

Fig. 7
figure 7

Four palmprint ROI images from a same person in PolyU database a two images captured in the first session b two images captured in the second session

This database was automatically preprocessed as described in the Section 3.1. Therefore, a square region is generally extracted as the ROI with a fixed size ( 128 × 128 pixels).

In the palmprint biometric identification system, the assessment of their performance obliges the construction of two image sets which are gallery and probe sets. Indeed, the gallery set includes the palmprint images classified for the training, which are known and noted by the system. The probe set includes the images organized for the testing, which are unknown palmprint images by the system and presented for identification phase. Each image of the probe set will be compared to the records of the existing images in the gallery set and thus found the class in which this palmprint image belongs.

We assess the performance of our approach for palmprint identification based on two experiments with this dataset.

In the first experiment, we randomly selected 7 samples of each person which were taken from the 1st session as the gallery set and the remaining samples of the same session (1st session) make up the probe set. Therefore, we used 2702 samples for gallery set (Training) and 1158 samples for probe set (Testing).

In the second experiment, all the 10 samples of each person which were taken from the 1st session built the gallery set and 5 randomly selected samples from the 2st session make up the probe set. Therefore, we used 3860 samples for gallery set (Training) and 1930 samples for probe set (Testing).

3.2.2 Experimental results and comparison

In our experiments, we presented two types of comparisons between the proposed approach and the most famous systems in the state-of-the-arts using similar protocols. The difference between the presented protocols lies in the number of users considered in both the gallery and the probe sets.

We present the results on this dataset in three different tables: Table 1 compares our principal line results with the principal lines based on the state-of-art methods. In Table 2, we present a comparative study of texture-based results of the proposed approach with the state-of-the art of texture based approaches. Table 3 presents the results of the proposed approach in comparison with the intra-modal state-of-the art methods.

Table 1 Recognition results of the extracted principal lines comparison of the state-of-the-art results on the PolyU-Palmprint database. Protocols Description: Samples of person (S), Users in dataset (U), All Users, Gallery Set (G), Probe Set (P)
Table 2 Recognition results of the extracted texture comparison of the state-of-the-art results on the PolyU-Palmprint database. Protocols Description: Samples of person (S), Users in dataset (U), All Users, Gallery Set (G), Probe Set (P)
Table 3 Recognition results of the extracted Intra-modal features comparison of the state-of-the-art results on the PolyU-Palmprint database. Protocols Description: Samples of person (S), Users in dataset (U), All Users, Gallery Set (G), Probe Set (P)

As illustrated in Table 1, although the number of users used for testing is bigger compared to well-known methods, where only 100 users are considered, the proposed approach outperforms the totality of previous approaches.

Obviously, this performance of Recognition Rate (RR) in the first experiment reaches 93,13 %, which brings the improvement rates of about 3.13 %, 0.63 % and 6.95 %, respectively to the result in [30], [23] and [46].

On the other hand, in the second experiment, we observed yet a better RR than the work in [31]. This performance of RR reaches 91,00 % when we use the 10 samples for the gallery set and 5 samples for the probe set in the matching phase. This scenario is important in a real application where gallery and probe sets are acquired in different sessions.

To make the fair comparison with that krishneswari et al.[31], we further test the proposed approach using the same setting of samples classification for gallery set and probe set proposed by the work described in krishneswari et al.[31] (5 samples in the gallery set and 10 samples for the probe set). It can be concluded from this result that the proposed approach likewise yields a better performance of recognition rate compared to the other work but it cannot gives a better recognition rate than the first sample classification. Hence, it is necessary to increase the number of samples (images) in the gallery set. This is in concordance with the results provided by Hammami et al. [21] and Tana et al. [45], where they show that it is important that the sample number in the training data (gallery set) be high to achieve a best performance of RR.

Table 2 summarizes approaches and results of texture analysis reported previously on the PolyU dataset, compared to those obtained in this work. The testing protocols used in the experiments are quite different, especially the number of samples in the gallery set and probe set, except in [29] where the number of sets of gallery and probe is not specified. The number of users considered represent all the users of the dataset (386 users) in several works, except in [23] and [31] where the number of users is restricted to 100.

In [48], the results are presented for palmprint recognition accuracy on all subjects picked out from the PolyU dataset. The authors improve the performance with two experiments. In the first experiment, 5 samples of each subject from the 1st session are selected for the gallery set and 5 samples of the same session for the probe set. In the second experiment, 5 samples from the 1st session are selected for the gallery set and 10 samples from the 2st session are selected for the probe set. The best recognition scores on PolyU for two experiments are reported by Wang et al. [48] which are 97.37 % and 95.48 %, respectively. In order to keep the comparisons fair, we kept the 5 samples in the gallery and 10 samples in the probe. It should be noticed that our method achieved a 95.00 % recognition which is close to the highest published result on this sample classification in the second experiment [48]. It can be concluded that the obtained results of our proposed approach give a better performance of Recognition Rates than the outcomes of the works in [29], [23] and [31]. Furthermore, according to this comparative analysis, the proposed approach is favorably comparable with the best state-of-the-art solutions [21] and [48].

Table 3 recapitulates a comparison of the obtained results of this work with previous works in intra-modal features extraction. Compared to our work, the works in [23] and [31] focus on a small dataset user number, which leaves uncertainty about the statistical significance of their performance on large and important challenging dataset. These two works fuse three features of palmprint. Nevertheless, Zhi et al. [51] combine two features of the palmprint as our proposed approach such as the texture pattern and the principal line shape of the palmprint. Hu et al.[23] and Zhi et al. [51] provide their experiments with the classification of samples in the same session for the gallery set and the probe set. The work of [31] that uses the mixture between two sessions of the dataset for samples classification as ours yielded 94.89 % recognition rate, whereas we reached a 96.99 % for 10 and 5 samples in the gallery set and probe set, respectively and 96.45 % during the use of the same sample classification in [31].

The most similar work to ours is provided in [51]. With all the subjects of PolyU and principal lines and texture fusion, [51] achieved 97.82 % recognition rate, while we achieved a much higher result of 98.32 % by combining principal line shape and texture pattern. From this table, it can be seen that the proposed feature fusion approach achieves the best performance.

3.2.3 Effect of number of ROI division

In order to validate the choice of 10 as a number of the ROI division into sub-regions, we presented the following Table 4. It shows the result of recognition rates of the PolyU-Palmprint dataset achieved in our experiments which was noticed for 12 different splits of ROI into sub-regions (See Fig. 8).

Table 4 The achieved recognition rates by applying the Fractal descriptor with different splits of ROI
Fig. 8
figure 8

Palmprint identification results using a Fractal Analysis Descriptor with different number of ROI Division

We studied the recognition rates of the PolyU-dataset achieved by applying the fractal descriptor for the different divisions of ROI palmprint image into sub-regions and thus it can be concluded that division 10 gives the best Recognition Rate. We apply the same number of divisions for all our experiments.

3.2.4 Effect of the number of trees in random forest algorithm

The performance of Random Forest classifier changes with the different number of trees. Therefore, we perform the experiments with several number of trees; thus, the obtained results are illustrated in Table 5 and depicted in Fig. 9. These experiments demonstrate that the recognition rates rise with the increasing number of trees until 250, and then become quite decrease. Thus, the best recognition rates are 96.65 %, 93.13 % and 98.32 % for texture analysis, shape analysis and intra-modal analysis, respectively; achieved by 250-Tree Random Forest.

Table 5 The different Recognition Rates with several number of trees of Random Forest
Fig. 9
figure 9

Palmprint identification results using a Random Forest when varying the number of trees: Intra-modal approach based fusion (Red curve), Principal lines based shape analysis (green curve) and Texture analysis (blue curve)

Since scans in PolyU dataset have high quality, many methods are able to provide good performance and especially when using the images from the same session. It is, thus, important to evaluate a method in other situations where the data quality is not as good. In the next two sections, we will consider those situations with the CASIA and the IITDelhi datasets involving the scale variations, the direction and posture challenges.

3.3 Comparative evaluation on the CASIA-Palmprint dataset

3.3.1 Dataset description and experimental setting

The CASIA-Palmprint dataset contains 5502 palmprint images captured from 312 subjects. This dataset is taken from Chinese Academy of Sciences of institution who Automation. All palmprint images of this dataset are 8 bits gray–level and JPEG format. For each subject, 8 palmprint images have been collected from both left and right palms. The resolution of original palmprint images is 640 × 480 pixels. The capture system captures the all palmprint images using a CMOS camera fixed on the top of the device. This latter supplies an evenly distributed illumination on the palmprint images. Since the hand of each person is not touching any platform plate, a presence of distance variation between the hand and the capture device which implies scale variations. In this dataset, the users are not used the guidance pegs to constraint their hands (unconstrained peg-free setup). Therefore, these hand images are captured with different directions and postures, which brings on contracting and stretching the principal lines and texture skin pattern at the region of interest of the palmprint. This dataset has many variations which make the identification of the persons more difficult. Figure 10 shows the capture system used and some examples of these palmprint images in the database.

Fig. 10
figure 10

Palmprint Image Scan of ”CASIA-Palmprint” dataset a the capture system b Some examples of the palmprints

This database was also automatically preprocessed as described in the Section 3.1. Furthermore, we randomly selected 5 samples of each person as gallery set and the remaining samples (3 samples) make up the probe set.

3.3.2 Experimental result and comparison

Since the CASIA dataset contains numerous challenging and variation in term of scales, position, direction, etc, the principal lines and Intra modal approaches with this dataset are currently almost absent in literature. Except, Bruno et al. [5] proposed an approach to detect the principal lines and thus achieved 80.00 % recognition rate, while we achieved a much higher result of 92.98 %, as mentioned in Table 6. Hence, we observed that the performance of the principal line extraction has a very high recognition rate compared to the work in [5].

Table 6 Recognition results comparison of the state-of-the-art results on the CASIA-Palmprint database. Protocols Description: Right hand (R), left hand (L), Samples of person (S), Gallery data (G), Probe Data (P)

In order to make the comparison of our texture extraction method fair, we kept the same setting protocols using by Jawsal et al. [27] (only the right hand of the dataset). Our method outperforms their approach with a performance of 96.52 %. Consequently, this result presents an improvement rate of about 5.76 %. We conducted tests of the texture extraction by following the same testing protocol proposed by the evaluation described in [21], we have achieved comparable results than in this work. It is worth noting that there is no prior result in the literature on palmprint identification by feature fusion (Intra-modal approaches) using this dataset, while, we have a significant recognition rate of about 98.00 %, which is the best result on this dataset.

3.4 Comparative evaluation on the IIT-Delhi-Palmprint-V1 dataset

3.4.1 Dataset description and experimental setting

The IITD-Palmprint dataset contains 2300 palmprints images captured from 230 individuals aged from 14 to 56 years. All images of this dataset have been acquired by using a digital CMOS camera and saved in bitmap format. For each individual, 5 palmprint images are acquired in varying hand pose variations and have been collected from both left and right palms. So, 1150 left palmprints and 1150 right palmprints. All the images are collected in the indoor environment and employ circular fluorescent illumination around the camera lens. Each person is provided to present his/her hand in the imaging region with different positions and directions which deal the scale variations.

In addition, the automatically cropped and segmented palmprint images (region of interest-ROI) are also available in this dataset with size 150 × 150 pixels. Moreover, our experiment setting for this dataset considered by the randomly selection of 3 samples of each person for the gallery data and the remaining samples are used as probe data (2 samples).

3.4.2 Experimental results and comparison

Tables 7 and 8 compare the results of our method with the previously published results following the principal lines and texture extraction, respectively. The setting protocols of the number of users considered represent all the users of dataset (230 users) in the several experiments works, except in [32] where the number of users is restricted to 100 users.

Table 7 Recognition results of the extracted principal lines comparison of the state-of-the-art results on the IIT-Delhi-Palmprint-V1 database. Protocols Description: Samples of person (S), Users in dataset (U), Gallery data (G), Probe Data (P)
Table 8 Recognition results of the extracted texture comparison of the state-of-the-art results on the IIT-Delhi-Palmprint-V1 database. Protocols Description: Samples of person (S), Users in dataset (U), Gallery data (G), Probe Data (P)

As noted, our approach provides the highest recognition rate for palmprint with principal line shape extraction (92.82 %) and texture pattern extraction (95.80 %).

We observed in Table 9 that the performance of our intra-modal approach is very high with a recognition rate of about 97.98 % compared to the work of Luo et al. [35] (92.00 %) which follows the same protocol as ours. Consequently, this result presents an improvement rate of about 5.98 %. We have achieved comparable results with Kumar et al. [32]. These researchers achieved a improvement of recognition rate about 0.77 % than us, they considered only 100 users while we achieved our results using all the subjects on database.

Table 9 Recognition results of the extracted Intra-Modal features comparison of the state-of-the-art results on the IIT-Delhi-Palmprint-V1 database. Protocols Description: Samples of person (S), Users in dataset (U), Gallery data (G), Probe Data (P)

3.5 Discussion

In order to study the performance of the proposed approach in the presence of different challenges, we have presented experimental results using three well-known palmprint datasets. The results of our proposed intra-modal approach which are achieved in these two experiments are shown in Table 10. It is clear that these results show that the first experiment reaches 98.32 % which is a superior compared to the second experiment. This is due to the protocols of the second experiment which are a mixture of images between two sessions. This is important in a real application thanks to the several changes and variability in terms of lighting conditions (illumination), which vary from one session to another. The objective of this experiment is to verify the robustness of our proposed method. This second experience recorded an identification rate of about 96.99 %. Another conclusion that can be made from Table 10 is that the fractal features achieve a better performance compared to the principal line features. The performance improvement is due to the combination of two palmprint representations can be observed in row three of Table 10.

Table 10 The different Recognition Rates of our experiments for the three datasets and the different experiments

4 Conclusion

In this paper, we have presented a framework for intra-modal palmprint identification. The texture and the principal lines are investigated separately. An elastic shape analysis framework is used in order to analyze the shape of the principal lines. This framework has the advantage to provide re-parameterization invariant distance between the principal lines and also to compute some statistics like the intrinsic means which are used to build the feature vectors for principal line classification. Then, the texture information is investigated using the fractal analysis. The fusion of the resulting features makes an efficient method for palmprint recognition with different challenges. This is validated by comparative studies on three challenging datasets; the proposed approach performs competitive results using the principal lines, the texture and the intra-modal fusion. As future work, we plan to integrate in our approach the feature selection techniques which can be used to reduce the dimensionality of the features while discarding the non discriminative features and thus increase the recognition accuracy. Addaboost for instance is adapted to form a strong classifier based on several weak classifiers (the feature vector elements). Additionally, we would encourage future works to extend our approach for investigating more assessment of our proposed system performance. Therefore, we plan to use the other biometric metric such as the Equal Error Rate (EER) which is an important metric in the performance estimation of a palmprint verification method.