Palm vein recognition based on a modified $\text{(2D)}^{2}\text{ LDA}$

Abstract

Biometric computing offers an effective approach to personal identification based on unique, stable physical or behavioral characteristics. A reliable and robust personal verification approach using palm vein patterns is presented in this paper. This approach has lower computational and memory requirements and a higher recognition accuracy than similar methods. In my work, a near-infrared charge-coupled device camera is adopted as an input device for capturing palm vein images, since it can provide low-cost, non-contact imaging. In the proposed approach, two finger-webs are automatically selected to define the region of interest (ROI) in the palm vein images. Modified two-directional two-dimensional linear discriminant analysis ($\text{(2D)}^{2}\text{ LDA}$), which performs an alternate two-dimensional LDA (2DLDA) in the column direction of images in the 2DLDA subspace, is proposed to exploit the correlation between rows and columns of palm vein features inside the ROI. The major advantage of the method is that it requires fewer coefficients for efficient palm vein image representation and recognition. A total of 4,140 palm vein images were collected from 207 persons to verify the validity of the proposed palm vein recognition approach. High accuracies ($>$99 %) have been obtained by the proposed method, and the speed of the method (response time $<$0.75 s) is rapid enough for real-time recognition. Experimental results demonstrate that the proposed approach is feasible and effective for palm vein recognition.

1 Introduction

Wherever security is required, whether to restrict access to buildings or to computer systems, it is necessary to employ an access control system. Traditional personal verification methods rely heavily on the use of passwords, personal identification numbers (PINs), magnetic swipe cards, keys, smart cards, etc. Each of these offers only limited security.

Fig. 1

The main steps of the proposed palm vein recognition system

Many biometric recognition systems have been proposed, based on various human physiological features or behaviors including facial images, hand geometry, fingerprints, palm prints, retinal images, handwritten signature and gait [1], to improve the security of personal verification. Each of these systems has merits and demerits. In the case of fingerprints, direct contact of the finger with the fingerprint-image-extracting sensor causes degradation in performance, especially in factories or construction sites where good quality fingerprints are hard to obtain due to oil from the finger, moisture, dirt, etc. Using a retinal scanner, users must place their eye close to the scanner, causing an uncomfortable feeling of invasion of privacy. With hand-shape recognition devices, problems may arise with users who suffer from arthritis or rheumatism, leading to poor performance. The palm vein recognition system has been developed to resolve these problems. Since it acquires a palm vein pattern image without direct contact with the palm or with the vein pattern-extracting sensor, there is no contamination. Using the palm vein image, both user comfort and biometric recognition are improved, producing a stable and reliable system.

Palm vein recognition appeared in the 1990s [2] and has been popular since 2000 because of the advantages it offers. A vein pattern reveals the vast network of blood vessels underneath a person’s skin. Like fingerprints, though it has never been proven in a strict scientific sense, the shapes of the vascular patterns of different individuals are believed to be distinctive [1–3] and stable over a long period of time. In addition, as the blood vessels are hidden beneath the skin and are mostly invisible to the human eye, vein patterns are much harder for intruders to copy compared to other biometric features. The properties of uniqueness, stability and strong immunity to forgery of the vein pattern make it a potentially good biometric which offers secure and reliable features for personal identity verification [4]. There is some evidence that the pattern may not be totally unique. Based on an investigation done by the Fujitsu Company [5], the system had a false rejection rate (FRR) of 1 % and a false acceptance rate (FAR) of 0.5 %. Nevertheless, it is a feasible biometric feature for a small population.

In this paper, an eigenspace-based palm vein recognition method is described in detail. Experimental results on my own near-infrared (NIR) palm vein image database, including more than 200 individuals, have demonstrated that the proposed method is feasible and effective for personal identification. The main steps in the proposed approach are shown in Fig. 1. The solid boxes show the processed data at different stages, and the dashed boxes denote the four different processing steps. First, palm vein images are captured by an NIR camera as the input data. Then, a median filter is employed on the palm vein images to remove some noise, and Otsu’s method is applied to select a suitable threshold to segment the hand region. Two baseline data points are found automatically, and a square region of interest (ROI) is obtained. Next, modified two-directional two-dimensional linear discriminant analysis $\text{((2D)}^{2}\text{ LDA}$) is employed on the vein pattern in the ROI to extract important discriminant features. Finally, the minimum distance classifier (MDC) is adopted to verify whether the template and testing samples are captured from the same person.

After analyzing related work in Sect. 2, I will briefly introduce the palm vein image collection device and process in Sect. 3. A detailed description of the proposed method for palm vein recognition is given in Sect. 4. Experimental results are presented and discussed in Sect. 5, prior to conclusions in Sect. 6.

2 Related work

There are many different features in palm vein images, such as the geometry, the principal line and the delta point. Furthermore, hand veins show significant textural differences and many minutiae, similar to the ridges and branches of fingerprints. [6–8]. These features offer stable, unique and reliable biometrics for personal identification [4]. Many verification technologies using biometric features of hand veins have been developed over the past decade [6–18]. Lin et al. [9] present personal verification results using palm dorsal images acquired from a thermal infrared (IR) camera operating in the 3.4–5.0 $\upmu \mathrm{m}$ wavelength range. Their approach is based on the combination of multi-resolution images obtained from pre-processed thermal vein images. Wang and Leedham [10, 11] present another approach for personal authentication using hand vein images acquired from thermal imaging. They employed the Hausdorff distance to generate matching scores between the extracted line patterns and illustrated promising results. In another paper, Wang et al. [8] compared shape- and texture-based methods for vein recognition, with shape similarity measured via Hausdorff distance and Line Edge Map (LEM) and texture similarity measured via Euclidean distance of Gabor magnitude features. In a dataset of 100 persons, Hausdorff-, LEM- and Gabor-based methods achieved accuracies of 58, 66, 80 %, respectively. Wang et al. [12] proposed a multimodal personal identification system where palmprint and palm vein modalities were combined in a single image. Locality Preserving Projection (LPP) was used to extract features of the fused images, which they called “Laplacianpalm.” Kumar et al. [13] presented a new approach to authenticate individuals using triangulation of hand vein images and simultaneous extraction of knuckle shape information. Wang et al. [14] proposed a hand-dorsal vein recognition method based on Partition Local Binary Pattern (PLBP) and assessed it using a similarity measure obtained by calculating the Chi square statistic between the feature vectors of the tested sample and the target sample. Crisan et al. [15] focused on improving the two essential parts of a vein-scanning device: the hardware lighting system and the feature extraction algorithms. Ramalho et al. [16] proposed a secure multimodal biometric recognition system with a multi-level fusion architecture. They used four biometric traits (i.e., palmprint, finger surface, hand geometry and palm veins) and three different feature extraction techniques (i.e., Orthogonal Line Ordinal Features, Competitive Code and PalmCode) for biometric identification. Sanchit et al. [17] presented a multimodal system that combines hand-palm vein and hand-dorsal vein biometrics information at the score level. Khan et al. [18] successfully used principal component analysis (PCA) to obtain eigenveins, which is a low-dimensional representation of vein pattern features. Fujitsu’s [5] palm vein verification product has high accuracy (1 % FRR and 0.5 % FAR), but to the best of my knowledge the features used have not been disclosed in any published research articles.

Although these authors claim the features they use to recognize the vein patterns can attain a high level of accuracy, most of them used small datasets to evaluate the performance of hand vein recognition, so the practical feasibility of these algorithms has not been demonstrated. To demonstrate the robustness of the proposed approach, I constructed my own near-infrared (NIR) palm vein image database, comprising 207 participants and 4,140 palm vein pattern images. It is based on a non-contact, non-invasive data acquisition method and requires no injection of any agents into the blood vessels. This is by far the best-known non-invasive option for acquiring palm vein pattern images.

Currently, palm vein recognition methods can be classified into two types, geometric approaches and holistic approaches. Geometric approaches extract local features such as the locations and local statistics of the principal veins, minutiae points, and ridge bifurcations. However, geometric features are difficult to extract, represent and compare, while the discriminability of geometric features such as texture energy is not strong enough for palm vein recognition. To overcome these problems, another set of methods called holistic approaches, which extract data from the palm vein image for identity recognition, are compared in this paper. Holistic methods, which represent intrinsic attributes of an image, can be extracted based on various algebraic transforms and matrix decompositions.

PCA [19, 20] and linear discriminant analysis (LDA) [21, 22] are two classical linear feature extraction and data representation techniques widely used in the areas of pattern recognition and computer vision. In the two holistic methods, the two-dimensional image matrices must first be transformed into one-dimensional image vectors column by column or row by row. However, concatenating two-dimensional matrices into one-dimensional vectors often leads to a high-dimensional vector space, making it difficult to evaluate the covariance matrix accurately due to its large size and the relatively small number of training samples. Furthermore, computing the eigenvectors of a large covariance matrix is very time consuming. To overcome those problems, in recent years, two-dimensional feature extraction methods, such as two-dimensional PCA (2DPCA) [23] and two-dimensional LDA (2DLDA) [24, 25], have been widely used. Work on these has mainly focused on constructing the image covariance matrix directly using the original image matrices. In [26], the efficiency of 2DPCA and 2DLDA has been demonstrated for many applications such as pattern recognition. In general, there has been a tendency to prefer 2DLDA over 2DPCA because, as intuition would suggest, the former deals directly with discrimination between classes, whereas the latter deals with the data in its entirety for principal components analysis without paying particular attention to the underlying class structure. However, it is observed that working either on a 2DLDA [25]-based approach along the row direction of images or on the alternative 2DLDA [27]-based approach along the column direction of images would still require many coefficients for image representation. To alleviate this problem, the concept of two-directional two-dimensional linear discriminant analysis $\text{((2D)}^{2}\text{ LDA}$) [27] has been proposed specifically for face recognition problems. Two-directional 2DLDA was developed by simultaneously considering the row and column directions of the original images.

In this paper, a feature extraction method of palm vein recognition based on a holistic approach is proposed. Motivated by the work of Noushath et al. [27], I present an analogous model, called modified $\text{(2D)}^{2}\text{ LDA}$, to improve the performance of the $\text{(2D)}^{2}\text{ LDA}$ method for palm vein recognition. The modified $\text{(2D)}^{2}\text{ LDA}$ method has the threefold advantage of higher recognition rate, lower memory requirements and better computing performance than the standard PCA, LDA, 2D-PCA, 2D-LDA, $\text{(2D)}^{2}\text{ PCA}$ [27], or $\text{(2D)}^{2}\text{ LDA}$ methods, as confirmed through extensive experiments conducted on the palm vein database. The overall system is reliable and robust, and the image quality is more consistent and uniform.

Fig. 2

Acquisition of palm vein images with a non-contact system

Fig. 3

Original gray-level images of palm veins captured from different persons by a near-infrared CCD camera

3 Palm vein image collection

In visible light, the vein structure of the palm is not always easily discernible. The visibility of the vein structure varies significantly depending on such factors as age, level of subcutaneous fat, ambient temperature and humidity, physical activity and hand position. In order to perform a preliminary analysis on the features of the palm vein pattern, a new NIR palm vein image database was constructed. A charge-coupled device (CCD), a low-cost camera traditionally used for surveillance (Sony XC711, costing about U.S. $1,500), was employed as the non-contact image collection device to acquire the vein images. The CCD is capable of detecting near-infrared radiation up to a wavelength of approximately 1mm, but cost cameras have a filter in front of the sensor since the main purpose of the camera is to see the maximum amount of visible radiation. This filter must be removed in order to gain access to the infrared part of the radiation spectrum.

Fig. 4

Location of ROI defined inside a palm vein image: a Original palm vein image, b palm region segmented by Otsu’s method, c palm border (white pixels) extracted by the inner border tracing algorithm, d two data points $P_1$ and $P_2$ are selected and the palm image is rotated an angle $\theta$ between line $\overline{P_1 P_2 }$ and the horizontal line, e based on $\overline{P_1 P_2 }$, a square region is located and denoted as the ROI, f distance distribution diagram of the palm border

Though principally designed for use in visible light, CCD cameras are also sensitive to NIR wavelengths of the electromagnetic spectrum up to about 1,100 nm. This is within the actinic IR range, which covers the NIR spectrum from 700 to 1,400 nm. In my work, LEDs producing near IR illumination are located around the camera, producing a peak at 750 nm. This wavelength lies in the medical spectral window (700–900 nm) in which illumination penetrates deeper into biological tissues [29]. The region of interest (ROI) is irradiated by the LED infrared light sources. Since the LEDs are focused on the ROI, a nearly black image surrounds the palm image. It is quite obvious that there are no backgrounds with the back of the palm facing the camera. In addition to the LEDs, another important part of this system is an infrared filter. To eliminate the effect of visible light, an optical infrared filter was mounted in front of the camera’s lens. For data collection in this work, a Hoya RM80 IR filter was used. Since there are no docking devices to constrain the participant’s hand, either the right or left hand can be imaged. During the collection of images for the database, I restricted the position of participants’ hands above the camera. A distance between the hand and the camera of approximately 20 cm gave the best acquisition. At this distance, there is acceptable tolerance for positioning the palm within the specified region. The acquisition of a typical non-contact palm vein image is shown in Fig. 2.

Using the NIR light source and the IR filter, the CCD sensor produces a high-quality image of the palm (each image has a resolution of $320\times 240$ in 8-bit grayscale per pixel.). When seen on the computer monitor, the most distinguishable component in the image is the superficial vein tree pattern. The color of the skin is immaterial: both a light skinned Caucasian and a dark skinned African have vein patterns that are clearly visually distinguishable in the images. Figure 3 shows several sample images from this database.

4 Proposed method for palm vein recognition

In this paper, a palm vein recognition algorithm is presented, which includes four basic processes: (i) extraction of the ROI; (ii) image enhancement; (iii) extraction of the palm vein features; and (iv) matching. Detailed descriptions of these four steps are introduced in the following sections.

4.1 Extraction of the ROI

Image segmentation is one of the most important steps in the analysis of processed image data. Its main goal is to separate the background from the components of the image that have a strong relevance to the analysis. When a palm image is obtained, the palm background is first segmented from the image. Binarization is used to segment the image into two levels: object (hand region) and background. For palm image segmentation, Otsu’s thresholding is applied to the palm image to estimate the palm region. Comparing the segmented palm shapes with the original palm images, the two are almost indistinguishable, as shown in Fig. 4a, b. This shows that Otsu’s method is quite effective in determining the threshold of palm images.

To increase verification accuracy and reliability, the principal palm vein patterns extracted from the same region in different palm vein images are compared. The region to be extracted is known as the region of interest (ROI). For this reason, it is important to fix the ROI in the same position in different palm vein images to ensure the stability of the principal extracted vein features. However, it is difficult to fix the ROI at the same position in different palm vein images without the use of a docking device to constrain the palm position when acquiring palm vein images. The ROI for palm vein recognition purposes is usually a square region in the central part of the palm. Many algorithms [30–32] discuss palm localization. The method of Sanches et al. [30] was adopted to locate the ROI in this paper. Two data points were employed ($P_1$ and $P_2$, as shown in Fig. 4d), to determine the approximate (not absolute) immovable ROI. The following processes determine the two data points in binary palm vein images. First, the inner border tracing algorithm [33] is employed to find the palm border. Figure 4c shows an example illustrating how the extracted border of a palm image perfectly matches the original palm contour. Then for each point on the palm contour, the distance between this point and the mid-point of the wrist is calculated. Figure 4f shows the distance distribution diagram. As can be seen, there are five local maxima and four local minima. The pattern in the diagram is quite similar to the geometric shape of a palm (see Fig. 4a), which also has five tips (local maxima) and four finger-webs (local minima). Experimenting on a wide variety of palm vein images, I found that the four local minima locations in the distance distribution diagram are the same as finger-web locations and the match between the two locations is very close. Finally, two valley points, $P_1$ (the valley point between the small finger and ring finger) and $P_2$ (the valley point between the middle finger and the index finger), are selected as the two key data points, as shown in Fig. 4d. These two data points ($P_1$ and $P_2 )$ are then employed to locate the ROI. The procedure is described as follows.

Fig. 5

Demonstration of the effects of the proposed scheme to automatically define the ROI. Each set of images, a1–a4, b1–b4, c1–c4 and d1–d4, is extracted from different palm vein images captured from one person and covers nearly the same region

First, the straight line $\overline{P_1 P_2 }$ is formed by the points of $P_1$ and $P_2$ as shown in Fig. 4d. To eliminate the influences of palm rotation and define the coordinates of ROI more conveniently, the palm image is rotated by the angle $\theta$ (Eq. 1) between line $\overline{P_1 P_2 }$ and the horizontal line.

$$\begin{aligned} \theta =\tan ^{-1}(Y_{P_2} -Y_{P_1} )/(X_{P_2}-X_{P_1} ), \end{aligned}$$

(1)

where ($X_{P_1 } ,Y_{P_1 } )$ is the coordinate of $P_1$ and ($X_{P_2 } ,Y_{P_2 } )$ is the coordinate of $P_{2}$.

This makes the direction of line $\overline{P_1 P_2 }$ horizontal. Next, $P_C$ is defined as the middle point between $P_1$ and $P_2$ on line $\overline{P_1 P_2 }$, as shown in Fig. 4e. A square region $R_{C_1 C_2 C_3 C_4 }$ whose corners are $C_1 ,\,C_2 ,\,C_3$ and $C_4$ is located and denoted as the ROI. The upper side of $R_{C_1 C_2 C_3 C_4 } ,\,\overline{C_1 C_2 }$ is of the same length and parallel to line $\overline{P_1 P_2 }$. $R_{C_1 C_2 C_3 C_4 }$ lies directly beneath $\overline{P_1 P_2 }$ at a distance one sixth of the length of $\overline{P_1 P_2 }$ below $\overline{P_1 P_2 }$, as shown in Fig. 4e. Finally, the upper-left point $C_1$ of the ROI is defined as the origin coordinate (0,0). Figure 5 depicts the original palm images and different ROIs captured from the same person. The results show that the ROIs are nearly identical. Using this procedure, a docking device is not necessary for acquiring the palm vein images, and high verification accuracy can simultaneously be maintained.

Fig. 6

Intermediate results of feature enhancement: a Original ROI image, b estimated local average intensity, c result after eliminating the local average intensity

4.2 Image enhancement

Figure 6 depicts sample image enhancement results from the palm vein database. After extraction of the ROI, as shown in Fig. 6a, the problems of low contrast and non-uniform illumination in palm vein images need to be mitigated. The method proposed by Lee et al. [34] to eliminate background brightness is used in the current paper. First, the palm vein image is divided into non-overlapping $16\times 16$ blocks. The mean grayscale value constitutes a coarse estimate of the background illumination for individual blocks. By using bicubic interpolation, each estimated mean value is expanded to the whole region of its own $16\times 16$ block. An example of the estimated background illumination is displayed in Fig. 6b. Each template block can then be converted to a uniform light condition by subtracting the background illumination from each original template, as shown in Fig. 6c.

4.3 Extraction of palm vein features based on modified $\text{(2D)}^{2}\text{ LDA}$

In this section, a straightforward image projection technique for feature extraction, called modified $\text{(2D)}^{2}\text{ LDA}$, is presented. Standard $\text{(2D)}^{2}\text{ LDA}$ methods [25] work in the row and column direction of the original images. I propose an alternate 2DLDA which works in the column direction of images in the low-dimensional 2DLDA subspace. The modified $\text{(2D)}^{2}\text{ LDA}$ method is defined in Sect. 4.3.2, after an overview of the standard approach.

4.3.1 Overview of the $\text{(2D)}^{2}\text{ LDA}$ approach

As in LDA, the goal of 2DLDA is to extract features that can best approximate the data. 2DLDA [25] works in the row direction reflecting the information between rows of images to learn an optimal matrix $X$ from a set of training images, and then projects an $m\times n$ image $A$ onto $X$, yielding an $m$ by $d$ matrix $Y=AX$. Suppose that there are $N$ training image samples in total, and the $j$th training image is denoted by an $m\times n$ matrix $A_j$ ($j=1,2,\ldots ,N)$, which contains $C$ classes, and the $i$th class $C_i$ has $n_i$ samples $(\sum _{i=1}^C {n_i =N} )$. 2DLDA attempts to seek a set of optimal discriminating vectors to create a transform $X_d =\left\{ {x_1 ,x_2 ,\ldots ,x_d } \right\}$ by maximizing the two-dimensional Fisher criterion [24], denoted as

$$\begin{aligned} J(X)=\frac{X^{T}G_\mathrm{b} X}{X^{T}G_\mathrm{w} X} \end{aligned}$$

(2)

In Eq. (2), $T$ denotes a matrix transposition, while $G_\mathrm{b}$ and $G_\mathrm{w}$ are between-class and within-class scatter matrices, respectively:

$$\begin{aligned} G_\mathrm{b}&= \frac{1}{N}\sum _{i=1}^C {n_i (\overline{A_i } -\overline{A} )^{T}(\overline{A_i } -\overline{A} )} ,\end{aligned}$$

(3)

$$\begin{aligned} G_\mathrm{w}&= \frac{1}{N}\sum _{i=1}^C {\sum _{j\in C_i } {(A_j -\overline{A_i } )^{T}(A_j -\overline{A_i } )} } , \end{aligned}$$

(4)

$\overline{A_i }$ and $\overline{A}$ denote the means of the $i$th class and the whole training set, respectively. $A_j$ is the $j$th image in the class $C_i$. The goal of the 2DLDA scheme is to find the optimal discriminating vectors $X_\mathrm{opt}$ in order to maximize $J(X)$. Obviously, the optimal discrimination vectors $X_\mathrm{opt}$ are the eigenvector corresponding to the dominant eigenvalues of eigenstructure $G_\mathrm{w}^{-1} G_\mathrm{b}$. It has been proved that the optimal value for the discriminating vectors $X_\mathrm{opt}$ is composed of the orthonormal eigenvectors $x_1 ,x_2 ,\ldots ,x_d$ of $G_\mathrm{w}^{-1} G_\mathrm{b}$ corresponding to the $d$ largest eigenvalues. Given an image $A_{m\times n}$, all the projections of the image matrix in the $d$-directions make up an $m\times d$ dimensional vector, which is the 2DLDA feature vector.

Similarly, the alternative 2DLDA [27] learns the optimal projection matrix $Z$ reflecting information between columns of images, and then projects $A$ onto $Z$, yielding a $q$ by $n$ matrix $B=Z^{T}A$. The optimal projection in the column direction $Z_\mathrm{opt} =(z_1 ,z_2 ,\ldots ,z_q )$ is obtained by the transposed space matrix. The (2D)$^{2}$LDA treats the feature matrix $A_j$ both in row and column directions by projecting the $m$ by $n$ image $A_j$ onto $Z_\mathrm{opt}$ and $X_\mathrm{opt}$ simultaneously, yielding a $q$ by $d$ matrix $S$:

$$\begin{aligned} S_j =Z_\mathrm{opt}^T A_j X_\mathrm{opt}. \end{aligned}$$

(5)

In this scheme, the resulting feature matrix $S_j$ is used to represent image $A_j$ for classification.

Fig. 7

Comparison of the feature extraction process: a $\text{(2D)}^{2}\text{ LDA}$, b modified $\text{(2D)}^{2}\text{ LDA}$

4.3.2 Proposed modified $(2D)^{2}\text{ LDA}$

$\text{(2D)}^{2}\text{ LDA}$ works simultaneously in the row and column directions of the original images. Although it is efficient for image representation and recognition, it is possible to improve the recognition rate and reduce computation time by modifying $\text{(2D)}^{2}\text{ LDA}$. In $\text{(2D)}^{2}\text{ LDA}$, the optimal projection matrix Z (described in Sect. 4.3.1) works in the column direction, capturing information between columns of images. In contrast, modified $\text{(2D)}^{2}\text{ LDA}$, which works in the column direction of images in the low-dimensional 2DLDA subspace, can reduce the number of coefficients in the feature matrix. The idea of modified $\text{(2D)}^{2}\text{ LDA}$ is described as follows: (1) The image matrix performs 2DLDA within rows (as described in Sect. 4.3.1) and reflects the information between row of images to find the optimal image matrices $Y=AX$. (2) Then image matrix $Y$ performs 2DLDA in the column direction in the 2DLDA subspace. Ultimately, the discriminant information of the whole image is packed into the upper-left corner of the image matrix. This method, which has been used efficiently in palm vein recognition, is described below:

Given a training set $\left\{ {A_1 ,A_2 ,\ldots ,A_M } \right\} ,\,M$ is the number of the training images and the size of each image matrix is $m\times n$. The $i$th training image is denoted by matrix $A_i$ ($i=1,2,\ldots ,M)$, and the average image of all training samples is denoted by $\overline{A}$.

• (1) After the first 2DLDA transform in the rows direction, the feature matrix Y of sample A is obtained using $Y=AX$ (as discussed in Sect. 4.3.1).

• (2) Constructing the image between-class and within-class scatter matrices $H_\mathrm{b}$ and $H_\mathrm{w}$ based on $Y^{T}$ yields

  $$\begin{aligned} H_\mathrm{b}&= \frac{1}{M}\sum _{i=1}^C {M_i (\overline{Y} _i -\overline{Y} )(\overline{Y} _i -\overline{Y} )^{T}}\end{aligned}$$

  (6)
  $$\begin{aligned} H_\mathrm{w}&= \frac{1}{M}\sum _{i=1}^C {\sum _{j=1}^M {(Y_j^i -\overline{Y} _i )(Y_j^i -\overline{Y} _i )^{T}} } \end{aligned}$$

  (7)

  where $Y_j^i =A_j^i X, \overline{Y} _i =\overline{A} _i^ X,$ and $\overline{Y} =\overline{A} X$. It can be observed that $H_\mathrm{b}$ and $H_\mathrm{w}$ in Eqs. (6) and (7) are obtained in this new formulation as the outer products of column vectors, unlike $G_\mathrm{b}$ and $G_\mathrm{w}$ (Eqs. 3, 4) in the case of the original 2DLDA.

• (3) We next find the optimal projection matrix $V_\mathrm{opt}$ so that the total scatter of the projected samples of the training images is maximized. For this purpose, Fisher’s criterion [24] is employed, given by

  $$\begin{aligned} J(V)=\frac{VH_\mathrm{b} V^{T}}{VH_\mathrm{w} V^{T}} \end{aligned}$$

  (8)

  Similarly, the eigenvectors of $H_\mathrm{w}^{-1} H_\mathrm{b}$ are computed and then $q$ eigenvectors $v_1 ,v_2 ,\ldots ,v_q$ corresponding to the first $q$ largest eigenvalues of $H_\mathrm{w}^{-1} H_\mathrm{b}$ are chosen.

• (4) Projecting $Y^{T}$ onto V, yields $C^{T}=Y^{T}V_\mathrm{opt}$. The modified $\text{(2D)}^{2}\text{ LDA}$ feature matrix of Y is given by

  $$\begin{aligned} C_i =V_\mathrm{opt}^T Y_i =V_\mathrm{opt}^T A_i X_\mathrm{opt}. \end{aligned}$$

  (9)

  The resulting feature matrix $C_i$ is a $q\times d$ matrix, which is smaller than the $\text{(2D)}^{2}\text{ LDA}$ feature matrix $S_j$ and the original image A. A comparison of $\text{(2D)}^{2}\text{ LDA}$ with modified $\text{(2D)}^{2}\text{ LDA}$ is illustrated in Fig. 7.

4.4 Matching

After feature extraction, it is essential to choose an appropriate method of similarity measurement for the matching of feature matrices. The aim of palm vein recognition is to match the unknown vein image with those known vein classes in the database0 and determine whether the unknown individual is authentic or an impostor. In my experiment, ten images of each palm vein class are chosen randomly to constitute the training samples, and the remaining images of each class are used as testing samples. The minimum distance classifier (MDC) is adopted for classification in a low-dimensional feature space.

$$\begin{aligned}&m=\arg \mathop {\min }\limits _{1\le i\le c} d(f,f_i ), \nonumber \\&d(f,f_i )=\sum _j {\left(f^{j}-f_i^j\right)^{2}} \end{aligned}$$

(10)

where $f$ and $f_i$ are the feature matrix of an unknown sample and the $i\text{ th}$ class, respectively. $f^{j}$ and $f_i ^{j}$ are the $j\text{ th}$ component of the feature matrix of the unknown sample and that of the $i\text{ th}$ class, respectively. $c$ is the total number of classes, and $d(f,f_i )$ denotes the Euclidean distance measure. The feature matrix $f$ is classified into the $m\text{ th}$ class by searching for the lowest distance value using the similarity measure $d(f,f_i )$.

5 Experimental results

To evaluate the effectiveness of the proposed method for palm vein recognition, experiments were conducted in two modes: identification (one-to-many matching) and verification (one-to-one matching). In identification mode, if the test sample and the identified template are from the same class, this is a correct recognition. Therefore, in identification mode, the algorithm can be measured by the correct recognition rate (CRR), the ratio of the number of samples being correctly classified to the total number of test samples. In verification mode, assuming that a test sample is from a specified subject, a one-to-one comparison is made to verify whether the test sample is from the specified subject. Such comparisons result in two independent error rates, false match rate (FMR) and false non-match rate (FNMR). The FMR (sometimes called false positive rate) is the probability that a test sample of an impostor is falsely declared to match the template of an authorized subject and the FNMR (sometimes called false negative rate) is the probability that a test sample of an authorized subject is falsely declared not to match his template. By adjusting the matching threshold, a receiver operating characteristic (ROC) curve can be created. The ROC curve is a plot of genuine match rate (1-FNMR) against false match rate for all possible matching thresholds and shows the overall performance of an algorithm. The ideal ROC curve is a step function at the zero false match rates. The equal error rate (EER) is the point where the FMR and the FNMR are equal in value. The smaller the EER, the better the algorithm. In these experiments, the measures described above are used for performance evaluation. The following subsections detail the experiments and results.

5.1 Palm vein database

A near-infrared CCD camera was used to acquire palm vein images. It has the benefits of easy availability, uniformity, low-cost and consistently high image quality. In this work, the camera used to acquire palm vein images was a digital noise reduction (DNR) DSP Camera, shown in Fig. 2. The captured palm vein images are 8-bit gray images with a resolution of $320\times 240$. The reason for using such a low spatial resolution is that both the data amount in palm vein images and noise sensitivity can be reduced while the features of palm veins can still be preserved. To obtain high verification accuracy, it is important to construct an objective verification template library. Therefore, 207 volunteers were enrolled in my study, and palm vein images were captured from their right hands at two different times after an interval of at least 1 month. Ten palm vein images were acquired each time for each person, so this database includes 4,140 palm vein images. A database constructed in this way includes possible variations of palm vein images under various conditions. The database includes several racial groups, including Chinese, African and Caucasian. The age range of the volunteers was between 18 and 60 years, and their occupations ranged from university students, professors and technicians to manual workers such as cleaners and electricians. No distinction was made between male and female when the samples were collected.

So far, only small datasets have been used to evaluate recognition performance for palm vein recognition for most methods. To demonstrate the robustness of the proposed approach, the current work includes more than 200 subjects. In the future, I hope to make the database available for free download for biometric verification research. The experiments conducted below were run on a 3.2 GHz PC with 2 GB RAM using Matlab 7.0.

Fig. 8

Different resolutions of the palm vein image

5.2 Performance evaluation of the proposed method

In order to evaluate the recognition accuracy, a large number of images from the palm vein database were collected (as described in Sect. 5.1). In this section, two experiments were conducted to demonstrate the performance of the proposed method for palm vein recognition. The first analyzes the relationship between recognition accuracy and different resolution ROI images. The second investigates the number of modified $\text{(2D)}^{2}\text{ LDA}$ eigenvectors selected for the proposed scheme and further evaluates the recognition performance of modified $\text{(2D)}^{2}\text{ LDA}$.

Fig. 9

FMR and FNMR distributions at each level of resolution: a $1\mathrm{st}$ level ($128\times 128)$, b $2\mathrm{nd}$ level ($64\times 64)$, c $3\mathrm{rd}$ level ($32\times 32)$, d $4\mathrm{th}$ level ($16\times 16)$

To investigate the relationship between the recognition accuracy and the resolution of palm vein images, each ROI image is decomposed into different levels of resolution ($128\times 128,\,64\times 64,\,32\times 32$ and $16\times 16)$ and the images are tested at each level. Therefore, there are four levels, the image with $128\times 128$ resolution being the 1st level and $16\times 16$ resolution being the 4th level (as shown in Fig. 8). The feature vector of each testing vein image is matched against each stored template at each level. A total of 428,490 ($10\times 207 \times 2\text{07})$ comparisons are performed at each level, of which 2,070 ($10\times 207)$ comparisons are genuine matches. FMR and FNMR for a given decision threshold $t$ are displayed for the genuine and impostor score distributions; FMR is the percentage of non-matching pairs whose matching scores are greater than or equal to $t$, and FNMR is the percentage of matching pairs whose matching scores are less than $t$. The FMR and FNMR distributions at different resolution are plotted in Fig. 9. There are two curves in each figure for each different resolution, one curve being genuine matching and the other impostor matching. When the decision threshold is set as the intersection of genuine and impostor distribution curves, the total error reaches a minimum, and the corresponding threshold and FNMR values at a fixed FMR of 1 % in each level are listed in Table 1. According to Fig. 9, the EERs of the 1st and 2nd level are much lower than those of other levels. In other words, palm vein images with resolutions of $128\times 128$ and $64\times 64$ are more suitable for feature extraction–based palm vein recognition than lower resolutions. Because the difference between the EERs at the 1st and 2nd level is very small ($<$0.02), it is difficult to decide which level is optimal for identity recognition.

Fig. 10

ROC curves for the $1\mathrm{st}$ and $2\mathrm{nd}$ level of resolution

Table 1 FNMR values at a fixed FMR of 1 % for each level

Further analysis of the images at the two levels (1st and 2nd) can be done by considering their ROC curves, which plots FMR against FNMR. Figure 10 plots the ROC curves of the 1st and 2nd level. From this figure, although the curve of the 2nd level is not always better than the 1st level, the CRRs of the 1st and 2nd levels are 99.18 and 99.41 %, respectively. Hence, the ROI images of palm vein at the 2nd level are better than those at the 1st level in the proposed method. Therefore, ROI images at the 2nd level ($64\times 64$ resolution) are optimal for this palm vein recognition system. The experimental results demonstrate that image resolution will affect recognition accuracy. A higher image resolution may be affected by noise and a lower image resolution may lose detailed information of the image.

Table 2 Recognition rate of seven methods with varying numbers of training samples

Table 2 shows the recognition accuracy obtained by different methods for varying numbers of training samples. It reveals that the proposed modified $\text{(2D)}^{2}\text{ LDA}$ method is comparable to PCA, LDA, 2DPCA, 2DLDA, $\text{(2D)}^{2}\text{ PCA}$ and (2D)$^{2}$LDA methods in terms of recognition accuracy. Due to the fact that images in the palm database were obtained under varying conditions, a lower number of training samples did not achieve encouraging results in palm vein recognition. As the number of training samples per class increased, the gain relative to other methods becomes more apparent. Experimental results demonstrate that my method can compete favorably with $\text{(2D)}^{2}\text{ LDA}$.

Table 3 The effect of $d_\mathrm{row}$ and $q_\mathrm{col}$ on the CRR obtained using the modified $\text{(2D)}^2\text{ LDA}$

Table 4 Recognition accuracy, feature vector dimensions, and matching time for seven methods

In the second test, the effect of parameter values on the recognition performance of the modified $\text{(2D)}^{2}\text{ LDA}$ is evaluated. The modified $\text{(2D)}^{2}\text{ LDA}$ introduces two parameters, the number of row eigenvectors $d_\mathrm{row}$ and the number of column eigenvectors $q_\mathrm{col}$. In this experiment, ten images of each class are selected randomly to constitute the training set, and the remaining images of each class are treated as the test set. Table 3 depicts the effect of $d_\mathrm{row}$ and $q_\mathrm{col}$ on the CRR obtained using the modified $\text{(2D)}^{2}\text{ LDA}$. As Table 3 shows, the maximum CRR (99.41 %) is obtained when $d_\mathrm{row} =8$, while the number of column eigenvectors $q_\mathrm{col} >6,\,q_\mathrm{col}$ has little effect on the CRR of the modified $\text{(2D)}^{2}\text{ LDA}$. Although the accuracy for $q_\mathrm{col} =10$ and $d_\mathrm{row} =8$ is only slightly better than that for $q_\mathrm{col} =6$ and $d_\mathrm{row} =8$, the $6\times 8$ feature matrix is faster in terms of matching time. In my experiment, I chose the value that corresponds to the best performance on my database, $q_\mathrm{col} =6$ and $d_\mathrm{row} =8$. Table 4 shows a comparison of several methods on recognition accuracy, corresponding dimensions of the feature vector (for PCA and LDA) or feature matrices (for the other methods) and matching times. It can be seen from Table 4 that the recognition accuracy of proposed modified $\text{(2D)}^{2}\text{ LDA}$ method is higher than other methods despite having a smaller feature matrix. Furthermore, the selected optimal value ($8\times 8)$ in $\text{(2D)}^{2}\text{ LDA}$ is slightly larger than the optimal value ($6\times 8)$ using modified (2D)$^{2}$LDA. Finally, Table 4 shows that modified $\text{(2D)}^{2}\text{ LDA}$ methods consume the least matching time among all the methods. The experimental results show that the proposed palm vein representation is effective and the proposed approach can extract promising features from each palm vein image and accomplish the task of palm vein recognition.

5.3 Comparison and discussion

The experimental results presented in Sect. 5.2 reveal that the proposed method is an effective scheme for feature extraction from palm vein images and achieves a CRR up to 99.41 % for this palm vein database. The approaches of PCA, LDA, 2DPCA, 2DLDA, $\text{(2D)}^{2}\text{ PCA}$ and $\text{(2D)}^{2}\text{ LDA}$ are well-known existing schemes for pattern recognition. Table 4 shows the results, comparing the current method with six other known approaches, using the 207 classes of the database. The number of eigenvectors are repeated 25 times by varying projection vectors $d$ (where $d = 1, 2, 3,{\ldots },$ 20, 25, 30, 35, 40, 45). Since $d$, the number of projection vectors, has a considerable impact on different algorithms, I chose the value that corresponds to the best classification result on the image set. As shown in Table 4 and Fig. 11, the proposed method can meet the demand of high accuracy necessary for very high security environments. Figure 11 displays the ROC curve of the seven methods. Experimental results demonstrate that the proposed method is much better than the four methods of PCA, LDA, 2DPCA and 2DLDA and compares favorably with the other two approaches, $\text{(2D)}^{2}\text{ PCA}$ and $\text{(2D)}^{2}\text{ LDA}$. Although the $\text{(2D)}^{2}\text{ PCA}$ and $\text{(2D)}^{2}\text{ LDA}$ curves are a little better than the modified $\text{(2D)}^{2}\text{ LDA}$ curve locally, it can be seen that modified $\text{(2D)}^{2}\text{ LDA}$ outperforms $\text{(2D)}^{2}\text{ PCA}$ and $\text{(2D)}^{2}\text{ LDA}$ overall. Table 5 shows the clock time for different stages of the proposed method and the total time. The whole process should take only about 0.72 s, which is fast enough for real-time verification. Compared with $\text{(2D)}^{2}\text{ PCA}$ and $\text{(2D)}^{2}\text{ LDA}$, modified $\text{(2D)}^{2}\text{ LDA}$ is faster in terms of the matching time. The reason is that the modified $\text{(2D)}^{2}\text{ LDA}$ method uses $6\times 8$ feature matrices, obtaining the same or better recognition accuracy compared to the $\text{(2D)}^{2}\text{ PCA}$ and $\text{(2D)}^{2}\text{ LDA}$ methods (which uses $8\times 8$ feature matrices). This advantage is important in the case of verification.

Fig. 11

ROC curves of seven eigenspace methods of palm vein recognition

Compared with the other eigenspace techniques surveyed [18–28] for palm vein classification or verification, this approach applies a modified $\text{(2D)}^{2}\text{ LDA}$, which requires fewer coefficients to extract principal vein features. The minimum distance classifier (MDC) is adopted to match the templates and testing samples. Table 4 summarizes the results generated by this approach with other techniques [18–28], with reference to our database. As shown in the Table 4, the new approach compares very favorably with other methods.

Table 5 Clock time for different stages of the proposed method

Table 6 Performance of several well-known methods on our palm vein database

Among existing methods for palm vein recognition, those proposed by Lin et al. [9], Kumar et al. [13] and Sanchit et al. [17] are the best known. Moreover, they characterize local details of the palm vein from different viewpoints, that is, a multi-resolution representation method, a geometric-based approach and a texture-based approach. To further prove the effectiveness of the proposed method, detailed comparisons were conducted between the proposed method and the above three methods on the new palm vein database. Comparisons show that my experimental results for each algorithm are consistent with the published results for each. Table 6 illustrates that the CRR of each algorithm is greater than 90 %, demonstrating the high accuracy of these methods. Figure 12 illustrates the ROC curves on the new palm vein database for the four algorithms compared, namely, multiple multi-resolution filters (MRFs), minutiae feature points, two-dimensional Gabor filter and the proposed method. Based on the results shown in Table  6 and Fig. 12, the proposed method has best performance, followed by the methods described in Sanchit et al. [17], Lin et al. [9] and Kumar et al.[13]. The main reason is that the eigenspace method can effectively analyze global variations of the intensity signals, which can reflect most of the random shape information of the palm vein and thus achieve better performance.

Fig. 12

ROC curves from the palm vein database with different approaches

Failure of verification can occur in some palm vein images. The reason for the failure can be categorized into three main groups by carefully observing these palm vein images.

• (1) In preprocessing, two data points are employed to find the ROI automatically. Although the palm vein images are located in almost the same region, there are still a few instabilities in preprocessing, such as translation or rotation. These may result from either misplacement of hands during palm vein acquisition or imperfect preprocessing. The proposed method is resistant to both translation and rotation, but it is not robust to extreme rotation or translation.

• (2) Sometimes the palm vein images captured by the image collection device will result in noise (such as moles, scars, and pigmentation). Too few principal vein features extracted for verification will result in verification failure as well.

• (3) Since the thickness of the vein will be affected by seasonal change, some principal vein features inside the ROI region may be lost as a result. Moreover, the boundary of the regions could form pseudo principal vein features. These issues might lead to match difficulties, but only in rare and extreme cases.

The conditions mentioned in (1)–(3) above cannot be avoided and lead to difficulties in distinguishing palm vein images using the proposed approach. This problem can be alleviated by utilizing other methods to extract vein features or combination with other biometric verification methods.

6 Conclusions

In this paper, a reliable and robust biometric-based verification approach using palm vein images is proposed. There are two main advantages of the proposed approach. The first is that a non-contact image collection device can be used to acquire palm vein images. The low-cost NIR charge-coupled device (CCD) camera is a complete, compact system that provides uniform and consistent image quality due to consistency of the light source and sensor. The second advantage is that a modified $\text{(2D)}^{2}\text{ LDA}$ method is proposed to extract the principal vein features. The modified $\text{(2D)}^{2}\text{ LDA}$, which works in the column direction of images in the low-dimensional 2DLDA subspace, can increase the recognition rate while reducing the coefficients of the feature matrix. The main advantage of modified $\text{(2D)}^{2}\text{ LDA}$ over $\text{(2D)}^{2}\text{ LDA}$ and $\text{(2D)}^{2}\text{ PCA}$ is that fewer coefficients are needed for palm vein representation and recognition, saving computing time.

We used the method of Sanches et al. [30] to locate the ROI in this paper. Under normal conditions, the ROIs should cover almost the same region in different palm vein images. Within the ROI, obvious principal vein features are extracted by applying the modified $\text{(2D)}^{2}\text{ LDA}$ method. Vein features are then matched with those from the template library by the minimum distance classifier (MDC) to verify the identity of the person. Experimental results demonstrate that the proposed approach can obtain acceptable verification accuracy. Such an approach can be applied in access control systems. The proposed algorithms are also compared with other published algorithms and the performance found to be suitable for real-time applications. Therefore, the proposed system is a novel and efficient method for personal identification using palm vein images.