Keywords

1 Introduction

The term biometrics refers to the measurement of human physical traits and behavioural traits. Several common physical traits used in biometric technologies are fingerprints, palm prints, face, iris while behavioural traits used are voice and gait [1]. Biometric technologies are widely used in various identity authentication and verification applications, namely customs, airports, banks and so on [2].

Among various biometrics developed, the fingerprint recognition system is the most established biometric technology where many applications can be seen utilising this system. Despite being the most common approach these days, a fingerprint recognition system still poses the risk of forgery due to the exposure of fingerprints easily. It also degrades the system’s performance when it comes to obtaining unclear fingerprint images due to sweat or dry condition [3]. In response to the addressed drawbacks of current biometrics, researchers have put considerable attention to personal authentication using finger vein recognition system [4]. This is because finger veins are hidden under the skin and therefore it is almost impossible to be forged. It also provides extremely high security due to its distinctive patterns that no one finger, even twins or our fingers have the same vein pattern. Moreover, the finger vein pattern can only be obtained from a living person. Due to these characteristics of increased security and reliability; finger vein biometric outperforms other biometric identifiers.

Typically, a finger vein recognition system (FVRS) consists of three stages, namely, image acquisition, image pre-processing as well as feature extraction and matching. Of all stages, region of interest (ROI) extraction during image pre-processing is considered one of the most critical elements which have a direct impact on the accuracy of the system [5]. ROI of a finger vein image refers to the region of a finger which is filled with an abundant finger vein pattern network. ROI extraction aims to decide which part of the image is fit for finger vein feature extraction, thereby reserving the useful information in the ROI and removing the useless information in the background [6].

Many researchers had contributed to different ROI extraction methods. Yang and Shi [7] proposed an ROI extraction by using distal interphalangeal joint localisation. Distal interphalangeal joint is the joint near the fingertip and has a higher sum of grey value. They first calculate the sum of pixel values in each row of the image. Then, they estimate the position of the distal phalangeal joint by pinpointing the maximum row-sum. Finally, they extract ROI using a window where the phalangeal joint position is located at 2/3 the height of the window. This method is able to obtain localized ROI by estimating the position of the distal phalangeal joint. However, the calculation of row-sum value is easily affected by illumination. Moreover, it used a fixed window instead of edge detection in extracting ROI, thus it is more prone to extracting unwanted background noise.

Yang et al. [6] also extracted the ROI based on phalangeal joints but used a single sliding window as an improvement to the method [7]. First, they used the Sobel edge operator to detect finger edges. Based on the finger edges, they calculate the skew angle to correct finger orientation. For ROI extraction, Yang et al. [6] used a sliding window with 50 rows to estimate the position of the distal and proximal phalangeal joint, which are two joints of the finger. They slide the sliding window row by row from the top to the bottom of the key area to calculate the sum of grey value, where two highest sums are the two phalangeal joints. Lastly, they crop out ROI with two phalangeal joint positions as the height and internal tangent of the finger edge as the width. Sobel edge operator is sensitive to illumination. Hence this method is not accurate for FV images affected by uneven illumination.

Shahrimie et al. [8] proposed a different ROI extraction method which involves orientation correction and height and width definition. They corrected the orientation of the FV image based on the finger edge obtained by Canny edge operator and connected component labelling. For ROI extraction, they cropped the ROI with 300 pixels of height which was empirically set and with the width that was adaptively defined based on the bottom end of finger geometry. The method proposed by Shahrimie et al. [8] provides an accurate straight middle line as the fingertip point is taken. This ensures the rotation of skewed FV image to be more precise. However, its finger edge detection was applied to FV images with a clear background and mild illumination. It is incapable of obtaining a clear finger edge if given a higher intensity of light interference.

Van et al. [9] proposed a method of ROI extraction that omits the orientation correction process. In the beginning, they detected finger edge with Canny edge detector. This separated finger image from the background. Then, they drew lines of finger edge with Hough transform. These lines served a means to estimate the direction of the finger. After that, Van et al. [9] synthesised a middle line based on finger edges. Finally, they cropped out the ROI by using a fixed rectangular window in the direction of the finger, with the synthesized middle line as reference and finger edges as boundaries. This method does not extract a good ROI as there is no localised benchmark or reference. ROI images extracted using this method are inconsistent as ROI image could be the higher or lower part of the finger depending on the placement of the finger. Moreover, the finger edge obtained could be inaccurate as the noise outside finger image was not masked out.

Most of the previously proposed ROI extraction techniques [6, 8, 9] utilise the edge-based method such as Canny and Sobel edge operators to detect the edge of the finger. Edge-based image segmentation method is not suitable to be applied on raw FV images captured under a severe non-uniform illumination. Apart from the edge-based image segmentation method, there is another category of segmentation method called region-based segmentation [10]. Region-based segmentation is based on continuity. It has not been used in FV image segmentation before because most FV images captured by different image acquisition device are not severely affected by background illumination. Some region-based segmentation methods are watershed segmentation, region growing, region splitting and region merging [11]. In this paper, we utilise watershed segmentation with a distance transform to detect the edge of the finger for ROI extraction. With the proposed method, a consistent ROI can be extracted from a finger vein image with non-uniform background illumination. This will increase the accuracy of an FVRS.

2 Proposed ROI Extraction Method

This research proposes an ROI extraction with three stages: Image segmentation and edge detection, image orientation correction and ROI extraction. For image segmentation and edge detection, instead of using an edge-based segmentation we propose a region-based segmentation, which is watershed segmentation with distance transform [12] and applied Canny edge detection. Edge-based segmentation detected finger edges along with many unwanted lines, which is not suitable for a finger vein image with a severe non-uniform background illumination. Details for each steps of the proposed ROI extraction method is explained in following sub-sections.

2.1 Finger Vein Image Segmentation and Edge Detection

First, we use a Laplacian filter with the 3 × 3 s derivative kernel to enhance the finger edge region. Then, we sharpen the FV image by subtracting the Laplacian filtered image from the original FV image. After that, we apply Otsu binary thresholding to the sharpened image to obtain a binary image. Following that, we perform distance transform on the binary image to find foreground area as depicted in Fig. 1b. In FV image, the foreground area is the finger itself, while the background area is the background region or noise other than a finger.

Fig. 1
figure 1

FV image segmentation and edge detection. a Raw image; b Distance transform; c Marker; d Mask; e Segmented FV image; f Finger edge

Full size image

Next, we apply a suitable threshold to obtain the marker for the foreground object. We label markers to mark the foreground region, and the background region. Figure 1c illustrates the marked regions. Next, we perform a watershed algorithm to obtain the shape of the foreground object and then invert it to become a mask image as shown in Fig. 1d. The FV image segmentation is completed as we applied the mask image onto the original image. Figure 1e shows the segmented FV image. Finally, we apply Gaussian blur and Canny operator to detect finger edges instead of Sobel operator as the Sobel operator is more sensitive to light interference. Figure 1f shows the Image of finger edges detected. Although there is little defect at the right side of the finger edge, it can be disregarded as it does not affect the process orientation correction and ROI extraction. Figure 2 shows the overall flow of FV image segmentation and edge detection.

Fig. 2
figure 2

Overall flow of the Finger Vein image segmentation and edge detection

Full size image

2.2 Finger Vein Image Orientation Correction

For FV image orientation correction, we adapt Yang et al.’s method [6]. First, from the finger edge detected in the previous step, as shown in Fig. 3a, we cropped out part of the top and bottom of FV image with a predefined window of (48, 10, 136, 246), where noises could still reside due to the floating position. We calculate two midpoints based on the first two detected edge pixels in the top row and the first two detected edge pixels from the last row of cropped FV image. This is because there might be some severe light interference at the horizontal middle row of finger edge caused by different position. Figure 3b depicts the drawn middle line intersection two midpoints. Lastly, we compute the skew angle of the image based on the middle line using Eq. (1) and rotate the segmented FV image as shown in Fig. 3c.

Fig. 3
figure 3

FV image orientation correction

Full size image
$$ \propto \,=\, \mathrm{arctan}\left(a\right)\times 360/2\pi $$
(1)

where \(\propto \) is the skew angle and \(a\) is the slope coefficient of the straight line obtained.

2.3 ROI Extraction

Finally, we adopt Yang et al.’s method [6] to extract ROI. First, we detect the inner contour of the FV image based on rotated finger edge as shown in Fig. 4a. Figure 4b shows the inner contour detected. Then we crop out the outside part of the contour as depicted in Fig. 4c. Following that, we enhance the remaining region with Contrast Limited Adaptive Histogram Equalization (CLAHE) using a clip limit of 6 as contrast limiting threshold and grid size for histogram equalisation of (6, 6). The use of CLAHE enhancement is to distinguish the grey values of the phalangeal joint further. Figure 4d illustrates the enhanced region.

Fig. 4
figure 4

Inner contour detection. a Rotated finger edge; b Inner contour drawn in vertical red line; c Cropped key region; d Enhanced image

Full size image

For ROI localisation, although Qiu et al.’s [5] dual sliding window method is better in terms of insensitivity to light interference, we still chose to employ single sliding window method due to several reasons. First, it takes shorter computational time. Second, a raw image captured by FVRS developed in-house only have interference in the background but not in the finger vein itself. Furthermore, the region beyond the inner contour was already discarded to eliminate light interference. Figure 5 illustrates the sliding window process.

Fig. 5
figure 5

Sliding windows’ (red box) sliding process

Full size image

In the first step of phalangeal joint detection, we divide FV image into two parts, top half and bottom half, as distal interphalangeal joint usually resides in the top half in FV image and proximal interphalangeal joint at the bottom half. We use a sliding window with 20 rows as shown in Fig. 6a to calculate the sum of grey value. Then, we slid the window from top to middle row of FV image and from the 100th row after top phalangeal joint to the bottom of FV image. This is because from Yang et al.’s [6] careful observation, distal phalangeal joint and proximal phalangeal joint are at least 100 rows apart. Next, we locate the phalangeal joints by obtaining the maximum sum of grey values in the top half and bottom half of FV image. Figure 6b depicts the estimated phalangeal joint position in the red line. Following that, we defined the height of ROI. For height, we select the rows at which top phalangeal joint and bottom phalangeal joint were detected. Lastly, we cropped out the ROI and resized it to 30 × 90 based on the row and width defined. Figure 6c shows the final ROI extracted.

Fig. 6
figure 6

ROI extraction. a Sliding window; b Estimated phalangeal joint in red line; d Resized ROI

Full size image

3 Results and Discussions

To demonstrate the effectiveness of the proposed region-based edge segmentation method two different types of experiments were conducted. First is the comparison between the proposed method with the existing edge-based segmentation methods [5, 11, 13] in the extracting of the finger edge. The second experiment is to evaluate the impact of the proposed method in the ROI extraction of FV image. In both of the experiments, we use 6 FV images with different finger positions to illustrate different scenarios where users may insert their fingers. The positions are as follows:

  1. 1.

    Straight (Not floating)

  2. 2.

    Slanted to the left (Not floating)

  3. 3.

    Slanted to the right (Not floating)

  4. 4.

    Straight (Floating)

  5. 5.

    Slanted to the left (Floating)

  6. 6.

    Slanted to the right (Floating)

All of the images used in the experiments are captured internally using our developed finger vein image acquisition device.

3.1 Comparison of the Proposed and Previously Proposed Finger Edge Detection Methods

Here, we compare our proposed watershed segmentation with distance transform and Canny edge detector with previously proposed edge-based segmentation methods [6, 8, 9, 13]. Figure 7 shows finger edges detected using different edge detection methods. Based on observation, finger edges of positions 1–3 (non-floating) are successfully detected by methods [6, 9, 13]. However, there are unwanted lines on both sides of the finger edges caused by illumination. Clearer finger edges are found with our proposed method and method [8], where the unwanted lines are eliminated. Nevertheless, the method [8] is still affected by illumination, particularly where the finger and background light seem to ‘touch’ each other at positions 1 and 3. Failure of the method [8]’s connected component labelling at the affected areas could be the reason for the incomplete detection of finger edges. Our proposed method is almost unsusceptible to illumination for positions 1–3. There is little noise for position 1, but that can be ignored.

Fig. 7
figure 7

Edge detection comparison of left index finger with various positions. a Raw images with various positions; b Canny edge detector [13]; c Sobel edge detector [6]; d Canny edge detector and Hough transform [9]; e Canny edge detector and connected component labelling [8]; f Proposed method

Full size image

On the other hand, methods [6, 8, 9, 13] failed to determine finger edges of positions 4–6 (floating). Most edges detected are incomplete or too short. As shown in Fig. 7f our proposed method can detect the finger edge efficiently for a finger with floating positions. Although some unwanted lines remained especially for position 5, the orientation of FV image can still be continued as long as both edges are present and connected.

3.2 Comparison of Various ROI Extraction Methods

In our proposed method, we utilise the ROI extraction method in [6] to correct the orientation of the finger and also to extract the ROI. To justify the adoption of the method proposed in [6], five different ROI extraction methods are compared in this section. The compared methods are as follows:

  1. I.

    Fixed window method

  2. II.

    Inner contour based [13]

  3. III.

    Adapted finger direction based [9]

  4. IV.

    Adapted maximum row sum value based [7]

  5. V.

    Adapted single sliding window based [6].

Figure 8 shows the ROI extracted using different methods. Based on observation, it can be clearly seen that ROI extracted with methods I and II are not consistent. For both methods, FV patterns vary with different finger positions. Besides that, there are noises in the ROI. It can also be observed that large patches of non-informative black pixels are present at the top and bottom of the images, especially for floating finger positions 4–6. Furthermore, inner contour based ROI extraction of method II shows that it is highly susceptible to slanted finger positions.

Fig. 8
figure 8

ROI extraction comparison. a Raw images with various positions; b Fixed window method I; c Method II [13]; d Method III [9]; e Method IV [7]; f Method V [6]

Full size image

For method III, FV patterns are more consistent than that of methods I and II. However, dark patches in ROI can still be seen in position 4–6. For methods IV and V, consistent ROIs were successfully extracted. There is little or no difference between these two methods because illumination affected area is eliminated before the process of localised benchmark estimation was carried out. Methods IV and V can be employed to extract ROI as it retains solely informative FV patterns and are consistent. In this case, method V was chosen. This is because there are still many variations that finger can be positioned. As illumination changes with the way finger is positioned shown in Fig. 7a position 5, FV images are still possible to be affected by illumination. Since it was proven by Yang et al. [6] that method V is more accurate in estimating phalangeal joints while method IV is more susceptible to illumination, method V was employed in ROI extraction.

4 Conclusion

In this paper, to detect the finger edge of a finger vein image with a non-uniform background illumination, we utilise watershed segmentation with distance transform and also Canny edge detector coupled with Gaussian blur. Following that, we correct the orientation of the image based on the edges obtained. Next, we push a single sliding window down the rows of FV image to calculate the sum of grey value to estimate the position of phalangeal joints. Finally, we extract the ROI based on the height of the phalangeal joint and width of inner contour of the finger edge. Experimental results prove that the proposed method can detect a consistent ROI from a finger vein image with various positions. This shows that the proposed region-based image segmentation plays an important role in ROI extraction of finger vein image.