Keywords

1 Introduction

The need for user authentication techniques and concerns about security and vast progression in networking, and communication has increased in the past few decades. Traditional methods are commonly used for authorizing and binding access to different systems even though these systems could be attacked and the security can be overridden. Biometrics technologies have replaced the traditional authentication methods due to their ability to authenticate the right personality of different people requesting a service [1].

Biometric recognition systems aim at the automation of recognition of a person’s identity based on physical or behavioral characteristics (something a person is or produces).

Since majority of biometric systems are single modal which rely on the single biometric information of authentication, problems with those biometrics trait information such as (noise in sensed data, intra-class variations, and inter-class similarities, etc), results in problems such as authenticating the unauthorized user as authorized users (FAR) and rejecting the authorized users (FRR). Usually, we are using FAR and FRR to measure the performance of biometric systems. Another measurement is (EER) could also be used, EER is a cross point when drawing FRR verses FAR (i.e., the equal values of FRR with FAR [2].

Nowadays there is more concern of solving some inherited problems of biometric systems (intra-class variations, inter-class similarity etc.). Possible solutions are to use more than one modality to reduce the classification problems which raise the intra-class variety and inter-class. Multiple biometric traits could be used to improve the performance of biometric systems. Combining or fusing of more than one biometric system is referred as Multi-biometric system [3]. The Multi-biometric systems can offer essential improvement in the authentication accuracy of a biometric system, as it depends on more than one biometric data.

The term Multi-biometric refers to the fusion of different types of biometrics according to the way of fusing the biometrics data as follows [2]: Multi-sensor: Multiple sensors are used to collect information of the same biometric. Multi-sample: more than one consideration of the same biometric is taken at the time of the enrollment and/or recognizing time, e.g. a number of face readings are taken from different sides for the same person. Multi-algorithms: different algorithms are used for extracting the same biometric features and matching them with the already obtained database. Multi-instance: means the use of the same biometric trait and processing on multiple instances of the similar biometric trait, (such as left and right irises) [1, 4]. Multi-modal: Multiple biometric modalities can be collected from the same person, e.g. fingerprint and face, which require different sensors.

Thus this paper evaluates the performance of multi-modal approach by fusing the data at match score level using Sum, Max, and Min rules. The rest of the paper is organized as follows: Sect. 2 presents related works, proposed method is given in Sect. 3, detailed experimental results are given in Sect. 4, fusion strategies in Sect. 5, result and discussion in Sect. 6, and the conclusion is mentioned in Sect. 7.

2 Related Works

Meraoumia et al. [5] proposed a personal identification multi-modal biometric system by using palm print and iris modalities. In this work, the authors describe the development of a multi-biometric system based on Minimum Average Correlation Energy Filter (MACE) method (for matching).

Morizet and Gilles [6] have suggested a new fusion technique to combine scores obtained from face and iris biometric modalities. Based on a statistical analysis of boots trapped match scores elicitation from similarity matrices, the authors show the usefulness of wavelet noise removal by normalizing scores.

Toh et al. [7] have proposed a diverse polynomial model increasing the number of parameters longitudinally with model order and the number of inputs. First, the model is subjected to a well-known pattern classification problem to elucidate the classification capability such as the above-mentioned methods and then followed by a biometrics fusion combining fingerprint and voice data.

Giot et al. [8] in their paper have proposed a lower cost multimodal biometric system fusing keystroke and 2D face recognition. The suggested multimodal biometric system has improved the recognition rate compared to the individual method.

Rodrigues et al. [9] have proposed two schemes that could increase the security of multi-modal biometric systems. Experimental result shows that the suggested methods are more sturdy against spoof attacks compared to classical fusion methods.

Shahin et al. [10] have introduced a multi-modal system based on the fusion of entire dorsal hand geometry and fingerprints that achieves right and left near-infra-red hand geometry and right and left index and ring fingerprints. Scores obtained from different biometric modalities matchers were fused using the Min–Max score fusion technique.

Wang et al. [11] have proposed a method to combine the face and iris features for developing a multi-modal biometric system. The authors pick out a virtuous feature level fusion plan for fusing iris and face features in sequence, and normalizing the pristine features of iris and face using z-score model to reduce estrange in the unbalance of girth.

3 Proposed Methodology

In this paper, different modalities have been used namely: Face modality of AR-Face database, iris modality of CASIA-Iris database, and Finger Knuckle Print (FKP) modality of D. Zhang FKP database. FKP refers to the image pattern of the outer surface around the phalangeal joint of one’s finger.

3.1 Preprocessing

This section describes the extraction of the Region of Interest (ROI). The process involved to extract ROI for FKP is shown in Fig. 1 and the process involved to extract ROI of Iris is shown in Fig. 2.

Fig. 1
figure 1

a Image acquisition device is being used to collect FKP samples; b sample FKP image; c ROI coordinate system, where the rectangle indicates the area d extracted ROI

Full size image
Fig. 2
figure 2

Region of interest extraction of Iris: a Original edge image. b Edge image after edge detection. c Edge image after deleting noise and thinning. d Circular Hough transformed is used to detect the iris border

Full size image

4 Feature Extraction

In this paper, the following feature extraction algorithms have been used to extract the features prior to fuse a different modalities combination.

(1) To extract the features from finger knuckle print, Log-Gabor filters have been used. Log-Gabor proposed by Field [12], suggests that natural images are better coded by filters that have Gaussian transfer functions as they are seen on logarithmic frequency scale. On the linear frequency, the Log-Gabor function has a transforming function of the form:

$$G\left( w \right) = e^{{{\raise0.7ex\hbox{${\left( { - \log\left( {{w \mathord{\left/ {\vphantom {w {w_{0} }}} \right. \kern-0pt} {w_{0} }}} \right)^{\text{2}} } \right)}$} \!\mathord{\left/ {\vphantom {{\left( { - \log\left( {{w \mathord{\left/ {\vphantom {w {w_{0} }}} \right. \kern-0pt} {w_{0} }}} \right)^{\text{2}} } \right)} {\left( {\text{2}\left( {\log\left( {{k \mathord{\left/ {\vphantom {k {w_{0} }}} \right. \kern-0pt} {w_{0} }}} \right)^{\text{2}} } \right)} \right)}}}\right.\kern-0pt} \!\lower0.7ex\hbox{${\left( {\text{2}\left( {\log\left( {{k \mathord{\left/ {\vphantom {k {w_{0} }}} \right. \kern-0pt} {w_{0} }}} \right)^{\text{2}} } \right)} \right)}$}}}}$$
(1)

where w 0 is the filter’s center frequency and k/w 0 is a constant for different w 0.

(2) To extract the iris’s features, the Local Phase Quantization (LPQ) methods have been used. LPQ introduced by Ojansivu et al. [13]. LPQ is based on the blur undisparity property of the Fourier phase spectrum. It uses the local phase information extracted by the 2D DFT which is computed over a rectangular M-by-M neighborhood Nx at each pixel of the image f(x) defined by:

$$F\left( {u,x} \right) = \sum\limits_{{y\varepsilon N_{x} }} {f\left( {x - y} \right)} e^{{ - j2\pi u^{T} y}} = w_{u}^{T} f_{x}$$
(2)

where w u is the basis vector of the 2D DFT at frequency u and f x is another vector containing all M 2 image samples from N x [13].

(3) Principal Component Analysis (PCA) was invented in 1901 by Karl Pearson. PCA is a mathematical procedure that uses an orthogonal transformation to convert a set of observations of correlated variables into a set of values with uncorrelated variables called principal components.

5 Biometric Fusion Strategies

A biometric system works in two different modes: enrollment and authentication. The two authentication modes are verification and identification. Combining biometric systems, algorithms, and/or traits is a good solution to improve the authentication performance of biometric systems. A lot of researchers have shown that multi-biometrics enhanced the authentication performance.

In biometric systems, fusion can be performed at different levels: sensor level, feature level, score level, and decision level fusions [14].

5.1 Sensor-Level Fusion

It is the integration of testimonials presented by different sources of raw data before throwing in one’s hand for feature extraction. Sensor-level fusion can be availed from multi-sample systems which grip multi-snapshots of the same biometric.

5.2 Feature-Level Fusion

In feature-level fusion, the feature suit constructed from multiple biometric algorithms are conjoined into a single feature set by applying a suitable feature normalization, transformation, and reduction planner [3, 14].

5.3 Score-Level Fusion

The matching scores output by verity of biometric matchers are joined to generate a new scalar. Score level fusion is shown in Fig. 3.

Fig. 3
figure 3

Basic concept of the score-level fusion

Full size image

5.4 Decision-Level Fusion

Fusion is achieved at the epitomized or decision level the only final decisions are obtainable (e.g. AND, OR, Majority Voting, etc.).

In all the experiments, the data have been fused at score level, using ‘‘Sum,’’ “Max,” and ‘‘Min’’ rules for two and three modalities combinations.

6 Results and Discussion

This section tackles the inquisition results of joining different biometric modalities at score level fusion with ‘‘Sum,’’ “Max,” and ‘‘Min’’ rules to measure the performance of multimodal system.

$${\text{Sum}}\;{\text{Rule}};\;S_{i} = \sum\limits_{m = 1}^{M} {S\begin{array}{*{20}c} m \\ i \\ \end{array} }$$
(3)
$${\text{Max}}\,{\text{Rule}};\;S_{i} = {\text{Max}}\left( {s\begin{array}{*{20}c} 1 \\ i \\ \end{array} ,s\begin{array}{*{20}c} 2 \\ i \\ \end{array} , \cdots s\begin{array}{*{20}c} M \\ i \\ \end{array} } \right)$$
(4)
$${\text{Min}}\;{\text{Rule}};\;S_{i} = {\text{Max}}\left( {s\begin{array}{*{20}c} 1 \\ i \\ \end{array} ,s\begin{array}{*{20}c} 2 \\ i \\ \end{array} , \ldots s\begin{array}{*{20}c} M \\ i \\ \end{array} } \right)$$
(5)

In all the experiments, performance is measured in terms of False Acceptance Rate (FAR in %) and corresponding Genuine Acceptance Rate (GAR in %). To start with the performance of a single modality biometric system is measured. Then, the results for multimodal biometric system are scrutinized. The results attained from single modality biometric system are shown in Table 1 and illustrated as Receiver Operating Characteristic (ROC) curve in Fig. 4.

Table 1 Performance of single modality
Full size table
Fig. 4
figure 4

The ROC curve performance of single modality

Full size image

By examining the data in Table 1 it can be noted that the FKP have the highest performance among the three modalities at all the FAR values and Iris has acceptable performance at FAR = 1 but low performance value at FAR = 0.01 compared to FKP. On the other hand, the face has substantial less performance values than other modalities at all values of FAR.

The results attained from two modality biometric systems are shown in Table 2, and illustrated as ROC curve in Fig. 5.

Table 2 Performance of two modality biometric systems
Full size table
Fig. 5
figure 5

The ROC curve performance of two modality systems at score level

Full size image

By examining the data in Table 2, it can be observed that the fusion of two modalities at score level (Sum Rule) when fusing FKP with any other modality, the performance has achieved good improvement at all FAR’s values compared to FKP as single modality performance and very high improvement compared to face or iris performance as a single modality. Fusing face with iris has achieved good performance improvement at FAR = 1; it could be noticed by comparing that the single modality result is 62.00 % for face and 74.00 % for Iris and the fused result is 93.50 %. At FAR = 0.01, it has less improvement performance than iris as a single modality which is 45.00 % with the fused result which is 46.00 %. The fusion of two modalities at score level (Max Rule) when fusing FKP with other modality performance degrade compare to the performance of FKP as a single modality. On the other hand, the performance of fusing face with iris has a good improvement than a single modality for either Iris or face; we could notice here the Max rule is giving good result for fusing two weak modalities but degrading the performance when there is a strong modality. The fusion of two modalities at score level (Min Rule) does not exhibit any performance improvement over a single modality. The performance fusion of two modality systems at decision level is illustrated as a ROC curve in Fig. 5.

The results attained from three modality biometric systems are shown in Table 3 and illustrated as Receiver Operating Characteristic (ROC) curve in Fig. 6.

Table 3 Performance of three modality biometric systems
Full size table
Fig. 6
figure 6

The ROC curve performance of three modality systems at score level

Full size image

By analyzing the data in Table 3, it can be observed that the fusion of three modalities at score level (Sum Rule) has good score improvement over the two modalities. It could be noticed by comparing the best two modality results at FAR values 0.01 and 1 which are 88.50 and 98.50 % with the best of three modality results which are 95.00 and 99.50 % at FAR values 0.01 and 1, respectively. The fusion of three modalities at score level (Max and Min Rules) has no performance improvement over the two modalities. The performance fusion of three modality systems at decision level is illustrated as a ROC curve in Fig. 6.

7 Conclusion

By analyzing the experimental results, it can be concluded that the performance fusion of two modalities at score level “Sum Rule” has some score improvement over the single modality. Fusing FKP with either face or iris, the performance has a higher score over the best single modality performance at all FAR values except for fusing face with iris at FAR = 0.01 with less score improvement 1 %. At score level “Max Rule” fusing FKP with either face or iris, the performance has degraded over a single modality at all values of FAR. But fusing face with iris has gained some improvement over single modalities. At score level “Min Rule,” the performance was almost the same as the best single modality.

The performance fusion of three modalities at score level “Sum Rule” has a good score improvement over the fusion of two modalities at FAR = 0.01 about 4 % over the highest performance of two modalities, but less improvement at FAR = 0.10 about 1 %. At score level “Max and Min Rules,” the performance has degraded over two modalities.