Keywords

1 Introduction

Electroencephalogram (EEG) is an appropriate signal for brain biometric identification and its effective connectivity features are resistance to spoof attacks and impossible to use under pressure and coercion states.Biometrics is based on the client identification and verification system can be divided into physiological, behavioural and biosignals features associated with the clients. Data analysis procedure for biometric system usually involve pre-processing, feature extraction, and classification techniques [2, 4, 8, 19, 22, 24, 30, 32, 34], which address the complexity of the input space and the possibility of using reduced amount of training data before the classifier model can learn to generalize. In contrast, the data-driven models for speaker verification task uses a large amount of labeled data which is not only time consuming, but also requires a huge amount of training data for better generalization. Thus, proper selection of feature extraction and classification techniques is important for the design and performance of the biometric system. Other types of biometrics, such as palm, finger print, signature, and keystroke [6, 11,12,13, 27, 29], are based on univariate model types of biometric systems. These traditional biometric systems rely on a single biometric with some form of limitations.

Researchers have shown that biometric traits extracted from a person (i.e., speech) tend to vary with time and vary from one person to another. Noisy biometric data such as inconvenient ambient conditions like heart sounds with loud background noise are example of noisy input that affects the performances of the system. Non universality can arise in speech technology. Similar to a fingerprint failure to enrol, some voice print might also be impossible to acquire its trait because of the poor quality of the voice, which in turn cause its failure to enrol the system. Most of the limitations of biometric systems are mainly because of their implementation by using unimodal biometric strategy. This unimodal biometric is totally relied on a single biometric trait. In order to address such issues, researcher works in this area overcome it by multi-biometric systems and using efficient fusion type to combine these systems. The information then is presented in a multiple biometric trait.

This paper propose a methodology based on specific features and classification techniques involving EEG data analysis for identification for healthy subjects. Thus, biometrics with multiple tasks or modalities can improve the system performance reliability of person recognition and also increase the difficulty of forging biometric data. This paper address the gap of impeding the implementation using EEG effective connectivity multivariate signal modal for biometric system.

2 Methods

2.1 Multi-channel EEG Biometric System

The application of multivariate EEG biometric utilizes the multi-channel time series modeling to infer the dynamic of physiological system. The Hidden Markov Model (HMM), is a well know classification method and has been proven tremendous performance, however with big data availability, and high computing power in processing, Deep Learning approach would be more reliable as it able to sustain good performance as parameter increased. In the last two decades, Deep Learning (DL) [1, 31, 36], a form of classifier is a family of Machine Learning (ML) or a traditional approach methods has gained considerable attention in the scientific community. In the exploration of DL it have succeeded in breaking the benchmark, in areas such as mining in biological data [17], reinforcement learning to biological data [16], classifying ECG signals [26, 28] and many more.

For this paper, HMM is used. The method of using HMM, has shown to provide a reliable performance in estimating single-channel EEG, heart sound, ECG, speech signal and etc., however, the limitation of using univariate data is the process includes only correlation in time precedence of a signal, while the correlation between the other input signals is ignored. The alternative to this problem is the generalization of univariate model to multivariate modeling.

Using multivariate model, the inter-channel correlation could give additional information to discriminate between the different input signals where the models or methods can measure the synchronization between these signals and the coherency among them. One distinct disadvantage of a multivariate signals with different inputs stand in its difficulty to implement multiple sensors input and it can be costly. An alternative solution is the use of multivariate signal such as EEG where it is one of the promising and potent biosignals for brain biometrics [3, 5, 18, 20, 23, 25, 35, 37]. The use of functional magnetic resonance imaging (fMRI) time series multivariate analysis with Independent Component Analysis (ICA) is also frequently used by researchers to analyse this type of signal [14, 33]. However, the main drawback of this multivariate method is that it only assesses the spatial correlation, while the temporal correlations were ignored which can lead to results misinterpretation. Thus, not only it requires specialized equipment, it may not be suitable for task-related or highly non-stationary time series signals.

The multivariate EEG biometric system experimental set up, where the raw data time series is modeled using MVAR or Time-Varying Vector Autoregressive (TV-VAR) as time-domain feature extraction method. Later, this MVAR will be converted to PDC or COH as the frequency domain features. The analysis of the model performance is based on different biometric EEG signals, the complexity of the HMM classifier hyperparameters (i.e. states and Gaussian mixtures). The dataset consists of EEG biosignals of multi-subjects, multi-channels, with multi-trails as password-stimulated data. The first evaluation of the biometric system is the client identification experiments. Clint identification requires a close community only, which mean the data are train and test within the client’s group.

EEG Database. We use a locally recorded EEG data obtained from the Computational Intelligence and Technologies (CIT) Research Group from Universiti Teknikal Malaysia Melaka (UTeM), to carry out experiments and validate the proposed methodology in this study. A summary of the data base is discussed here, and the full detail of the work can be found in [15]. During the data acquisition, eight Visual Evoked Potential (VEP) electrodes are used to record the EEG signals of control subjects. These occipital electrodes (i.e., PO7, PO3, POZ, PO4, PO8, O1, OZ and O2) were placed on subject’s scalps which the EEG signals are digitized 256 Hz for intervals of one-second. A total of 37 subjects with each subject involved in 120 trials where 60 trials were shown different stimuli and 60 trials for password stimuli.

The 37 subject trials were randomly partitioned into 50% for training and 50% for testing. The HMM is trained using the training set to construct the subject-specific client’s individual models. The system utilized continuous density HMM which further evaluated on different experiment setups, where different number of Markov states and Gaussian mixtures are used to construct the HMM models with different complexities. The number of trials conducted wss 1 s password-stimulli and 1.5 s of inter stimulus intervals.

The data were collected with the subject seated on a back-rested chair in front of a computer display which located 1 m away from the subject’s eye level. The random or password pictures were displayed in sequence at the screen center with a fixation point. The collected data are divided into two conditions, a quiet environment and a noisy environment where an external audio clip of recorded office noise effects played through the audio speaker.

Subjects were asked to recognized the pre-selected password picture among the sequentially visualized pictures on the screen. The inter-stimulus interval for each trial was set to 1.5 s and the picture remained on the computer screen for 1 s followed by 1.5 s of white-blank screen. A short break of 5 min was interspersed in between the recording sessions to provide rest time to the subject to ensure good attention from the subject during the experiments [15].

Feature Extraction for Multi-channel EEG. VAR is a stochastic process model which can be used to capture the linear inter-dependencies among multiple time series of the EEG signals. VAR models generalize the univariate Autoregressive (AR) model by allowing for more than one evolving variable. VAR captures the spatial underlying dynamics of these variable using their own lagged values. These dynamics are represented by coefficient matrices and an error term. The only prior knowledge required is the past observations of all variables which can be hypothesized to affect each other inter-temporally. The EEG signals can be presented as a stationary piece-wise TV-VAR process \(\mathbf{\Phi }_{\ell }\). TV-VAR is a natural extension of the VAR model to dynamic multivariate time series.

The rapid changes of the EEG signals can be efficiently modeled using TV-VAR model \(\mathbf{\Phi }_{\ell t}\). The VAR model can be utilised to estimate time–domain directed EEG connectivity between various scalp regions. If the signal \(\mathbf{y}_t=[y_{1t}, \ldots ,y_{Nt}]'\),

   \(t=1, \ldots ,T\) represent the EEG signal then:

$$\begin{aligned} \mathbf{y}_t = \sum _{\ell =1}^{L} \boldsymbol{\varPhi }(\ell t) \mathbf{y}_{t-\ell } + \boldsymbol{\epsilon }_t, \ \ \ \boldsymbol{\epsilon }_t \sim N(\mathbf{0},\boldsymbol{\varSigma }) \end{aligned}$$
(1)

where \({[\boldsymbol{\varPhi }_{\ell t}]}_{\ell =1}^{L}\) are \(N \times N\) matrices of time-varying AR coefficients at time t and \(\boldsymbol{\epsilon }_t\) is a \(N \times 1\) i.i.d. Gaussian observational noise with mean zero and covariance matrix \(\boldsymbol{\varSigma }\), \(\boldsymbol{\epsilon }_t \sim N(\mathbf{0},\boldsymbol{\varSigma })\sim W(\mathbf{0},\boldsymbol{\varSigma })\).

TV-VAR is an extension of the stationary AR model that can describe instantaneous correlation in time series, by allowing the coefficient matrix to change with time. Hence, a sequential implementation of Ordinary Least Square (OLS) algorithm is used to estimate the TV-VAR coefficients. To estimate the state noise covariance Q and the autoregressive coefficients, \(\varPhi \), for each state dynamics, the OLS method was used.

$$\begin{aligned} \hat{\varPhi } = (y_{t-1} ^\prime y_{t-1})^{-1} (y_{t-1} ^\prime y) \end{aligned}$$
(2)

where \(y_{t-1}\) is 1 \(\times \) P and contains the P = 1 lag observations of Y as an example. The OLS procedure is to estimate VAR coefficient using raw EEG data that is needed as the input features to the HMM. The classification potential of the static frequency-domain EEG connectivity to our knowledge is rarely explore in this area. For this purpose, the frequency domain of Generalized Partial Directed Coherence (GPDC), PDC, Directed Coherence (DC), and COH was extracted from the VAR parameters.

2.2 Classification (Hidden Markov Model)

The HMM model is a probabilistic model relies on Markov chains theory, wherein each Markov state does not directly correspond to observation data, instead it connects to a set of probability distributions of a state. The HMM is a common effective model which has shown to be successful in speech processing, biometrics, and medical applications [7, 9, 10, 21].

The input to HMM is the vectorized matrices of wither Mel Frequency Cepstral Coefficents (MFCC), VAR, or TV-VAR connectivity coefficients. The HMM uses dynamic programming (Vertibi algorithm) to maximize the objective function of log likelihood. The model states are initialized using K-means clustering to distribute the VAR frames for the Gaussian mixture.

Fig. 1.
figure 1

Show the flow of the block diagram of training the HMM. Vector Quantization (VQ) was used to obtain the mean and the variance to train the model with specific parameters (A,B,\(\pi \)).

Full size image

As shown in Fig. 1 the input features of the univariate or the multivariate signal, either as a static or dynamic (windowed) features are fed to the HMM classifier. The continuous HMM was used to model such univariate or multivariate features for the 37 classes representing the enrolled 37 clients. The HMM is a probabilistic method that can be used to statistically model the dynamical changes of the different types of input signals by making inferences about the likelihood of being in certain discrete states. In this study, a continuous HMM with Gaussian mixtures consisting of five states (left-to-right) and different Gaussian mixtures Probability Density Functions (PDF) for each state was used. In specific cases where the whole trial of the signals is assumed to be stationary, the Gaussian model is the preferred choice rather than HMM. An HMM can be defined as \(\lambda \) = (A, B, \(\pi \)), where, \(\pi \) is the initial state probability vector, A = \([a_{ij}]\), \({1 \le i,j \le N}\) is the state transition matrix, and B = \([{b_{j}}(X)], {1 \le j \le N}\), is the observation probability function for each state j.

The observation probability B can be modeled by the continuous PDF of predefine Gaussians mixtures, such that, \({b_{j}}(X)\) can be calculated as in Eq. (3).

$$\begin{aligned} b_j(x)=\sum _{m=1}^{M} C_{jm} \mathcal {N}(x,\mu _{jm},\varSigma _{jm}), \quad 1\le j \le N \end{aligned}$$
(3)

where x represents the vector that is modelled, \(C_{jm}\) refers to the mixture coefficient of \(m^{th}\) mixture component present in the state j, \(\mathcal {N}(x,\mu _{jm},\varSigma _{jm})\) is a multivariate Gaussian probability distribution function with mean vector \(\mu _{jm} = [\mu _{jmd}]\) and the covariance matrix of \(\varSigma _{jm} = [\varSigma _{jmd}]\) in the case of the \(m^{th}\) mixture component present in the state j, for \(1 \le d \le D\), where D is the dimension of the feature vectors. Before the evaluation process, training and testing of the system were conducted. Training is the initial part of the signal before it is being modelled. The parameters that were considered included the percentage split, in where one recording of data as an example can be split to 50%. The first 50% was used for training the data and the second 50% was used for testing.

3 Results and Discussions

To evaluate the HMM ability for EEG biometric system, several experimental setups were carried out. The first experiment test the multivariate signals with one source of input sensors and the second experiment seek to understand the unimodal with univariate signal from different input sensors.

3.1 Multi Channel EEG Biometric

The proposed model of biometric system used EEG data of healthy subjects which were processed with VAR to extract more meaningful effective brain network features. This section compares the performance of the stationary and non-stationary of the visual cortex area by modeling the signals using TV-VAR and TV-frequency domain, respectively.

With window size set at 25 ms and 10 ms overlaps, the TV-VAR performed sightly better with an accuracy of 95.33% using three Markov states and 6 Gaussian mixture as shown in Table 1. The initial findings suggest that the biometric system can use the static VAR (assume the brain signal is stationary across the entire scanning time) or the TV-VAR model, however, further experiments should be carried out to confirm the reliability of the two system.

Table 1. Shows the performance of different time-series TV-VAR features of multivariate EEG biometric system using Markov state. The performance is based on accuracy averaged over 37 subjects.
Full size table

The time series approach TV-VAR shows almost similar between the state performance results while the frequency domain also perform but significantly depends on frequency features used. The results show the presence of the effective connectivity of the VAR changes over time. These changing connectivity structures can be said to reflect the behaviour of underlying brain states from the 37 subjects. To detect the state related change of brain activities based on effective connectivity, the next experiment will explore on the frequency domain of the EEG brain signal.

Table 2. Shows the performance of different frequency-domain features of multivariate EEG biometric system using single Markov state. The performance is based on accuracy averaged over 37 subjects.
Full size table

Table 2, shows the static EEG brain signal in the frequency domain performance using one state and different Gaussian parameters. The best performance form different type of feature of HMM in the frequency domain from GPDC gives an accuracy of 77.77%, PDC, 94.59%, DC, 62.16%, and COH 94.59%.

Fig. 2.
figure 2

Model performance accuracies based on different frequencies and time domain features.

Full size image

Figure 2, shows the performance of TV-VAR (1 state) and the frequency domain features. The best performance is obtained from TV-VAR (1 state) time domain with an accuracy of 94.84% with Gaussian 5 State 1 complexity. As for the frequency domain features for PDC and COH was able to achieve almost the same performances with the Gaussian 1 State 1 complexity when compared with the VAR model. However, extensive experiments conducted by the TV-VAR at different state have shown a slight improvement of 95.33% for a Gaussian 6 State 3 complexity.

3.2 Conclusion

The use of unimodel biometric system such as speech, does suffer from specific condition such as the speech can be contaminated with noise or a cold can seriously affect its performance. On the other hand, heart sound biometric can be exposed to non-universality where a faint heart sound from a overweight client or heart sound which can be contaminated with the sound of lungs and artefact can seriously damage the information of heart sound. This may result in the possibility of a client being unable to enrol into the biometric system. Furthermore, the univariate model is also likely to be exposed to spoof attack. The disadvantage of the univariate modal biometric is its reliance on a single biometrics trait. In order to address the said issues, an extension to the univariate model, we applied the multi model biometric system. The biometrics with multiple modalities have shown to improve the accuracy of client recognition and also increase the difficulty of forging biometric data. Acquiring data from a variety of input sensors such as discussed above can be time-consuming because the tasks must be performed sequentially. We proposed a new design scenario for multiple tasks in a biometric system, where, it only involve a series of tasks in a single paradigm rather than having users perform several tasks one by one. The key issue is to handle the single task paradigm proposed in this paper with multivariate signal EEG classification using MVAR rather than univariate model. The study on univariate model is limited with unrealistic assumptions such as stationary and linearity which are far from real in relationship to the behavioural signal (speech) as well as the biosignal (heart sound, ECG and EEG) for the biometric task.

3.3 Future Work

The discussions are part of the proposed framework to estimate the underlying dynamic effective connectivity in brain imagery EEG time series and the frequency domain. This paper applied the time series VAR as well as two derived frequency domain measures, example, PDC, to investigate dynamic causal interactions between parietal-occipital (PO) and occipital (O) areas in discriminating visual imagery. These findings result in some unanswered questions which will be address in future work. As an example, is it reliable to assume the brain signal to be stationary across the entire scanning time or it is dynamically changing and thus how to handle the effective connectivity of the brain? In addition, the time domain as well as the frequency domain seems to provide significant performance. Which type of features would be appropriate to estimate the dynamic changes in effective connectivity as well as how to design a proper multi model biometric that will increase the performance of this system?