Keyword

1 Introduction

With the development of functional brain imaging technology and electrophysiological technology, researchers have discovered the cerebral cortex and subcutaneous structures involved in social decision-making and value judgment [1]. In the cerebral cortex, the dorsolateral prefrontal cortex (DLPFC) and the temporo-parietal junction (TPJ) are two brain regions related to decision-making. Neural imaging studies found that DLPFC participated in cognitive functions such as working memory, rule learning, planning, attention and motivation [2]; TPJ is a multimodal brain area related to cognitive functions such as “Theory of Mind”, moral judgment and empathy [3]. In addition, there exists functional connection between the DLPFC brain area and the TPJ brain area, forming a neural circuit related to decision-making function [4].

In recent years, the fields of machine learning and deep learning have developed rapidly. Based on the neural activity of the brain, neural activity signals can be decoded by using relevant algorithms. On the one hand, it is helpful to understand the complex cognitive activity process, and on the other hand, the construction of the transmission path between the nervous system and the external information is conducive to the development of the brain-computer interface (BCI) field [5, 6]. BCI is being applied to our lives, especially in the medical field, helping patients recover sensory and perceptual functions or control artificial limbs [6]. BCI use invasive and non-invasive technology to collect the neural activity of the brain. Based on the advantages of high time resolution and low cost, functional near-infrared spectroscopy (fNIRS) is widely used in BCI field [7]. fNIRS generates multi-dimensional data through multiple measurement channels. Therefore, how to extract the key features of neural activities is conducive to promoting the further development of BCI.

In this paper, we explored the feature extraction method of fNIRS-BCI system. Based on the behavioural data and fNIRS data of relevant study [8], we use support vector machine to find that synchronization of neural activities between the right temporo-parietal junction (rTPJ) and the right dorsolateral prefrontal cortex (rDLPFC) encodes the effective information of the decision process and can be used as a key feature in fNIRS-BCI system.

2 Preliminary

2.1 Functional Near-Infrared Spectroscopy (fNIRS)

After decades of exploration, researchers have developed a variety of methods to measure brain activity to comprehensively understand the potential neural mechanisms and signal pathways of the nervous system. Among them, fNIRS is a relatively new non-invasive technology, which uses two or three wavelengths of near-infrared light to record the neural activity of the brain by measuring the changes in the concentration of oxyhemoglobin (HbO) and deoxyhemoglobin (HbR) [9] simultaneously. fNIRS system is composed of light source, electronic driving device, optical detector, signal processing device and recording device. The light source and detector form the measurement channel of fNIRS together: the light source emits near-infrared light and reaches the detector through scattering (Fig. 1).

Fig. 1.
figure 1

The schematic diagram of fNIRS.

Full size image

2.2 FNIRS-BCI System

A typical BCI system consists of five main parts: signal acquisition, preprocessing, feature extraction, classification and application interface [6]. The signal acquisition stage is mainly to acquire brain nerve activity through invasive and non-invasive technology [5]. Due to the risk of surgery and the decline of signal quality, non-invasive methods are more convenient to serve human than invasive technology. Compared with other non-intrusive technologies, fNIRS has many advantages such as low cost, high spatial and temporal resolution [10].

In the BCI system based on fNIRS (fNIRS-BCI) (Fig. 2), the pre-processing stage is mainly to remove global noise and motion artifacts in the signal through principal component analysis (PCA), independent component analysis (ICA) or wavelet denoising [10]. In the process of feature extraction, classification and application, BCI mainly extracts relevant features based on the time series of de-noised blood oxygen signal or its corresponding statistical indicators such as mean, median, standard deviation, slope and skewness, and uses machine learning or deep learning algorithms to predict behaviour or control external devices [10, 11].

Fig. 2.
figure 2

fNIRS-BCI system

Full size image

3 Methods

We use fNIRS data and behavioural data of individuals in multi-round game to carry out our research [8]. In each round of game, individuals determine the degree of cooperation (a number from 0 to 100, the greater the number, the higher the degree of cooperation), and use fNIRS technology to record the neural activities of individual rTPJ and rDLPFC.

3.1 Neural Synchronization Between Brain Regions

Based on the fNIRS data of relevant experiments, we conduct data preprocessing. First, the principal component analysis (PCA) and wavelet domain denoising are used to remove the global noise and motion artifacts of the data. Then, we use the bandpass filter to extract the data of the frequency band we care about and remove the frequency band related to physiological noise, including: low-frequency fluctuations and high-frequency physiological noise.

After data pre-processing, we calculate the Pearson correlation coefficient between individual rDLPFC and rTPJ brain regions to measure the neural synchronization between brain regions. As shown in the Fig. 3, each subject's rDLPFC and rTPJ brain area has 7 measurement channels (CH). By calculating the Pearson correlation coefficients of all channel pairs in different brain regions of individuals and converting them into Fisher z-value, we can obtain 7 × 7 (rDLPFC-rTPJ) synchronization value matrix.

Fig. 3.
figure 3

The neural synchronization between rTPJ and rDLPFC.

Full size image

3.2 Decoding of Neural Synchronization

We use neural synchronization to decode behaviour. First, in order to facilitate training and prediction, we convert the strategy value into the label. According to the strategy value in the behaviour data (the strategy value is a number from 0 to 100), we define a classification threshold (CTH) and divide it into cooperation (C) and defection (D). Specifically, when the value is greater than the threshold value CTH, the label is C, otherwise it is D. Then, we divide the data set composed of neural synchronization data and behavioural data into two parts: training set and test set (training set: test set = 8:2). Finally, based on the data set, we use support vector machine to train and predict.

4 Results

Here, we introduce four evaluation criteria of classification performance to measure the decoding effect of SVM: Accuracy, Precision, Recall and F1-Score. In order to determine the optimal threshold CTH of the classifier and test the stability of the decoding effect, we traverse the threshold CTH in the range of 50 to 95 (for example, CTH = 95 indicates that the strategy is D when it is less than 95, and C when it is greater than 95). Then, based on the data set obtained under each CTH, we train the SVM classifiers respectively, and use the corresponding classifiers to predict the behaviour.

Fig. 4.
figure 4

The result of behavior prediction based on SVM

Full size image

As shown in the Fig. 4, blue, red, yellow and purple dotted lines represent the accuracy, accuracy, recall and F1-Score corresponding to the SVM model when the CTH is at different values. The results show that under different classification threshold CTH, the four evaluation indicators of SVM model based on synchronization value are relatively stable and the performance of classification is relatively good. Our results show that prediction of behaviour can be realized only according to the synchronization value between individual rTPJ and rDLPFC brain regions.

5 Conclusion

In conclusion, based on the above results, we find that the neural synchronization between rTPJ and rDLPFC encode the effective information of behaviour, which can be accurately decoded by machine learning.

In the field of BCI, feature extraction of neural signals is the key to achieve behaviour prediction. Neural synchronization signals between brain regions reflect the relationship between neural signals at different locations in the brain, and can be used as the key feature of decoding and prediction behaviour. On the premise of achieving dimensionality reduction of data structure and reducing training costs, the high-precision prediction is guaranteed. Our work provides a new perspective for research in related fields.