Keywords

1 Introduction

Galvanic Skin Response (GSR), also known as ElectroDermal Activity (EDA) or Skin Conductance (SC), is the electrical signal of the sympathetic activity of sweat glands. In detail, GSR is typically recorded as the electric conductance between a pair of electrodes placed over a person’s skin, near high density regions of sweat glands (e.g., hand palm or fingertips). The GSR signal is characterized by two main components: (i) a slowly varying tonic component, or SC level (SCL); (ii) a phasic component or SC response (SCR) where several peaks appear corresponding to specific events. The hypothesized connection between variations in a subject’s skin conductance and psychological state has been confirmed at the beginning of the \(21^{st}\) century by means of a simultaneous analysis of brain function, using functional Magnetic Resonance Imaging and GSR [10]. In the last decade, wearable devices have given us the opportunity of a non-invasive measure of the GSR signal through simple settings [8].

GSR peaks are difficult to extract from an observed GSR signal for a number of reasons, such as potentially overlapping SCR, or a predominant SCL [29]. In recent years, various signal decomposition approaches have been proposed to overcome these difficulties [1, 5, 6] but the problem still remains. The conventional approach to evaluate GSR peaks is based on the trough-to-peak detection [25] and it has already been explored depending on three different types of acoustic stimuli in [23]. This work investigates, instead, a Compressed Sensing (CS) [16] approach that exploits the intrinsic sparse nature of peaks. CS is, indeed, a technique able to exploit the signal sparsity in some domains [11,12,13,14,15, 22]. As shown in [24], CS exhibits better performance in terms of peak count, if compared to Ledalab automatic toolbox for GSR processing. It is important to consider that not only CS allows to perform a correct count of the number of GSR peaks, but it also works on compressed samples, while automatic toolboxes (such as Ledalab or EDA Explorer) do not implement any compression mechanism. Moreover, compression mechanisms are generally a valid instrument to solve the problems in Internet-of-Things (IoT) paradigm adopted for healthcare monitoring and management (such as big data quantity, security, privacy) [18].

The aim of this work is to investigate the effectiveness of a decomposition-based CS method [26] for reconstruction of GSR phasic component. The method proposed in [26] is firstly evaluated by using synthetically generated signals. Then, the method is analyzed also on GSR signals experimentally acquired by an IoT-enabled wrist-worn device, the Empatica E4 wristband [20], in reaction to unpleasant acoustic stimuli.

The paper is organized as follows: Sect. 2 shortly describes the GSR signal, reviewing the state-of-the-art about GSR signal analysis in time domain with the related issues. Section 3 presents the acquisition IoT-enabled sensing device, the protocol to collect data, as well as the methods used to process it. Section 5 presents and discusses the results obtained. Finally, Sect. 6 concludes the paper.

2 Background

2.1 GSR Signal

The GSR signal is a physiological signal reflecting changes in the electrical properties of the human skin, resulting from the activity of the Sympathetic Nervous System (SNS, a branch of the autonomic nervous system) [17]. As such, GSR values provide an optimal marker of both psychological and physiological arousal, being associated to emotional and cognitive human activities [9].

Research studies have proposed two different approaches to measure GSR signals: the exosomatic method (applying a direct current (DC) or an alternating current (AC)) and the endosomatic method (without applying any external current or voltage) [30]. The former approach is the most commonly used, with an external constant voltage source that is connected to the human skin through electrodes. Generally, a GSR sensing device consists of the following components: two electrodes to collect signal, an amplifier to increase the signal amplitude, and a digitizer to convert analog raw signals into digital form; in the case of a wireless device, also data transmission modules (e.g., Bluetooth or WiFi transceivers) are added for communicating with a recording system [25]. Both the measuring methods mentioned above produce a GSR signal, that can be always decomposed in the two components: SCL and SCR. A single SCR, as shown in Fig. 1 from [3], is characterized by: i) peak amplitude (amplitude difference between the onset and the maximum of the peak - SCR amplitude); ii) latency (time interval between the stimulus onset and the GSR peak onset - SCR latency); iii) rise time (time interval between the onset and the maximum of the peak - rise time); iv) recovery time (time interval from peak to total recovery - rec.t/2 and rec.tc, 50% and 63% recovery respectively).

Fig. 1.
figure 1

Shape of a SCR and the related descriptive metrics [3].

Full size image

2.2 GSR Signal Analysis in Time Domain

For the purpose of both emotion detection and recognition, the GSR signal is one of the most investigated physiological signals. In particular, the response to specific external stimuli, such as audio [23] and video [31], is analysed in time domain in terms of SCR peaks and number of peaks over time.

Among the algorithms developed to analyse the SCR signal, the detection of both SCR peaks and troughs is the most used. Such algorithm identifies a peak or trough, by determining the time-series points where the derivative of the signal is zero; then, the corresponding SCR peaks amplitude and rise-time are computed with respect to the peak onset [1]. However, a weakness of this implementation is evident when short inter-stimulus intervals are established. In fact, it may happen that two close SCR peaks overlap, with the tail of the preceding SCR peak hiding the initial onset of the next SCR peak [21]. A misdetection can result in a distorted estimation of the event-related values, and a pair of events may be detected as a single one. To face this problem, several studies have proposed mathematical models to decompose the GSR signal into its two components. As an example, Alexander et al. [1] proposed an automated analysis based on the mathematical process of deconvolution to extract the phasic activity from the GSR signal [6]. This approach relies on the physiological assumption, according to which the GSR signal results from a convolution between the sudomotor nerve activity (corresponding to a driver function) exhibiting peaks in response to a stimulus, and an Impulse Response Function (IRF). By deconvolving the GSR signal with a specific IRF, the driver function (sequence of discrete peaks) is revealed, and the peaks can be identified to reconstruct the corresponding SCR [5]. In particular, the IRF depicts the SCR shape resulting from an impulse. In 1987, Schneider et al. [28] modeled the SCR shape with a bi-exponential function, called Bateman function, in which time constants represent the steep onset and slow recovery. However, the inter- and intra- individual variability, that are significantly evident in SCR shape, can affect the performance of this approach [27].

Another GSR signal decomposition method, presented by Benedek and Kaernbach in [6], is based on nonnegative deconvolution, called Discrete Decomposition Analysis (DDA). This approach lies on the nonnegativity of the driver function and maximal compactness of its impulses (i.e. peaks). Same authors also proposed the Continuous Decomposition Analysis (CDA) based on a standard deconvolution algorithm. Both the overmentioned decomposition methods are widely used and freely available in the Ledalab toolbox [4]. In this case, the concept of single and discrete response is replaced by a continuous measure of SCR (i.e. phasic activity - an indicator of sympathetic activity) and a response window (indicator of event-related activity).

3 Materials and Methods

3.1 The IoT-Enabled Sensing Device

Experimental sessions were performed using a single wearable, the Empatica E4 [19], an IoT-enabled and multi-sensor wristband device designed to comfortably acquire in real-time data during everyday life. As specified by the manufacturer, the device hosts four sensors, namely a photoplethysmographic sensor (PPG) sampled at 64 Hz, a 3-axial MEMS accelerometer (sampling frequency, \(f_s\) = 32 Hz), a GSR sensor (\(f_s\) = 4 Hz), and an optical thermopile (\(f_s\) = 4 Hz).

This study only considers the signals generated by the GSR sensor (see Table 1 for further technical details), which measures the electrical conductance of the skin from the bottom of the wrist, by applying an extremely small amount of alternating current between two silver-coated electrodes embedded into the device bracelet.

Table 1. Technical specification of the GSR sensor.
Full size table

Experimental tests involving the IoT-enabled E4 device can be conducted in two different modalities: streaming and recording mode, with a claimed battery life >20 hours and >36 hours, respectively. In our study, participants used the Empatica device in real-time streaming mode, connected via Bluetooth Low Energy (BLE) to the mobile App (i.e., E4 realtime) running on a smartphone. The recorded samples are automatically uploaded and safely stored in the E4 Connect cloud-based repository, where also the session duration, device serial number and session date are available as well. Following the data collection session, data acquired can be downloaded as a compressed folder (.zip), containing one .csv file for each sensor and an additional file (named tags.csv) related to events marked during a session. The overall GSR sensing and data acquisition process is graphically shown in Fig. 2.

Fig. 2.
figure 2

The E4 IoT-enabled GSR data acquisition process: following the measuring session, data can be retrieved from the Empatica cloud platform.

Full size image

Concerning the GSR files, data samples given in microSiemens (\(\mu S\)) are listed in a single-column format, after indicating the starting time (\(t_{0}\), expressed in UTC) of the acquisition process in the first row, and the sampling rate in the second one.

3.2 Data Collection

Six healthy subjects (3 males and 3 females, aged between 20 and 60 years old) were enrolled in this study. Before starting the data collection, participants were briefed on the study procedure, and an informed consent was signed.

To avoid as much as possible any distraction during the signal acquisition sessions, the participants were left alone in their room, and asked to lay on a bed and relax with closed eyes. The E4 was placed on the dominant wrist to acquire the skin electrical signal. Prior to signals registration performed, volunteers were asked to push the event-marker button located on the wristband, at the start and at the end of each acoustic stimulus, to enable the automatic real-time annotation of sessions (in the tags.csv file).

The overall data acquisition lasted about 12 min (see Fig. 3): i) 5 min at resting condition where the physiological reference baseline was collected with the subject relaxed; ii) 2 min (from 05:30 to 07:00 min:s; minutes:seconds) where physiological changes were measured. In particular, sound stimuli were played at 05:30 min:s, 06:30 min:s, and 07:00 min:s through headphones to the subject; iii) 5 min again at resting condition.

Fig. 3.
figure 3

Schematic representation of the temporal structure of the auditory stimulation sessions presented to the participants.

Full size image

Audio clips, lasting 6 seconds per each, were extracted from the International Affective Digitized Sounds (IADS-2) database [33], that contains a collection of sounds rated in terms of valence (pleasantness), arousal (intensity of sensations) and dominance (control over sensations) according to the 9-point Self-Assessment Manikin (SAM) scale [7]. Based on the unpleasant valence score, the sounds listed in Table 2 were selected for the current study.

Table 2. Valence, arousal and dominance scores (dimensionless values, mean value ± standard deviation) of the selected sounds.
Full size table

4 Data Processing

The GSR signal acquired has to be first pre-processed in order to remove noise and possible motion artefacts. Electrical noise is reduced by applying a low-pass filter with a small cut-off frequency (generally <1 Hz), and similarly the motion artefacts with both a low pass filtering and manual inspection of the signal [32]. Once the GSR signal has been pre-processed, it can be decomposed in SCL and SCR components.

4.1 Synthethic GSR Signals

Based on the work by Jain et al. [26], and in order to evaluate the recovery accuracy of the proposed approach, the following parameters were defined:

  • impulse response vector \(\mathbf {h}\);

  • time steps \(\gamma \) and \(\delta \);

  • approximately sparse vectors \(\mathbf {x}\) \(\in \) \(\mathbf {X}_\delta ^{s}\) and \(\mathbf {b}\) \(\in \) \(\mathbf {B}_\gamma ^{c}\) with a certain number s of peaks and a number c of baseline jumps; \(\mathbf {x}\) was obtained by taking the s significant components uniformly at random, filling such components with a random vector (characterized by independent and identically exponentially distributed entries) and then, by adding a rescaled standard Gaussian random vector with \(l_{1}\) norm \(\delta \); \(\mathbf {Db}\) was computed by selecting the c significant components uniformly at random, filling such components with a standard Gaussian variable, and then by adding a rescaled standard Gaussian random vector with \(l_{1}\) norm \(\gamma \);

  • the noise \(\mathbf {n}\), as rescaled Gaussian random vector with \(l_{2}\) norm equal to \(\varepsilon \) = 0.01;

All these parameters allow to define:

$$\begin{aligned} \mathbf {D y}=\mathbf {DT_{h}x}+ \alpha \mathbf {Db} + \mathbf {n}, \end{aligned}$$
(1)

where \(\alpha \) is the scaling factor applied to \(\mathbf {Db}\) related to \(\mathbf {DT_{h}x}\) and \(\mathbf {T}\) a Toeplitz matrix constructed from the vector \(\mathbf {h}\).

5 Test Implementation and Results

To verify the feasibility and accuracy of the proposed CS-based approach, the experimental tests were preliminarily carried out on synthetic GSR signals, for which the amount of SCR peaks was defined a priori, and then on real GSR signals to acoustic stimuli of Table 2.

5.1 Preliminary Analysis on Synthetic GSR Signals

In a preliminary analysis the feasibility of CS framework for GSR signals was evaluated on synthetic GSR signals. In this test only the peak vector \(\mathbf {x}\) was reconstructed. The figure of merit used to analyze the reconstruction performance is the Recovery Error (RE) [2], defined as the norm of the difference between the original peak vector \(\mathbf {x}\) and the reconstructed peak vector \(\mathbf {\hat{x}}\), divided by the norm of the original peak vector. Thus, RE can be written as:

$$\begin{aligned} RE=\frac{\left\| \mathbf {x}-\hat{\mathbf {x}} \right\| _2}{\left\| \mathbf {x}\right\| _2}. \end{aligned}$$
(2)

Figure 4 illustrates the RE diagrams obtained for three different values of the scaling factor \(\alpha =~\{0.01,0.1,1\}\). The diagrams were computed depending on the number of SCR peaks chosen in the set \(s=~\{10,20,\dots ,240\}\) and the number of baseline jumps in the set \(c=~\{10,20,\dots ,350\}\). Specifically, 5 synthetic GSR signals were randomly generated for each value of peaks amount in the set s. RE was firstly computed for each synthetic GSR signal and, then, the obtained RE values were averaged. All the possible combinations between peaks and baseline jumps of average RE are reported as colored pixels. The three diagrams show that when scaling the magnitude of the baseline component, through a lower factor \(\alpha \), a lower average RE is obtained.

Fig. 4.
figure 4

Diagrams of RE depending on the number of GSR peaks s and baseline jumps c, for three values of the scaling factor \(\alpha \): a) 0.01, b) 0.1, c) 1.

Full size image
Fig. 5.
figure 5

Step-by-step reconstruction for subject 1.

Full size image
Fig. 6.
figure 6

Step-by-step reconstruction for subject 2.

Full size image

5.2 Test on Real GSR Signals

In the second test the entire phasic component of real GSR signals was recovered. In particular, the phasic component was estimated as the convolution between the reconstructed peak vector \(\mathbf {\hat{x}}\) and the impulse response vector \(\mathbf {h}\), obtained by sampling at 4 Samples/s the bi-exponential function:

$$\begin{aligned} f(u)= 2 (e^{-u/\tau _1}-e^{-u/\tau _2}) \end{aligned}$$
(3)

with \(u\in [0,40]\), \(\tau _1=10\) and \(\tau _2=1\).

Fig. 7.
figure 7

Step-by-step reconstruction for subject 3.

Full size image
Fig. 8.
figure 8

Step-by-step reconstruction for subject 4.

Full size image

Figures 5, 6, 7, 8, 9 and 10 show the reconstruction of the phasic component of the GSR signals acquired by 6 subjects, as described in Subsect. 3.2. The figures report all the intermediate signals step-by-step, till the final reconstruction phase. The colored vertical lines indicate the moments when the acoustic stimuli occurred. In particular, each figure illustrates: a) the acquired GSR signal \(\mathbf {y}\); b) the GSR signal filtered by the matrix \(\mathbf {D}\); c) the reconstructed peak vector \(\mathbf {\hat{x}}\); d) the reconstructed phasic component \(\mathbf {\hat{x}}*\mathbf {{h}}\). The obtained results exhibit a good reconstruction of GSR phasic component for all the 6 subjects.

Fig. 9.
figure 9

Step-by-step reconstruction for subject 5.

Full size image
Fig. 10.
figure 10

Step-by-step reconstruction for subject 6.

Full size image

6 Conclusion

In this work, a decomposition method based on the approach proposed by Jain et al. [26] has been employed, to reconstruct GSR signals through CS framework. In particular, the reconstruction was preliminarly evaluated on peaks of GSR signals that were synthetically generated, in order to verify the feasibility of the decomposition method and its reliability in peaks reconstruction. Subsequently, the reconstruction was tested on real signals, experimentally acquired with a wearable device from 6 subjects, in response to unpleasant acoustic stimuli. In this case, the entire phasic component of the GSR signal was reconstructed. From the obtained results, a good reconstruction of GSR phasic component can be observed. The proposed method could be exploited to analyse the reaction of subjects exposed to unpleasant sounds in the long-term, as it may be the case for workers in constructions or harsh industrial environments.

Future investigations will be devoted to recover the whole GSR signal, including the tonic component, that is determined by several individual factors (e.g. gender and age), irrespective of any stimulus. Besides, future studies will be performed on a bigger set of signals, by involving more subjects as well as other devices for GSR acquisition.