1 Introduction

Air travel has become the preferred mode of long-distance transportation. Approximately, 1.5–2 billion passengers travel by civil airlines each year, and this number is expected to double in the next two decades as larger aircraft and more routes come on line [1]. Taking the demographic trend of an aging population into consideration, the likelihood of an older passenger population increases and thus the airplane will soon be part of our environment in which people may need assistive support [1–3].

The European SEAT-project (http://www.seat-project.org) addresses this issue by extending existing airplane seats with new technologies in order to assess and to improve the comfort of the passengers. The seat is chosen because it is the main interaction point of a passenger with the airplane. Besides the health monitoring for elderly people, the project tries to provide robust and reliable indicators for the well-being (e.g. stress or fear of flight) of a flight passenger. This offers the opportunity for a more appropriate support by the stewardess or for personalized feedback implemented in the entertainment system [4]. A first step in this direction is the reliable recording of the relevant physiological signals. Therefore, a hardware setup is proposed that records the Electrocardiogram (ECG), the Electrodermal Activity (EDA), the respiration and the skin temperature of a person sitting in an airplane seat. To achieve the acceptance and hence the use of the system, the sensors need to be attached in a non-obtrusive way or even be totally integrated into the seat. Thus, common body worn systems (e.g. with electrodes attached to the chest) are not feasible in this context. However, a comfort-optimized sensor placement limits the signal quality. Like in every system for personal healthcare settings, motion artifacts are a common issue [5, 6]. Therefore, the decrease of signal quality due to movement artifacts was taken into account throughout the whole design of the system and a suitable method for artifact detection was developed.

The article is structured as follows: Sect. 2 gives a brief overview about existing work addressing physiological monitoring in a seat environment. Afterwards, relevant work in the field of artifact detection for real-life applications is presented. Section 3 presents the developed physiological measurement system, that is unobtrusively integrated into an airplane seat. Beside the physiological sensors, the system incorporates additional sensors in order to perform an automatic artifact detection through direct sensor fusion. The method used for the automatic artifact spotting and the validation of this approach is presented in Sect. 4. To evaluate this concept, Sect. 5 shows the results of a small experiment as a proof of concept. Different physiological signals were measured during a representative set of airplane passenger activities. In addition to the seat integrated sensors, a body worn system was used to obtain a ground truth measurement of the physiological signals. This measurement enabled the calculation of a Quality Index in order to quantify the performance of our artifact detection method. Sect. 6 discusses the overall work and proposes further steps.

2 Related work

2.1 Physiological monitoring in a seat environment

We present two examples exploiting a seat environment for automatic prediction of stress and frustration respectively. Healey et al. [7] showed the possibility to detect stress during real-world driving tasks using physiological sensors. Their analysis revealed that for most of the drivers studied, the EDA and ECG were most closely correlated with driver stress level. Not a car seat but an office seat was used by Kapoor et al. [8] in order to predict frustration during learning processes. As sensor modalities, they used a camera, a pressure-sensitive mouse, an EDA sensor, and a pressure-sensitive chair. With these sensors, they achieved an automatic prediction of frustration with 79% accuracy.

Concerning the integration of physiological monitoring in a seat, the contactless measurement of the heart activity represents a promising approach. Three different research activities in this direction could be identified. Lim et al. and Steffen et al. both designed a system for measuring the ECG of a sitting person capacitively [9, 10]. Another method is shown by Zakrezwski et al. [11] and Suzuki et al. [12]. They present the possibility of detecting the heart rate using a radar system. A further approach is proposed by Junnila et al. [13]. They use sensors of electromechanical film (EMFi) integrated into a seat pan. The heart rate measurement with this sensor is based on ballistocardiography (BCG), that measures the recoil that spreads through the body as a result of a heartbeat [14]. All three measurement principles are not only unobtrusive but also sensitive to artifacts evoked through movements.

2.2 Dealing with artifacts

The conflict to reach good signal quality while enabling acceptable comfort levels for the user is highlighted by Such et al. in [6]. The authors point out that an indication of the signal confidence would often be preferred over an automatic compensation algorithm. The automatic compensation can lead to plausible but incorrect features and, therefore, to a false sense of safety. Concerning automatic artifact detection, two different approaches are distinguished. The single parameter approach only uses the characteristics of the corrupted sensor signal itself to detect the artifacts while the multiparameter approach uses additional sensor channels to identify these artifacts. An example of the single parameter approach is given by Such et al. [5]. They present an algorithm that segments the signal into “artifact-free” intervals before performing a signal analysis. An example for an artifact compensation based on the multiparameter approach is presented in different publications [15–17]. The authors use a MEMS accelerometer to recover signals of a photoplethysmograph sensor corrupted by body motion. All three research activities focus on a wearable photoplethysmograph sensor. In [18], the authors show the effect of movements on the electrodermal response. In accordance to [6], they suggest to measure the signal confidence in order to describe the artifacts.

3 Sensor system in an airplane seat

In order to detect the well-being of a passenger, both the mental and the health state have to be addressed. The ECG, EDA, respiration and skin temperature have proven to provide important features concerning these two states [7, 19–22]. Therefore, these signals are chosen to be monitored in this work. When measuring these signals unobtrusively in a “real-life scenario” we face the trade off between sensor placement and signal quality. Conventional sensors and measurement locations, such as wet electrodes at the chest for cardiovascular monitoring, are not feasible due to the lack of acceptance by the passengers. Therefore, we integrated all sensors unobtrusively into an airplane seat (see Fig. 1). Because of the non-optimal measurement locations and the non-controllable environment, a main concern of the system design was to make sure that signal disturbances can be detected. Thus, additional sensors for measuring the movements of the fingers and the contact pressure at the back rest were integrated into the setup (see Fig. 1). These additional sensors, referred to as artifact sensors, are able to indicate movement patterns that provoke artifacts, which influence the signal quality of the physiological signals.

Fig. 1
figure 1

Airplane seat with integrated unobtrusive sensors

Full size image

Besides the comfort, the spatial requirement of the sensors was an important aspect throughout the development of the measurement setup. Therefore, the sensors were designed so that the natural environment of the airplane seat is exploited and no additional space is required. Regarding the required computational power for the data acquisition the overall system is designed in such a way that the computation can be performed by the in-flight entertainment system (1 GHz CPU, 500 MB RAM). Such a state-of-the-art system is installed in the airplane seats targeted at the SEAT project (http://www.thales-ifs.com).

This section briefly introduces the measurement concepts for the physiological signals and the above mentioned artifact sensors. It concludes with an exemplary presentation of the recorded signals.

3.1 Measuring physiological signals

For the ECG measurement, two different systems were integrated into the seat. They provide a different level of comfort and hence the expected signal quality also differs. One system, referred to as “Contactless-ECG”, is a research prototype developed by the RWTH Aachen [10, 23]. It measures the ECG capacitively without direct skin contact. We integrated it into the backrest of the airplane seat and connected it to the overall setup by a self-developed analog/digital conversion board. As this ECG system is unobtrusively integrated into the seat, it does not disturb the user, but it is sensitive to body movements and is only capable of measuring the ECG while the passenger is leaning back.

The other ECG system, referred to as “Finger-ECG”, measures the ECG at the index finger of both hands. The system uses dry electrodes that are fixed to the fingers with small strips as indicated in Fig. 2. The electrodes can easily and comfortably be attached and detached from the fingers and therefore provide a higher user comfort compared to wet electrodes attached to the chest. Due to the direct skin contact, this system is more obtrusive than the “Contactless-ECG” but also more reliable.

Fig. 2
figure 2

Finger-Strip with dry electrode, skin temperature sensor and accelerometer

Full size image

Similar to the Finger-ECG, the measurement of the EDA needs direct skin contact. The EDA is recorded by measuring the conductivity of the skin. As proposed in the literature, the medial and distal phalanges of the index and middle fingers are taken as the measurement position for the EDA [24]. The implemented measurement principle is referred to as an exosomatic quasi constant voltage method [25]. Hereby, a constant voltage (500 mV) is applied to the electrode at the index finger, leading to a current flowing through the skin to the other electrode. This current is measured and thereby the skin conductance can be assessed.

In order to reduce the number of electrodes and thus increase the comfort while using both the Finger-ECG and the EDA measurement, a novel concept for combining the measurement of the EDA and ECG at the fingers was developed. The electrode at the index finger of the left hand is used for both measurements, ECG and EDA, and thus the amount of electrodes decreases from four to three (see Fig. 3). This raised the need of special design considerations for the ECG measurement, in order to handle the high offset voltage at the left index finger, caused by the EDA measurement.

Fig. 3
figure 3

Electrode placement at the finger for measuring ECG, EDA and skin temperature

Full size image

For the measurement of the respiration, a special characteristic of the airplane seat, the safety belt, is exploited. In contrast to common methods with a belt around the chest, the respiration sensor is directly integrated into the safety belt of the airplane seat (see Fig. 4). A strain sensitive resistor [26] measures the mechanical expansion of the seat belt around the waist. As the inhalation and exhalation yield a deformation of the waist, the respiration is directly correlated with the expansion of the seat belt.

Fig. 4
figure 4

Safety belt with integrated textile strain sensor for respiration measurement

Full size image

As a skin temperature sensor, the commercially available TSic™-506F was chosen. Due to the small size (5mm × 5mm × 1mm) of the sensor, it can be fixed beside the left electrode of the EDA measurement and does therefore not further decrease the perceived comfort (see Fig. 2). As this sensor is not sensitive to movement artifacts, it is not further evaluated in this work.

3.2 Measuring artifact signals

Movements of the fingers and the hands, respectively, evoke artifacts on the EDA and the ECG signals measured at the fingers. Main reasons are the shifting of the electrodes and the stretching of the skin beneath the electrodes. To spot these finger movements, the finger stripes for both index fingers are equipped with 3-axis accelerometers (see Fig. 2). The detection of artifacts evoked by movements can be achieved by an overall indication of the finger movements. Therefore, the three dimensions of both accelerometers were combined to a single artifact signal, referred to as FingerMovement. It is calculated by the following formula

$$ FingerMovement = \sum_{n}\sum_{d}Acc_{nd} , $$

with n being the index for the left and right sensor and d being the index for the three dimensions x, y, and z.

For the Contactless-ECG and the respiration signals, contact pressure at the back rest and general movements of the upper part of the body lead to artifacts. Therefore, four pressure sensors were integrated behind the electrodes of the Contactless-ECG system, one at the top and one at the bottom of each electrode. The Contactless-ECG system only delivers good signal quality if a good and constant contact pressure exists between the upper part of the body and the back of the seat [23]. As this information can be extracted from the pressure sensors, they are supposed to deliver meaningful information about the signal quality of the Contactless-ECG. For the respiration signal, the pressure sensors also deliver artifact information as they can indicate a leaning forward or backward of a person sitting in the seat. Such movements change the stretching of the respiration sensor without being related to inhalation or exhalation processes. Similar to the combined FingerMovement signal, the four pressure signals are combined to one signal, the ContactPressure

$$ ContactPressure = \sum_{n}\sum_{i}P_{ni}, $$

with n being the index for the left and right sensor and i being the index for the upper and lower sensor.

3.3 Signal examples

In Fig. 5, an example of the Finger-ECG signal and the corresponding artifact signal (FingerMovement) is shown. The example shows the correlation between the FingerMovement signal and the signal quality of the ECG. Regions of low movement deliver a good signal quality, whereas in the regions of high finger movement, the signal is clearly corrupted and cannot be used to extract ECG features such as the QRS-complexes.

Fig. 5
figure 5

Example of movement artifacts influencing the Finger-ECG signal

Full size image

In the same way, Fig. 6 shows an extraction of the signal recorded by the Contactless-ECG and the ContactPressure signal. It can be seen that the fast change in the ContactPressure due to movements of the upper body yields a temporary disturbance of the ECG signal, which renders a detection of the QRS complexes impossible.

Fig. 6
figure 6

Example of movement artifacts influencing the Contactless-ECG signal

Full size image

The signal recorded by the respiration sensor is depicted in Fig. 7. The periodic peaks reflect the expansion of the seat belt due to inhalation and exhalation. It can be seen that the ContactPressure signal indicates significant disturbances which have to be addressed, when using the respiration signal for physiological monitoring.

Fig. 7
figure 7

Example of movement artifacts influencing the respiration signal

Full size image

A strong artifact on the EDA signal can be seen in Fig. 8. As the EDA is measured at the left hand, only the sum of the three axes of the left accelerometer are used as artifact signal. The movements of the fingers corrupt the signal such that no analysis based on the EDA can be done for this region. We have already shown a systematic analysis of the effect of movements on the EDA in [18]. Therefore, the further evaluation focuses on both ECG and the respiration signals.

Fig. 8
figure 8

Example of movement artifacts occurring on the EDA signal

Full size image

4 Signal appraisal through sensor fusion

Artifacts can lead to erroneous features extracted from a signal and thereby lead to a wrong interpretation of the signal. The most important step when dealing with signals disturbed by artifacts is the detection of those artifacts. The introduced system offers the possibility to detect artifacts with a multiparameter approach. Fig. 9 depicts the working principle of the senor fusion. The artifact sensors are used to spot regions of artifacts and remove the features extracted in these regions. In terms of computational complexity this kind of direct sensor fusion can be performed by the in-flight entertainment system.

Fig. 9
figure 9

Artifact removal through sensor fusion

Full size image

As can be seen exemplarily in Fig. 10, fast movements of the fingers evoke artifacts on the Finger-ECG. For an automatic spotting of these regions, appropriate features have to be calculated from the artifact signals. For the Finger-ECG, the derivative of the lowpassed filtered FingerMovement signal is chosen as artifact feature (see Fig. 10).

Fig. 10
figure 10

Artifact feature extracted from the FingerMovement signal

Full size image

The derivate indicates the incline in the finger acceleration and when it exceeds a certain threshold, the temporal period contains high finger movements. This area is going to be marked as corrupted by artifacts.

For the artifact detection on the Contactless-ECG signal, the contact pressure between the body and the backrest is of high importance (see Fig. 6). If no contact or fast changes due to movements of the upper body signal are detected, the signal is marked as corrupted. Similar to the FingerMovement, the derivative of the lowpassed filtered ContactPressure signal is calculated as an indicator for those fast changes.

For the respiration signal, body movements resulting in fast changes in the ContactPressure are expected to evoke artifacts (see Fig. 7). Therefore, the respiration signal is marked as artifact-affected when fast changes in the ContactPressure are detected. In the same way as for the Contactless-ECG, the fast changes are indicated by the derivative of the lowpass filtered ContactPressure signal.

In order to increase the stability of the artifact detection, two post processing steps were performed. On the one hand, artifact regions that are separated by small non-artifact spots were merged together and on the other hand single isolated artifact spots were discarded and not taken into consideration for further purposes.

4.1 Definition of a quality index

As indicated by Such et al. [6], the compensation of artifacts by, e.g. adaptive filters or decorrelation methods can lead to a false sense of safety and has to be done carefully. Hence, instead of compensation, we propose the removal of the affected signal parts. To validate this approach, we define a Quality Index (QI) that describes the correctness of characteristic features extracted from the ECG and respiration signals. The calculation of the QI is based on the ANSI/AAMI Norm [27]. The norm defines the appraisal of the quality of a peak detection algorithm. We adopted this procedure to our purposes and analogously appraise our signals based on the ability to identify characteristic features. We therefore compare a characteristic feature extracted from the signal to be observed with the same feature extracted from a simultaneously recorded ground truth signal. When comparing the QI of the entire signal to be observed, with the QI of the parts marked as artifact free, the difference indicates the ability to detect regions destroyed by artifacts.

From the ECG, the R-peaks are taken as feature to calculate the QI. For the extraction of this feature, an algorithm proposed by Hamilton and Tompkins [28] is used. For the respiration signal, the peak evoked by the inhalation is the characteristic feature for the QI. These inhalation peaks are detected with a peak detection algorithm proposed by [29].

Figure 11 shows exemplarily with an artificial ECG signal how the feature comparison for the calculation of the QI is done. The QI of the depicted ECG signal shall be evaluated.

Fig. 11
figure 11

Example showing the calculation of the Quality Index. Eight out of ten peaks were correctly identified, therefore the QI for this signal part results in \({\frac{8}{10}}=80\%\)

Full size image

The stars in the boxes at the bottom indicate the timestamps where R-peaks were detected in the ground truth signal. Correct hits (True Positives), a missing peak (False Negative) and an erroneously detected nonevent (False Positive) are shown. According to the ANSI/AAMI Norm, the interval X in which a peak is still counted as a True Positive is chosen to be 150 ms. More than one peak inside the interval X or a peak outside the interval results in a False Positive. Both kind of errors, False Negatives and False Positives, are false detected R-peaks and therefore both count as false R-peak. The Quality Index for the ECG signals is chosen to be the ratio of correctly detected R-peaks divided by the total amount of R-peaks detected in the ground truth signal. For the example shown in Fig. 11, 8 out of 10 peaks were correctly identified, therefore the described procedure results in a QI of 80%.

For the respiration, the Quality Index is calculated analogously with respect to the detected respiration cycles. Since a lower temporal resolution is sufficient for the respiration, the interval X was chosen to be half the distance of two consecutive ground truth events in both directions.

5 Proof of concept

This section shows the results of a feasibility study with the developed sensors and implemented artifact detection algorithms. A small experiment with a person performing typical passenger activities was conducted. During these activities, all the described signals were measured and the Quality Index before and after the artifact removal was calculated. For every activity, the difference in the QI indicates the performance of the artifact removal. In order to calculate the QI, the commercial Mobi device (http://www.tmsi.com) was used to measure the ground truth for the ECG and the respiration. It measures the ECG with wet electrodes attached to the chest and the respiration with a belt directly attached to the upper part of the chest. In addition to the QI, the amount of remaining data after the artifact removal was calculated.

5.1 Experiment description

Since this works targets an airplane environment, typical passenger activities such as being entertained, working, reading, sleeping and eating were addressed, as depicted in Fig. 12. The choice of activities was based on the statistical distribution of passengers’ activities during a long term flight, as shown in Table 1 [30]. One subject was sitting in the airplane seat performing each activity for 10 min. This resulted in a total of 50 min of data. During the activity “Entertainment”, the subject watched a relaxing movie on a laptop placed on the front-table. For the activity “Working”, the subject had to perform normal office tasks, like writing a text and editing a table. For the next activity “Reading”, the subject was reading a journal and during “Sleeping” the subject was asked to lean back, close the eyes and relax. During the last activity “Eating”, the subject ate a piece of cake and drunk some juice. For the evaluation of the results, the activities are divided into two classes, the active class consisting of “Working”, “Reading” and “Eating” and the calm class consisting of “Entertainment” and “Sleeping”.

Fig. 12
figure 12

Typical passenger activities during a long term flight

Full size image
Table 1 Temporal distribution of passengers activities during a long term flight
Full size table

5.2 Results for the Contactless-ECG

The results for the Contactless-ECG are depicted in Table 2. The Quality Index of the calm class (Entertainment: 71% and Sleeping: 63%) differs substantially from the more active class (Working: 16%, Reading: 32% and Eating: 17%). These results reflect the constraint of the system: the ECG can only be measured when sitting with constant back contact. The fact that the QI, before artifact removal, for the activities “Working” and “Eating” still exceeds 15%, can be explained by the nature of the R-peak detection algorithm, that also extracts features in pure noise data. This leads to randomly correct hits even though the signal was totally disturbed by artifacts. When analyzing the results achieved after the artifact removal, it can be seen that the signal for these activities is almost totally discarded. For the activity “Entertainment” and “Sleeping”, also a high percentage of the signal (approx. 60%) is eliminated. However, for the remaining signal, the QI raises by 23 and 53%, respectively. Taking all the activities together, almost 84% of the data are removed, but the mean QI for the remaining data increased from 40 to 88%

Table 2 Contactless-ECG results
Full size table

5.3 Results for the Finger-ECG

Table 3 shows the results for the Finger-ECG. It can be seen, that except for the activity “Eating” the QI increases due to the artifact removal. During eating, a high amount of finger movement was detected and almost the whole signal was discarded. The resulting QI for this activity is thus not reliable. For the calm classes, less than 50% of the signal is discarded and an overall high QI is achieved. The unexpected high QI for the activity “Working” is probably related to the relatively constant hand position with only short finger movements for typing on the keyboard. Considering the mean value of all activities, the QI increased from 75 to 88%.

Table 3 Finger-ECG Results
Full size table

5.4 Results for the respiration

In Table 4, the results for the respiration signal are depicted. For every activity, the artifact removal led to an increase in the Quality Index. Analyzing the results for the respiration signal shows that about 32% of the data was removed but an increase of the QI from 76 to 82% was achieved. Furthermore, the results show no general pattern, neither for the active classes nor for the calm ones. There exists a general decrease of the QI during the experiment. This decrease can probably be explained by a loosening of the saftey belt during the course of the experiment which led to a decrease in the signal quality.

Table 4 Respiration Results
Full size table

6 Discussion

We identified the airplane as part of our environment in which assistive support would be appreciated. To address this issue, we have incorporated sensor technology into an airplane seat with the goal to unobtrusively measure physiological signals in order to enable the modelling of the well-being of a passenger. The trade-off between comfort and signal quality was identified to be an important issue for every system supporting persons in their daily life environment. Therefore, a sensor fusion of physiological sensors with artifact sensors for an automatic detection of artifacts for physiological signals was introduced. For validation purposes, we proposed a Quality Index that appraises the signal of interest based on a ground truth signal. In a feasibility study, we recorded the signals delivered from the smart airplane seat during typical passenger activities of a long term flight. During this study, we compared two different ECG measurement systems: the Contactless-ECG and the Finger-ECG. The Contactless-ECG is completely unobtrusive at the expense of signal quality, while the Finger-ECG is less prone to artifacts but more obtrusive. For all activities, an initial mean Quality Index of 40% was achieved by the Contactless-ECG, whereas the Finger-ECG reached 75%. These numbers show the importance of using artifact detection algorithms in real-life settings. By performing the described artifact removal based on sensor fusion, we identified signal regions in which both ECG systems achieved a Quality Index of 88%. Taking the remaining signal length into account, the Finger-ECG clearly outperforms the Contactless-ECG: 84% of the data was discarded for the Contactless-ECG while only 55% was removed for the Finger-ECG. However, it is worth stating that not all activities occur with the same frequency during a long term flight. According to Table 1 in Sect. 5, passengers are calm (i.e. sleeping or being entertained) during more than 60% of the flight duration. If the remaining signal length found in our study is scaled up to a 12 h flight, 2.8 h of the Contactless-ECG data and 6.8h of the Finger-ECG data is expected to be almost artifact free and useful for analysis.

Regarding the respiration, the results show that the respiration measurement is not as sensitive to artifacts as the ECG measurement. When upscaling the results for the respiration, a total amount of 8.5 h of good quality for the respiration signal can be expected during a long-term flight.

7 Outlook

As a possible alternative to the Finger-ECG the integration of the electrodes into thrombosis socks could be evaluated. In a feasibility study, we integrated the sensors into thrombosis socks and we were able to measure both the ECG and the EDA. This electrode placement is expected to be less obtrusive to the passenger as no wired connection at the hands would be needed. Even though the signal to noise ratio of the ECG measured at the feet is supposed to be smaller than measured at the finger, this electrode placement may suffer less from movement artifacts and lead therefore to an overall good QI.

A further step towards the goal of the SEAT-project is the implementation of a human interaction loop: if a low QI is measured over a long time period, the passenger could be asked to adopt a posture which enables a measurement with a high signal quality.