Abstract
Background
Polysomnography systems used for the diagnosis of sleep respiratory disorders are comprised of multiple sensors, including abdomen and thorax respiratory inductance plethysmography (RIP) belts to record respiratory effort. However, RIP belts are known to be susceptible to signal loss. To resolve this, we utilized a contactless three-dimensional (3D) time-of-flight (TOF) camera to monitor respiratory effort.
Objective
We aimed to show that respiratory effort monitoring can be achieved by 3D TOF camera recording instead of RIP belts.
Materials and methods
The use of RIP belt signals is twofold. Firstly, the signals are used to classify the apnea events into obstructive, central, and mixed. Additionally, the American Academy of Sleep Medicine (AASM) Scoring Manual recommends the scoring of apneas and hypopneas using RIPSum (the sum of the abdomen and thorax RIP signals) when the airflow sensors are unavailable. We therefore used the 3D effort signal to classify apneas and compared it to the RIP signal classification. Reduced effort events from RIP and 3D effort signals were compared to the apnea and hypopnea events. Furthermore, the changes in effort during the events were compared between the two effort signals.
Results
Classification by 3D effort signal performed well, with 80% accuracy. It worked best for central apneas, with an accuracy of 99%. There was a high correlation of r = 0.88 (r ≠ 0, p = 0.0001) between the 3D effort signal events and RIPSum events. There was also a significant correlation of 0.62 (r ≠ 0, p = 0.0001) between 3D effort signal and RIPSum in the decrease of effort during apnea and hypopnea events.
Conclusion
We conclude that respiratory effort derived from a 3D TOF camera can be used as an alternative to RIP belts for scoring of apneas and hypopneas and classification of apneas.
Zusammenfassung
Hintergrund
Die Polysomnographie für die Diagnose von schlafbezogenen Atemstörungen verlangt die Montage von vielen Sensoren am Patienten, u. a. für die respiratorische Induktionsplethysmographie (RIP) zur Messung der Atemanstrengung. Die verwendeten Gurte sind anfällig für Signalstörungen. Die Autoren verwendeten daher eine kontaktlose 3‑D-Time-of-Flight(TOF)-Kamera, um die Atmungsanstrengung zu messen.
Zielsetzung
Ziel der Studie war es zu zeigen, dass die Atemanstrengung mit einer 3‑D-TOF-Kamera anstelle der üblichen RIP gemessen werden kann.
Material und Methoden
Die Signale der RIP dienen 2 Anwendungen: Sie werden verwendet, um Apnoen als obstruktiv, zentral oder gemischt zu klassifizieren. Des Weiteren erlauben die Regeln der American Academy of Sleep Medicine (AASM) die Detektion von Apnoen und Hypopnoen unter Verwendung des RIP-Sum-Signals (Summe des Abdomen- und Thorax-RIP-Signals), falls kein Atemflusssignal verfügbar ist. Die Autoren verglichen die Klassifikation von Apnoen mit RIP und der dreidimensional erfassten Atemanstrengung („3D effort“). Episoden mit reduzierter Atemanstrengung – basierend auf den beiden Sensoren – wurden mit den Apnoe- und Hypopnoe-Ereignissen verglichen. Des Weiteren verglichen die Autoren die Änderung der beiden „Effort-Signale“ während dieser Ereignisse.
Ergebnisse
Die Klassifikation mit dem 3‑D-Effort-Signal erreichte eine Genauigkeit von 80 %, für zentrale Apnoen bestand sogar eine Genauigkeit von 99 %. Es gab eine starke Korrelation r = 0,88 (r ≠ 0; p = 0,0001) zwischen 3‑D-Effort-Ereignissen und RIP-Sum-Ereignissen. Die Signalabschwächung während Apnoen und Hypopnoen zwischen den beiden Effort-Signalen war auch signifikant korreliert mit r = 0,62 (r ≠ 0; p = 0,0001).
Schlussfolgerung
Aufgrund des ähnlichen Verhaltens der beiden Effort-Signale kann eine 3‑D-TOF-Kamera als Alternative zur RIP für die Detektion und Klassifikation von Apnoen verwendet werden.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
The diagnosis of sleep apnea syndrome is performed in sleep clinics by a full-night polysomnography (PSG) recording. A PSG system comprises multiple sensors, including two respiratory inductance plethysmography (RIP) belts over the thorax and abdomen to measure the respiratory effort. According to the American Academy of Sleep Medicine (AASM) Scoring Manual v2.4, the respiratory effort signals measured by the two belts are to be used to determine the apnea types. The signals are inspected for continuous or increased breathing effort (obstructive), absence of breathing effort (central), or a combination of both (mixed). Additionally, when airflow signals from the primary sensors, such as nasal pressure cannula or nasal thermistor, are not available or are unreliable, the RIPSum—the sum of the thorax and abdomen RIP belt signals—can be used for detection of respiratory events [1]. RIPSum is listed as an alternative because there is an absence or a reduction in respiratory effort during apneas and hypopneas [2,3,4].
The drawback of RIP belts is that they are attached to the body and connected by cables. These belts and all the other sensors included in a standard PSG are uncomfortable and may contribute to the first night effect (FNE). FNE is a term used to describe loss of sleep quality during PSG recordings [5]. Also, the signals from respiratory belts are often distorted and more susceptible to signal loss when patients are moving [2, 6]. In line with this, alternative sensors have been introduced to measure respiratory effort for the detection of apnea syndrome, e. g., two Doppler radars were placed under the mattress to measure vibrations to detect apnea and hypopnea [7]. With Doppler radars, 94% correlation was achieved with PSG scoring for the respiratory disturbance index (RDI). RDI reports respiratory events during sleep, including respiratory effort-related arousals. The Sonomat system (Sonomat Pty Ltd, Balmain, NSW, Australia), on the other hand, uses four electrodes in the mattress foam in locations near the abdomen and thorax [8]. The validation study of the Sonomat system reported a correlation with PSG of higher than 0.89 for the apnea–hypopnea index (AHI). Another alternative system is the SleepMinderTM (Biancamed Ltd, Dublin, Ireland), using an ultra-low-power radiofrequency transceiver to record respiration [9]. The AHI derived from SleepMinder was shown to have a 91% correlation with PSG scoring.
Another alternative sensor to monitor respiration during sleep is a contactless three-dimensional (3D) time-of-flight (TOF) camera. A 3D TOF camera measures the distance of objects in its field of view with a given spatial resolution and frame rate. The use of 3D cameras for medical applications is not entirely new; previous investigations have been performed using depth cameras to monitor body movements [10,11,12,13]. To our knowledge, depth cameras for the investigation of respiratory events were first introduced in by Falie and Ichim [14], but no test results from overnight sleep recordings were provided.
We used a respiratory effort signal derived from a 3D TOF camera to monitor respiration during sleep. With the camera recording for 30 frames/s, it captured the depth changes following the movement of the abdomen and thorax from a person’s breathing. We investigated whether this respiratory effort signal derived from a 3D camera is comparable to RIP belt signals. We performed the following comparisons between the two respiratory effort signals according to how RIP belts are used in the AASM Scoring Manual:
-
absence or presence of effort for the classification of apneas into obstructive, central, or mixed
-
decrease in effort for the scoring of apnea or hypopnea events when primary sensors are unavailable.
Materials and methods
Study design and investigation methods
In this study, 52 full-night PSG and 3D camera recordings of patients with suspected sleep apnea syndrome from the Advanced Sleep Research GmbH (Berlin Clinic) and Kepler University Clinic (Linz Clinic) were included. The median age of the patients was 57 years, the youngest 25 and the oldest 80 years. There were 20 female patients. Median body mass index (BMI) was 28.5 kg/m2 (range 21.8–45.4 kg/m2). This study was approved by the ethics committee of the state of Upper Austria (B-130-17) and the ethics committee of the Charité—Universitätsmedizin Berlin (EA1/127/16).
The 3D camera (KINECT V2; Microsoft, Redmond, WA, USA) setup was integrated with the PSG to synchronize recording time. The camera was mounted above the bed so that it captured the whole bed and the patient.
Apnea and hypopnea events
Overall, 639 apnea and 2235 hypopnea events were manually scored according to the AASM Scoring Manual v2.4 and using the 1B rule for hypopneas. The scoring task was carried out in our laboratory, with consultations with sleep experts from the Medical University of Vienna. These annotations served as ground truth for our investigations. From the annotations, there are 7 recordings with healthy AHI (<5), 15 recordings with mildly disturbed AHI (5–15), 9 recordings with moderately disturbed AHI (15–30), and 21 with severely disturbed AHI (>30). The severity category of AHI was based on the recommendation of the AASM Task Force [15]. The AHI was computed using the total sleep time from the PSG’s hypnogram.
As we are only concerned with apnea and hypopnea events, when respiratory events are mentioned in this paper, it only pertains to these two unless stated otherwise.
RespEffort3D
RespEffort3D is the respiratory effort signal derived from the depth signal of the 3D camera. The relevant pixels over the thorax and abdomen region were selected automatically and the depth signals of the selected pixels were then averaged per frame to create the respiratory effort signal.
Classification of apnea types
The respiratory effort signals monitored by the abdomen and thorax RIP belts were used to classify apnea into three categories: obstructive, central, and mixed. We show that the RespEffort3D signal can be used to classify the events, with the classification by RIP belts serving as the ground truth. The RespEffort3D and the RIP belts signals were visually inspected to classify the apnea events. Accuracy and per-type classification rate were computed, wherein classification by RIP belts served as ground truth. In total, 618 apnea events were tested, whereas the remaining 21 apnea events were excluded due to unreliable effort signals due to body movements.
Effort signals during respiratory events
According to the AASM Scoring Manual, RIPSum can be used as an alternative sensor when the primary sensors are not working or unreliable. RIPSum typically exhibits a decrease in signal during an apnea and hypopnea, which is the basis for scoring. In hypopneas, a 50% decrease in thoracoabdominal movements is usually observed and the decrease in apneas is usually larger [3, 16]. Consequently, we expect RespEffort3D to do the same, so that it can be used to score events, too. In order to compare RespEffort3D to RIPSum, we computed the size of the decrease in effort before and during the event for both signals. This is achieved by calculating a feature we call the effort reduction factor (ERF). ERF is the ratio of the effort during the event compared to the effort before the event. It is calculated for every apnea and hypopnea by the following steps:
-
1.
Create an event segment of the respiratory effort during an apnea or hypopnea episode.
-
2.
Create a baseline segment from the 15 s segment right before the event. If an event occurrs within the 15 s segment, the baseline segment is shortened from the end of the earlier event to the start of the current event. The 15 s duration was chosen as we found this to be the optimal segment length to estimate the pre-event baseline.
-
3.
For each segment, calculate the effort strength from the average peak-to-peak amplitude of the effort.
-
4.
Calculate the ERF from the ratio of the event effort strength to the baseline strength.
As such, an ERF value greater than 1 means there is an increase in effort during the event segment compared to the baseline segment, while a value of less than 1 means a decrease in effort during the event. The ERFs calculated for both RespEffort3D and RIPSum for all respiratory events excluding events with unreliable effort signal are tested for correlation using the Pearson’s correlation coefficient (Pearson’s r). The significance of the correlation is evaluated for p = 0.0001.
To assess the ability of scoring by RespEffort3D, a simple algorithm to find events of reduced effort in RIPSum and RespEffort3D was used. These reduced-effort events are candidate respiratory events, as further rules are needed according to the AASM manual. The reduced efforts from the two sensors were compared against each other. They were also compared to the annotated respiratory events. Pearson’s correlation coefficient was used to measure the correlation.
Results and discussion
This study is based on 52 full-night recordings including PSG and 3D camera signals. For each PSG recording, manual scoring was performed according to the AASM Scoring Manual v2.4. RespEffort3D was calculated for all recordings. RespEffort3D, as shown in Fig. 1, is similar to the RIPSum, nasal thermistor, and nasal pressure cannula signals from standard PSG during and between respiratory events, as it measures the respiration and reduction in respiration effort.
Classification of apneas by RespEffort3D
A total of 618 apneas were classified by the thorax and abdomen RIP belts by visual inspection using the AASM rules, serving as the ground truth. The same set of apneas were classified by visual inspection using RespEffort3D. The results of these classification by RespEffort3D are shown in Table 1, with 80% of the apneas classified correctly by RespEffort3D. Breaking this down into the individual types, RespEffort3D performed very well in classifying central apneas, with only 1% error. An example of RespEffort3D showing an absence of respiratory effort during a central apnea event can be found in Fig. 2. For obstructive apneas, 79% were classified correctly. During an obstructive apnea event, a decrease in respiratory effort in RespEffort3D is commonly observed (see Fig. 3).
Of the 87 misclassified obstructive apneas, 83 were classified as central and the remaining as mixed. Of the 32 misclassified mixed apneas, 28 were classified as central apneas and 4 as obstructive. This misclassification of obstructive and mixed apneas as central only happened in 14 recordings, and in three of these, misclassification is more prevalent. However, no distinguishable pattern can be seen among these recordings with respect to the patients’ BMI or their body position during sleep. One explanation for no effort seen in RespEffort3D for obstructive and mixed apnea is paradoxical breathing, which is the out-of-phase movement of the abdomen and thorax [4]. In order to improve the classification, it might be worthwhile to separate the 3D-derived effort signal into abdomen and thorax parts in future work.
In contrast to other classification studies, 90% test accuracy was achieved by using the thorax effort signal to train a neural network model [17]. A 73.1% classification accuracy was reported in one study using midsagittal jaw motion [18]. In the current study, no algorithm was yet developed for apnea classification using RespEffort3D, but the visual inspection method already provided information on the capability of RespEffort3D, where 80% is a good indicator. As such, in order to improve accuracy, separating the RespEffort3D into its abdomen and thorax parts is important for better classification of mixed and obstructive events.
Effort signals during respiratory events
The ERFs of all apnea and hypopnea events were calculated for both RIPSum and RespEffort3D and compared to each other. Fig. 4 shows the ERF score comparison between RIP and RespEffort3D, where each point corresponds to a respiratory event. Between the two respiratory effort signals, Pearson’s r is 0.62 (r ≠ 0, p = 0.0001). This suggests similarity in behavior during respiratory events and validates the decrease in RespEffort3D similar to RIPSum. The average decrease in RIPSum during hypopneas is 54 and 51% for RespEffort3D, which is not far off the reported 50% reduction in effort in the thoracoabdominal movement [16] and the reported 50% reduction in polyvinylidene fluoride belts’ sum (PVDFSum), another alternative sensor [3]. On the other hand, the average decrease in effort during apneas of all recordings for RIPSum and RespEffort3D is 72 and 73%, respectively. In the literature a 90% reduction was reported for PVDFSum in apneas [3]. The decrease in RespEffort3D by less than 90% can be attributed to the continuous breathing during obstructive apneas. In some events an increase is observed for both RespEffort3D and RIPSum; however, this is not surprising, because as specified in [1], continuous or increased inspiratory effort occurs in obstructive cases.
In order to strengthen our claims that RespEffort3D can score apneas and hypopneas, we applied a simple algorithm to find events of reduction of more than 10 s to RespEffort3D and RIPSum. In Fig. 5, the number of RespEffort3D events per recording is plotted against the number of apnea and hypopnea events. There is a moderate correlation between the two, with Pearson’s r = 0.75 (r ≠ 0, p = 0.0001). Between RespEffort3D events and RIPSum events there is a moderate correlation of r = 0.78 (r ≠ 0, p = 0.0001). Between RIPSum events and RespEffort3D events there is a strong correlation of r = 0.88 (r ≠ 0, p = 0.0001), see Fig. 6. These events of reduction in the effort signals serve as candidates for respiratory events, as further rules are needed to score hypopneas. This also explains the systematic overestimation of the reduced effort events. One limitation of this comparison between reduced effort events and respiratory events (combination of apnea and hypopneas) is that there is no distinction between apneas and hypopneas. Further investigation is necessary to identify the difference between apneas and hypopneas in RespEffort3D.
Our results show that RespEffort3D decreases similarly to RIPSum during respiratory events. We found a strong correlation between the number of events of reduction in RespEffort3D and RIPSum. Taken together, this is a very strong indication that RespEffort3D can be used for the scoring of respiratory events, similarly to the RIPSum.
Conclusion and recommendation
We were able to derive a respiratory effort signal, RespEffort3D, from a 3D TOF camera recording mounted above the bed. The RIPSum and the RespEffort3D were shown to have similar behavior for the two crucial tasks, i. e., classification of apneas and scoring of respiratory events. We can therefore recommend the use of 3D cameras as a replacement for the RIP belts for sleep apnea syndrome screening. This will lead to a better sleep quality for the patients, as the uncomfortable RIP belt sensors can be omitted.
For future work, deriving separate abdomen and thorax signals from the 3D TOF camera is worth looking into to improve the apnea classification rate. Furthermore, encouraged by the strong signal drops during respiratory events, we are planning to investigate the descriptive power of the RespEffort3D signal for the detection of respiratory events when combined with other noninvasive sensors.
Abbreviations
- AASM:
-
American Academy of Sleep Medicine
- AHI:
-
Apnea–hypopnea index
- ERF:
-
Effort reduction factor
- FNE:
-
First night effect
- PSG:
-
Polysomnography
- RDI:
-
Respiratory disturbance index
- RIP:
-
Respiratory inductance plethysmography
- TOF:
-
Time of flight
References
Berry RB, Brooks R, Gamaldo CE (2017) The AASM manual for the scoring of sleep and associated events: rules, terminology, and technical specifications. Version 2.4. American Academy of Sleep Medicine, Darien
Farré R, Montserrat JM, Navajas D (2004) Noninvasive monitoring of respiratory mechanics during sleep. Eur Respir J 24:1052–1060. https://doi.org/10.1183/09031936.04.00072304
Berry RB, Budhiraja R, Gottlieb DJ, Gozal D, Iber C, Kapur VK, Marcus CL, Mehra R, Parthasarathy S, Quan SF, Redline S, Strohl KP, Ward SLD, Tangredi MM (2012) Rules for scoring respiratory events in sleep: update of the 2007 AASM manual for the scoring of sleep and associated events. J Clin Sleep Med 8:597–619. https://doi.org/10.5664/jcsm.2172
Kaniusas E (2012) Biomedical signals and sensors I: linking physiological phenomena and biosignals. Springer, Berlin Heidelberg
Agnew HW, Webb WB, Williams RL (1966) The first night effect: an Eeg studyof sleep. Psychophysiology 2:263–266. https://doi.org/10.1111/j.1469-8986.1966.tb02650.x
BaHammam AS (2005) Signal failure of type 2 comprehensive unattended sleep studies in patients with suspected respiratory sleep disordered breathing. Sleep Breath 9:7–11. https://doi.org/10.1007/s11325-005-0001-6
Kagawa M, Tojima H, Matsui T (2016) Non-contact diagnostic system for sleep apnea-hypopnea syndrome based on amplitude and phase analysis of thoracic and abdominal Doppler radars. Med Biol Eng Comput 54:789–798. https://doi.org/10.1007/s11517-015-1370-z
Norman MB, Middleton S, Erskine O, Middleton PG, Wheatley JR, Sullivan CE (2014) Validation of the sonomat: a contactless monitoring system used for the diagnosis of sleep disordered breathing. Sleep 37:1477–1487. https://doi.org/10.5665/sleep.3996
Zaffaroni A, de Chazal P, Heneghan C, Boyle P, Mppm PR, McNicholas WT (2009) SleepMinder: An innovative contact-free device for the estimation of the apnoea-hypopnoea index. In: 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp 7091–9094
Otte K, Kayser B, Mansow-Model S, Verrel J, Paul F, Brandt AU, Schmitz-Hübsch T (2016) Accuracy and reliability of the Kinect version 2 for clinical measurement of motor function. PLoS ONE 11:e166532. https://doi.org/10.1371/journal.pone.0166532
Yu M‑C, Wu H, Liou J‑L, Lee M‑S, Hung Y‑P (2012) Multiparameter sleep monitoring using a depth camera. In: Biomedical engineering systems and technologies. Springer, Berlin, Heidelberg, pp 311–325
Garn H, Gall M, Kohn B, Wiesmeyr C, Kloesch G, Wimmer M, Stefanic-Kejik A, Boeck M, Mandl M, Ipsiroglu OS, Seidel S (2018) 0678 contactless 3D detection of leg movements in sleep. Sleep 41:A251–A251. https://doi.org/10.1093/sleep/zsy061.677
Garn H, Kohn B, Wiesmeyr C, Dittrich K, Wimmer M, Mandl M, Kloesch G, Boeck M, Stefanic A, Seidel S (2017) 3D detection of the central sleep apnoea syndrome. Curr Dir Biomed Eng 3:829–833. https://doi.org/10.1515/cdbme-2017-0174
Falie D, Ichim M (2010) Sleep monitoring and sleep apnea event detection using a 3D camera. In: Communications (COMM), 2010 8th International Conference on. IEEE, pp 177–180
American Academy of Sleep Medicine Task Force (1999) Sleep-related breathing disorders in adults: recommendations for syndrome definition and measurement techniques in clinical research. Sleep 22:667–689. https://doi.org/10.1093/sleep/22.5.667
Gould GA, Whyte KF, Rhind GB, Airlie MA, Catterall JR, Shapiro CM, Douglas NJ (1988) The sleep hypopnea syndrome. Am Rev Respir Dis 137:895–898. https://doi.org/10.1164/ajrccm/137.4.895
Guijarro-Berdiñas B, Hernández-Pereira E, Peteiro-Barral D (2009) Classifying sleep apneas using neural networks and a combination of experts. In: Current topics in artificial intelligence. Springer, Berlin, Heidelberg, pp 271–280
Senny F, Destine J, Poirrier R (2008) Midsagittal jaw movement analysis for the scoring of sleep apneas and Hypopneas. Ieee Trans Biomed Eng 55:87–95. https://doi.org/10.1109/TBME.2007.899351
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
C. Coronel, C. Wiesmeyr, H. Garn, B. Kohn, M. Wimmer, M. Mandl, M. Glos, T. Penzel, G. Klösch, A. Stefanic-Kejik, M. Böck, E. Kaniusas, and S. Seidel declare that they have no competing interests.
This study was approved by the ethics committee of the state of Upper Austria (B-130-17) and ethics committee of the Charité—Universitätsmedizin Berlin (EA1/127/16).
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Coronel, C., Wiesmeyr, C., Garn, H. et al. Measurement of respiratory effort in sleep by 3D camera and respiratory inductance plethysmography. Somnologie 23, 86–92 (2019). https://doi.org/10.1007/s11818-019-0203-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11818-019-0203-y