Introduction

Obstructive sleep apnea (OSA) is a systemic syndrome characterized by episodes of apnea and hypopnea during sleep, due to complete or partial pharyngeal obstruction [1]. From estimations in the general population, moderately severe OSA with an apnea–hypopnea index (AHI) >15 is present in 11.4% of men and 4.7% of women [2, 3]. With the trend of increasing prevalence of obesity and aging of the population, the incidence of clinically significant OSA is expected to increase five- to tenfold over the next decade [4]. There is growing evidence implicating OSA as a causal pathway to the development of cardiorespiratory diseases [5–12]. and it is estimated that nearly 80% of men and 93% of women with moderate to severe sleep apnea are undiagnosed [4].

In-laboratory polysomnography (PSG) is the gold standard for diagnosing OSA [13]. However, in-laboratory PSG is a time-consuming and costly procedure. Furthermore, the growing awareness of sleep apnea has exacerbated the long waiting list in many sleep laboratories [14]. In-laboratory PSG is also not practical in some situations, such as postoperative sleep evaluation. To deal with this issue, a number of portable sleep monitoring devices have been developed [15–20]. To provide evidence-based recommendations on the use of unattended portable monitors in the diagnosis of obstructive sleep apnea in adult patients, a clinical guideline has been published by the Portable Monitoring Task Force of the American Academy of Sleep Medicine in 2007 [21–24]. Since then, many new models of portable sleep monitors have been developed.

Embletta with proxy X100 is a new portable sleep study device with ten channels, which is a level 2 diagnostic tool [25]. Although different models of this family have been evaluated and used in research and clinical settings [26–33], there is no available published literature to validate this model of Embletta.

Patients coming to surgery may have increased risk of obstructive sleep apnea. Patients with obstructive sleep apnea are known to have an increased risk of perioperative complications [34–37]. Therefore, preoperative screening for obstructive sleep apnea is essential and the American Society of Anesthesiologists has published a guideline recommending preoperative screening for obstructive sleep apnea [38]. The STOP questionnaire [39], a four-question formatted with yes/no answer has been used as a screening tool in many preoperative clinics. In those surgical patients with the preoperative clinical suspicion of OSA, portable PSG offers an alternative tool in diagnosing OSA. This will expand the realm of anesthesiologists as a perioperative physician. The objective of the study is to test the hypothesis that installed by well-trained technicians, a level 2 portable sleep device (Embletta X100) is a reliable and practical alternative for standard in-laboratory PSG.

Methods

Study subjects

The study was carried out in Toronto Western Hospital and Mount Sinai Hospital, Toronto. Ethics approvals were obtained from both institutions. Patients 18 years or older, who were ASA I–IV and scheduled for elective procedures in general surgery, orthopedics, urology, plastic surgery, gynecology, spine surgery, or neurosurgery were included for the study. Patients who were unwilling or unable to give informed consent, patients previously diagnosed with OSA or any other sleep breathing disorder, or patients who were expected to have abnormal EEG findings (e.g., brain tumor, epilepsy surgery, patients with deep brain stimulator) were excluded. All patients who visited the preoperative clinics for their scheduled surgery and met the inclusion criteria were approached by the research staff. After informed consent was obtained, patients were invited to undergo one to five nights of sleep study using the Embletta X-100.

For the patients recruited for validation study, they underwent standard PSG and Embletta X100 simultaneously in a sleep laboratory before surgery. The other patients received sleep studies with Embletta X100 preoperatively at home. Some patients had sleep studies with Embletta X100 on postoperative nights 1, 3, 5, and 7. In patients with positive PSG indicating clear pathology, their family physicians were notified so that the patients could be referred to sleep physicians regarding further clinical management.

The standard in-laboratory PSG

A one-night, in-laboratory PSG was conducted before surgery at Toronto Western Hospital Sleep Research Laboratory. During the overnight PSG study, every patient went to bed at his/her usual bedtime. Collection of continuous sleep architectural data was accomplished using a standard EEG montage consisting of an electroencephalogram, electro-oculogram, submental electromyogram and electrocardiogram using surface electrodes. Ancillary channels were used to specifically record respiratory parameters including respiratory effort by thoracoabdominal excursion, respiratory inductive plethysmography, and oronasal airflow by nasal airflow pressure. Oxygen saturation was measured with a pulse oximeter.

One certified PSG technologist with 10 years experience scored all the sleep records under the supervision of a sleep physician (C.M.S.). The certified technologist was blinded to the results of the sleep study conducted with Embletta X100 and clinical information of the patients. Sleep stages and the AHI were scored according to the standard criteria [40].

Ambulatory PSG

Full-night unattended PSG recording with Embletta X100 were performed at the patient’s home preoperatively and on postoperative nights 1, 3, 5, or 7 in hospital or at home when feasible. The PSG recording montage consisted of two electroencephalographic channels (C3 and C4), left or right electroculogram, chin muscle electromyograms, nasal cannula (pressure), thoracic and abdominal respiratory effort bands, body-position sensor, and pulse oximetry. The device measures the oxyhemoglobin saturation at a rate of three samples per second.

At bedtime, the device was hooked up to the patients by well-trained PSG technicians. The overnight recording was unattended. The patients were taught how to dismantle the device, which was picked up by the same sleep technician or study coordinator the following morning. The patients were asked to keep a sleep diary (Appendix 1). The sleep technician or study coordinator picking up the device ensured that the sleep diary was completed.

One certified PSG technologist with more than 8 years experience scored recordings from Embletta. He was blinded to the clinical information and the results of in-laboratory PSG. The scoring of recordings from Embletta was carried out in two ways: manual scoring and automatic scoring. Somnologia Studio 5.0 was the scoring platform for both scoring methods. In manual scoring (manual-scoring), the PSG recording was manually scored epoch by epoch by the PSG technologist, according to the manual published in 2007 by American Academy of Sleep Medicine [40]. In automatic scoring (auto-scoring), the PSG recording was scored fully with Somnologia Studio 5.0.

Data analysis

Sample size estimation

According to a previously published study to validate an Embletta model with nasal pressure transducer, thoraco-abdominal movement detection, oximeter, and body position detection [24], the correlation coefficient between AHI from standard PSG and Embletta was 0.98. If we consider that the null Pearson coefficient is 0.3, the expected Pearson correlation coefficient is 0.8, alpha = 0.05, power = 0.9 and with two sides; the estimated sample size will be 20.

The demographic data is presented with descriptive statistics, mean ± SD was used for continuous data with normal distribution, median (inter-quartile range, IQR) for continuous data with skewed distribution, and frequency (%) for categorical data. The correlation analysis and paired Student t test between variables from Embletta with manual scoring or automatic scoring and the corresponding variables from standard laboratory PSG were used to evaluate the accuracy of Embletta with manual or automatic scoring. Bland Altman analysis for AHI and the agreement between two methods at different AHI cutoffs was also analyzed. Descriptive statistics with frequency of different quality category of PSG recording with Embletta was summarized across the different types of PSG.

Results

Validation of Embletta

Twenty-four patients were recruited for the validation study. Of them, three patients did not successfully complete both in-laboratory PSG and Embletta PSG: standard in-laboratory PSG not done due to a misunderstanding between technicians in one patient, Embletta did not start recording in one patient, and Embletta only recorded for 40 min in another patient. Twenty-one patients who had complete sleep study with good quality of PSG recordings on both systems were included in the final data analysis. There were ten females and 11 males. The average age was 54 ± 11. Body mass index was 36 ± 9 kg/m2.

Table 1 summarizes the results of correlation analysis and paired Student t test in sleep parameters from in-laboratory standard PSG and Embletta with manual scoring. Except for the central apnea index, mixed apnea index, average duration for apnea hypopnea episodes, longest duration for apnea–hypopnea episodes, and average wake SaO2; all the corresponding parameters analyzed from the two methods are significantly correlated. This suggests that results from the Embletta are good predictors for the results of laboratory PSG. However, at the same time, there was a significant difference of absolute value between the two methods in sleep latency, sleep stage 1, AHI, NREM AHI, respiratory arousal index and spontaneous arousal index. For example, AHI from Embletta with manual scoring was higher than that from laboratory PSG: 11.5 (18.7) vs. 8.1(18.3) in median (IQR). The mean difference is 2.3 ± 4.7 (mean ± SD) or 1.2(2.6) (median(IQR)), p = 0.038. The relationship for AHI was further demonstrated in Fig. 1a and b.

Table 1 Comparison of results from Embletta (Manual Scoring ) and laboratory PSG
Full size table
Fig. 1
figure 1

a Scatter plot of apnea-hypopnea index (AHI) recorded by laboratory standard PSG and Embletta with manual scoring. b Bland-Altman analysis of apnea hypopnea index (AHI) recorded by laboratory standard PSG and Embletta with manual scoring. The difference was calculated as AHI from Embletta (manual scoring) minus AHI from laboratory PSG

Full size image

The inter-rater agreement between the laboratory PSG and Embletta with manual scoring at AHI > 5, AHI > 10, and AHI > 15 as cutoffs was almost perfect with Kappa coefficient = 1 for AHI > 5 and AHI > 15, and 0.811(0.566–1.000) for AHI > 10. The inter-rater agreement at AHI > 30 was substantial with Kappa coefficient = 0.69 (95% CI: 0.29–1.00; Table 2).

Table 2 Inter-rater agreement (Kappa Coefficient) between Embletta (Manual Scoring or AutoScoring) and laboratory PSG at different AHI cutoffs
Full size table

Table 3 shows the comparison between parameters from laboratory PSG and Embletta with automatic scoring. A significant correlation was observed in sleep period time, AHI, REM AHI, NREM AHI, obstructive apnea index, hypopnea index, and lowest SpO2. There was no correlation between any sleep architecture parameters from two methods. There was no significant difference in REM AHI, NREM AHI, obstructive apnea index, average SpO2 and lowest SpO2 from both methods. The difference of AHI (AHIEmblettaAuto – AHILab) was −3.7 ± 7.7 (mean ± SD ) or −0.9(7.0) (median(IQR)). The relation of AHI from two methods was further demonstrated in Fig. 2a and b. The inter-rater agreement between laboratory PSG and Embletta with autoscoring was moderate to substantial at different AHI cutoffs (Table 2).

Table 3 Comparison of sleep study results from Embletta (Auto Scoring) and lab PSG
Full size table
Fig. 2
figure 2

a Scatter plot of apnea-hypopnea index (AHI) recorded by laboratory standard PSG and Embletta with automatic scoring. b Bland-Altman analysis of apnea hypopnea index (AHI) recorded by laboratory standard PSG and Embletta with automatic scoring. The difference was calculated as AHI from Embletta (automatic scoring) minus AHI from laboratory PSG

Full size image

The quality of Embletta PSG

In 385 patients, 1,002 perioperative PSGs were carried out with Embletta: preoperative—385, postoperative night 1—298, night 3—208, night 5—56, night7—55. There were 204 females and 181 males. The average age was 59 ± 13 years and body mass index was 39 ± 5 kg/m2.

The quality of 1002 PSG recordings was summarized in Table 4. The majority of PSG recordings (88.7%) were technically good, which is defined as more than 4 h recording with good quality on all channels. Nine percent of PSG recordings were technically acceptable, which is defined as more than 4 h recording with defect on one or two channels, but the AHI was still viewed as reasonably reliable. This includes the PSG recordings with no thorax and/or abdominal effort recording, no EOG, heavy EKG artifact on EEG, no EMG, or only one channel of EEG. Only 23 (2.3%) PSG recordings with Embletta failed, including 6 (0.6%) without EEG recording, 4 (0.4%) with battery failure within 4 h and 13 (1.3%) with electrodes removed by patients within 4 h. The PSG recordings on nights 5 and 7 after surgery had a higher rate of PSG recordings with good quality. The scoring of AHI was viewed as reliable in 98.2% of home PSG recordings.

Table 4 Summary of the quality of PSG recordings with Embletta
Full size table

Discussion

This study demonstrated that the parameters of sleep architecture, sleep respiratory events, arousals, and pulse oxygen saturation level from manually scored PSG recording with Embletta X100 were highly correlated with the those from in-laboratory standard PSG . However, the AHI was overestimated by Embletta X100 by 2.3. It was also noted that the significant correlation between the parameters from Embletta X100 with automatic scoring and standard PSG was found in sleep period time, AHI, REM AHI, NREM AHI, obstructive apnea index , hypopnea index, and lowest SpO2. The automatic scoring by Somnologia Studio 5.0 is not reliable, especially for the sleep architecture. Out of a total of 1,002 recordings with Embletta X100, 88.7% of PSG recordings was technically good for sleep architecture and AHI scoring. The scoring of AHI was reliable in another 9% of recordings with minor technical defects. The major limitation for the study is that the validation sample size was not big enough to do predictive analysis and receiver–operator characteristic (ROC) analysis.

Based on these results, when it is installed by a well-trained technician, Embletta X100 has a high rate of good-quality recordings and can be used as an alternative when in-laboratory standard PSG is not available. The recordings should be scored by a certified PSG technologist and reviewed by a physician trained in sleep medicine. Total automatic scoring by Somnologia Studio 5.0 is not reliable, especially regarding the sleep architecture. However, the absolute value from Embletta is not equal to the value from standard PSG. Some adjustments may be needed.

The major focus of previous studies on portable sleep devices is the ability of the devices to detect sleep breathing disorder [16, 22, 41, 42]. Tiihonen and colleagues reported that 77.2% of PSG recording with a type 3 Embletta was technically good and 80.8% of recordings was diagnostically acceptable in technician-assisted installation [22]. Our study comprehensively compared major sleep parameters between two systems. This provides valuable information on the accuracy of portable sleep study device on the sleep architecture parameters. Compared with a previous study [22], the rate of technically good and diagnostically acceptable recordings was higher in our study, 88.7% and 97.7%, respectively. We trace this to the well-trained PSG technicians, education and cooperation of patients, and continuous feedbacks from the scoring technologist. The PSG recordings done on the 5th and 7th night after surgery had a higher percentage of good quality recordings. This indicated that patient's experience with the system could increase the success rate of unattended PSG recording. The patients on the first and third postoperative night had significant pain and this may reduce the level of compliance and recording quality.

The high correlation between the results from manually scored PSG recordings using Embletta X100 and laboratory standard PSG suggests that if Embletta X100 is used properly, it could provide useful information of sleep physiology, including sleep stages, sleep respiratory events, arousals, and oxygen saturation level. However, we could not find a significant correlation in central sleep apnea and mixed sleep apnea detected by Embletta X100 and laboratory PSG. Since the frequency of central apnea and mixed sleep apnea in the study patients was very low (close to 0), we may need a bigger sample size or a study in patients with higher frequency of central sleep apnea to further evaluate the ability of Embletta on detecting central sleep apnea.

Although the difference of AHI between Embletta with manual scoring and laboratory-standard PSG was statistically significant, it is comparable with previously published studies on other portable devices, [16, 22, 41, 42]. There were several possible reasons for the difference. The most significant one is that the scoring platform is different. Embletta recording was scored on Somnologia Studio 5.0 and the standard PSG was scored on Sandman version 7.2. The other possible causes include the difference between PSG technologists and different sensitivity of nasal flow sensor and oximeter. Collon demonstrated that there was a significant variation in AHI and sleep stages from the same PSG recordings scored by different PSG technologists [43]. The difference suggests that validation and possible adjustment is necessary before adopting a portable system.

When coming to inter-rater agreement between Embletta X-100 and laboratory PSG in terms of diagnosing OSA at different cutoffs of AH, it was almost perfect to recognize all patients with OSA (Kappa Coefficient=1). Although the performance of Embletta to identify the patients with severe OSA was not as perfect as to recognize all patients with OSA, it is acceptable and better than the performance of electrocardiography to detect the pathologic Q-wave in myocardial infarction patients ( Kappa Coefficient = 0.64) [44]

Our results also support that fully automatic scoring by Somnologia Studio 5.0 is not as reliable, especially on sleep stages. However, there was also a significant correlation between results from the automatic scoring of Embletta PSG and standard laboratory PSG in AHI, REM AHI, NREM AHI and lowest SpO2. This indicates that when manual scoring by a certified technologist is not available, AHI and lowest SpO2 from the automatic scoring with Somnologia Studio 5.0 could be used. But caution must be taken.

In conclusion, Embletta X100 installed by a well trained sleep technician is a good alternative when standard PSG was not available or impractical. Manually scoring by a certified PSG technologist is the key for reliable results. There is a potential of significant difference between two systems, so that validation is necessary.