Introduction

The Epworth sleepiness scale (ESS) has been developed by Johns as a self-reported questionnaire to assess excessive daytime sleepiness (EDS) and daytime sleep propensity [1, 2]. EDS is generally considered to be the result of disturbed or inadequate sleep which is very common in modern societies [3]. EDS is one of the most disabling symptoms of obstructive sleep apnea syndrome and some other sleep disorders including narcolepsy [4], insomnia [5], hypersomnia [6], and restless leg syndrome [7]. Epidemiologic studies also revealed presence of EDS in patients with common medical disorders such as ulcers, migraines, depression [8], obesity, and diabetes [9], but with no history of sleep disordered breathing. ESS is widely used in daily clinical practice as well as in research protocols [10] for the evaluation of EDS due to its simplicity, high reliability, and internal consistency. However, results derived from ESS are limited by the subjective perception effects, and thus, the ESS should be combined with more objective research methods on evaluation of sleep in patients with various sleep disorders. The questionnaire consists of eight items which aim to assess the degree of sleepiness during common daily situations for which the respondents evaluate their propensity to fall asleep on a scale from 0 to 3 (Appendix 1, 2). Although it was originally created for the English-speaking population, it has been translated and proven for its reliability and validity in different languages [11–18]. It is important to have a standardized version of the questionnaire since subjective evaluation of sleepiness and the tendency to fall asleep could be related to cultural, social, and language factors. Although ESS is used in Croatia, validation of the Croatian language version is yet to be performed.

There is an increasing awareness that obstructive sleep apnea syndrome (OSAS) is a common disorder and may have adverse medical complications so that the development of simple clinical screening methods [19] is extremely useful to identify subjects who really need whole-night polysomnography (PSG) [20–22]. The Snoring, Tiredness, Observed Apnea, and High Blood Pressure (STOP) questionnaire, a concise and easy-to-use screening tool for OSAS with high sensitivity, has been developed and validated in surgical patients during preoperative assessment and compared with Berlin questionnaire [23]. The 4-item STOP questionnaire (Appendix 3, 4) can classify patients as being at high risk of having OSAS if they answer yes to two or more questions. As of yet, no previous study has examined sensitivity of the STOP questionnaire in a broader, more diverse population.

There is still a need for research studies on clinical predictions in patients with a risk for OSAS to be conducted in non-English-speaking countries where lifestyle habits and language expressions may be quite different and, as such, have an impact on the final score of the screening method used. The present study was undertaken to provide reliable and validated ESS in the Croatian language and to evaluate the STOP and ESS questionnaires as screening instruments for OSAS.

Methods

Translation of ESS and STOP questionnaires into the Croatian language

The ESS and STOP questionnaires were translated into the Croatian language by two psychologists and two clinicians. The questionnaires were then administered to a small group of ten patients seen at the Split Sleep Medicine Center to ensure clarity and face validity. The questionnaires were back-translated from Croatian into English by a bilingual professional translator for comparison with the original text. The sentence structure and presentation of the ESS (Appendix 1 and 2) and STOP questionnaires (Appendix 3 and 4) were similar to those of the English version.

Subjects

A total of 425 subjects, including patients admitted to the Split Sleep Medicine Center, postgraduate students enrolled at the University of Split School of Medicine (academic year 2009/2010), and their relatives were assessed.

All subjects completed Croatian versions of the ESS (Appendix 1) and STOP questionnaires (Appendix 3), which had been translated according to the procedures described above. Complete results of the PSG, including electroencephalogram, electrooculogram, electromyogram, arterial oxygen saturation, abdominal and thoracic respiratory efforts, oronasal pressure, snoring, monitoring electrocardiogram, etc., were available from 217 patients admitted to the Split Sleep Medicine Center. An apnea was defined as complete cessation of airflow for at least 10 s, whereas hypopnea was defined as a decrease in airflow by more than 50% from baseline for at least 10 s in association with a fall in arterial oxygen saturation of at least 3%. In order to correlate severity of OSA with ESS scores, we accessed the patients' PSG data including the value of apnea–hypopnea index (AHI). Subjects with sleep apnea were classified into three subgroups according to their AHI: mild subgroup (subjects with AHI <15/h), moderate (subjects with AHI between 15 and 30/h), and severe comprised those with AHI ≥30/h.

The questionnaire was administered to 208 control subjects: 22 postgraduate students from the University of Split School of Medicine and 186 of their relatives older than 18 years who had no history of sleep disorders. Test–retest reliability of the total score was assessed in 20 randomly selected subjects who completed ESS on two occasions, 1 month apart.

Statistical analysis

Continuous data are presented as means ± standard deviation, whereas categorical variables are presented as whole numbers and percentages. The comparison of continuous variables was done using Student's t test and one-way analysis of variance. The internal consistency and reliability were assessed using a Cronbach's alpha coefficient. The test–retest variability was examined by the mean difference of the scores and intra-class correlation coefficient. A probability of p < 0.05 was considered statistically significant.

Results

There were a total of 425 subjects involved in this study, and their characteristics are presented in Table 1. The mean score of ESS was significantly higher for the patients referred to the Split Sleep Medicine Center compared to control group (8.2 ± 5.0 vs. 5.9 ± 3.8, p < 0.001; Tables 1 and 2). When each item was analyzed, the highest score was obtained for the fifth item (lying down to rest in the afternoon) and was statistically different in patients and control group (1.8 ± 1.0 vs. 1.4 ± 0.8, respectively, p < 0.001; Table 2). On the other hand, the lowest score was obtained for the sixth item (sitting and talking to someone) in patients and control subjects (0.3 ± 0.6 vs. 0.2 ± 0.5, respectively, p > 0.05; Table 2). Among patients with suspicious sleep disordered breathing referred to the Split Sleep Medicine Center, the STOP questionnaire classified 96% of those patients as being at risk of having OSAS (Table 1). The STOP questionnaire revealed excellent performance in both sensitivity (0.96) and specificity (0.83) in the detection of risk for OSAS.

Table 1 Patients and control subjects' characteristics
Full size table
Table 2 Item analysis of the Croatian version of ESS
Full size table

Construct validity of the Croatian version of ESS

ESS score and severity of sleep apnea

To assess the validity of the Croatian version of ESS, correlations of three polysomnographic measures of sleep apnea severity (AHI, the minimum recorded SaO2, and mean SaO2) with the total ESS score were calculated. There was a significant correlation between AHI and total ESS scores (r = 0.243, p < 0.001). Negative correlations were found between total ESS scores and the lowest SaO2 recorded (r = −0.319, p < 0.001), as well as between total ESS score and mean SaO2 (r = −0.149, p = 0.038).

ESS score was significantly greater in the severe compared with the mild subgroup according to AHI (9.5 ± 5.5 and 6.5 ± 4.3, respectively, p < 0.05; Table 3). However, there was no significant difference between the moderate and the remaining two groups (p > 0.05, Table 3).

Table 3 ESS score, age, and BMI of patients according to severity of OSAS
Full size table

In a receiver operating curve (ROC), the true positive rate (sensitivity) is plotted in function of the false positive rate (100-specificity) for different cutoff points of ESS and STOP variables. Sensitivity was defined as probability that a test result will be positive when the OSAS is present (true positive rate, expressed as a percentage), while specificity was defined as probability that a test result will be negative when the OSAS is not present (true negative rate, expressed as a percentage). The ROC curve of the ESS for identification of cases with AHI >5/h was 0.64 which means that the probability that ESS predicted AHI >5/h correctly was 64% (Fig. 1). The cutoff value of the ESS score of 4 provided best sensitivity (0.81) and specificity (0.42) in detecting AHI >5/h. Values lower than this point in the curve had higher sensitivity but lower specificity, whereas greater values were more specific but less sensitive.

Fig. 1
figure 1

ROCs of the Croatian version of the Epworth sleepiness scale (dashed line) and STOP questionnaire (plain line) in detecting patients with AHI >5/h. Area under the curve was 0.64 for the ESS and 0.84 for STOP questionnaire. Sensitivity (true positive rate, y axis) is plotted in function of the 100-specificity (false positive rate, x axis)

Full size image

Internal consistency and test–retest reliability

Cronbach's alpha coefficient for the ESS Croatian version was 0.84 which indicated an excellent internal consistency. After deleting some specific items, particularly the sixth item (sitting and talking to someone), the seventh item (sitting after lunch without alcohol), or the eighth item (in a car while stopped for a few minutes in the traffic), there were no substantial changes in the internal consistency (Cronbach's alpha 0.81–0.82, Table 2).

Reproducibility was tested in 20 subjects, and no significant differences were found in each item or in the total score in the first and second assessments (ESS score was 5.0 ± 2.9 vs. 5.2 ± 2.2. p > 0.7). The intra-class correlation coefficient was 0.72 (95% CI 0.30–0.89).

STOP questionnaire

The STOP questionnaire as a screening tool for the patients with high risk for OSAS (STOP score ≥2) demonstrated a high level of sensitivity and specificity in patients referred to the Split Sleep Medicine Center. The ROC curve of the STOP questionnaire for identification of cases with AHI >5/h was 0.84 which means that the probability that STOP predicted AHI >5/h correctly was 84% (Fig. 1). The cutoff value of 2 provided best sensitivity (0.96) and specificity (0.83). Positive predictive value of those ranked by the STOP questionnaire as being at high risk of having OSAS was 0.61, while negative predictive value was 0.95. The STOP questionnaire identified 95% of patients with mild AHI, 92% of patients with moderate AHI, and 98% of patients with severe AHI as being at risk of having OSAS (Table 3).

Discussion

The results of our study indicate that both questionnaires, ESS and STOP, successfully distinguished healthy subjects from subjects with OSAS. The STOP questionnaire had better probability to correctly predict high-risk patients for OSAS compared to ESS. The Croatian version of ESS showed excellent internal consistency and reliability which is in accordance to that of other languages, despite the existing cultural and social differences [11–13, 15, 17, 18, 24–26]. Due to the relatively high prevalence of unrecognized and undiagnosed OSAS, there is a need to develop a reliable screening tool for the OSAS in the general population. Questionnaires that have high methodological validity, reasonable accuracy, and easy-to-use features are appropriate for research protocols as well as for routine daily practice [10, 19].

The ESS is one of the widely used simple, self-administered questionnaires for measuring daytime sleepiness that efficiently distinguishes healthy subjects from subjects with sleep disorders [1, 2]. It was not specifically designed to detect OSAS, but ESS score significantly correlated with AHI and the lowest SaO2 among subjects with sleep disordered breathing [15, 18]. Similarly, results of our study showed positive correlation between ESS score and AHI, reporting that the ESS score increased with increasing AHI from 6.5 in subjects with mild OSAS up to 9.5 in subjects with severe OSAS. Furthermore, ESS score obtained in our control group was significantly lower compared to ESS score in patients with OSAS (5.9 vs. 8.2).

In general, excessive daytime sleepiness is often the result of poor sleep quality, not exclusively associated to OSAS. The mean score of the Croatian version of ESS obtained in the control subjects (5.9 ± 3.8) was similar to the previously reported studies in English-speaking populations [1, 24], as well as for non-English-speaking populations [1, 11, 14, 26]. However, in the patients' group, the mean ESS score (8.2 ± 5.0) was lower than those reported in former studies with the English version of ESS (12.7 ± 5.5 [27] and 11.1 ± 5.2 and 11.2 ± 5.3 [10]). Other, non-English-speaking population studies have also found higher ESS scores, for example the German version of ESS (13.0 ± 5.1) [11], the Greek version (11.3 ± 5.1) [14], the Turkish version (12.6 ± 6.0) [15], the Thai version (9.9 ± 5.3) [26], and the Serbian version (10.0 ± 5.2) [18]. It is well accepted that the ethnic variability, differences in lifestyle habits, and social and cultural factors are important determinants that may influence daytime sleepiness measured by the ESS score. Therefore, we could only assume that the aforementioned variables might influence the results of our study. Among eight items of the Croatian version of the ESS questionnaire, items 3 (sitting inactive in a public place), 5 (lying down to rest in the afternoon), 7 (sitting quietly after a lunch), and 8 (in a car while stopped) should be rather annotated. Since our study was performed in the Mediterranean part of Croatia, there are some specific lifestyle habits. The lack of “traffic rush hour” that is related to Western countries results in less time spent in a car during traffic jams; therefore, item 8 was often scored as 0 or not scored at all. It is interesting to report that in our study, both patients and controls had the highest score for item 5 and a relatively high score for item 7. Both of these items could be related to the well known phenomenon of “Mediterranean siesta,” that is subjective perception of tiredness and lack of energy in the afternoon [14]. In addition, there was a significant proportion of missing data in responses to item 3 in both groups that could be explained by the fact that in the Croatian language, the term meeting is usually related to the business meeting. Overall, although influenced by language and cultural differences, ESS might be used as a screening tool for excessive daytime sleepiness in the Croatian population in a similar way as in the case of other non-English- and English-speaking populations previously reported.

Due to the relatively high prevalence of OSAS in the general population, an ideal screening test should maintain relatively high specificity, and it should be sensitive enough to detect most of the patients with OSAS [19, 28]. Previous studies have shown that the use of the ESS as a screening tool for OSAS in the general population is limited by its low sensitivity and specificity [26, 29, 30]. The most common cutoff point for determining excessive daytime sleepiness is an ESS score of 9 or greater [13, 18, 25]. Assuming the validity of this cutoff point, this translation resulted in high specificity but surprisingly low sensitivity.

With regard to predicting OSAS, a relatively novel STOP questionnaire has been shown to have the highest methodological validity, reasonable accuracy, and easy-to-use features [19]. It was originally developed and validated as an OSAS screening tool that demonstrated a moderately high level of sensitivity and specificity among surgical patients [23]. However, in this study, we tested the accuracy of the STOP questionnaire in the non-surgical population. In terms of its predictive parameters in our study, the STOP questionnaire itself demonstrated a high level of sensitivity (96%) and specificity (83%) at a cutoff point of 2 for determining the risk for OSAS. Moreover, ROC curve analysis found the STOP questionnaire to have very good discriminative properties for OSAS.

In a recent systematic review that aimed to identify and evaluate the available questionnaires for screening OSAS, the STOP questionnaire was suggested to be used in surgical patients [19]. In our study, the STOP questionnaire was proven useful in the detection of OSAS in non-surgical patients. We propose that the STOP questionnaire could be used as an easy-to-use and accurate screening tool in identification of patients with risk for OSAS in the general population, but it is waiting to be proven in the Croatian population as well as other populations.

There are some limitations of the present study. The “gold standard” for diagnosis of OSAS is laboratory polysomnography. Although control subjects did not undergo polysomnography, those with known sleep disorders were excluded from the control study group. Also, we did not investigate use of caffeine or medications and different lifestyle habits, as possible confounding factors among subjects, but we expected equal distribution of those factors in both studied groups. The patients with mild OSAS did not differ significantly from the control group in terms of ESS score. One might conclude that the Croatian version of ESS can distinguish between controls and severe/moderate OSAS patients, and more specifically, it might not be valid for mild OSAS patients. This might be concluded on the basis of a rather weak correlation (albeit statistically significant) between AHI and ESS score, which is in accordance with previous studies [11, 15, 26].

Current screening strategies for OSAS, such as ESS and STOP questionnaires, identify high-risk patients based on their self-reported symptoms. Novel screening strategies are trying to combine patients' symptoms with physical examination findings to further increase strength of the existing screening tools. By using these principles, the novel Neck Circumference, Airway Classification, Comorbidities, Epworth Sleepiness Scale, and Snoring questionnaire (NAMES) has been shown to efficiently screen for OSA in the high-risk population and is easy to use in daily practice [31]. However, validation of the NAMES protocol in different populations is yet to be performed.

In conclusion, ESS could be used in assessment of excessive daytime sleepiness in the Croatian population, and despite language and cultural differences, it demonstrated excellent internal consistency and reproducibility. The simple and easy-to-use STOP questionnaire, irrespective of any cultural and language differences, could serve as a reliable and quick tool in screening for OSAS.