Introduction

Adolescents experience marked changes in sleep–wake regulation and sleep patterns, including reduced time in slow-wave sleep and latency to rapid-eye-movement sleep and a preference for a delayed sleep phase [1, 2]. Adolescents are at increased risk of insufficient sleep, irregular sleep patterns, sleep disturbance and daytime sleepiness [3, 4]. Insomnia is one of the most prevalent sleep problems in adolescents and is associated with daytime distress, functioning impairment, substance use, depression, and self-harm [5, 6]. The prevalence of insomnia symptoms in previous epidemiological studies varies from 7 to 37% [7,8,9], possibly mainly due to different definitions or measures used to assess one or more insomnia symptoms or insomnia disorder. Furthermore, despite its high prevalence and significant impact on health and daytime functioning, insomnia often remains unrecognized, underdiagnosed and undertreated in adolescents possibly due to lack of brief and valid screening scales for population-based epidemiological studies and routine clinical assessments at school or general pediatric practice setting [9,10,11].

Although insomnia has been used in different contexts and refers to either a symptom or a specific disorder, the definition of insomnia disorder is very similar as defined by the International Classification of Sleep Disorders, 3rd Edition (ICSD-3) [12] and the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) [13]. According to ICSD-3, an insomnia disorder is defined as a subjective report of difficulty initiating sleep (DIS), difficulty maintaining sleep (DMS), waking up earlier than desired or early morning awakening (EMA), which occurs despite adequate opportunity for sleep and results in daytime consequences (distress or functioning impairment). According to DSM-V, insomnia is defined as complaint of sleep quantity or quality associated with problems falling asleep, staying asleep, and/or early morning awakenings, which causes distress and/or impaired cognitive, social, family, occupational or academic performance. For the diagnosis of chronic insomnia disorder, both ICSD-3 and DSM-V require the sleep disturbance and associated daytime symptoms occur at least 3 nights/week and has been going on for at least 3 months.

The prevalence rates of insomnia vary across studies and cannot be directly compared due to differences in the definition/criteria used to define insomnia and/or populations studied [14,15,16]. For example, in a recent study of insomnia in Hong Kong Chinese adolescents, the prevalence of overall insomnia symptoms using the Insomnia Severity Index (ISI) [17] total score ≥ 9 was 37.1%, while the prevalence of DIS, DMS, and EMA with the symptoms presenting at least 3 times a week was 6.1, 2.6, and 1.9%, respectively [8]. Most epidemiological studies of insomnia in adolescents have used one single or several items to measure insomnia symptoms [7, 18]. For example, insomnia is often defined by a positive response to either question, “Do you have trouble sleeping?” or “Do you have difficulty falling or staying asleep?” [7, 14, 19]. Some studies have used insomnia measures for adults to estimate insomnia in adolescents, such as ISI [17], the Pittsburgh Sleep Quality Index (PSQI) [20], and the Athens Insomnia Scale (AIS) [21]. A few studies have used clinical diagnostic instruments based on DSM or ICSD criteria to estimate the prevalence of insomnia disorder [16, 22, 23]. Another difference is the time frame to ask about insomnia, such as currently, last week, last month, last year or lifetime. In addition, the frequency to define insomnia symptoms also varies from currently having sleep disturbance to at least 3 times/week. In addition, different cutoffs of a scale score used to define insomnia can also result in great variation in the reported prevalence rates of insomnia symptoms or insomnia disorder. Furthermore, whether negative consequences are included to define insomnia can result in huge differences in the prevalence estimates [14].

As mentioned, there are currently several self-reported measures available for assessing frequency and severity of insomnia symptoms, including the ISI, PSQI, and AIS. Each of these instruments has its own advantages and limitations [24, 25]. However, all the measures are developed for adults. There are several brief, self- or parent-report questionnaires for evaluating sleep disturbances in adolescents, such as Pediatric Sleep Questionnaire [26], Sleep Habits Survey [27], and Sleep Disorders Inventory for Students [28], but none of them is specifically developed to assess self-report insomnia for adolescents [9, 10]. Given the sleep characteristics in adolescents, absence of the insomnia scale for adolescents, and inconsistent findings depending on measures used in previous studies, it is important to develop a simple, reliable, valid, and easily self-administered insomnia scale for clinical screening and research of insomnia in the general adolescent population.

We developed the Youth Self-Rating Insomnia Scale (YSIS), an 8-item self-administered questionnaire to measure insomnia severity for adolescents. The objective of the current study was to assess the psychometric properties of the YSIS in a large sample of Chinese adolescents. Specifically, our first aim was to assess the factor structure of the YSIS by exploratory and confirmatory factor analyses. The second aim was to assess the internal consistency reliability and test–retest reliability of the YSIS. The third aim was to assess its criterion validity in terms of sleep duration, sleep problems, daytime sleepiness, and depression. The fourth aim was to explore the cutoff of the YSIS for predicating general sleep disturbance.

Methods

Participants and procedure

Subjects used for the psychometric assessment of the YSIS were participants of the Shandong Adolescent Behavior and Health Cohort (SABHC). Detailed sampling and data collection have been described elsewhere [29,30,31,32]. In brief, participants included 11,836 adolescents from 5 middle and 3 high schools in 3 counties of Shandong Province, China. These counties and schools in Shandong were selected with consideration of their social demographics, the representativeness of adolescent students in the region, prior study collaboration, convenience, and budget to conduct the study.

In November–December 2015, participants were invited to complete a self-administered, structured adolescent health questionnaire (AHQ) to assess sleep, risk behaviors and mental health. After getting permission from the target schools, trained master-level public health workers administered the AHQ to participants in their classrooms during regular school hours. Before filling out the questionnaire, participants were instructed to read the instructions carefully and informed that the survey was anonymous, and their participation was voluntary without any penalties for nonparticipation. The questionnaire was re-administered to two classes of middle school students and two classes of high school students 2 weeks later to assess the test–retest reliability of the YSIS.

We obtained permission to conduct the study from the principals in the target schools and informed consent from participants before the survey. The study was approved by the research ethics committee of Shandong University School of Public Health and target schools.

Measures

YSIS

The YSIS is a self-administered questionnaire. It consists of 8 questions that ask about insomnia symptoms and waking symptoms or daytime consequences [24] within the past month. The symptoms assessed include: DIS, DMS, EMA, unrefreshing sleep, poor sleep quality, sleep insufficiency, sleep dissatisfaction, and interference of sleep difficulties with daytime functioning. The 8 items were developed based on ICSD-3 [12] and DSM-V [13] diagnostic criteria, several measures of sleep and insomnia for pediatric populations and adults, including the Sleep Habits Survey [27], Child Sleep Habits Questionnaire (CSHQ) [33], Pediatric Sleep Questionnaire [26], ISI [17], AIS [21] and the PSQI [20], and our series of sleep and health surveys in Chinese adolescent populations [18, 19, 30, 34, 35]. Example items are “During the past month, how often would you say you feel difficulty falling asleep?” “During the past month, how often would you say you have insufficient sleep?” “During the past month, how would you rate the quality of your sleep overall?” Sleep quality is rated from 1 = Very good to 5 = Very poor and sleep satisfaction is rated from 1 = Very satisfied to 5 = Very unsatisfied. The remaining 6 items are rated on a 5-point scale from 1 = Never, 2 = Rarely (< 1 time/week), 3 = Sometimes (1–2 times/week), 4 = Often (3–5 times/week), to 5 = Almost every day (6–7 times/week). Summing the scores on the 8 items yields a total YSIS score, ranging from 8 to 40. A higher total score of the YSIS indicates a greater insomnia severity during the past month. The YSIS was initially developed in English and translated into Chinese via translation and back translation by 4 bilingual (Chinese and English) sleep or psychological scientists. The English and Chinese version of the YSIS can be found in the “Appendix”.

Daytime sleepiness

The Chinese adolescent daytime sleepiness scale (CADSS) was used to assess daytime sleepiness [36]. The CADSS consists of 7 questions that ask about adolescents’ general feeling of drowsiness and dozing off at different situations during the daytime in the past month. Example items are “During the past month, how often would you say you feel sleepy during the day?” “During the past month, how often would you say you have dozed off in the morning classes?” All 7 items are rated on a Likert scale from 1 = Never to 5 = Almost every day. Summing up the scores of the 7 items yields a total CADSS score. A higher total score of the CADSS indicates a greater daytime sleepiness during the past month. The CADSS has satisfactory psychometric properties [36]. The Cronbach alpha with the current sample was 0.89.

Restless Legs Syndrome and Other sleep problems

Four questions developed by the 2003 NIH Restless Legs Syndrome (RLS) Diagnosis and Epidemiology workshop were used to assess RLS symptoms [30, 37]. (1) Do you have strange or uncomfortable sensations in your legs combined with an urge or need to move your legs? (2) Do these feelings occur mainly or only at rest and do they improve with movement? (3) Are these feelings worse in the evening or night than in the morning? (4) How often do these feelings occur? If the participant answered “yes” to all the first 3 questions, he/she was considered having RLS symptoms [30].

Three questions adapted from the CSHQ [33] were used to ask about loudly snoring, breath holding or pauses, and snorting/gasping during sleep. Hypnotic use was asked using the question “How often did you take medicine to help sleep?” The participants answered each question with a response from 1 = never, 2 = rarely (< 1 time/week), 3 = sometimes (1–2 times/week), 4 = often (3–5 times/week), to 5 = almost every day (6–7 times/week). Sleep duration was asked by “During the past month, on an average school day, how many hours of actual sleep did you get at night?” One question adapted from the Centre for Epidemiologic Studies Depression Scale (CES-D) [38] was used to ask about general sleep disturbance, i.e., “Please indicate how often you’ve felt you could not sleep well during the past week on a 4-point scale from < 1 day, 1–2 days, 3–4 days, to 5–7 days.”

Anxious/depressive symptoms

Anxiety and depression were measured by the Chinese Youth Self-Report (YSR) of Child Behavior Checklist [39]. The YSR anxious/depressed subscale consists of 16 items. Each item is rated on a 3-point scale: “0” = not true, “1” = somewhat or sometimes true and “2” = very true or often true over the last 6 months. Summing the scores of the 16 items yields a total anxious/depressive score. The higher the total score, the more severe the anxiety/depression. The Cronbach alpha with the present sample was 0.88 for the subscale.

Statistical analysis

Means and standard deviations for continuous variables or frequencies for categorical variables were reported. Participants were randomly divided into two subsamples, one for exploratory factor analysis (EFA) and one for confirmatory factor analysis (CFA), to assess construct validity and to compute internal consistency reliability of the YSIS scale. Correlation coefficients were computed to estimate the 2-week test–retest reliability. Concurrent validity was examined by comparing mean YSIS scores across sleep variables (sleep disordered breathing symptoms, RLS symptoms, and hypnotic use) and by computing correlations between YSIS scores and sleep duration, daytime sleepiness and anxiety/depression.

A receiver operating characteristic (ROC) curve analysis was performed to estimate the sensitivity and specificity of the YSIS at different cutoffs for predicting self-reported general sleep disturbance (almost every day). The accuracy of prediction was determined by the area under the ROC curve, with an area of 1 representing a perfect test and an area of 0.5 representing a worthless test.

EFA and CFA were conducted in Mplus 8.0. All other statistical analyses were conducted in IBM SPSS Version 25 (Armonk, NY: IBM Corp.).

Results

All sampled students attending school on the day of the survey (n = 11,836) were invited to participate in the SABHC baseline survey. A total of 210 students did not provide a response to any item of the YSIS and were thus excluded, leaving 11,626 participants for the current analysis (98.2%). Mean age of the sample was 15.0 (SD = 1.5) and 51% were boys. A total of 242 students from 4 classes returned usable retest questionnaires 2 weeks later. Mean age of the test–retest subsample was 14.7 (SD = 1.6) and 47% were boys.

YSIS item analysis

Table 1 shows descriptive statistics of YSIS items. The responses of item 1 (poor sleep quality) and item 2 (sleep dissatisfaction) appeared to be symmetric with the mode being the category “3” (Fair), items 3 (DIS), 4 (DMS), 5 (EMA), and 8 (Daytime functioning impairment) tended to be positively skewed with the mode of the category “1” (Never), while the other two items (Sleep insufficiency and Unrefreshing sleep) tended to be negatively skewed with the mode of the category “5” (5–7 times/week). Items 1, 2, 6, 7, and 8 were highly correlated with each other (rs ranged from 0.49 to 0.85); items 3, 4, and 5 were moderately correlated with each other (rs ranged from 0.36 to 0.48). All items were positively and moderately or highly correlated with the total scale score.

Table 1 Descriptive statistics of YSIS items in Chinese adolescents
Full size table

Factor analysis

To examine the factor structure underlying these 8 items, the sample was randomly divided into two subsamples, one for EFA (n = 5912) and the other for CFA (n = 5843). EFA and CFA were conducted in Mplus 8.0 using unweighted least squares estimation method with mean and variance adjustment (i.e., ULSMV) based on polychoric correlation matrix.

Exploratory factor analysis The polychromic correlation matrix based on the EFA subsample had two eigenvalues being greater than one (3.85 and 1.54), followed by the third eigenvalue being 0.72. We conducted two EFA models: one-factor and two-factors. For the two-factor model, a geomin rotation was performed allowing factors to be correlated. As shown in Table 2. The one-factor model did not fit adequately to the data. The fit of the two-factor model was much better although it was still not adequate. A clear factor-structure was observed from the two-factor model: items 1, 2, 6, 7, and 8 mainly loaded on the first factor, while the items 3, 4, and 5 mainly loaded on the second factor. Upon an examination of item content, modification indices, and interpretability of factor structure, we added a residual covariance between item 1 and item 2. The revised two-factor model fit the data reasonably very well. The loadings of items 3, 4, and 5 on the target factor was 0.44–0.74, and the loadings of the other five items on the second target factor was 0.64–0.92. The correlation between the two residuals was 0.73.

Table 2 Model-data fit information from exploratory factor analysis based on subsample 1
Full size table

Confirmatory factor analysis The two-factor model with a residual covariance was subjected to a validation through confirmatory factor analysis using with the second subsample. The first factor was measured by items 1, 2, 6, 7, and 8. The second factor was measured by items 3, 4, and 5. This model fitted adequately to the data (χ2(18) = 486.24, p < 0.01, RMSEA = 0.067 (90% CI 0.062–0.072), CFI = 0.965. The standardized parameter estimates were shown in Fig. 1. The standardized loadings ranged from 0.50 to 0.90. The correlation between the two factors was 0.44.

Fig. 1
figure 1

Standardized Parameter Estimates from Confirmatory Factor Analysis (N = 5843). Item 1 = Poor sleep quality; Item 2 = Sleep dissatisfaction; Item 3 = Difficulty initiating sleep. Item 4 = Difficulty maintaining sleep; Item 5 = Early morning awakening. Item 6 = Sleep insufficiency; Item 7 = Unrefreshing sleep. Item 8 = Daytime functioning impairment

Full size image

The 2-factor model was consistent with our conceptualization that the items were essentially measuring 2 domains of insomnia. Factor 1 consisting of items 1, 2, 6, 7, and 8 are related to daytime distress or impairment. Factor 2 consisting of 3 items measuring DIS, DMS and EMA is defined as insomnia symptoms.

Internal consistency reliability and test–retest reliability

The internal consistency reliability coefficient computed by coefficient omega [40] was 0.82 while Cronbach’s alpha was 0.80. Table 3 shows the test–retest (2-week interval) correlations for individual YSIS items were moderate to high, ranging from 0.42 (EMA) to 0.75 (daytime functioning impairment). The test–retest coefficient of the YSIS total scores was 0.82 (p < 0.0001).

Table 3 Mean item scores and correlations of YSIS 2-week test–retest
Full size table

Criterion validity

Table 4 shows the mean YSIS scores across sleep variables. Mean YSIS scores were significantly elevated in adolescents with shorter sleep duration, hypnotic use, sleep disordered breathing problems, and restless legs syndrome.

Table 4 Criterion validity: comparisons of mean YSIS total scores across sleep variables
Full size table

In addition, the YSIS score was moderately or highly correlated with night sleep duration (r = − 0.325), daytime sleepiness score (r = 0.667), and anxiety/depression score (r = 0.451) (all p < 0.001).

Exploratory cutoffs for insomnia severity

The total YISI score ranged from 8 to 40, with a mean of 19.12 (SD = 6.20) in the sample. Figure 2 displays the distribution of YSIS score and selected percentiles in the sampled adolescents. As shown in Fig. 2, YSIS scores were not normally distributed, with 70% adolescents being scored under 22, 90% under 28 and 95% under 30.

Fig. 2
figure 2

YSIS score distribution in adolescents (n = 10,819). YSIS Youth Self-rating Insomnia Scale

Full size image

ROC curve analysis was performed to demonstrate sensitivity and false positivity at different cutoffs of YSIS scores for predicting reported general sleep disturbance (5–7 days/week). As shown in Fig. 3, the area under the curve was 0.79 (SE = 0.008, 95% CI 0.77–0.80, p < 0.0001), indicating that YSIS was acceptable for discriminating those participants with reported general sleep disturbance. Using 22 (70th percentile), 26 (85th percentile), and 30 (95th percentile) as cutoffs, the sensitivity and specificity were 71% and 75, 48 and 90%, and 19% and 98%, respectively.

Fig. 3
figure 3

ROC curve of YSIS total scores for predicting general sleep disturbance. ROC receiver operating characteristic

Full size image

Based on these percentiles and the results from ROC curve, the possible cutoffs for insomnia severity were recommended below.

$$ \begin{aligned} & {\text{Normal}}:\, < 2 2\;( < 70{\text{th}}\;{\text{percentile}}) \\ & {\text{Mild}}\;{\text{insomnia}}: 2 2\;\left( { 70{\text{th}}\;{\text{percentile}}} \right){-} 2 5\\ & {\text{Moderate}}\;{\text{insomnia}}/{\text{clinical}}\;{\text{insomnia}}: 2 6\;\left( { 8 5 {\text{th}}\;{\text{percentile}}} \right){-} 2 9\\ & {\text{Severe}}\;{\text{insomnia}}/{\text{clinical}}\;{\text{insomnia}}:\, \ge 30\;\left( { 9 5 {\text{th}}\;{\text{percentile}}} \right). \\ \end{aligned} $$

Discussion

Insomnia is prevalent in adolescents. Although several insomnia scales/questionnaires such as ISI [17] and PSQI [20] are available to assess sleep disturbance or insomnia for adults, no insomnia scale has been specifically developed for adolescents. The YSIS is an 8-item self-rated scale for measuring insomnia severity in the past month in adolescents. The current study assessed psychometric properties of the YSIS in a large sample of Chinese adolescents (n = 11,626). The psychometric properties of the YSIS are summarized and discussed below.

First, the YSIS demonstrates satisfactory construct validity as assessed by exploratory and confirmatory factor analyses. The YSIS had 2 factors, one factor involving 3 insomnia symptoms (i.e., DIS, DMS, and EMA) and another factor consisting of 5 items related to insomnia distress or impairment (e.g., dissatisfaction with sleep, unrefreshing sleep, daytime functioning impairment). The 8 items had acceptable standardized loadings ranging from 0.50 to 0.90. The correlation between the two factors was 0.44. The 2 factors support the definitions of insomnia by ICSD-3 [12] and DSM-V [13] (i.e., a subjective report of insomnia symptoms and associated daytime consequences).

Second, the YSIS scale has satisfactory reliability as assessed by internal consistency reliability and 2-week test–retest reliability. The Cronbach’s alpha was 0.80 and coefficient omega was 0.82. The test–retest correlations for individual YSIS items ranged from 0.42 to 0.75. The test–retest coefficient of the YSIS total scores was 0.82.

Third, the YSIS has good criterion validities as assessed by significant correlations with short sleep duration, daytime sleepiness, and anxiety/depression. Significantly higher YSIS scores were also found in adolescents who slept less, had SDB and RLS symptoms, and used hypnotics than those who did not. All the concurrent associations between YSIS and multiple sleep-related problems or consequences are consistent and supportive of the validity of YSIS for assessing insomnia severity in adolescents.

Fourth, the YSIS demonstrates acceptable discriminating properties to differentiate adolescents with general sleep disturbance as assessed by the area under the curve of 0.79 in the ROC curve analysis. There is a chance of close to 80% that the YSIS will be able to distinguish between adolescents with and without general sleep disturbance. Based on ROC curve and sensitivity and specificity analyses, scores of 22, 26, and 30 were proposed as cutoffs of the YSIS total scores for mild, moderate, and severe insomnia for adolescents.

Despite the sound psychometric properties of the YSIS in a large sample of Chinese adolescents, several limitations need to be considered when interpreting the findings. First, the study did not include clinical samples of adolescents or clinical interview of adolescents. Its psychometric properties for clinical use as compared with other measures developed for adults need to be assessed in clinical samples of adolescents with insomnia. Second, EFA and CFA yielded 2 factors (i.e., insomnia symptoms and insomnia distress or impairment). Sleep insufficiency was statistically grouped into factor 2 possibly due to the fact that sleep insufficiency is a subjective perception of sleep associated with both sleep loss and poor sleep quality. Further studies with different samples including adolescent patients with insomnia are needed to validate the 2 factors and their items. Third, the cutoffs were proposed using ROC curve analysis of a community sample based on a single question assessing the participants’ perception of having a sleep problem or not. It is unknown if the cutoffs can be generalized to clinical populations. Further validation with a gold criterion based on clinical diagnosis with clinical samples is needed. Fourth, the measures used to validate the YSIS such as sleep duration, anxiety/depression, RLS and SDB symptoms, and hypnotic use were all self-reports. In addition, only adolescents from mainland China were included in the study, it is unknown if the findings could be generalized to other Chinese adolescents outside of mainland China and to non-Chinese adolescents. Further studies are needed to assess its psychometric properties and utility to adolescents in other countries or regions.

To our knowledge, ISI [17] and PSQI [20] may be the most commonly used scales to assess sleep disturbance or insomnia in epidemiological and clinical studies. However, both ISI and PSQI were developed for adults. PSQI was specifically developed to assess sleep quality rather than insomnia severity. ISI was developed to assess current insomnia severity (i.e., in the past 2 weeks). Three insomnia symptoms (i.e., DIS, DMS and EMA) are rated on a 5-point severity scale from none to very severe [17]. In addition, the item in the ISI, “How NOTICEABLE to others do you think your sleep problem is in terms of impairing the quality of your life?” may be hard to answer by adolescents. Compared with ISI, the YSIS was developed to assess insomnia symptoms and waking symptoms or daytime consequences in the past month and the insomnia symptoms in the YSIS are rated on a 5-point frequency scale from none to almost every day (i.e., 5–7 times/week). Furthermore, the 8 items are easily understood and responded by adolescents.

In summary, the YSIS consists of 8 items that measure insomnia symptoms and daytime consequences as self-reported by adolescents. The scale can be easily administered, scored, and interpreted. A series of reliability and validity analyses with a large sample of adolescents demonstrates that YSIS appears to be a suitable scale to assist clinical diagnosis and to measure insomnia in the past month among adolescents. Further assessment with objective sleep measures and with clinical samples would provide new evidence for its clinical, epidemiological and research use as a brief, reliable and valid tool to assess insomnia in youth.