Obstructive sleep apnea (OSA) in adults is commonly treated with continuous positive airway pressure therapy (CPAP) [1]. While CPAP is recognized as an efficacious treatment for OSA [2], the effectiveness of CPAP is significantly limited by patients’ use of CPAP or adherence [3–5]. Studies that have examined the pattern of CPAP use in newly diagnosed and treated OSA suggest that the first week of home treatment use reliably reflects long-term CPAP use patterns [6–8]. Specifically, defining consistent users as those who use CPAP ≥4 h/night on 70 % of nights and intermittent users as those who fail to meet this criteria, consistent users demonstrate higher rates of adherence at 1, 3, and 12 months [6, 7]. Early CPAP use as a predictor of long-term CPAP use has been replicated in other studies [5, 8], and these studies suggest that patients establish a pattern of CPAP use during the earliest period of home CPAP treatment by days 2 to 4 of treatment [6, 7] and remain consistent in that pattern of use for the long-term treatment period. It is therefore critical to reduce the incidence of early intermittent patterns of CPAP use that are not conducive to long-term effective treatment of OSA.

Many studies have sought to identify baseline or pre-treatment characteristics or factors as predictors of CPAP nonadherence [9]. OSA severity or the Apnea-Hypopnea Index (AHI) and subjective sleepiness are relatively consistent, yet weak pre-treatment predictive factors that have emerged from this area of study [3]. After 1 week or more of home CPAP treatment, more robust predictors of CPAP adherence have emerged. These factors include spousal or immediate social support for treatment use/troubleshooting [10, 11], psychological factors such as treatment self-efficacy, or the belief in one’s ability to use a challenging treatment [12–15] and initial or early problems with CPAP [10, 16]. Although these factors are seemingly influential on CPAP adherence when measured at 1 week, the pattern of CPAP use is likely well-established by this point in treatment. Several, relatively small, exploratory studies have identified the initial exposure to CPAP and experience with CPAP during titration polysomnogram (PSG) may be consequential to early CPAP use (i.e., adherence) [10, 16]. As factors of influence at baseline or prior to any treatment exposure are not consistently robust predictors of short- or long-term CPAP adherence and CPAP adherence behaviors are established within the first week of home treatment, we sought to examine the first exposure to CPAP (i.e., CPAP titration PSG) as a potential opportunity to prospectively identify adult OSA patients’ risk for CPAP nonadherence.

We hypothesized that a critically timed risk screening questionnaire, employed immediately after CPAP titration PSG, will reliably identify adults with newly diagnosed OSA who are at risk for subsequent nonadherence to CPAP at 1-month. A prospective, longitudinal study was conducted to address the following specific objectives: (1) test a risk screening questionnaire to prospectively identify CPAP nonadherence risk among adults with newly diagnosed OSA, (2) reduce the questionnaire to a minimum set of items that effectively identifies 1-month CPAP nonadherence, and (3) examine the diagnostic utility of the screening index.

Methods

A prospective longitudinal study was conducted at two accredited clinical sleep centers in the USA. One center was a suburban, community-based clinical center. The second was a large, academic clinical sleep center located in a suburban setting. The study was approved by the respective Institutional Review Boards (IRB).

Participants

Consecutive newly diagnosed OSA patients were recruited for study participation. Inclusion criteria were the following: (1) newly diagnosed OSA with AHI ≥5 events/h on in-laboratory PSG conducted and scored in accordance with standard criteria [17], (2) referral to CPAP titration PSG, and (3) able to speak and read English. Exclusion criteria were the following: (1) supplemental oxygen or bilevel positive airway pressure required during titration PSG, (2) new diagnosis of psychiatric disorder within previous 6 months prior to study enrollment, and (3) any medical contraindication to using CPAP for treatment of OSA. All potential participants were CPAP-naïve, with no previous exposure or experience with CPAP prior to study enrollment. Initial pre-enrollment screening was conducted by a study investigator, and informed consent was scheduled and completed thereafter prior to CPAP titration PSG (n = 102). The majority of participants with complete data (n = 97) were from one site (n = 89, 91.8 %); the second site, a community-based clinical sleep center, referred and enrolled eight participants (8 %) during the 18-month study period. In total, five participants were lost to follow-up (2), refused CPAP during titration PSG (2), or did not return for titration PSG (1).

Procedures

After informed consent, baseline measures including self-reported demographics, medical record extraction of medical history, and OSA and physical characteristics from diagnostic PSG were collected. Participants then completed an in-laboratory, full-night CPAP titration PSG conducted and scored according to standard procedures [1, 17]. Prior to participants’ diagnostic PSG, all patients received usual care procedures for patient education, including a pamphlet about OSA and treatment options and an educational video that provided information about OSA, PSG, treatment options, CPAP titration, and CPAP treatment. Following the titration PSG, participants completed a risk screening questionnaire, Index for Nonadherence to Positive Airway Pressure (I-NAP). Initiation of CPAP treatment was managed by the respective clinical sleep provider, which included device-specific instruction by a home medical equipment provider at the time of CPAP initiation for home treatment, access to sleep center personnel for troubleshooting and questions by telephone, and a scheduled follow-up visit with a sleep center provider after at least 30 days of CPAP treatment. Study participants returned to the sleep center for a scheduled research visit after 1-month (i.e., 30 days) of home CPAP treatment, in all cases, prior to the scheduled clinical visit with a provider.

I-NAP was developed by the investigators based on previous work and the extant literature. The risk screening questionnaire included previously validated instruments that were publicly available or permission was obtained from license holder/developer. No alterations to the instruments were made for the conduct of this study. All instruments included in I-NAP have established validity and reliability for being employed in studies of OSA and CPAP-treated OSA. The instruments addressed the following evidence-based factors of influence on CPAP adherence: subjective sleepiness, daily function, health literacy, sleep-related symptoms, OSA/CPAP knowledge, perceived risks of OSA, treatment expectations, CPAP self-efficacy, and social support. The risk screening questionnaire was comprised of 111 items, inclusive of 14 subscales and 14 independent items (self-reported patient characteristics, physical characteristics, symptoms, and health literacy). The following measures were included in the full risk screening questionnaire:

ESS

The Epworth Sleepiness Scale (ESS) measures subjective sleepiness [18] and includes eight items with response range of 0–3. A total score range of 0–24 indicates subjective sleepiness experienced with higher scores indicating more daytime sleepiness than lower scores.

FOSQ

The Functional Outcomes of Sleep Questionnaire (FOSQ) measures sleep-related functional impairment [19] that includes 30 items with a response range of 1–4. The total score, or global FOSQ score with a range of 5–20, based on sum of sub-scale scores, suggests overall daily functional impairment, with lower scores equating to more dysfunction. Five FOSQ subscales include activity, general productivity, social outcome, vigilance, and intimacy and sexual activity. Each FOSQ subscale has an item response range of 1–4, with lower scores equating to more dysfunction for the subscale domain. Weighted means are calculated for each subscale score, and the global FOSQ score is a total score of subscale scores.

HLS Questionnaire

The Health Literacy Screening Questionnaire (HLS) is a general assessment of health literacy, not disease-specific [20]. The screening questionnaire is comprised of three items, each with a score range of 0–4, with 0 indicating no health literacy difficulty and 4 indicating significant difficulty for health literacy. The total score range is 0–12. Individual HLS items, dichotomized at specific cut points, have demonstrated sensitivity and specificity for determining risk of inadequate health literacy [20]. HLS item 1 (HLS1) addresses problems with learning health-related information. HLS item 2 (HLS2) addresses self-confidence with completion of health-related forms. HLS item 3 (HLS3) addresses problems with reading [20]. To our knowledge, no previous studies have examined health literacy as an influential factor on CPAP adherence. Therefore, the total HLS score and individual items for inadequate health literacy risk were examined.

Sleep symptoms

A single item measured sleep symptoms for which participants sought care at a sleep center. Respondents selected all sleep-specific symptoms that were applicable from a response set of five options.

Social Cognitive Theory Questionnaire in OSA

The questionnaire includes four subscales: knowledge of OSA and CPAP (12 items), outcome expectancies with treatment (4 items), treatment-specific self-efficacy (5 items), and social support (9 items) [13]. Each subscale score range is 1–5 with exception of the knowledge subscale, which is scored percent of correct responses based on true/false responses.

SEMSA

The self-efficacy measure in sleep apnea (SEMSA) is a disease-specific instrument that includes three subscales that measure risk perception (8 items), outcome expectancies (9 items), and treatment-specific self-efficacy (9 items) [21]. The response range for each item is 1–4. A weighted mean is calculated for each subscale; no total score is calculated for the SEMSA.

Study variables, outcomes, and measures

Participant characteristics

A questionnaire eliciting demographic information and presenting symptoms was completed by all study participants at enrollment. Weight and height for calculation of body mass index (BMI) were extracted from the diagnostic PSG.

Diagnostic PSG

Diagnostic PSG data was collected at study enrollment. Studies were scored based on standard criteria [17] employing the alternative hypopnea definition, a 50 % decrement in nasal pressure of 10 s or more and ≥3 % desaturation [17]. Variables included AHI, oxygen nadir in non-REM, and total sleep time ≤90 % (minutes and %).

I-NAP

Risk screening questionnaire was administered on a single occasion immediately after CPAP titration in the laboratory. The questionnaire, as described above, was completed by participants, on average, in 15 min. Only instrument subscales (14), total instrument scores (2), and 14 individual descriptive items were statistically examined. I-NAP is assessed at a 5th grade reading level.

CPAP use

Objective 1 month CPAP use was collected using the internal microprocessor on standard CPAP devices. Use was recorded as mean hours/night at effective pressure >20 min.

Analysis

All variables, including screening questionnaire subscale scores and scale total scores, if appropriate were summarized using standard descriptive statistics. All potential predictors with a priori-defined level of CPAP nonadherence <4 h mean CPAP use (clinical benchmark) were entered as predictors in corresponding logistic regression models. CPAP use was categorized as nonadherent at <4 h ([n = 38; 39.2 %] or adherent at ≥4 h (n = 59; 60.1 %).

Terms were removed from the model using a backward elimination procedure, deleting factors wherein removal caused an insignificant change in the predictive ability of the model as determined by Wald statistic. Variables included in the model were restricted such that there were at least 10 cases of nonadherence/adherence for each variable in order to minimize type I errors in variable selection. A final logistic regression model with receiver operating characteristic curve (ROC) was produced for CPAP adherence level of <4 h use per night. A regression equation was derived from the final logistic regression model to produce a composite score which was evaluated as a diagnostic test. An additional logistic regression model was then fit containing only the composite score as a predictor, and a corresponding ROC curve was produced. The area under the curve (AUC) of this ROC curve was compared to the nominal value of 0.5 of an uninformative model within the logistic regression procedure. In addition, a simplified composite score was estimated from the regression coefficients rounded to one decimal place. The corresponding ROC curve from the simplified composite score was compared to that of the original composite score to determine whether the simplified version could be applied in practice. The comparison of the AUC of the two ROC curves was also performed within the logistic regression procedure.

Multiple test cutoff points were then assessed for the composite score, and each of these cutoff points was associated with a true-positive rate (TPR) and false-positive rate (FPR) based on actual outcome. ROC curves graphically describe the relationship between the choice of cutoff and the associated test characteristics. The AUC associated with the ROC curve measures the ability to discriminate between nonadherers and adherers. The ROC curve and respective sensitivity, specificity, TPR, and FPR were used to choose the optimal cutoff point for each test.

Because the division of the data into training and validation data sets would have resulted in an unacceptably small training data set, cross-validation was used, providing an unbiased assessment of the model without reducing the training data set. Cross-validation was performed within the logistic regression model by ignoring one observation at a time and using the remaining observations to compute the predicted probability for the ignored observation. The cross-validated predicted probabilities were then used in an ROC analysis.

To determine the required sample size for the study based on having adequate statistical power to determine that a predictive model for nonadherence at 1 month was more sensitive than chance, an AUC of at least 0.70 was considered significant. Based on prior work, we assumed a 50 % nonadherence rate, 80 % power, and alpha = 0.05. In order to determine if one model was at least 25 % more accurate than another with ROC area under the curve of 0.70, a sample size of 82 subjects was required. Our previous experience enrolling similar participants in studies of CPAP adherence guided our application of a 20 % attrition rate, resulting in 102 enrolled subjects to achieve the required sample size of n = 82 with complete data.

Results

Sample description

The sample (n = 97) included predominantly males (54.6 %) and females (45.4 %), middle-aged (49.5 ± 11.6 years), with severe OSA (AHI 36.8; SD 19.7 events/h; Table 1). Participants were married (69.1 %), educated at high school level or college (96.9 %), employed part-time or full-time (74.2 %), and predominantly self-identified their race as white (86.6 %) or black (6.2 %). The majority of the sample was referred for sleep evaluation by a family physician/primary care provider (66 %), pulmonary specialist (13.4 %), or cardiologist (5.2 %). Mean 1 month CPAP use was 4.25 h/night (SD 2.35).

Table 1 Sample description
Full size table

I-NAP Risk Assessment Questionnaire

The risk assessment for nonadherence to CPAP questionnaire included Epworth Sleepiness Scale total score, Functional Outcomes of Sleep Questionnaire total score and five subscale scores, three subscale scores for the Self-efficacy Measure in Sleep Apnea, Health Literacy Screening Questionnaire total score and three individual items, and four sub-scale scores for Social Cognitive Theory Questionnaire for OSA. Individual instrument total scores and subscores are reported in Table 2.

Table 2 Risk assessment scores by domain total scores and/or subscale scores (n = 97)
Full size table

Item selection for I-NAP

Bivariate associations of participant characteristics and risk assessment questionnaire total scores and/or subscale scores were examined with 1 month CPAP use. Mean CPAP use was dichotomized at <4 h/night. Variance inflation factors (VIF) were examined to remove variables based on collinearity for the CPAP use model, and criteria for 10 cases per adherence category were examined. These diagnostic procedures resulted in the removal of FOSQ total score and HLS total score from the model (collinearity); rotating shift was also removed from the model (case number criteria). A logistic regression model for <4 h/night CPAP use was reduced by backward stepwise selection and resulted in eight significant subscale scores or characteristic variables remaining in the final <4 h/night CPAP use I-NAP model (Table 3).

Table 3 Logistic regression model for <4 h/night CPAP use I-NAP (n = 97)
Full size table

I-NAP <4 h/night CPAP use

The final I-NAP predictors for <4 h/night of CPAP use included SEMSA outcome expectancy, Social Cognitive Theory Questionnaire (SCT) self-efficacy, HLS1, BMI, marital status, sleepiness as a presenting symptom with the referent group being absence of symptom, restlessness during sleep as a presenting symptom with the referent group being absence of symptom, and gender, with male as the referent group. The final model, inclusive of the eight variables, is well-fit (likelihood ratio test [LR], X 2[8] = 34.67, p < 0.0001).

I-NAP <4 h/night CPAP use sensitivity and specificity

The ROC curve was examined for predictive utility of the reduced I-NAP model for the CPAP adherence cut point of <4 h/night in the study sample. With an AUC of 0.83 (95 % CI 0.74–0.91), the I-NAP demonstrates good diagnostic utility in terms of differentiating those likely to be nonadherent at <4 h/night after 30 days of CPAP treatment. A gamma value of 0.66 for the final ROC suggests that 66 % fewer errors are made employing the model to identify “nonadherers” at the 4 h cut point than by chance alone. To examine sensitivity and specificity, the I-NAP regression equation for the 4-h cut point model was produced:

$$ \begin{array}{l}\mathrm{INAP}=\left(-0.8\times \mathrm{meanSEMSAO}\right)-\left(0.6\times \mathrm{meanSCTSE}\right)-\left(0.6\times \mathrm{HLS}1\right)+\left(0.05\times \mathrm{BMI}\right)-\left(1.1\times \mathrm{married}01\right)-\hfill \\ {}\left(1.5\times \mathrm{sleepiness}\right)+\left(1.1\times \mathrm{restless}\ \mathrm{sleep}\right)+\left(0.7\times \mathrm{female}\right)\hfill \end{array} $$

where meanSEMSAO is the mean SEMSA outcome expectancy subscale score, meanSCTSE is the mean social cognitive theory in sleep apnea self-efficacy subscale score, HLS1 is the health literacy screening question 1 score, BMI is the body mass index, married01 is a binary variable (unmarried = 0; married = 1), sleepiness is a binary variable (sleepiness as presenting symptom, absent = 0; present = 1), restless sleep is a binary variable (restlessness during sleep as presenting symptom, absent = 0; present = 1), and female is a binary variable (male = 0; female = 1). I-NAP composite score is well-fit (X 2[1] = 20.25, p < 0.0001) with an OR of 2.75 (95 % CI 1.77–4.26). Employing only the I-NAP composite score, a second receiver operating characteristic curve demonstrated AUC 0.83 (95 % CI 0.74–0.91; p < 0.001 vs. noninformative model). The estimation of the cumulative score with coefficients rounded to one decimal place was not significantly different from the full expression of the predicted probabilities (p = 0.71), which supports the use of the cumulative score expression shown above. The cross-validation procedure produced an ROC curve with a similar AUC of 0.81 (95 % CI 0.72–0.90). As this is a diagnostic screening instrument, an optimal cut point that maximized sensitivity while preserving specificity was desired, which resulted in a cut point for I-NAP ≥−4.8 (Fig. 1). At this I-NAP optimal cut point for identifying adults with risk of CPAP adherence <4 h/night after 30 days of treatment, sensitivity was 87 % and specificity was 63 % (Table 4).

Fig. 1
figure 1

Predictive utility of the I-NAP composite score for <4 h/night CPAP use (n = 97). ROC receiver operating characteristic curve; AUC = 0.83 (95 % CI 0.74–0.91); Somers’ D = 0.65

Full size image
Table 4 I-NAP cut point ≥ −4.8 predictive utility (n = 97)
Full size table

Discussion

Our prospective, longitudinal study sought to test, reduce, and examine the predictive utility of a risk screening questionnaire employed after first exposure to CPAP among CPAP-naïve adults with OSA to identify subsequent risk of CPAP nonadherence at 1 month. To our knowledge, no previous published studies have prospectively tested a simplistic mechanism, such as a screening questionnaire, to identify newly diagnosed OSA adults who are likely to be nonadherent to subsequent CPAP treatment. I-NAP screening questionnaire for CPAP use <4 h/night reliably identified subjects in our study at risk for CPAP nonadherence at 1 month and has good sensitivity (i.e., >0.80). I-NAP was reduced to eight measured domains, including outcome expectancies, treatment self-efficacy, health literacy, BMI, marital status, symptoms on presentation, and gender, consisting of a total of 19 individual questionnaire items (Table 5).

Table 5 I-NAP items and subscales
Full size table

Screening tests or instruments are commonly used when (1) the gold standard test or instrument is invasive, expensive, or generally inaccessible; (2) assessment of risk is cost-effective and/or reduces morbidity and/or mortality; and/or (3) when prevalence of the condition is common and treatment is available [22]. As nonadherence to CPAP is estimated at 50 % [3] and contributes to persistent poor health and functional outcomes in OSA [4], a screening instrument for nonadherence to CPAP is a reasonable approach to improve cost-effective OSA care and lessen morbidity associated with nonadherence. It is well-established that CPAP nonadherence patterns develop in the earliest phase of CPAP treatment, and these usage patterns are reflective of long-term adherence outcomes [6–8]. For this reason, it is imperative to identify CPAP nonadherence at the earliest possible point in the course of treatment. Testing of the I-NAP screening questionnaire immediately after first exposure to CPAP, i.e., CPAP titration, supports identification of nonadherers before these nonconducive patterns are established and is also consistent with evidence that suggests early CPAP experiences are influential on initial CPAP acceptance and early patterns of use [8, 16, 23]. As the purpose of the I-NAP is to prospectively identify adults with risk of subsequent nonadherence to CPAP, sensitivity or true-positive case identification was prioritized when determining the I-NAP cutoff point with careful consideration of maintaining a moderate level of specificity, or true-negative case identification, greater than 60 %. A negative predictive value >85 % was identified for the optimal screening cut point, suggesting that the risk screening questionnaire effectively identifies cases that are true negatives based on objective CPAP use at 1 month.

Similar to our own findings, Balachandran and colleagues [24] retrospectively examined the predictive utility of a CPAP perception survey as a brief screening tool to predict CPAP nonadherence at 1 month. The survey procedure was initially designed as a quality improvement project. Assessing similar domains as the I-NAP, with several items addressing CPAP treatment self-efficacy and outcome expectancies and collecting survey responses immediately after CPAP titration PSG, Balachandran and colleagues’ study [24] and our own study provide preliminary evidence that risk assessment surveys for CPAP nonadherence are a valid and clinically feasible approach to determining which patients require CPAP adherence intervention. Though prior reported research has applied and tested a similar strategy of risk assessment in a retrospective fashion and successfully identified adherers [25], our own study and Balachandran and colleagues’ study importantly differentiated both adherers and nonadherers.

This preliminary study is not without limitations. The clinical sample characteristics included predominantly high school or higher educated subjects, minimal representation of diversity in terms of race and ethnicity, and a relatively low representation of absolute nonadherers (i.e., 0 h use). These limitations may have implications on the overall predictive validity of I-NAP in general sleep clinical settings. Future testing of the I-NAP in a larger sample and across multiple sites to increase sample heterogeneity is needed. For the purpose of validating the I-NAP, a second validation sample or subsample from the full sample would ideally have been included in the initial preliminary testing. We conducted a priori power analysis that suggested the retained sample size, n = 97, was adequate to test for statistically significant differences in ROC-determined I-NAP score cut points. We also conducted a cross-validation procedure to assess the model fit in order that the sample size would not be reduced for validation purposes. As this study was designed as a preliminary study to develop and test a simplistic risk screening instrument, additional predictive utility testing is recommended. Such studies will necessarily include a validation sample to better delineate the clinical utility of the I-NAP in the care of CPAP-treated OSA.

Several studies have examined the cost-effectiveness of CPAP treatment for OSA in adults [26–28]. When quality of life, costs of therapy, and motor vehicle crashes were considered, CPAP was a highly efficient use of health-care resources; CPAP use less than 4 h per night on 70 % of nights, modeled as no CPAP, was not cost-effective. Based on the principles of resource allocation and cost-effectiveness, combined with our current understanding of the efficacy of CPAP in OSA, risk assessment for CPAP nonadherence is a simplistic, economical, and low-burden approach to potentially contribute to improved outcomes and resource utilization in the OSA population. Furthermore, as is recommended by the current clinical guidelines for diagnosis, treatment, and long-term management of OSA in adults [29], implementation of an “early compliance program” may be facilitated by risk screening of OSA patients after CPAP titration PSG, at the earliest phase of treatment exposure, to permit targeted delivery of compliance programs to those adults identified as at risk for CPAP nonadherence.

Building upon a relatively large body of evidence addressing CPAP nonadherence and influential factors on short-term CPAP adherence outcomes, we developed a risk assessment for CPAP nonadherence in adults with OSA. I-NAP, identifying CPAP use <4 h/night at 1 month, includes a minimum set of items that address multiple domains of recognized influence on CPAP adherence and is both simplistic and consistent in delivery with standardized care intervals at sleep centers. By incorporating a validated risk assessment of nonadherence to CPAP prior to treatment initiation, the chronic care of CPAP-treated OSA in adults may be improved. Specifically, clinical decision-making and resource allocation may be enhanced by early identification of adults most likely to require targeted or personalized interventions to improve CPAP adherence.