Introduction

There is evidence that the profile of substance use behaviors and substance use disorders (SUDs) among individuals entering substance abuse treatment has changed dramatically in the past two decades according to the Treatment Episode Data Set, which collects data on admissions to U.S. substance abuse treatment facilities [1, 2]. More specifically, there has been a significant shift in the primary substances of abuse observed among those entering substance abuse treatment programs in the U.S. over the past two decades. The percentage of substance abuse treatment admissions reporting alcohol as the primary substance of abuse decreased from 57 % in 1993 to 41 % in 2010, while the percentage of substance abuse treatment admissions for marijuana, opiates, and stimulants as the primary substance increased from 22 % in 1993 to 47 % in 2010 [1, 2]. These shifts suggest the importance of obtaining accurate estimates of substance use behaviors and SUDs in the U.S. general population.

There is a general trend toward decreasing response rates in population surveys [3], and selective nonresponse represents a major source of potential bias in survey-based estimates of drug use behaviors. This is especially true for estimates of change in the prevalence of these behaviors and disorders based on longitudinal designs. Therefore, the degree to which different types of substance use behaviors and SUDs among survey respondents are representative of the U.S. general population has important implications for screening, prevention efforts, and substance abuse treatment programs. Previous population surveys have found evidence that alcohol use behaviors were significantly associated with attrition in cross-sectional and longitudinal designs [4–6]. For instance, Dawson and colleagues [4] recently examined whether alcohol consumption differed for respondents and nonrespondents in a longitudinal study. These authors found that the degree to which the prevalence of drinking was underestimated in the total U.S. adult population was 1.6 %, while the extent to which alcohol consumption was overestimated ranges from 1.7 to 2.4 % among drinkers, after correcting for their sociodemographic differences. Dawson and colleagues [4] concluded that the extent of nonresponse bias in estimates of alcohol use behaviors should not affect drinking guidelines and planning for prevention and treatment programs.

At least two other longitudinal investigations have found evidence that alcohol use was not significantly associated with attrition while marijuana and other drug use was an important source of nonresponse bias in longitudinal surveys of young adults in the U.S. military [7] and U.S. college students [8]. However, there have been no nationally representative longitudinal studies of the U.S. adult general population examining the potential nonresponse bias associated with substances other than alcohol. This gap in knowledge needs to be addressed, because non-college and non-military young adults tend to have much higher rates of illicit and nonmedical drug use than their college-attending and military counterparts [9–11]. Thus, investigations are needed to assess potential nonresponse bias associated with estimates of the prevalence of substance use behaviors and SUDs for substances other than alcohol based on prospective nationally representative samples, and especially those that take into account frequency of substance use behaviors. The main objective of this study is to examine whether substance use behaviors measured at the baseline wave of data collection in a nationally representative U.S. population survey are associated with the likelihood of re-interview 3 years later at a follow-up wave, and examine the sensitivity of inferences regarding substance use behaviors to potential differential attrition.

Methods

This study used the 2001–2002 National Epidemiologic Survey on Alcohol and Related Conditions (NESARC, Wave 1) and 2004–2005 NESARC (Wave 2) as the primary datasets to obtain information regarding substance use behaviors and SUDs among the general population in the United States. The target population for the 2001–2002 NESARC (Wave 1) was the civilian, noninstitutionalized population residing in the United States, 18 years of age and older. Wave 1 and 2 data were collected via face-to-face interviews conducted in respondents’ households. The Wave 1 target population included persons living in households, military personnel living off base, and persons residing in certain group quarters: boarding or rooming houses, nontransient hotels, shelters, facilities for housing workers, college quarters, and group homes. The NESARC used a multistage stratified sample design, and had two sampling frames: one for housing units and one for group quarters. The United States Bureau of the Census trained interviewers using the Alcohol Use Disorder and Associated Disabilities Interview Schedule DSM-IV (AUDADIS-IV). This fully structured diagnostic interview was completed via face-to-face personal interviews. The response rate for Wave 1 was 81.0 %; the response rate among those eligible for Wave 2 was 86.7 %, resulting in a cumulative response rate of 70.2 % (the product of the response rates from Waves 1 and 2). More details about the NESARC sample design and data collection methods for both Waves 1 and 2 are available elsewhere [12, 13]. The United States Census Bureau and the United States Office of Budget and Management approved the NESARC research protocol. The University of Michigan Institutional Review Board approved the present study.

Sample

At Wave 1, the sample consisted of 43,093 adults 18 years of age and older, and represented a population that was 52 % women, 71 % White, 12 % Hispanic, 11 % African American, 4 % Asian, and 2 % Native American or another racial category. In addition, an estimated 20 % of the population lived in the Northeast, 35 % in the South, 23 % in the Midwest, and 22 % in the West. Approximately 12 % of the population were 18–24 years of age and 88 % were adults 25 years of age or older. The mean age of the Wave 1 sample (unweighted) was 46.4 (SD = 18.2), and the corresponding weighted estimate of the mean age of the target population was 45.2 (SE = 0.2). Without adjusting the Wave 1 sampling weights for attrition, the respondents in both waves represented a population that was 53 % women, 73 % White, 10 % Hispanic, 11 % African–American, 4 % Asian, and 2 % Native American or another racial category, with a similar regional distribution. After applying weighting adjustments for attrition from Wave 1, the sample responding in both waves (n = 34,653) represented a population with the same gender, region, and race/ethnicity distributions from Wave 1, and an estimated mean age of 44.9 (SE = 0.2).

Measures

Demographic and background characteristics

Demographic and background characteristics collected at the Wave 1 interview included age, sex (male, female), race/ethnicity (white, black, Native American, Asian, Hispanic), educational level (less than high school, high school, some college or higher), employment status (full-time, part-time, not working), income (less than $19,999, $20,000–$34,999, $35,000–$69,999, $70,000 or higher), United States Census geographical region (Northeast, Midwest, South, and West), metropolitan statistical area (MSA) (central city in MSA, not in central city in MSA, not in MSA), and relationship status (married/cohabiting, widowed/separated/divorced, never married).

Past-year substance use behaviors

Alcohol use was assessed at each wave by asking respondents if during the last 12 months they had at least one drink of any kind of alcohol (e.g., beer, champagne, coolers, wine, and liquor), not counting small tastes or sips. Other substance use behaviors were assessed at each wave by asking respondents about use of a variety of medications and other drugs without a doctor’s prescription. Respondents were asked about use of medications in the last 12 months that were not prescribed to them by a doctor or that were used in a manner not intended by the prescribing clinician (e.g., more often than prescribed, longer than prescribed, or for a reason other than prescribed, such as to get high). Substances included marijuana, cocaine or crack, heroin, hallucinogens, inhalants or solvents, prescription opioids, sedatives, stimulants, and tranquilizers or anti-anxiety drugs.

Past-year substance use disorders

Substance use disorders at each wave were based on DSM-IV criteria from the AUDADIS-IV, which contains symptom questions used to operationalize DSM-IV abuse and dependence separately for all ten substances (alcohol, marijuana, cocaine, heroin, hallucinogens, inhalants, prescription opioids, sedatives, stimulants, and tranquilizers). A past-year diagnosis of abuse requires the absence of a diagnosis of dependence and at least one of four DSM-IV abuse criteria in the 12 months preceding the interview. A past-year dependence diagnosis is based on presence of at least three of the seven DSM-IV dependence criteria. The diagnoses of abuse and dependence were combined to create binary substance use disorder outcomes. Reliability and validity of the DSM-IV AUDADIS-IV substance use disorder diagnoses have been documented in numerous psychometric studies [14–23], with test–retest reliability ranging from good to excellent (0.70–0.91).

Data analysis

This study compared baseline substance use behaviors among initial respondents who did respond (n = 34,653) and did not respond (n = 8440) to a 3-year follow-up interview in a prospective study of the general U.S. adult population [12]. Differences in nonresponse rates were assessed as a function of past-year substance use behaviors both before and after adjustment for socio-demographic differences potentially associated with these behaviors, and the effects of interactions of the substance use behaviors with the socio-demographic characteristics were assessed in multivariate logistic regression models for a nonresponse indicator at the 3-year follow-up.

We considered three design-based logistic regression models for the indicator of nonresponse at Wave 2, accounting for the complex sample design features of the NESARC (i.e., using the Wave 1 weights to estimate the model parameters, and the NESARC stratum and cluster codes to estimate sampling variability). This approach enabled us to make representative statements about the expected response behaviors of the larger NESARC population. The initial model only included socio-demographic covariates (age, sex, race/ethnicity, education, and marital status). The second model included categorical variables measuring the self-reported frequency of selected past-year substance use behaviors from Wave 1 (alcohol, marijuana, cocaine, stimulants, opioids, hallucinogens, sedatives, and tranquilizers), in addition to binary indicators of any past-year use of inhalants or heroin. The third model then examined two-way interactions between the substance use measures and the socio-demographic predictor variables; only significant two-way interactions according to a design-adjusted Wald test [24] were retained in the third model. Each model was tested for goodness of fit using design-based methods described by Archer and colleagues [25].

Finally, to examine the possibility that there may be unique subclasses of substance users at Wave 1 based on these frequency measures and indicators, we performed an exploratory latent class analysis (LCA) using all of these categorical substance use measures at Wave 1, and identified the latent class solution with the optimal fit based on likelihood ratio tests and the BIC criterion. We then grouped Wave 1 respondents into subclasses based on the latent class with the highest posterior predicted probability for each individual, described the resulting subclasses, and compared these subclasses in terms of nonresponse prevalence at Wave 2.

Results

Wave 2 nonresponse rates by Wave 1 substance use behaviors and gender

Table 1 presents both weighted and unweighted nonresponse rates at NESARC Wave 2 for different subgroups of individuals defined by past-year drug use behaviors at Wave 1. Unweighted nonresponse rates varied from 17.45 % (for past-year users of alcohol) to 27.65 % (for past-year cocaine users). While the nonresponse rate for cocaine users is based on a smaller subsample of individuals (n = 217), this does suggest that slight negative biases may be present for Wave 2 estimates describing correlates of cocaine use if Wave 1 cocaine use is not used in computing nonresponse adjustments. The weighted estimates, representing expected nonresponse rates if everyone in the target population had been included in the sample, tracked the unweighted response rates fairly closely, ranging from 13.56 % (for past-year heroin users at Wave 1, with n = 11 only) to 24.08 % (for past-year cocaine users at Wave 1).

Table 1 Nonresponse rates at 3-year follow-up, by baseline substance use behaviors in the U.S.
Full size table

Table 1 also presents design-based comparisons of nonresponse rates for males and females, among each subgroup of users defined by Wave 1 behaviors. Significant differences were found between males and females among Wave 1 users of: (1) any substance (p < 0.001, with males having a higher nonresponse rate at Wave 2), (2) alcohol (p < 0.001, with males having a higher nonresponse rate at Wave 2), and (3) two or more substances (p < 0.05, with males having a higher nonresponse rate at Wave 2).

We replicated the Table 1 analyses for different subgroups defined by substance use disorders (SUDs) in the past year at Wave 1 (results not shown). In general, we found evidence of higher nonresponse rates for subgroups of individuals having various SUDs at Wave 1. This was especially true for those with a cocaine use disorder (n = 97; weighted nonresponse rate = 26.38 %, unweighted nonresponse rate = 28.87 %), those with a tranquilizer use disorder (n = 51; nonresponse rate = 27.45 % regardless of whether weights were used), and those with SUDs associated with two or more substances (n = 462; weighted nonresponse rate = 23.16 %, unweighted response rate = 23.22 %). In addition, we found evidence of males once again having higher nonresponse rates among individuals with any SUDs at Wave 1, individuals with alcohol use disorders, and individuals with marijuana use disorders, suggesting that longitudinal estimates of change in drug use behaviors for these subgroups may be biased in the direction of the change for females.

Wave 2 nonresponse rates by frequency of Wave 1 substance use behaviors

Table 2 presents weighted and unweighted estimates of nonresponse rates at NESARC Wave 2 for different subgroups of individuals defined by frequency of substance use behaviors in the past year at Wave 1. Substances with notable associations between frequency of use at Wave 1 and nonresponse at Wave 2 included alcohol, where daily drinkers were found to have the highest nonresponse rate; cocaine, where daily and annual users at Wave 1 had relatively higher rates of nonresponse at Wave 2; and stimulants and tranquilizers, where after incorporating the NESARC weights into the estimation, daily and weekly users had a higher nonresponse rates at Wave 2. The general patterns of differences in nonresponse rates were not particularly sensitive to the use of the NESARC weights in estimation, but individual estimates of nonresponse rates did in some cases vary substantially with and without the use of weights in estimation (e.g., cocaine and stimulant use). This suggests that the NESARC weights may be informative about response behavior for selected subgroups, and should thus be used when estimating nonresponse rates for the larger populations represented by these subgroups.

Table 2 Nonresponse rates at 3-year follow-up, by baseline frequency of past-year substance use behaviors in the U.S.
Full size table

Logistic regression models predicting nonresponse at Wave 2

Table 3 presents results from the multivariate logistic regression analyses, where an indicator of nonresponse at Wave 2 was predicted by various socio-demographics and indicators of different drug use behaviors at Wave 1. Notably, several two-way interactions between socio-demographic covariates and the past-year substance use indicators at Wave 1 emerged as significant predictors, so we focus our discussion on Model 3, which was found to have an improved fit relative to the other models according to the smaller value of the design-based goodness-of-fit test statistic proposed by Archer and colleagues [25].

Table 3 Estimated coefficients in design-based logistic regression models predicting nonresponse at NESARC Wave 2 as a function of demographics and Wave 1 substance use behaviors (n = 43,030 with non-missing data on all variables, for all models)
Full size table

In general, we found significant two-way interactions involving age, race/ethnicity, education, and alcohol use at Wave 1. Among individuals who did not use alcohol at Wave 1, older individuals, Asians, Hispanics/Latinos, and those with less-than-high school education tended to have higher rates of nonresponse (holding other covariates fixed). The age effect was significantly reduced for different types of users, becoming essentially zero for all four different groups of users defined by frequency of use. A negligible difference in nonresponse rates between black and white persons among non-users became substantially larger for annual users, where blacks had a substantially higher nonresponse rate. In addition, the already positive effects of being Asian and Hispanic/Latino on nonresponse rates became even stronger among monthly users, suggesting that more frequent alcohol users in these demographic groups have an increased risk of being lost over time in longitudinal data collections. Finally, a non-significant negative effect of having more than High School education on nonresponse among non-users becomes even stronger and significant among annual, monthly, and weekly users of alcohol, suggesting that past-year users of alcohol with more than High School education tend to have an increased probability of responding relative to other more frequent users with less education. These effects are illustrated via plots of marginal predicted probabilities of nonresponse in Fig. 1 (see supplementary files online).

Predicted probabilities of nonresponse at 3-year follow-up, two-way interactions (Fig. 1, Note: Supplemental files for online publication only).

In addition to the interaction effects found in Model 3, we also find very strong main effects of sex and marital status, where being married tends to reduce the probability of nonresponse at Wave 2, while being male substantially increases the probability of nonresponse at Wave 2. We also find that daily users of cocaine and prescription stimulants have substantially increased odds of not responding at Wave 2, holding all other covariates in Model 3 fixed. In general, these results supported initial design-based tests of bivariate associations between the different socio-demographic predictors and the indicator of nonresponse at Wave 2 (results not shown) and the tests of association presented in Table 2.

Latent class analysis results

The optimal latent class solution (Entropy = 0.935) identified four classes (see Table 4; Fig. 2, Note: Supplemental files for online publication only), which could be described as follows:

  • Class 1 (estimated percentage in population: 4.25 %) includes less severe “polydrug” users, who experiment with multiple substances but not at high rates.

  • Class 2 (estimated percentage: 0.86 %) includes users of multiple nonmedical prescription drugs.

  • Class 3 (estimated percentage: 94.36 %) includes users who drink occasionally but do not tend to use other substances.

  • Class 4 (estimated percentage: 0.54 %) is the more severe group of “polydrug users” that includes heavy users of all drugs, and is at the high end of drinking frequency.

We re-fit the three logistic regression models including this four-category predicted latent class variable (instead of the aforementioned categorical measures of substance use at Wave 1), and found that these four classes did not differ significantly in terms of the probability of nonresponse at Wave 2. This was likely due to the fact that such a high proportion of the population was predicted to be in Class 3, limiting the power of our comparisons. Despite the potentially interesting presence of these four different classes, we cannot say that individuals belong to particular classes with certainty (despite the strong fit of the LCA model), and the individual measures of the different substance use behaviors at Wave 1 (especially past-year alcohol use) appear to be more sensitive predictors of nonresponse at Wave 2.

Discussion

The findings of the present study suggest that nonresponse rates at follow-up waves in national longitudinal surveys of substance use vary considerably between alcohol users and users of other individual drug classes, such as cocaine and stimulants. As a result, these differences should be accounted for when computing nonresponse adjustments to reduce nonresponse bias, given that baseline measures of these drug use behaviors have significant associations with both Wave 2 response propensity and the same measures of interest at Wave 2 [26]. While previous national studies have found evidence that alcohol use behaviors were significantly associated with attrition in cross-sectional and longitudinal designs [4–6, 27], the findings of the present study extend prior longitudinal work on selected samples (e.g., college students, military personnel) that indicated other drug use behaviors were important sources of nonresponse bias in longitudinal surveys [7, 8].

The findings of the present study indicated notable associations between frequency of drug use behaviors at Wave 1 and nonresponse at Wave 2. Most notably, for alcohol, non-drinkers were found to have the highest nonresponse rates. Previous work has shown that alcohol abstainers have the highest attrition in population surveys relative to occasional drinkers and this could be associated with individual characteristics of abstainers that may influence ongoing participation in surveys [5, 6]. However, this same trend was not found for abstainers of other drug use behaviors, suggesting a unique effect for alcohol use. We found evidence of higher nonresponse rates for subgroups of individuals having various substance use behaviors (e.g., daily cocaine and stimulant use) and SUDs at Wave 1 (e.g. cocaine use disorders, tranquilizer use disorders, SUDs involving multiple drug classes). While the sample sizes of these subgroups defined by frequency of use and SUD were fairly small, accounting for frequency of use and SUDs in nonresponse adjustments may be especially important if one also desires subsequent analyses of frequency of use and SUD prevalence at later waves or time points.

The results also suggest that nonresponse rates in follow-up waves for specific subgroups of substance users may vary depending on gender. This implies that nonresponse adjustments only accounting for Wave 1 substance use behaviors and SUDs that do not incorporate gender and interactions between substance use behaviors and gender may be less effective than adjustments that incorporate these factors. Thus, our findings provide strong evidence of the need to consider interactions between gender and indicators of substance use behaviors and SUDs when performing nonresponse adjustments.

The findings indicated that being married tends to reduce the probability of non-response, while higher age, being male, being Asian or Hispanic, and having lower education all substantially increase the probability of nonresponse at Wave 2. Furthermore, multivariate analyses provided evidence of interactions suggesting that some of these effects are exacerbated by frequency of alcohol use (e.g., being Asian or Hispanic). These findings are similar to previous cross-sectional and longitudinal studies and have important nonresponse bias implications for any drug use behaviors or disorders that are strongly related to these specific socio-demographic features, as the composition of the respondents is clearly changing from Wave 1 to Wave 2 [4, 27]. For example, Wave 2 respondents will tend to be younger, female, non-Asian, non-Hispanic, higher educated, and married. If individuals with these features tend to have unique drug use behaviors or changes in drug use behavior over time, weighted analyses that do not adjusted for these demographic features may produce biased estimates of these behaviors and trends.

The conclusions of the present study apply primarily to a longitudinal setting and should not be used to make inferences about differences between respondents and non-respondents in Wave 1 of the NESARC. Past survey nonresponse studies have developed a theoretical model for nonresponse in later waves of a longitudinal survey, and found empirical support for a number of predictors of nonresponse in follow-up waves that have also been found to predict nonresponse in cross-sectional studies [28, 29]. Despite these previous studies, caution should be taken not to infer that the same predictors of nonresponse would be operating in the initial wave. While our study primarily focuses on the effects of various drug use behaviors on response in the second wave of a longitudinal study of substance abuse, our findings related to socio-demographic predictors are largely consistent with these previous studies.

The findings of the present study do suggest that the NESARC sampling weights did carry some information about nonresponse tendencies in particular subgroups, and this has important implications for interpretation of nonresponse rates (depending on whether weights were used) and nonresponse adjustments. Increasingly, the survey statistics literature suggests that base sampling weights should be used when computing nonresponse adjustments [30, 31]. Based on some of the slight differences found in the present study, future research should examine the sensitivity of adjusted estimates (and their standard errors) to the use of weights in computing the adjustments. Some of the differences that were observed in this study were for subgroups with fairly small sample sizes (e.g., cocaine and stimulant users), especially when stratifying the results by frequency of use.

The findings from the present study extend our knowledge regarding the nonresponse bias associated with estimates of substance use behaviors and SUDs. This investigation is one of the few national and prospective studies to examine substance use behaviors and SUDs based on DSM-IV criteria, and future research is needed to examine potential nonresponse bias associated with SUDs based on DSM-5 criteria. The relatively large sample size of the NESARC allowed for the calculation of prevalence estimates for several drug classes. Finally, the nationally representative nature of the sample allows for generalization to the civilian non-institutionalized population, 18 years of age and older residing in the United States.

Despite these strengths, there are some limitations that must be weighed carefully when considering the implications of the findings. First, the findings from the NESARC should not be generalized to populations outside of the United States. Second, the sub-samples of some drug classes were relatively small (e.g., heroin and inhalants). Third, this study represented a secondary analysis and not all of the survey items were ideal for our purposes. For example, some of the most commonly misused prescription drugs were not specifically listed as examples in the Wave 1 NESARC survey, including hydrocodone (e.g., Vicodin®) and methylphenidate (e.g., Ritalin®) [11, 32]. Finally, although the NESARC includes persons formerly institutionalized who are now living in households and high risk group quarters such as nontransient public lodging, including hotels, shelters, and group homes, the exclusion of currently institutionalized individuals omits an important subgroup at increased risk for chronic DUD and relapse [33].

We also emphasize that this was a study of nonresponse bias in a longitudinal setting, where the baseline responses provide a convenient set of auxiliary variables for examining nonresponse bias, including measures on key survey variables. Assessments of nonresponse bias in cross-sectional national studies of substance use will tend to be much more difficult. In these cases, potential candidates for auxiliary variables that could be used to assess nonresponse bias include interviewer observations of features known to be correlated with substance use (e.g., drug papers, beer bottles, etc.) recorded for all sampled households [34], or survey-based estimates of drug use behaviors for large geographic regions (where, for example, if nonresponse rates are higher in regions known from previous studies to have higher rates of substance use, estimates may be biased). Identification of other effective auxiliary information for assessing potential nonresponse bias in cross-sectional substance use surveys is certainly an area where more work is needed.

In conclusion, the findings of the present study should provide the substance abuse field with a methodology that users of publicly available longitudinal data can apply to test the sensitivity of their inferences to assumptions about attrition patterns. In addition, the findings from these analyses provide researchers with empirical evidence in support of differential tracking strategies for individuals having different baseline substance use behaviors, frequencies and disorders in longitudinal studies of substance use, possibly enabling reductions in the risk of nonresponse bias in estimates based on longitudinal data through evidence-based study designs.