Introduction

Use of substances such as marijuana remains widespread among adolescents in the United States (Johnston et al. 2009). Media-based campaigns are attractive as intervention strategies given the potential economies of scale associated with media interventions as well as youth involvement with and influence by media (Hornik 1988; Klein et al. 1993; see Green 1977).

The data presented and analyzed here evaluate the impact of a replication and extension of an in-school and community based media intervention, “Be Under Your Own Influence,” found to reduce marijuana uptake trajectories in an earlier randomized community trial (Slater et al. 2006). The planned evaluation was adapted while in the field to incorporate measurement of exposure to the (Office of National Drug Control Policy’s) rebranded national anti-drug media campaign, “Above the Influence,” after it was launched concurrently in 2005. While the unplanned concurrence of these very similar campaigns complicated the original evaluation plans for “Be Under Your Own Influence,” it also represented a unique opportunity to a) provide two simultaneous tests of autonomy and aspiration perceptions as mediators of impact on marijuana use as a consequence of exposure to each of these campaigns, b) conduct the first independent assessment of the ONDCP media campaign, which did not have a formal independent evaluation in place during the years of this study, and c) assess the simultaneous impact of a national campaign and a similar community/in-school effort.

“Be Under Your Own Influence,” Non-use and Youth Aspirations and Autonomy—A Theory of Influence Based on Developmentally Appropriate Goals for Adolescence

“Be Under Your Own Influence” was a media-based substance-use prevention program developed over 5 years of formative research and testing (Kelly et al. 1996, 2000; see also Kelly et al. 2006). The theme was adapted for use in a randomized community trial of an intervention using in-school media and promotional materials combined with community-based efforts, and showed significant intervention reductions in marijuana uptake trajectories (Slater et al. 2006). In the study, eight communities received the in-school and community media, and eight did not. Within each community, two middle schools were recruited, one of which received a classroom-based intervention and one that did not. Main effects and interactions involving the classroom intervention did not reach significance.

The “Be Under Your Own Influence” intervention strategy sought to align messages with developmentally appropriate goals. The campaign positioned non-use as supporting the goals of autonomy and achievement or competence that have been conceptualized as innate psychological needs that persist over the lifespan (Ryan and Deci 2000). The “Be Under Your Own Influence” campaign designed messages that would take into account the salience and unique manifestations of these needs in adolescence. For example, developmentally appropriate drives for adolescents towards autonomy and competence (Peterson 1988) were addressed by emphasizing that substance use would undermine the ability to plan for the future, to achieve goals, to act independently, and to aspire to excellence in sports, creative pursuits, and other activities.

While the results from the experiment and the first intervention supported the developmentally-appropriate behavior influence strategy of the “Be Under Your Own Influence” campaign, several major questions remained that could be addressed only with a replication study. One was whether the intervention was robust enough to provide significant effects upon replication and, if so, if both the community and in-school media components were needed or if either element alone could generate significant effects (see Flay (2000) regarding the importance of disentangling community from school intervention effects). Therefore, in the present intervention, the community component was randomized by community, and the in-school media component was randomized to one of two middle schools within each community. Another research objective was to more fully examine the roles of aspirations and autonomy as possible indirect avenues for campaign effects on substance use behavior.

The ONDCP National Media Campaign: “Above the Influence”

The ONDCP’s National Youth Anti-drug Media Campaign was initiated in 1998 after authorization from Congress. The campaign, initially branded “My Anti-drug,” was on-air during the same years as the initial randomized trial of the “Be Under Your Own Influence” campaign. Creative work on advertisements was overseen by the Partnership for Drug Free America (PDFA) through pro bono efforts of leading U.S. advertising agencies.

The ONDCP campaign was initially funded at nearly $200 million per year, mostly for purchase of advertising time to reach both youth and parents. Media buys were supplemented with a two for one requirement, effectively doubling advertising placement. An evaluation based on a representative national panel of youth and parents, using computer-assisted household interviews over four waves of data collection, found no evidence of effectiveness of the “My Anti-drug” campaign and indeed suggested possible iatrogenic effects (Hornik et al. 2008). A re-analysis of the same data set found evidence supporting possible iatrogenic effects for younger but not older respondents (Scheier and Grenard 2010). Another evaluation effort using a regional field test method suggested positive campaign effects (Palmgreen et al. 2007).

Following public release of the negative findings for the “My Anti-drug” campaign, the ONDCP campaign was rebid and a new advertising firm received the contract. In 2005, a rebranded marijuana prevention campaign, “Above the Influence,”Footnote 1 was introduced and ran concurrently with the second trial of “Be Under Your Own Influence” reported here. Funding levels were about half of initial levels. The campaign aimed at parents was eliminated to help maintain levels of campaign exposure to adolescents (Eddy 2006). This national campaign remains ongoing at time of writing.

No formal external evaluation of the rebranded campaign was funded. However, internal rolling cross-sectional surveys conducted weekly over the course of the campaign indicate that while awareness of the “My Anti-drug” campaign had been negatively associated with anti-drug attitudes, awareness of the “Above the Influence” campaign showed significant positive associations with such attitudes (White 2008).

The ONDCP “Above the Influence” campaign also linked substance use with autonomy and aspiration threats. It differed from the “Be Under Your Own Influence” campaign in several respects. The ONDCP campaign involved television advertising produced by leading national advertising agencies placed on broadcast programs frequently watched by teens at levels of exposure comparable to nationally advertised brands. In contrast, the “Be Under Your Own Influence” campaign did not utilize television. The ONDCP campaign effort was supplemented by print advertising, which was the predominant medium used in the “Be Under Your Own Influence” campaign (e.g., posters). Far more creative executions in total were used in the ONDCP campaign effort given the funding levels of that campaign. Creative approaches in print messages generally were similar in concept but somewhat different in execution.

Research Questions and Hypotheses

Given the concurrence of the “Be Under Your Own Influence” and “Above the Influence” campaigns, the close resemblance between the campaign themes, and the parallel mediating mechanisms addressed by both campaigns, effects of the two campaigns must be addressed in tandem. First (RQ1), we assess the extent of exposure to the ONDCP’s “Above the Influence” in treatment and control sites for the “Be Under Your Own Influence” intervention. Second, (RQ2) we examine whether either campaign increases perceptions that marijuana use is inconsistent with aspirations or autonomy. Third (RQ3), we test whether either campaign reduced marijuana uptake, despite the presence of the other campaign, and (RQ4) whether aspirations and autonomy provide an indirect path between the campaigns and marijuana uptake. Finally, (RQ5) we ask whether the presence of the “Be Under Your Own Influence” campaign, a) complements effects (if any) of the ONDCP campaign, (e.g., by reinforcing a similar message); b) interferes with such effects, perhaps by providing a somewhat different presentation of the same message that dilutes the effect of the ONDCP campaign; or, c) has no net impact on effects of the ONDCP campaign.

Methods

Design and Intervention

The study design was a randomized community and school trial with four conditions. Ten communities were randomly assigned to receive the “Be Under Your Own Influence” community-media intervention, which involved a 1-day community-readiness training including training in developing local media materials and working with local press (Slater et al. 2005), provision of media materials including posters, banners, and brochures intended to be used in community settings such as stores, libraries, and recreation centers, and localized press releases about prevention topics, and periodic follow-up. Ten communities did not receive these community-level trainings or materials.

Within each of the 20 communities, two middle schools were recruited and randomized to receive or not receive in-school media including a series of posters for display within the school. Posters included versions pretested for cultural appropriateness for schools with large Hispanic or African-American populations. Banners and promotional materials such as pens, key chains, lanyards, stickers, and t-shirts were also distributed within the school (see Kelly et al. 2006, for details regarding the in-school and community intervention materials). All materials emphasized the link of substance non-use to achieving personal autonomy and aspirations and displayed the campaign slogan “Be Under Your Own Influence.”

To summarize the four experimental conditions: Ten schools were in communities receiving both the community intervention and the in-school media materials, 10 schools received the community intervention without the in-school media, 10 schools received the in-school media and no community intervention, and 10 schools served as controls, receiving no intervention.

Students in all four conditions, of course, were exposed to the nation-wide “Above the Influence” campaign. The re-orientation of the national campaign in 2005 to adopt the same approach as the “Be Under Your Own Influence” intervention was unanticipated and was addressed by adding to the evaluation instrument measurement of exposure to the national campaign.

Four waves of data were collected in each school. The first wave was a pre-intervention baseline in the fall and spring of two school years, beginning in the 7th grade and ending in the 8th grade, followed by data collection at the end of that school year and in the fall and spring of the subsequent year. Schools were recruited based on National Center for Educational Statistics (NCES) district listings; eligible districts were not in the largest urban category (because of the long delays required to obtain project approval in such districts) and had at least two middle schools of reasonably comparable size. Random assignment used a group-matching procedure: NCES data on community demographics and location were used to generate possible randomization schemes in which major demographics and location were balanced to the degree possible across experimental conditions and one of the acceptable schemes was randomly selected. Two communities were in California, two in Oregon, and two in Texas; three were in Louisiana, one in South Carolina, one in Kentucky, and one in Virginia; two were in Minnesota, one in South Dakota, one in Missouri; two in (northeast) Ohio, one in Pennsylvania, and one in New Jersey. While the intervention took place over a 2-year period, because of the logistic demands of implementation, intervention was staggered across three groups, the first beginning fall 2005, the second in fall of 2006, and the third in fall of 2007, with data collection concluding in spring of 2009.

Participants

3,236 students participated in at least one survey, with 48% males, 52% females and a mean age at baseline of 12.4 years (SD = 0.6); 75% were European-American, 11.5% African-American, and 13.5% of other racial backgrounds. One-quarter of the youth were of Hispanic ethnicity. Students were recruited using active consent procedures. The average rate of student participation in each school was 32% of total student enrollment, lower than the prior study because of stricter IRB requirements being imposed on recruitment procedures. 57.1% of respondents provided data at all four measurement occasions; 27.2% provided data on three, 9.4% provided data on two and 5.3% provided data on just one of the measurement occasions. Missed surveys appear to be a matter more of absenteeism or slips in getting students to survey sessions, than of panel mortality; 84.5% of participants filled out the wave 1 survey, 86.2% wave 2, 86.1% wave 3, and 81.3% wave 4. Students who responded that they had tried all drugs listed including one that had been invented were considered exaggerators and were excluded from analyses; there were no more than 0.4% of such exaggerators in any given wave of data collection.

Measures

Autonomy and Aspirations Inconsistent With Marijuana Use

Autonomy inconsistent with marijuana use was measured using responses to four items following the phrase “Not using marijuana”: 1) is a way to be true to myself; 2) is an important part of who I am; 3) is a way of being in control of my life; and 4) is a way of showing my own independence, where responses ranged from 1 = definitely disagree to 4 = definitely agree. Similarly, aspirations inconsistent with marijuana use were measured using the responses to three items following the phrase “Using marijuana would: 1) keep me from doing the things I want to; 2) mess up my plans for when I am older; and 3) get in the way of what is important to me.” Because responses to each scale’s items were heavily skewed, with 82% of respondents selecting “definitely agree” for all aspiration items and 84% of respondents selecting “definitely agree” for all autonomy items, each scale was dichotomized such that a “1” was assigned if all responses to the scale items were “definitely agree” and a “0” otherwise. The Cronbach’s alpha values (Cronbach 1951) for each dichotomized measure were .9 or greater at each of the four waves.

Marijuana Use

Lifetime use of marijuana was measured at each measurement wave using four questions: “How old were you the first time you used marijuana?”, “How often in the last month have you used marijuana?”, “How often in the last 3 months have you used marijuana?”, and “Have you ever tried marijuana? (pot, grass, hash, etc.)?” If a subject responded affirmatively to any one question (or indicated an age when they first used marijuana), lifetime marijuana use was scored a “1”, while an indication of never using marijuana resulted in a score of “0”. The reliability for the scale was above 0.7 for the first two measurement occasions, .64 on the third occasion, and .69 at the fourth occasion.

Exposure to ONDCP’s “Above the Influence” Campaign and Response to Foils

Exposure to the ONDCP “Above the Influence” campaign was evaluated using the following stem, “Have you seen the following lines in ads or posters about drugs or alcohol?” The lines included two foils or fake campaign slogans, “Use pot and booze, you lose” and “Don’t drink, don’t smoke, don’t croak,” and the ONDCP campaign slogan, “Above the Influence” (see Southwell et al. 2002, re validation of exposure self-reports). This question was asked at measurement occasions 2 through 4 but not at measurement occasion 1. Response options ranged from 1 “definitely have seen” to 3 “definitely have not seen,” with “might have seen” as the middle category. Based on earlier validation research regarding use of foils and recognition measures (Slater and Kelly 2002), only endorsement of “definitely have seen” was regarded as evidence for exposure.Footnote 2 A dichotomous measure for exposure to “Above the Influence” was 1 if the subject reported definitely having seen “Above the Influence” and 0 otherwise. Similarly, a dichotomous variable measuring self-reported exposure to at least one of the foil campaigns was calculated.

Treatment Variables

A dichotomous school-level variable measured whether a school participated in the in-school media effort and a similar community-level variable denoted community-wide participation in the campaign.

Data Analysis

The models used to test the research questions were generalized linear mixed models (Skrondal and Rabe-Hesketh 2004) with binary outcomes. The outcomes were measured within four levels of clustering—multiple occasions within individual within school within community.Footnote 3 Testing of random effects was accomplished using the likelihood ratio statistic (Goldstein 1990; Snijders and Bosker 1999). In general, the test compares a model under a null hypothesis of no random effect (H0) at a particular level of clustering against the alternative model containing the effect (H1).

In the analyses reported in this paper, we assumed the missing data mechanism was missing at random (MAR). A violation of this assumption will still result in parameter estimates that exhibit little if any bias if an appropriate method for missing data is applied during the analytic phase (Arbuckle 1996; Wothke 2000). Among the principal methods for addressing MAR is direct maximum likelihood (ML), which we employed.

We assessed our first research question concerning the extent of exposure to the ONDCP’s “Above the Influence” campaign in our intervention and control sites using a cross-tabulation showing the proportion of subjects in each of the response categories. Next, we examined the effects of the “Be Under Your Own Influence” campaign on aspirations and autonomy (RQ 2) and on marijuana uptake (RQ 3) by estimating multilevel growth models for three outcome variables—aspirations, autonomy, and marijuana use. Within these models, we looked at both growth trajectories associated with exposure over time as well as post-test comparisons. Multi-level growth models permit assessment of growth trajectories associated with exposure to the campaigns. Growth trajectories have the advantage of utilizing all available measurement occasions to estimate differences in rate-of-growth as a function of cumulative exposure differences, and as such are both conservative and take into account developmental process and cumulative exposure over time. Post-test comparisons look simply at intercepts placed at the last measurement time point, and have the primary advantage of not assuming linearity (a necessary assumption for our trajectory analyses given the number of exposure time points available for both campaigns).

A four-level model was initially estimated (measurement occasion within the individual within a school within a community) for each outcome variable. Random effects were retained if they were significant at the p < .1 level. Neither community nor school random effects for intercepts or slopes met this criterion. Parameters for these random intercept and slope effects ranged between 1e-8 and 1e-15 (precise parameters are difficult to obtain reliably as the model was difficult to estimate when random effects this close to zero were included in the model).

The fixed effects portion of the model treated each outcome variable as a function of time, school treatment, and community treatment while also controlling for self-reported exposure to the ONDCP message and any foil messages. The fourth research question hypothesized a mediating mechanism through which these media campaigns influence marijuana use via impact on perceptions concerning the relationship of marijuana with aspirations and personal autonomy. Indirect effects were tested using a Taylor series expansion of the Sobel (1982) test that had been previously shown to perform very well in multi-level models (Krull and MacKinnon 2001). The final research question concerned the possibility that the effects of the ONDCP “Above the Influence” campaign might be reinforced or diluted by the similar “Be Under Your Own Influence” intervention. This was tested by adding an interaction term between the intervention treatment effects and the measure of self-reported exposure to the ONDCP campaign to the model used to test RQ 3.

Results

Extent of Exposure to ONDCP Campaign in “Be Under Your Own Influence” Treatment and Control Communities

The extent of exposure to the ONDCP “Above the Influence” campaign (RQ1) was assessed by cross-tabulating the measures of self-reported exposure to this campaign with each of the four treatment/control cells at the fourth wave of data collection, the point by which such exposure would have taken place for all study participants. Of youth in the control community/control school cell, 73% said they definitely had seen the ONDCP “Above the Influence” campaign. The self-reported exposure to the ONDCP campaign was similar in the three treatment cells (68–79%).

We can be confident that this exposure is in large measure due to actual exposure and not false recognition or youth providing what they may have believed to be a socially desirable response, as the percentage reporting they had definitely seen one of the two foils, or fake campaigns, was much less than the percentage reporting exposure to the ONDCP campaign. For example, in the control cell in which 73% of youth said they’d definitely seen the “Above the Influence” campaign, 14.6% reported definitely seeing one foil and 20.2% claimed they definitely saw the other (false recognition of the two foils was highly correlated—67% of those claiming recognition of the first foil also claimed recognition of the second). Similar differences were found in the other cells. These levels of false recognition are typical in response to survey questions about self-reported exposure that do not include actual images of an advertisement (see Shapiro (1994) for a discussion of false recognition of messages).

Effects of the “Be Under Your Own Influence” and the “Above the Influence” Media Campaigns on Beliefs about the Association of Marijuana with Aspiration and Autonomy

Beliefs about the association of marijuana with personal aspirations and autonomy were proposed as possible proximal effects and a possible mechanism for effects of either or both campaigns (RQ2). Therefore, we first estimated two multilevel growth models, the first using the outcome variable, aspirations, and the second using the outcome variable, autonomy. Tests for growth model random effects at levels 3 and 4, as described above, showed that those effects were not significant. Therefore, they were removed from the model. Because development is not always linear, we recoded time so that the growth model intercept was at the last measurement occasion. Placing the intercept at this location provided a direct test for the treatment on the outcome at the end of the treatment application: If the treatment had an effect by the end of the campaign, experimental groups would differ at this measurement occasion.

The community-level treatment in the “Be Under Your Own Influence” campaign predicted increased perceptions that marijuana use was inconsistent with personal aspirations at the last measurement occasion, controlling for the “Above the Influence“ campaign and foil effects (β = 0.54, p = 0.006). The school-level treatment did not have a significant effect. The ONDCP “Above the Influence” campaign also exhibited a higher intercept for aspirations at the last measurement occasion (β = 1.05, p < 0.0005) but had no detectable effect on the linear rate-of-change (see Table 1).

Table 1 “Be Under Your Own Influence” (Intervention) and “Above the Influence”(ONDCP) campaign effects on perceptions that personal aspirations and autonomy are inconsistent with marijuana use: multi-level latent growth model results
Full size table

For autonomy, the community-level treatment resulted in a more positive linear rate-of-change (β = −0.48, p = 0.023) but did not affect post-test scores. The school-level treatment had no significant effect at either the last measurement occasion or on the linear rate-of-change. The “Above the Influence” campaign exhibited a positive effect on autonomy at the last measurement occasion (β = 0.599, p < 0.0005) but did not influence the linear rate-of-change (see Table 1).

Effects of the “Be Under Your Own Influence” and the “Above the Influence” Media Campaigns on Marijuana Outcomes

Table 2 presents results for the multilevel growth models that use marijuana use as the outcome variable (RQ3). The positive coefficient for the variable “time” implies that as time progresses, the odds of a positive response to one of the marijuana ever-use indicator variables increases (OR = 3.8: 1, p < 0.005). The community treatment parameter estimate was significant (β = −0.51, p = 0.026), indicating that community-level treatment lowered the propensity to use marijuana at the last measurement occasion over and above the effect of the “Above the Influence” campaign. There was no evidence that school-level treatment affected the marijuana use at the last measurement occasion. Neither the community-level nor the school-level treatment for the “Be Under Your Own Influence” campaign provided evidence of an effect on the linear rate-of-change for marijuana use. In other words, while there was evidence of a significant effect of the community-based component of the “Be Under Your Own Influence” campaign on post-test marijuana use, the strong and consistent effects of the prior campaign on both post-tests and reduced linear trajectory of marijuana use were not replicated.

Table 2 “Be Under Your Own Influence” (Intervention) and “Above the Influence”(ONDCP) campaign effects on marijuana uptake: multi-level latent growth model results
Full size table

However, there was clear evidence that exposure to the “Above the Influence” campaign, as measured by respondent self-report, predicted reduced marijuana uptake. While controlling for “Be Under Your Own Influence” effects and recognition of foils, exposure to ONDCP’s “Above the Influence” campaign prospectively predicted reduced marijuana use at the last measurement point (β = −1.35, p < 0.005) and a reduced linear rate-of-change of marijuana use (β = 0.34, p = 0.042). Expressed in terms of odds ratios, there was a main effect of “Above the Influence” suggesting that, at the last measurement occasion, those who had been exposed to the ONDCP campaign were less likely (OR = 3.85: 1, p < 0.0005) to use marijuana compared to those not exposed to the campaign.

Indirect Effect Tests: Campaign Effects on Marijuana Use Via Aspirations and Autonomy

One of the aims of the present research was to test the hypothesis that media campaigns such as these might influence substance use behavior by reinforcing beliefs that substance non-use is consistent with youths’ aspirations and autonomy drives (RQ4). We provide inference tests for indirect effects of the campaigns on marijuana use through aspirations and autonomy, using the Taylor-series expansion of the Sobel test.

For the “Be Under Your Own Influence” campaign, we look only at the community-level treatment, for which evidence for statistically-significant effects on marijuana uptake at post-test were found. The path from community-level treatment through aspiration to the marijuana use at post-test was significant (z = 2.58, p < 0.01). No other indirect effect involving the community-level treatment was significant. The indirect effect of the ONDCP campaign on marijuana use as measured at post-test through aspiration was significant (z = 5.2, p < 0.005). The indirect effect of the ONDCP campaign on linear trajectory via aspiration was non-significant (z = 1.31, p = 0.19). The ONDCP campaign also had an indirect effect on marijuana use at post-test through autonomy (z = 3.07, p = 0.002). There was no evidence that autonomy mediated the effect on the rate-of-change for marijuana use (z = 0.58, p = 0.56).

Interaction Effects of Exposure to the Two Campaigns

The simultaneous presence of the two campaigns in our study sites permitted us to test interactions to assess whether the intervention reinforced, interfered with, or had no net effect with respect to ONDCP campaign impacts (RQ5). We found no statistically significant interaction between intervention exposure and ONDCP campaign exposure on marijuana use (β = 1.004, p = 0.361).

Discussion

These analyses provide independent evidence that the ONDCP’s revised “Above the Influence” campaign predicted lower marijuana use by the final wave of data collection. These results suggest that the change of campaign theme from the earlier “My Anti-drug” effort, which was associated with no or perhaps even iatrogenic effects (Hornik et al. 2008), was well-advised.

ONDCP campaign effects on marijuana uptake appeared more robust in examination of post-test results than in tests of effects on linear trajectory, though the latter was statistically significant. Existing data on marijuana uptake over the course of adolescence indicates a non-linear trajectory sharply increasing around ages 14 and 15 (many 8th graders are or turn 14 years old), with initiation increasing by a factor of five over rates from that at ages 12 and 13 (Substance Abuse and Mental Health Administration 2010; see also Tang and Orwin 2009). Finding maximum impact, then, for our respondents who are at the end of 8th grade is consistent with such epidemiological findings. The significant indirect effects indicate that the ONDCP campaign indirectly affected post-test marijuana uptake through both aspirations and autonomy. There was no indirect effect on linear trajectory, perhaps because trajectories are in fact not best modeled as linear, as discussed previously.

With respect to the “Be Under Your Own Influence” community and in-school intervention replication and extension study, the robust effects found in the previous community randomized trial on both post-test and linear trajectory effects replicated only with respect to post-test results. This is not surprising given the study’s control conditions were contaminated by high levels of exposure to the ONDCP “Above the Influence” campaign, which was closely parallel in theme and emphasis to “Be Under Your Own Influence.” Even in the treatment conditions, exposure levels were as high or higher to the “Above the Influence” national campaign as in control conditions, swamping the impact of the intervention. The relative inability of the in-school/community media effort to replicate previous effects in the presence of this very similar national campaign underscores the relative power of national cable and broadcast advertising with high levels of exposure to communicate essentially the same message.

The community component of the “Be Under Your Own Influence” campaign showed evidence for effects in the post-test analysis, while the in-school campaign did not. Under ordinary circumstances, this would be surprising: The level of exposure to campaign content was generally higher in the in-school setting, in which posters and other materials were highly visible, than in the community setting. However, the community effort included media and community coalition activities that were less likely to be redundant with the ONDCP’s national “Above the Influence” campaign. This may explain evidence for community but not in-school treatment effects. Given this anomalous context of a very similar national media campaign in the information environment that was likely to compete more directly with the in-school than the community treatment component, we do not believe our results are informative regarding the question of the comparative effects of an in-school versus community-based prevention effort that we had initially hoped to help answer.

It is worth noting that the presence of a national media campaign per se is not the explanation for limited results, as ONDCP’s “My Anti-drug” campaign ran concurrently with the first, clearly successful iteration of the “Be Under Your Own Influence” effort. However, “My Anti-drug” focused on negative consequences, refusal skills, and normative influences (Office of National Drug Control Policy 1998), and had problematic outcomes according to the funded evaluation (Hornik et al. 2008). The overlap of theme and strategy seems the key factor.

The circumstances also permitted a test as to whether combining a national media campaign and an in-school/community media program targeting the same proximal variables would enhance the effects of the national campaign—or, perhaps, undermine them by causing some confusion of the message with slightly different brands and different creative executions and styles. The lack of interaction effect for exposure to the two campaigns suggests neither happened (or both happened and the effects cancelled one another out). We hasten to note, however, that to therefore infer a national campaign should not consider supplementing efforts through in-school and community media efforts would be premature. It remains quite possible, if an in-school/community effort used exactly the same theme and creative work specifically designed to complement the national campaign, that effects of the national campaign could be measurably increased.

This study has a number of design strengths, including longitudinal data collection in varied communities across the U.S., measurement of mediating variables, and a randomized community trial design for assessment of the “Be Under Your Own Influence” intervention. However, the analyses reported here also have limitations to be taken into account when drawing conclusions from these findings.

Analyses concerning the ONDCP campaign depend on a single-item question concerning exposure to the campaign “Above the Influence” theme. Recognition memory tends to be relatively reliable, although false recognition can be a problem (Shapiro 1994). We controlled for false recognition by incorporating recognition of foil themes consistent with validation analyses from a previous study (Slater and Kelly 2002). The ONDCP exposure item was in the data set for waves 2, 3, and 4 but not in wave 1, as the wave 1 scannable forms had already been printed before the “Above the Influence” campaign had been announced. Having to work with three, rather than four, waves in assessing the predictive power of ONDCP campaign exposure tends to reduce the statistical power to test associations, with a conservative impact on these analyses. It would not have been possible to get a pre-exposure baseline for the ONDCP campaign even if we had been able to get the ONDCP exposure item into all four waves, as wave 1 data were collected after the launch of the “Above the Influence” campaign for nearly all study participants. Therefore, we would be analyzing associations of trajectories even with inclusion of wave 1 data for ONDCP. Another associated limitation is that with three waves we can test only linear relationships and trajectories.

The fundamental limitation of the analyses of the ONDCP campaign rests in the associational, non-experimental nature of these analyses and the resulting uncertainties regarding causation. There are several obvious threats to causal interpretation of this association. The primary threat is social desirability bias—a belief among respondents that claiming recognition of the “Above the Influence” campaign is a way to express socially approved anti-drug norms and values. If responses typically reflected such bias, they might generate spurious associations between claimed recognition and attitudinal and marijuana use uptake outcomes. However, our use of recognition measures of two foils—fake campaign slogans—as controls should largely eliminate this effect: If social desirability is driving affirmative responses to recognition of anti-drug campaigns, they should be more likely to affirm exposure to one or both foils.

It is also possible that those interested in experimentation with marijuana are more likely to remember messages about the product of interest; advertising researchers note that advertisements are best recalled by people interested in using the product advertised (Kokkinaki and Lunt 1999). However, if this is the case, the better recall of the campaign by those interested in marijuana use should lead to negative and not positive associations with anti-drug beliefs and marijuana uptake. Such a bias would therefore render these tests more conservative, and not provide an alternative explanation for results.

Another possibility is that youth who had other protective factors in their lives would be more sympathetic to the aspirational messages, and as a result notice and remember them, producing higher self-reported campaign recognition and spurious positive predictive effects. This explanation, while it cannot be excluded, seems to pose only a modest threat to inference for several reasons. One is that those interested in using a product are more likely to attend to relevant messages, as noted above. Another reason is that evaluation of the prior “My Anti-drug” campaign found at best neutral and often clear tendencies toward negative associations between earlier self-reported recognition measures of the campaign. It is not clear why spurious positive relationships would be found for the aspirational messages in “Above the Influence” and not for the negative consequence, refusal skill, and normative messages found in the “My Anti-drug” effort (Hornik et al. 2008). Even if there was something uniquely protective and compelling about the aspirational theme, this would suggest that the messages were well-targeted, but that the causal process was more complex than captured here, involving reinforcement of existing positive perceptions (Slater 2007). This would qualify but not change the basic findings of these analyses. Finally, the analyses of mediation reported above provide some support for our hypothesized causal processes.

Therefore, despite the uncertainties associated with use of the self-report measure, ONDCP exposure predicting lower uptake and greater association of non-use with personal aspirations and autonomy seems plausible. At minimum, these results provide reason to believe that the possibly iatrogenic effects of the earlier version of the ONDCP campaign are not evident in response to the rebranded campaign. The negative findings from the evaluation of the “My Anti-drug” campaign (Hornik et al. 2008) were also based on self-report and associational data in a panel data set.

The construction of the youth sample is also a limitation. First, while community and school variability is accounted for in these multi-level models, the sample is biased toward smaller towns and suburbs, as it was too difficult to manage logistics of administrative clearances to conduct research such as this efficiently in large metropolitan school districts. In one respect this bias is likely to result in conservative estimates of ONDCP effects. Advertising exposure to the ONDCP campaign was relatively heavier in larger than in smaller media markets, so average exposure of youth in our study communities to the ONDCP campaign was probably at least somewhat less than for typical early adolescents in the U.S. However, it may be that the media campaign is more effective with youth in smaller than large metro communities, perhaps because the larger communities have richer media environments causing more competition with the campaign messages; therefore, likely effects on youth in large metropolitan areas cannot be assessed confidently from these data.

Non-response bias among youth is also an issue. Stricter IRB demands resulted in lower participation rates in the current study compared to the prior project, and appear to have resulted in participants with lower initial use of marijuana. It is possible that this sample under-represents youth at highest risk. Insofar as the media campaigns were both focused on delaying onset among non-users, a bias towards non-users is not necessarily a bad thing, but it does raise questions about effects on users. To the extent that there may be more users in a truly random sample, it is possible that these analyses may overstate average effects overall assuming effects are greater on non-users. As with most longitudinal research, sample attrition also reduces confidence in generalizability of results, although in this study missed surveys seemed due more to erratic patterns of absenteeism or problems at schools early on in getting all students to data collection, rather than actual panel mortality; such missing data points can be adequately managed using the trajectory analyses in a multi-level latent model.

Even granting constraints in inference associated with these limitations, and resulting uncertainties about precise parameter estimates for U.S. younger adolescents as a whole, these findings provide independent evidence that the ONDCP “Above the Influence” campaign is trending towards positive impacts on attitudes and behavior, and that these effects may be explained in part by impact on perceptions that personal autonomy and aspirations are linked to substance non-use. The autonomy and aspiration messaging approach pioneered in “Be Under Your Own Influence” and adapted by “Above the Influence” continues to have noteworthy potential.