Introduction

The stigma of mental illness poses a pronounced threat to individuals with psychiatric conditions and, at the same time, represents a fundamental social injustice (Corrigan et al. 2005; Hinshaw 2007). Public stigma (i.e., derogatory attitudes and discriminatory practices) fosters a detrimental social environment. Indeed, individuals influenced by such attitudes and practices may internalize feelings of shame, leading to self-stigma (Vogel et al. 2013). These negative self-views compound the already distressing symptoms characteristic of most forms of mental disorder (Holmes and River 1998). In a review of the association between stigma and treatment seeking, Corrigan and Rüsch (2002) asserted that the perceived costs of public stigma (e.g., loss of opportunity) and self-stigma (e.g., weakened self-esteem) linked to a diagnosis of mental illness may outweigh the desire to seek, or fully participate in, treatments that otherwise help to ameliorate symptomatology. For example, Sirey et al. (2001) found that perceived/internalized stigma among outpatients with depression, controlling for depressive symptoms, predicted early withdrawal from treatment.

Antistigma efforts have increasingly been promoted to reduce the negative impacts associated with a psychiatric label (Estroff et al. 2004; Pescosolido et al. 2010). Such campaigns have generally come in the form of education or contact; protest against stigmatizing messages may also be featured (Corrigan et al. 2012; Corrigan and Penn 1999). Education-based programs aim to increase knowledge about mental illness in order to equip individuals with accurate information about mental disorder, with the potential for disconfirming false assumptions and negative stereotypes (Corrigan and Penn 1999). Contact strategies are concerned with positive, cooperative interactions with members of a given “outgroup” as a means for changing generalized attitudes toward that group (Corrigan and Penn 1999; for a general review, see Pettigrew and Tropp 2006). Protest is intended to remove stigmatizing messages from the public eye, although concerns have been raised that protest efforts might lead to “rebound” salience of the damaging messages (Corrigan and Penn 1999).

To place antistigma solutions within a broader theoretical framework, stigma is believed to incorporate three key elements: stereotypes, prejudice, and discrimination (Corrigan et al. 2001; Corrigan and Shapiro 2010; Hinshaw 2007; Thornicroft et al. 2007). In other words, stigma comprises cognitive, affective, and behavioral processes (Hinshaw 2007). This conceptual model has been used as the basis for measurement in stigma research. For instance, in a review of antistigma educational interventions related to mental disorder, Yamaguchi et al. (2011) showed that most evaluations included measures of some combination of knowledge (the “cognitive” aspect), attitudes/attributions (the “affective” aspect), and social distance (a proxy for the “behavioral” aspect). (Actual behavior change, however, is almost never included as a measure.) The most comprehensive evaluations include measures of all three and should ideally include a measure of social distance when measuring actual behavior is not possible (Schachter et al. 2008; Yamaguchi et al. 2011).

Many existing efforts focus on education as the primary means for reducing stigma (e.g., DeSocio et al. 2006; Esters et al. 1998; Wahl et al. 2011; Watson et al. 2004). In fact, in a systematic review of school-based antistigma interventions directed toward youth, Schachter et al. (2008) found that all of the reviewed interventions included at least one education component. Yet short-term, lecture-style educational approaches often fall short of the goal of globalized and lasting stigma reduction (e.g., Mann and Himelein 2008; Pinfold et al. 2003; Watson et al. 2004). In fact, one particular aspect of many educational programs is the tenet that biological explanations of mental illness (e.g., “mental illness is a brain disease”) will reduce blame and thereby reduce stigmatization. However, biogenetic ascriptions may actually intensify stigma in several key ways (Kvaale et al. 2013; see also Corrigan and Watson 2004). Accordingly, antistigma approaches often incorporate other elements, such as videos about the lives of people with mental illness (Mann and Himelein 2008; Pinfold et al. 2003; Stuart 2006), theater and drama performances (Essler et al. 2006; Roberts et al. 2007), and/or direct contact with persons with mental illness (Schulze et al. 2003). Stigma researchers generally agree that antistigma interventions need to move beyond literacy- and knowledge-based programs (Crisp et al. 2000; Mann and Himelein 2008; Pinfold et al. 2003; Schulze et al. 2003; Stuart 2006).

Even so, instructive teaching of mental health concepts is at the core of most interventions. Eminent examples are The Science of Mental Illness, a five-lesson curriculum supplement for middle-school (grades 6–8) science classes focused on the biological and psychosocial causes of mental illness (Watson et al. 2004), and Breaking the Silence: Teaching the Next Generation About Mental Illness, a program providing middle school teachers with lesson plans and teaching packages related to the signs and treatment of mental illness and how to fight stigma (Wahl et al. 2011). Another curriculum supplement, in the form of a three-day instructional unit in high school health classes, focuses on the professional roles and basic facts about mental illness and its treatment (Esters et al. 1998; see also Pinfold et al. 2003). It is noteworthy that (a) each of these programs requires teachers or outside facilitators and class time to implement the curriculum; (b) whereas program evaluations have showed positive changes in all instances, no indices of actual behavior were used as outcomes—and significant changes in social distance are not consistently found; and (c) all authors uniformly supported the need for longer-term antistigma strategies to maintain short-term impacts.

More specifically, Pinfold et al. (2003) evaluated mental health awareness workshops in UK secondary schools delivered by mental health professionals. In addition to didactic information delivery (i.e., lesson plans) on mental illness and mental health, the workshops included a video of people living with depression and schizophrenia. In some cases, sessions were co-facilitated by persons who had personal experiences with living with mental health conditions. There were improvements in students’ knowledge of mental health problems (e.g., more students correctly rejected the statement “People with schizophrenia have a split personality”) and attitudes (e.g., more students disagreed that “People with mental health problems are unpredictable”) from pre- to post-test. Yet social distance ratings (e.g., “I would be afraid to talk to someone with mental health problems”) were not significantly improved, even though “personal experience talks” were independently associated with the observed (non-significant) improvements. All gains declined substantially by the 6 month follow-up. The length of the program (two, 1-h long sessions) may have been too brief to challenge students’ longstanding aversion to interacting with persons with mental illness.

From a developmental perspective, stigmatizing tendencies emerge during middle childhood and worsen with age (Wahl et al. 2007). Most antistigma efforts focus on adult populations (Hinshaw 2005; Wahl 2002), yet core, negative attitudes may have become solidified by adulthood. Programs for youth are therefore a priority but (a) have often, as just noted, focused on knowledge-based education strategies (e.g., Esters et al. 1998; Pinfold et al. 2003; Wahl et al. 2011; Watson et al. 2004); and (b) have been insufficiently evaluated (Link et al. 2004; Wahl 2002). Moreover, they typically require already overburdened teachers or trained professionals for their implementation. Evidence for lasting impacts from such programs is not strong (for reviews, see Schachter et al. 2008; Yamaguchi et al. 2011).

Evidence is accumulating in favor of an approach that includes open and discursive (vs. traditional and didactic) learning settings, mutual and informal face-to-face interactions with outgroup members (vs. contrived, formal interactions), and an emphasis on the humanity of the targeted outgroup through empathy enhancement and perspective taking (Corrigan and Penn 1999; Estroff et al. 2004; Hinshaw 2007; Mann and Himelein 2008; Penn et al. 1994; Pettigrew and Tropp 2008; Pinfold et al. 2003; Schachter et al. 2008; Schulze et al. 2003). Disclosure of mental illness (and related topics, like sexual orientation) in a welcoming peer community has also been featured as a way of reducing self-stigma by way of increasing group identification and social support (see Corrigan et al. 2013 for a review). Corrigan et al. (2013) concluded that there is a need for peer support programs that encourage “coming out” and thus contribute to self-empowerment. Relatedly, Estroff et al. (2004) reviewed exemplary antistigma programs, emphasizing the following for successful interventions: fundamental principles that address the needs of community members (e.g., inclusion, empowerment, choice, self-determination); ongoing personal and interpersonal opportunities and contexts where both persons with mental illness and others can experience equal value and respect; and high reach, visibility, and impact (e.g., programs in schools, consultation with researchers). Importantly, Estroff et al. also called for innovation and growth for future antistigma programs.

Adolescent interventions, in particular, should be relevant and developmentally-appropriate (Schachter et al. 2008). Schulze et al. (2003) cautioned that programs, particularly those focused on knowledge enhancement, may not be well-received by teens who find the delivery of mental illness information unrelated to their own experiences. They proposed that interventions should emanate from students’ personal experiences. Furthermore, interventions may optimally target themes of “difference” more broadly, rather than focusing on a single condition (Heary et al. 2014). Pinto-Foltz and Logsdon (2009) suggested that adolescent-based interventions should be universal (i.e., administered to all adolescents) rather than targeted (i.e., directed only at those with mental health problems): “Reducing stigma among all adolescents will help to reform the culture, within adolescent social networks, to one that embraces discussion of mental disorders and is inclusive of adolescents with mental disorders” (p. 789).

Let’s Erase the Stigma (LETS) Educational Foundation features an innovative, youth-focused conceptual framework blending personal discussion and disclosure as well as youth-directed ideas and action plans to combat mental illness stigma. The core framework of the LETS model is that youth, with a club advisor (typically a high school teacher), hold weekly discussion clubs that promote peer-to-peer interaction, discussion, and community action geared toward stigma reduction. A LETS Start-up Guidebook consists of ideas for club activities and fundraising opportunities, yet LETS has no structured educational curriculum. Rather, under the guidance of the volunteer advisor, whose role is to sponsor the club and deal with potential legal and supervisory issues surrounding intense club discussions on sensitive topics (e.g., suicide attempts), youth are responsible for selecting discussion topics and deciding on school-based or community activities intended to reduce stigma. Because it consists of after-school clubs, LETS occurs outside the classroom, formal assemblies, or clinical environments.

Sample LETS club activities are articulated in the following. The Stigma Box is presented as a way to spark discussion. Students write down any topic they would like to talk about (e.g., depression) and anonymously insert it into the stigma box, from which topics are drawn at random and opened for discussion. LETS members will invite guest speakers (e.g., LETS alumni, community members) to tell their stories about being stigmatized—via sexual orientation persecution or being bullied—with the goal of making the detrimental effects of stigma more real and personal. Similarly, students also participate in anonymous story sharing, through which club members write about their stigma-related struggles, which are then read out loud. Perspective-taking is included in direct relation activities: if a student discloses depression, students then partake in an exercise that allows them to experience what it feels like to be depressed, such as walking around holding heavy objects, symbolizing the “heaviness” characteristic of depression. Students also engage in hands-on projects working directly for, or with, stigmatized persons within the community (e.g., at local hospitals) or school.

In short, although traditional youth antistigma programs are brief, knowledge-focused, and usually conducted by adults, LETS provides a continuous dialogue in an interactive, youth-directed format to instill members with stigma-reducing attitudes and increased awareness of mental illness. LETS club meetings are a time for students, with or without mental illness, to come together and discuss issues related to “difference” (including mental illness) and effect change strategies, with the overall aim of promoting destigmatization. Our belief is that a preliminary evaluation of the feasibility and potential benefits of this innovative model is in order, to pave the way toward randomized trials.

Our aim herein is therefore to provide an initial, quasi-experimental evaluation of LETS, given our belief that the LETS model adheres to many of the guiding frameworks that might lead to effective antistigma promotion for youth. Moreover, our outcome measures adhere to the conceptual framework presented above. Specifically, quantitative measures assessed knowledge, attitudes, social-distancing tendencies, and self-professed positive actions, all related to mental illness. Inclusion of the positive-actions measure was prompted by the fact that such indices are very seldom used in stigma research (Link et al. 2004), offering an opportunity to probe this underexplored behavioral element of stigma. Furthermore, participants provided open-ended, written responses to questions related to defining stigma, positing why stigma should be reduced and what strategies would be optimal for doing so. The goal here is to probe whether LETS participation would be associated with deeper understanding of stigma processes and antistigma strategies.

We hypothesized that a semester’s participation in a LETS club, contrasted with non-participation, would be associated with better attitudes about, reduced social distance toward, and enhanced self-professed positive actions performed, all related to mental illness. Given the student-led and non-curricular nature of LETS clubs, we did not predict that participation would be related to greater knowledge about mental illness. Moreover, we hypothesized that, on the basis of open-ended responses, LETS participation would yield more awareness of stigma and a greater sense of the rationale for and nature of antistigma efforts.

Method

Procedures

For this program evaluation we collected non-identified questionnaires and open-ended responses from high school students whom we designated as belonging to two groups: those interested in joining a LETS club but with no prior involvement in such a club, versus those who had completed a semester’s worth of weekly LETS club meetings of approximately 1 h each. Analyses were accordingly performed at the group level, as detailed below. Because both groups consisted of youth with motivation to join a club, it is likely that prior motivation is consistent between the groups, although this variable was not directly measured. At this initial stage of the development and evaluation of the LETS model, we deemed such a quasi-experimental design—which did not require identifiers on forms and questionnaires and therefore did not require Institutional Review Board approval—as optimal. In the reporting of all information below, we have followed the Transparent Reporting of Evaluations with Nonrandomized Designs (TREND) guidelines (Des Jarlais et al. 2004) in order to ensure accurate reporting of our quasi-experiment. Note that none of the authors has any conflicts to report with regard to this research.

No-LETS Group

Two subgroups of participants comprised the No-LETS group: (a) all high school students who attended the May 2011 LETS Youth Summit held in the Los Angeles area, the goal of which was to introduce students to the concepts and ideas behind LETS clubs, which had not yet begun or had just been initiated; and (b) Los Angeles-area high school students during the Fall of 2011 who attended school meetings at which LETS clubs were described by LETS staff, with the intention of initiating clubs. For the latter, the surveys included a question related to attendance at the May 2011 Summit, and only students who answered “no” were included in the No-LETS condition. Across subgroups (a) and (b), N = 415 for quantitative questionnaires and N = 396 for open-ended responses.

LETS Group

Surveys for the LETS group emanated from Los Angeles-area school meetings held in the Winter of 2012, after LETS clubs had been operating at several schools. These participants attended one of three schools at which a LETS administrator verified that participants were students who had certainly participated in a LETS club for one full semester (N = 39 for quantitative questionnaires and N = 39 for open-ended responses). For both conditions, the LETS staff who administered the surveys, as well as the investigators, were not involved in implementing the LETS club meetings that are the object of the current evaluation. As noted above, after-school club meetings were conducted by youth club members, under the supervision of a volunteer adult advisor.

Participants

The sample was self-selected, in that attendance at the LETS Summit, LETS school meetings, and LETS clubs was voluntary. Demographic data were solicited on the quantitative questionnaires. The vast majority (95.4 %) of participants fell within the age range of 14- to 18-years (i.e., high school-aged adolescents); the overall range was 12–19 years (M = 16.6, SD = 1.26). The No-LETS group mean age was 16.5 (SD = 1.24) and the LETS mean age was 17.4 (SD = 1.13), a significant but small difference. Proportions of ethnic-group identifications were as follows: 62 % Hispanic or Latino; 13 % Black or African-American; 12 % Caucasian/White; 16 % Asian; 3 % American Indian/Alaska Native; 3 % Native Hawaiian or other Pacific Islander; and 5 % “other.” The Hispanic or Latino category was the only participant-selected ethnicity that differed by group, χ2(1, N = 449) = 13.00, p < .001. There were no significant differences between the two groups with respect to sex. Informed consent was not required, because participation was voluntary and individual identifying information was not obtained on any questionnaire or open-ended response forms. As noted above, the project was exempted from institutional review board approval.

Quantitative Measures

These scales emanate from Wahl et al. (2011). Wahl and colleagues first established the psychometric properties of the scales in a pilot investigation and subsequently in their youth antistigma program evaluation. The test–retest correlations reported after each measure represent combined averages between these two studies; internal consistency calculations (Cronbach’s alpha) stem from the pilot.

First, the attitude scale contains 17 items tapping attitudes about mental illness (e.g., “A person with mental illness is able to be a good friend”). Responses were indicated on a 5-point Likert scale from strongly disagree to strongly agree. Scores were averaged to yield an overall score on this 5-point scale, where higher mean scores indicated more positive attitudes. There were 10 reverse-scores (e.g., “It is a good idea to avoid people who have mental illness”), for which weaker agreement indicated more positive attitudes. This measure has been found to have strong test–retest reliability (r = .80) and internal consistency (α = .83).

Second, the Social Distance Scale, originally created by Bogardus (1925) for the study of race and ethnic relations, is regarded as one of the oldest and most widely used psychological scales tapping stigma and prejudice (Wark and Galliher 2007) and has generally been used as a proxy for discriminative behavior (Corrigan et al. 2001). The overall framework is to ask participants about their willingness to interact with particular social groups. These interactions are ordered by the closeness of contact required, from rather distant interactions to extremely close ones. In the Wahl et al. (2011) version, items were worded to be relevant to youth (e.g., working on a class project instead of working on a job). The eight items, in order from least to most intimate contact, ranged from “Have someone with a mental illness as a neighbor” to “Go on a date with someone with a mental illness.” Willingness to interact was measured on a 5-point scale for each item, ranging from definitely unwilling to definitely willing. Higher scores were suggestive of more desire for contact with people who have mental illness (i.e., less social distance; greater acceptance/inclusiveness). Mean scores across the eight items were calculated. This measure demonstrated extremely robust test–retest reliability (r = .88) and internal consistency (α = .91).

Third, regarding actions related to mental illness stigma, Wahl et al. formulated a behavioral measure that included a list of positive (N = 18) and negative (N = 2) actions that could reasonably be performed as a result of the antistigma program tested in their evaluation. We considered only positive actions herein because the two negative actions were too few to constitute a reliable scale. Moreover, only 16 of the original 18 positive action items were used because, prior to data analyses, we judged two of the actions as inadequate indicators of deliberate actions performed to help reduce stigma (e.g., “I heard other people use slang mental illness terms to put down people or their ideas”; “I heard others make fun of or tease someone with a mental illness”). Here, participants were asked to answer yes or no to indicate whether they had performed a series of specific (positive) actions (e.g., “I befriended someone who has a mental illness”; “I talked to someone about their use of slang mental illness terms to put down other people or their ideas”) within the past month. Positive action scores were calculated as proportions (e.g., a yes response to 12 out of 16 positive actions would yield a score of .75). In their pilot, Wahl and colleagues established the reliability of the positive action measure by calculating the percentage of instances in which a student was consistent in indicating the same action from the first to the second administration, which was 76 % of the time.

Finally, with respect to mental illness knowledge, this set of 18 items related to factual questions about mental disorders (e.g., “People with Obsessive Compulsive Disorder (OCD) often feel they must repeat behaviors over and over”). Like the attitude items, overall knowledge scores were calculated as averages on a 5-point Likert scale (strongly disagree to strongly agree), where higher mean scores suggested more accurate knowledge. Eight of the items were reverse-scored (e.g., “Schizophrenia is a mental illness that involves multiple personalities”), for which weaker agreement revealed more accurate knowledge. Test–retest reliability was lower for this measure (r = .58), as was internal consistency (α = .54). Wahl et al. (2011) proposed that the weaker psychometrics exhibited by the knowledge measure may be related to the diversity of items and may thus reflect specific gaps in knowledge about mental illness rather than a general lack of knowledge about mental illness.

Open-Ended Responses

Participants also completed open-ended questionnaires containing three questions (A, B, and C), created to assess participants’ understanding of mental illness stigma: (A) “Can you think of an example in your life, your family, or your school of what stigma means?” (stigma awareness); (B) “What are the best ways that youth can be the ones to overcome the stigma of mental illness and make history?” (potential antistigma actions); and (C) “Why is it important to erase the stigma of mental illness?” (antistigma rationale). Extensive coding manuals were created for each question, with scoring of responses on a 5-point scale (no understanding to exemplary understanding). Scores (1–5) were based on quality of relevant content, as determined by specific criterion lists, plus level of coherence.

Responses to Question A were judged against a list of aspects of stigma (e.g., “stereotypes,” “prejudice,” etc.) and related themes (e.g., “us vs. them mentality”) commonly found in the stigma literature. Responses demonstrating no understanding (i.e., a score of 1) were generally ignorant in their message about stigma (e.g., “Stigma is helping people with mental disorders and not judging people”; “It when people make fun of other just because they have stigma”). Exemplary responses (a score of 5) clearly articulated a relevant stigma-related message (e.g., “When I think of stigma, the first thing that pops into my head is stereotyping and one of my best friends that unfortunately suffers from depression and thoughts of suicide. Today, and for the past generations, society has bombarded us with this image of being perfect, and when people come across someone who acts or appears different, they get confused and treat them unfairly or differently, which isn’t right. There is no such thing as being normal”). Other responses fell somewhere in between, showing a hint of (2), firmer (3), or satisfactory (4) understanding.

For Question B, participants were asked to write about actions that can be taken to overcome the stigma of mental illness (e.g., “befriending someone with mental illness”). Sometimes responses offered goals of stigma-reducing methods without actually stating, or alluding to, a specific action (e.g., “raising awareness about mental illness”). An elaborated conveyance of either type earned a 5 (e.g., “The first step that we can take to overcome stigma is to veer away from the idea that we all must strive to become normal. Normality doesn’t exist because each of us is different in our unique, special ways. Once we can expose the public and growing children about the fact that inside we are all human, and that each of us deserves a voice and support, then history can be made”). Those that failed to convey relevant actions or goals were given a 1 (e.g., “By putting it all aside. Focusing on bad brings it all down”).

Question C evaluated participants’ reasons for why stigma should be eliminated. Responses often either stated a benefit of stigma reduction (e.g., “people would feel free to get help or seek treatment”) or made a claim about stigma’s current negative effects (e.g., “prevents people from getting help or seeking treatment”); responses earning a score of 1 included neither (e.g., “Media has a huge influence on stigma, but it’s not always negative”). Responses earning a score of 5 showed exemplary understanding of why stigma should be erased (e.g., “People with mental illness have been injured emotionally and physically because of stigma. Some people even get to the point where they commit suicide due to all of the discrimination. By erasing stigma, we can show the public that it’s ok to be different and give everyone support. The most important part is that lives can be saved”).

Initial reliability of the manuals was established by three independent coders, unaware of participant group status, with pairwise correlations at or near r = .80 for each manual. Subsequently, a highly trained volunteer coded all responses, unaware of the respondents’ LETS versus No-LETS status. A 25 % inter-agreement check with the first author (who wrote the coding manuals) yielded pairwise correlations of r = .91, .88, and .86 for the A, B, and C codes, respectively. To lessen the potential threat of demand characteristics, the volunteer’s scores were the ones used for the present analyses.

Results

We predicted that those students who participated in a LETS club for one full semester, compared to those who did not, would exhibit better attitudes, less social-distancing tendencies (more acceptance), and more positive actions performed regarding stigma reduction, all related to mental illness. However, we did not predict that LETS group members would show enhanced knowledge about mental illness, given that the LETS activities were focused on changing attitudes and behavior rather than supporting acquisition of facts about mental disorder. We also predicted that LETS participation would be associated with a greater understanding of stigma-related ideas and actions, as measured via the open-ended questionnaires. We ultimately determined that Questions A (stigma awareness) and C (antistigma rationale) were conceptually similar and thus averaged scores across these two variables into one called “stigma understanding”—which essentially included what stigma is and why it should be reduced—for our analyses. Doing so reduced the N of outcome measures. In all, it was hypothesized that LETS club members would demonstrate more stigma understanding and would yield a greater sense of potential antistigma actions.

We considered LETS vs. No-LETS participants as a between-groups factor, given that within-subjects, pre–post evaluations were not possible with our quasi-experimental, program evaluation design. The two groups (LETS vs. No-LETS) were compared via independent-samples t tests. Again, we expected higher mean scores—which represent the “positive” end of all measures—for LETS participants with respect to attitudes, social distance, performed positive actions, stigma understanding, and potential antistigma actions, but we did not predict a significant difference with respect to mental illness knowledge per se. For the analyses below, all data were analyzed via SPSS, v. 20 (IBM Corp. 2011). Table 1 shows the means and effect sizes for each group with respect to all outcome variables. Sample sizes vary slightly across dependent variables because of missing items (see Table 1). Missing data were rare (and related to participants’ omission of selected items); thus, we did not impute missing values but instead used only cases with full records for each analysis.

Table 1 Summary of descriptive analyses and results of independent-samples t tests for group differences in quantitative scales and open-ended responses
Full size table

As shown in Table 1, differences in mean scores between the two groups (LETS vs. No-LETS) for all four of the quantitative scale variables were statistically significant, favoring LETS participation. Effect sizes (Cohen’s d) were greatest for social distance (d = .61) and positive actions (d = .53), indicating that, beyond statistical significance, those who participated in a LETS club were more willing than their No-LETS counterparts to engage in various social interactions with persons with mental illness and performed more discrete actions related to stigma reduction, with medium strength. The effect size for attitudes was almost medium (d = .46), whereas knowledge showed the smallest effect (d = .37).

In addition, statistically significant mean differences between the LETS group and No-LETS group were found for the two variables assessed by the open-ended questionnaires. Those in the LETS group demonstrated better stigma understanding (d = .69) and articulated stronger potential antistigma actions (d = .89) than those in the No-LETS group. These represent medium and large effects, respectively.

Discussion

In the absence of convincing trends in favor of its overall decline (see Link et al. 1999; Pescosolido et al. 2010), the stigma of mental illness remains a major threat to the lives of individuals touched by mental illness (Hinshaw 2007; Thornicroft 2006). Antistigma program evaluations and ideas for future interventions point toward the need for dynamic and innovative approaches to mental illness stigma reduction. Past methods have often been brief and focused on strategies directed toward adult populations that have shown, at best, short-lived effects. Particularly for youth, it is essential that interventions be relevant and developmentally-appropriate, as well as interactive and experiential. Moreover, evaluations should include outcomes explicitly related to relevant stigma domains, including attitudes, social distance, and behavioral actions, beyond knowledge enhancement per se. LETS, and its evaluation contained herein, incorporate all of these guiding frameworks.

The results of this preliminary, quasi-experimental program evaluation support our hypothesized outcomes for both the quantitative and qualitative measures. Compared to the No-LETS condition, the quantitative scales reveal that high school-aged youth participating in a school LETS club demonstrated better attitudes about mental illness, showed less social-distancing tendencies related to individuals with mental illness (i.e., more acceptance), and reported performing higher proportions of positive antistigma actions. The LETS members also had greater knowledge of mental illness than non-participants, although this finding was not anticipated. Moreover, as gleaned from reliably coded open-ended responses, LETS club participants showed greater stigma understanding and articulated better potential stigma-reduction activities. Our findings provide initial support for the notion that participation in a LETS club may facilitate globalized improvements in stigmatizing tendencies toward people with mental illness. Such participation may also enhance stigma understanding—in terms of what it is and why it should be overcome—and awareness of how best to reduce stigma.

Because measures used to assess stigma reduction are typically constructed for adults (Link et al. 2004), finding adolescent-focused measures was a challenge. One of our objectives was to compare our quantitative findings to those from the Wahl et al. (2011) program evaluation, but the absence of reported standard deviations from their evaluation limited our ability to make effect size comparisons. Instead, we compared two of our findings (attitudes and social distance) to a meta-analysis of studies focused on adolescent antistigma interventions (k = 19) (Corrigan et al. 2012). Corrigan et al. divided their findings into education vs. contact (in person vs. by video). Because LETS most closely resembles an in-person contact strategy, we therefore contrasted our findings to that condition. Our attitude-related d was .46, compared to the meta-analytic d = .37. For behavioral intentions (including social distance), the respective d’s were .61 vs. .46. (Given the rarity of actual behavior assessment, we could not find a measure comparable to our positive-action outcome.) In short, our pilot evaluation of LETS participation yielded effect sizes comparable to (and even higher than) those from a definitive review of relevant literature.

Linking our unanticipated knowledge outcome to other studies was also difficult, partly because of missing standard deviation summaries (e.g., DeSocio et al. 2006; Essler et al. 2006; Pinfold et al. 2003; Roberts et al. 2007; Stuart 2006; Wahl et al. 2011). Watson et al. (2004) utilized a knowledge measure with 13 true or false (or “not sure”) items in their middle-school education program evaluation, finding, from pre- to post-intervention, a knowledge effect size of d = 1.22. This large effect—more than triple the knowledge effect size found in the present investigation (d = .37)—is not surprising, given that Watson et al. targeted knowledge enhancement. Of note is that their effect size for attitudes did not even qualify as small (d = .08). Again, educational antistigma strategies work to improve knowledge but may not be wholly sufficient.

Whereas the quantitative scales were indices of participants’ stigmatizing beliefs and tendencies toward people with mental illness, the open-ended responses embodied participants’ own stigma-related ideas and experiences related to mental illness. Our reliably coded responses revealed that LETS participants showed enhanced understanding of mental illness stigma and more sophistication in terms of justifying and instantiating means of reducing such stigma, compared to No-LETS participants. We could find no comparable measures from previous research. Of course, whether those open-ended responses can and will result in actual behavior change is the ultimate question.

Indeed, behavior change is widely regarded as the most important antistigma domain (e.g., Corrigan and Shapiro 2010; Schachter et al. 2008; Thornicroft et al. 2007). Our results are provocative, in that we found smaller effect sizes for knowledge and attitudes, as compared to the more behaviorally-driven constructs (i.e., social distance and positive actions) we measured, even though the latter were self-reported in our evaluation. Bolstering the contention that behavior change is the “gold standard,” Corrigan and Shapiro (2010) quoted a person with mental illness: “It would be nice if people didn’t think bad things about me but what I really want is for them to stop blocking me from getting back to work” (p. 913). Thus, it is important to measure relevant behaviors specifically, beyond social distance scales (Link et al. 2004; Thornicroft et al. 2007). Our findings provide initial support for the value of LETS in prompting specific behavioral practices intended to reduce stigma.

Whereas educational methods aim to influence stigmatizing attitudes and behavior primarily by increasing knowledge about the outgroup, LETS features a multidimensional, experiential strategy that incorporates mutual peer-to-peer interactions within a context that promotes shared discussions of “difference.” Furthermore, most youth-related antistigma efforts are brief, thus limiting the potential for gains related to antistigma ideals. LETS, however, provides the opportunity for youth to connect in a growing and continuous relationship with one another via a sturdy platform of acceptance and change through regularly scheduled meetings across multiple semesters. An encouraging consideration is that our confirmatory findings were realized without the explicit assistance of trained teachers or outside professionals. Other antistigma efforts (especially in the form of education-based curricula) inherently require some form of lecture-style instruction, yet LETS clubs operate with minimal adult support. Also, the ease with which clubs can be started and maintained makes them particularly appealing. In all, the LETS foundation’s divergence from educational paradigms makes our distinctive findings eminently realistic.

Beyond the quasi-experimental nature of our evaluation, particular mechanisms that may underlie the success of LETS clubs are difficult to identify. Thus, we can only make educated claims in this area. A potential mechanism is contact with outgroup members. We note that membership in LETS clubs was not restricted to participants with mental disorders; indeed, the hope is that antistigma promotion via LETS occurs for a wide range of adolescents. Although we could not directly ascertain the diagnostic status of club members, we assume that members were exposed, on some level, to individuals with mental illness. This assumption is reinforced by the fact that LETS provides a multitude of opportunities for intergroup contact, including (but not limited to) in-person contact during club meetings and community-outreach activities, interactions with guest speakers who share their experiences of living with mental health problems, plus video-based contact. Although direct contact marks the ideal, indirect contact through videos showing first-person perspectives of mental illness also has strong support for reducing stigma (e.g., Mann and Himelein 2008; Reinke et al. 2004; Stuart 2006). Tolomiczenko et al. (2001) cautioned, however, that videos viewed without opportunity for follow-up discussion may actually have an opposite effect, intensifying negative attitudes. LETS appears exempt from this precaution, given its discussion-focused nature.

Contact with outgroup members may decrease stigma through a number of mechanisms, three of which were identified in a seminal meta-analytic review of the processes that may mediate the association between contact and reduced prejudice (Pettigrew and Tropp 2008): increased knowledge, empathy enhancement (together with perspective taking), and anxiety reduction. Interestingly, of the three, knowledge had the smallest effect size. Pettigrew and Tropp propose that “Affective factors, such as anxiety reduction and empathy, are clearly major mediators relative to the more cognitively oriented mediator of knowledge” (p. 929). A viable explanation related to the present investigation is that the mutual and benign intergroup interactions promoted in LETS club meetings may work to reduce stigmatizing tendencies by acting on these important affective mediators and only secondarily through enhancing knowledge of outgroup members. By encouraging disclosure and “togetherness” across social groups—particularly with respect to those with mental illness—LETS members may identify with their diverse group of fellow club members while incidentally learning about mental health conditions.

Although the findings presented in this evaluation are promising, several key limitations must be highlighted. First, this study necessitated a quasi-experimental design, which prohibited the opportunity to control for confounding variables that may have contributed to the observed differences between the two conditions. A randomized experimental trial would be possible through random assignment of interested youth to LETS participation versus delay of a semester or year. Important, but more difficult, would be an alternative intervention, such as knowledge enhancement about mental illness. Second, we did not obtain club-specific practices in this evaluation, precluding determination of the exact processes or mechanisms by which LETS clubs may facilitate stigma reduction. Future research should include assessment of records (by club historians) about the particular activities in which clubs engaged. We do know, however, from the LETS Start-up Guidebook, that suggested activities included experiential (e.g., participation in community events, collaboration with other school clubs) and contact-based (e.g., guest speakers, movie screenings, open discussions about members’ issues and experiences) opportunities.

Third, the sample was self-selected, suggesting that participants with a predisposed interest in antistigma ideals may have been the ones more likely to join a LETS club. It is therefore uncertain whether their positive scores were related to features of the LETS program versus selection bias. We hasten to point out, however, that the No-LETS group also constituted participants motivated to join, as they voluntarily attended the LETS Summit and LETS school meetings, which were designed as introductions to LETS for motivated students. We simply measured them before they had the opportunity to begin LETS clubs. Accordingly, motivation should have been relatively equalized across both conditions.

Finally, the use of self-reported surveys posed limitations related to validity. As is the case with all self-report measures, those used in the current study may have been subject to social desirability bias: participants may have been inclined to answer more positively for reasons related to social norms. Yet the open-ended responses did not always conceal participants’ detrimental views. For instance, one participant from the No-LETS group wrote in response to Question A: “Stigma means something that is wrong with you or like a disease.” Another mentioned, as a way to overcome stigma: “We can make a medicine that helps prevent problems from inside the womb.” Both responses reveal the underlying belief that stigma is a core deficit within stigmatized individuals, rather than an external phenomenon that is imposed upon those who are stigmatized. Although the support here is anecdotal, it is noticeable how stigmatizing views may subtly present themselves in an open-response format. As we have indicated throughout this report, we emphasize that measures of direct behavioral change should be prioritized in subsequent research, despite the inherent problems in mounting such observational indicators.

In conclusion, youth-focused antistigma efforts have been relatively neglected (Link et al. 2004; Wahl 2002) or have focused rather exclusively on knowledge enhancement related to mental illness. Our support for stigma reduction—as facilitated by youth-directed, empathy- and action-based LETS clubs, within an adolescent sample—is therefore encouraging, particularly in light of the developmental trend for mental illness stigma to intensify with age from childhood through adolescence (Wahl 2002; Wahl et al. 2007). Thus, prevention and early education remain a top priority. The current findings suggest that prevention of negative attitudes, which may otherwise become deeply ingrained by adulthood, is possible, and that an easily-initiated, youth-led club design can provide a platform for such preventive efforts. Clearly, more rigorous and definitive clinical trials are required to bolster this claim.