Introduction

Suicide is the third leading cause of death for young people ages 10–24 and accounts for 12.3% of all deaths annually among 15- to 24-year-olds (Centers for Disease Control and Prevention 2007). In fact, historically more youth and young adults die from suicide than by cancer, heart disease, AIDS, birth defects, stroke, pneumonia, influenza, and chronic lung disease combined (U.S. Public Health Service 1999). Of course, youth suicide attempts are much higher than deaths and have additional and significant deleterious impact on peers, families, and communities. Recent research underscores the need for broad community-based prevention efforts to reduce the tragedy of suicide among the youngest members of society: Students with the greatest risk (i.e., those who report suicidal ideation) are significantly less likely than others to favor help-seeking through school personnel (Wyman et al. 2008). Clearly, prevention approaches cannot rely solely on suicidal youth to identify themselves and to seek help through formal channels.

A growing body of research on community-based youth suicide prevention efforts has tested several recommended strategies (e.g., Center for Mental Health in Schools at UCLA 2007; Centers for Disease Control and Prevention 1992). One promising approach, gatekeeper training, teaches community members to identify signs of depression and other behaviors that put individuals, including youth, at heightened risk for suicide (Centers for Disease Control and Prevention 1994; Eggert et al. 1997; Gould and Kramer 2001; Isaac et al. 2009; Kataoka et al. 2007; Keller et al. 2009; King and Smith 2000; Mackesy-Amiti et al. 1996; Mann et al. 2005; Quinnett 1995; Reis and Cornell 2008; Rodgers et al. 2007; Tierney 1994; Turley and Tanney 1998). The gatekeeper training model depends upon changing individuals’ index of suspicion and willingness to inquire directly about distress, persuade suicidal youth to accept help, and provide the bridge to local referrals. Gatekeeper training for suicide prevention may target teachers, coaches, or other key stakeholders in the community who are in a natural position to carry out informal surveillance, detection, and assistance for youth in need of intervention.

Assessing the direct influence on rates of suicide of any prevention program, including gatekeeper training, is challenging given the relative infrequency of suicidal behavior and the numbers of study participants required (Brown et al. 2006). Testing theory-based proximal variables is one strategy to address this challenge, and several community-level gatekeeper training studies in a variety of settings have shown changes in participants’ knowledge and attitudes (i.e., self-efficacy) after training and at follow-up (Brown et al. 2007; Cross et al. 2010; Keller et al. 2009; Matthieu et al. 2008; Tompkins and Witt 2009; Tompkins et al. 2010; Wyman et al. 2008). For example, a randomized control trial of gatekeeper training for adults in 32 schools found significant increases in participants’ knowledge of youth risk factors for suicide, intentions, and perceived efficacy for intervening (Wyman et al. 2008). There was a more moderate effect, however, on the number of students asked about suicide by trained compared to non-trained staff at 1-year follow up. Moreover, the increases in asking about suicide occurred primarily among school staff who reported they inquired about suicide prior to gatekeeper training. The authors suggest that direct skills training may improve the transfer of knowledge and enhance self-efficacy for greater ability to communicate with students. Most training programs for suicide prevention do not employ active learning strategies such as role play to practice talking about suicide, despite the social taboo and inherent difficulty in doing so. Moreover, few studies have rigorously assessed gatekeeper skills directly or examined maintenance of demonstrated skills (Beidas and Kendall 2010; Cross et al. 2010). Clearly, skill development and assessment are important foci for gatekeeper training studies if the goal of training is behavior change and subsequent use of gatekeeper skills to intervene with those at-risk. In this way, examination of gatekeeper skills in a role play scenario, and maintenance of those skills over time, may serve as proxy measures of suicide prevention program effectiveness along with changes in knowledge, attitudes, and referral rates.

Adult learning theories indicate that behavior change, and thus skill development, is most likely when active learning strategies are employed (Beidas and Kendall 2010; Humair and Cornuz 2003; Joyner and Young 2006). The rationale for including practice in skill development is simple: Simulation and role play experiences have been shown to enhance learning, particularly when learning new or challenging skills (Joyner and Young 2006; Kemeny et al. 2006; Proude et al. 2006; Taylor et al. 2005). Practice improves skills partly because it stimulates a degree of affect shown to enhance skill development and retention (Preuss and Wolf 2009; Smeets et al. 2009). The more a training situation reflects real world application, the more likely the trained behaviors will be used (Knowles et al. 2005). In the case of intervening with a suicidal youth, increased affect during skill practice may help manage the heightened emotions that are likely to arise when faced with a potentially suicidal youth. In the present study, we test the impact of role play practice on learning outcomes, including skills, in a school-based gatekeeper training program.

School-based suicide prevention for youth may require a broad network of gatekeepers. As noted, recent evidence indicates that students with the greatest risk for suicide, those who report experiencing suicidal ideation, are less likely than others to endorse help-seeking through school personnel (Wyman et al. 2008). Moreover, Keller et al. (2009) found that the impact of gatekeeper training for school personnel was significantly less than for other child service workers such as child welfare workers, juvenile justice employees, and health department nurses. Parents are integral to school communities and may be important partners for comprehensive suicide prevention. They are resources for their own children and their children’s friends and serve as informal gatekeepers for youth in a variety of contexts outside of school, including teams, religious groups, and neighborhoods. Including parents in community-based gatekeeper training efforts may be particularly effective because they have opportunities to observe risky behaviors not necessarily observed in schools; they have long-term, consistent relationships with youth in their children’s friendship groups; and they may assume a broad responsibility for the youth in the community. To date, despite the potential to be part of an effective surveillance network for youth, parents have not been targets of gatekeeper training studies for school-based suicide prevention. The current study begins to address the gap in knowledge about parents’ willingness and ability to contribute to school-based gatekeeper training.

The purpose of the current study is to (a) compare two training conditions by testing the impact of adding an active learning strategy (behavioral rehearsal) to standard gatekeeper training on knowledge, attitudes, and skills in a randomized controlled trial and (b) examine participant group differences in training outcomes in a broadly defined school community that includes school personnel and parents. We hypothesized that knowledge and attitudes would improve from pre to post training and be maintained at follow-up for both training groups but that the behavioral rehearsal group would report more positive attitudes (e.g., greater self-efficacy) after training. More importantly, participants in the behavioral rehearsal condition were expected to demonstrate better gatekeeper skills after training and at follow-up than those who received the standard training. Secondarily, we anticipated that school personnel would outperform parents on baseline knowledge about youth suicide and efficacy for intervening, based on daily exposure to youth and youth-related information, but that gatekeeper training would erase those differences.

Method

Study Setting

A community-based partnered research project (Israel et al. 1998) involved collaboration between a university medical center and a rural–suburban school district in upstate New York. The school district is made up of approximately 4,200 students in six schools. The partnership formed in response to the school district’s Youth Risk Behavior Survey (YRBS; Centers for Disease Control and Prevention 2005) findings showing higher rates of self-reported student suicidal thoughts, plans, and attempts than the surrounding communities and county. The team consisted of university investigators and district mental health intervention and prevention leaders. We invited a variety of school personnel and parents to participate in training about how to intervene with distressed youth. In addition to higher rates of suicidal thoughts and behaviors, the YRBS responses revealed concerning rates of self-reported depression and bullying among district students. Therefore, the training was expanded to also include some information about these topics. The standardized brief community gatekeeper training (Question, Persuade, Refer [QPR]; Quinnett 1995) was preserved intact as the core of the training, however, and the focus of the current study.

Participants

The community partners’ goal was to infuse gatekeeper training across many levels of the entire school community. Therefore, school personnel (including mental health professionals, teachers, and bus drivers) were offered gatekeeper training for youth as part of professional development in-service and were subsequently invited to participate in research. One hundred fifty-nine school personnel attended the training. Seventy-two percent (72%, n = 114) of these trainees agreed to participate in the study. Parents were initially notified about the opportunity for gatekeeper training and the research program by letter from the district. The letter was sent to approximately 800 parents who were participating in the district’s Safe Homes Project, pledging to keep their students safe from substance use at home. Research staff then personally contacted 206 parents from the Safe Homes directory, including those who had shown an interest in student activities in the past, to further describe the training and associated research with the goal to train and complete follow up with approximately 50 parents. Thirty-four percent (n = 71) of those contacted agreed to participate; however, 15 consented parents subsequently did not attend the training. Our relatively high recruitment rate may reflect the study’s relationship with the district’s well-integrated Family Support Center that provides free mental health services to students and their families, as well as the outreach to parents through personal phone calls. All participants were compensated with a gift card ranging in value from $10 to $50 depending on their level of participation. All participants gave written informed consent, and the project was approved by the university Institutional Review Board as well as the school district board of education.

Demographic variables and other descriptors of the sample are provided in Table 1. Of 170 participants in the study, 114 were school personnel (79 teachers/aides/administrators, 22 mental health professionals, and 13 bus drivers), and 56 were parents. The sample was primarily Caucasian (93%) and female (87%), with an average age of 42 years (SD = 9). Within the parent group, 84% reported full- or part-time employment outside of the home, and 86% were married at the time of the pre-assessment. There were no differences between non-mental-health-trained school personnel and parents in terms of number of previous suicide prevention trainings or previous contact with a suicidal individual. Differences between mental health professionals and other groups were significant and expected on these variables. There were significantly more females in the parent sample, χ2(1) = 6.15, p < .01, but no differences in age compared to the school personnel sample. Highest education level differed between the groups, χ2(4) = 50.46, p < .001: School personnel (excluding mental health professionals due to required education level) had more masters degrees, and parents held more associates degrees. Participants self-rated the quality of their relationships with youth (e.g., “students/youth talk to me about their thoughts and feelings”) at baseline. Mental health professionals had higher scores on Natural Gatekeeper Relationship status, χ2(2) = 18.83, p < .001, but no difference was found between school personnel and parents.

Table 1 Sample demographic and baseline variables
Full size table

Gatekeeper Training Conditions

Standardized community gatekeeper suicide prevention training for youth (Question, Persuade, Refer [QPR]; Quinnett 1995) was conducted by two certified QPR trainers in the school district. We chose this program because it met the goals of the community–university partnership. QPR is relatively brief, broadly disseminated, and the focus of previous research. Training groups averaged 14 participants. Separate trainings were held for parents and school personnel. Participants were randomly assigned to condition, blocked for equal numbers, prior to training. Consistent with QPR standardized trainings, the 1-h program consisted of a lecture, a 10-minute introductory video, distribution of booklets and referral cards, and a question-and-answer discussion period (Quinnett 1995). The lecture provides an overview of the epidemiology of suicide and current statistics, myths and misconceptions about suicide and suicide prevention, general warning signs of suicide, and three target gatekeeper skills. The short video includes interviews with people who have experienced suicide in their families, schools, and neighborhoods. The booklet contains an overview of the didactic presentation and reviews the gatekeeper role. Referral cards are distributed and function as wallet cards with prompts to recall gatekeeper skills emphasized in the training and information about referral resources. Referral information was tailored to the groups being trained; names and phone numbers of local and national resources were provided. Supplementary information on depression and bullying was presented for an additional 10 min. All participants (training as usual [TAU], training plus behavioral rehearsal) attended the large group training.

Participants in the standard training plus behavioral rehearsal (T + BR) condition were then provided an additional small group practice opportunity after the large group presentation. These participants first observed a brief role play by the trainers who discussed and demonstrated “wrong way/right way” interactions between a caring adult and suicidal student. T + BR participants were then divided into groups of three for role play practice. Each group was provided three scenarios, back stories for the suicidal student and adult gatekeeper roles, and instructions for an observer role. Participants were instructed to rotate through the roles over the course of three role play opportunities. Scenarios for parent and school personnel were tailored to match practice experiences. (See “Appendix 1” for an example of role play materials.) The small group practice activity was 25 min in duration.

There were no significant differences between the two training conditions for participant demographic or descriptive variables such as gender, age, education, prior training experience, or contact with a suicidal person.

Data Collection Procedures

All participants completed study measures assessing knowledge and attitudes prior to and immediately following training and again at 3-month follow-up when questions about use of gatekeeper skills in the intervening period and diffusion of the training were also administered.

Immediately after training and at follow-up, all participants interviewed a “distressed youth” portrayed by a trained actor. The role play interaction required participants to respond to a high school student in their role as a school employee or parent. Research staff provided each participant with a brief, standardized “back story” which included the setting and details about the youth (i.e., actor portraying a student or neighborhood youth). A research team member accompanied each participant to a room set up with a video camera and a waiting actor and was present throughout the interaction. The actor followed a standardized protocol that increasingly signaled signs of distress covered in the training. The actor also signaled the end of the interaction assessment by delivering a specific phrase at which time the study staff member escorted the participant out of the room. Thus, the interview ended when the actor completed all the scripted prompts to the participant who had the clear opportunity to demonstrate each of the QPR steps. Participants were instructed to converse naturally with the actor for 5–10 min and respond to the best of their ability within the context of their role (e.g., teacher, parent). (See “Appendix 2” for an illustration of the actor script.) After the videotaped interaction, a brief informal assessment was conducted by a study team member to ensure the participant was not distressed by the experience and willing to continue with the study. A follow-up appointment was then scheduled.

At 3-month follow-up, all participants were videotaped interacting with a different actor using a different script matched for difficulty. Procedures for the actor–participant observed interaction were the same with one exception: Following the interaction, the study team member offered brief feedback to the participant on his or her demonstrated gatekeeper skills. The feedback focused on general communication skills (e.g., active listening) and suicide specific skills (e.g., asking about suicide directly). The study team member provided at least one positive feedback and, if the participant struggled, encouragement to ask directly about suicide and/or provide an effective referral. Participants were debriefed about the study goals, completed the questionnaire, and given compensation.

Actor Training

Actors were screened by the principal investigator for mental health concerns prior to participating in the study. Procedures for training actors for standardized “patient” assessment (Blatt et al. 2007; Hardoff and Schonmann 2001; Humair and Cornuz 2003) of gatekeeper skills replicated previous studies (e.g., Cross et al. 2010). For this school-based study, the patient was a suicidal high school student. Eleven college student actors (6 male, 5 female) who could credibly present as 16-year old students were hired and trained for 6 h to criterion which included providing standardized responses to specific questions from participants (e.g., “do you want to hurt yourself?”) and inquiries about feeling suicidal, pacing challenges (e.g., being asked about feeling suicidal early in the interaction), ability to deliver stimuli/clues naturally and in the correct order, and repeating referrals provided by the participant. Two “suicidal adolescent” scenarios were developed for the actors to learn for the study. Each script was tailored and matched for school personnel and parent contexts, resulting in two parent and two school personnel scripts. The order of script delivery (Script 1 or 2) was randomized at post-training and the other script was delivered at follow up. In order to aid character development and interactive yet standardized delivery of the script, the actors were provided with a detailed back story outlining their adolescent character’s role as a youth at risk for suicide, the context of the interaction, and details on the history of risk factors leading up to the suicidal crisis. Scenarios included a precipitating event and increasingly direct warning signs of suicide (e.g., hopeless statements, irritability, suicidal ideation and plans). Although some vague ad lib and non sequitur comments were allowed, as would be made by someone in distress (e.g., “I don’t know”), actors were trained to deliver all scripted lines in order. Actor training included role play “test” interactions with the study team, which were scored for adherence to the script. Feedback was provided to actors until 100% adherence was reached.

Measures

Demographics and Descriptive Variables

The pre-training measurement packet included sociodemographic items: age, gender, education, race, ethnicity, employment status (for parents), job role (for school personnel), previous experience with suicide prevention training, and exposure to suicide. Based on the Wyman et al. (2008) finding that asking about suicide post-training was related to baseline “natural gatekeeper” status with students, we assessed participants’ relationship status with students using the same items. Natural gatekeeper status with students (i.e., “students/youth talk to me about their thoughts and feelings”) was operationalized according to Wyman et al. (2008) using a median split (excluding mental health professionals): “Low natural gatekeeper” had means ranging from 0 to 2 (N = 85), and “high natural gatekeeper” had means greater than 2 (N = 60).

Declarative Knowledge

Participants completed a 14-item assessment of declarative knowledge about suicide-related facts (Cross et al. 2007; Wyman et al. 2008) provided in the training at pre- and post-assessment as well as at 3-month follow-up. Items include multiple choice and true/false questions. The knowledge score is the percentage of correct responses.

Attitude Measures

Self-perceived knowledge about suicide (Cross et al. 2007, 2010; Matthieu et al. 2008). Participants were asked to respond to 5 items about their perceived knowledge about suicide (e.g., “Please rate your knowledge of warning signs of suicide”) using a 5-point Likert scale, 0 (poor) to 4 (excellent), at all three assessment points. In the current study, Cronbach’s alpha was .94. Results are presented as an average score.

Self-efficacy for intervening. A 5-item measure of efficacy for intervening with a suicidal individual (e.g., “I feel confident that I can identify signs of emotional distress in students”) used previously (Cross et al. 2010; Matthieu et al. 2008) was slightly modified for the current study. Due to an error in administration, this measure was administered to participants recruited in the second year of the study only (N = 67). Baseline demographics and other baseline variables were similar across those who were missing values versus those who completed these items. Cronbach’s alpha was .81 for the current sample. Results are presented as an average score.

Observational Rating Scale of Gatekeeper Skills (ORS-GS)

Minor contextual revisions were made to the ORS-GS scoring system (Cross et al. 2010) based on the youth and school-based scenarios used in the present study. The scale has five items resulting in four domains: General Communication (two items: active listening, clarifying questions) and three suicide-specific skills (asking a direct Question about suicide, Persuasiveness, Referral). Each item is rated on a 4-point scale with specific behavioral descriptions for each item and rating. The lowest rating (0) indicates an absence of skill and the highest rating (3) indicates competent demonstration of the skill. The ORS-GS scores are combined for a Total Gatekeeper Skills score. Four raters, blind to condition as well as observation time point, were trained to code the videotaped participant–actor interactions. Inter-rater reliability was calculated for each of the four domains at post- and follow-up assessment using intra-class correlation ([ICC]; Shrout and Fleiss 1979). Thirty percent (30%) of observations were double coded, and a randomly selected 10% were also coded at two time points to assess rater group drift over time. All ICCs were adequate to excellent (range 0.65–0.92) except for the Persuade item (0.40). Disagreements were resolved through consensus meetings and consensus scores were used in the analyses. Drift analyses showed coders maintained adherence to coding rules over the duration of the 17-month rating phase of the study and did not drift (range across 4 domains = .80–.97) between Rating Time 1 and Rating Time 2.

Actor Adherence

To examine if participants received equivalent stimuli during the observed skills assessment, actor adherence to the prompts and scripted lines in scenarios was scored dichotomously (yes/no) and rated independently for each observation. Adherence ratings were conducted separately from ORS-GS coding. We found that actors were highly adherent and delivered the scripted information as written. At post-training, actors averaged 98.76% of the standardized script delivered as written, and at follow-up, actors averaged 99.25%. The actors were blind to condition; however, there is a possibility that they could have become aware of condition at post-training due to timing of training logistics. In order to rule out potential variations in actor performance, we conducted the following comparisons: (a) we compared actors’ adherence to the script across both conditions at post-training, and (b) we compared actors’ adherence at post versus follow-up. These analyses indicated that actors’ adherence did not differ by condition and actors’ adherence did not differ at either time point.

Gatekeeper Behavior and Diffusion

At 3-month follow-up, participants self-reported use of gatekeeper skills since training (i.e., referrals), their experience of being a gatekeeper at work and in the community, and diffusion of the training content and materials to others.

Data Analysis

Repeated measures ANOVA were conducted to examine changes in variables over time and by both training condition and participant group. We used t-tests to examine group differences, impact of previous exposure to suicidal person or training, and differences in natural gatekeeper relationship on referrals at 3-month follow-up. Chi-square analyses were conducted to examine group difference in gatekeeper behaviors and diffusion of gatekeeper training information. We also used t-tests to compare total groups reached in diffusion by training condition and participant group. All data analyses were conducted using the SPSS for Windows Version 16.0 (SPSS 2007).

Results

Participant Retention

Overall, 96% of study participants were retained across three assessment points, although not all participants completed standardized actor videotaped skill assessments at both post and follow-up. Retention in the school sample was 97%. Of the school sample, one participant withdrew prior to the training, one participant declined to complete the 3-month follow-up video skills assessment but completed the survey, one participant withdrew before the follow-up, one participant was lost to follow-up, and five participants have incomplete skill assessment observational data due to technical difficulties. Retention in the parent sample was 95%. One parent participant withdrew after the post-assessment, two participants did not present for the 3-month follow-up, and two participants declined to complete the 3-month follow-up video skills assessment but completed the survey. Five parent participants also have incomplete observational data due to technical difficulties.

Declarative Knowledge

A repeated measures ANOVA tested changes in Knowledge about Suicide from pre to post and at follow-up for the two training conditions (TAU, T + BR). There was a significant main effect for time but no effect for condition. That is, knowledge increased from pre- to post-training and was maintained at follow-up regardless of training condition, F(2, 156) = 17.73, p < .001. We examined participant group differences and found that, consistent with our expectation, mental health professionals’ knowledge was not enhanced by the training. They had significantly higher knowledge at pretest (mean = 77.92%, SD = 8.51) than other school personnel or parents, and this knowledge was maintained at post (mean = 81.49%, SD = 10.95) and follow-up (mean = 80.84%, SD = 11.54). Training enhanced knowledge at posttest, however, for other school personnel and parents; there was no difference between these two groups. Higher knowledge scores were maintained at 3-month follow-up for both participant groups

Because the study is a test of community-level gatekeeper training, and because mental health professionals are trained clinicians who demonstrated high levels of knowledge and positive attitudes prior to training, we focus the remainder of analyses on the non-clinician school personnel and parent participant groups.

Attitudes: Self-Perceptions and Self-efficacy

A repeated measures ANOVA was conducted to test changes in Self-Perceived Knowledge about Suicide and Self-efficacy to Intervene at all three time points. Contrary to our expectation, there was no effect for training condition, and no time-by-training condition interaction for either attitude measure. On Self-Perceived Knowledge, participants reported an increase from pre to posttest and maintained their self-perceptions at follow-up, F(2,139) = 302.99, p < .001. The same pattern was found for Self-efficacy to Intervene, F(2,77) = 90.35, p < .001 (time effect). We examined the two participant groups (school personnel, parents) on the attitude measures and found no significant difference between the groups on Self-Perceived Knowledge about Suicide or Self-Efficacy to Intervene. Pretest, posttest, and follow-up means, along with effect sizes, are reported in Table 2.

Table 2 Knowledge, attitudes, observed gatekeeper skills, and referrals for training conditions and training groups
Full size table

Observed Gatekeeper Skills

This set of analyses was first run including both training condition (TAU, T + BR) and participant group (school personnel, parents). No condition-by-participant group interactions were significant, however, and the sample n within cells were relatively small. We therefore ran analyses separately for training condition and participant groups. Main effects for time are not different between sets of analyses. Thus, our primary analyses focused on the impact of behavioral rehearsal compared to gatekeeper training as usual, immediately after training, and at follow-up on observed skills (Table 2). We examined ORS-GS scores for the two conditions (TAU, T + BR) and two participant groups (school personnel, parents) across time. In repeated measures ANOVA, participants in the T + BR condition scored significantly higher than those in the TAU condition for the Total Gatekeeper Skills score, F(1,127) = 6.25, p < .05), consistent with our hypothesis. Examination of domains showed a significant effect for condition on General Communication, F(1,126) = 16.31, p < .001, and a trend for Asking about Suicide, F(1,127) = 3.05, p = .08. There was also a main effect for time for the Total Gatekeeper Skills score, F(1,127) = 11.18, p < .001. Looking at the individual domains, there is a significant main effect for time for asking a direct Question about Suicide, F(1,127) = 16.20, p < .001, and for making an appropriate Referral, F(1,127) = 4.64, p < .05. In each case, follow-up scores were significantly lower than posttest scores. There was not a significant interaction of condition by time in any analysis. A comparison of participant group scores on the ORS-GS revealed that school personnel performed significantly better than parents on one suicide specific item: Referral, F(1,127) = 5.88, p < .05. There are no other differences on observed skills for the two participant groups.

Gatekeeper Behavior and Diffusion

We examined the two training conditions in terms of self-reported referrals at 3-month follow-up and found no difference between TAU and T + BR groups. Compared to parents, however, school personnel reported significantly more referrals during the intervening period, t(120.80) = 2.50, p < .05. We considered the impact of previous exposure to suicide on outcomes by combining two items (prior contact with someone who was suicidal and participation in previous suicide prevention training) and found that exposure was associated with more referrals at follow-up, t(130.84) = −3.19, p < .01; previous exposure = .97 (SD = 1.73), and no previous exposure = .33 (SD = .63). We also assessed participant self-reported gatekeeper behavior at work and in the community (e.g., “since the training have you acted more like a gatekeeper”) and found no effect for training condition. We examined the percent of school personnel and parents who responded “yes” and found that there was no difference in terms of acting like a gatekeeper in the community (school personnel = 54.1%, parent = 56.6%) but that school personnel were significantly more likely to report engaging in gatekeeper behaviors at work (school personnel = 83.9%, parents = 50.0%, χ2(1) = 17.59, p < .001) than employed parents. Finally, we performed t-tests to assess if baseline Natural Gatekeeper Relationship status with students predicted gatekeeper skills or referrals after training and found there was not a significant relationship, t(137) = −1.67, p = .10.

Participants reported diffusion of the gatekeeper training information (e.g., whether they discussed the training or showed materials to others) and suggestions that others take the training at 3-month follow-up. We found that diffusion of gatekeeper training information was extensive. Table 3 shows that almost all participants (96%) discussed the training with others, and about one-third recommended the training to others. More participants in the T + BR condition suggested training to at least one group (78.3%, n = 54) than those in the TAU condition (61.6%, n = 45), χ2(1) = 4.64, p < .05. Overall, parents discussed the training with significantly more groups of people than school personnel, t(140) = 3.20, p < .01. Examination of individual items revealed that a large percentage of participants in both groups discussed the training with coworkers (73% of school personnel, 64% of parents) and spouses (73% of school personnel, 81% of parents), but parents were significantly more likely to discuss the training with friends, χ2(1) = 8.96, p < .01; acquaintances, χ2(1) = 6.72, p = .01; and others, χ2(1) = 16.74, p < .001, in their community. Chi-square analyses showed that school staff were significantly more likely to suggest the training to coworkers, χ2(1) = 10.47, p = .001, and parents were more likely to suggest the training to friends, χ2(1) = 7.62, p < .01, and other people, χ2(1) = 10.94, p = .001, in their community. Parents also showed the training materials to their children, χ2(1) = 6.72, p = .01, and spouses, χ2(1) = 6.58, p = .01.

Table 3 Self-reported diffusion of training at 3-month follow-up
Full size table

Discussion

The goal of the current study is to assess the potential for enhanced proximal outcomes of brief, community-based gatekeeper suicide prevention training for youth by (a) testing the impact of a theoretically supported, active learning behavioral rehearsal condition and (b) broadening the targets for a school-based training strategy to include parents. We used a randomized control design and standardized patient assessment methods to evaluate gatekeeper skills following training and at follow-up. As expected, knowledge and attitudes improved from pre to post and were maintained at follow-up for both training condition groups. Our findings are consistent with previous research showing a brief (1-h) gatekeeper training program effectively changes knowledge and attitudes over time (Keller et al. 2009; Tompkins et al. 2010; Wyman et al. 2008). Contrary to our hypothesis, however, practicing gatekeeper skills did not further improve these enhanced knowledge or attitude outcomes.

We also compared the impact of gatekeeper training on school personnel and parents. Our results showed no difference between school staff and parents in terms of knowledge or attitudes about youth suicide prevention at baseline or after training. We were initially surprised that teachers and other school personnel who work daily with youth were no more knowledgeable or confident about intervening with suicidal youth than parents. Upon reflection, however, we realized that, with the exception of mental health professionals, school personnel do not typically receive training about mental health issues including suicide prevention (Koller et al. 2004; Walter et al. 2006). School personnel and parents are, therefore, equally uninformed when it comes to suicidality and, without specific training, hold similar attitudes and beliefs about youth suicide. This finding is particularly salient for the community partner because, unlike parents, teachers deal daily with a high concentration of youth, and up to 5–10% of students may have diagnosable mental health symptoms that are likely to impact learning and student well-being. The implication is that being a classroom teacher does not guarantee an educator will have a working knowledge of mental health concerns, especially suicidality, which might affect students. Considering the significance of suicidal thoughts and behavior among youth, this finding suggests that school-based training of teachers and administrators about how to identify and respond effectively to mental health issues that arise in the classroom should be part of a comprehensive approach to mental health issues in the school setting. Moreover, the training can be as brief as 1 h to have a positive and enduring impact on knowledge and attitudes. The community partners were particularly interested in this finding.

We anticipated that participants who had the opportunity to practice would demonstrate significantly better gatekeeper skills after training and at follow-up than those who received the standard training. Results showed that rigorously assessed, observed gatekeeper skills were improved with role play practice. In particular, the general ability to communicate comfortably with a youth in distress was enhanced, as was the specific ability to ask directly about suicide. Although the positive impact of practice persisted over time, both training conditions showed significant decrements at follow-up. Thus, while practice did improve skills the impact was certainly not perfect.

How can enhanced training skills be maintained over time? Because interacting with suicidal youth is a low-base-rate event, there is little opportunity for newly trained gatekeepers to use, and thus retain, their abilities. Simply put, skills decay without use. One clear implication is that boosters may be needed for effective gatekeeper training. Several strategies may be enlisted to support maintenance of gatekeeper skills including reminders via video applications for phones or web-based interactive practice opportunities (Hanauer et al. 2009).

Positive practice effects may be further enhanced and maintained with feedback. In the current study, we provided feedback to participants on their skills assessment with actors only at follow-up as part of the debrief process. Adult learning models indicate that feedback on performance during the learning process enhances outcomes (Hattie and Timperley 2007). Future studies that incorporate feedback on demonstrated gatekeeper skills during role play practice and, potentially, following skills assessment could strengthen gatekeeper skills which may improve maintenance over time.

We anticipated that role play practice would improve participants’ ability to ask directly about suicide. The difference between the two training conditions was consistent with our expectation, but it was only a trend. What accounts for this finding? One factor may be that there was an attempt to desensitize participants during the large group training attended by all participants: Trainers led the whole group in repeating “are you thinking of killing yourself” and “are you feeling suicidal.” This small addition to the standard training may have been sufficient practice to confound the difference between conditions. Nevertheless, survey feedback from participants across conditions indicated that asking directly about suicidal thoughts and feelings was extremely difficult. Given our findings, community-level gatekeeper training requires modifications to prepare participants to develop and comfortably use gatekeeper skills. Alternatively, it may not be reasonable to expect this level of gatekeeper training to transform all comers into active gatekeepers in their communities. A selection process that targets those who are most likely to learn and use gatekeeper skills within a brief training program may be necessary.

Another way to evaluate training outcomes is to define adequate gatekeeper behaviors. Cross et al. (2010) defined participants as having adequate gatekeeper skills if they asked directly about suicidality (Q-score = 3), used persuasive communication for help-seeking (P-score ≥2), and provided a referral that would successfully connect the distressed person to appropriate assistance (R-score ≥2). Based on their stringent criteria, 10% of participants demonstrated adequate gatekeeper skills at baseline, and 54% met criteria after training. In the current study, there was no difference between training conditions, and 56% of participants met this criteria immediately post training, with a decrement to 41% 3 months later. Clearly, there is room for improvement on suicide-specific skills developed in training and for maintenance of those skills.

We found one difference between parent and school personnel groups on observed gatekeeper skills: School personnel scored higher on the ability to provide an effective referral for suicidal youth. Moreover, at follow-up, school personnel reported greater use of the gatekeeper training by referring more students than parents during the intervening time period. Although it is likely that school personnel have greater opportunity to encounter distressed and suicidal youth than most parents as well as greater access to on-site referral sources, it is also possible that the gatekeeper training may require modification to prepare parent participants to effectively refer youth for intervention. It is likely that the school context, which has built-in processes for student intervention, provides a clear and normative referral process for school personnel. Unlike school staff, who use a variety of services to meet students’ academic, physical, and socioemotional needs as a matter of course, parents are faced with an unfamiliar, anxiety-provoking task of approaching a suicidal youth’s parents or initiating community services. Gatekeeper training for parents, or others who are not trained in the context of a supportive institutional setting, may require tailoring to meet the challenge of encountering a suicidal youth in the community and referring them for services. One solution for school districts with mental health services would be to institutionalize a link between counselors and trained gatekeeper parents. Although school districts are unique and may have specific processes to accomplish this link, we offer one for consideration. Many school systems have a designated prevention or outreach counselor on staff who could be identified as the link for trained gatekeeper parents to access a school-based referral process for a distressed or suicidal youth. This and other avenues could potentially assist trained parents to have access to mental health professionals in the school setting.

Two other comments are warranted with respect to participants’ use of this gatekeeper training. We found that previous exposure to suicide (either through contact with someone who was suicidal or other prevention training) was associated with referrals 3 months after training. This finding is consistent with previous studies that show exposure to training content is associated with positive post-training outcomes (e.g., Cross et al. 2010), and with adult learning theory (Knowles et al. 2005). The suggestion by Wyman et al. (2008) that communication with students may be enhanced with training is supported by the current study: Observed general communication skills were enhanced by a brief role play practice exercise. The extent to which targeted behavioral rehearsal can enhance adult relationships with students who may be at risk for suicide is an important focus of future studies. Communication skills may be particularly important when students approach school personnel or other adults for assistance and training, such as a brief role play rehearsal, may be an effective strategy.

In fact, a comprehensive approach to youth suicide will likely include both adult-targeted programs such as gatekeeper training and programs that target youth themselves as part of a multilevel, comprehensive approach to community-based suicide prevention (Fountoulakis et al. 2011). One program, Sources of Strength (SoS; LoMurry 2005), aims to increase youth–adult connectedness by engaging diverse adolescent peer leaders to help change student norms regarding the acceptability of suicide, help-seeking, and youth–adult communication. A recent study of SoS found a positive impact of the program on school-wide norms about help-seeking and suicidal behavior, with the largest, most positive changes occurring among suicidal youth (Wyman et al. 2010). The integration of adult- and youth-focused interventions is a promising next step for school-based suicide prevention. Assessing skills and changes in behavior as well as their use in practice over time are important outcomes of these interventions.

Finally, brief gatekeeper training information and materials were broadly diffused both inside and outside the school community network; almost every participant discussed and shared training information with coworkers, friends, and family members. Clearly, the impact of gatekeeper training extends far beyond the participants. Given the strong societal taboo against talking about suicide (Cvinar 2005; Lester and Walker 2006; Sudak et al. 2008), there may be a previously unacknowledged public health benefit with broad-based enhanced awareness of and comfort addressing youth suicide. Our finding that a brief behavioral rehearsal experience enhanced the spread of gatekeeper information beyond training-as-usual indicates that active learning strategies such as role play practice may expand the impact of training further. Parents spread the suicide prevention message broadly to their network including children, spouses, friends, and coworkers. They were more likely to suggest the training to friends and other people in their community whereas school personnel were more likely to recommend the training to coworkers. It may be that the context in which gatekeeper training is conducted, either as professional development or as part of a parent training, defines how participants share and use the training. Our finding that school personnel are more likely than (working) parents to report acting like a gatekeeper at work (in their case, school) also suggests that the context of the training may be a factor in subsequent gatekeeper behaviors. Future studies may test this hypothesis by explicitly asking participants to share what they have learned with others in their social and/or professional network and compare ‘targeted’ diffusion outcomes using social network methods.

There are several limitations to the study. The sample is relatively small with limited diversity and may not be generalizable. Although professional development for school staff is required, attendance at specific programs, including our gatekeeper training, was voluntary. School participants along with parent volunteers, therefore, may not be representative. In addition, and consistent with a previous study (Cross et al. 2010), the Persuade domain of the observational measure of gatekeeper skills had inadequate inter-rater reliability. This step in the Question, Persuade, Refer gatekeeper process may be somewhat elusive for a brief observational measure of skills using standardized patient methodology. Finally, our follow-up period was fairly short and not likely to capture use of training, particularly for parents. On the other hand, it was sufficient to reveal decrements in skills in both training conditions.

Conclusion

In conclusion, we found in a randomized control trial that incorporating active learning strategies into standard gatekeeper training enhanced observed gatekeeper skills. Practice, however, did not make perfect. The ability to directly ask about suicide was not sufficiently enhanced, and skills deteriorated over time for participants in both training groups, though those in the behavioral rehearsal group continued to demonstrate better overall retention of skills than those trained in the usual manner. Future studies to test a variety of procedures (e.g., technological booster applications) to maintain gatekeeper skills over time are needed. Parents and non-clinician school personnel did not differ in terms of learning outcomes, with the exception that school staff had greater facility with referrals and made more referrals over a 3-month period, likely due to greater opportunity. Parents demonstrated less facility with community-based referral procedures than school-based personnel and may benefit from a referral process that is linked to school services to be effective. Both parents and school staff diffused the training broadly, although in somewhat different patterns, likely covering different domains of a youth’s community. Brief gatekeeper training infused a broad network with the knowledge, attitudes, and, for many, the skills to help prevent suicide in their students and children.