Addressing youth suicide in the USA is a public health imperative: suicide is the second leading cause of death among young adults 10–34 years old (Centers for Disease Control [CDC], 2016). Further, 17.2% of high school students seriously considered attempting suicide and 7.4% attempted suicide (CDC, 2017). Accordingly, prevention of youth suicide through the mitigation of risk factors, enhancement of emotional and behavioral regulation skills, early identification of at-risk youth, dissemination and development of evidence-based approaches, and development of new treatment options is a national mental healthcare priority (Gould, Greenberg, Velting, & Shaffer, 2003; Wilcox & Wyman, 2016).

With the passage of the 2004 Garrett Lee Smith Memorial Act (GLSMA) and subsequent reauthorization in 2011, an appropriation was made by Congress for the Substance Abuse and Mental Health Services Administration to fund grants to states, tribal communities, and campuses for suicide prevention (Goldston et al., 2011). More than 96% of State and Tribal grantees conduct gatekeeper training, or programs that trained professionals and community members who regularly interact with youth to identify those at-risk and refer them for mental health support (Burnette, Ramchand, & Ayer, 2015). It does not refer to training of mental health professionals, but instead, is intended for individuals like teachers, administrators, and other school staff who interact with students on a regular basis (Burnette et al., 2015). In 2012, the Surgeon General released the National Strategy for Suicide Prevention report, highlighting the promise of gatekeeper training programs (Department of Health and Human Services [HHS], 2012). This report also underscored the key role that schools can play in preventing adolescent suicide through the dissemination of a continuum of mental health services, ranging from universal prevention programs to targeted treatment. Schools are the major providers of mental health services for youth (Farmer, Burns, Philips, Angold, & Costello, 2003). In fact, of youth who receive mental health services, 70–80% access these services in schools (Rones & Hoagwood, 2000).

While schools have the capacity to reach youth at-risk for suicide, there remains a gap between the number of youth with mental health issues and those who receive services (Kataoka, Zhang, & Wells, 2002; Mills et al., 2006). The gap for students at-risk for suicide may be more stark: suicidal youth are under-identified, and therefore, few access services (Brown, Wyman, Brinales, & Gibbons, 2007). Given their consistent contact with youth and the closeness of their relationships, teachers are uniquely positioned to play a role in suicide prevention by connecting youth to mental health services (Gould & Kramer, 2001). Qualitative studies reveal that teachers desire to play a role in suicide prevention and would like to receive training to identify warning signs and address youth’s suicide risk (Hatton, Heath, Gibb, Coyne, & Hudnall, 2017; Nadeem et al., 2011). Further, twenty-nine states now require school personnel to complete suicide prevention training (American Foundation for Suicide Prevention [AFSP], 2018). In Maryland, for example, HB 0920 mandates the State Board of Education to require all certificated school personnel who have direct contact with students to complete training to understand and respond to youth suicide risk (Maryland Primary and Secondary Education—Certificated School Personnel Training Act, 2016).

Despite a potential need and desire to intervene, teachers may not know the risk factors of suicide and how to engage youth in a conversation about suicide risk (Isaac et al., 2009). To fill this gap, gatekeeper training programs teach teachers prevalence rates of suicide, signs of psychological distress, and strategies to make referrals to school mental health treatment when necessary (Burnette et al., 2015; Gould & Kramer, 2001; Isaac et al., 2009). Reviews of adolescent suicide prevention have found that, for uncontrolled studies, gatekeeper training is related to increase in trainees’ knowledge of suicide risk and self-efficacy in engaging in suicide prevention (Isaac et al., 2009; Robinson et al., 2013; Yonemoto, Kawashima, Endo, & Yamada, in press). While gatekeeper training is a promising practice to increase attitudes and beliefs, the research is inconclusive in determining whether gatekeeper training changes gatekeeper behaviors, or the number of youth identified, approached, and referred (Katz et al., 2013; Isaac et al., 2009; Robinson et al., 2013). Several studies have found increases in the number of individuals approached and referred after gatekeeper training (Albright et al., 2012; Rallis et al., 2018; Wyman et al., 2008). However, other studies have either found no impact on gatekeeper behaviors (Terpstra et al., 2018) or an impact of number of students approached, but not referred (Ewell-Foster et al., 2016). Though not exclusive to schools, an analysis of 10 years of gatekeeper training implemented by GLS grantees in 48 states found reductions in suicide attempts and deaths (Garraza, Kuiper, Goldson, McKeon, Walrath, in press; Walrath, Garraza, Reid, Goldston, & McKeon, 2015). Gatekeeper training implemented in school settings has found that gatekeeper training is related to increases in knowledge, beliefs (e.g., self-efficacy), and behavioral intentions (e.g., likelihood to intervene) (Cross et al., 2011; Katz et al., 2013; Lamis, Underwood, & D’Amore, 2017; Mo, Ko, & Xin, 2018; Pistone, Beckman, Eriksson, Lagerlof, & Sager, in press; Robinson et al., 2013; Tompkins, Witt, & Abraibesh, 2010; Wyman et al., 2008). Studies of school-based suicide prevention have found that identification of students increased for staff communicating about suicide at baseline (e.g., Katz et al., 2013; Mo et al., 2013; Robinson et al., 2013; Wyman et al., 2008). Nevertheless, there is no documented relationship between school-based gatekeeper training and reductions in suicide attempts (Pistone et al., in press).

In addition to setting, gatekeeper training varies by delivery method, duration, and emphasis on behavioral rehearsal. The majority of gatekeeper training programs that have been developed and evaluated are conducted in person (Cross, Matthieu, Lezine, & Knox, 2010; Cross et al., 2011; Ewell-Foster et al., 2016; Wyman et al., 2008). Yet, given the time constraints, limited funding, and competing priorities for schools (Reinke, Stormont, Herman, Puri, & Goel, 2011; Stephan, Brandt, Lever, Acosta-Price, & Connors, 2012), gatekeeper training programs that are flexible and available online may be more feasible for schools to implement. Preliminary research indicates that online gatekeeper training is related to increases in knowledge, attitudes, and beliefs (e.g., self-efficacy) (Albright, Goldman, Shockley, McDevitt, & Akabas, 2012; Albright, Adam, Goldman, & Serri, 2013; Bartgis & Albright, 2016; Lamis, et al., 2017; Smith, Silva, Covington, & Joiner, 2014). There is preliminary support the relationship between duration of gatekeeper training and its impact: a higher proportion of K-12 school personnel who participated in in-depth trainings, compared to brief trainings, identified youth at-risk for suicide (Condron et al., 2019). Finally, Cross et al. (2011) examined the impact of gatekeeper training compared to gatekeeper training and behavioral rehearsal, finding that both groups had similar increases in knowledge and self-efficacy and neither group had significant impacts on referrals.

Prior research on gatekeeper training has also aimed to examine mechanisms of change. One line of research has examined the relationship between “natural gatekeepers,” or non-mental health professionals who report communicating with youth about psychological distress before receiving gatekeeper training. Wyman et al. (2008) found that, among teachers, an increase in questioning students about suicide only occurred for these “natural gatekeepers,” while Ewell-Foster et al. (2016) found that “natural helper” status was unrelated to change in identification behaviors, and negatively related to change in response and referral behaviors. Health behavior change theories posit that there are relationships among attitudes, beliefs (e.g., self-efficacy), behavioral intentions, and behavioral change. Specifically, the theory of planned behavior posits that attitudes, perceived behavioral control (e.g., personal beliefs about how difficult the behavior will be), and subjective norms influence a behavioral intention, which, in turn, influences behavior (Godin & Kok, 1996; Madden, Ellen, & Ajzen, 1992). One component of perceived behavioral control is self-efficacy, or a person’s beliefs about their capability to perform a behavior. According to both the theory of planned behavior, theories of health behavior change (Hayden, 2009), and Bandura (1977)’s self-efficacy model, self-efficacy indirectly and directly influences behavior (Madden et al., 1992). Further, one study of gatekeeper training found that self-efficacy mediated changes in behavior (Osteen, Frey, Woods, Ko, & Shipe, 2017), which suggests that teachers’ change in self-efficacy after the gatekeeper training program would be related to changes in behavior.

The Current Study

The focus of this study is the evaluation of the online Kognito gatekeeper training program for K-12 teachers implemented in Maryland schools from 2014–2017. Despite wide-scale national adoption and being listed in evidence-based practice repositories, the only evaluations of Kognito have been published by the program’s developer (e.g., Albright et al., 2012, 2013; Bartgis & Albright, 2016), pointing to the need for an independent evaluation. There were two main questions of inquiry: (1) What are the outcomes (e.g., belief, behavioral intention and behavior change) of the Kognito gatekeeper training? (2) What are the predictors of change of beliefs and behaviors? In line with the theory of planned behavior (Madden et al., 1992), to capture perceived behavioral control, we examine self-efficacy and how prepared participants believe they are to engage in gatekeeper intervention behaviors (e.g., detecting signs of psychological distress) (Albright et al., 2016). To capture behavioral intention, we assess an individual’s self-reported likelihood of engaging in gatekeeper intervention behaviors (e.g., intervening with students at-risk). For simplicity, throughout the paper, we refer to “perceived behavioral control” as “beliefs,” which are comprised of self-efficacy and preparedness, and we refer to “behavioral intention” as likelihood.

Given prior research on gatekeeper training, we hypothesized that beliefs (i.e., preparedness and self-efficacy) and behavioral intentions (i.e., likelihood) to intervene with youth in psychological distress would substantially (medium or large effect size) increase from baseline to follow-up, and there would be a more modest (small effect size) increase in gatekeeper intervention behaviors (e.g., approaching students and referring them to school support services) from baseline to follow-up. Given Wyman et al. (2008)’s findings, we hypothesized natural gatekeepers would have greater increase in gatekeeper intervention behaviors (proportion of students approached and referred) from baseline to follow-up compared to those who are not natural gatekeepers. Further, we hypothesized that a change in self-efficacy (from baseline to post) would predict a change in gatekeeper behaviors over time (from baseline to follow-up).

Methods

Procedure

As part of our GLS youth suicide prevention project, our team contacted school administrators, student services directors, and mental health professionals to throughout the state of Maryland to advertise the Kognito training modules. Some schools required teachers to take the training, whereas in others, teachers self-selected to take the Kognito training. Teachers completed a survey before they took the training (baseline), immediately post-training (post), and 3 months following training (follow-up).

Gatekeeper Training

Kognito’s At-Risk for Educators Suite (Albright, 2009) is an online gatekeeper training program. There are three modules targeted for K-12 students: At-Risk for Elementary School Educators, At-Risk for Middle School Educators, and At-Risk for High School Educators. Participants take the training that corresponds to their current level of teaching. These modules are approximately one-hour-long simulations, which can be completed in one sitting or at a pace that suits the participant. A mental health clinician, “Jackie,” represented as an avatar, introduces signs of emotional distress (e.g., worrisome behaviors and signs of appearance, and troublesome academic performance), followed by demonstrating motivational interviewing techniques to approach youth who may be at risk and refer them to mental health support if necessary. Participants then practice identifying signs of psychological distress and motivational techniques through interactive exercises with three virtual students (i.e., avatars). Specifically, “Jackie” introduces a student’s profile, which includes information about their academics, behaviors, and moods. Then, the trainee practices approaching and referring students through a virtual role play with students. Trainees select from a menu of options (e.g., reflections, questions, and recommendations), and the mental health clinician (“Jackie”) provides feedback throughout the simulation. At the end of the module, the trainees are provided with a list of state and county-specific mental health resources.

Participants

The primary sample consisted of teachers that completed the baseline, post, and follow-up survey (see Fig. 1), and the secondary sample consisted of teachers who completed baseline and post. Teachers were from public schools that were both co-educational and single sex. Participants who were not teachers (e.g., administrators, nurses, and mental health clinicians) or who did not complete a baseline, post, and/or follow-up survey were excluded from the sample. There were 781 participants in the primary sample: 48.0% (n = 375) were elementary school teachers, 25.2% (n = 197) were middle school teachers, and 26.8% (n = 209) were high school teachers. When comparing the primary sample to the secondary sample, there were no significant differences in racial demographics or school-level teacher/student ratio. However, the primary sample had a higher percentage of females (83.9% compared to 78.7% in the secondary sample, X2(2) = 10.89, p = .004); higher mean age (M = 44.01 compared to M = 40.14 in secondary sample, t(965.7) = 7.93, p < .001); and higher percentage of students receiving free and reduced lunch (31.75% compared to 29.78% in the secondary sample, t(4048) = 2.22, p = .026).

Fig. 1
figure 1

The inclusion and exclusion of teachers in the primary sample

Full size image

Demographics

Participants were asked to indicate their gender, race, ethnicity, and age (see Table 1). The majority of participants in the primary sample were female (n = 647, 83.9%) and White/non-Hispanic (n = 617, 80.8%). Seventy (9.2%) teachers identified as Black/African-American, twenty-one (2.8%) teachers identified as White/Hispanic, and fifty-five teachers (7.2%) identified as Asian, Multiracial, American Indian, Native Hawaiian, or other. The average age of participants was 44.01 (SD = 11.77), and the average years of teaching experience was 14.87 years (SD = 9.22).

Table 1 Demographics of primary sample
Full size table

School-Level and County-Level Characteristics

Teacher/student ratio and percentage of students on free and reduced lunch (FRL) were entered as continuous variables. Teachers who participated in the Kognito gatekeeper training were employed in schools spanning seventeen of Maryland’s twenty-four counties. The majority of teachers in the sample came from Howard County (n = 459, 64.7%), followed by Baltimore County (n = 138, 19.5%), Queen Anne’s County (n = 23, 3.2%), and Montgomery County (n = 23, 3.2%). The number of suicide deaths per county (from the Maryland Violent Death Reporting System) and percentage of youth per county who seriously considered suicide (from the 2015 YRBS survey) were also entered.

Measures

Teacher Status

Participants selected one of six job roles: teacher or teacher’s aide; mental health counselor; school administrator; administrative assistant; social worker; or other. Participants who marked “other” and indicated that they were in a role that involved educating students (e.g., teacher’s assistants or special educators) were recoded to be teachers. Participants who selected mental health counselor, school administrator, administrative assistant, social worker, or other (and did not educate students) were excluded from the primary and secondary samples.

Natural Gatekeeper Status

Prior studies (e.g., Wyman et al., 2008) defined “natural gatekeepers” as participants that communicated with students about psychological distress before taking the gatekeeper training (i.e., baseline). In Wyman et al. (2008)’s study, the authors use three questions to create a “Natural Gatekeeper scale” to assess participant’s communication with students. Because we did not use the natural gatekeeper scale in our study, we used a question about the number of students approached at baseline to define natural gatekeepers. Accordingly, natural gatekeeper status was dichotomized, such that participants who approached students about distress at baseline were considered “natural gatekeepers.”

Gatekeeper Behavior Scale

The Gatekeeper Behavior Scale (GBS) is an empirically validated measure that teachers take before and after the completion of the Kognito module (Albright, Davidson, Goldman, Shockley, & Timmons-Mitchell, 2016). The GBS was empirically validated through assessing its reliability, construct validity (e.g., confirmatory factor analysis), criterion validity (e.g., correlations with behaviors), and convergent validity (e.g., correlations with general self-efficacy). There are three subscales: preparedness to aid those in psychological distress, likelihood to help those in psychological distress, and self-efficacy to help those in psychological distress. The composite GBS is the average of the three subscales. Middle school and high school teachers answered the questions as originally worded from the Gatekeeper Behavior Scale. However, elementary school teachers were asked questions with wording focused on parental involvement for two of the preparedness questions (e.g., “How would you rate your preparedness to motivate a parent whose child is exhibiting signs of psychological distress to seek help?”), one of the likelihood questions (e.g., “How likely are you to try helping parents be informed about mental health support services?”), and two self-efficacy questions (e.g., “I feel confident in my ability to discuss with a parent my concern about the signs of psychological distress their child is exhibiting”). Further, elementary school educators were asked one additional question on the self-efficacy scale (e.g., “I feel confident in my ability to apply communication strategies such as reflective statements and open-ended questions in discussions with parents”).

Gatekeeper Intervention Behaviors

There were three questions to determine gatekeeper intervention behaviors: the number of students that the gatekeepers had (a) been concerned about because of psychological distress (“students identified”), (b) discussed their concerns with (“students approached”), and (c) referred to appropriate services in the past two months (“students referred”). Given the number of students identified at baseline and follow-up likely influences the number of students approached and referred over time, the authors compared the proportions of students approached and referred over time. That is, the authors compared the proportion of: (a) students approached out of total identified, and (b) students referred out of total identified over time. These intervention behaviors were assessed at the baseline-survey and follow-up survey. Prior research has not been conducted on the validity of gatekeeper intervention behaviors, but the items have acceptable face validity.

Analyses

Data were analyzed using SPSS version 25.0. Covariates included race, gender, age, school-level free and reduced lunch ratio (FRL), school-level teacher/student ratio, district-level percentage of students with suicidal ideation, and district-level suicide deaths. These covariates were correlated with criterion variables (beliefs, behavioral intentions, and gatekeeper behaviors) to determine significant correlations. Only significant covariates were included in the analyses.

Results

Missing Data and Reliability

On the Gatekeeper Behavior Scale, less than 5% of the participants had one or more missing items. For those participants who were missing only one item on each scale, their responses were averaged. Similarly, less than 5% of the sample had missing items for the gatekeeper behaviors (e.g., number of students identified, approached, and referred). Participants who were missing items for gatekeeper behaviors were excluded from the relevant analyses. The overall GBS scale, preparedness subscale, and self-efficacy subscale had acceptable within scale reliability (α = .82 to .93); however, the likelihood subscale had low levels of within scale reliability (α = .51–.88). Given that the intraclass correlations (ICC) were sufficiently close to zero (i.e., ICC = .002–.103), multilevel modeling was not necessary (Hayes, 2006).

Belief and Behavioral Intention Change

Preparedness, Self-efficacy, and Likelihood

One-way repeated measures ANCOVAs were performed to evaluate whether teachers’ preparedness, likelihood, and self-efficacy to intervene with students expressing psychological distress increased over time (e.g., baseline, post, and follow-up). After controlling for teachers’ race, gender, age, school-level teacher/student ratio, and district-level suicide deaths, the overall F difference in mean preparedness was statistically significant: F(1.97, 1280.70) = 16.01, p < .001. Consistent with our hypothesis, Bonferroni pairwise comparisons revealed statistically significant increases in preparedness between baseline and post, decreases between post and follow-up, and increases between baseline and follow-up (see Table 2). The overall F difference in mean self-efficacy was not statistically significant: F(1.97, 1283.13) = 2.02, p = .133) after controlling for individual’s race, school-level teacher/student ratio, and district suicide deaths. Consistent with our hypothesis, Bonferroni pairwise comparisons revealed there were statistically significant increases in self-efficacy between baseline and post and between baseline and follow-up, but there were no differences between post and follow-up (see Table 2).

Table 2 Belief and behavioral intention change from baseline to follow-up
Full size table

After controlling for individual and school/district-level variables, the overall F difference in average likelihood to intervene was not statistically significant: F(2, 1320) = 0.76, p = .466); however, Bonferroni pairwise comparisons revealed there were statistically significant increases in likelihood between baseline and post, decreases between post and follow-up, and increases between baseline and follow-up (see Table 2). Teachers’ likelihood that they would intervene with a student expressing psychological distress increased over time; however, the results revealed that the effect size was small.

After controlling for individual’s race, gender, age, school-level teacher/student ratio, and district suicide deaths, the overall F difference in mean GBS was statistically significant: F(1.96, 1279.43) = 4.83, p = .008). Consistent with our hypothesis, Bonferroni post hoc comparisons revealed there were statistically significant increases in beliefs and behavioral intentions between baseline and post, decreases between post and follow-up, and increases between baseline and follow-up (see Table 2).

Behavior Change

Behavior Change

To assess for behavior change, the researchers used repeated measures ANCOVAs, controlling for significant covariates. After controlling for age, the repeated measures ANCOVA revealed that the proportion of students approached out of those identified over time was not significant (F(1, 343) = 0.745, p = .389). After controlling for gender and teacher/student ratio, the repeated measures ANCOVA revealed that the proportion of students referred out of those identified over time was not significant (F(1, 349) = 2.694, p = .102) (see Table 3). Subanalyses revealed there were significant differences (F(3265) =  9.99, p = .002) in proportion of referrals by school level (e.g., elementary, middle, and high school); however, there were no significant differences in proportion of students approached by school level (F(2, 260) = 1.04, p = .825). Specifically, middle school and high school teachers referred a greater proportion of students compared to elementary school teachers (Mdif = .30, p < .001; Mdif = .27, p < .01 respectively). Overall, these findings indicate the proportion of students approached and referred stayed roughly the same from before and after the Kognito training.

Table 3 Gatekeeper behavior change from baseline to follow-up
Full size table

Predictors of Behaviors Change and Behaviors at Follow-Up

Natural Gatekeeper Status and Behavior Change

Researchers created difference scores between proportion of students approached and referred out of those identified follow-up and baseline. In contrast to our hypothesis, natural gatekeeper status did not predict proportion of students approached (t = 1.53, p = .127) after controlling for significant covariates. Natural gatekeeper status predicted changes in referrals over time, however, in the opposite direction that we predicted (t = 9.44, p < .001) over time (see Table 4).

Table 4 Natural gatekeeper status and behavior change from baseline to follow-up
Full size table

Self-efficacy and Behavior Change

To evaluate whether change in self-efficacy (from baseline to post) predicted change in behaviors over time (from baseline to follow-up), multivariate regressions were run. Consistent with our hypothesis, a multivariate regression revealed that change in self-efficacy change in proportion of students approached (t = 2.47, p = .014). However, change in self-efficacy did not predict change in proportion referred (t = 2.83, p = .728) from baseline to follow-up when controlling for gender, FRL, teacher/student ratio, district-level suicide deaths (see Table 5).

Table 5 Self-efficacy change and behavior change from baseline to follow-up
Full size table

Discussion

Consistent with prior research on gatekeeper training tailored to teachers (e.g., Lamis et al., 2017; Mo et al., 2018; Pistone et al., in press; Robinson et al., 2013; Wyman et al., 2008), our findings suggest that teachers’ preparedness to detect signs of psychological distress, likelihood they would intervene with students at-risk, and self-efficacy in detecting and referring students at-risk increased from baseline to follow-up. Similar to in-person trainings, Kognito, an online gatekeeper training, may equip teachers with the knowledge (e.g., signs of psychological distress) and skills (e.g., strategies to detect and refer at-risk youth) necessary to increase beliefs and behavioral intentions about suicide prevention. However, beliefs and behavioral intentions could have increased due to flaws in the study design (e.g., attrition and within-subjects). Consistent with prior research, teachers’ beliefs and behavioral intentions also decreased from post to follow-up, indicating there may need to be additional supports to maintain beliefs through in-person role plays or booster sessions (Cross et al., 2011; Wyman et al., 2008).

Nevertheless, the majority of teachers did not increase the proportion of students approached or referred over time. Prior research has found limited impact of gatekeeper training on behavior change, with Ewell-Foster et al. (2016) finding a small but significant increase of gatekeeper training on behaviors and Wyman et al. (2008) finding an increase in number of students approached only for teachers communicating with students at baseline. In our study, there are several possible explanations for the lack of behavior change, one being that a one-time, one-hour gatekeeper training is not sufficient to connect students to mental health support (Kalafat, 2003), and strategies to supplement gatekeeper training may be necessary to increase the number of referrals. Research on gatekeeper training suggests that, among teachers, in-depth gatekeeper training, compared to brief trainings, is related to a higher proportion of identifications (Condron et al., 2019). Another explanation for the lack of behavior change is that increased knowledge about signs of distress results not in greater number of referrals, but rather, in more accurate targeting of which students need mental health support. In other words, the overall number of students identified and referred did not change, but those students who were detected may have more likely been expressing signs of distress. Finally, teachers may not have had enough time to implement the skills to identify and refer at-risk students, given the follow-up survey occurred only three months after completion of the Kognito training.

More generally, individual and school-wide factors influence the implementation of interventions, and accordingly, the change of behaviors. Implementation science and prior research suggests that one-time workshops may increase knowledge of trainees, but are insufficient to change practices and behaviors (Herschell, Kolko, Baumann, & Davis, 2010; Forman et al., 2013). Barriers specific to the implementation of interventions for teachers and schools include limited funding, competing initiatives and priorities, time constraints, and administrator support for the training (Reinke et al., 2011; Stephan et al., 2012).

Elucidating the factors that predict detection and referrals of at-risk youth can help providers and policymakers develop effective strategies and practices for suicide prevention. In contrast to our hypothesis, our study revealed teachers who were not approaching students about psychological distress at baseline (e.g., those who were not “natural gatekeepers”) were more likely to increase the number of students they referred after Kognito. Teachers’ increase in self-efficacy resulted in an increase in proportion of youth approached (e.g., Bandura, 1977), but not referred, which may suggest that self-efficacy change may effect changes in gatekeeper behavior.

One strength of this study is that, to our knowledge, it is the first independent evaluation of Kognito, and first evaluation of Kognito’s K-12 trainings for teachers. Prior studies that evaluated Kognito (e.g., Albright et al., 2012, 2013, Bartgis & Albright, 2016) have been published by Kognito’s developer. Another strength of this study was the range of schools and counties represented in the sample. Though one jurisdiction constituted the majority of the sample (64.7%), there was representation from 16 other jurisdictions in Maryland. Moreover, there were 131 schools represented in the sample. We also included potentially confounding school (FRL ratio and teacher/student ratio) and district-level (suicide deaths and percentage of students with suicidal ideation) variables in the analyses. Given the diversity of schools represented and the inclusion of school-level and district-level covariates in the study, the findings could likely be generalized to other school settings.

Despite this study’s strengths, it is important to acknowledge the limitations. The most glaring limitation is that this study relied on a within-group repeated measures design, as opposed to a randomized control trial. A rigorous randomized trial of Kognito is critical to assert causal inference for its impact on beliefs, behavioral intentions, and behaviors (Brown et al., 2007). On a related note, there was a high rate of attrition, likely due to the lack of incentives. Additional limitations include its reliance on teacher self-report data, the use of a tool developed by Kognito’s developers, and the lack of attention to the relationship between prior suicide prevention training and outcomes. This study focused on teachers’ perceptions of their ability to detect those at-risk of suicide, rather than their mental health literacy and students’ ideation, attempts, or deaths. Further, our study followed up with teachers three months after the completion of Kognito, which may not be sufficient to detect and evaluate long-term effects of the training.

Finally, there may be limitations in the training approach itself: gatekeeper training hinges on the accurate detection of those at-risk, and detection must be followed by effective treatment to successfully reduce suicide deaths and attempts (Brent & Brown, 2015; Kalafat, 2001). Furthermore, this gatekeeper training is only a one-time support, and prior research has documented the need for ongoing professional development and support of teacher learning to effect behavior change (Herschell et al., 2010; Kalafat, 2003).

Future research could advance this research study by addressing its limitations. For instance, a randomized control trial (Katz et al., 2013; Wilcox & Wyman, 2016), greater retention of teachers in follow-up surveys, and a more robust measure of the number of students identified and referred would enhance our understanding of gatekeeper training. Acquiring psychometrically sound behavioral data is challenging in the context of school-based research, given the reporting of student referral and mental health data varies by each school and school district. Nevertheless, reliable and valid measurement could be improved by asking teachers to fill out weekly logs that document the number of students identified, approached, and referred. Studies could explore whether different individual characteristics, such as personal experience with suicide, or school-level factors, such as school mental health resource availability, are related to appraisal or behavior change.

Implications and Conclusion

Our study found that Kognito gatekeeper training is related to an increase in beliefs and behavioral intentions, but not, however, related to changes in proportion of students approached and referred. Nevertheless, Kognito was online and one hour long, which points to the need to further examine the impact of gatekeeper training program by duration (e.g., Condron et al., 2019) and delivery method. Taken together, our findings suggest that gatekeeper training may be a necessary, but not sufficient, component of suicide prevention. As the majority of youth at-risk for suicide do not seek professional support independently (Kalafat, 2003), teachers can play a critical role in detecting subtle changes in appearance, mood, or behavior that indicate that a child is at-risk for suicide. Schools could pair gatekeeper training with universal interventions that teach youth emotion regulation skills, targeted interventions that provide mental health support to groups of students who are expressing signs of moderate psychological distress, and intensive therapeutic support for students expressing more severe symptoms of psychological distress (Slade, 2002; Wyman, 2014; Wilcox et al., 2008). Further, school connectedness is related to reduced suicidal behavior (Marraccini & Brier, 2017), suggesting building a positive school climate through teacher/student and peer relationships and providing more funding for mental health providers are also important components to suicide prevention.

These findings have implications for school practices and local, state, and federal policies, as many states require suicide prevention training for teachers (AFSP, 2018). Yet, our findings suggest that the Kognito gatekeeper training alone for this population group is not related to change of behaviors, underscoring that state and local policies should likely pair gatekeeper training with other approaches, such as socioemotional learning interventions, programs to increase school connectedness, and restrictions to means of suicide (Katz et al., 2013; King, 2001; Mann et al., 2005). School practices ideally should increase the opportunities in the school day for teachers to have meetings to discuss students who may need mental health support. Moreover, gatekeeper training needs to be paired with increased support for clinicians in school settings: referral of youth to mental health support is futile without funding for mental health providers (Brown & Brent, 2015). In conclusion, this gatekeeper training alone appears insufficient to change teachers’ intervention behaviors, and accordingly, those implementing systems of suicide prevention should employ a comprehensive approach to link youth at-risk for suicide with mental health treatment.