Introduction

Suicide in the United States is a significant public health issue with a rate of 13.26 suicides per 100,000 (age adjusted) in 2015 (CDC 2017). Suicide outnumbers homicides by more than two to one, and in 2015, there were more deaths by suicide than motor vehicle accidents. In Missouri, suicide represents an even greater issue, as the state has the 17th highest rate in the country with 16.98 per 100,000. Additionally, rates have been increasing over time; in 2000, the overall suicide rate in Missouri was 12.49 (CDC 2017). Rates are almost four times higher among men than women (27.17 vs. 7.49 per 100,000) and are highest among older (75+) white males (53.48 per 100,000) followed by middle-aged (45–54) white males (41.28 per 100,000). While rates among those under 25 are generally lower, among those 15–19 and 25–34, it is the 2nd leading cause of death and the third leading cause of death among 10–14 and 20–24 year olds (CDC 2017). In Missouri, 15.6% of all high school students reported seriously considering suicide within the past 12 months and 7.3% attempted suicide during this same time interval, equating to approximately 1.5 million suicide attempts per year based on 2015 U.S. Census population estimates (Missouri Student Survey 2016; CDC 2017). Moreover, the impact of suicide extends beyond the individuals themselves. Previous research has shown that suicide can also dramatically impact survivors including family, friends, schools, and even entire communities (Cerel et al. 2008; Bozigar et al. 1993).

The recognition of suicide as a public health issue has resulted in the development of evidence-based prevention programs to decrease suicide-related mortality. Specifically, suicide prevention gatekeeper training programs have emerged in the wake of the recognized urgent need to address suicide. Gatekeepers have been defined as “those people who regularly come into contact with individuals or families in distress” (U.S. Department of Health and Human Services, Public Health Service 2001). Gatekeepers can include anyone who could potentially assist someone who is suicidal, including family members, friends, teachers, members of the clergy, law enforcement, co-workers, correctional personnel, or primary and mental health care professionals (U.S. Department of Health and Human Services, Public Health Service 2001). “Question, Persuade, Refer” or “QPR” is an hour-long gatekeeper training program that has been disseminated widely and has been cited by the QPR Institute as being “the most widely taught gatekeeper training in the world” with more than 1,000,000 adults trained (QPR Institute 2017). Specifically, the QPR program attempts to increase knowledge and dispel myths about suicide and suicidal behaviors, including warning signs and available resources. The program also aims to strengthen the ability of gatekeepers to ask individuals about their suicidal thoughts and/or intentions, persuade them to obtain help, and accompany them to the appropriate service provider (QPR Institute 2014). QPR is listed in the Substance Abuse and Mental Health Services Administration (SAMHSA) National Registry of Evidence-based Programs and Practices (NREPP) and the Suicide Prevention Resource Center’s (SPRC) Programs with Evidence of Effectiveness (SAMHSA 2017; SPRC 2017).

Considerable research has been conducted to test the effectiveness of QPR. Several studies have focused on its immediate effects using pretest/posttest-only designs and found positive changes in knowledge, self-efficacy, or help-giving behaviors among program participants (Cross et al. 2007, 2010; Matthieu et al. 2008). These studies focused on specific subgroups, including staff in universities, hospitals and the Veteran’s Administration. Follow-up studies have also demonstrated positive effects. Using a randomized-trial design, Wyman et al. (2008) found increases in preparedness and efficacy of gatekeepers with a 1-year average follow-up among secondary school staff. Keller et al. (2009) found immediate positive improvements in knowledge and self-efficacy among youth-serving adults. At 6 months, these effects had decreased but still remained higher than baseline scores. Matthieu et al. (2009) conducted a 1-year follow-up of Veterans Administration (VA) hospital employees who had completed the training and found that self-efficacy and knowledge both increased from pretest to posttest, with sustained self-efficacy effects at 1-year follow-up but decreases in knowledge to almost baseline levels. Mitchell et al. (2013) conducted a 3–6 month follow-up study with college students and staff who had completed QPR and found sustained effects on eight items that assessed knowledge, attitudes, and skills. Godoy Garraza et al. (2015) evaluated the effectiveness of the Garrett Lee Smith programs nationwide and found reductions in suicide attempts, but this study aggregated findings across all suicide prevention programs offered through the GLS grants and therefore included a wide array of suicide prevention programs beyond QPR only.

While these studies serve to demonstrate QPR’s short-term effectiveness on attributes such as knowledge and self-efficacy, to date there are no published articles on its ability to influence trainees beyond 1 year. Furthermore, most published studies targeted specific subpopulations (e.g., college students and staff, secondary school staff, VA employees and hospital staff). This paper expands the growing body of knowledge regarding the program’s effectiveness by presenting findings from a 2-year follow-up study of adults in Missouri who serve youth. Given QPR’s goals as outlined above, we anticipate our analysis to show sustained increases in gatekeeper knowledge about suicide prevention, including attitudes, behaviors and warning signs, improved gatekeeper self-efficacy and increases in the number of youth who are helped by the gatekeeper. We also hypothesize that improvements will occur regardless of the gender, race or age of the training participant. We will also explore the effect of prior suicide prevention training on gatekeeper outcomes.

Methods

A total of 3692 adults received gatekeeper suicide prevention training in Missouri between July of 2010 and September of 2011 through funding from the Garrett Lee Smith (GLS) Youth Suicide Prevention Initiative through SAMHSA and Missouri Department of Mental Health (DMH) Mental Health Block Grant Funds. Of those adults, 2988 received training in QPR. Because most of the funding received was through the GLS initiative, the majority of adults who participated in QPR served youth (ages 10–24) in some capacity.

Procedures

The longitudinal research design included measurements immediately prior to and after each training and at 2 years post-training. Pretest and posttest surveys were completed using paper/pencil methods and the follow-up survey was completed using SurveyMonkey, an online survey tool. Pre/post surveys were completed in 2010 and 2011, and all survey data collection was completed by January of 2014.

Posttests included a section requesting consent to contact participants 2 years following the QPR training. Those who agreed to be contacted provided their names and email addresses to the researchers. Follow-up reminders were sent to those individuals who did not immediately respond to the survey 3 months after initial contact. Of the 2988 adults who participated in both the pretest and posttest, 491 received a follow-up survey and 98 completed the survey.

Participants

Pre/Post participants (n = 2988)

The average age of the QPR participants who completed pretests and posttests was 40 and ranged from 17 to 95. Approximately three-fourths (76.1%) were female, 88.3% were white, 7.2% were African American, and the rest identified as another race. Participants included school staff (40.9%), youth service providers (14.9%), students (9.9%), and parents (9.2%) with the remaining individuals representing a wide array of professions, including physical and mental health care professionals, clergy, mental health and substance use professionals, and probation and parole officers. A majority had participated in at least one suicide prevention program before the QPR training (58.3%).

Two-Year Follow-up Participants (n = 98)

The average age of participants who completed follow-up was 42.6 and ranged from 18 to 77, and 82.4% were female. Almost all (95.8%) were white, and 4.2% were African American. Of those reporting their role, 40.0% took this training as school staff, 20.5% as youth service providers, 11.1% as students, and 2.6% as parents. Almost two-thirds (63.8%) of the participants had suicide prevention training prior to QPR.

Instrumentation

The survey instrument was designed to capture key elements of the QPR training, including suicide prevention knowledge, self-efficacy, and help-giving behaviors, three constructs specifically addressed in the QPR training program. Measures were adapted from a gatekeeper training survey developed at the QPR Institute and from a study of QPR effectiveness by Wyman et al. (2008).

Self-efficacy

Because QPR was designed to equip individuals with the skills needed to talk directly to suicidal individuals and get them the help they need, participants were asked three questions regarding their perceptions of effectiveness. Specifically, they were asked how comfortable they felt asking someone if they were suicidal, how prepared they felt connecting them to help, and how confident they felt in knowing where to refer them for help. Answer choices included “not very,” “somewhat,” “very,” and “extremely,” with “not very” coded as 1 and “extremely” coded as 4. Self-efficacy items were collapsed into one scale yielding an alpha coefficient of 0.79.

Knowledge

Four items related to suicide knowledge and warning signs were included to examine QPR’s effectiveness in dispelling myths regarding suicidal behaviors and increasing knowledge regarding warning signs. First, participants were asked to indicate whether they agreed that “Asking directly, ‘Are you thinking about suicide?’ is an important step in preventing suicide”. This question was included because a key component of QPR is educating participants that asking direct questions regarding suicidal thoughts and behaviors is more effective in eliciting accurate information than asking questions indirectly. Second, gatekeepers were asked whether they agreed that “Sometimes when people say, ‘I want to die,’ or make a suicide attempt, they might really want help”. This question was included to address the incorrect perception that when individuals tell others that they want to die, they are merely asking for attention. Third, they were asked whether they believed that “Even if a person makes a plan to kill themselves, it might still be possible to stop them”. Again, this was included to assess whether QPR dispelled the myth that suicide attempts are inevitable once a person has developed a suicide plan. Finally, because QPR intends to educate gatekeepers as to how to identify a potentially suicidal individual, gatekeepers were asked whether they could name three suicide warning signs. Questions were adapted from a QPR-developed survey and modifications were made so that all statements were statements of fact rather than myth. Answer choices were “no,” “maybe,” and “yes,” with “no” coded as 1 and “yes” coded as 3.

Help-Giving Behaviors

Because the QPR program ultimately aims to increase the degree to which suicidal individuals seek and get help, three items explored the extent to which participants had interacted with and helped a suicidal individual both 3 months before the QPR training and 3 months prior to the administration of the 2-year follow-up. These items included (1) the number of suicidal individuals who had approached them with thoughts or feelings of suicide; (2) the number of youth whom they themselves had approached and (3) their actions based upon these conversations (e.g., did they refer them to help, seek advice from a mental health professional, contact the caregiver/youth, help them without additional assistance). We hypothesized that the number of youth who the participant approached would rise and that participants would be more likely to refer individuals to mental health professionals after the training.

The questionnaire also included gender, age, race, their role as it relates to serving youth (teacher, mental health professional, etc.), and whether they had participated in a suicide prevention program in the past.

Data Analysis

Data was analyzed using IBM SPSS Statistics Version 22 for Windows. RM-ANCOVA was selected as an omnibus test of differences across the three time points and was used to assess differences in the pre/post analysis to have a consistent and comparable approach. Missing data were imputed using estimated-maximization missing value analysis (maximum iterations = 25) controlling for race, age, and gender. Covariates included gender, race, previous suicide prevention training, age, and role with respect to youth (parent, school staff, youth service provider, student, and other). Preliminary models controlled for these covariates and those found to be insignificant were removed in a step-wise manner until all covariates in the model were significant at the 0.10 level. The 0.10 level was used because, though covariates may not exhibit significance at the standard 0.05 level, they may still be relevant to the research question and may still have an important effect on the model. Analyses included calculation of significance levels, estimated marginal means (EMMs), effect size (Hill et al. 2008), and least significant difference (LSD) post-hoc tests for the overall model, as well as for subgroups of those covariates included in final models.

Results

Pre- Post-analysis

Self-efficacy

To assess the short-term effectiveness of QPR related to self-efficacy, a RM-ANCOVA was conducted with an initial model that controlled for gender, race, age, role, and previous suicide prevention training. All variables with the exception of gender were found to be significant at the 0.10 level and were retained in the final model. Analysis indicated positive and statistically significant interaction effects for role, previous suicide prevention training, race and age. The EMMs significantly increased for all subgroups (see Table 2), with an overall increase in EMMs, yielding a significant positive main effect and a medium effect size [F(1,2431) = 244.67, p = 0.000; d = 0.63] (see Table 1).

Table 1 Repeated measures analysis for self-efficacy scale and knowledge before and after training (n = 2988)
Full size table
Table 2 Pretest, posttest subgroup analysis
Full size table

Knowledge

Four items related to participant knowledge of suicide prevention facts. First, participants were asked to indicate whether they believed that directly asking someone about potential suicidal thoughts and/or behaviors could lead to a suicide. Specifically, they were asked to indicate whether they believed that “Asking directly, ‘Are you thinking about suicide?’ is an important step in preventing suicide” is true. The final model included role, race, previous suicide prevention training and age. EMMs of all subgroups of these covariates and overall EMMs increased significantly (see Table 2). There was also a significant positive main effect and a medium effect size [F(1,2336) = 47.31, p = 0.000; d = 0.63] (see Table 1).

With respect to the item, “Sometimes when people say, ‘I want to die.’ or make a suicide attempt, they might really want help,” no significant interaction effects were detected, so no covariates were retained in the final model. QPR trainees showed significant overall increases in EMMs, yielding a significant main effect and a small-to-medium effect size [F(1,2842) = 138.52, p = 0.000; d = 0.44] (see Table 1).

For the item, “Even if a person makes a plan to kill themselves, it might still be possible to stop them,” gender, previous training and age were included in the final model. Females exhibited a significant increase in EMMs from pretest to posttest, while males did not, though the p value approached significance (see Table 2). White individuals exhibited significant increases in EMMs, but positive results were not found among African American individuals. Those who did not have previous suicide prevention training exhibited significant increases in EMMs, while those who did have previous suicide training did not, likely due to higher pretest EMMs. QPR trainees overall did not exhibit a significant increase in EMMs for this item likely due to ceiling effects, yielding a non-significant main effect [F(1,2311) = 0.517, p = 0.472; d = 0.00] (see Table 1).

Gender, previous suicide prevention training, race, and role [F(4,2215) = 12.41, p = 0.000; d = 0.30] were included in the final model for the knowledge item, “I can name 3 warning signs of suicide.” EMMs of all subgroups and overall EMMs increased significantly (see Table 2), and a significant main effect and medium effect size were detected [F(1,2215) = 305.81, p = 0.000; d = 0.74] (see Table 1).

Two-Year Follow-up Analysis

Self-efficacy

In the follow-up self-efficacy analysis, previous suicide prevention training was included in the final model. Both those who indicated that they did have previous suicide prevention training and those who did not increased in self-efficacy from pretest to posttest and from pretest to follow-up (see Table 4). Overall, there were sustained increases in self-efficacy, as posttest and follow-up measures were significantly higher than pretest (p = 0.000 and p = 0.000 respectively). A significant overall main effect was observed and a large effect size was detected [F(2,89) = 34.68, p = 0.000; d = 1.77] (see Table 3).

Table 3 Two-year follow-up repeated measures analysis for self-efficacy scale and knowledge (n = 98)
Full size table
Table 4 Two-year follow-up subgroup analysis
Full size table

Knowledge

For the item, “Asking directly, ‘Are you thinking about suicide?’ is an important step in preventing suicide,” role and age were included in the final model. School staff exhibited an increase in EMMs from pretest to posttest, but this was not sustained at follow-up, as 2-year EMMs were not significantly higher than at pretest. There were no significant differences found in EMMs at different time points for the other roles, possibly due to small sample sizes of these subgroups (see Table 4). The same trend was seen overall, as there was an increase from pretest to posttest (p = 0.003), but there was not a significant difference between pretest and follow-up (p = 0.098), though the p value approached significance. There was not a significant overall main effect for this model perhaps due to the small sample size, but a small effect size was detected [F(2,79) = 1.58, p = 0.213; d = 0.40] (see Table 3).

There were no significant interaction effects for the items “Sometimes when people say, “I want to die” or make a suicide attempt, they might really want help” and “Even if a person makes a plan to kill themselves, it might still be possible to stop them,” at the 0.10 level so no covariates were included in these models. There were also no significant differences detected between any time points for the first item, likely due to the already high means at pretest. For the latter item, posttest scores were significantly higher than pretest scores (p = 0.014), but follow-up scores were not (p = 0.154). A significant main effect and medium effect size were found for the item “Even if a person makes a plan to kill themselves, it might still be possible to stop them” [F(2,92) = 3.28, p = 0.042; d = 0.54]. For the item, “Sometimes when people say, “I want to die” or make a suicide attempt, they might really want help,” no significant main effect was found, but a small effect size was detected [F(2,94) = 0.57, p = 0.570; d = 0.22] (see Table 3).

For the item, “I can name 3 warning signs of suicide”, significant interaction effects were observed in the final model for previous suicide prevention training and age. Those who had no previous suicide prevention training showed sustained significant increases in EMMs, as both posttest and follow-up scores were higher than pretest. Participants who had previous suicide prevention training showed a significant increase in EMMs from pretest to posttest, but this increase was not sustained at follow-up, likely due to high means at pretest (see Table 4). Overall, both posttest and follow-up scores were higher than pretest (p = 0.000 and p = 0.000 respectively). The main effect of the model approached significance and a medium effect size was found [F(2,84) = 2.90, p = 0.061, d = 0.53] (see Table 3).

Help-Giving Behavior Questions (2-Year Follow-up)

Prior to QPR training, among those who completed a 2-year follow-up (n = 98), six participants reported having approached a young person about suicide in the 3 months before they completed the QPR training, while 12 participants reported having youth come to them about suicide in the same time period. In the 2 years after the training, 25 reported having initiated contact with youth about suicide, and 25 reported that youth came to them. In other words, before the training, participants were half as likely to approach youth about suicide as youth were to approach them (6/12), and after the training, they were equally likely to approach youth as youth were to approach them (25/25).

Table 5 displays types of help-giving behaviors given by participants 3 months before the training and 3 months immediately prior to the 2-year follow-up. There was a significant increase in the number of adults who were able help the youth by themselves (pre-training: 31.3%; follow-up: 56.3%; p = 0.017). There were also large increases in the percentage of adults who sought advice from a mental health professional (pre-training: 31.3%; follow-up: 56.3%; p = 0.197), referred the youth to a hotline or mental health professional (pre-training: 56.3%; follow-up: 81.3%; p = 0.375), or told the youth’s parent or caregiver (pre-training: 50.0%; follow-up: 60.8%; p = 0.106). However, these increases were not found to be significant due to the small sample size (n = 16).

Table 5 Help-giving behavior responses for gatekeepers who reported having contact with a suicidal individual for both pretest and 2-year follow-up (n = 16)
Full size table

Discussion

As hypothesized, the QPR gatekeeper training program resulted in both immediate and long-term positive effects in suicide prevention knowledge, self-efficacy and help-giving behaviors. On the whole, effects were sustained regardless of gender, age, race or role and participation in a prior suicide prevention program.

Analysis of the knowledge items indicated that the program was effective in dispelling suicide prevention myths and training participants about warning signs. Specifically, participants were less likely to believe that directly talking about suicidal thoughts or behaviors planted seeds in a youth’s mind that would result in a suicide. Additionally, they were still more likely to be able to identify warning signs even given the large gap in time between the training and follow-up survey. The remaining two items had ceiling effects at baseline which resulted in non-significant findings from pretest to 2-year follow-up. Findings for all four knowledge items indicated no significant decrease from posttest to 2-year follow-up, suggesting no significant loss of knowledge between these time points.

The multi-dimensional components of the self-efficacy scale included comfort in talking directly to individuals about suicide intentions, preparedness in finding help for those who indicate they are suicidal, and confidence that they can connect them to help. Follow-up analyses of the self-efficacy scale indicated sustained effects over time. Significant positive findings for this construct are highly encouraging given that key to being an effective gatekeeper is having the ability to motivate a person with suicidal thoughts to seek help and to assist them in obtaining that help.

In addition to exploring participant knowledge and self-efficacy, we also explored whether the training program resulted in behavior change. Analysis of help-giving behaviors at pretest and 2-year follow-up indicate that after completing the QPR training, more adults approached youth to ask whether they had suicidal intentions than prior to the training. This finding suggests that QPR encourages adults to seek out youth who may be at risk for suicide rather than waiting for the youth to come to them. Moreover, there was a significant increase in the number of adults who were able to help the youth by themselves and large, though non-significant, increases in the percentages of adults who sought advice from a mental health professional, referred the youth to a hotline or mental health professional, and told the youth’s parent or caregiver. The lack of significant findings for these variables is likely due to the small number (n = 16) of adults who had communicated with possibly-suicidal youth both in the 3 months before and in the 2 years following QPR training.

Limitations

The results of this study are somewhat constrained due to modifications to existing questionnaire items, elements of the research design, and sample size. While knowledge items were largely drawn from the existing QPR surveys, modifications were made to eliminate negatively worded “myth” statements. Additionally, items had to be removed to shorten the survey so that it could be administered during the relatively short time period in which QPR is offered. For that reason, only items perceived to be more central to the tenets of QPR were included. Furthermore, because knowledge items were reversed, baseline ceiling effects were detected for two items, limiting the ability to detect effects for two rather than four items. Nonetheless, most items in the questionnaire were sensitive to change over time and resulted in positive long-term effects.

Second, the use of a convenience sample limits generalizability, especially with respect to the 2-year follow-up analysis. Compared to the Missouri adult population, the follow-up sample had a higher percentage of females (82.4 vs. 51.0%) and whites (95.8 vs. 83.0%) and a lower percentage of African Americans (4.2 vs. 11.5%) (US Census Bureau 2015). Accordingly, the follow-up results by-and-large reflect changes in knowledge, self-efficacy and help-giving behaviors as they related to white females. However, RM-ANCOVA models included gender and race covariates when significant interaction effects existed to control for possible effects of these demographic characteristics on results.

Third, while findings for many follow-up items are highly suggestive of significant improvements, the small follow-up sample size somewhat reduced the ability to detect statistically significant differences. This is particularly true regarding the help-giving questions related to referrals, because only 16 adults reported having contact with possibly suicidal youth both before and in the 3-months prior to the 2-year follow-up survey. Nonetheless, the significant increase in the number of youth approached by the trainees at follow-up is very promising, suggesting that the ultimate aim of QPR, to get people help, was achieved. A randomized control group design with a larger sample size would strengthen the generalizability of the study and help to identify other factors that may have influenced the outcomes suggested in the analyses presented in this article. Wyman et al. (2008) used a similar research design which strengthened their assertions that QPR positively changed participant knowledge and behaviors. However, that study was limited to assessing change over a 1-year period only. Future studies of long-term QPR effectiveness would benefit from a larger and representative sample of the target population to build on the work of Wyman et al. and the research presented here.

Implications

The 2-year follow-up study design is the longest QPR follow-up study to date. The changes in self-efficacy and knowledge add to an already large evidence base supporting the effectiveness of the training. Specifically, this study suggests evidence of effectiveness much longer than had previously been researched. Furthermore, effects were detected regardless of age, race, gender, and role as well as with individuals who had already received suicide prevention training, implying that QPR can be efficacious in a variety of settings. These findings suggest that this relatively short suicide prevention gatekeeper training can have an impact far into the future, making it a viable option for organizations with limited time and resources.

Results also suggest that QPR continue to be recommended in state suicide prevention plans as an effective gatekeeper program with long-term effects. Many plans currently endorse QPR on their menu of suggested gatekeeper programs (e.g., Alaska, North Carolina, Missouri, Montana, Tennessee, Washington). However, many state plans are either under development or in the process of being updated, including that of Missouri. This research suggests that QPR be included in any future plans or plan revisions. As youth-serving agencies with limited time and resources search for programs with long-term impacts, this relatively short program stands out. If more youth-serving agencies can offer suicide prevention programming to more individuals, the number of trained gatekeepers will increase. Consequently, expansion of this pool increases the number of people who can be helped and ultimately can reduce suicide rates in Missouri and elsewhere.