“Mental health has been hidden behind a curtain of stigma and discrimination for too long…We need to enhance our investment in mental health substantially and we need to do it now” (World Health Organization 2003, p. 3).

Despite this call to action over a decade ago, current estimates indicate that one in five adults in the U.S. have experienced a mental health issue (U.S. Department of Health and Human Services [HHS] 2017), with similar estimates in other developed countries including Australia (ABS 2009), Canada (Canadian Mental Health Association 2017), and the United Kingdom (Mental Health Foundation 2017). Depression and anxiety are among the most common of these issues and may be exacerbated by workplace characteristics and experiences (HHS 1999). At the same time, many workers experience bouts of psychological distress that do not reach clinical levels, but pose challenges to their mental health and ability to be productive members of the organization (LaMontagne et al. 2014). Overall, estimates suggest that challenges associated with mental health can carry costs of up to $225 million for U.S. businesses due to lost workdays, and as much as $36 billion in lost productivity (Kessler et al. 2009).

Given these trends, workplace interventions aimed at improving worker well-being are increasingly common; however, recent reviews suggest that most indirectly target mental health through individual stress reduction or the provision of better coping techniques, while relatively fewer address mental health or illness specifically (Czabała et al. 2011). This is troubling given stigma associated with mental illness that may make recognizing mental health issues, seeking treatment, and appropriately intervening at work unique to other contexts (Krupa et al. 2009). For instance, fear of appearing incompetent or jeopardizing one’s position in a weak economic climate, the supportiveness (or lack thereof) of the workplace culture, and one’s relationship with their co-workers and direct supervisor present unique aspects of the workplace that could impact how workers cope with mental health challenges. Thus, workplace development opportunities that specifically address mental health are a potential opportunity to reduce widespread stigma and positively affect worker well-being.

One potential leverage point for organizations to improve the management of mental health is the development of supervisors (Skakon et al. 2010). Emerging research suggests that leaders play an important role in facilitating and fostering employee well-being (Kelloway et al. 2012; Van Dierendonck et al. 2004). For instance, transformational leadership style has been found to relate to employee physical and psychological health through enhancing employee perceptions of meaningfulness (Arnold et al. 2007), self-efficacy (Nielsen et al. 2008), and trust (Kelloway et al. 2012); however, despite a growing body of literature linking leadership to employee well-being (Kuoppala et al. 2008), and calls for more leadership development in occupational health psychology (Kelloway and Barling 2010), there remains relatively few reports of either well-being- or mental health-related leadership training (for an exception see Dimoff et al. 2015).

Given the potential negative impact of undiagnosed and untreated mental health conditions on the effectiveness and well-being of employees, in combination with the relative lack of studies targeting the development of direct supervisors in this context, this study addresses an important gap in the current literature. First, our training program focuses on combating supervisors’ negative attitudes (stigma) about mental illness, as well as providing supervisors with the knowledge and skills to become aware of and manage their personal mental health. We take into account emerging research suggesting that supervisors themselves need to be skilled at managing their own well-being as a first step toward supporting their employees’ well-being (Byrne et al. 2014). Roche et al. (2014) point out that research attention focused directly on leaders and their personal experience of mental health in the workplace is scarce. These authors reasoned that supervisors, perhaps more so than non-supervisory employees, are often faced with challenging circumstances at work that make them especially vulnerable to increased risk of reduced well-being, including mental health issues such as depression and anxiety. When supervisors experience these states of ill-being, the resultant impact is not only personal, but may negatively impact employees (Sy et al. 2005).

In addition to addressing supervisors’ personal awareness and strategies for coping with mental health concerns, the current training targets knowledge and skills related to recognizing and assisting employees to manage mental health challenges at work. Kitchener and Jorm (2002, 2004) argued for equipping the public with the capability to assist others in responding to mental health emergencies as a means of reducing negative attitudes, delays in recognition, and increasing help-seeking (Mental Health First Aid training; (https://www.mentalhealthfirstaid.org/cs/). In line with this rationale, we assert that supervisors are an important part of a workers’ social context and therefore well positioned to observe, impact, and intervene with respect to the mental health of employees. However, our approach is distinct in that our focus is not necessarily on crises or emergencies related to mental illness, but the effective support of workers’ mental health in the long-term. This approach also differs from other supervisor-focused organizational interventions that have primarily focused on helping supervisors to build general leadership competencies that might indirectly impact employee well-being (Kelloway and Barling 2010). It has been argued that specific leadership trainings may be needed when the outcomes to be impacted span multiple life domains or are highly specific (cf. Hammer et al. 2011); and, indeed initial efforts to train supervisors specifically as it relates to mental health in the workplace have shown promising results (e.g., Dimoff et al. 2015).

Drawing on these findings, the current training took a two-pronged approach. First, the training targeted the effective management of supervisors’ own personal mental health. Specifically, the focus was on empowering supervisors to take actions to monitor and manage personal distress and use strategies to increase their ability to recognize and face challenges. A second objective of the training was to encourage supervisors to engage in behaviors that build positive leader-follower relations and create an environment conducive to the open exchange of information about mental health challenges, accessibility of resources, and mutual understanding. For example, such behaviors include checking in with employees with respect to levels of anxiety or distress, supporting employees who appear upset, supporting employees who resist help from others, and referring employees to company well-being resources (e.g., EAP). A concerted focus on these areas serves to enhance both the direct and indirect influence supervisors can have on employees’ overall mental health in the workplace.

Training Design

The training program was developed in consultation with the executive leaders of the organization to ensure alignment with the organization’s strategic plan for their employees’ health and well-being. Further, a Senior Consultant interviewed a sample of frontline employees and their direct supervisors to understand the unique challenges in the company work environment and explore challenges to mental health and well-being. These interviews in combination with a review of preventative mental health interventions informed the selection of key concepts and skills targeted in the training.

Consistent with the position of the Centers for Disease Control (CDC), we treated mental health and mental illness as related but distinct constructs addressed by the training. The CDC defines mental health as “a state of well-being in which the individual realizes his or her own abilities, can cope with the normal stresses of life, can work productively and fruitfully, and is able to make a contribution to his or her community” (2017). Mental illness is defined as, “collectively all diagnosable mental disorders or health conditions that are characterized by alterations in thinking, mood, or behavior (or some combination thereof) associated with distress and/or impaired functioning” (2017). Thus, while we aimed to provide information on both mental health and illness, challenges to mental health were defined broadly in the context of the current training, and were considered any conditions or psychological states that derailed optimum mental health, which may or may not qualify as a diagnosable mental illness.

The training consisted of a four-hour classroom session that was delivered to groups of 20–30 supervisors per session. Each training session was conducted by two trained and registered psychologists with experience in clinical settings. The classroom-based session was divided into several sections. The first section introduced a model of physical, psychological, and social well-being, and through facilitated activities and discussion, helped participants to identify their own mental health challenges. For example, attendees were invited to consider the differences between ‘surviving’ and ‘thriving’ at work, and then identify specific issues at work that may be contributing to their personal levels of mental health. Kitchener and Jorm (2002, 2004) noted that the public have difficulty recognizing specific disorders or instances of psychological distress contributing to widespread stigma and delays in both recognition and help seeking related to mental illness. As such, the next section raised awareness of mental health issues through sharing of prevalence statistics via an animated short film, followed by detailed information about common mental health challenges including depression, psychosis, and anxiety-related challenges. In the third section, the facilitators explained the concept of ‘mental toughness’ and provided tools, strategies, and concepts to increase adaptability in the context of adversity including reframing (based on cognitive-behavioral therapy techniques), locus of control, focusing on personal strengths, developing social support networks, and managing personal stress. The final section outlined a process to conduct ‘supportive conversations’ with employees experiencing mental health issues or temporary psychological distress, and participants were given time to practice this model and receive constructive feedback from the expert facilitators.

Training Evaluation

Arguably the most popular model of training evaluation is that identified by Kirkpatrick (1976) and later augmented by Kraiger et al. (1993), which identifies four levels of criteria relevant to training evaluation—reactions, learning, behavior, and results. Consistent with this framework, we investigated changes in participants’ perceived knowledge, attitudes toward mental illness, and personal reports of changes in well-being-targeted behaviors before and after the training. We also examined the role of supervisors’ perceptions of well-being self-efficacy (i.e., supervisors’ confidence to manage their personal well-being as it relates to mental health, and support employees’ mental health) to better understand the process by which the training had its effects.

Participant Reactions to the Training

Participants’ experiences of a training event are important indicators of engagement, and although there is mixed evidence supporting their predictive utility, such measures are widely used in organizations and typically demonstrate a positive, albeit weak relationship with learning and transfer (Ruona et al. 2002). Given the training program was designed specifically for the target organization, it is important to establish that participants’ reactions to the training were positive and contribute to the transfer of relevant knowledge and skills. As the training was designed according to adult learning principles, such as high interactivity and leveraging the experiences of participants to maximize learning, we anticipated that reactions would be favorable (Fig. 1).

  • Hypothesis 1: Participants in the training groups will provide positive ratings of (a) their satisfaction with the training, (b) perceived usefulness of the training, and (c) their motivation to apply new knowledge and skills to the job.

Fig. 1
figure 1

Map of Measurement Occasions and Corresponding Hypotheses. Note. H2ab, H4ab, H6ab refer to within-group changes from before to after participating in the training. H3ab, H5ab, H7ab refer to between-group comparisons evaluating differences between control groups (Time 1) and training groups (Time 2/Time 3). Bolded text and lines represent hypotheses that were supported. Ns = Hypothesis not supported. + = Partial support for hypothesis

Full size image

Knowledge and Attitudes Toward Mental Health

In the context of supporting mental health within one’s work group, the effectiveness of supervisors is in part dependent on knowledge and skills that go beyond general leadership competencies. For instance, some research suggests that factors intrinsic to work are associated with mental health challenges among employees; as such, the symptoms and expression of mental health issues in the workplace context may be distinct from those in other life domains (Bogg and Cooper 1995). Therefore, knowledge specific to mental health conditions and symptoms at work, such as the attitudinal or behavioral indicators that suggest someone may be suffering from a serious mental health condition are important informational elements that we expected would help prompt supervisors to recognize mental health challenges both internally (relevant to their own well-being) and externally with employees.

Similarly, the training program included a concerted effort to address potentially negative or unhelpful attitudes toward mental health, as attitudes predict behavioral intentions (Azjen 1991; Armitage and Conner 2001) and should, therefore, impact the extent to which supervisors are motivated to respond to personal and employee mental health challenges in an open and constructive manner. For example, some supervisors may feel that mental health issues suffered by employees are outside their scope of influence (i.e., “none of my business”) or may harbor negative attitudes and beliefs about mental health challenges and those who suffer from them. Therefore, in addition to unique knowledge and skills related to the recognition of mental health issues in the workplace, supervisors need to be willing to engage with employees around these sensitive topics. The provision of information that counteracts inaccuracies or stigmas about mental health conditions and challenges unhelpful assumptions represents another space in which specific leadership development related to supporting mental health is necessary. Drawing on these arguments, the following hypotheses were proposed and represented in Fig. 1:

  • Hypothesis 2: There will be a (a) significant increase in perceived knowledge targeted by the training and (b) a significant decrease in negative attitudes toward mental health from Time 1 to Time 2 within training groups.

  • Hypothesis 3: (a) Perceived knowledge targeted by the training will be significantly greater, and (b) negative attitudes toward mental health will be significantly lower in training groups at Time 2 compared to control groups at Time 1.

Personal Well-Being Behavior and Supervisor Well-Being Support

An additional goal of the training was to facilitate a change in supervisors’ well-being-related behavior on the job. Therefore, critical to an evaluation effort is the specification of behavioral outcomes that evidence the transfer of learned knowledge, skills, and attitudes to the job. In the current study, we differentiated between personal well-being behavior (PWB) and supervisor well-being support that is employee-targeted (SWS). With regard to the former, these behaviors refer to work and non-work behaviors that supervisors engage in to support and facilitate their own personal well-being, as it relates to mental health (e.g., managing personal distress). Conversely, SWS refers to supervisors’ reports of their behaviors that support their employees’ mental health (e.g., supporting distressed employees). If successful, participation in the training should contribute to increased efforts by supervisors to facilitate and improve both PWB and SWS. Therefore, we proposed the following hypotheses:

  • Hypothesis 4: There will be a significant increase in supervisors’ reports of (a) personal well-being behavior (PWB) and perceptions of (b) supervisor well-being support (SWS) from Time 1 to Time 3 within training groups.

  • Hypothesis 5: Supervisors’ reports of (a) PWB and (b) SWS will be significantly greater in training groups at Time 3 compared to control groups at Time 1.

The Role of Well-Being Self-Efficacy

Particularly in the current context, there may be a number of challenges that dissuade supervisors from actively applying the skills and knowledge related to supporting positive mental health in the workplace, including a sense of confidence and motivation to apply new knowledge and skills on the job (Colquitt et al. 2000). Simply being exposed to new information about mental health in the workplace may be insufficient. Instead, supervisors need to be willing and able to use the knowledge and skills learned in the training with employees on a day-to-day basis. Indeed, self-efficacy has been consistently noted as a moderate to strong predictor of training transfer as it directly encourages trainees to apply their learning (Blume et al. 2010) and has been shown in prior research to be modifiable in the context of mental health interventions (Dimoff et al. 2015; Kitchener and Jorm 2002, 2004).

In the current training, we focused on supervisors’ personal well-being self-efficacy (PWSE) and supervisors’ well-being support efficacy (SWSE). The former refers to supervisors’ perceived capability with respect to managing personal well-being challenges, such as distress at work. The latter refers to supervisors’ perceived capability to engage in behaviors that will support their employees’ well-being and mental health, such as supporting employees who show signs of psychological distress. In both cases we expected to see positive change following participation in the training. As such, the following hypotheses were proposed:

  • Hypothesis 6: There will be a significant increase in (a) personal well-being self-efficacy (PWSE) and (b) supervisor well-being support efficacy (SWSE) from Time 1 to Time 2 within training groups.

  • Hypothesis 7: (a) PWSE and (b) SWSE will be significantly greater in training groups at Time 2 compared to control groups at Time 1.

Consistent with prior research on the role of training self-efficacy in predicting training transfer (Blume et al. 2010), we expected that participants’ levels of well-being self-efficacy immediately following the training would be positively related to reports of well-being behavior and support demonstrated post-training. Further, given the specific focus on conducting supportive conversations with employees and generally ‘checking in’ more frequently with employees about their mental health, we expected that SWSE would also be related to supervisors’ self-reported effort to communicate about mental health with employees—that is, the frequency and percentage of the team that supervisors reported carrying out well-being conversations after their participation in the training. Specifically, we hypothesized the following:

  • Hypothesis 8a: Supervisor-reported PWSE at Time 2 will be positively related to PWB at Time 3.

  • Hypothesis 8b: Supervisor-reported SWSE at Time 2 will be positively related to perceived SWS at Time 3.

  • Hypothesis 8c: Supervisor-reported SWSE at Time 2 will be positively related to reports of well-being communication effort at Time 3.

Method

Participants & Procedure

A total of 949 frontline, middle, and senior supervisors from five geographical regions of a large Australian financial services institution participated in the training. Participants were encouraged to participate in the study as part of a broader implementation of a customized mental health awareness and personal well-being program. This training was delivered by the authors’ previous employer (see conflict of interest statement), who was contracted by the organization to design and implement a supervisor-focused well-being training. Supervisors could opt-out of the study at any point without penalty. Approximately 81% of participants reported company tenure of 5 years or more and 43% were aged between 30 and 39 years. Approximately 42% had between five and ten subordinates reporting directly to them at Time 1. Participating supervisors represented a range of different job positions throughout the ‘Personal Insurance’ division of the company; however, the primary role served by most supervisors was a team-leader function overseeing a group of employees that act as customer contact personnel via the company’s telephone and email inquiry lines.

Data were collected at three time points: Time 1 (pre-training), Time 2 (immediately post-training), and Time 3 (1 month post-training). This approach was used to capture baseline measures prior to any training occurring, measure changes in proximal variables, obtain post-training reactions, and assess the impact of the training on self-assessed behavior after participants returned to work. At Time 1, an initial invitation email containing a link to the online survey was distributed to all potential participants approximately 2 weeks prior to the training. Included in this communication were informed consent flyers and contact information for the research team in case employees wanted to discuss any questions or concerns related to their participation in the study. The Time 1 measure collected demographics and baseline measures for study variables. At Time 2, participants completed a paper-and-pencil survey following their participation in the training which assessed proximal outcomes for variables likely to change immediately following the training (e.g., perceived well-being self-efficacy). Finally, the Time 3 measure was a follow-up survey distributed via email approximately 4 weeks after the training.

Response rates at each data collection point varied, with 57% (n = 543) of participants responding at Time 1, 84% (n = 793) responding at Time 2, and 31% responding at Time 3 (n = 296). Given the lower response rates to the Time 1 and Time 3 surveys, a χ 2 test was conducted to determine whether the proportions of respondents according to geographical region differed from the proportions observed in the employee population. Although the results of the test were statistically-significant at a p < .05 level (χ 2(5) = 12.88, p = .02), only two of the five regions differed from the full employee participant population figures (one area slightly underrepresented and the other overrepresented), so overall, the study participants were taken as representative of the regions included within the broader training implementation. Further, we evaluated the extent to which non-respondents differed to respondents on key variables that might have confounded the evaluation results including amount of knowledge and skills gained, evaluations of the training (reactions such as enjoyment), well-being self-efficacy, and attitudes toward mental health. The two groups did not differ significantly on any of these variables across either of the time-point comparisons, which suggests that the training experiences of respondents and non-respondents were comparable.Footnote 1

Finally, to combat the reduction in sample size given removal of duplicate cases and respondent attrition due to study drop out,Footnote 2 we created separate datasets for each analysis. Matched cases from the Time 1 and Time 2 surveys totaled 229 (24% match rate). Matched cases from the Time 1 and Time 3 surveys totaled 94 (10% match rate). Finally, matched cases across all three time points totaled 78 (8% match rate).

Measures

Survey items for all scales are included in the Appendix.

Training reactions

A series of 11 items were used to measure participants’ reactions to the training at Time 2. Five items measured participants’ affective ratings of the training delivery (e.g., “The training was carried out in a professional manner”). Cronbach’s alpha for the five items was .84. Two items measured ‘training transfer readiness’ or motivation to apply learned concepts and skills to the job (Casey and Krauss 2013). An example item reads: “I am eager to apply what I learned during this training to my job”. Additional single-item measures were used to assess overall satisfaction with the training and utility reactions (e.g., “Please rate the extent to which you found this training to be useful to your job”).

Attitudes Toward Mental Health

A total of four items were selected from the nine-item Depression Stigma Scale (Griffiths et al. 2004) and customized to suit the study context (See Appendix for list of items used). Participants provided responses to these items at Time 1 and Time 2. Higher scores indicated more negative attitudes. A six-point Likert agreement scale was used to record responses (1 = Strongly Disagree to 6 = Strongly Agree). Internal consistency alpha was .70 and .69 at Time 1 and Time 2 respectively.

Perceived Knowledge

A five-item custom scale was created through a process of content review and consultation with key stakeholders involved in the development of the training. Supervisors reported the amount of knowledge they currently have about key topics using a five-point scale (1 = None to 5 = A lot). The five key topics included: Differences between a mental health issue and a normal reaction to work stress, behaviors that suggest an employee is struggling to cope, behaviors that suggest an employee is performing at his/her optimum level, adapting to stress and adversity, and the steps involved in a supportive conversation. These responses were provided at Time 1 and Time 2. Internal consistency alpha was .86 at Time 1 and Time 2.

Well-being Self-Efficacy

Following the guidelines outlined by Bandura (2006), two three-item domain-specific self-efficacy scales were created to measure the level of confidence supervisors held in their ability to a) show training-targeted skills at work with respect to their personal well-being (PWSE) and b) show training-targeted skills at work with respect to supporting their employees’ well-being (SWSE). Well-being self-efficacy was assessed at Time 1 and Time 2. Respondents used a six-point Likert agreement scale (1 = Strongly Disagree to 6 = Strongly Agree). Internal consistency alpha for PWSE was .85 and .91 at Time 1 and Time 2, respectively. Internal consistency alpha for SWSE was .87 and .88 at Time 1 and Time 2, respectively.

Perceived Well-Being Behavior

A similar process to that described above was used to create two scales to measure the extent to which participants reported engaging in training-targeted well-being supportive behaviors in reference to their a) personal well-being (PWB; 3 items) and b) their employees’ well-being (SWS; 3 items). At Time 1 and Time 3 participants were asked to self-report the extent to which they performed the behaviors at work within the past month, with the option to select ‘not applicable’ if an opportunity to perform had not been experienced. Internal consistency alpha was .81 and .80 at Time 1 and Time 3, respectively. A six-point extent scale was used to record responses (1 = Not at all to 6 = Very great extent).

Perceived Well-Being Communication Effort

The following single-item measures assessed supervisors’ perceptions of their well-being communication effort: (1) “Over the past month, on average how often did you carry out informal conversations with your subordinates regarding their well-being?”, and (2) “Over the past month, with what percentage of subordinates have you carried out informal conversations regarding their well-being?”

In order to assess the discriminant validity of the study measures, we followed the process outlined by Byrne (2001), in which a series of nested models were estimated and evaluated using the Satorra-Bentler (Satorra and Bentler 2010) corrected χ2 statistic. First, a single congeneric model was estimated and summary statistics indicated it was a very poor fit to the model. Five subsequent models were then estimated, differentiating perceived knowledge, then self-assessed behavior, and finally attitudes and self-efficacy. The final measurement model exhibited acceptable fit statistics: χ2 (174) = 460.70, p < .01; RMSEA = .07, p < .01; CFI = .91; TLI = .89.

Analytical Strategy

The current study employed a quasi-experimental rolling groups evaluation design (RGD; Quinones and Tonidandel 2003). Similar to a “switching replications” (Cook and Campbell 1979) or a “staggered intervention” (Robson et al. 2001) approach, under the RGD method, all participants receive the training but do so in ‘waves’ that align with the implementation schedule. This strategy means that later groups serve as controls for groups conducted earlier in the schedule; for instance, the Time 1 results from the ‘control’ group are compared with the Time 2 results from the ‘training’ group.

According to the RGD method, a training has had a meaningful impact on measured outcomes if there are statistically-significant differences on outcomes (1) between Time 1 and Time 2 results of the training group (within-group changes pre and post exposure to the training), (2) between Time 1 and Time 2 results of the ‘control’ group (replicated pre-post change within the control group after this group is exposed to the training), and (3) no significant difference between Time 1 results of the training and ‘control’ groups (reflecting equivalence of the two groups before exposure to the training). Finally, there should be a significant difference between Time 1 results of the ‘control group’ and Time 2 results of the ‘training group’ (between-group differences in results). This final step verifies that contextual factors are likely not responsible for any detected pre-post changes in the training group, given that the control group is based in the same organization and so would be exposed to the same maturation effects.

Consistent with this approach, we divided the training cohort into four ‘waves’ of participants that represented the major geographical regions for the Australia-wide implementation. Random assignment to these groups was not logistically possible. A series of χ 2 tests were used to assess whether any statistically-significant differences in study participants’ demographics existed between each of the groups. No evidence of significant differences in tenure (χ 2 (9) = 9.46, p = .40), age range (χ 2 (12) = 10.60, p = .56), or number of subordinates (χ 2 (9) = 9.37, p = .40) were found. As a result of this method, Time 1 was not the same for all participants. Participants completed the Time 1 survey over a period of 3 months (as employees were allocated to training programs).

Results

Table 1 presents descriptive statistics and bivariate correlations between study variables. At a general level, we found positive associations between pre- and post- measures of knowledge targeted in the training (r = .33, p < .001), attitudes toward mental health (r = .23, p < .05), PWSE (r = .37, p < .001), SWSE (r = .39, p < .001), PWB (r = .42, p < .001), and SWS (r = .60, p < .001). The correlation between PWSE and PWB did not reach statistical significance (r = .14, ns), nor did the correlation between SWSE and SWS (r = .19, ns); however, the relationship between PSWE and well-being communication effort in the form of frequency of supervisor check-ins was positive and significant (r = .28, p < .001), but not the percent of team with whom check-ins were completed (r = .16, ns).

Table 1 Intercorrelations among study variables
Full size table

Training Reactions

First, we examined participants’ affective and utility reactions to the training content and delivery, and motivation to apply learned concepts and skills in the workplace. Overall, supervisors’ reactions to the training delivery were favorable (M = 5.40, SD = .51) with 75% (395) reporting that the content included in the training was either ‘extremely’ or ‘very’ useful to their job. Further, 75% (395) of attendees reported that the training was either ‘extremely’ or ‘very’ enjoyable. Nearly all (98%, 516) of participants reported that they would recommend the training to other people. On average, participants reported high motivation to apply what they learned on return to the workplace (M = 5.44, SD = .62). Based on these results, Hypothesis 1 was supported.

Examining Changes in Perceived Knowledge, Attitudes, and Self-Assessed Behavior

Hypothesis 2 concerned perceived knowledge targeted by the training and attitudes toward mental health. Table 2 provides the details of each step in the RGD approach. In summary, within each wave of participants, we found a significant increase in perceived knowledge from Time 1 to Time 2 (Wave 1: t (108) = 12.35, p < .001; Wave 2: t (34) = 8.48, p < .001; Wave 3: t (46) = 7.85, p < .001; Wave 4: t (37) = 5.78, p < .001), supporting Hypothesis 2a. With regard to attitudes toward mental health, while there was a significant decrease in negative attitudes toward mental health for Waves 1 and 2 (Wave 1: t (108) = −2.02, p < .05; Wave 2: t (34) = −2.73, p < .001), there was not a significant decrease within Waves 3 and 4 (Wave 3: t (46) = −1.56, ns; Wave 4: t (37) = .84, ns), providing only partial support for Hypothesis 2b. A comparison of differences in perceived knowledge between the control group at Time 1 and the training group at Time 2 (Wave 1 compared to Wave 2; Wave 3 compared to Wave 4) showed that there was a significant difference in perceived knowledge in both instances (Wave 1 compared to Wave 2: t (143) = 8.29, p < .001; Wave 3 compared to Wave 4: t (84) = 5.68, p < .001), providing support for Hypothesis 3a. However, there was no evidence for significant between-group differences when comparing Time 2 attitudes of the training groups to Time 1 attitudes of the control groups Wave 1 compared to Wave 2: t (143) = 1.68, ns; Wave 3 compared to Wave 4: t (84) = .73, ns), indicating no support for Hypothesis 3b.

Table 2 Means, SDs, sample sizes, and t tests of training outcomes (rolling groups design)
Full size table

Hypothesis 4 concerned changes in supervisors’ self-assessed PWB and SWS. Results supported a significant increase in PWB from Time 1 to Time 3 for all waves of participants (Wave 1: t (31) = 3.40, p < .001; Wave 2: t (9) = 2.14, p < .05; Wave 3: t (15) = 5.18, p < .001; Wave 4: t (19) = 3.82, p < .001), supporting Hypothesis 4a, and this pattern was mirrored for SWS (Wave 1: t (31) = 3.47, p < .001; Wave 2: t (9) = 2.53, p < .05; Wave 3: t (15) = 4.63, p < .001; Wave 4: t (19) = 5.72, p < .001), supporting Hypothesis 4b. When looking across groups, results showed that there was a significant difference between PWB of the training group at Time 3 and PWB of the control group at Time 1 (Wave 1 compared to Wave 2: t (40) = 4.19, p < .001; Wave 3 compared to Wave 4: t (34) = 2.50, p < .05), and significant differences between reported SWS when training and control groups were compared (Wave 1 compared to Wave 2: t (40) = 4.90, p < .001; Wave 3 compared to Wave 4: t (34) = 2.54, p < .05), providing support for Hypothesis 5a and 5b.

Examining the Role of Well-Being Self-Efficacy

Hypothesis 6a and 6b concerned changes in supervisors’ PWSE and SWSE, respectively. We found significant increases in PWSE (Wave 1: t (108) = 3.45, p < .001; Wave 2: t (34) = 2.50, p < .05; Wave 3: t (46) = 5.66, p < .001; Wave 4: t (37) = 5.46, p < .001) and SWSE (Wave 1: t (108) = 10.36, p < .001; Wave 2: t (34) = 9.25, p < .001; Wave 3: t (46) = 5.29, p < .001; Wave 4: t (37) = 5.40, p < .001) from Time 1 to Time 2 for all waves of participants, satisfying steps 1 and 2 of the RGD and providing support for Hypothesis 6a and 6b. When looking across waves, results showed that when comparing PWSE for the training group at Time 2 to the control group at Time 1, there was a significant difference only between Waves 1 and 2 (t (142) = 2.37, p < .05), but not for Waves 3 and 4 (t (83) = 1.87, ns). Thus, Hypothesis 7a was only partially supported. In the case of supervisors’ reports of SWSE, all steps in the RGD process were satisfied indicating significant between-group differences in Time 2 SWSE of the training group compared to Time 1 SWSE of the control group (Wave 1 compared to Wave 2: t (142) = 10.54, p < .001; Wave 3 compared to Wave 4: t (83) = 5.06, p < .001), therefore supporting Hypothesis 7b.

Hypotheses 8a concerned the relationship between supervisors’ reports of PWSE and PWB after participating in the training. Using ordinary-least squares multiple regression, we found that Time 2 PWSE positively predicted supervisor-reported PWB at Time 3 (β = .40, p < .01; R2 adj = .15, F (1,79) = 14.74, p < .01), which supports Hypothesis 8a. Hypothesis 8b was concerned with the relationship between supervisors’ reports of SWSE and SWS. Consistent with our hypotheses, the overall model was statistically-significant, R2 adj = .31, F (1,77) = 36.68, p < .01. Time 2 SWSE positively predicted supervisor-reported SWS at Time 3, β = .57, p < .01, thus, Hypothesis 8b was supported. Finally, Hypotheses 8c proposed that SWSE at Time 2 would positively predict supervisors’ self-reported well-being communication effort at Time 3—that is, how often supervisors’ reported checking-in with their team with regard to well-being and the percentage of team members checked. Given that these dependent variables were single indicators measured using ordinal response scales, ordinal logistic regression was used to evaluate the hypotheses. Results showed that for every unit increase in supervisors’ reported SWSE at Time 2, the odds of a supervisor checking in with their team regarding their well-being increased by 3.18 times (95% CI: 1.24–8.12, z = 2.40, p < .05). Next, supervisors’ reports of the percentage of team members checked in with at Time 3 was regressed on SWSE at Time 2. For each unit increase in SWSE, the odds of a supervisor checking in with a higher percentage of their team increased by 2.34 times (95% CI: 0.98–5.57, z = 1.92, p = .05). These results supported Hypothesis 8c.

Discussion

Recent reports from across the developed world indicate that adverse mental health conditions related to the experience and lack of management of stress are prevalent in society in general and the workplace in particular (ABS 2009). The financial and social impact of work-related mental health concerns encourage oganizational researchers to prioritize mental health in the workplace and find ways to support employees. The current study aimed to address two primary objectives. First, we sought to evaluate the impact of a supervisor-focused mental health training intended to equip supervisors to become advocates for their own mental health, as well as serve as a resource for employees facing mental health challenges at work. Second, we specifically examined well-being self-efficacy (personal well-being self-efficacy and supervisors well-being support efficacy) as one personal factor that was targeted by the training and that we expected would impact supervisors’ personal well-being behavior and well-being support for employees.

Our results supported the general effectiveness of the training in terms of increasing supervisors’ perceived knowledge related to mental health in the workplace, as well as increasing supervisor-reported personal well-being behavior and employee-targeted, supervisor well-being support, from before to after participation in the training. Notably, attitudes toward mental health significantly improved in only two out of four waves of participants, indicating that the training was not equally successful in improving attitudes toward mental health. However, one possible explanation for this result was the already relatively positive attitudes of participants at Time 1, which may indicate the potential for a greater effect of the training in other industries or occupations in which attitudes toward mental health are generally less positive (e.g., heavy industry) at the outset.

However, we did find that the training successfully increased both personally-targeted well-being self-efficacy and supervisor well-being support efficacy and both were positively associated with supervisor-reports of their respective well-being behavior. That is, personally-targeted well-being self-efficacy was positively associated with personal well-being behavior 1 month later; and, supervisor well-being support efficacy was positively associated with supervisor-reports of subsequent supervisor well-being support targeted toward employees, the frequency with which supervisors reported checking in with their employees regarding well-being, and the percentage of the team with whom supervisors reported checking in. Simply put, supervisors who felt confident to manage their personal mental health were more likely to report engaging in behaviors supporting their well-being. Further, supervisors who felt confident to support their employees’ mental health reported enacting supportive behaviors more often and with a larger proportion of their team.

These findings are consistent with prior research that has found significant increases in self-efficacy as a result of participation in mental health awareness training (Dimoff et al. 2015), and further supports research that suggests that domain-specific self-efficacy is an important antecedent to the successful transfer of training (e.g., Blume et al. 2010). These results provide important insight into the potential mechanisms that underlie the current training and the translation of knowledge and skills learned in training to reported changes in well-being-targeted behavior on the job. However, it should be noted that despite the implied indirect relationships among these variables, we did not test full mediation models given power constraints due to the small sample size. These tests are important for future research as they control for the effects of the outcome variable at Time 1, and therefore provide a more nuanced look at changes in the focal outcomes as a result of the intervening variable (e.g., self-efficacy).

In addition, our analyses revealed the potential importance of focusing on supervisors’ own well-being as a supplement to efforts aimed at supporting others. Although causality cannot be established due to the cross-sectional nature of the assessment of supervisor-reported well-being behaviors, we did find that supervisors’ personal well-being behaviors were positively correlated with their reports of supervisor well-being support at Time 3. It is possible that by addressing supervisors’ well-being and providing them with skills to effectively manage personal challenges to mental health, supervisors are able to effectively role model positive well-being behavior and self-care. At the same time, doing so may enable important energetic and attentional resources to be made available that can be redirected at supportive leadership behavior. Consistent with this notion, Byrne et al. (2014) found that supervisors who suffered from diminished psychological resources as indicated by depressive symptoms, anxiety, and workplace alcohol consumption were less likely to exhibit transformational leadership behaviors and more likely to display abusive supervisory behaviors. Building on these findings and others, the current study supports an approach that combines development of supervisors’ well-being with the provision of practical skills needed to support employees’ mental health, specifically. Future research aimed at better understanding the directionality of these relationships is an important next step.

The current research has a number of strengths and limitations that are important to consider when evaluating our findings and can inform future research in this area. First, we answered the call for evidence-based leadership development that equips and empowers supervisors to improve employee well-being (Kelloway and Barling 2010; Nielsen et al. 2010). Despite a growing body of research suggesting that leaders play an integral role in employee mental health, few reports of large-scale supervisor-focused interventions exist. To address this, the current evaluation effort was designed to the best extent possible in accordance with evidence-based training evaluation guidelines (Kirkpatrick 1976; Kraiger et al. 1993) including assessments at multiple time points and contextualized measurement tools to assess learning and behavior. This approach underpins the effectiveness of this training and adds confidence to the findings reported here.

It should be noted that the aim of the current study was to equip supervisors with the tools and resources to support their personal mental health and that of their employees; however, the current training did not explicitly address ‘upstream’ or macro organizational factors that may be contributing to mental health issues in the workplace. Although we view supervisors as an important factor in an employees’ work environment, there are, of course, other relevant organizational and work-related features that influence employee mental health at work and may serve as important levers in organizational interventions (e.g., see Stansfeld and Candy 2006 for a meta-analysis). For example, high job demands, lack of role clarity, and poorly managed organizational change can influence mental health at work. Importantly, we do not advocate for stress management or tertiary approaches in lieu of primary intervention techniques aimed at eliminating stress at its source, but rather as an important supplement to these efforts. Indeed, future applications of the current training could incorporate more directly supervisors’ role in managing and shaping various work characteristics or other features of the work environment to reduce exacerbating circumstances (e.g., stress) for employees.

Another limitation is that we were not able to assess employee reports of changes in supervisory behavior; thus, the primary outcomes in the current study rely on supervisors’ perceptions and reports of their own experiences and behaviors. Unfortunately, practical constraints precluded our ability to access employees during the training. Future research that includes employee reports of both supervisor behavior as well as perceived changes in the mental health or well-being of employees would contribute to our understanding of the impact of the training. For instance, Kelloway and Barling (2010) note that leadership development is largely an indirect phenomenon in which the success or failure of these efforts are seen in outcomes at the employee level. Indeed, employee reports of changes in supervisory behavior would serve to further validate our current findings, and would enable a test of whether such changes in behavior had a measurable impact on the well-being and mental health of employees—which, of course, is the ultimate goal of such a training. Despite this limitation, prior intervention research aimed at improving mental health literacy (e.g., Kitchener and Jorm 2002, 2004) and within occupational health more broadly (e.g., Unsworth and Mason 2012), has relied on self-reports of participant changes and has provided useful and important preliminary information that can serve as a foundation for future training and intervention efforts.

Conclusion

A critical challenge for organizations who endeavor to build a culture of employee well-being and health is first, expanding the discussion of employee well-being beyond the traditional concepts of stressors and strains to acknowledge the widespread prevalence of many mental health issues faced by workers; and to identify ways to support employees to effectively cope with such challenges. The current training focused on increasing supervisors’ knowledge of mental health issues, reducing negative attitudes toward mental health, and providing tools for the effective management of personal mental health and the support of employees’ mental health. We suggest that creating a context in which employees are made aware of and encouraged to openly access resources that can support them in times of particular distress without repercussion or negative consequence are important goals for organizations. Identifying critical leverage points within the organization (such as supervisors) that can serve as targets for training and development are important preventative measures central to creating a positive organizational culture around employee health and well-being.