Introduction

Supported Employment (SE) is an evidence-based practice that assists people with severe mental illness to obtain and maintain employment [1]. For people with mental health issues, SE programs, in particular the Individual Placement and Support model (IPS), have better vocational successes than other vocational rehabilitation programs. Although SE programs share the same core principles, IPS programs emphasize the close collaboration between the vocational team (employment specialists) and the mental health treatment team. IPS programs also promote working closely with employers to facilitate obtaining and maintaining competitive employment while recommending a caseload size of approximately 20 clients per employment specialist [1–4]. In randomized controlled trials an average of 60 % of people with severe mental illness obtained employment through IPS whereas <25 % of people in other vocational rehabilitation programs obtained employment [1, 5]. Despite these conclusive results, approximately one-third of SE participants do not find employment [6]. Even if SE programs have a high level of fidelity for the IPS model standards, investigators observe significant variation in vocational successes from one study to another [5].

An interesting but rarely investigated field is employment specialist competencies. Drake et al. [7] reported a large variation in vocational success rates (25–75 %) for the employment specialists in several IPS programs, possibly reflecting differences in competency or working alliance. More recently, Bond et al.’s study [8] showed that in 15 randomized controlled trials conducted using IPS programs, the competitive employment rate was poorer in Canada and European countries (about 47 %) compared to the USA, Australia and Asia (about 62 %). Explanations for these differences included the disability policies of Canada and the European countries as well as the lack of training of employment specialists working in SE programs [8–10].

Since many programmatic and other variables have already been studied in Canada [2] and several European countries (e.g., the Netherlands [10]), the employment specialist competencies should be examined, specifically how the competencies contribute to the success of SE programs. Inspired by IPS principles [11], the employment specialist competencies are generally described as [12, 13]: (1) To work closely with clients to plan work integration while considering work interests and preferences; (2) to coordinate with other health or vocational professionals around clients’ career objectives; (3) to work with several stakeholders from the workplace or community (e.g., employers, case managers, family members) for helping clients obtain and keep jobs; (4) to provide advice to clients about the benefits and social advantages of work integration; (5) to work primarily in the community, meeting with potential employers rather than engaging in office-based activities; (6) to provide support to help clients maintain competitive employment once they secure it; (7) to recommend as needed to employers and supervisors in the workplace feasible and economic work accommodations to facilitate client work integration. Furthermore, employment specialists are recognized to play a key role because they interact with numerous stakeholders [12, 14], especially workplace stakeholders such as employers [1]. This type of interaction with people from the workplace has been defined as the competency of Job development when aimed at creating jobs or helping a client get a job [1, 15].

In recent years, many qualitative studies have investigated SE components, including employment specialist competencies, to identify those most helpful for the work integration of people with severe mental illness [16–18]. As Corbière and Lanctôt [19] noted, these competencies can be grouped in relation to the working alliance (e.g., [20]), the recovery philosophy (e.g., [13]), the supports offered to people with severe mental illness (e.g., [21]) and disclosure and work accommodations issues (e.g., [22]). These authors also found that clients enrolled in SE programs often cited employment specialist competencies supporting their work integration, particularly, using job search strategies, marketing clients’ abilities and establishing a working alliance. The SE programs encourage these competencies through their application of the recovery philosophy. Whitley et al.’s study [18] on the other hand focused more on personal characteristics such as initiative, empathy, and persistence.

More recently, Drake et al. [1] reported that employment specialist competencies likely influence outcomes but studies that have rigorously analyzed them are rare. To address this, Dreher et al. [23] developed the IPS-Q, a 30-item multiple-choice quiz (i.e., includes incorrect answers) for employment specialists, covering fundamental IPS principles and key SE components. This tool gathers information about the level of knowledge of SE program employment specialists. However, the authors admitted that it does not identify which skills the employment specialists applied effectively in practice. We agree with Dreher et al. [23] that the level of knowledge is a valid criterion for assessing competencies and add that the attitudes and behaviors relating to skills in different fields (e.g., communication, negotiation, relationships, marketing) should also be assessed to get a complete picture of employment specialist competencies [9]. Overall, we need to better understand how vocational successes in SE programs are affected by employment specialist competencies. To our knowledge, the IPS-Q is the only measure that evaluates employment specialist knowledge but it does not address other important competency components such as attitudes and behaviors. A more comprehensive tool for assessing employment specialist competencies should also identify this professional’s training needs to improve SE program outcomes in any country.

The three objectives of this study are: (1) To develop a questionnaire covering a broad range of employment specialist competencies—attitudes, behaviors and knowledge—recognized as important for assisting clients with work integration; (2) to conduct exploratory factor analyses using a priori conceptual categories to explore emerging factors from the questionnaire; and (3) to identify the most significant employment specialist competencies for helping clients obtain and maintain competitive employment.

Methods

Development of the BAKES

The Behaviors, Attitudes and Knowledge in Employment Specialists—BAKES [24] tool was developed to better understand the most important competencies for SE program employment specialists assisting clients with severe mental illness to obtain and maintain employment. To accurately identify employment specialist competencies (i.e., attitudes, behaviors and knowledge), we: (1) reviewed the relevant literature; (2) operationalized competencies items in attitudes, behaviors or knowledge, in the context of SE programs; (3) tested the first version of the tool with experienced employment specialists and made revisions based on their feedback; (4) created conceptual categories for employment specialist competencies; and (5) translated the BAKES from English to Dutch. These steps were based on several methods: literature review, development of tools, the Delphi method and evaluation of content validity, the use of concept mapping and, finally, the back-translation procedure.

First, we consulted the literature to identify items for inclusion in the BAKES (including major search terms such as competencies, skills, questionnaires, tools, counselors, employment specialists, mental health professionals). Since little has been written about the competencies of employment specialists, the majority of articles consulted were from the areas of psychiatric rehabilitation, vocational rehabilitation and career counseling. We also considered articles dealing with mental illness and stigma, self-efficacy theory and recovery to get a more complete perspective of the work of employment specialists. Information specific to employment specialists was supplemented through general Internet searches, grey literature searches, SE program guidelines and through direct communications with employment specialists. This review, which included papers on employment specialists, psychotherapists, psychologists, and other professional competencies in mental health, was large enough to identify a broad range of competencies for an employment specialist, such as clinician–client relationship, rehabilitation and empowerment, knowledge in mental health, attitudes and behaviors for approaching an employer or for working with different stakeholders (e.g., [6, 7, 25, 26–31]). For example, the Career Counseling Self-Efficacy Scale [32] assesses a counselor’s confidence in providing career counseling to their clients and includes both knowledge and attitudes the SE program employment specialist may use. Other scales were helpful in creating the format, layout and content for the BAKES. Most relevant was Coursey et al’s study [27] exploring the fundamental attitudes, behaviors and knowledge in mental health professionals working in the rehabilitation field. Hagner et al.’s qualitative study [29] focusing on competencies needed to handle work-related incidents (e.g., work absenteeism) was also useful for developing the BAKES since it included stakeholder interactions in community employment program staff competencies, particularly Working with employers and Working with other professionals and agencies. In addition, the influential “consumer–survivor” movement contributed articles about empowerment and recovery. Since empowerment and recovery are central to the SE model, we included this perspective [33].

Second, we formulated items related to employment specialist attitudes, behaviors and knowledge considered to be potentially important for facilitating work integration in people with severe mental illness. We believed that it would be more useful to ask employment specialists to describe what they knew or did to help their clients achieve their vocational goals, rather than to ask them evaluate their confidence in specific vocational rehabilitation skills. We chose a 7-point Likert scale for responding to behaviors and attitudes items (1 = never to 7 = always) and for knowledge items (1 = not at all to 7 = perfectly). The first pool from the literature review consisted of 405 items representing behaviors, attitudes and knowledge. We included the maximum number of items to be certain we would assess all skills related to employment specialist competencies. The BAKES authors met repeatedly to discard irrelevant items and adapt others to be specific to employment specialists (resulting in a total of 280 items kept at this stage).

Third, in the Vancouver area (British Columbia, Canada), seven experienced employment specialists working in SE programs (most using the IPS model) were invited to select the most essential items listed in the 280 version scale. This procedure allowed us to ensure the content validity of the BAKES. The choice of opening up to other types of SE programs allowed us to enlarge the pool of employment specialist competencies and SE components [2]. The instructions to these employment specialists were: Here is a large pool of behaviors, attitudes and knowledge in employment specialist skills and we would like to get your opinion regarding the most essential ones in your work with the perspective to indicate to future employment specialists what are the most important to develop in your profession. Possible responses were recorded on a scale of 7 points (1 = not at all essential to 7 = completely essential). This consultation allowed us to reduce the original 280 items to a final pool of 156 items, all retained by at least one of the seven experienced employment specialists (score greater than 5 for each item) and revised by our team to avoid redundancy.

Fourth, two authors (M. C. and N. L.) assigned a priori groups of items to larger conceptual categories using elements of the concept mapping procedure [34]. Concept mapping consists of a structured conceptualization process that helps identify main components or conceptual dimensions from a studied phenomenon (employment specialists’ competencies). As suggested by Kane and Trochim [34], all 156 items were written on a separate sheet, classified first by attitudes and behaviors together and, separately, by knowledge. These authors realized that attitudes and behaviors could be difficult to distinguish. For example, an attitude could refer to Maintain a positive attitude toward your clients regarding their employability while a behavior could refer to Listen carefully to your clients and support them throughout their work integration process. After this classification procedure, items were then put together according to their content and, finally, resulting in five conceptual categories: (1) Teaching proactive skills and strategies to obtain and maintain employment (n = 48); (2) Teaching social skills and self-management (n = 13); (3) establishing relationships with stakeholders and service providers (n = 26); (4) informing clients about stigma and using a client-centered approach (n = 36); and (5) knowledge about services, workplaces and policies pertaining to work disability (n = 33).

Fifth, we conducted a translation of the BAKES in several steps. Two researchers (E. B. and J. vW.) translated the BAKES questionnaire from English to Dutch. Both translators spoke mother tongue Dutch and were familiar with the area of research. During this step, they communicated with the authors of the English version to accurately translate the technical elements. The first version of the translated questionnaire was then tested with seven native Dutch-speaking researchers. After integrating the Dutch-speaking researchers’ comments, four employment specialists were asked to fill out the questionnaire and gave their feedback. They received concise information about the BAKES questionnaire and completed the questionnaire without supervision via a web form application (SurveyMonkey). Following this, a final version of the Dutch version of the BAKES was created, integrating the comments received during the testing steps.

Data Collection

The BAKES was validated by administering it to employment specialists in two countries, Canada and the Netherlands. The questionnaire was available in three languages, French and English for Canada, and Dutch for the Netherlands. The preliminary section of the BAKES relates to sociodemographics (e.g., age, sex, work history) and caseload (number of clients per month). In addition, the questions about vocational successes were: During the past year, how many of your clients have found competitive employment (i.e., jobs not reserved for people with a disability that pay at least minimum wage and that are part- or full-time)? Of those clients who have found a job on the competitive job market, how many have kept their job for more than 6 months? The project was approved by the ethics committee of the Université de Sherbrooke.

The Canadian participants were recruited from a larger study on the implementation of SE programs in three Canadian provinces (British Columbia, Ontario, and Quebec) [19, 35]. The 97 participating employment specialists came from 24 different SE programs dedicated to people with severe mental illness. Eligible employment specialists: (1) Worked in a SE program identified for the larger study described above; and (2) worked in SE programs for longer than 1 year. The average age was 41.07 years (SD 11.12) and 78.4 % (N = 76) of the sample was female. The average work history in the SE program was 5 years (SD = 5.71). Of the 97 employment specialists, 50.5 % (n = 49) worked in a SE program based on the IPS model while the rest (n = 48) worked in SE programs using other models (e.g., choose-get-keep model). The average caseload was 24 clients (SD 12) for employment specialists working in IPS programs and 37 clients (SD 23) for employment specialists working in other SE programs.

The Dutch participants were recruited from 12 mental health care institutions and/or vocational rehabilitation agencies in the Netherlands. Employment specialists (n = 56) from SE programs were only recruited if their daily caseloads consisted mainly of people with severe mental illness. Most of the invited employment specialists were involved in the nationwide learning collaborative implementing SE/IPS [36]. During a meeting of this learning collaborative they received oral information about the study from one of the researchers (J. vW.). In addition, one of the authors (J. vW.) organized special information meetings about the study at several agencies. He asked the invited employment specialists to pass the study information along to eligible colleagues who might be interested in participating in the Internet-based survey (SurveyMonkey, http://www.surveymonkey.com). The mean age was 41.3 years (SD = 11.08) and 71.4 % (N = 40) of the sample was female. The average work history in the SE program was 4.6 years (SD = 5.32). Of the 56 employment specialists, 32.1 % (n = 18) indicated that they worked in a SE program based on the IPS model while the rest (n = 38) worked in programs using other types of models. The average caseload was 27 clients (SD = 13) for employment specialists working in IPS programs and 32 clients (SD = 20) for other SE programs.

Analyses

First, we carried out exploratory factor analyses using the Statistical Package for the Social Sciences (SPSS–IBM Corporation, Armonk, New York) program principal components analyses (PCA) to allow factor extraction from the BAKES items. Five PCAs with orthogonal rotation (varimax) were applied to the BAKES items’ conceptual categories: (1) Teaching proactive skills and strategies to obtain and maintain employment (n = 48); (2) teaching social skills and self-management (n = 13); (3) establishing relationships with stakeholders and service providers (n = 26); (4) informing clients about stigma and using a client-centered approach (n = 36); and (5) knowledge about services, workplaces and policies pertaining to work disability (n = 33). The reasons for using five PCAs were: (1) to explore potential subscales from each of the five conceptual categories; and (2) to reduce the number of items for each conceptual category (or subscale). Our goal was not only to have a more concise version of the BAKES, since 156 items is cumbersome, but also to specify items belonging to each emerging factor in relation to the five conceptual categories. Cronbach’s alpha coefficient was calculated for all emerging factors from each PCA to obtain the level of internal consistency of each new dimension or subscale.

Second, by using a MANOVA with Tukey’s-b post hoc test, we evaluated the potential differences between four groups of employment specialists, with respect to BAKES subscales, by identifying the employment specialist’s country of origin (Canada or Netherlands) and the type of SE program (IPS model or other types of SE programs) they used. We expected that regardless of the country of origin, employment specialists working in IPS programs would have lower caseloads (about 20 clients) [4] and higher scores on the emerging BAKES subscales specifically relevant to the IPS model (such as: Relationships with mental health professionals [2, 3]; and Relationships with employers and supervisors [1]).

Third, linear regression equations were calculated for employment specialists’ competencies using a stepwise method to determine the most significant competencies for predicting the clients’ vocational success, i.e., the number of clients who obtained competitive employment in the previous year and, of those, the number who maintained competitive employment for more than 6 months. Both regressions were adjusted for the employment specialist variables of age, gender, and work history in the SE program. The regressions controlled for the type of SE program (IPS or other types) and other variables related to work outcome, such as caseload size as a predictor of obtaining employment, and the number of clients who obtained competitive employment in the past year as a predictor of maintaining employment.

Results

Before conducting exploratory factor analyses, the Dutch and Canadian samples were compared with respect to sex, age, work history, and type of SE programs the employment specialists worked in. No differences were detected for sex (χ² = 0.93, df = 1, p = 0.34), age (t = −0.109, df = 144, p = 0.91), caseload (t = −0.64, df = 99, p = 0.52), and work history (t = −0.404, df = 148, p = 0.69). The only significant difference was related to the type of SE program (χ² = 4.87, df = 1, p = 0.03): Canadian employment specialists worked in IPS programs more often than other types of programs compared to Dutch employment specialists. All subsequent analyses were conducted with the whole sample from the Netherlands and Canada.

Five PCAs were carried out on the five conceptual categories of the BAKES. The results for all PCAs had first eigenvalues between 5.1 and 9.4 and accounted for 49.5, 54.5, 63.6, 50.6 and 66.7 % of the total variance for each conceptual category, respectively (Tables 1, 2, 3, 4, 5). A total of 66 items did not load significantly on one specific factor (<0.40) and were dropped, resulting in 90 items retained for the final PCA solutions. Each of the five PCAs included two or three dimensions or subscales, for a total of 12 subscales (Tables 1, 2, 3, 4, 5). Alpha coefficients for the results from each conceptual category ranged from 0.77 to 0.94, indicating a satisfactory internal consistency for the 12 subscales entitled: Outreach and work accommodations (12 items), Job search strategies (6 items), Support in identifying work interests and labor market (6 items), Social behaviors and healthy lifestyle (6 items), Adaptation to and dealing with the workplace culture (6 items), Relationships with employers and supervisors (10 items), Relationships with mental health professionals (10 items), Dealing with stigma and self-stigma (7 items), Support and client-centered approach (8 items), Knowledge of clinical symptoms and health/vocational services (11 items), Knowledge of the workplace (5 items), and Knowledge of the laws and policies related to disability (3 items).

Table 1 Factor structure of the BAKES—teaching proactive skills and strategies to obtain and maintain employment (N = 153)
Full size table
Table 2 Factor structure of the BAKES—teaching social skills and self-management (N = 153)
Full size table
Table 3 Factor structure of the BAKES—establishing relationships with stakeholders and service providers (N = 153)
Full size table
Table 4 Factor structure of the BAKES—informing clients about stigma and using a client-centered approach (N = 153)
Full size table
Table 5 Factor structure of the BAKES—knowledge about services, workplaces and policies pertaining to work disability (N = 153)
Full size table

A MANOVA (Wilk’s Lambda multivariate test) revealed that the BAKES subscales were significantly related to country (F = 13.13, df = 12, p < 0.001) and specific type of SE program (IPS or other) (F = 3.30, df = 12, p < 0.001), but there was no significant interaction between country and SE program type (F = 0.53, df = 12, p > 0.05). As expected, the results indicated that Canadian and Dutch employment specialists working in IPS programs had significantly higher scores on three subscales (see Table 6): Outreach and work accommodations (F = 15.39, df = 1, p < 0.001); Relationships with employers and supervisors (F = 6.98, df = 1, p < 0.01); and Relationships with mental health professionals (F = 11.59, df = 1, p < 0.001). However, regardless of the type of SE program, Canadian employment specialists had significantly higher scores than Dutch professionals for support and client-centered approach (F = 39.6, df = 1, p < 0.001), job search strategies (F = 11.25, df = 1, p < 0.001), adaptation to and dealing with the workplace culture (F = 9.24, df = 1, p < 0.001), support in identifying work interests and labor market (F = 6.16, df = 1, p < 0.05), knowledge of clinical symptoms and health/vocational services (F = 19.67, df = 1, p < 0.001), and knowledge of the workplace (F = 29.94, df = 1, p < 0.001). No significant differences (p > 0.05) were observed for the country or the type of SE programs for the following BAKES scales: social behaviors and healthy lifestyle, dealing with stigma and self-stigma, and knowledge of the laws and policies related to disability.

Table 6 Mean (SD) for each BAKES subscale according to specific groups of employment specialists
Full size table

When employment specialists (n = 101) who answered the two questions on vocational successes (obtaining and maintaining competitive employment in their clients) were compared to employment specialists who did not (N = 52), the t test and χ² values did not detect significant differences for sex (χ² = 0.32, df = 1, p = 0.57), age (t = 0.28, df = 144, p = 0.77), work history (t = −1.40, dl = 148, p = 0.16), type of SE program (χ² = 0.91, df = 1, p = 0.34), country (χ² = 1.97, df = 1, p = 0.16), and caseload size (t = 1.83, df = 146, p = 0.07).

Linear regressions using a stepwise method indicated that 51.1 % of the adjusted variance linked to the number of clients who obtained competitive employment in the previous year was accounted for by the caseload (β = 0.58, t = 8.03, p < 0.001), country (β = −0.47, t = −6.35, p < 0.001) and the BAKES scale Relationships with employers and supervisors (β = 0.19, t = 2.54, p < 0.01).

For linear regression results using a stepwise method for predicting maintaining client employment, 70.2 % of the adjusted variance associated with the number of clients who maintained competitive employment for more than 6 months was accounted for by the number of clients who obtained employment (β = 0.75, t = 12.80, p < 0.001), the number of years the employment specialist participated in the SE program (β = 0.16, t = 2.69, p < 0.01), and three types of competencies: those related to Relationships with employers and supervisors (β = 0.20, t = 3.12, p < 0.01), those related to Adaptation to and dealing with workplace culture (β = −0.21, t = −2.92, p < 0.01), and those associated with support and a client-centered approach (β = 0.14, t = 2.04, p < 0.04).

Discussion

The goal of this paper was to develop a questionnaire to assess employment specialists’ competencies to facilitate obtaining and keeping competitive work of people with severe mental illness enrolled in SE programs. Our results provide a detailed picture of the employment specialists’ competencies, identifying which competencies relate to client vocational success. From a large pool of attitudes, behaviors and knowledge required in the rehabilitation field, this study identified the specific employment competencies needed to help people with several mental illness obtain and maintain competitive employment. In particular, 90 competencies spread over 12 subscales emerged from factor analyses. Most of the 12 BAKES subscales have been highlighted in the literature as critical ingredients in SE programs and employment specialists’ competencies. Some of the competencies are exclusively related to the IPS model whereas others are more generic, applicable to vocational counselors in various practices. These results will be discussed below, followed by an examination of the dimensions significantly related to work outcomes.

The BAKES subscale related to Relationships with mental health professionals is considered an essential principle in the IPS model and distinguishes IPS from other SE programs [1–3, 37]. Regular, frequent communication with treatment team members to share information in both directions can help clients implement strategies to facilitate work integration or to coordinate and integrate vocational services into mental health treatment [1]. This type of collaboration can also inform the mental health team about the advantages of working for clients in their recovery process and, thus, educate the mental health team about the work capabilities of people with mental illness [37].

The BAKES subscale Outreach and work accommodation included several items such as Re-contact your clients when you do not hear from them or Spend most of your time meeting with clients and employers outside your office and was depicted by the Outreach dimension [18, 38], where employment specialists actively establish contacts with clients and spend most of their time in the community. It is interesting to note that other proactive attitudes, such as Encourage your clients to obtain competitive jobs as soon as possible after registering in the Supported Employment program and Inform your clients about the impact obtaining a job may have on their benefits related to their disability compensation were loaded on the same dimension. These two items more precisely reflect the two IPS principles: rapid job search and benefits counseling [1]. Furthermore, the Outreach and work accommodation scale is of interest since work accommodation is essential for helping people maintain competitive employment [16, 39]. These work accommodations can be implemented inside or outside the workplace, for example, accessing transportation with the help of a co-worker [16] or enlisting a co-worker as a mentor (buddy-system) to ensure the work activity of the person with a mental illness [40].

Different authors have observed the competencies of the BAKES scale Relationships with employers and supervisors as essential and they are often translated in the literature as job development and follow-along support [1, 15]. Leff et al. [15] defined Job development as any direct or indirect contact with employers or networking with individuals possessing job information, such as supervisors. Job development involves cultivating a relationship with employers [1], illustrated by the BAKES scale item: Establish and maintain trusting relationships with employers in the competitive job market. Follow-along support is defined as individualized and continued assistance for as long as the client wants and needs the support [1]; this IPS principle can also refer to workplace support, as illustrated in the BAKES tool item, Work in collaboration with employers to modify job tasks when your clients encounter difficulties. Corbière and Lanctôt’s [19] review of the literature, supplemented by interviews conducted with employment specialists, directors of SE programs and clients, emphasized the specific role of the employment specialist in building partnerships with stakeholders (e.g., employers and other mental health professionals).

The BAKES dimension Job search strategies, particularly the use of these strategies during work integration, has been considered essential for finding a job [41]. Clients enrolled in supported programs consider these strategies to be essential for obtaining competitive employment [19]. Some job search strategies may be more effective than others for obtaining employment, for example, calling employers directly [42]. Using job search strategies requires the employment specialist to correctly evaluate both the clients’ work preferences and the local labour market (another BAKES subscale) to achieve the best fit between workplace requirements and individual interests and competencies. This increases work satisfaction and the likelihood of maintaining employment [1, 43].

Employment specialists must make use of knowledge from a variety of fields to suggest appropriate work integration skills and strategies to clients. Three types of knowledge emerged from the BAKES questionnaire: workplace, clinical and policy knowledge. When comparing the competencies of employment specialists versus return to work coordinators, Loisel and Corbière [9] observed that knowledge was important for employment specialists, particularly workplace and policy knowledge. Clinical knowledge was considered secondary since too much focus on clinical characteristics may hamper people with a disability returning to or obtaining employment [9].

The BAKES also identified several factors in other categories of employment specialist competencies not exclusively related to SE programs. For example, Bond and Drake [44] found that process variables, such as the support and client-centered approach, have rarely been explored and should be analyzed. To our knowledge, only a few recent publications [15, 19, 20, 45, 46] have stressed the importance of evaluating process variables, e.g., Use supportive interpersonal skills in working with your clients, such as listening and empathizing (item from Table 4). These behaviors and attitudes tap the ability of employment specialists to demonstrate empathy, develop rapport, and establish effective working relationships with clients. This solid relationship better helps clients to learn about dealing with stigma and prejudices that exist in the workplace [47, 48].

Two new BAKES subscales emerged from factor analyses related to the ability of employment specialists to help clients deal with potential difficulties faced in the workplace such as understanding the expected code of conduct: Social behaviors and healthy life style and Adaptation and dealing with the workplace culture. Many authors have studied several models for learning these skills, such as The workplace fundamental skills module [49, 50], which helps clients to deal with conflicts arising in the workplace, improve conversational skills and more. Similarly, Baker et al. [51] developed an intervention focusing on the adoption of healthier lifestyle choices by encouraging smoking cessation and improving diet and physical activity. Employment specialists may use these training tools to help their clients obtain and maintain competitive employment [52], though Baker’s tool requires more investigation regarding its impact on work outcomes.

The third objective of the study was to identify the most significant competencies associated with obtaining and maintaining employment. After controlling for individual (i.e., sex, age) and programmatic (i.e., caseload size, location, type of SE model) characteristics, linear regression results indicated that the subscale Relationships with employers and supervisors was the most significant competency related to job acquisition. This competency was also significant for helping people maintain employment [1, 15, 53]. In other words, this competency involves not just approaching employers, but maintaining relationships with them over time, in order to facilitate job retention. These results reflect the cornerstone of this competency which includes systematically approaching employers and other stakeholders from the workplace [14, 15]. The Relationships with employers and supervisors competency was recently added to the list of essential IPS principles, which Drake et al. [1] termed Systematic job development. Whitley et al. [18] noted that employers may not necessarily be open to being approached by employment specialists, perseverance eventually becomes profitable, enabling the employment specialists to develop relationships with many employers. These BAKES subscale items also correspond to the definition of follow-along support such as Involve supervisors at your clients’ workplaces to help them keep their jobs. We know that the supervisor’s role is essential for the work integration of people with a disability or for the implementation of work accommodations [54]. As noted above, the implementation of work accommodations and follow-along support are essential for helping people maintain employment [39, 53, 55], therefore, these competencies are some of the most helpful for maintaining competitive employment.

Two additional dimensions were significantly related to job tenure: Support and client-centered approach and Adaptation to and dealing with the workplace culture. Interestingly, Leff et al.’s study [15] highlighted support from employment specialists as an essential attitude and behaviour to adopt for helping people with severe mental illness maintain employment. The negative and significant relationship between the BAKES subscale Adaption to and dealing with the workplace culture and job tenure was counterintuitive. One hypothesis to account for these results is that when employment specialists intervene too often on behalf of their clients (e.g., to de-escalate a crisis), co-workers or employers may become less tolerant of these interventions and develop increased stigma towards people with severe mental illness. Unfortunately, we do not know if or how often the employment specialists intervened in the workplace—in fact most items on this subscale do not make explicit reference to supports in the workplace. This negative result could suggest that the employment specialist uses several strategies when the client is having significant difficulties at work (e.g., encouraging appropriate behavior, dealing with crisis situations, coping with stress, dealing with conflict, encouraging friendliness) and this may be linked to shorter job tenure, even when support is provided. More investigation of this subscale is warranted.

Finally, regarding the linear regression results, it is noteworthy to mention that the type of SE program, IPS or other models, was not a significant variable to predict work outcomes (obtaining and maintaining employment). These last results put emphasis on the importance of the employment specialist competencies and, as reported by Drake et al. [7], reflect the large variation in vocational success rates observed by employment specialists even within IPS programs. These elements are discussed in the next paragraph (i.e., limitations).

This study has a few limitations. First, the vocational outcome measure is self-reported by the employment specialists and could be biased by social desirability. Second, vocational outcomes were assessed in the preceding year and could not represent the actual competencies of employment specialists. The most significant learning and application of knowledge, attitudes and behaviors by employment specialists usually occurs at the beginning of their training. It is likely that fewer changes in practice occur after their initial training. Note that one of our inclusion criteria was to recruit employment specialists working in SE programs for longer than 1 year to control for potential changes in competencies. The average work histories in SE programs in our samples were 4.6 and 5 years for the Netherlands and Canada, respectively. Accordingly, we believe that even though the vocational successes were evaluated in the past year, no significant (or only very subtle) changes in the employment specialists’ competencies would have occurred since then. Third, we did not verify whether employment specialists came from a SE program with a high or low level of fidelity to SE program standards, particularly when employment specialists indicated that they were working in IPS programs. A proxy variable, the caseload size, informed us that employment specialists working in IPS programs had a lower caseload (25 clients) compared to other SE programs (>32 clients), thus, indicating that IPS programs in this study conformed to the expected caseload size. In addition, the MANOVA results indicated a significant relation between employment specialists working in IPS programs and those working in other types of SE programs, particularly for the BAKES scales related to IPS programs (i.e., relationships with mental health professionals and employers). In summary, a lower caseload size and frequent adoption of specific attitudes and behaviours can reflect the variety of SE programs (IPS model or other types) or a higher level of fidelity of IPS programs [56]. Fourth, the BAKES does not specifically assess competencies for handling cognitive impairment in SE program clients. McGurk and Mueser [57] have found that specific strategies for helping clients cope with cognitive impairments increase work outcomes. Finally, it is noteworthy that some BAKES dimensions (e.g., support and client-centered approach) have higher scores when comparing employment specialists working in Canada to those working in the Netherlands. These results could be explained partly by employment specialist training in the Netherlands, which is less thorough than the training provided in Canada [10]. These limitations could be overcome by assessing individual characteristics of clients, employment specialist competencies and SE program fidelity in the same multilevel analysis.

The clinical implications of the BAKES suggest it would be a useful tool for SE programs to verify that new employment specialists have integrated the competencies needed to improve their training, thus, creating conditions for more significant vocational success for their clients. This evaluation could be used when a new employment specialist is trained, with a follow-up after each trimester over 1 year. Supervisors could also teach and guide new employment specialists, particularly at the commencement of SE program implementation, to ensure a consistent and effective service [9]. Appropriate training is essential for new employment specialists joining an SE team to ensure appropriate interventions and accurate implementation of this evidence-based practice [58–60].

In conclusion, during the implementation of SE programs, it is not only important to evaluate the program using fidelity scales but also to systematically evaluate employment specialist competencies. The BAKES questionnaire is a solid and comprehensive evaluation of employment specialist competencies. Even though all BAKES dimensions are important for helping people with severe mental illness obtain and maintain competitive employment, two are significant for predicting vocational successes: (1) Relationships with employers and supervisors, and (2) Support and client-centered approach. We believe that the development and validation of the BAKES will better define a job summary for this professional and improve training and supervision of future employment specialists working in SE programs.