Professionals working in community mental health are at high risk for experiencing staff burnout (Oddie and Ousley 2007; Rohland 2000; Siebert 2005), which is frequently conceptualized as high levels of emotional exhaustion and depersonalization (negative or cynical attitudes), along with a diminished sense of personal achievement (Maslach 1993). Burnout has been associated with negative consequences for mental health providers, including physical health problems (e.g., insomnia, headaches, poor overall health), relationship problems, reduced job satisfaction, and increased mental health problems (e.g., depression, anxiety, substance abuse) (Carson et al. 1999; Heaney et al. 1995; Killian 2008; Long et al. 1990; Perrone et al. 2006; Rohland 2000; Smoot and Gonzolas 1995; Stalker and Harvey 2002). Burnout is also problematic for organizations, and literature reviews frequently report increased absenteeism, tardiness, reduced job commitment, and, in some studies, poor job performance and increased turnover (Burke and Richardsen 1993; Schwab et al. 1986; Smoot and Gonzolas 1995; Stalker and Harvey 2002). Staff burnout is widely believed to impact quality of care; however, little direct research has examined burnout in relation to mental health services.

Quality of care provided in healthcare settings may involve a number of possible domains. Specifically in mental health, measurement of quality has been identified as “particularly challenging,” because of a lack of measurement tools, as well as a lack of consistent infrastructure and strategy for assessing quality (Kilbourne et al. 2010). While consensus on specific assessment approaches is lacking, the Institute of Medicine has highlighted six domains of quality, specifying that care should be safe, effective, patient-centered, timely, efficient, and equitable (Institute of Medicine 2001). Quality of care could also be conceptualized as the vehicle by which mental health clinicians impact consumer outcomes. As an example, seminal meta-analytic work by Horvath and Symonds (1991) and later updated by others (Lambert and Barley 2001; Martin et al. 2000) have indicated a link between the quality of the therapeutic relationship (e.g., patient-centeredness and therapeutic alliance) and psychotherapy outcomes, over and above the use of specific therapeutic techniques. Thus, any influences of staff burnout on quality of care could be particularly important in impacting consumer outcomes.

Burnout could impact quality of care in a number of different ways. At the organizational level, burned out workers who take excessive time off or leave their jobs will disrupt the continuity of mental health care (Boyer and Bond 1999). Further, studies have shown that burnout among some workers is likely to have a contagion effect, damaging the morale of other employees and leading to further staff turnover (Bakker and Demerouti 2007; Bakker et al. 2005). At the individual level, burnout has been associated with cognitive impairments, including decreased attention (van der Linden et al. 2005), which can lead to errors and less engagement. Burnout has been associated with decreased empathy (Astrom et al. 1991), and in a recent study of medical residents, reduced empathy mediated the relationship between burnout and lower self-ratings of patient-centered care (Passalacqua and Segrin 2012). Burnout has been associated with negative feelings about mental health consumers (Holmqvist and Jeanneau 2006) and negative expectations about consumer recovery (Salyers et al. 2013). Negative staff attitudes, in turn, have been linked with poorer outcomes among consumers with serious mental illness, for example, in the area of employment (Gowdy et al. 2003).

One indirect indicator of high quality care may be satisfaction with care, and several studies have examined the links between staff burnout and patient satisfaction in healthcare settings. Higher levels of burnout among doctors (Argentero et al. 2008; Halbesleben and Rathert 2008) and nurses (Argentero et al. 2008; Leiter et al. 1998; McHugh et al. 2011; Vahey et al. 2004) have been associated with lower patient satisfaction. We found one study specifically examining burnout and satisfaction in community mental health. Garman et al. (2002) surveyed clinicians and consumers from 31 psychosocial rehabilitation programs. Using team-level aggregation, higher levels of emotional exhaustion were associated with less consumer satisfaction related to the environment, treatment, and autonomy; depersonalization was associated with less satisfaction with treatment; and low personal accomplishment was associated with less satisfaction with therapists.

A more rigorous indicator of high quality of care is improved consumer outcomes, and a few studies have gone beyond satisfaction to assess outcomes. Priebe et al. (2004) studied 24 assertive outreach teams working with adults with severe mental illness in London. Team level burnout was associated with increased hospitalizations 9 months later; however, multivariate tests showed different findings, indicating other factors may moderate/mediate these relationships. Halbesleben and Rathert (2008) surveyed 178 matched pairs of physicians and patients who had been hospitalized in the prior year. Using path analysis, physician depersonalization was related to lower patient satisfaction and longer patient reported recovery times. More recently, a cross-sectional study of 89 outpatient substance abuse treatment facilities aggregated staff and consumer surveys at the level of the facility, finding burnout not directly related to treatment participation (Landrum et al. 2012). While the relationship between burnout and consumer satisfaction is generally consistent, research linking burnout and consumer engagement and outcomes are few and have mixed findings.

Another approach to assessing quality of care is by asking providers their perceptions of care, an alternative that may be less time and labor-intensive. In addition, staff-reported quality of care may be a good proxy measure when actual client outcomes or satisfaction data are not readily available. In studies of health care professions that empirically examine the relationship between burnout and self-reported quality of care, physicians reporting higher levels of burnout (emotional exhaustion and/or depersonalization) tend to report lower levels of quality of care (Gopal et al. 2005; Shanafelt et al. 2002; Williams et al. 2007). One study found that burnout mediated the relationship between work load and poor self-reported quality of care (Shirom et al. 2006). Nurses with high levels of burnout also tend to report lower quality of care (Halbesleben et al. 2008; Van Bogaert et al. 2009). We found only one study examining staff-reported quality of care and staff morale in the mental health field. Van Bogaert et al. (2012) surveyed psychiatric nurses in inpatient settings and found that better work engagement (conceptualized as the opposite of burnout) was associated with higher self-reported quality of care. To our knowledge, no studies have examined burnout in relation to staff-reported quality of care in community mental health settings.

In a related area of work, several studies have found higher levels of burnout were correlated with more self-reported errors among physicians (Fahrenkopf et al. 2008; Hayashino et al. 2012; Prins et al. 2009; Shanafelt et al. 2010; Williams et al. 2007). One longitudinal study found a prospective relationship; residents with high levels of burnout were more likely to report having made an error 3 months later (West et al. 2006). The relationship between burnout and self-reported errors in nurses, however, is mixed. Laschinger and Leiter (2006) found that nurses with high levels of burnout reported more patient safety adverse events. In another study of nurses, burnout was not related to self-reported errors, but was related to frequency of reporting near misses—though in the opposite direction; nurses with high emotional exhaustion were less likely to report near misses (Halbesleben et al. 2008).

We were interested in examining the relationship between burnout and self-reported quality of care among community mental health workers. Prior studies have used a variety of methods to assess self-reported quality of care, none specific to community mental health care. For example, Van Bogaert et al. (2012) asked three global indices of care (e.g., “The quality of care on the unit is fair or poor”). Shanafelt et al. (2002) developed an 8-item scale of more specific behaviors (e.g., “I found myself discharging patients to make the service ‘manageable’ because the team was so busy”) and attitudes (“I had little emotional reaction to the death of one of my patients”). Items from this measure have been used in other studies of medical doctors (Gopal et al. 2005; Williams et al. 2007). However, we know little about what constitutes quality of care as perceived by community mental health workers and how the constructs are related to burnout. Building on this prior work, we developed a scale that could be used to examine the relationship between quality of care and burnout in community mental health.

Because a new scale was created, we first examined psychometric properties through factor analysis, internal consistency, and correlations with related constructs. We then tested the relationship between burnout and quality. We hypothesized that community mental health staff with higher levels of emotional exhaustion and depersonalization, and lower levels of personal accomplishment, would report lower levels of quality of care.

Method

Participants

Employees at a Community Mental Health Center (CMHC) in a Midwestern city were invited to participate. As part of an ongoing program evaluation, participants completed a semi-annual staff survey that included scales to assess burnout, job satisfaction, intentions to turnover, and expectations of consumers. We added a new scale to assess quality of care specifically developed for this study. All staff were invited to participate, not just direct clinical care providers. Of approximately 156 possible staff, 113 (72 %) responded with complete data. Participants were primarily female (83 %) and white (96 %). Staff education levels included 36 % with less than a bachelor’s degree, 38 % with a bachelor’s, and 26 % with a graduate degree. Staff reported spending an average of 50.2 % (SD = 32.7) of their time in direct care, ranging from 0 to 100 %. Staff also reported spending an average of 21.7 % (SD = 28.2) of their time in administrative duties, ranging from 0 to 100 %. The mean length of time at the agency was 6.7 years (SD = 6.2), ranging from a month to 29 years, and the mean length of time in the mental health field was 10.5 years (SD = 8.2), ranging from one month to 38 years.

Measures

The survey included limited demographic information, our primary independent variable (burnout), our primary dependent variable (Quality of Care), and other job-related variables of job satisfaction, intentions to turnover, and expectations of consumers.

Burnout was assessed with an adapted version of the Human Service Provider version of the Maslach Burnout Inventory (Maslach et al. 1996) that used 7-point anchors ranging from 1 = ‘strongly disagree’ to 7 = ‘strongly agree’. This widely-used scale measures three subscales: emotional exhaustion, depersonalization, and personal accomplishment. The subscales have shown good internal consistency, stability over time, and convergent validity with related constructs (Maslach et al. 1996).

Self-Reported Quality of Care We created a quality of mental healthcare scale for this study. Items were generated by input from clinicians and administrators from the participating CMHC, listing areas of care that they thought could be impacted by provider well-being. We combined clinical input with ideas from scales that have been developed in other healthcare settings (Gopal et al. 2005; Shanafelt et al. 2002; Williams et al. 2007). Items were vetted and refined with input from four other mental health service researchers (all clinical psychologists), supporting face validity of the scale. The scale was pared to 25 items (see Table 1), and respondents were asked to report on the frequency of each occurring in the past month (0 = not at all to 5 = very often). Ten items were reverse scored so that higher numbers indicate better quality.

Table 1 Descriptive information and factor loadings for Quality of Care items (N = 113)
Full size table

Job satisfaction was assessed with 5 items from the Job Diagnostic Survey (Hackman and Oldham 1974). The scale has good internal consistency (Hackman and Oldham 1974) and evidence of convergent and divergent validity (Fried 1991).

Intentions to turnover were assessed by two individual items: “How often have you seriously considered leaving your job in the past six months?” (1 = never to 6 = several times a week) and “How likely are you to leave your job in the next six months?” (1 = not likely at all to 4 = very likely). These have been used in other studies of mental health providers (Salyers et al. 2011a, b).

Expectations of consumer recovery were measured with the Provider Expectations for Recovery Scale, a 10-item scale adapted from the 16-item Consumer Optimism Scale (Salyers et al. 2007). Respondents are asked to think about consumers they currently work with and to answer questions about how many consumers they expect to have specific outcomes (e.g., in housing, employment) on a 5-point scale ranging from 1 = almost all to 5 = none. The scale has been shortened to 10 items through Rasch analyses and has strong internal consistency (α = .91) (Salyers et al. 2013).

Procedures

The research team attended an all-staff meeting in November 2012 and distributed surveys to staff members as they arrived. Participation was voluntary and anonymous, and staff could return completed surveys directly to the team throughout the morning. We randomly selected five participants to receive a $25 gift card. The study and procedures were approved by Indiana University Purdue University Indianapolis (IUPUI) Institutional Review Board.

Analyses

An exploratory factor analysis with a principal axis factoring method and promax rotation was conducted to uncover the underlying structure of the Quality of Care (QOC) scale. Convergent validity of the scale was also examined via correlations with related constructs, including job satisfaction, turnover intentions (past 6 months and next 6 months), and provider expectations for consumer recovery. Finally, in order to test our main hypothesis that burnout is associated with lower QOC, we conducted a multiple regression analysis predicting total QOC scores and the three QOC subscale scores from burnout (emotional exhaustion, depersonalization, and personal accomplishment) controlling for demographics [gender, education level, length of time at the agency, length of time in the mental health field, and job role (time spent in direct care, administration)]. In order to account for missing data, a covariance matrix obtained by an Expectation–Maximization algorithm was used for the factor analysis and multiple imputation (30 imputed datasets) was utilized for the correlation and regression analyses (Enders 2010). SAS ver. 9.2 was used for the analyses.

Results

The exploratory factor analysis retained 17 items and extracted three factors, which we labeled Client-Centered Care, General Work Conscientiousness, and Low Errors. See Table 1 for item descriptives and factor loadings. The number of factors were determined based on scree tests, Kaiser’s criterion (i.e., eigenvalues greater than one), and interpretability. Eight items eliminated were those with either poor factor loading for all factors (i.e., <.30), cross-loadings, freestanding, or high correlations with other items. Kaiser–Meyer–Olkin measure of sampling adequacy was .77. The internal consistency (Cronbach’s αs) for each factor was .88, .72, and .71, respectively, and the total QOC was .84. As shown in Table 2, correlations between factors were low to moderate, ranging from .05 to .46. Client-Centered Care appeared relatively independent of the other two domains of General Work Conscientiousness (r = .16, p = 0.12) and Low Errors (r = .05, p = 0.63).

Table 2 Correlations among quality of care, burnout, and other job-related variables (N = 113)
Full size table

As hypothesized, burnout was associated with self-reported quality of care (also shown in Table 2). Total QOC was related to higher personal accomplishment and less depersonalization. Total QOC was also related to more positive expectations of consumer recovery. At the factor level, Client-Centered Care was positively correlated with more positive personal accomplishment and expectations of consumer recovery (r = .29, p < .01), but had almost no relationship to emotional exhaustion, job satisfaction, or turnover intentions. In contrast, General Work Conscientiousness was positively correlated with job satisfaction (r = .24, p < .01), and negatively correlated with emotional exhaustion, depersonalization, and turnover intentions in the past six months (r = −.30, p < .01) and the next six months (r = −.29, p < .01). Errors were not significantly related to burnout, job satisfaction, turnover intentions, or expectations of consumer recovery.

The multiple regression analysis (Table 3) revealed that participants with higher personal accomplishment (p < .001), lower depersonalization (p < .05), and higher education (more than a bachelor’s degree) (p < .05) had higher total QOC. Similarly, participants with higher personal accomplishment (p < .001) and higher education (p < .05) had higher Client-Centered Care. The overall regression models for two of the QOC subscales (i.e., General Work Conscientiousness and Low Errors) were not statistically significant (p > .05).

Table 3 Summary of multiple regression results predicting Total QOC scores and the three subscale scores across 30 imputed datasets (N = 113)
Full size table

Discussion

This was the first study to assess self-reported quality of care (behaviors and attitudes) in community mental health and to link it with staff burnout. Our newly developed scale demonstrated good internal consistency and correlated with other job related variables (including job satisfaction, turnover intentions, and providers’ expectations of consumer recovery). In addition, burnout was related to quality of care over and above background variables.

Quality of care factored into three distinct areas: client-centeredness, conscientiousness, and reporting low errors. Client-centeredness was comprised of items tapping into how the clinician is working directly with consumers, including attending to, interacting in a collaborative and compassionate way, and helping consumers with goals and treatment plans. In terms of the IOM definitions of quality (Institute of Medicine 2001), this domain may most tap into being patient-centered, effective, and equitable. The second factor of conscientiousness related to aspects outside of direct client care (missing deadlines or appointments, being late) and may fit best into areas of timeliness and efficiency. The third factor, assessing self-reported errors, aligns most closely with safety concerns.

The first two factors appeared to be the most robust, each with different patterns of relationships to other job-related variables. Conscientiousness was moderately related to a range of other variables including job satisfaction, turnover intentions, as well as emotional exhaustion and depersonalization. Conscientiousness may reflect a level of organizational commitment (i.e., to their employer) (Mowday et al. 1979), whereas client-centered care is more indicative of a clinician’s commitment to their consumers. Staff who are dissatisfied with their jobs and are considering leaving the organization, are not putting in as much effort into meeting deadlines and punctuality. Although we can’t determine direction of relationships, this pattern would be consistent with the Conservation of Resources theory of burnout—that being exhausted leads to a reduction of investment to conserve one’s energy (Hobfoll 1989). Conversely, client-centeredness was relatively independent of job satisfaction, turnover, and exhaustion. Even when experiencing work-related stress, mental health workers may be able to (or choose to) protect the quality of their direct clinical interactions.

The third factor (low errors) did not significantly correlate with other variables in this study. This factor was comprised of only two items, one of which had floor effects (reporting major mistakes in one’s work). Although similar items have been used in other samples (Shanafelt et al. 2002, 2010), it may be that errors look different in a community mental health context or for different roles (e.g., case manager versus prescriber), or that reporting behavior differs in these settings. Alternatively, mistakes and errors reported in global self-ratings could simply be more vulnerable to biased reporting. Further work is needed to better assess errors or even “near misses” as they constitute important domains of care quality (Kohn et al. 2000). Institutional-level incident reports of mistakes or near misses could be a better method for assessing safety aspects of care quality, although even those are reliant on clinician reports of incidents in most cases.

The findings linking burnout to quality of care in our sample show some consistency with prior work on burnout and self-reported quality of care in medical residents. For example, our bivariate correlations showed significant relationships between both emotional exhaustion and depersonalization with one aspect of quality—lower general work conscientiousness (e.g., missing appointments, being late). This is similar to work showing depersonalization and/or emotional exhaustion are related to self-reported quality of care in residents (Gopal et al. 2005; Shanafelt et al. 2002; Williams et al. 2007). In our study, although depersonalization was related to overall quality of care, emotional exhaustion was not, and neither emotional exhaustion nor depersonalization were related to client-centered aspects of care quality (e.g., spending time with consumers, helping them meet goals).

Instead, personal accomplishment was the aspect of burnout most closely associated with both overall self-reported quality and client-centered care. Even after controlling for background variables, staff with higher levels of personal accomplishment reported higher levels of client-centered care. This is consistent with a recent qualitative study of physicians who described the important role of patient-doctor relationships and feeling effective in their work as two key areas of gratification in their work (Zwack and Schweitzer 2013). The direction of these relationships, however, is not clear. It may be that staff who have a sense of pride and accomplishment in their work are then more dedicated to the consumers they serve. But it may also be that providing client-centered care is rewarding in itself, leading to a greater sense of personal accomplishment. The other studies examining self-reported quality of care and burnout, however, did not examine personal accomplishment (Gopal et al. 2005; Shanafelt et al. 2002; Williams et al. 2007). Replication and further modeling in longitudinal studies is needed to sort out these relationships over time.

Taken together, an intriguing implication from our findings suggests different routes to improving quality of care through addressing staff burnout. Increasing positive affect (e.g., feelings of personal accomplishment) of staff may be more important for improving client-centered quality than decreasing negative affect (e.g., addressing emotional exhaustion and depersonalization). Conversely, for issues of conscientiousness, it may be more important to find ways to decrease the negative affect. These are hypotheses worth testing in the context of future burnout interventions. For example, one recent study found that a one-day stress management workshop (BREATHE) was associated with reduced emotional exhaustion and depersonalization, but not increased personal accomplishment (Salyers et al. 2011a). It may be that the BREATHE intervention would have more impact on conscientiousness, and other interventions may be needed to improve client-centeredness.

Our study used self-reported quality of care, and we are limited in our ability to comment on more objective indicators of care quality. For example, supervisor or consumer ratings of staff behaviors may be helpful in assessing quality of care being provided. In addition to perceived quality of care (by staff, supervisors, or consumers), actual consumer outcomes will be important to examine as well. There are many theoretical and practical reasons to believe that staff well-being and consumer well-being are related. However, research linking burnout with patient reported outcomes (Halbesleben and Rathert 2008; Landrum et al. 2012) or more objective indicators such as hospital use (Priebe et al. 2004) is in the early stages. In addition, these types of indicators could also be used to better validate the new measure of quality of care. Finally, our study is limited in the sampling frame. We created the scale and tested it in one site that was comprised of predominantly white women, which limits generalizability of the findings. Additional studies are needed in other settings, with a more diverse sample of participants.

Although preliminary, the Self-Reported Quality of Care scale could be useful in future efforts to measure quality of care and understand how burnout impacts quality in mental health organizations. A strength of the scale is that it was developed in conjunction with stakeholders, as recommended by Kilbourne et al. (2010) as one step to overcome challenges in quality assessment in the mental health field. The scale could be administered as a user-friendly and economical tool for program evaluation intending to monitor and improve mental health care quality at both individual and agency levels. In addition, the scale revealed different aspects of quality of care, which were associated with different aspects of burnout and other job-related constructs (including job satisfaction, turnover intentions, and expectations of consumer recovery). These are often concerns for administrators as indicators that may damage organizational climates. Future research is of course needed to validate the scale in other samples, and with other indicators of quality, but this study provides an initial foray into an important domain of inquiry.