BACKGROUND

Burnout among medical students, residents, and practicing physicians has become an increasingly important concern, due both to its high prevalence and its reported associations with patient care, personal well-being, and professionalism.1–10 The gold standard for the measurement of burnout, the Maslach Burnout Inventory (MBI),11 presents challenges for its use in large, multifaceted surveys due to its length (22 items across three domains: emotional exhaustion, depersonalization, and personal accomplishment). A brief measure of burnout that could be used in such settings to allow accurate analysis of how burnout relates to other variables would be useful.

Many burnout studies have focused on the presence of high levels of either emotional exhaustion or depersonalization as the foundation of burnout among high-achieving medical professionals for whom low levels of personal accomplishment may be less likely.12,13 Two single items adapted from the emotional exhaustion (i.e., “How often do you feel burned out from your work?” ) and depersonalization (i.e., “How often do you feel you’ve become more callous toward people since you took this job?”) domains of the MBI may be useful screening questions for burnout in these dimensions.14 These items exhibit the highest factor loadings with their respective burnout domains,11 and have been shown to correlate strongly with the emotional exhaustion and depersonalization domains of burnout as measured by the full MBI in a sample of over 10,000 medical students, residents, and practicing physicians.14 The area under the receiver-operating characteristic curve for the emotional exhaustion and depersonalization single items against their respective full MBI domain measure is 0.94 and 0.93.14 The positive predictive values of the single-item thresholds for high levels of emotional exhaustion and depersonalization are 88.2% and 89.6%, with positive likelihood ratios of 14.9 and 23.4, respectively.14

Despite this evidence supporting the validity of these two items for measuring burnout, their utility relative to the full MBI for evaluating associations between domains of burnout and published outcomes such as suicidality or self-reported major medical errors is unknown. Therefore, to assess the concurrent validity (a type of criterion-based validity occurring when two measures are obtained simultaneously) of these single-item measures for burnout, we compared their performance with that of the full MBI instrument in multiple predictive models.

METHODS

We evaluated models assessing the relationship between burnout and outcomes reflecting important issues involving patient care or physician well-being reported in multiple separate published studies involving medical students, internal medicine residents, and practicing surgeons.4–9 Detailed methods for each of the five evaluated studies are contained within the referenced published manuscripts. Briefly, linked longitudinal data on burnout and suicidality were collected in 2006 and 2007 for 858 medical students at five United States medical schools (Study 1),4 cross-sectional data on burnout and serious thoughts of dropping out of medical school were obtained in 2007 for 2222 medical students at seven United States medical schools (Study 2),6 and cross-sectional data on burnout and unprofessional attitudes and behaviors were collected in 2009 for 2566 medical students at seven United States medical schools (Study 3).7 Linked longitudinal data on burnout and perceived major medical errors were obtained from 2003 to 2009 for 321 Mayo Clinic Rochester internal medicine residents (Study 4).5 Cross-sectional data on burnout and both perceived major medical errors8 and suicidality9 were collected in 2008 for 7,905 members of the American College of Surgeons responding to a national survey (Study 5).

For the present analysis, we compared predictive associations within each study for the single-item burnout measures versus the full MBI. Because emotional exhaustion is measured on a 0–54 scale on the full MBI and the single emotional exhaustion item score ranges from 0–6 (response options for each question on the MBI are on a 7-point Likert scale ranging from “Never” to “Daily”), each 1-point change on this single-item measure equates to a 9-point change in the emotional exhaustion domain of the full MBI. Similarly, because depersonalization is measured on a 0–30 scale on the full MBI and the single depersonalization item score ranges from 0–6, each 1-point change on this single-item measure equates to a 5-point change in the depersonalization domain of the full MBI. Thus, to obtain model estimates comparable with the full MBI subscale scores the emotional exhaustion and depersonalization scores on the single-item measures were multiplied by 9 and 5, respectively. Associations between the previously reported outcomes and burnout as defined by the single-item measures were then compared with associations between these outcomes and burnout as defined by the full MBI. The primary criterion for similarity of the estimates of effect was overlapping confidence intervals.

Although analyses utilizing raw scores in each burnout dimension are preferred for the MBI, it is also common to categorize scores in each of the burnout domains into low, average, or high levels based on the published normative scoring.13 Therefore, we also conducted comparative analyses for categorical burnout domains (i.e., high emotional exhaustion and high depersonalization) comparing the single-item emotional exhaustion and depersonalization measure results to the emotional exhaustion and depersonalization results from the full MBI. For this purpose, high levels of emotional exhaustion and depersonalization on the single items were defined as occurring at least weekly, in accord with thresholds previously reported.14 High levels of emotional exhaustion and depersonalization on the full MBI were defined according to the MBI Manual.11 Finally, overall burnout was assessed, where burnout was defined from both the single-item and full MBI measures by the presence of high levels of emotional exhaustion and/or depersonalization as has been described in prior literature.12,13

Where results were not previously reported, the original data sets were re-analyzed to provide the necessary results. For example, associations between the categorical emotional exhaustion and depersonalization domains and all outcomes other than serious thoughts of dropping out of medical school were not reported in the cited manuscripts,4,5,7–9 and were therefore calculated for the current paper from the original data using methods identical to those detailed in the original references. Similarly, associations between overall burnout and outcomes for internal medicine residents and surgeons were not reported in the cited manuscripts5,8,9 and were generated from the original data.

All contributing studies had approval from the relevant institutional review boards. The MBI for these studies was used under licensed agreement with Consulting Psychologists Press, Inc.

RESULTS

We first evaluated the association of emotional exhaustion as measured by both the full MBI and the single-item measure with patient care and physician well-being outcomes (Table 1). In both the continuous models of raw scores and the dichotomized scoring (high vs. not high) both the single-item and full MBI emotional exhaustion scores were strongly related to all outcomes in medical students, internal medicine residents, and practicing surgeons. The single-item emotional exhaustion measure also exhibited generally consistent exposure-response associations with each outcome, both for the continuous (Fig. 1a) and dichotomized forms (Fig. 1b) of the single-item measure. The magnitudes of association for equivalent changes in raw scores on the full MBI and the single-item measure as assessed by the odds ratios were generally similar to each other, although the single-item measures tended to slightly underestimate the magnitude of association. The magnitudes of association using the dichotomized scoring for both the full MBI and single-item measures were also similar. In no case was the overall conclusion regarding the relationship between emotional exhaustion and each outcome variable altered by use of raw or dichotomous scores for the single-item measure rather than the full MBI.

Table 1 Associations of Emotional Exhaustion (EE) with Patient Care and Physician Well-Being Outcomes, Comparing Full Maslach Burnout Inventory (MBI) Results with Results from Single-Item Assessment of Emotional Exhaustion
Full size table
Figure 1.
figure 1

Association of the (A) continuous.* and (B) dichotomized† single-item measure of emotional exhaustion with outcomes. *Columns within each outcome indicate rate as emotional exhaustion single-item measure ranges from 0 (Never) to 6 (Daily), read from left to right. †Columns within each outcome indicate rate for emotional exhaustion single-item measure categorized in green as “less than once weekly” (not high) and in red as “weekly or more often” (high).

Full size image

We next evaluated the association of depersonalization as measured by the full MBI and the single-item measure with patient care and physician well-being outcomes (Table 2). In both the continuous models of raw scores and the dichotomized scoring both the single-item and full MBI emotional exhaustion scores were strongly related to all outcomes. The single-item depersonalization measure again exhibited generally consistent exposure-response associations with each outcome, both for the continuous (Fig. 2a) and dichotomized forms (Fig. 2b) of the single-item measure. The magnitudes of association for equivalent changes in raw scores on the full MBI and the single-item measure as assessed by the odds ratios were similar to each other, although the single-item measures once again tended to slightly underestimate the magnitude of association. The magnitudes of association using the dichotomized scoring for both the full MBI and single-item measures were also similar, and no conclusion regarding the relationship between depersonalization and each outcome variable was altered by use of raw or dichotomous scores for the single-item measures rather than the full MBI.

Table 2 Associations of Depersonalization (DP) with Patient Care and Physician Well-being Outcomes, Comparing Full Maslach Burnout Inventory (MBI) Results with Results from Single-Item Assessment of Depersonalization
Full size table
Figure 2.
figure 2

Association of the (A) continuous* and (B) dichotomized† single-item measure of depersonalization with outcomes. *Columns within each outcome indicate rate as depersonalization single-item measure ranges from 0 (Never) to 6 (Daily), read from left to right. †Columns within each outcome indicate rate for depersonalization single-item measure categorized in green as “less than once weekly” (not high) and in red as “weekly or more often” (high).

Full size image

Finally, we evaluated the association between overall burnout as measured by the full MBI and the single-item measures with the patient care and physician well-being outcomes (Table 3). Dichotomized overall burnout based on both the single-items and the full MBI was again strongly related to all outcomes (all p < 0.001). The magnitudes of association based on dichotomized overall burnout from both the full MBI and the 2 single-items were similar to each other, and again no conclusions were altered by use of the 2 single-item measures rather than the full MBI.

Table 3 Associations of Burnout* with Patient Care and Physician Well-being Outcomes, Comparing Full Maslach Burnout Inventory (MBI) Results with Results from Single-Item Burnout Assessment
Full size table

DISCUSSION

This study extends evidence in support of a brief burnout assessment tool by confirming the concurrent validity of two items relative to the full MBI. The single-item measures of emotional exhaustion and depersonalization exhibited excellent performance across a broad range of predictive models for high-impact outcomes, including suicidality, professionalism, and medical errors, assessed in medical students, internal medicine residents, and practicing surgeons,.

Although appropriate in many contexts, it is often not feasible to incorporate instruments as long as the MBI into large-scale national surveys covering a variety of topics. Abbreviated burnout measures have been evaluated previously, including a single-item measure focusing only on the emotional exhaustion domain of burnout.15,16 The prior studies of this approach have been limited by low response rates and relatively small sample sizes, in addition to their restricted focus on only a single domain of burnout. Furthermore, because the single item used in these reports is distinct from items within the MBI, the single item does not directly benefit from the three decades of extensive validity evidence that has been established in support of the MBI and its component items since its initial development in 1981.17 The current study has notable strengths in these areas that have limited prior work. In addition to drawing on the existing validity evidence for items comprising the full MBI, the total sample size across this study is large, including well over 10,000 medical students, internal medicine residents, and practicing surgeons.

Given the favorable performance of the single-item measures of burnout, they appear to be useful substitutes for the assessment of burnout in medical students, residents, and practicing physicians when the full MBI cannot be implemented. For example, positive single-item tests of emotional exhaustion and depersonalization might trigger deeper evaluations of distress for individuals. In addition, evaluations of groups of medical professionals using the single-item measures could be used to identify “hot spots” where efforts to improve the learning or working environments may best be directed. This application is illustrated by a recent national study of internal medicine resident distress which employed the single-item burnout items to uncover differences in burnout across demographic factors, including year of training, sex, medical school location, and amount of educational debt.10

This study does have limitations. First, given the vast literature supporting the validity and reliability of the MBI for the assessment of burnout in medical professionals, the single-item measures of burnout evaluated in this study should not be viewed as substitutes for the full MBI when administration of the longer instrument is possible. Second, response rates in the individual samples ranged from 32–84%,4–9 so that nonresponse bias could occur. The consistency of our findings across the samples suggests that the likely impact of any such bias is small. Third, although the medical student and surgeon samples included in this analysis were derived from multi-center national studies, the internal medicine resident sample reflects a single academic institution. Therefore, the validity characteristics of these items in residents should be further confirmed in additional settings.

In summary, the single questions “I feel burned out from my work” and “I have become more callous toward people since I took this job,” evaluated on the 7-point Likert scale originally developed by Maslach, exhibit strong associations with multiple key published outcomes. These associations are consistent with those reported between these outcomes and the full Maslach Burnout Inventory, providing added support for the utility of these two questions as an abbreviated burnout assessment tool. This may in turn facilitate future studies aimed at understanding the consequences of burnout and inform potential interventions to reduce burnout.