Abstract
Background
Burnout affects nearly half of all U.S. nurses and physicians, and has been linked to poor outcomes such as worse patient safety. The most common measure of burnout is the well-validated Maslach Burnout Inventory (MBI). However, the MBI is proprietary and carries licensing fees, posing challenges to routine or repeated assessment.
Objective
To compare a non-proprietary, single-item burnout measure to a single item from the MBI Emotional Exhaustion (MBI:EE) subscale that has been validated as a standalone burnout measure.
Design
Cross-sectional online survey.
Participants
A sample of primary care providers (PCPs), registered nurses, clinical associates (e.g., licensed practical nurses (LPNs), medical technicians), and administrative clerks in the Veterans Health Administration surveyed in 2012.
Main Methods
We compared a validated one-item version of the MBI:EE and a non-proprietary single-item burnout measure used in the Physician Work Life Study. We calculated kappa statistics, sensitivity and specificity, positive predictive (PPV) and negative predictive values (NPV), and area under the receiver operator curve (AUC). We conducted analyses stratified by occupation to determine the stability of the correlation between the two measures.
Key Results
We analyzed responses from 5,404 participants, including 1,769 providers and 1,380 registered nurses. The prevalence of burnout was 36.7 % as measured on the single MBI:EE item and 38.5 % as measured on the non-proprietary single-item measure. Relative to the MBI:EE, the non-proprietary single-item measure had a correlation of 0.79, sensitivity of 83.2 %, specificity of 87.4 %, and AUC of 0.93 (se = 0.004). Results were similar when stratified by respondent occupation.
Conclusions
A non-proprietary single-item measure served as a reliable substitute for the MBI:EE across occupations. Because it is non-proprietary and easy to interpret, it has logistical advantages over the one-item MBI.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Background
Burnout is an occupational condition characterized by emotional exhaustion, depersonalization, and a low sense of personal accomplishment.1,2 In healthcare, burnout is thought to contribute to poor outcomes, including worse patient safety,3–7 and to lower patient satisfaction.8–10 Burned-out employees are more likely to leave their jobs,1,11–15 take sick leave, and suffer from depression and relationship problems.16–18 Burnout affects nearly half of all U.S. nurses and physicians,1,19 and is significantly more prevalent among physicians than in the general U.S. population.20 For that reason, burnout has been a major topic of health services research in efforts to better understand its causes and solutions.21–25
The most common measure of burnout is the Maslach Burnout Inventory (MBI), a well-validated,26 widely-used self-survey measure.27,28 The MBI comprises three scales: 1) emotional exhaustion (nine items), a state of chronic emotional and physical depletion; 2) depersonalization (five items), a sense of disconnection from coworkers and clients; and 3) diminished personal accomplishment (eight items), a negative sense of self-value and ability.29,30 The MBI has been validated among both physicians31 and nurses,32 and has been found to accurately discriminate between populations of employees who are not suffering from burnout and those clinically diagnosed with burnout.33 While the original MBI Human Services Survey comprises 22 items,34 West and colleagues have validated single items from the MBI emotional exhaustion (MBI:EE) and MBI depersonalization (MBI:DP) subscales as standalone measures.35 They found that, compared to the full scales, single items demonstrated strong psychometric validity, both in terms of meaningfully stratifying risk of high burnout35 and exhibiting strong, consistent associations with outcomes (e.g., suicidality, perceived major medical error, serious thoughts of dropping out of medical school), such that the association between burnout and the outcome was not altered by the use of the single-item MBI versus the full MBI.36 The single-item version of the MBI scales was subsequently used in a major national survey of medical residents.37
The MBI is a proprietary assessment tool and carries licensing fees, making its routine or widespread use, such as in repeated monitoring of employee burnout, potentially cost-prohibitive. An alternative non-proprietary, single-item measure of burnout exists, and has been used successfully in a variety of settings. It was first introduced in a survey of HMO physicians by Schmoldt and colleagues,38 and has also been used in the Physician Work Life Study,39 the Minimizing Error, Maximizing Outcome (MEMO) study,40 and the Mississippi Workforce Study.41 Rohland and colleagues compared this single-item version with the full MBI-HSS in a cohort of 307 Texas medical school graduates,42 and Hansen and Girgis compared the two versions in a survey of 740 Australian oncology health care professionals that included physicians and nurses, among others.43 Despite the differences in sample size and populations, the psychometric findings in these two studies were almost identical: the ANOVA R2 between the MBI:EE subscale and the non-proprietary single-item burnout measure was 0.50 in both, while the simple correlation (r) between the two measures was 0.64 in the study by Rohland and colleagues42 and 0.68 in Hansen and Girgis’ study.43 They both concluded that this single-item measure of burnout could be effectively used as an alternative to the MBI:EE to screen for burnout, especially where emotional exhaustion was the primary subscale of interest and a shorter survey was needed. However, the sample sizes in these studies were small, and to our knowledge, no one has evaluated the performance of this non-proprietary single-item measure of burnout in subsets of U.S. health care workers other than physicians, such as non-physician providers and primary care clinical staff.
Objective
We compared the non-proprietary single-item burnout measure from Rohland and colleagues to a single-item MBI:EE using a large national sample of health care workers from primary care.44 We assessed the performance of the two measures across four distinct occupational classes: providers, registered nurses (RNs), clinical associates (e.g., licensed practical nurses (LPNs), medical technicians), and administrative clerks.
Design
We conducted a cross-sectional survey, administered online. Data were collected during a six-week period in May–June 2012. This work was conducted as part of an operational quality improvement evaluation of VHA primary care. Per VHA Handbook 1058.05, we obtained signed documentation of non-research status from the VA program office overseeing the evaluation (documentation provided upon request.) The survey was voluntary, and we offered no individual-level incentives.
Participants
The target population was all Veterans Health Administration (VHA) primary care personnel, and included respondents from 630 of the 913 primary care clinics operating at the time. We focused our analyses on respondents from the four occupations that comprise primary care teams: providers (physicians, nurse practitioners, and physician assistants), RNs, clinical associates (e.g., LPNs, medical technicians), and administrative clerks. The response rate was approximately 25 %.44
Main Measures
The non-proprietary single-item burnout measure instructs respondents to define burnout for themselves: “Overall, based on your definition of burnout, how would you rate your level of burnout?” Responses are scored on a five-category ordinal scale, where 1 = “I enjoy my work. I have no symptoms of burnout;” 2 = “Occasionally I am under stress, and I don’t always have as much energy as I once did, but I don’t feel burned out;” 3 = “I am definitely burning out and have one or more symptoms of burnout, such as physical and emotional exhaustion;” 4 = “The symptoms of burnout that I’m experiencing won’t go away. I think about frustration at work a lot;” and 5 = “I feel completely burned out and often wonder if I can go on. I am at the point where I may need some changes or may need to seek some sort of help.” This item often is dichotomized as ≤2 (no symptoms of burnout) vs. ≥3 (1 or more symptoms).38,39
The single item from the MBI:EE that West and colleagues validated as a standalone burnout assessment is36: “I feel burned out from my work.” This item appears in the MBI:EE subscale of both the MBI Human Services Survey and MBI General Survey (the two variants of the MBI typically used in health services research), and responses are measured on a seven-point frequency scale ranging from 0 “Never” to 6 “Every day”. They defined “high levels of burnout” as feeling burned out at a frequency of “once a week” or more (a score greater than or equal to 4). This item and two other items from the emotional exhaustion subscale of the MBI-GS were chosen by Leiter and Shaughnessy based on results from a structural equation model in which they minimized correlations among item errors.45 The two other emotional exhaustion items were: “I feel tired when I get up in the morning and have to face another day on the job,” and “Working all day is really a strain for me.” In sensitivity analyses, we also compared the single-item burnout measure to Leiter and Shaughnessy’s three-item MBI:EE.
Respondent Characteristics
The survey also collected respondent characteristics, including tenure with the VA (eight categories), supervisory level (six categories), age (six categories), sex, race (six categories), Latino ethnicity (dichotomous), and occupation (we used four categories).
Analytical Procedure
In order to compare the performance of the non-proprietary single-item measure to the single MBI:EE item, we examined the Pearson correlation between the original ordinal scale values of the two measures as well as several measures of agreement and discrimination between the dichotomized measures (i.e., based on a positive screen for burnout): kappa values, sensitivity and specificity, and positive and negative predictive values (PPV & NPV). We also conducted a receiver operator curve analysis to calculate the area under the curve (AUC). Analyses were conducted with all respondents combined and separately within each of the four occupations: PCP, RN, clinical associate, and clerk. We calculated all statistics and conducted analyses using Stata (SE) version 12 (StataCorp., College Station, TX, USA).
Key Results
Our analysis included 5,404 respondents (1,769 providers, 1,380 RNs, 1,358 clinical associates, 557 clerks). Fifty-two percent were 50 years of age or older, 38 % had been with the VA more than 10 years, and 39 % had some supervisory responsibilities (Table 1).
The prevalence of burnout was 36.7 % as measured on the single MBI:EE item and 38.5 % as measured on the non-proprietary single-item measure (Table 2). Burnout varied substantially by occupation, and was highest for providers and lowest for clinical associates. Results from the measures of agreement are presented in Table 3. The Pearson correlation between the two burnout measures was 0.79. A full cross-tabulation of the frequencies for the two single-item burnout measures is included in Table 4. In a comparison of measures of agreement on whether the respondent was burned out, the inter-rater agreement (kappa) was 0.70, indicating 70 % greater agreement between the two measures than by chance alone. Compared to the single MBI:EE item, the non-proprietary single-item measure had sensitivity of 83.2 %, specificity of 87.4 %, PPV of 79.3 %, and NPV of 90.0 %. The AUC was 0.93 (se = 0.004). When we assessed discrimination statistics separately by respondent occupation, we found similar results across occupations. We also repeated analyses with the three-item version of the MBI:EE used by Leiter and Shaughnessy,45 and found similar results (electronic supplementary material, available online).
Conclusions
We found that a non-proprietary single-item measure of burnout was a viable substitute for the one-item MBI:EE validated by West and colleagues, with a high Pearson correlation and area under the curve. We compared the burnout measures among four very different occupational classes (providers, RNs, clinical associates, and administrative clerks), with significant differences in prevalence of burnout, and found remarkably consistent results. The single-item measure has the important advantage of carrying no licensing fee, which for the MBI was $2.00 per use for small-volume administration at the time of this paper. The single-item measure is also easy to interpret, with the response scale explicitly indicating where a change in values signals symptoms of burnout (3 = “I am definitely burning out”) versus no burnout (2 = “don’t feel burned out”).
Our findings are similar to two previous, smaller studies that compared the single-item burnout measure to the full MBI:EE in a sample of physician alumni from a single institution42 and in a sample of Australian cancer care workers.43 In the latter, the investigators found a Pearson correlation of 0.66 between the two versions, with the MBI:EE producing a burnout prevalence of 32.0, vs. 28.2 % for the non-proprietary single-item measure. In other analyses, we identified elements of medical home team-based care that were associated with burnout, using both the non-proprietary single-item measure and the three-item MBI:EE, and found that the factors associated with each measure were virtually identical,44 suggesting that small absolute differences between the two measures in the prevalence of burnout would not likely bias studies of factors contributing to burnout.
Limitations
The present study has three notable limitations. First, we compared the single-item measure to a one-item MBI:EE,36 not the full MBI:EE, and it is possible that different results would be achieved with the full MBI:EE. We did not have the full MBI:EE available in this dataset, but did have a three-item MBI:EE (that included the one MBI:EE item tested), and using it produced very similar results.
Second, because we were comparing single-item measures, we could not test internal consistency reliability, which requires scales to have three or more items to assess. Until test–retest analyses are conducted, the reliability of these items will not be known. However, the primary threat from poor reliability is that it introduces measurement error, biasing our results towards the null (i.e., toward lower correlation between the measures).
Finally, because the surveys had to be anonymous, our sampling frame was estimated, and it is impossible to know for certain who participated in the survey and, consequently, what the true response rate was. We estimated the response rate at 25 % based on administrative data.44 There may have been important differences between those who participated and those who did not, which could have introduced selection bias. There may be concerns that our results do not generalize beyond this sample. However, the demographics of our respondents were very similar to those of primary care participants in the VA All Employee Survey, which was fielded the month prior, that was sent to all VA employees and achieved a 62 % response rate.44 Our sample was also representative of primary care clinics, with respondents from 69 % of the 913 VHA primary care clinics, including sites in all 50 states. This gives us added confidence that our findings are generalizable to the VA primary care population, and possibly to other primary care employees.
Overall, we found that a non-proprietary single-item measure of burnout could serve as a valid substitute for the MBI:EE. Because it is not licensed, and its scores are easy to interpret, this assessment tool has important logistical advantages over the single-item MBI:EE.
References
Shanafelt TD, Boone S, Tan L, et al. Burnout and satisfaction with work-life balance among US physicians relative to the general US population. Arch Intern Med. 2012;172(18):1377–85.
Aiken LH, Clarke SP, Sloane DM, Sochalski J, Silber JH. Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. JAMA. 2002;288(16):1987–93.
Cimiotti JP, Aiken LH, Sloane DM, Wu ES. Nurse staffing, burnout, and health care-associated infection. Am J Infect Control. 2012;40(6):486–90.
Halbesleben JR, Wakefield BJ, Wakefield DS, Cooper LB. Nurse burnout and patient safety outcomes: nurse safety perception versus reporting behavior. West J Nurs Res. 2008;30(5):560–77.
Spence Laschinger HK, Leiter MP. The impact of nursing work environments on patient safety outcomes: the mediating role of burnout/engagement. J Nurs Adm. 2006;36(5):259–67.
West CP, Tan AD, Habermann TM, Sloan JA, Shanafelt TD. Association of resident fatigue and distress with perceived medical errors. JAMA. 2009;302(12):1294–300.
Shanafelt TD, Balch CM, Bechamps G, et al. Burnout and medical errors among American surgeons. Ann Surg. 2010;251(6):995–1000.
Haas JS, Cook EF, Puopolo AL, Burstin HR, Cleary PD, Brennan TA. Is the professional satisfaction of general internists associated with patient satisfaction? J Gen Intern Med. 2000;15(2):122–8.
Halbesleben JR, Rathert C. Linking physician burnout and patient outcomes: exploring the dyadic relationship between physicians and patients. Health Care Manag Rev. 2008;33(1):29–39.
Argentero P, Dell’Olivo B, Ferretti MS. Staff burnout and patient satisfaction with the quality of dialysis care. Am J Kidney Dis. 2008;51(1):80–92.
Shanafelt T, Sloan J, Satele D, Balch C. Why do surgeons consider leaving practice? J Am Coll Surg. 2011;212(3):421–2.
Van Bogaert P, Clarke S, Roelant E, Meulemans H, Van de Heyning P. Impacts of unit-level nurse practice environment and burnout on nurse-reported outcomes: a multilevel modelling approach. J Clin Nurs. 2010;19(11–12):1664–74.
Linzer M, Manwell LB, Williams ES, et al. Working conditions in primary care: physician reactions and care quality. Ann Intern Med. 2009;151(1):28–36. W26-29.
Landon BE, Reschovsky JD, Pham HH, Blumenthal D. Leaving medicine: the consequences of physician dissatisfaction. Med Care. 2006;44(3):234–42.
Hinami K, Whelan CT, Wolosin RJ, Miller JA, Wetterneck TB. Worklife and satisfaction of hospitalists: toward flourishing careers. J Gen Intern Med. 2012;27(1):28–36.
Shanafelt TD, Bradley KA, Wipf JE, Back AL. Burnout and self-reported patient care in an internal medicine residency program. Ann Intern Med. 2002;136(5):358–67.
Parker PA, Kulik JA. Burnout, self- and supervisor-rated job performance, and absenteeism among nurses. J Behav Med. 1995;18(6):581–99.
Sargent MC, Sotile W, Sotile MO, Rubash H, Barrack RL. Quality of life during orthopaedic training and academic practice. Part 1: orthopaedic surgery residents and faculty. J Bone Joint Surg Am. 2009;91(10):2395–405.
Aiken LH, Clarke SP, Sloane DM, et al. Nurses’ reports on hospital care in five countries. Health Aff. 2001;20(3):43–53.
Shanafelt TD, Boone S, Tan L, et al. Burnout and satisfaction with work-life balance among us physicians relative to the general us population. Arch Intern Med. 2012:1–9.
Spence Laschinger HK, Leiter M, Day A, Gilin D. Workplace empowerment, incivility, and burnout: impact on staff nurse recruitment and retention outcomes. J Nurs Manag. 2009;17(3):302–11.
Houkes I, Winants Y, Twellaar M, Verdonk P. Development of burnout over time and the causal order of the three dimensions of burnout among male and female GPs. A three-wave panel study. BMC Public Health. 2011;11(1):240.
Friedberg MW. The potential impact of the medical home on job satisfaction in primary care. Arch Intern Med. 2012;172(1):31–2.
Dyrbye LN, Varkey P, Boone SL, Satele DV, Sloan JA, Shanafelt TD. Physician satisfaction and burnout at different career stages. Mayo Clin Proc. 2013;88(12):1358–67.
Linzer M, Levine R, Meltzer D, Poplau S, Warde C, West CP. 10 bold steps to prevent burnout in general internal medicine. J Gen Intern Med. 2014;29(1):18–20.
Worley JA, Vassar M, Wheeler DL, Barnes LLB. Factor structure of scores from the maslach burnout inventory: a review and meta-analysis of 45 exploratory and confirmatory factor-analytic studies. Educ Psychol Meas. 2008;68(5):797–823.
Schaufeli W, Enzmann D, Girault N. Measurement of burnout: A review. Prof Burnout: Recent Dev Theory Res. 1993:199–215.
Wheeler DL, Vassar M, Worley JA, Barnes LL. A reliability generalization meta-analysis of coefficient alpha for the Maslach burnout inventory. Educ Psychol Meas. 2011;71(1):231–44.
Cropanzano R, Rupp DE, Byrne ZS. The relationship of emotional exhaustion to work attitudes, job performance, and organizational citizenship behaviors. J Appl Psychol. 2003;88(1):160.
Maslach C, Jackson SE. The measurement of experienced burnout. J Organ Behav. 1981;2(2):99–113.
Rafferty JP, Lemkau JP, Purdy RR, Rudisill JR. Validity of the Maslach burnout inventory for family practice physicians. J Clin Psychol. 1986;42(3):488–92.
Poghosyan L, Aiken LH, Sloane DM. Factor structure of the Maslach burnout inventory: An analysis of data from large scale cross-sectional surveys of nurses from eight countries. Int J Nurs Stud. 2009;46(7):894–902.
Schaufeli W, Bakker A, Hoogduin K, Schaap C, Kladler A. On the clinical validity of the Maslach Burnout Inventory and the Burnout Measure. Psychol Health. 2001;16(5):565–82.
Maslach C, Jackson SE, Leiter MP. Maslach Burnout Inventory Manual, Third Edition. Third ed: Mind Garden; 1996.
West C, Dyrbye L, Sloan J, Shanafelt T. Single item measures of emotional exhaustion and depersonalization are useful for assessing burnout in medical professionals. J Gen Intern Med. 2009;24(12):1318–21.
West CP, Dyrbye LN, Satele DV, Sloan JA, Shanafelt TD. Concurrent validity of single-item measures of emotional exhaustion and depersonalization in burnout assessment. J Gen Intern Med. 2012;27(11):1445–52.
West CP, Shanafelt TD, Kolars JC. Quality of life, burnout, educational debt, and medical knowledge among internal medicine residents. JAMA. 2011;306(9):952–60.
Schmoldt RA, Freeborn DK, Klevit HD. Physician burnout: recommendations for HMO managers. HMO Pract. 1994;8(2):58–63.
McMurray JE, Linzer M, Konrad TR, Douglas J, Shugerman R, Nelson K. The work lives of women physicians results from the physician work life study. The SGIM career satisfaction study group. J Gen Intern Med. 2000;15(6):372–80.
Williams ES, Manwell LB, Konrad TR, Linzer M. The relationship of organizational culture, stress, satisfaction, and burnout with physician-reported error and suboptimal patient care: results from the MEMO study. Health Care Manag Rev. 2007;32(3):203–12.
Cossman JS, Street D. Mississippi burnout. Part I: personal characteristics and practice context. J Miss State Med Assoc. 2009;50(9):306–10.
Rohland BM, Kruse GR, Rohrer JE. Validation of a single-item measure of burnout against the Maslach Burnout inventory among physicians. Stress Health. 2004;20(2):75–9.
Hansen V, Girgis A. Can a single question effectively screen for burnout in Australian cancer care workers? BMC Health Serv Res. 2010;10(1):341.
Helfrich C, Dolan E, Simonetti J, et al. Elements of team-based care in a patient-centered medical home are associated with lower burnout among VA primary care employees. J Gen Intern Med. 2014;29(2):659–66.
Leiter M, Shaughnessy K. The areas of worklife model of burnout: tests of mediation relationships. Ergonomia Int J. 2006;28:327–41.
Acknowledgments
Our thanks to Julie Kurutz and John Witzlib of the VA Healthcare Analysis and Information Group (HAIG) for fielding the survey; to John Messina for assistance with communication, scheduling, and data tables; and to members of the PACT Demonstration Laboratory Coordinating Center Organizational Function Working Group, who contributed invaluable expertise and feedback.
This work was undertaken as part of the VA’s PACT Demonstration Laboratory initiative, supporting and evaluating the VA’s transition to a patient-centered medical home model. Funding for the PACT Demonstration Laboratory initiative is provided by the VA Office of Patient Care Services. The views expressed in this article are those of the authors, and do not necessarily reflect the position or policy of the Department of Veterans Affairs.
Conflict of interest
The authors each declare that they have no conflict of interest.
Author information
Authors and Affiliations
Corresponding author
Additional information
The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States government.
Electronic supplementary material
Below is the link to the electronic supplementary material.
ESM 1
(DOCX 34 kb)
Rights and permissions
About this article
Cite this article
Dolan, E.D., Mohr, D., Lempa, M. et al. Using a Single Item to Measure Burnout in Primary Care Staff: A Psychometric Evaluation. J GEN INTERN MED 30, 582–587 (2015). https://doi.org/10.1007/s11606-014-3112-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11606-014-3112-6