Abstract
BACKGROUND
Medical knowledge and clinical performance ratings are major criteria for assessing the competence of resident physicians. However, these assessments may be influenced by residents’ mental health. The relationship between residents’ well-being and empathy and assessments of their global performance remains unclear.
OBJECTIVE
To determine whether resident well-being and empathy are associated with assessments of their medical knowledge and clinical performance.
DESIGN, SETTING, AND PARTICIPANTS
We studied 730 clinical performance assessments completed by peers, supervising residents, and allied health professionals; 193 mini-clinical evaluation exercise (mini-CEX) evaluations; and 260 in-training examinations (ITE) of Mayo Clinic internal medicine residents between January 2009 and August 2010. Multivariate generalized estimating equations were used to evaluate associations between residents’ well-being and empathy and assessments of their knowledge and clinical performance.
MEASUREMENTS
Independent variables were empathy using the Interpersonal Reactivity Index (IRI), burnout using the Maslach Burnout Inventory, depression using a standardized two-question screening instrument, and quality of life using a Linear Analog Self-Assessment item and the Medical Outcomes Study Short Form (SF-8). Dependent variables were mini-CEX, ITE, and the validated six-item Mayo clinical performance assessment.
RESULTS
202 residents (64.7%) provided both well-being and at least one category of assessment data. In multivariate models, residents’ scores on the IRI empathy measure of “the tendency to adopt the psychological view of others” were associated with higher peer ratings on “desirability as a physician for a family member” (beta = 0.023, 95% CI = 0.007–0.039, p = 0.004). Additionally, burnout was associated with higher supervisor ratings of communication (beta = 0.309, 95% CI = 0.100–0.517, p = 0.004). There were no observed associations between ITE or mini-CEX scores and resident quality of life, burnout, fatigue, depression, or empathy.
CONCLUSIONS
Most dimensions of resident well-being were not associated with residents’ knowledge scores and assessments of their clinical performance by other members of the health care team, which supports the trustworthiness of these measures. Nonetheless, correlations of resident empathy and burnout with assessments completed by peers and supervising residents suggest that some ratings of residents may be influenced by interpersonal factors.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
BACKGROUND
Medical knowledge and clinical performance ratings are major criteria for assessing the competence of resident physicians,1 so it is essential that these standards are trustworthy. Assessments of medical knowledge and clinical performance should primarily reflect residents’ abilities to care for patients, but these assessments may be influenced by other factors. For example, research has shown that resident empathy affects assessments of faculty members,2 female teachers receive lower ratings than males with similar skills,3 and assessment scores for a single instrument may vary substantially between learning environments within the same institution.4
It has been theorized that physician health and well-being could have an impact on medical knowledge acquisition.5 While some studies have linked resident well-being and medical knowledge,6,7 other research has failed to demonstrate this relationship.8–10 It is unknown whether resident well-being shapes overall assessments of residents’ competency.
Research has shown that internal medicine residents’ well-being is affected by features of the work environment such as duty-hours11 and the perception of experiencing medical errors.12 Because it is known that learning environments11,12 and interpersonal relationships13–16 play crucial roles in learning, we postulated that residents’ well-being and empathy would influence assessments of their medical knowledge and clinical performance.
Studies have indicated that well-being affects residents’ attitudes towards patients and the quality of care that they provide,11,12,17 but no studies have examined interactions between residents’ well-being and the global assessments they receive from other members of the health care team. Therefore, we used a prospective longitudinal study design to investigate the hypothesis that resident well-being (measured by the Maslach Burnout Inventory, Linear Analog Self-Assessment item, Medical Outcomes Study Short Form health survey, and a standardized two-question depression screen) and empathy (measured by the Interpersonal Reactivity Index) are associated with numerous dimensions of competency including assessments of knowledge on the medical in-training examination (ITE), clinical performance on the mini-clinical evaluation exercise (mini-CEX), and multi-source assessments by peers, supervising residents, and allied health professionals.
METHODS
Learning Environment and Participants
This study involved Mayo internal medicine residents in training between January 2009 and August 2010, and included scores from all knowledge and clinical performance assessments performed on the 264 resident physicians enrolled during this time period in the Mayo Clinic Internal Medicine Well-being (IMWELL) Study, which is described below. This study was approved by the Mayo Institutional Review Board.
The Mayo IMWELL Study
Resident characteristics were obtained from the Mayo Clinic-Rochester IMWELL study, a longitudinal study of resident physician well-being. Since the 2003 academic year, all categorical and preliminary residents in the Mayo Clinic Rochester Internal Medicine Residency program have been invited to participate in the IMWELL study during their first-year orientation. For the time period of this study, 264 of 312 (84.6%) eligible residents volunteered to participate. All residents provided written consent and were surveyed at regular intervals throughout their residency training. An instrument that measures quality of life (QOL) was administered quarterly, and instruments that measure burnout, empathy, and depression were administered biannually. To maintain anonymity, study participants’ identities were blinded by using numerical codes during data collection and analyses.
Instruments Comprising the Mayo IMWELL Study
The IMWELL study utilized a linear analog self-assessment (LASA) scale of QOL and survey items from the Maslach Burnout Inventory (MBI), Interpersonal Reactivity Index (IRI), Medical Outcomes Study Short Form (SF-8) health survey, and a depression screen by Spitzer et al. Notably, these instruments are supported by sources of validity evidence18,19 that are essential to medical education studies.20,21
The LASA for measuring QOL is a single item with scores ranging from 0 (as bad as it can be) to 10 (as good as it can be). LASA QOL scores have been validated in varied populations including the general public,22 cancer patients,23,24 and physicians.25
The MBI is a 22-item instrument with Likert scales ranging from 0 (never) to 6 (daily).26 Content validity has been demonstrated by reviewing established scales and surveying professionals who are at risk of experiencing burnout, including physicians.26 Many studies have shown that the MBI is an effective measure of burnout in resident physicians.12,27–30 Factor analysis demonstrated that the MBI consists of three dimensions: depersonalization, emotional exhaustion, and sense of low personal accomplishment.26 As has been done in previous studies of physicians, we considered residents with a high score on either the depersonalization or emotional exhaustion subscale as having at least one manifestation of professional burnout.25,27 Additionally, the MBI has high internal consistency, acceptable test-retest reliability, moderate correlation with other measures of burnout, and poor correlation with constructs that are likely confounded with burnout.26
The IRI is a 28-item instrument for measuring empathy with Likert scales ranging from 0 (does not describe me well) to 4 (describes me well). Factor analysis revealed four dimensions: perspective-taking, personal distress, empathic concern, and fantasy.31 These seven-item subscales correspond to the two-dimensional cognitive and emotive model, and may be evaluated separately.31,32 A model focusing on empathic concern and perspective-taking has proven especially useful when evaluating empathy among resident physicians.17,30,33,34 On this basis, these two subscales were assessed in this study. Additional validity evidence for the IRI includes good internal consistency,32,35,36 and significant correlations between IRI subscales and other recognized measures of empathy.36,37
The Medical Outcomes Study Short Form (SF-8) health survey has eight items with 5- and 6-point Likert scales.38 Content validity is supported in that survey items represent ideas that are commonly included in widely used health measures.38 The SF-8 generates scores that are assigned to domains of mental and physical health.38 Further SF-8 validity evidence includes score reliability, high correlation with existing measures of the same concepts,38 and convergence between measurements of patients with migraine and those with other conditions.39
Spitzer et al. described a depression screening method consisting of two questions: “During the past month, have you often been bothered by feeling down, depressed and hopeless?” and “During the past month, have you often been bothered by little interest or pleasure in doing things?”40 These two questions perform well in screening for depression relative to several widely used depression inventories, including the Beck Depression Inventory, the Center for Epidemiological Studies Depression Scale, and the Medical Outcomes Study depression measure.41 Spitzer et al.’s screening questions have been used to identify depression in various populations, including resident physicians.11,27
Outcome Measures
Strong validity evidence supports the outcome measures used in this study. ITE scores, the validity and use of which are well described,42,43 have been shown to correlate strongly with resident conference attendance and self-directed reading,44,45 and to have no association with resident physician empathy.10 The mini-CEX has impressive validity and reliability as demonstrated by previous studies at the Mayo Clinic46,47 and elsewhere.48–52 Our institution uses the traditional version of the mini-CEX, except that the items are on 5-point scales.
Clinical performance assessments of resident physicians at the Mayo Clinic are comprised of forms completed by peers, senior medical residents, and non-physician professionals. After critically analyzing the items for content most plausibly related to resident well-being, the following items (scale: 1 = needs improvement, 3 = average, 5 = top 10%) were chosen for this study: (1) desirability as a physician for one of your family members, (2) desirability as a future co-worker or team member, (3) effectiveness and completeness of sign-outs, (4) coverage of cross-cover issues and completeness of tasks when on call, (5) demonstrates empathy and compassion for patients, and (6) communication skills with patients, family, allied health, and other providers. Content validity for these clinical performance assessments is based on assessment elements that are represented in previously published instruments and were selected by experts with experience in scale design. A factor analytic study revealed that several items within these Mayo clinical performance assessments are multi-dimensional and have excellent internal consistency reliability.53
Data Analysis
A repeated measures design, analyzed using multivariate generalized estimating equations, was employed to evaluate associations between resident clinical performance assessments, mini-CEX evaluations and ITE examinations, and residents’ QOL, burnout, empathy, and depression over the 4 points in time. Scores from the clinical performance assessment items were evaluated individually and also averaged within assessor group to form an overall score ranging from 1 to 5. Covariates included resident well-being (QOL, burnout, depression), empathy, gender, year of training, program (categorical or preliminary), debt, relationship status (single, married, divorced, partner), and children (yes or no). Univariate associations were examined, and a multivariate model was developed using standard forward and backward stepwise selection techniques. The threshold for statistical significance was set at p < 0.01 to account for multiple comparisons. The study sample of 202 residents provided 80% power for a medium-to-small Cohen’s f2 effect size of 0.04 for a univariate association between clinical performance assessment scores, mini-CEX evaluations, and ITE examinations, and any well-being or empathy variable. Statistical analyses were conducted using SAS version 9.1 (SAS Institute Inc., Cary, NC).
RESULTS
We studied 730 clinical performance assessments completed by peers, supervisors, and allied health professionals for Mayo Clinic internal medicine residents in January 2009, August 2009, January 2010, and August 2010. We also examined 193 mini-clinical evaluation exercise (mini-CEX) evaluations and 260 in-training examinations (ITE) during the same time frame. Of 312 eligible residents, 202 (64.7% of all eligible) provided well-being and at least one category of assessment data for this study. Demographic characteristics for this sample are shown in Table 1. Data for responders were similar to those of non-responders on measured factors including year of training, age, sex, and program type (categorical or preliminary). In addition, United States Medical Licensing Examination Step 1 and Step 2 Clinical Knowledge (CK) scores were similar for responders and non-responders (Step 1 mean scores 232.3 vs 231.2, respectively, p = 0.74; Step 2 CK mean scores 241.3 vs 236.1, respectively, p = 0.10). As the overall results did not differ for categorical and preliminary residents, data were pooled across these categories. Baseline well-being, empathy, assessment, mini-CEX, and ITE scores at the start of the current study are shown in Table 2. In addition, response rates for each outcome are detailed in Table 2.
Univariate associations of measures of well-being and empathy with summary scores in each resident performance assessment domain are shown in an on-line appendix. In multivariate models, there were no statistically significant associations between resident ITE or mini-CEX scores and QOL, burnout, depression, empathy, or demographic characteristics. Both forward and backward model selection approaches yielded the same results. However, residents’ scores on the IRI Perspective Taking scale, a measure of “the tendency to adopt the psychological view of others,” were associated with higher peer ratings on “desirability as a physician for a family member” (multivariate beta = 0.023, 95% CI = 0.007–0.039, p = 0.004). Consequently, a 5-point increase in this empathy score was associated with a small but statistically significant 0.12-point increase in residents’ ratings by their peers as desirable physicians, as shown in Table 3. Additionally, having at least one manifestation of professional burnout was associated with higher resident supervisor ratings of communication with patients, families, allied health, and other providers (multivariate beta = 0.309, 95% CI = 0.100–0.517, p = 0.004). Hence, burnout was associated with a 0.3-point increase in resident communication score, as shown in Table 3.
DISCUSSION
To our awareness, this is the first study to show that multiple dimensions of resident performance assessment are not significantly influenced by various aspects of well-being including QOL and depression. Nonetheless, the detected correlations between resident empathy and burnout and assessments completed by other doctors suggest that observation-based ratings of residents may be influenced by both intrapersonal and interpersonal factors.
Resident well-being has received national attention because the Accreditation Council for Graduate Medical Education has recently mandated further restrictions on resident duty hours,54 based on the assumption that these restrictions will enhance patient care by improving resident fatigue and well-being. Consequently, it is important to examine the potential relationships between resident well-being and assessments of resident competency. Furthermore, many residency programs assess resident performance on tests of knowledge and clinical skill using the ITE and mini-CEX, respectively. Thus, it is encouraging that we identified no associations between resident well-being and performance on the ITE and mini-CEX. Our findings build upon research that has demonstrated validity of these measures,43,46–52 while underscoring the fact that these measures should not be viewed as reliable means for detecting variations in resident empathy and well-being. Our findings also suggest that increasing resident well-being through duty hour restrictions may not be a comprehensive, stand-alone strategy for improving resident performance and enhancing patient care.
It has been theorized that resident performance is partly determined by depression, burnout, and distress related to the work environment.5 Girard et al. found that American Board of Internal Medicine (ABIM) Certification Examination score variation among internal medicine residents was largely attributable to residents’ psychological states.6 Filho et al. have shown that anesthesiology residents’ knowledge of basic science was associated with academic performance anxiety, but was not associated with QOL.9 However, other research has identified no relationship between resident well-being and knowledge on specific topics or standardized examinations.8 Likewise, the current study found no association between resident well-being and medical knowledge on the ITE. A potential explanation for this lack of association is that an insufficient number of residents in our study sample displayed the extremes of low well-being that would be required to negatively impact on the acquisition or display of medical knowledge; indeed, West et al. identified an association between resident well-being and medical knowledge in a much larger sample with wider ranges in well-being scores.7 Another explanation for our negative findings may be that the ITE—as opposed to the ABIM certification—is a lower stakes examination and intended only for formative feedback. This may be the reason why Girard et al. identified a relationship between resident mood disorders and score variation in the ABIM certification examination.6
We found an association between resident empathy and desirability as a physician. In particular, residents who scored high on the IRI Perspective-Taking scale, which is “the tendency to adopt the psychological views of others,” were considered to be desirable physicians for a family member. It is common wisdom that to be entrusted with the care of a colleague’s loved one is the highest compliment. This finding also reflects two core virtues of medicine, which include empathy and compassion.55,56 Additionally, this finding extends previous studies that have shown relationships between scores on the IRI Perspective-Taking scale and higher performance among medical students,57 as well as decreased likelihood of medical errors by residents.12
We observed that supervising residents perceived interns with higher burnout to have better communication with patients, families, allied health, and other providers. Although this finding could seem counterintuitive, many physicians who experience burnout may sustain high levels of professional achievement for long durations. Furthermore, the most dedicated physicians might be more likely to place professional duties—including the time-consuming task of effectively communicating with patients, family members, support staff, and colleagues—above all other aspects of personal life.58 Therefore, such physicians could be viewed favorably by supervisors in the workplace, even though the personal aspects of these physicians’ lives may suffer. Ultimately, this finding should prompt residency programs to reflect on the optimal balance between patient care responsibilities and resident burnout.
This study has limitations. It involved participation by only 64.7% of eligible residents. Of the 264 of 312 (84.6%) residents volunteering for the IMWELL study, there were additional missing data due to incomplete surveys, missing evaluation measures at the same point in time as well-being assessment, and the fact that not all rotations have clinical evaluations. Nonetheless, the participation rate in this study was favorable relative to that typically seen in physician studies.59,60 Our study sample was largely comprised of first-year residents and resident work-load—which may influence well-being—varies by PGY year; yet, our study results were adjusted for PGY level in the multivariate analyses, and none of the results were significantly different for the various PGY levels. Since this was a single-institution study, one should generalize the findings to other settings with caution. However, the range of well-being and empathy scores in this study is similar to those reported in previous studies at other institutions.28,33,34 Furthermore, the outcomes variables in this study for medical knowledge (ITE) and clinical skill (mini-CEX) are widely used among US residency programs, which should broaden the importance and relevance of the study findings. While the results may have been affected by non-response bias, data for responders were similar to those of non-responders on measured demographic factors and USMLE scores. The small but statistically significant associations between resident empathy and burnout and assessments by other physicians might be viewed as clinically insignificant; however, the actual range of scores among residents at the Mayo Clinic for communications competency is very narrow, so the observed 0.3-point change in communication score with the presence of burnout (yes/no) could have a substantial impact on a Mayo resident’s relative standing within this competency. Additionally, the positive associations between resident empathy and burnout and assessments by other physicians persisted after multivariate adjustments that incorporated a number of potential confounding covariates, so we believe that these findings are educationally relevant and add new knowledge regarding the relationships between resident empathy and desirability as a physician, as well as between resident burnout and communication. Finally, we acknowledge that this study did not examine the influence of several potential confounders, including residents’ learning styles, personality types, and major life events.
In this study sample, multiple dimensions of resident performance were generally not influenced by various aspects of well-being, which lends credibility to standardized measures of knowledge (ITE) and clinical performance (mini-CEX). However, the lack of association between well-being and medical knowledge requires further study involving high-stakes examinations. The association between empathy and desirability as a doctor for a family member—which is widely accepted as a characteristic of excellence among physicians—suggests the need to emphasize the identification and promotion of empathy in medical learners. However, the positive association between burnout and the perception of excellent clinical performance should stimulate discussion about the best ways to engage residents in meaningful clinical experiences without compromising their overall well-being.
REFERENCES
ACGME. ACGME outcome project. 2004; www.acgme.org/outcome/.
Beckman TJ, Reed DA, Shanafelt TD, West CP. Impact of resident well-being and empathy on assessments of faculty physicians. J Gen Intern Med. 2010;25:52–6.
Beckman TJ, Mandrekar JN. The interpersonal, cognitive and efficiency domains of clinical teaching: construct validity of a multidimensional scale. Med Educ. 2005;39:1221–9.
Beckman TJ, Cook DA, Mandrekar JN. Factor instability of clinical teaching assessment scores among general internists and cardiologists. Med Educ. 2006;40:1209–16.
Mitchell M, Srinivasan M, West DC, et al. Factors affecting resident performance: development of a theoretical model and a focused literature review. Acad Med. 2005;80:376–9.
Girard DE, Hickam DH. Predictors of clinical performance among internal medicine residents. J Gen Intern Med. 1991;6:150–4.
West C, Shanafelt TD, Kolars JC. Quality of life, burnout, educational debt, and medical knowledge among internal medicine residents. JAMA. 2011;306:952–60.
West CP, Shanafelt TD, Cook DA. Lack of association between resident doctors' well-being and medical knowledge. Med Educ. 2010;44:1224–31.
de Oliveira Filho GR, Vieira JE. The relationship of learning environment, quality of life, and study strategies measures to anaesthesiology resident academic performance. Anesth Analg. 2007;104:1467–72.
West CP, Huntington JL, Huschka MM, et al. A prospective study of the relationship between medical knowledge and professionalism among internal medicine residents. Acad Med. 2007;82:587–92.
Goitein L, Shanafelt TD, Wipf JE, Slatore CG, Back AL. The effects of work-hour limitations on resident well-being, patient care, and education in an internal medicine residency program. Arch Intern Med. 2005;165:2601–6.
West CP, Huschka MM, Novotny PJ, et al. Association of perceived medical errors with resident distress and empathy. JAMA. 2006;296:1071–8.
Beckman TJ, Lee MC. Proposal for a collaborative approach to clinical teaching. Mayo Clin Proc. 2009;84:339–44.
Williams GC, Wiener MW, Markakis KM, Reeve JM, Deci EL. Medical students' motivation for internal medicine. J Gen Intern Med. 1994;9:327–33.
Wortham S. The interdependence of social identification and learning. Am Educ Res J. 2004;41:715–50.
Haidet P, Stein HF. The role of the student-teacher relationship in the formation of physicians. The hidden curriculum as process. J Gen Intern Med. 2006;21:S16–20.
Shanafelt TD, West C, Zhao X, et al. Relationship between increased personal well-being and enhanced empathy among internal medicine residents. J Gen Intern Med. 2005;20:559–64.
Messick S. Validity of psychological assessment. Validation of inferences from persons' responses and performances as scientific inquirey into score meaning. Am Psychol. 1995;50:741–9.
Downing SM. Validity: on the meaningful interpretation of assessment data. Medical Education. 2003;37:830–7.
Beckman TJ, Cook DA, Mandrekar JN. What is the validity evidence for assessments of clinical teaching? J Gen Intern Med. 2005;20:1159–64.
Beckman TJ, Ghosh AK, Cook DA, Erwin PJ, Mandrekar JN. How reliable are assessments of clinical teaching? A review of the published instruments. J Gen Intern Med. 2004;19:971–7.
Gudex C, Dolan P, Kind P, Williams A. Health state valuations from the general public using the visual analog scale. Qual Life Res. 1996;5:521–31.
Spitzer WO, Dobson AJ, Hall J, et al. Measuring the quality of life in cancer patients: a concise QL-index for use by physicians. J Chron Dis. 1981;34:585–97.
Rummans TA, Clark MM, Sloan JA, et al. Impacting quality of life for patients with advanced cancer with a structured multidisciplinary intervention: a randomized controlled trial. J Clin Oncol. 2006;24:635–42.
Shanafelt TD, Novotny P, Johnson ME, et al. The well-being and personal wellness promotion strategies of medical oncologists in the North Central Cancer Treatment Group. Oncology. 2005;68:23–32.
Maslach C, Jackson SE, Leiter MP. Maslach Burnout Inventory Manual (3 rd ed). Palo Alto, CA: Consulting Psychologists Press; 1996.
Shanafelt TD, Bradley KA, Wipf JE, Back AL. Burn-out and self-reported patient care in an internal medicicine residency program. Ann Intern Med. 2002;136:358–67.
Thomas NK. Resident burnout. JAMA. 2004;292:2880–9.
Gopal R, Glasheen JJ, Miyoshi TJ, Prochazka AV. Burnout and internal medicine resident work-hour restrictions. Arch Intern Med. 2005;165:2595–600.
Rosen IM, Gimotty PA, Shea JA, Bellini LM. Evolotion of sleep quality, sleep deprevation, mood disturbances, empathy, and burnout among interns. Acad Med. 2006;81:82–5.
Davis M. A multidimensional approach to individual differences in empathy. JSAS Catalog Selected Documents Psychol. 1980;10:85.
Davis M. Measuring individual differences in empathy: evidence for a multi-dimensional approach. J Pers Soc Psychol. 1983;44:113–26.
Bellini LM, Baime M, Shea JA. Variation of mood and empathy during internship. JAMA. 2002;287:3143–6.
Bellini LM, Shea JA. Mood change and empathy decline persist during three years of internal medicine training. Acad Med. 2005;80:164–7.
Yarnold PR, Bryant FB, Nightingale SD, Martin GJ. Assessing physician empathy using the Interpersonal Reactivity Index: A measurement model and cross-sectional analysis. Psychology, Health, and Medicine. 1996;1:207–21.
Bryant FB, Yarnold PR, Grimm LG. Toward a measurement model of the Affect Intensity Measure: a three-factor structure. Journal of Research in Personality. 1996;30:223–47.
Lawrence EJ, Shaw P, Baker D, Baron-Cohen S, David AS. Measuring empathy: reliability and validity of the empathy quotient. Psychological Medicine. 2004;34:911–24.
Ware JE, Kosinski M, Dewey JE, Gandek B. How to score and interpret single-item health status measures: a manual for users of the SF-8 Health Survey. Lincoln, RI: QualityMetric Incorporated; 2001.
Turner-Bowker D, Bayliss M, Ware JEJ, Kosinski M. Usefulness of the SF-8 health survey for comparing the impact of migraine and other conditions. Qual Life Res. 2003;12:1003–12.
Spitzer RL, Williams JB, Kroenke K, et al. Utility of a new procedure for diagnosing mental disorders in primary care: the PRIME-MD study. JAMA. 1994;272:1749–56.
Whooley MA, Avins AL, Miranda J, Browner WS. Case-finding instruments for depression. Two questions are as good as many. J Gen Intern Med. 1997;12:439–45.
Garibaldi RA, Trontell MC, Waxman H, et al. The in-training examination in internal medicine. Ann Intern Med. 1994;121:117–23.
Garibaldi RA, Subhiyah R, Moore ME, Waxman H. The in-training examination in internal medicine: an analysis of resident performance over time. Ann Intern Med. 2002;137:505–10.
McDonald FS, Zeger SL, Kolars JC. Factors associated with medical knowledge acquisition during internal medicine residency. J Gen Intern Med. 2007;22:962–8.
McDonald FS, Zeger SL, Kolars JC. Associations of conference attendence with internal medicine in-training examination scores. Mayo Clin Proc. 2008;83:449–53.
Cook DA, Beckman TJ. Does scale length matter? A comparison of nine- versus five-point rating scales for the mini-CEX. Adv Health Sci Educ Theory Pract. 2009;14:655–64.
Cook DA, Beckman TJ, Mandrekar JN, Pankratz VS. Internal structure of mini-CEX scores for internal medicine residents: factor analysis and generalizability. Adv Health Sci Educ. 2010;15:633–45.
Norcini JJ, Blank LL, Duffy FD, Fortna GS. The mini-CEX: a method for assessing clinical skills. Ann Intern Med. 2003;138:476–81.
Kogan JR, Bellini LM, Shea JA. Feasibility, reliability, and validity of the mini-clinical evaluation exercise (mCEX) in a medicine core clerkship. Acad Med. 2003;78(10 suppl):S33–5.
Hatala R, Ainslie M, Kassen BO, Mackie I, Roberts JM. Assessing the mini-clinical evaluation exercise in comparison to a national specialty examination. Med Educ. 2006;40:950–6.
Holmboe ES, Hout S, Chung J, Norcini JJ, Hawkins RE. Construct validity of the mini-clinical evaluation exercise (miniCEX). Acad Med. 2003;78:826–30.
Durning SJ, Cation LJ, Markert RJ, Pangaro L. Assessing the reliability and validity of the mini-clinical evaluation exercise for internal medicine residency training. Acad Med. 2002;77:900–4.
Beckman TJ, Mandrekar JN, Engstler GJ, Ficalora RD. Determining the reliability of clinical assessment scores in real time. Teach Learn Med. 2009;21:188–94.
ACGME. Common Program Requirements 2011. http://www.acgme.org/acwebsite/home/Common_Program_Requirements_07012011.pdf. Accessed August 22, 2011.
Pellegrino ED. Toward a reconstruction of medical morality. The American Journal of Bioethics. 2006;6:65–71.
Rosenow EC III. Recertifying in the art of medicine: what I would tell young physicians. Mayo Clin Proc. 2000;75:865–8.
Stratton TD, Elam CL, Murphy-Spencer AE, Quinlivan SL. Emotional intelligence and clinical skills: preliminary results from a comprehensive clinical performance examination. Acad Med. 2005;80:S34–7.
Spickard A, Gabbe SG, Christensen JF. Mid-career burnout in generalist and specialist physicians. JAMA. 2009;288:1447–50.
Asch DA, Jedrziewski MK, Christakis NA. Response rates to mail surveys published in medical journals. J Clin Epidemiol. 1997;50:1129–36.
Kellerman SE, Herold J. Physician response to surveys. A review of the literature. Am J Prev Med. 2001;20:61–7.
Acknowledgments
Presented at the Association for Medical Education in Europe Meeting in Vienna, Austria, on August 29, 2011. This study was supported by a Mayo Program in Professionalism and Ethics (PPE-5) Award.
Conflict of Interest
None disclosed.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
ESM 1
(DOC 68 kb)
Rights and permissions
About this article
Cite this article
Beckman, T.J., Reed, D.A., Shanafelt, T.D. et al. Resident Physician Well-Being and Assessments of Their Knowledge and Clinical Performance. J GEN INTERN MED 27, 325–330 (2012). https://doi.org/10.1007/s11606-011-1891-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11606-011-1891-6