BACKGROUND

Surveys of department and academic promotion chairpersons in the United States revealed that teaching effectiveness, which is largely determined by learners’ assessments of faculty, is an important criterion for promoting clinician-educators.1,2 Although it is essential that criteria for advancing clinician-educators are trustworthy, emerging evidence suggests that learners’ assessments of teachers may be influenced by factors unrelated to teaching effectiveness. For example, studies have demonstrated that skilled female faculty receive lower teaching assessment scores than males3 and that assessment scores vary among medical specialties because of differences between learning environments.4

Previous studies have shown that high well-being among internal medicine residents correlates positively with empathy5 and the implementation of work-hour limitations6 and negatively with the likelihood of perceived medical errors.7 Consequently, because it is known that interpersonal relationships between teachers and learners play a crucial role in learning,8–11 we postulated that residents’ mental health would influence their perceptions and assessments of attending physicians.

Resident physicians’ well-being affects their attitudes towards patients and the quality of care that they provide.5–7 However, the interactions between residents’ well-being and empathy, and their assessments of faculty physicians, are unknown. Therefore, we utilized a prospective longitudinal study design to investigate the hypothesis that resident physicians’ well-being and empathy would be associated with faculty teaching assessment scores.

METHODS

Learning Environment and Participants

This study included all resident physicians enrolled in the Mayo Internal Medicine Well-Being (IMWELL) Study and the attending physicians that they assessed between August 2007 and August 2008. The study was approved by the Mayo Clinic Institutional Review Board.

The Mayo IMWELL Study

Resident characteristics were obtained from the Mayo IMWELL study, a longitudinal Mayo Clinic study of resident physician well-being. Since the 2003 academic year, all categorical and preliminary residents in the Mayo Clinic Rochester Internal Medicine Residency program have been invited to participate in this study during their first-year orientation or by telephone if they cannot attend orientation. Approximately 85% of residents have volunteered to participate and signed informed consent. Residents are surveyed at regular intervals throughout their training. An instrument that measures quality of life (QOL) is administered quarterly. Instruments that measure burnout, empathy, and depression are administered twice yearly.

Instruments

The clinical teaching assessments are comprised of 16 items structured on 5-point Likert scales (1 = strongly disagree, 5 = strongly agree). Content validity is based on assessment elements represented in previously published clinical teaching instruments,12,13 observations of bedside teaching,14 and expert review.3 Factor analysis revealed that Mayo clinical teaching assessments are multi-demensional.3,4 Additional validity evidence includes excellent internal consistency and interrater reliability.3,15

The Mayo IMWELL study utilizes survey items from the linear analog self-assessment (LASA) scale of QOL,16 Maslach Burnout Inventory (MBI),17 Jefferson Scale of Physician Empathy (JSPE),18,19 and depression screening questions by Spitzer et al.20 These instruments are supported by sources of validity evidence21–23 essential to medical education research.12,13

A LASA scale was used to measure QOL. This single-item scale ranges from 0 (as bad as it can be) to 10 (as good as it can be). LASA QOL scores have been validated in diverse populations including the general public,24 cancer patients,16,25 and physicians.26

The MBI is a 22-item instrument with Likert scales ranging from 0 (never) to 6 (daily).17 Content validity was demonstrated by reviewing the items of established scales and surveying professionals who are at risk of experiencing burnout, including physicians.17 Numerous studies have shown that the MBI is an effective measure of burnout among resident physicians.7,27–30 Factor analysis demonstrated that the MBI consists of three dimensions: depersonalization (possible scores 0 to 30), emotional exhaustion (possible scores 0 to 54), and sense of low personal accomplishment (possible scores 0 to 48).17 Additional validity evidence includes high internal consistency, acceptable test-retest reliability, moderate correlation with other measures of burnout, and poor correlation with constructs that are likely confounded with burnout.17

The JSPE is a 20-item instrument for measuring empathy with Likert scales ranging from 1 (strongly disagree) to 7 (strongly agree).18,19 The range of possible overall JSPE scores is 20 to 140. Content validity was established by careful literature review, obtaining expert input from physicians (including residents) and revising the instrument through an iterative process.18 Factor analysis revealed four dimensions: physician’s view from the patient’s perspective, understanding patient’s feelings, ignoring patient’s emotions and thinking like the patient.18 Other validity evidence included high internal consistency reliability, acceptable correlation with other measures of empathy, and poor correlation with measures irrelevant to empathy.18,19,31

Spitzer et al. first described a depression screening method consisting of two questions: “During the past month, have you often been bothered by feeling down, depressed, and hopeless?” and “During the past month, have you often been bothered by little interest or pleasure in doing things?”20 Whooley et al. demonstrated that these two questions perform as well at screening for depression as several widely used depression inventories, including the Beck Depression Inventory, the Center for Epidemiological Studies Depression Scale, and the Medical Outcomes Study depression measure.32 Spitzer et al.’s screening questions have been used to identify depression in various populations,20,32 including resident physicians.6,27

Data Analysis

We studied 1,191 monthly assessments of 356 faculty members by 209 unique internal medicine residents in August 2007, February 2008, and August 2008. Resident characteristics were obtained from the Mayo Clinic IMWELL Study and assessment scores from the Mayo clinical teaching assessment form. A repeated measures design, analyzed using multivariate generalized estimating equations, was employed to evaluate associations between residents’ assessments of faculty and residents’ quality of life, burnout, empathy, fatigue, and depression over the three points in time. Scores from the 16 teaching assessment items were averaged to form a continuous overall faculty score ranging from 1 to 5. Covariates included resident well-being (quality of life, burnout, depression), empathy, gender, year of training, program (categorical or preliminary), debt, relationship status (single, married, divorced, partner), and children (yes or no). An additional covariate was the global faculty assessment over the previous year (defined as the mean of all teaching assessments received for each faculty member during the prior academic year). Univariate associations with residents’ assessments of faculty were examined, and a multivariate model was developed using standard forward and backward stepwise selection techniques. Two-tailed statistical significance was set at an α of 0.05. The study sample of 149 respondents provided 80% power for a medium-to-small Cohen’s f2 effect size of 0.06 for a univariate association between faculty assessment score and any well-being or empathy variable.33 Statistical analyses were conducted using SAS version 9.1 (SAS Institute Inc., Cary, NC).

RESULTS

Of 241 eligible residents, 209 (86.7%) provided well-being and/or assessment data for this study. Demographic characteristics for this sample are shown in Table 1. As results did not differ for categorical and preliminary residents, data were pooled across these categories. Baseline well-being, empathy, and evaluation scores at the start of the current study are shown in Table 2.

Table 1 Baseline Characteristics of Resident Physicians Enrolled in the Mayo IMWELL Study from August 2007 to August 2008
Full size table
Table 2 Well-Being, Empathy, and Assessment Scores for Resident Physicians Responding to the Mayo IMWELL Study at baseline in August 2007
Full size table

The 149 residents providing well-being, empathy, and assessment data were included in all analyses. The demographic, well-being, empathy, and evaluation scores for this sample did not differ from those of the partial responders or for the residency as a whole. Univariate associations between residents’ assessments of faculty and residents’ well-being, empathy, and demographic characteristics are presented in Table 3. In multivariate models of the 149 residents providing well-being, empathy, and assessment data, there were no statistically significant associations between faculty assessments and resident quality of life, burnout, fatigue, depression, or demographic characteristics (Table 4). Both forward and backward model selection approaches yielded the same model. The factor most strongly correlated with residents’ assessments of faculty in every model was the individual faculty member’s average assessment rating for the year preceding the study. Across models, a 1-point increase in the baseline rating was associated with an approximately 0.64-point increase in the current rating (beta = 0.6379, 95% CI = 0.4182–0.8575, p < .0001, Table 4). Consequently, a 0.5-point increase in a resident’s baseline global faculty rating was associated with a 0.32-point increase in resident-of-teacher assessment score. Empathy as measured by the Jefferson Scale of Physician Empathy was significantly associated with clinical teaching assessment ratings (beta = 0.0063, 95% CI = 0.0018–0.0108, p = 0.0061, Table 4). Therefore, a 10-point increase in empathy was associated with a 0.063-point increase in residents’ ratings of faculty.

Table 3 Univariate Associations Between Well-Being and Empathy and Resident-of-Teacher Assessment Scoresa
Full size table
Table 4 Association Between Well-Being and Empathy and Resident-of-Teacher Assessment Scores in Multivariate Analysisa
Full size table

DISCUSSION

To our knowledge this is the first study to show that resident physicians’ assessments of faculty teachers are not influenced by dimensions of well-being including quality of life, burnout, and depression. The legitimacy of teaching assessments is further supported in that previous faculty performance was the strongest predictor of resident-of-faculty assessment scores. Nonetheless, the association between resident empathy and residents’ assessments of faculty suggests that faculty assessments may be affected by residents’ intrinsic characteristics.

Clinician-educators are vital to the success of academic medical centers. Teaching effectiveness, which is largely determined by learners’ assessments, is an important criterion for promoting clinician-educators.1,2 This study lends to the credibility of clinical teaching assessments, which should be supported by diverse validity evidence.21–23,34,35 This study also builds upon our previous research,3,4,14 which demonstrated validity of faculty teaching assessments.

Resident physicians’ well-being is related to their attitudes toward work and patients. Authors have reported associations between decreased work-hours and improved resident career satisfaction6 and emotional vitality.6,29 Residents who perceive more frequent medical errors experience increased depression and burnout.7 Residents with burnout are more likely to report providing suboptimal patient care.7,27 The current study builds upon previous literature on resident well-being by revealing that residents’ quality of life, burnout, and depression do not impact on their assessments of faculty.

However, this study demonstrated that residents’ empathy was associated with resident-of-teacher assessment scores. Residents’ JSPE empathy scores over the duration of this study varied by up to 61 points, which in the multivariate models would translate to a 0.38-point (95% CI 0.11–0.66) change on the relatively narrow 5-point faculty teaching assessment instrument. Based on experience with teaching assessments at our institution, mean score variations in faculty assessments similar to those found in this study may influence important judgments about faculty performance, and would thus be educationally meaningful.

Empathy may differ from dimensions of well-being because evidence supports that it is an innate characteristic and develops from an early age.36–38 Nevertheless, learners’ empathy may vary over time and with exposure to special circumstances. Among medical students, empathy wanes over years of schooling,39–41 is higher among students choosing primary care specialties,39,41 and is lower among males and in situations that emphasize efficiency, patient objectification, and medical hierarchies.42 Empathy also declines over the internship year,30,43 and this decline persists throughout residency training.44 Additionally, residents’ empathy is reduced by perceptions of low well-being5 and experiencing medical errors.7 The current study extends this literature by demonstrating that residents’ empathy is associated with another important educational variable: the assessment of faculty teachers. Rogers observed that deep empathetic concern of teachers for learners is essential for learning.45 Therefore, future research should determine whether learning also depends on residents’ empathy for teachers.

This study has limitations. The study was conducted at one institution, and the results may have restricted generalizability. However, the range of well-being and empathy scores in this study are similar to those reported in previous studies at other institutions.6,39–41,43,44,46,47 Also, it is possible that non-response bias may affect the results, although data for responders were similar to those of non-responders where comparisons were possible (e.g., for gender). Additionally, although we are unaware of other medical education studies that examined the influence of resident or student empathy and well-being on their assessments of faculty, we acknowledge that this study did not investigate the possible influence of many potential confounders, such as time of year and capabilities of the resident. Finally, we recognize that while empathy was associated with teaching assessments, it is unknown whether assessments are influenced by other resident characteristics like aggressiveness, career orientation, or learning style.

We found that resident physicians’ assessments of faculty were not associated with multiple dimensions of resident well-being, but were strongly associated with previous assessments of faculty teaching effectiveness. This should be reassuring to clinician-educators whose advancement may depend upon resident-of-faculty assessments1,2 and to promotion committees that use this data to measure teaching effectiveness. Nonetheless, we also found that residents’ assessments of faculty are modestly influenced by residents’ empathy. Consequently, future studies should further explore the impact of environmental factors and intrinsic learner characteristics on clinical performance assessment scores.