INTRODUCTION

The Association of Program Directors in Internal Medicine (APDIM) and the American College of Physicians (ACP) have noted that Internal Medicine residency training is heavily weighted towards management of acute, inpatient care needs at the expense of ambulatory education.1,2 The ACP states, “A resident in a longitudinal weekly outpatient session is frequently preoccupied with problems on the inpatient service, and may view the outpatient time as a distraction from what is considered a higher inpatient priority.” The Residency Review Committee (RRC) for Internal Medicine has increased training requirements for ambulatory care education in response to the shift of patient care from inpatient to outpatient, but APDIM and the ACP describe outpatient training as “dysfunctional” and “inadequate.”1–3 Both APDIM and the ACP call for changes in residency training that include enhancing ambulatory education.

The call to improve ambulatory care education is hampered by the lack of validated outcomes measures and multi-institutional studies.4–7 Although Internal Medicine residency training programs are complying with RRC guidelines for ambulatory care training, gaps exist in the evaluation of this experience that might provide guidance for residency program directors to improve ambulatory training.3,5,8 The learning environment, staff personnel, patient mix, and teaching quality have each been cited as potentially relevant to ambulatory training.9–14

Residents can be expected to value knowledge that they plan to use in their career. Yet, whereas the majority of patient–physician encounters occur as outpatient visits, Internal Medicine residents are described as “frustrated” rather than excited with ambulatory clinic.2 The value that residents place on their ambulatory training in relation to other aspects of residency training is unknown. Our objectives were to determine resident satisfaction with continuity clinic, as well as resident valuation of components of residency training, and to use those results to determine those factors associated with valuation of clinic.

METHODS

Survey Creation, Distribution, and Data Collection

We surveyed residents at 3 Internal Medicine residency training programs using questions adapted from surveys used in prior studies of ambulatory care.10,15 We evaluated 3 domains of residents’ opinions: patient characteristics, preceptor characteristics, and clinic operations, as well as resident personal characteristics, intent to specialize, and valuation of key training experiences. Sites for study were residency training programs that were among those subscribing to the Hopkins online ambulatory curriculum, and they were chosen because of differences in rates of specialization of residency graduates and the presence of program directors who expressed interest in participating. The study sites were Christiana Care Health System, an 1,100-bed hospital system with 45 categorical Internal Medicine house officers, which is a teaching affiliate of Thomas Jefferson University in Wilmington, DE; Henry Ford Hospital, a 903-bed hospital with 109 categorical Internal Medicine house officers, which is a teaching affiliate of Wayne State University School of Medicine in Detroit, MI; and the Johns Hopkins Hospital, a 958-bed hospital with 106 categorical Internal Medicine house officers, which is a teaching affiliate of the Johns Hopkins University School of Medicine in Baltimore, MD. None of the study sites had a primary care track; Christiana Care offers an Internal Medicine/pediatrics track. All 3 study sites used the Hopkins ambulatory curriculum, distributed online through the Hopkins Internet Learning Center (www.hopkinsilc.org). At each site, institutional review board approval was obtained prior to study.

The survey was distributed online through the Hopkins Internet Learning Center and completed by postgraduate year (PGY)-1 through PGY-3 residents from October 1 through December 31, 2005.16 Survey responses were tabulated electronically by the software on the web site, which removed the resident’s name from survey responses. As an incentive to participate, respondents at each site were entered into a raffle for $500.

Respondents indicated their likelihood of pursuing a career in general Internal Medicine (GIM), using a Likert scale of probabilities (1 = 0%, 2 = 10%, 3 = 25%, 4 = 50%, 5 = 75%, 6 = 90%, 7 = 100%). Residents were also asked to rate how highly they valued or how much they agreed with the importance of various patient characteristics, desired and actual preceptor characteristics, and clinic operations using a 5-point Likert scale. The Likert scales were anchored with terms for either value (1 = no value, 3 = neutral, 5 = high value) or agreement (1 = disagree strongly, 3 = neutral, 5 = agree strongly). The value of different training experiences to residents was surveyed by asking residents to rate the value of each training experience to their overall residency training on a Likert scale (1 = no value, 3 = neutral, 5 = high value).

STATISTICAL ANALYSIS

We used Wilcoxon rank sum tests to evaluate differences in mean response scores among residents in different programs and also for the different clinical training sites. We performed univariate and multivariate logistic regression to calculate odds ratios for assigning high value to clinic with various independent variables.

Because the distributions of most responses were skewed towards higher values, we dichotomized the dependent and most independent variables as “strongly agree/high value” (5) versus “less than strongly agree/less than high value” (1,2,3 or 4). Thus, the analysis evaluated the odds of residents indicating the highest level of satisfaction with their clinic versus being less than highly satisfied. In some questions, residents were asked to rate whether or not the amounts of certain features were sufficient. For these questions, we dichotomized questions as “just right” (3) versus “too little/too much” (1,2,4 or 5). Questions about residents’ predicted likelihood of pursuing a primary care career and about the percentage of clinic patients discussed with preceptors were dichotomized as “50% or more” and “less than 50%.”

We were concerned that collinearity would exist among the independent variables. Collinearity occurs when 2 strongly related variables control for each other’s effect in a multiple regression model, causing both variables to incorrectly appear statistically insignificant. Thus, we checked for correlation among variables within each domain using Spearman’s rho. If variables exhibited correlation ≥0.5, we chose only the variable that made the most sense based on review of prior literature of clinic satisfaction for inclusion in multivariate analysis. These variables were also the ones with the largest adjusted odds ratios in univariate analysis.

Variables that were significant at p < .25 in univariate analysis were further examined using the stepwise regression technique described by Hosmer and Lemeshow.17 This method compared nested models using different permutations of independent variables, allowing us to choose the most statistically significant variables for further analysis. Those variables that contributed to models significant at p < .05 using likelihood ratio tests were included in the final multivariate logistic regression model, which was performed with clustering by residency programs, using the cluster function in our statistical software package. Odds ratios were adjusted for the high proportion of residents assigning high value to clinic using the method described by Zhang and Yu.18 All calculations were performed using Stata 9.2 (Stata, College Station, TX, USA).

RESULTS

The survey was completed by 218 of a possible 260 residents (response rate 83.8%), of which 117 (53.7%) were male and 101 (46.3%) were female. Seventy one were PGY-1 (32.6%), 70 were PGY-2 (32.1%), and 77 were PGY-3 (35.3%). The 42 nonresponders, when compared to responders, were more likely to be male and at a lower level of postgraduate training. At Christiana Care Hospital and the Johns Hopkins Hospital, less than 5% of residents were international medical school graduates, whereas at Henry Ford Hospital, 76% were international medical school graduates.

The 3 programs differed in resident intent to pursue a career in GIM (p < .05). The mean response on the Likert scale for all programs was 3.2, corresponding to a 25–50% likelihood of pursuing a career in GIM, but responses ranged from 2.1 (likelihood of 10–25%) at the Johns Hopkins Hospital to 4.2 (likelihood of 50–75%) at Christiana Care Hospital. Of those with intent to specialize, 51 residents (23.0%) chose cardiology, followed by infectious diseases (28, 12.6%), hematology/oncology (27, 12.2%), pulmonology (25, 11.3%), gastroenterology (24, 10.8%), nephrology (16, 7.2%), and endocrinology (16, 7.2%).

Patient Characteristics and Clinic Operations

At all 3 study programs, residents were assigned to a single clinic site during their residency training. Table 1 shows responses to questions on patient characteristics and clinic operations. Residents reported seeing a mean of 1.6 “new” (i.e., not previously seen in the practice) and 2.6 “return” patients per session, which were felt to be “just right” for both types of patients. On average, residents felt that about half of patients seen considered the resident to be their primary care physician. Residents reported treating patients with a diversity of medical conditions and range of ages, which were well-balanced between men and women. However, residents disagreed that they treated patients from a wide range of economic backgrounds in their clinics.

Table 1 Resident Responses on Patient Characteristics and Clinic Operations
Full size table

All 3 study sites used electronic medical records. When evaluating clinic operations, residents agreed that their clinic medical record system (mean rating 4.1), the physical location of clinic (mean rating 3.9), clinic computer availability (mean rating 3.8), and stocking of supplies (mean rating 3.8) were good. They also felt that clinic scheduling supported continuity of precepting (mean rating 3.7) but rated support for continuity of patient care as average (mean rating 3.3). Patient flow (defined as “patient flow from check in to check out”), protection from interruption by pages from the inpatient wards, and having a secure space in clinic to place valuables received the lowest ratings among operational issues.

Preceptor Characteristics

All preceptors at each residency program were General Internists with dedicated precepting responsibilities ranging from 1 to 3 weekly sessions for the entire year. Resident responses to desired and actual characteristics of preceptors are shown in Table 2. The most highly desired characteristics of a preceptor were receptiveness to questions about patient management and positive role modeling of professional behavior. Also desired were preceptors who allowed residents to have autonomy in patient care, were effective at assisting with managing medical issues, were respectful of resident judgment, were enthusiastic teachers, and were readily available. Less important to residents (yet still desired) were preceptors’ availability between clinic sessions, teaching physical exam skills, and teaching history-taking skills.

Table 2 Resident Responses on Preceptor Characteristics
Full size table

There was significant overlap between desired and measured characteristics of preceptors. Overall ratings of preceptors were high. Residents rated their preceptors most highly for being receptive to questions about patient management. They also gave high ratings to preceptors for serving as role models, being effective in assisting with medical issues, allowing autonomy, respecting resident clinical judgment, teaching with enthusiasm, and being effective at generating a differential diagnosis. Preceptor ratings were lower (mean score 3.5) for teaching history taking and physical exam skills. On average, residents discussed more than three-quarters of all patients with their preceptor. The largest differences between desired and measured preceptor characteristics were on teaching physical exam skills and providing constructive feedback.

Value of Training Experiences

The value that residents place on different training experiences to their overall residency training is shown in Figure 1. The highest mean rating was for the medical wards (mean rating 4.7), followed by the medical intensive care unit (4.6) and the coronary care unit (4.3). The value of outpatient continuity clinic to residency training had the lowest mean rating (4.1). The differences in ratings between continuity clinic and each of the other clinical experiences were statistically significant (p < .05), and this trend was maintained when the results were stratified by residency program.

Figure 1
figure 1

Mean value of key clinical training experiences as determined by residents is shown (1 = no value; 5 = high value)

Full size image

Finally, residents were asked to agree or disagree with the statement “My continuity clinic experience makes me confident that I could safely and competently practice GIM after residency” (1 = disagree strongly, 3 = neutral, 5 = agree strongly); the mean response was 4.0.

Variables Associated with Valuing Clinic

Univariate odds were calculated between resident ratings of the different domains surveyed and a high rating (i.e., Likert rating of 5) of the value of continuity clinic. Figure 2 shows the univariate odds of residents indicating they were highly satisfied versus being less than highly satisfied for various features of the clinic. Patient characteristics, operational issues, and preceptor characteristics all influenced residents’ opinion on the overall value of clinic to residency training. A high odds ratio, as seen with “preceptor is a great role model,” indicates that residents who rated their preceptors highly as role models were also likely to value clinic highly. A low odds ratio, as see with “physical location of clinic is ideal,” indicates that residents who rated the physical location of clinic highly were neither likely nor unlikely to value clinic highly. Patient characteristics that were associated with clinic satisfaction were having patient panels with a diversity of ages and balanced between men and women. High ratings on all preceptor characteristics were each associated with high ratings of clinic value. Important operational issues associated with valuing clinic were timely referrals, smooth patient flow, adequate nursing support, clinic scheduling issues, and well-stocked exam rooms. Resident characteristics associated with valuing clinic included those who valued morning report, those with a greater than 50% likelihood of a career in GIM, those who valued the inpatient ward services, and those who felt confident in their skills as an Internist. When compared to PGY-1 house officers, PGY-2 but not PGY-3 residents were less likely to value clinic.

Figure 2
figure 2

Odds ratio of clinic variable associated with high valuation of clinic

Full size image

Correlation Among Independent Variables

Except for moderate correlation between residents’ perceptions of having adequate nursing support and smooth patient flow (r s = .61), we found minimal correlation among responses to questions about patient characteristics, clinic characteristics, or the value assigned to the different key clinical experiences. However, responses to several questions about residents’ perception of preceptor characteristics grouped into 3 clusters with moderate to strong correlation. Preceptors’ availability in clinic correlated moderately with their availability between clinic sessions (r s = .58). Residents’ perceptions that preceptors are good role models were also moderately correlated with their perception that preceptors communicate their clinical reasoning (r s = .57), are receptive to questions about patient management (r s = .55), are enthusiastic teachers (r s = .52), and are effective in generating differential diagnoses (r s = .51) and managing medical issues (r s = .56). Lastly, residents’ perceptions that their preceptors give constructive feedback also correlated with preceptors providing timely feedback (r s = .73), teaching history-taking skills (r s = .55), teaching physical examination skills (r s = .49), and exhibiting enthusiasm (r s = .52). Residents’ perceptions that they receive sufficient autonomy, are respected by their preceptors, and receive help obtaining clinic and social resources for their patients exhibited only modest correlation with other factors.

In multivariate analysis, residents who rated their preceptor as a good role model were most likely to assign the highest value to clinic, after adjusting for other factors. Residents who valued the general medical wards were also likely to value clinic, as were residents who felt their preceptor was aware of clinic and social resources. Residents valued clinic when patient flow was smooth, and patients represented diversity in age. Year of training, intent to pursue a career in GIM, and other factors studied had no effect on the multivariate model (Table 3).

Table 3 Multivariate Analysis of Variables Associated with Valuing Clinic
Full size table

DISCUSSION

We found that patient, preceptor, and operational characteristics are associated with how highly residents value continuity clinic. Residents were more likely to value continuity clinic when they treated patients with a wide range of ages, felt their preceptors were good role models, and when patient flow ran smoothly. Resident characteristics also affected their valuation of clinic. Residents who highly valued the medical wards were also more likely to value clinic. Although residents feel adequately trained in GIM, continuity clinic was a less-valued component of residency training than the inpatient wards, the medical intensive care unit (MICU), or the critical care unit (CCU).

Educational factors, specifically precepting, dominate variables that are associated with valuing continuity clinic. In our univariate analysis, high ratings on all preceptor characteristics were positively associated with the perceived value of continuity clinic. Multivariate analysis also showed that when preceptors were rated as good role models and were rated as aware of clinic and social resources, residents valued clinic highly. For residents to value clinic, preceptors should also be outstanding clinicians; in our multivariate analysis, ratings of preceptors as role models correlated moderately with preceptors who communicated clinical reasoning, were effective at managing medical issues, and skilled at generating differential diagnoses. These findings support the findings of other studies,13,19–21 which suggest that teaching ability positively influences resident perception of continuity clinic, and extend this knowledge by defining the teaching behaviors and clinical skills most likely to influence the meaningfulness of continuity clinic. Continuity clinic is likely to be valued more in residency training programs that build a cadre of preceptors who serve as great role models and who have accomplished teaching, feedback, and clinical skills. Both APDIM and the ACP recommend faculty development for improved teaching in the ambulatory setting as a priority for redesigning training for Internal Medicine.1,2 Academic recognition of precepting and clinical skills may improve ambulatory training.

On average, residents gave clinic operations lower ratings than they gave preceptors. It has been shown that the organizational environment of continuity clinic influences learning.22 We showed that, similar to precepting, high ratings of most operational issues were associated with the valuation of clinic. In multivariate analysis, patient flow was associated with how clinic was valued. Having a gender-balanced practice with a wide range of ages and an acceptable volume of new patients were the patient factors associated with valuing clinic. In contrast, many patient factors, such as diversity of medical problems and economic backgrounds, were not associated with resident satisfaction, confirming what has been shown elsewhere.14

Career choice does not impact how residents value key clinical experiences of residency training, including continuity clinic. The general medical wards and MICU were the highest rated clinical experiences, despite the fact that the majority of surveyed residents did not intend to pursue a career as a Hospitalist or intensivist. Residents most likely to pursue a career in GIM were no more likely to value continuity clinic than those with other career paths. The reasons for this are unclear and merit further study. Possible explanations include that residents pursuing a career in GIM do not feel that continuity clinic replicates what they will see in the “real world.” It may also be that preceptors serve as role models for all residents on characteristics that are independent of career choice, such as professionalism. Because career choice does not impact valuation of key clinical experiences, continuity clinic has the potential to be among the most highly rated training experiences, but its value lags below the others. By focusing on improved precepting, operational issues (especially patient flow), and some aspects of patient diversity, training programs may improve both ambulatory education and how continuity clinic is valued.

This study has several limitations. This study was limited to 3 residency training programs and may not be generalizable to all programs. However, the programs studied differed in their orientation towards generalist medicine or subspecialty care, suggesting applicability to a wide range of residency programs. Factors not studied here are likely to impact resident attitudes about clinic, including patient, preceptor, and operational characteristics not included in our study, as well as the national financial environment and the teaching and clinical environment on the inpatient wards. Our survey was purely quantitative; qualitative methods such as open-ended questions or focus groups might reveal other themes that impact resident attitudes. Whereas resident intent to pursue a career in GIM was surveyed, actual career choices at the completion of residency training were not tracked. In addition, we did not distinguish residents who plan a career as a Hospitalist in our survey. As a career as a Hospitalist grows in popularity, future studies will need to determine how this influences perceptions on ambulatory training. Finally, our survey format allowed residents to rate each item independently, rather than using a forced ranking system, and as a result, most items were ranked highly. Results might have differed had a forced ranking system been used.

Although nearly all subspecialty careers include an ambulatory component, residents do not value their continuity clinic training at the same level as other key clinical experiences. An area of future study would include a survey of residency program graduates on how well different aspects of residency training, including continuity clinic, have prepared them for their current job. Residency training programs may improve how residents value their continuity clinic by careful selection of preceptors who serve as good role models, attention to patient flow through clinic, and ensuring diverse patient panels.