ABSTRACT
BACKGROUND
Remediation in the era of competency-based assessment demands a model that empowers students to improve performance.
AIM
To examine a remediation model where students, rather than faculty, develop remedial plans to improve performance.
SETTING/PARTICIPANTS
Private medical school, 177 medical students.
PROGRAM DESCRIPTION
A promotion committee uses student-generated portfolios and faculty referrals to identify struggling students, and has them develop formal remediation plans with personal reflections, improvement strategies, and performance evidence. Students submit reports to document progress until formally released from remediation by the promotion committee.
PROGRAM EVALUATION
Participants included 177 students from six classes (2009–2014). Twenty-six were placed in remediation, with more referrals occurring during Years 1 or 2 (n = 20, 76 %). Unprofessional behavior represented the most common reason for referral in Years 3–5. Remedial students did not differ from classmates (n = 151) on baseline characteristics (Age, Gender, US citizenship, MCAT) or willingness to recommend their medical school to future students (p < 0.05). Two remedial students did not graduate and three did not pass USLME licensure exams on first attempt. Most remedial students (92 %) generated appropriate plans to address performance deficits.
DISCUSSION
Students can successfully design remedial interventions. This learner-driven remediation model promotes greater autonomy and reinforces self-regulated learning.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
INTRODUCTION
Medical educators need to revisit how we identify and assist “struggling” learners.1 The current emphasis on examining competence in multiple domains (e.g., professionalism, clinical reasoning, etc.), across the medical education continuum, increases the likelihood we will identify and diagnose individuals with several performance deficits beyond the familiar and easy to measure.2–6 By casting a wider net, we will probably encounter more learners who require remediation. Therefore, it is essential we identify mechanisms to maximize remedial processes that benefit a growing number of struggling medical learners without overburdening our faculty.7
Limited evidence currently exists in the literature about best remediation practices in medical education.1,8 Published reports suggest faculty are reluctant to fail struggling learners, given the effort required to document performance deficits, participate in appeal processes, or provide remedial instruction.9,10 Furthermore, faculty may not identify underperforming learners in sufficient time to intervene effectively,1,11 or know which remedial techniques to use for learners with hard-to-assess competencies, such as professionalism and communication,11 or who require extensive guidance and mentoring.1,5 A “one-size-fits-all” approach cannot possibly address the myriad of factors (e.g., poor study skills, mental health issues, family demands, etc.) that may contribute to inconsistent or unsatisfactory performance.7,12 We cannot assume all faculty have the time, resources, or expertise to “fix the problem” or judge when learners have successfully remediated all deficiencies.1,2,13 In light of these challenges, some institutions have built programs where teams of experts, from multiple professions, manage remediation processes.5–7,12 Working together, such teams develop guidelines and best practices to provide more efficient and successful approaches to assist struggling learners.
Whether remediation is overseen by individual faculty, without specific expertise, or by specialized faculty teams with multiple skills,5–7 the learners’ role appears to be limited to following faculty-designed remedial interventions.1,5–8,11,12 This trend is surprising, given strong research evidence that enhanced learner autonomy has positive influences upon psychological states (motivation, interest, satisfaction, etc.) and actions (acceptance of shortcomings, persistence, etc.) required for remediation to succeed.14 Are we unintentionally encouraging struggling learners to become passive recipients of faculty-designed remedial interventions? Is there another way we can provide greater autonomy to underperforming learners where they, rather than faculty, select remedial strategies?
We present an innovative, systematic approach to remediation purposefully designed to engage learners with the development and implementation of remedial plans to improve their performance. We first describe our approach to remediation and then report outcomes for students who were and were not placed in remediation during medical school. We conclude with lessons learned and future directions.
SETTING AND PARTICIPANTS
The Cleveland Clinic Lerner College of Medicine (CCLCM) is a 5-year undergraduate medical school program with an assessment system designed to document student progress longitudinally and inform high-stakes decisions in nine competency domains.15 Students complete organ-based courses, complemented by longitudinal clinical experiences, in Years 1–2, and core clerkships, electives, and a graduate-level thesis in Years 3–5. All CCLCM students are assigned a physician advisor (PA) at matriculation. The PA meets regularly with their assigned students to discuss progress and performance. A 24-member Medical Student Promotion and Review Committee (MSPRC), comprised of senior-level basic scientists and clinicians, reviews each student’s performance (as documented in student-generated portfolios) at the end of Years 1, 2, and 4. The MSPRC uses these reviews, described elsewhere,16 to judge each student’s fulfillment of promotion criteria. Faculty may refer students to the MSPRC, through the dean of student affairs, if concerns arise about a student’s progress or behavior.
PROGRAM DESCRIPTION
Our program views remediation as an opportunity for students to reflect upon and improve their performance through use of the self-regulation cycle.17 The MSPRC plays a pivotal role by asking remedial students to develop formal remediation plans, where each student reflects upon targeted areas for improvement (TAFI) and proposes specific strategies to address these TAFIs. PAs provide information about external resources such as counseling, study/time management skills, and wellness care. Though PAs may help learners diagnose problem(s) and consider intervention strategies, students take full responsibility for constructing remediation plans. The MSPRC reviews each student’s remediation plan for appropriateness and measurable outcomes, and then sets a due date for the first, of possibly several, student-generated progress reports. During their monthly meetings, MSPRC members vote to approve students’ progress reports and judge if students achieved remediation goals. The MSPRC chair communicates all committee decisions in writing to each student and his/her PA.
PROGRAM EVALUATION
We used a retrospective, cohort-based approach to examine our learner-driven remediation model.
Data Sources
We consulted MSPRC records to identify all CCLCM students from six class cohorts (2009–2014) placed in remediation during medical school, and compared remedial students to their classmates not placed in remediation on variables collected for program evaluation purposes:
-
Baseline characteristics (gender, age, US citizenship, MCAT score)
-
USMLE licensure exam performance (pass-rate for each student)
-
National Residency Matching Program data (clinical specialty for each student)
-
Satisfaction with medical school (collected from graduating students using questionnaire)
Then, we read MSPRC correspondence to each remedial student to extract referral mechanism (portfolio-based decision or faculty-initiated), unsatisfactory competency domain(s), remediation duration (days), and completion of students’ remediation goals. Finally, we read students’ remedial plans to identify strategies they proposed to the MSPRC to address performance deficits in specific competency areas. We manually recorded a subset of these strategies to provide examples of students’ approaches to remediation.
All performance decisions, referral mechanisms, and outcomes were manually entered into SPSS. Chi-square and ANOVA tests were used to compare baseline characteristics and satisfaction scores of the two student groups (remediation vs. other), while the Mann Whitney U test was used to examine if remediation duration (days) differed by students’ gender. Statistical significance was established at p < 0.05 for all comparisons. We obtained ethical approval from the Cleveland Clinic’s Office of Institutional Research.
Results
Participants include 177 of 187 medical students from six class cohorts (2009–2014) who consented (95 %) to release program evaluation data for research purposes. Of these, the MSPRC placed 26 students (17 men and nine women) in formal remediation. Half (13) were identified by the MSPRC for not meeting year-end performance standards assessed via portfolios. The dean of student affairs referred the remaining 13 students based on faculty recommendation. We did not detect significant differences between remedial students and their classmates for baseline characteristics (Table 1). Overall, three-quarters of remediation decisions occurred during the first (10, 38 %) or second (10, 38 %) year of medical school.
With regard to specific domains in which students failed to meet year-end standards or were otherwise referred, professionalism (18, 69 %), communication (10, 38 %), research (9, 35 %), and reflective practice (5, 19 %) represent frequent domains of unsatisfactory performance. Fewer students did not meet performance standards for clinical reasoning (4, 15 %), medical knowledge (3, 11 %), clinical skills (2, 8 %), personal development (2, 8 %), or health care systems (2, 8 %). Unprofessional behavior (e.g., not logging patient encounters, unexcused absences, chronic tardiness, etc.) represented the most common reason for student remediation during Years 3–5 of medical school. Ten students had deficient performance in one competency, while others had unsatisfactory performance in two (7, 27 %) or more (9, 35 %) competencies. Three students were placed in remediation twice, with one repeating for unprofessional behavior. Time in remediation (determined by dates in MSPRC correspondence to students) ranged from 39 to 1696 days (Median = 448 days) and did not differ by student gender (p > 0.05).
Remediation plans revealed various student-proposed strategies to address TAFIs for multiple competency domains (Table 2). We observed that students’ remedial strategies resembled approaches faculty would recommend. We also observed the MSRPC’s correspondence to students consistently had an encouraging, supportive tone for students’ remedial efforts (See Appendices for MSPRC’s correspondence to one remedial student where committee asks student to develop a remediation plan, comments on student’s progress report, and releases student from remediation). The MSPRC “prescribed” remediation plans for two students only after these students did not provide adequate assessment evidence in progress reports.
The MSPRC determined that most students (24, 92 %) generated sufficient evidence (predominately obtained from faculty and peers) to document their competence. Table 1 reveals that some remedial students experienced negative outcomes, as two did not graduate and three did not pass USMLE licensure exams on first attempt. Proportionally, more remedial students (11, 46 %) pursued surgical specialties than their classmates (50, 33 %). Table 1 also shows graduating remedial students were just as likely as their classmates to recommend the CCLCM program to future students, even though fewer were as satisfied with their overall experience at CCLCM as their classmates.
DISCUSSION
Most programs struggle with how to identify and assist learners requiring remediation. We explored if students, rather than faculty, can take ownership for remedial plan design and implementation. We review key findings and conclude with future directions.
The locus of control for selecting remedial interventions and assessment evidence typically rests with faculty. In our model, the MSPRC charged students to develop remediation plans, with PA guidance as needed, and obtain assessment evidence to document progress and achievement. Most remediating students (92 %) successfully met these expectations and graduated from medical school.
Three-quarters of remedial students were identified in Years 1–2 of medical school. Early identification provides opportunities for multiple, tailored interventions to occur. Professionalism, communication skills, and research performance were frequent reasons the MSPRC placed students in remediation. Few students were identified for gaps in medical knowledge or clinical reasoning. The literature suggests some competencies, like professionalism, require direct observation and ongoing assessment for learner success.1,18 Our students’ remedial strategies (Table 2) ranged from specific to holistic approaches and often cut across courses and contexts, thereby providing them with more opportunities to improve and document their performance in complex domains. This emphasis on competence may explain why students’ time in remediation was longer (Median = 448 days) than normally needed for “more of the same,” such as retaking a test or course.2
Some learners may not know how to navigate performance obstacles, making it important to offer guidance and oversight.19,20 For this reason, we train a core group of faculty to help students apply the self-regulation cycle.17 PAs coach students in reflection, self-assessment, and goal-setting.2,20,21 They also form long-term relationships with students and frequently serve as student advocates. The MSPRC encourages students to identify measurable outcomes to document competence, while offering positive support in written correspondence to students (See Appendices for series of letters to one student throughout the remediation process). Both faculty groups work in tandem to support this learner-driven remediation model while ensuring student accountability.
Our evaluation is limited to one program, at one school, with a small class size. Other programs may foster remedial learners’ autonomy more than conveyed in the literature.5,6,8 Our student satisfaction measures originate from two questionnaire items obtained at graduation and do not capture students’ satisfaction with specific academic referral policies or remedial processes.22
Remediation is complex and involves multiple stakeholders and contexts. Future studies should capture learners’ perceptions of remediation to complement the faculty point-of-view currently reported in the literature. We discovered that more men than women were identified for remediation, yet we found no research exploring this observation in depth. Additionally, few longitudinal or multi-institutional studies currently exist, suggesting an area ripe for research.
We discovered, in our 10-year experience with this remediation model, that medical students can select appropriate remedial interventions with the support of well-trained faculty. Our learner-driven approach to remediation may transfer to graduate medical education (GME), where Clinical Competence Committees (CCCs) meet regularly to discuss and review resident performance. Having struggling residents create remediation plans and submit progress reports to CCCs may present few barriers, given GME’s competency-based assessment framework and emphasis on providing residents with frequent, formative feedback. Our approach requires a learning environment where faculty must trust students have the motivation and capabilities to identify and implement appropriate remedial strategies. We have created a systematic process where students must reflect on their performance, develop plans tailored to meet learning needs, and collect assessment evidence to document competence. To conclude, we believe educators who wish to adopt a learner-driven approach to remediation should view remediation as a growth opportunity rather than as a punishment.
REFERENCES
Hauer KE, Ciccone A, Henzel TR, Katsufrakis P, Miller SH, Norcross WA, Papadakis MA, Irby DM. Remediation of the deficiencies of physicians across the continuum from medical school to practice: a thematic review of the literature. Acad Med. 2009;84(12):1822–32.
Cleland J, Leggett H, Sandars J, Costa MJ, Patel R, Moffat M. The remediation challenge: theoretical and methodological insights from a systematic review. Med Educ. 2013;47(3):242–51.
Wilkinson TJ, Tweed MJ, Egan TG, Ali AN, McKenzie JM, Moore M, Rudland JR. Joining the dots: conditional pass and programmatic assessment enhances recognition of problems with professionalism and factors hampering student progress. BMC Med Educ. 2011;11:29.
Brokaw JJ, Torbeck LJ, Bell MA, Deal DW. Impact of a competency-based curriculum on medical student advancement: a ten-year analysis. Teach Learn Med. 2011;23(3):207–14.
Guerrasio J, Garrity MJ, Aagaard EM. Learner deficits and academic outcomes of medical students, residents, fellows, and attending physicians referred to a remediation program, 2006–2012. Acad Med. 2014;89(2):352–8.
Ricketts C, Bligh J. Developing a “frequent look and rapid remediation” assessment system for a new medical school. Acad Med. 2011;86(1):67–71.
Geurrasio J. Remediation of the Struggling Medical Learner. Irwin: Association for Hospital Medical Education; 2013.
Frellsen SL, Baker EA, Papp KK, Durning SJ. Medical school policies regarding struggling medical students during the internal medicine clerkships: results of a national survey. Acad Med. 2008;83(9):876–81.
Dudek NL, Marks MB, Regehr G. Failure to fail: the perspectives of clinical supervisors. Acad Med. 2005;80(10):S84–7.
Cleland JA, Knight LV, Rees CE, Tracey S, Bond CM. Is it me or is it them? Factors that influence the passing of underperforming students. Med Educ. 2008;42(8):800–9.
Saxena V, O’Sullivan PS, Teherani A, Irby DM, Hauer KE. Remediation techniques for student performance problems after a comprehensive clinical skills assessment. Acad Med. 2009;84(5):669–76.
Kalet A, Chou CL, eds. Remediation in Medical Education: A Mid-Course Correction. New York: Springer; 2014.
Winston KA, van der Vleuten CPM, Scherpbier AJJA. At-risk medical students: implications of students’ voice for the theory and practice of remediation. Med Educ. 2010;44(10):1038–47.
Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol. 2000;55(1):67–78.
Dannefer EF, Henson LH. The portfolio approach to competency-based assessment at Cleveland Clinic Lerner College of Medicine. Acad Med. 2007;82(5):493–502.
Bierer SB, Dannefer EF. Does students’ gender, citizenship, or verbal ability affect fairness of promotion decisions? Results from one medical school. Acad Med. 2011;86(6):773–7.
Sandars J, Clearly TJ. Self-regulation theory: applications to medical education: AMEE Guide No. 58. Med Teach. 2011;33(11):875–86.
Hickson GB, Pichert JW, Webb LE, Gabbe SG. A complementary approach to promoting professionalism: identifying, measuring, and addressing unprofessional behaviors. Acad Med. 2007;82(11):1040–8.
Yao DC, Wright SM. The challenge of problem residents. JGIM. 2001;16(7):486–92.
White CB, Ross PT, Gruppen LD. Remediating students’ failed OSCE performance at one school: the effects of self-assessment, reflection, and feedback. Acad Med. 2009;84(5):651–4.
Durning SJ, Clearly TJ, Sandars J, Hemmber P, Kokotailo P, Artino AR. Viewing “strugglers” through a different lens: how a self-regulated learning perspective can help medical educators with assessment and remediation. Acad Med. 2011;86(4):488–95.
Wear D, Keck-McNulty C, Jones B, Penn M, Moss P. Medical students’ experience of academic review and promotions committees. Teach Learn Med. 2004;16(3):226–32.
Acknowledgements
The authors wish to thank Drs. Colleen Colbert, Dale Dannefer, Lily Pien, and James Young for their feedback on this manuscript.
Conflict of Interest
The authors declare that they do not have a conflict of interest.
Funders
None.
Author information
Authors and Affiliations
Corresponding author
Electronic Supplementary Material
Below is the link to the electronic supplementary material.
ESM 1
(DOCX 67 kb)
Rights and permissions
About this article
Cite this article
Bierer, S.B., Dannefer, E.F. & Tetzlaff, J.E. Time to Loosen the Apron Strings: Cohort-based Evaluation of a Learner-driven Remediation Model at One Medical School. J GEN INTERN MED 30, 1339–1343 (2015). https://doi.org/10.1007/s11606-015-3343-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11606-015-3343-1