Assessment and Remediation of Clinical Reasoning

Abstract

Struggles with clinical reasoning are common among medical learners. Assessment of clinical reasoning skills, both diagnosis and management, can be challenging in the clinical environment. The clinical reasoning process is commonly broken down into: generating hypotheses, gathering data, forming a problem representation, creating a differential diagnosis through refinement of initial hypotheses, selecting a working diagnosis, and developing a management plan. This chapter describes a sequenced approach to the remediation of learners who struggle with clinical reasoning, which includes timely identification, global and targeted assessment, coaching, and continuing evaluation and assessment along this clinical reasoning pathway. Coaching sessions use case-based exercises to specify the primary area to support and standardized direct observation to help the learner translate the emerging skillset into practice.

Introduction

Ethan (he/him), a second-year internal medicine resident, is called to meet with the program director, Dr. Ramirez (she/her), to discuss concerns about his clinical performance. Dr. Ramirez alerts Ethan that senior residents and faculty have raised concerns in their written evaluations about his clinical decision-making. Select comments include: “cannot tell a story,” “struggles when things are complex, or when the service is busy,” “cannot put the pieces together,” and “has tunnel vision at times.” One evaluator did note that he “can make an accurate decision when given all of the data.”

Clinical reasoning is a term used to describe the processes of making a diagnosis and managing a patient [1]. Learners can struggle with one or both of these components, commonly referred to as diagnostic reasoning and management reasoning [2]. Effective remediation of learners who struggle with clinical reasoning includes timely identification, global and targeted appraisal, coaching, and continuing evaluation and assessment. The goal of this process is to ensure patient safety and move the learner toward consistent expert performance. National organizations and landmark publications, such as The National Academy of Sciences’ “Improving Diagnosis in Healthcare,” have called for enhanced teaching of clinical reasoning to reduce clinical errors [3]. Unfortunately, the rate of diagnostic and management error is difficult to determine in the routine assessment of individual trainees. Clinical evaluators may lack the skills necessary to assess the clinical reasoning of their learners, limiting intervention efforts. Once identified, dedicated coaching and deliberate practice are the most effective means of moving these learners toward expert performance [4, 5]. However, effective clinical reasoning remediation requires significant time investment by both coach and learner [6]. In addition, learner buy-in is essential to a successful remediation process. We follow many of the steps listed in Chap. 6; the first steps in this process are asking the learner about their perspective, empathizing with what they almost assuredly see as a predicament, and drawing out the learner’s perspective on their own plight.

Identification

Dr. Ramirez: Ethan, do these comments resonate with your experience?

Ethan: It’s definitely not the first time I’ve been given feedback like this, but I am not exactly sure what is going on. I felt that I was improving in these areas as an intern, but now that I have more responsibility, I am struggling with my decision-making.

Dr. Ramirez: I know this can be hard, and I hear that this is not a shock to you. If we can figure out where exactly you’re struggling, we can tackle these problems together.

Ethan: Sometimes I’m just not sure what is wrong with my patients, and which steps I should take next, especially with the more complex patients. Maybe I just need to read more? I always did well in medical school.

Struggles with clinical reasoning can be difficult to recognize and are often “misdiagnosed” as struggles either with fund of knowledge or organization and efficiency ([7]; see Chaps. 7, 11). Clinical reasoning is not recognized as a distinct clinical competency by the Accreditation Council for Graduate Medical Education (ACGME), and thus data on the prevalence of struggles with clinical reasoning are often not captured in surveys about learners who struggle [8,9,10]. According to reports from two centralized remediation programs across the continuum of medical education, clinical reasoning was identified in 25–45% of learners who struggled [5, 11].

Struggles with clinical reasoning can impact performance in many ways. Learners may have trouble with patient presentations or making clinical decisions. Table 9.1 includes some common descriptive phrases used to describe the learner struggling with clinical reasoning.

Table 9.1 Commonly used phrases to describe the learner struggling with clinical reasoning

Often, the evaluator recognizes that a learner is struggling, but cannot quite pinpoint the problem. Learners who make confusing, disorganized, or inaccurate case presentations or write disappointedly inadequate clinical notes are worrisome because they potentially endanger patient safety. Clinical reasoning is a complex cognitive process, dependent on, but not limited to, adequate fund of knowledge. Because many clinical supervisors may lack a framework for analyzing clinical reasoning struggles, they are less likely to effectively coach learners to improve clinical reasoning. Moreover, we often do not spend enough time directly observing our learners’ clinical skills and are therefore left to infer a great deal about their performance from how they answer factual questions on rounds or in conference. Many learners struggling with clinical reasoning are reflexively advised to “read more.”

Ethan’s case is typical of learners who struggle with clinical reasoning. Based on test scores, his knowledge base is sound. However, his performance drops during clinical encounters that require a structured approach and application of knowledge.

Clarification: Global and Targeted Appraisal of the Learner Struggling with Clinical Reasoning

Further conversation between Ethan and Dr. Ramirez revealed that Ethan’s performance on standardized tests was good to average. He remembers struggling on Observed Structured Clinical Examinations in medical school. Dr. Ramirez determines that Ethan struggles to apply knowledge to the clinical environment and refers him to Dr. Williams, a clinical reasoning remediation coach.

Global Appraisal

Effective remediation hinges on accurate determination of specific performance gaps, which is best accomplished with a systematic review of the learner’s performance in the current and, when available, prior program(s). As described in Chap. 6, a comprehensive investigation includes talking with the learner about underlying causes of unsatisfactory clinical performance such as impairment, performance anxiety, or burnout [11]. It is important to review educational history for evidence of primary struggles with medical knowledge; after all, one needs to know the alphabet to spell. Signs consistent with clinical reasoning struggles include failure in one or more objective structured clinical examinations (OSCEs) in medical school; average or above average standardized knowledge test scores coupled with consistently low scores in clinical performance; and comments from clinical rotations (Table 9.1). We recommend that the remediation coach attempt to speak directly with a few clinical educators who have worked with the learner across clinical contexts. And, as described in Chap. 7, knowledge can be assessed by asking direct, factual questions during direct observation of the learner’s performance in the clinical learning environment, or through discussing cases in a coach’s office.

Targeted Appraisal

If global appraisal leads to the determination that the learner’s struggle is predominantly related to clinical reasoning, the next step is a targeted appraisal [5]. The goal of targeted appraisal is to identify strengths and challenges along the clinical reasoning pathway. It is important to determine if the primary struggle is ineffective diagnostic or management reasoning by a thorough review of clinical evaluation data. In fact, clinical evaluation data are critical because they add context to the targeted assessment which can limited by case specificity and a learner’s content knowledge. For the purposes of appraisal of clinical reasoning, it is useful to simplify and consider diagnosis and management as a linear pathway, starting with diagnostic reasoning and then basing management strategies on that diagnosis (although in practice, there is a dynamic and iterative relationship between the two). A clinician must always reconsider the diagnosis as new information emerges from the patient’s response to treatment and the evolution of the clinical predicament.

For learners who struggle with diagnosis, the initial objective is to further specify an area to support along the following diagnostic reasoning pathway: hypothesis generation; data collection; problem representation; refinement of hypotheses; and development of working diagnosis (script selection). We suggest using a seven-step, case-based approach (Fig. 9.1) that incorporates these steps.

Fig. 9.1

Case-based identification of struggles with diagnostic reasoning: a seven-step appraisal. Use this case-based appraisal method to identify areas to support along the diagnostic reasoning pathway. It is critical to use this approach across a range of clinical cases and include varying contextual factors

Hypothesis Generation

The diagnostic reasoning process begins with the development of a broad list of potential diagnoses based on a limited set of key pieces of information from the chief complaint and patient demographics. This list of hypotheses is generated using two well-described cognitive systems of decision-making. Dual-process theory summarizes a vast cognitive psychology evidence base and provides a robust framework for understanding clinical reasoning [12]. The dual-process theory describes two systems that are relatively independent but work together, enabling a physician to reason rapidly and deliberately. System 1 is non-analytical, intuitive, and efficient [13]. The basic clinical reasoning skill in System 1 is pattern recognition. System 1 thinking is most obvious when an experienced physician immediately recognizes a well-established illness script when presented with a patient presentation. In contrast, System 2 is an analytical process. It is slow, deliberative, conscious application of an analytical approach to arrive at a diagnosis [13]. Which system is activated depends on the clinician’s prior experience with a given clinical presentation, and their ability to activate the appropriate illness script [14]. To avoid mistakes, experts consciously toggle between systems, confirming a diagnosis they reached quickly through System 1 by applying System 2 reasoning to the case [15]. Novices also use both systems; however, given their limited experience, their System 1 is likely to be less accurate, and they are more likely than experts to anchor on a final diagnosis based on their initial thoughts. Because of this, inexperienced clinicians are at risk of prematurely committing to a diagnosis with inadequate information.

Data Gathering

Data gathering involves asking questions about the patient’s history (see Chap. 7), performing the physical examination (see Chap. 8), and collecting initial laboratory and imaging results. Consideration of diagnostic hypotheses prior to data gathering, termed hypothesis-driven data gathering, improves diagnostic accuracy [16].

Problem Representation and Illness Scripts

The problem representation is a an abstraction of the important features of a case using paired, opposing descriptive terms referred to as semantic qualifiers [17]. When in the form of a verbalized or written summary statement, the problem representation is commonly referred to as the “one-liner” and is used to summarize patient cases when clinicians communicate with each other (oral presentations on rounds, handoffs, progress notes, calling consults). When done correctly, formulating problem representations strengthens clinical reasoning [18, 19] by activating or accessing illness scripts, or mental representations (schemas) from the clinician’s long-term memory. Illness scripts, which usually include key risk factors, pathophysiology, and clinical presentation, reflect the clinician’s organized stored knowledge of a given disease [14, 20]. With experience and attention to accuracy, learners should enhance their illness scripts to better estimate the likelihood of a diagnosis when a clinical feature is present or absent [21].

The ability to formulate an effective problem representation is a fundamental skill and one that, for many learners, needs to be taught explicitly. Difficulties with problem representation can manifest in many ways. For example, a learner may be unable to provide effective handoffs of care, as observed in their “sign-out” to other providers. Other signs of gaps in this domain are struggles with succinct and accurate presentations, calling consults, or managing more than one complex patient. Evaluators may comment that these learners lack an understanding of the big picture with their patients, or “just don’t get it.”

Management Reasoning

Cook et al. recently defined management reasoning as “the process of making decisions about patient management, including choices about treatment, follow-up visits, further testing, and allocation of limited resources” [2]. They also identified some key differences between diagnostic and management reasoning. For instance, diagnostic reasoning is a classification task of assigning a single diagnosis, operates independently of context, and does not require patient interaction. In contrast, management reasoning is a task involving shared decision-making and monitoring, can include multiple solutions, depends on context (e.g., patient, provider, and system preferences), and requires patient communication [2]. Diagnostic reasoning likely ends with activation of a management script, the first step in management reasoning. Like illness scripts, management scripts are activated in real time in response to encountering a clinical problem [22]. Once a management script is activated (accessed, recalled, retrieved) for a given condition, clinicians can then select actions (i.e., laboratory, imaging, procedures, consultants, medications, monitoring, etc.) based on the needs of a specific patient and the current situation. This process, termed management option selection [22], requires the learner to estimate the harms and benefits of each intervention, taking patient preferences into account. Though the evidence base supporting our understanding of management reasoning is less robust than that for diagnostic reasoning, the two processes are highly analogous, and the literature is growing.

Intervention: Coaching Clinical Reasoning

Setting: Ethan’s first meeting with Dr. Williams (she/her), the clinical reasoning coach

Dr. Williams: Ethan, I look forward to working with you on this. Let’s begin by setting some specific goals. Then I will introduce you to a standardized approach to clinical reasoning, which will give us a shared language we can use to talk about and work on clinical reasoning and clinical decision-making.

Once the targeted appraisal is complete, coaching should begin with a discussion of expectations and goals to obtain a commitment from the learner, because successful clinical reasoning coaching is time-intensive and requires deliberate practice. As clinical reasoning may be a new language to the learner, an explicit discussion of the reasoning processes should follow closely after the initial appraisal. The discussion should include an introduction or review of key clinical reasoning terms. The learner should be informed that the coaching process will include working through segmented cases and clinical reasoning exercises, employing frequent “stops” to determine the reasoning behind the learner’s decisions. Some learners find this approach needlessly theoretical and must be convinced that the ability to think critically and reflect on their own thought processes is critical to development of strong clinical reasoning (see Chap. 4).

The coach and learner together should design a remediation strategy that employs exercises (Fig. 9.2) targeted to the identified area requiring support [23,24,25]. Because successful clinical reasoning depends on context, coaching must include a substantial number of cases across a broad array of clinical conditions. We suggest that coaching encounters begin with simple, typical presentations of common problems that progressively increase in complexity [26, 27]. The coach should give the learner cases with varied chief complaints and demographic information. The coach’s approach, based on the reasoning level of the learner, should provide scaffolding for the learner, at first being very structured and supportive even to the point of sharing a detailed “worked example” if needed, and then fading back as the learner becomes more self-sufficient (see Chap. 19, cognitive apprenticeship). The coach should aim to create a safe atmosphere where the learner can develop strong self-regulatory skills, metacognition, and reflective practice (see Chaps. 4, 15).

Fig. 9.2

Targeted strategies to coach struggles with diagnostic reasoning. The process begins with hypothesis generation and ends with a working diagnosis. Each component of the process has two recommended coaching exercises [23,24,25]. Adapted with permission

Hypothesis Generation (Fig. 9.2, Purple Gear)

Dr. Williams: From the cases we have worked through together so far, I notice that you do not always begin by creating a broad differential diagnosis.

Ethan: Yes, I didn’t feel like I had enough information. I usually go see the patient right away, collect all the information I can, and then think about what may be causing the patient’s symptoms. At times, I come out of the patient’s room feeling confused and disorganized.

Dr. Williams: Can you tell me more about what you mean by disorganized?

Ethan: Yes. I feel overwhelmed, especially when the patient is critically ill.

Ethan struggles to generate potential diagnoses with limited information and distinguish between concerning and less concerning diagnoses. With limited information, and usually limited experience, the learner must rely on System 2 to build or broaden their initial list of hypotheses. Learners can use frameworks or organizing scaffolds (Fig. 9.2, purple gear), to systematically approach this process. Because clinical information can be retrieved and manipulated as a single item within the working memory, the use of frameworks to develop schemas may help learners manage their cognitive load [28, 29]. As certain frameworks may be more appropriate for a given chief complaint, we recommend use of one or more of the following frameworks [27]. We guide the learner toward the appropriate framework through repetitive case-based application and have provided some guidance below.

Schema 1: Pre-test Probability

Consider the probability or likelihood of suspected diseases, based on their prevalence before any diagnostic tests are conducted (also referred to as base rate or prior-probability), specific to a given patient from a particular population. Common diseases are common, and rare diseases are rare. It is important to encourage the learner to routinely familiarize themselves with the epidemiology of the geographic locale and understand how referral filters and recency or availability effects may bias their judgement on what is most likely in a particular clinical setting. We encourage use of this framework for all cases.

Example: While a complaint of chronic cough is usually post-nasal drip, reactive airway disease, or gastroesophageal reflex, during a viral pandemic, adjustments in prior probability need to be carefully considered.

Schema 2: Anatomical

Consider the organs and surrounding structures in a given location and what can go wrong with each. We encourage use of this framework for localized complaints such as pain, redness, swelling, or other signs of inflammation.

Example: For a complaint of chest pain, think of anatomical features in the chest (skin, muscles, ribs, mediastinum, heart, lungs, spine, and associated bony structures) as well as the few instances where this is potentially confusing because pain may be referred from elsewhere.

Schema 3: Pathophysiology

Consider the physiological processes of disease leading to the chief complaint. We encourage use of this framework for isolated abnormal vital signs, laboratory values, or imaging findings.

Example: For an abnormally elevated creatinine level in outpatient clinic, think of pre-renal, intrinsic, and post-renal causes.

Schema 4: Systems

Consider various organ systems and disease processes in each. We encourage use of this framework for nonspecific complaints such as fatigue, weight loss, and fever.

Example: For a complaint of unintentional weight loss in a refugee patient

• Neurological—Depression, eating disorder, laxative abuse

• Gastrointestinal—Malabsorption, dental disease

• Endocrine—Thyroid disease, diabetes mellitus

• Neoplastic—Many forms of cancers

• Infectious—Tapeworm, dysentery

• Vascular—Ischemic bowel disease

• Social—Poverty, isolation

Schema 5: Worst-Case Scenario

Consider specific conditions that can lead to significant morbidity or death, or are time-urgent. These are “cannot-miss” diagnoses. We encourage use of this framework for all cases, while emphasizing that inappropriate consideration of such diagnoses can, in some cases, lead to over-testing.

Example: For a complaint of acute shortness of breath, consider pulmonary embolism, decompensated heart failure, and myocardial infarction.

Data Gathering (Fig. 9.2, Pink Gear)

Dr. Williams: Now that you have a framework for generating a broad differential diagnosis, let’s work through some more cases.

Ethan: I have been told that I take a long time to see patients. This approach might make it worse. How do the attendings always seem to know what questions to ask to get to the heart of the matter quickly?

Dr. Williams: Great question. As we discussed, clinical reasoning is a complex process. Efficiency and accuracy come with practice in developing hypotheses to guide our data gathering. Let’s work on that.

Students are generally taught how to take a comprehensive patient history using a structured approach (i.e., chief complaint, history of present illness, past medical history, etc.). However, learners may struggle to adapt and refine their history-taking based on a given patient’s clinical presentation. To improve hypothesis-driven data gathering, learners need dedicated case-based coaching and deliberate practice. We recommend a “search for scripts” exercise (Fig. 9.2, pink gear) where the learner is provided a specific chief complaint, asked to generate a differential diagnosis, and then asked to propose 3–5 history items and 3–5 physical exam findings that would be expected for each item on the differential [23,24,25, 29]. This exercise forces the learner to consider differentiating and distinguishing features of each diagnosis. The coach should ask the learner to compare and contrast key features of each diagnosis. This exercise should be repeated for multiple chief complaints. Additionally, coaches should employ an exercise known as “highlight key features” (Fig. 9.2, pink gear) [23,24,25, 29]. In this exercise, a learner is provided comprehensive written H&Ps for unfamiliar patients. The learner is asked to literally highlight key features of the history and physical exam while reading the note from beginning to end. This exercise asks the learner to identify differentiating and distinguishing features of a case while simultaneously considering multiple diagnoses.

Problem Representation (Fig. 9.2, Orange Gear)

Ethan: I still receive some pushback from other services when calling consults. Maybe I am not communicating effectively?

Dr. Williams: It sounds like you may struggle with problem representation.

Successful development of an accurate and concise problem representation is reliant upon a strong repository of illness scripts in the learner’s long-term memory and a solid base of biomedical knowledge [17]. There are several targeted exercises for learners who struggle with problem representation. We recommend “reversing the presentation” [23,24,25, 29], a technique in which the learner begins the oral presentation at what is typically expected at the end, with the assessment to prime the coach for feedback on selection of subjective and objective data. This allows the learner to proactively support their initial assessment. Second, learners should be asked to explicitly create and state their problem representation for each patient. A thorough problem representation should answer three questions [30]:

• (a) Who is the patient? Include pertinent demographics and risk factors

• (b) What is the temporal pattern of illness? Length (acute, subacute, chronic) and tempo (stable, progressive, resolving, intermittent, waxing and waning)

• (c) What are the key signs and symptoms?

The learner should also be asked to refine their problem representation once new clinical data are collected or revealed in various case-based scenarios.

Hypothesis Refinement (Fig. 9.2, Green Gear)

Dr. Williams: Ethan, your presentations are greatly improved. But now you seem to be presenting expansive differential diagnosis lists even after the diagnosis is relatively certain.

Ethan: Yes, now that you mention it, I feel that I am now commonly considering many more diagnoses than I did previously. I thought that was a good thing.

Dr. Williams: It is, but the next step is to focus in on a few most likely diagnoses.

Learners sometimes assign diagnoses to individual pieces of data, but fail to consider the pattern in the data. This behavior emphasizes how knowledge is necessary but not sufficient for strong clinical reasoning, because this behavior can lead to over-testing and overtreatment when the clinician has a low tolerance for risk and uncertainty. An effective targeted exercise to address this phenomenon is known as “identify findings that matter” [23,24,25, 29], which asks the learner to identify findings that have the biggest impact on increasing or decreasing the probability of certain diagnoses. This helps them build more robust illness scripts.

Another exercise is to have the learner assume the role of a patient [23,24,25, 29]. The learner then describes how they would convince the coach (role playing a clinician) of a specific diagnosis in order to force prioritization of clinical details.

Working Diagnosis (Fig. 9.2, Blue Gear)

Ethan: After all of the negative feedback that I have received, I guess I am afraid getting it wrong.

Dr. Williams: Ethan, this a common response. But you have worked hard, and your skills are improving. Now we need to work on building your confidence in your approach.

Ethan: Are there any exercises that I can use to check myself once I select a diagnosis so that I am less prone to bias and cognitive error?

The coach can have the learner practice visual diagnosis using images, video, or bedside findings to enhance pattern recognition [23]. In addition, common cognitive biases include anchoring, confirmation, availability, and premature closure. Structured reflection on each step of a clinical case and engaging their metacognitive awareness may help learners prevent reasoning errors. More specifically, structured assessment of fit, a systematic procedure of reflective reasoning, can counteract bias [31, 32]. In this exercise, the coach asks the learner to list findings in support of the diagnosis (which also may result in confirmation bias), findings against the diagnosis, and findings expected for the given diagnosis, but not present.

Management Reasoning

Dr. Williams: Let’s spend some time on management reasoning so that you select the correct tests and treatments for your patients. This should enhance confidence.

Creating management plans can be difficult, especially for early learners who lack context or significant clinical experience. Use of a management script template provides a scaffold, forcing the learner to consider all of the potential management options for a given diagnosis [22]. A sample management script template is given below (Table 9.2).

Table 9.2 Management script template worksheet

Next, the coach should ask the learner to select which interventions to perform (i.e., management option selection) based on patient-specific characteristics.

The management script template can be particularly effective for coaching learners who struggle with urgent clinical encounters where the clinician is typically required to make management decisions prior to having a refined differential diagnosis. In these cases, the patient’s clinical response to management interventions, specifically testing and treatment, may iteratively guide prioritization of the differential diagnosis. Use of the management script template allows learners to practice delineating a broad list of potential management options from which to select interventions. Coaches can discuss with the learner the benefits and risks of each management option specific to a given patient. Improving management reasoning using this exercise can both improve the efficiency of decision-making in urgent clinical encounters and guide hypothesis generation.

Continuing Observation and Reassessment

Lessons learned through direct coaching are ideally fed forward, with the learner’s permission, by the clinical reasoning coach to faculty evaluators for use during subsequent direct observation on scheduled clinical rotations. Equipped with data from the global and targeted appraisal, a coach can gather real-time feedback from these clinical rotations, creating a dynamic process of coaching and feedback (Fig. 9.3) [4].

Fig. 9.3

A tool for obtaining feedback from direct observation. Use this communication tool to obtain informed feedback from continuing direct observation of the learner who has undergone targeted coaching

In most cases, learners benefit from ongoing practice with the skills developed with the coach. All of the exercises summarized earlier can be used in the clinical learning environment, under the supervision of peer evaluators (i.e., supervising residents or fellows) or the attending evaluator of record. This practice tends to promote more frequent direct observation of the learner and provide structure and impetus for more regular, higher quality, formative feedback. In the authors’ experience, summative evaluations in which the evaluator has participated in these exercises are generally more substantive and, often, more positive.

Conclusion

The approach and tools described here are derived from the latest research on the teaching and coaching of clinical reasoning. The reasoning process is inherently complex, and we recognize that many educators were never explicitly taught a process for clinical reasoning. For both educators who are undertaking remediation efforts and learners in need of additional coaching, we have designed a relatively linear approach that we have found to be highly effective. This coaching process works best when the learner takes ownership of their own educational development, and the learning becomes more self-directed (see Chap. 4). One of the main tenets of adult learning theory is that adults learn best when they are actively engaged in the learning process and self-direct their own learning goals and activities [33]. In the authors’ experience, learners who learn to gain comfort in feeding forward, soliciting feedback, bringing this feedback back to the coach, and continuing to self-reflect and identify areas for continued work, are most likely to succeed.