Introduction

The terms outcome based education and competency based education describe educational models, which differ in detail but share the assumption that medical education should be guided by predetermined outcomes. Since the introduction of OBE (we use this abbreviation to cover both outcome and competency based education for reasons explained later) to undergraduate medical education more than a decade ago, the approach has been adopted internationally and is now widely used. Today it seems that the historical roots, theoretical assumptions, and evidence-base are very much taken for granted in the medical education community. Not every author has been convinced, however, that OBE provides a comprehensive solution for the future of medical education. Norman (2006), for example, asked readers of Advances in Health Sciences Education: “How strong is the link between teaching, learning, and performance outcomes?” Malone and Supri (2012) wrote that experience from 40 years’ suggests that the uncritical application of OBE to medical curricula may not achieve the intended aims. Fernandez et al. (2012) demonstrated that the deceptively simple term ‘competency’ has been interpreted in importantly different ways, a point we return to later in this paper. We set out to answer Norman’s question about the strength of the link between teaching, learning, and learning outcomes in undergraduate medical education by reviewing not only recent empirical research, but also historical roots and conceptual frameworks that underlie the OBE movement. This review, therefore, has three parts: First, we trace the movement back half a century, identifying why OBE was first recommended; second, we review underlying conceptual frameworks and theories; and third, we examine the available empirical evidence base.

Review of historical landmarks in the evolution of OBE

The first cycle of advocacy and critique—the 1960s through to the 1980s

In 1957, the Soviet Union launched the first satellite—Sputnik I—into orbit around the Earth. Realisation that the USA had fallen behind in the space race sent shock waves through American society, which held its educational system accountable for this failure (Hodge 2007). As a result, large sums of money were directed into education and there was vigorous federal intervention in education and training. The ensuing debate and research ignited the development of OBE and its implementation in schools and teacher training. The theoretical orientation of early OBE was towards behaviourism as represented in the works of experimental psychologists like Watson, Pavlov, Thorndike, and Skinner, whose legacy was a focus on observable behaviours. The basic idea of emphasizing educational objectives had been given its definitive form by Tyler as early as 1949 (Tyler 1949). He argued that curriculum design should be determined by explicit objectives expressed in terms of changes the learning was supposed to produce in the behaviour of students. Tyler was followed by Bloom (Tyler’s former PhD student), who developed a taxonomy of educational goals in the cognitive domain (Bloom 1956). He classified them into knowledge, skills, and attitudes, and stressed that they should be communicable. It seems that the first wave of advocacy OBE came to a head in the early 1970s with Mager’s instructional objectives (Mager 1997), Bloom’s endorsement of mastery learning (Bloom 1968), and Gagné’s work on instructional design (Gagné and Briggs 1974).

By the mid-1970s, the tide was turning against the behavioural objectives curriculum model. Stenhouse was a very influential critic, who rejected the view that the only way to organise a curriculum was to pre-specify outcomes in terms of measurable changes in student behaviour (Stenhouse 1975). Whilst learning objectives concerning factual knowledge and simple skills made good sense, he argued that educational processes, which influenced the development of values, insight, and judgement, could not be subsumed into a curriculum model that focused strictly on behavioural objectives. When students learned affectively, socially, culturally, aesthetically, or ethically from experience, it was not possible to specify goals or assess them objectively but that did not mean such types of learning were unimportant. By placing more emphasis on optimising students’ learning than on measuring outputs or outcomes, Stenhouse (1975) was emphasising education (literally, from the Latin origins of the word, drawing learning out from students) as opposed to training (putting intended learning outcomes into students).

A revival of OBE began in the 1980s (Spady 1988, 1994), which was not based on any new theoretical insights and adhered closely to the behaviourist principles of the earlier competency movement. It was in some respects more restrictive, particularly regarding affects (attitudes, emotions, and values). Spady (1994, pp 55–60) acknowledged their importance but regarded them as preconditions for outcomes rather than outcomes in themselves. He called affects ‘goals’, which he distinguished from ‘outcomes’ because they were not directly observable and could not, therefore, be included in the specification of an outcome-based curriculum (Spady 1988, 1994, p55 et seq). So, the revival of OBE specifically excluded affects as learning outcomes.

Second phase of advocacy—the end of the twentieth century

Inspired primarily by Spady, Harden et al. (1999) published a report describing OBE as “a performance-based approach at the cutting edge of curriculum development (that) offers a powerful and appealing way of reforming and managing medical education” 1999 was, according to Albanese et al. (2008), “pivotal in the evolution of the competency movement” and not just because of the advocacy of Harden and colleagues. Two other important events took place: the Medical School Objectives Project (MSOP) of the Association of American Medical Colleges (AAMC) produced a report advocating competency based education; and the Accreditation Council for Graduate Medical Education (ACGME) and the American Board of Medical Specialties jointly agreed on six competencies for certification. Only 1 year previously, the groundbreaking Educating Future Physicians for Ontario Project that sparked the development of the internationally widespread CanMEDS competency framework had also been published (Neufeld et al. 1998).

Harden and colleagues wrote that OBE was a visionary new approach to medical education because it emphasised the type of doctor produced rather than the process of education (Harden et al. 1999). Curriculum content and structure, teaching and learning activities, educational environments, assessments, and programme evaluation followed logically from an unambiguous outcome specification. In the words of Harden et al. (1999): “OBE has two requirements. First that the learning outcomes are identified, made explicit and communicated to all concerned (…). Second, the educational outcomes should be the overriding issue in decisions about the curriculum.” OBE, the argument continued, had many benefits: relevance; controversy; acceptability; clarity; provision of a robust curriculum framework; accountability; encouragement of self-directed learning; flexibility; guidance for assessment; participation in curriculum planning; curriculum evaluation; and continuity (Harden et al. 1999).

Implementation of OBE followed by critique—into the new millennium

Reports and publications over the first decade of the new millennium showed it was possible to meet the first requirement of OBE, to define learning outcomes. The Scottish Deans’ Medical Curriculum Group, for example, defined 12 key outcome domains, which were subdivided into 86 learning outcomes, leaving individual schools to break them down further according to the needs of their curricula (Simpson et al. 2002). The learning outcome “take a history from patients, relatives and others”, for example, included a next level of detail concerning: All age groups; local/multi-ethnic factors; a wide range of different contexts; and a patient-centred, sensitive, structured and thorough approach with demonstration of principles of good communication. Simpson et al. (2002) concluded that even schools with very different styles of curriculum could reach consensus on learning outcomes. Other groups reached the same conclusion. The CanMEDS competency framework was extensively adopted, not only in Canada, but around the world (Frank and Danoff 2007) and the Tuning Project for Medicine also succeeded in developing a pan-European learning outcomes framework (Cumming and Ross 2007). Concern that medical schools would have to reconcile overlapping or competing outcome frameworks was addressed in a report published in 2007, which successfully cross-referenced The Scottish Doctor and the UK General Medical Council’s framework, Tomorrow’s Doctors (Ellaway et al. 2007). Critical opinions were also to be heard, however, at this time when OBE was being so avidly adopted (Prideaux 2004; Rees 2004; Talbot 2004). Rees (2004), quoted Grundy as having argued that a focus on the outcomes rather than educational processes of curricula was antithetical to good educational practice; she argued that OBE was a disempowering educational strategy and she proposed cooperative curriculum control as an alternative.

Flexner centenary recommendations—2010 and onwards

In 2010 an influential report by Cooke et al. (2010)—The Carnegie Foundation’s Flexner centenary report, concerning the future of medical education in the USA -endorsed a competency based approach to curriculum development as a gold standard. The report linked public concern about patient safety and minimum competencies of medical graduates to a general lack of defined outcomes for medical education programmes (Cooke et al. 2010). It recommended widespread adoption of OBE and asked programme leaders to set clear, progressive expectations of learners. Adoption of OBE would better equip medical graduates to respond effectively in complex situations and efficiently continue to expand the depth and breadth of the requisite competencies. That, according to the report, could not happen without assessment, which must determine that learners had achieved benchmarked standards of competence. Assessment had to be integrated and cumulative, cover professional formation as well as formal knowledge and clinical performance, and include formative feedback, guidance, and mentoring as well as summative certification of competence at each level of development.

Whilst strongly endorsing OBE, the Flexner centenary report noted two important, unresolved issues. The first was how to define and assess humanism, accountability, and altruism, which are unique to individuals and thus difficult to reconcile with an OBE framework. That concern is reminiscent of Stenhouse’s critique over 30 years earlier (Stenhouse 1975), which Spady (1994) resolved by excluding the affective domain from the formulation of curriculum outcomes. Affective elements of professionalism, the Flexner centenary report stated, would “need to be kept in focus and reflected on over time.” The second issue was how to promote excellence when competencies are targeted at “good enough” performance (Talbot 2004), captured in Sir John Tooke’s aphorism, “Good enough is not good enough” (Tooke 2008).

Hodges (2010), also, identified and discussed important issues that OBE left unresolved. He wrote that the Flexner centenary recommendations were being made because a lack of clearly defined and assessable outcomes made it impossible for individual trainees and medical education leaders to be held accountable to society. OBE seemed an obvious remedy because it provided precisely what was missing; however, standardization of learning outcomes, according to Hodges (2010), is based on a “production discourse”, which is dominant in industry and appeals because it promises to reduce education costs and training time. Individualization, however, is a substantial distance away from standardization. For doctors-in-training to learn tolerance of ambiguity, agility in the face of complexity, habits of mind such as curiosity, innovation, and a commitment to lifelong learning, curricula have to be rich in practice and feedback and provide continuous formative assessment and a stepwise developmental approach. Hodges (2010) further pleaded that “in fashioning discourses about future physicians, we should try to use language and concepts that emphasize the growth and development of people.”

It is striking how vigorous and alive the discussion of OBE remains more than 40 years after the invention, implementation, rejection, revival, advocacy and critique cycles began. Recently, a discourse analysis of the early development of the CanMEDS Roles highlighted the contextual and negotiated nature of their construction (Whitehead et al. 2011). It was also suggested that the supporting non-medical expert roles perhaps play the part of ‘armour’ for the authority of medical expertise under threat. A commentary from the Royal College of Physicians and Surgeons of Canada retorted that this was a significant misinterpretation of one of the world’s most widely used competency frameworks (Sherbino et al. 2011). It commented, also, that the commonly used term ‘non-medical expert roles’, which caused confusion over definitions and terminology, was developed and adopted by educators and clinicians outside the initiative.

A striking feature of the discourse of OBE is that individual teachers and learners are more or less unrepresented in it, except as objects of regulation. The rationale of Cooke et al. (2010) for adopting OBE, for example, is almost entirely expressed in terms of assessment and accountability, with little direct link to teaching and learning. Hodges’ analysis of the purported benefits of OBE is, likewise, dominated by the discourses of assessment and accountability (Hodges 2010).

To summarise this review of historical landmarks in the evolution of OBE, there appears to be a more or less unbroken line of inheritance between behaviourist psychology of the 1940s and contemporary advocacy of OBE. It was widely implemented, then heavily critiqued in the 1970s for reducing values, insight, and judgement to simple behavioural objectives and for not placing affective, social, cultural, aesthetic, and ethic learning processes at the core of education. OBE was revived in the 1980s and vociferously advocated for medical education before and on into the new millennium. The revival focused on just the same purported benefits of emphasising the product of education—an unambiguous outcome specification—to curriculum design, assessment, programme evaluation, and accountability. Advocacy was followed by implementation, which re-aroused more or less the same critique, now from medical educationalists and practitioners.

Review of frameworks and theories

Having traced the historical landmarks underlying modern OBE, we now turn to the second part of this review and consider a potential conceptual framework or theory. However, before doing so, problem with terminology have to be reconciled.

Review of terminology

Whilst outcome based and competency based education have in common a main focus on the product, as opposed to the process, of curricula, any differences between them are subtle. Shumway and Harden (2003) wrote that the learning outcomes specified by bodies like MSOP, AAMC, ACGME, CanMEDS, and the Scottish medical schools “have similarities and embrace a similar set of competencies or abilities.” According to Albanese et al. (2008), the difference between outcomes and competencies is in the words “want” and “need”. An outcome defines what skills and qualities we want students to have, whereas a competency is a determination of what skills and qualities doctors need to have to care for patients.

The Tuning Project (Cumming and Ross 2007) distinguished between outcomes, objectives, and competencies in this way: Learning objectives are set and described by teaching staff and relate to a particular component. Learning outcomes are described with a hierarchy of levels, with the top level consisting of large domains. Within those domains, more detailed outcomes are described to a level that is useful for blueprinting assessment programmes. Finally, competencies belong to students or graduates rather than teachers. The competencies of someone who has successfully completed a degree programme should be at least equivalent to the prescribed learning outcomes. The terms are used more or less interchangeably in the Tuning Project.

A recent review by Fernandez et al. (2012) concluded that health science educators agree that competence is composed of knowledge, skills and other components, though they found clear differences between different authors in how they used the term. Having found no essential difference between outcome and competency based education in practice; this article has aligned itself with Hodges (2010) in using the term OBE to refer to both. It has also standardised on ‘competency’ (rather than competence) except when quoting an author who uses the alternative term.

Review of learning theory

OBE, it may be concluded from the analysis thus far, links naturally with assessment but not so naturally with teaching and learning activities. It seems there is a gap in OBE between defining outcomes and supporting the teaching and learning activities of medical curricula. We are not alone in making that observation; Prideaux (2004) and Norman (2006), before us, questioned the nature of the link between learning outcomes and teaching/learning. Another gap is between the roots of OBE in behaviourism, which could be considered an outdated theoretical orientation, and modern learning theory.

An approach that seemed able to help us think more widely about affordances and limitations of OBE was Biggs and Tang’s constructive alignment curriculum framework, which emphasises teaching/learning processes as well as assessment and aligns them both with learning outcomes (Biggs and Tang 2007). Like OBE, constructive alignment calls for intended learning outcomes to be stated clearly and unambiguously. Like OBE, also, it puts a strong emphasis on assessment systems that call for learners to demonstrate proficiency in the intended learning outcomes. Teaching–learning activities, from a constructive alignment perspective, should be planned in such a way as to increase the likelihood of students being assessably proficient in the learning outcomes. The approach has been predicted to foster a deep approach to learning, since it encourages students to engage with appropriate activities and reach desired levels of performance. Thus, it broadens the theoretical foundation of OBE to include learning; it can be argued, however, that Biggs and Tang’s theory also narrows learning to observable behaviours and leaves out elements of learning related to affects and professional attributes. Their framework may be characterised as a constructivist one. The fact that OBE could be reconciled with a constructivist theory is not, however, convincing evidence that OBE has moved beyond its behaviourist origins.

Having looked in vain for a contemporary learning theory, which could reconcile society’s demand for doctors to show complex and affectively-laden personal and professional attributes with OBE, whose focus is on assessing demonstrable aspects of competence, it seemed logical to go back and scrutinise the basis on which OBE has been so strongly recommended in recent pivotal publications. Cooke et al. (2010) quoted authority statements of bodies like ACGME, but did not cite theory or empirical evidence. Harden et al. (1999) supported their advocacy by reference to Spady (1988, 1994) who, it should be remembered, said affects could not be learning outcomes because that conflicted with his behaviourist stance. So, no new theory lay behind the second cycle of advocacy and OBE is no closer to providing a curriculum framework for the affective and complex elements of professionalism than it was in the 1970s.

Review of empirical literature

Review question and approach

The foregoing conclusion that OBE was advocated primarily because of its ability to support assessment of skills and knowledge and that, on theoretical grounds, its ability to support the development of complex personal and professional attributes is unclear led us to turn to the empirical literature in order to frame the exploratory review question: How do predetermined learning outcomes affect learning and teaching in undergraduate medical education? Our approach was to search research publications since a review by Carraccio et al. (2002), which had summarised evidence up to the millennium and concluded that further research effort was needed to find if OBE produced more competent physicians.

Method

Two major search engines—Scopus and Cambridge Scientific Abstracts (CSA) natural and social sciences—were chosen because they include most large, relevant databases like Medline, Embase, Web of Science, ERIC, PsychInfo. We also searched CINAHL, which is not covered by those two search engines. All searches covered the period 1999–2010. Because of inconsistency in how databases classify performance outcomes, we conducted a simple keyword search in title, abstract, and descriptors. Our strategy was to combine context and population AND concept AND impact variables. Context and population were defined as undergraduate medical education, medical students, or medical faculty; concepts were defined as learning outcomes, outcome-based education, competence, competencies, competency based education, learning objectives, goals, or checklists; impact variables were teaching or learning. The final search syntax was too large and complex to be reproduced here but is available on request.

‘Hits’ were imported into Refworks software (Proquest, Bethesda, USA). 4,356 hits in Scopus were sorted by relevance and the 1,000 most relevant included. 1,400 CSA hits from natural sciences and 469 hits from social sciences were all included, as were all 161 hits from CINAHL. We also hand-searched our own reference databases to increase the sensitivity of the search. In all, 2,168 articles remained after manually deleting duplicates and papers without abstracts. Studies that met all the following four criteria were included:

  1. 1.

    The paper concerned undergraduate medical education, medical students, or medical faculty

  2. 2.

    And investigated learning outcomes as defined in OBE

  3. 3.

    And provided some verifiable, observational data from an experiment or observation of the effects learning outcomes

  4. 4.

    And reported the influence, effects, or impacts of learning outcomes on teaching or learning.

Based on these four criteria, one author (TD) excluded 2,126 obviously irrelevant hits by title and abstract. A second researcher was not involved in this phase because we have found there is little to be gained by having more than one person screen out the large number of completely irrelevant papers that have to be excluded in the first round of an education review (Dornan et al. 2007). The first iteration left us with 42 papers to review in full text. In the following iterations, all papers were reviewed by two authors and inclusions and exclusions decided by consensus. All 42 of them fulfilled criterion 1; 30 of the 42 papers were excluded on criteria 3, or 4, leaving 12 papers. Four of the 12 papers were excluded in the third iteration on criterion 2, because they were based on lists of topics that structured teaching–learning activities, but did not indicate levels or the performance expected of students, and could not therefore be classified as sets of learning outcomes. Eight of the 12 papers included outcomes with clear, unambiguous, observable performance elements and were included in this review.

Results

The eight included papers are presented in Table 1.

Table 1 Analysis of the eight included papers
Full size table

Study designs

Seven of the papers concerned students and one concerned teachers. One study was conducted in a skills laboratory setting, five were conducted in the context of clinical courses, and two were conducted in school level settings. One study was qualitative; one study was a controlled experiment with 2 groups of 8 students followed by performance assessment; one study was a controlled complex intervention comparing questionnaire-based self-assessment of respectively, 100 and 351 students; one study was a pre-post controlled complex intervention comparing MCQ test-scores from 52 and 42 students, respectively; one study was an intervention involving 81 students followed by a comparison of test scores to questionnaire responses; finally three studies were uncontrolled complex interventions involving from 54 to 120 medical students followed by questionnaire-based self-assessment.

Findings

Learning outcomes, teachers, and teaching

Very little research explicitly addressed how learning outcomes influenced medical teachers’ teaching. The exception was a single qualitative interview study, which found that school level learning outcomes influenced teachers’ course planning while lesson level outcomes influenced their teaching (Bolander et al. 2006). On the negative side, the study indicated that teachers with a richer view of expertise, who encouraged learning by participation, might be limited by a focus on outcomes.

Learning outcomes, students, and learning

Some students liked to be given learning outcomes, which were shown in one study to promote active learning in a laparoscopy trainer, although the number of participants was small and the result was not statistically significant (Gonzalez et al. 2004). Outcomes led students to be better prepared for teaching sessions (Waydhas et al. 2004) and resulted in higher scores on post-clerkship tests (Kuo and Slavin 1999). In other studies, learning outcomes were part of complex curriculum interventions, which made it hard to determine their independent contribution. These studies showed increases in students’ self-evaluated competence and confidence (Brody et al. 2003; Jacobs et al. 2005; Lai and Ramesh 2006). There was one important, negative finding from a study, which asked students how they used course objectives, exam blueprints, and the content of teaching sessions for examination preparation (McLaughlin et al. 2005). Only 34 % of students used the course objectives and they were significantly less likely to have above-average test scores.

Discussion

Principal findings and meaning

Widespread advocacy of OBE, we have shown, can be traced back to the launch of Sputnik. Its theoretical origins were in the behaviourist psychology of Tyler and his immediate successors. By the turn of the millennium, advocacy for OBE had moved from being supported by theoretical argument to appeals to common sense, backed up by authority pronouncements from regulatory bodies. The last 50 years have seen two clear cycles of advocacy, then critique, of OBE. In the last decade OBE has been advocated and implemented in undergraduate medical education, but this has not been followed by substantial research on the impact of learning outcomes on teaching and learning medicine.

Threats to validity

The validity of our conclusions rests primarily on the three-pronged approach we adopted. The historical review had to be selective because a comprehensive review of the history of OBE would be a huge task. Likewise, our review of theory was necessarily limited. That means important facts and lines of argument will, inevitably, be missing from the analysis. Even the third component—a systematic literature review—may not have been complete. The paucity of evidence we found might be seen as evidence of its incompleteness. Evidence reviews in the domain of medical education, however, consistently find that what is taken to be ‘normal’ rests, in reality, on little empirical evidence. Whilst acknowledging that informative evidence might have been missed, it seems unlikely we missed enough strong evidence to invalidate our findings. Everything we found, however, triangulates around the interpretation presented in the previous section. It is not intended to be the last word on the topic, or even the only possible interpretation of the evidence; far from it, the Implications section suggests next steps that might reasonably be taken in the light of our findings.

Relationship to other publications

Our analysis is confined to English language publications. Scandinavian and German language publications about education (Gundem and Hopmann 1998) present a rather different discourse of education, epitomised by the word ‘Bildung’. Education, from that point of view, extends far beyond acquiring skills and knowledge. German language authors such as Klafki (1995), with a starting point in the humanities, clearly focus on ideal goals like developing ability and willingness to critique and self-critique, to argue, to be empathic, and to think contextually and coherently. In this framework, education and curriculum mainly concerns Bildung (the process of striving towards such ideal goals); acquiring knowledge and skills are tools in that process, not the endpoint learning outcomes in themselves.

This review is complementary to the critiques of Prideaux (2004), Rees (2004), Talbot (2004), and Tooke (2008), which address the potential of OBE to disempower learners and set adequacy rather than excellence as the goal of medical curricula. Whilst our historical review and theoretical critique drew on education scholarship as a whole, our evidence review was confined to just one part of the medical education continuum—undergraduate medical education. The recommendations of Harden et al. (1999) and the Flexner centenary recommendations (Cooke et al. 2010), however, apply to medical education as a whole and the latter are supported by an exhaustive literature review. Rather than refer selectively to publications from graduate medical education to extend our argument to it, we leave the effectiveness of OBE in graduate medical education for future researchers to examine, as discussed below.

Implications

For practice

Rather than leading to a position that OBE is or is not appropriate for undergraduate medical education, this review indicates that it may be more appropriate to some than other aspects of it. It follows from our analysis that OBE provides a logical basis for blueprinting assessments of competence, particularly those that assess observable behaviours. On the other hand, even the theorist on whose writing the recommendations of Harden et al. (1999) rest said affects cannot be reduced to observable behaviours. It seems illogical, therefore, to subsume personal and professional attributes that develop through education into the same process as competences that are developed by training. To do so would, potentially, be to deny the importance of relationships in medical practice. Schön (1984) famously referred to practice as “messy”. OBE lends itself to tidying up those parts of practice that can be tidied up, but its danger is that inappropriate application could devalue those parts of the essence of practice that are inherently messy; particularly, complicated human ones that underpin effective therapeutic relationships.

For scholarship

The single most pressing scholarly task identified by our review is to examine OBE from theoretical perspectives other than behaviourism; cognitive and social theory, for example. Research designs answering the question “does OBE work?”, we suggest, will be of little use because the field is just too complicated for that. Rather, “clarification” (Cook et al. 2008) studies answering the question “how does OBE work, for whom, and in what circumstances?” are sorely needed and lend themselves to action or design based research conducted within curriculum implementations. We know little about what OBE is good for; what the connections are between learning outcomes and teaching; when outcomes are helpful for teachers and when not; when and how they are useful for self-directed learning.

Conclusion

Our review shows Harden et al. (1999) were right when they said: “Although OBE has obvious appeal, research documenting its effects is fairly rare.” As stated by Harden (2007), many “peacocks” have specified sets of outcomes but few “beavers” have used learning outcomes as a basis for curriculum related decisions (Harden 2007). Whilst agreeing, we suggest that beavers who draw on theory as well as authority statements to craft and carefully evaluate curriculum revisions will be the ones to move the field, as well as their own curricula, forward.