Introduction

Evidence-based medicine (EBM) and healthcare (EBHC) have received a remarkable attention during the last decade after the term had been coined in 1992 by a Canadian workgroup at the McMaster University in Hamilton [1, 10]. Hundreds of textbooks and thousands of articles in medical journals show the term “evidence-based” in their title and far more use it in the text. Even a superficial glance at these publications, however, quickly reveals an enormous range of perceptions and associations with this notion. In addition to being fashionable, EBM is certainly also one of the most misused notions in recent years. The misuse may be deliberate in some cases but often seems to stem from insufficient examination and knowledge of the underlying concepts.

EBM in its present form is, neglecting the discussion about its historical roots, a collection of concepts developed for a better integration of research results into healthcare and medical practice [7]. Complaints about the deficiencies of the interaction of new knowledge and the implementation in medical practice have a long history [2]. The modern development has been motivated by deeper insight into the mechanisms of the research transfer. Empirical research showed that in several cases, large numbers of patients were affected by morbidity or mortality, which could have been avoided if the existing knowledge would have been earlier appropriately summarized, provided and applied [26]. Three arguments are essential in this discussion: first, the percentage of medical decisions based on high-quality scientific evidence has often been estimated to be as low as 15–20% in some specialties, sometimes even lower [34]; second, the enormous time lag between the generation of knowledge and its introduction into practice has been empirically shown to be 8–10 years [2] and has even been estimated to be as high as 15 years [17], regularly confirmed by striking examples (e.g. [36]); and third, medical knowledge is, at an ever increasing speed, becoming outdated with a decreasing half-life period of about 5–45 years today, depending on speciality [13]. These figures, especially the percentage of evidence-based decisions, have been challenged by several authors and should not be taken too literally. Interpreted as order of magnitude, however, they describe an enormous problem and have alerted professionals in health care systems around the world to better meet the challenges of these trends.

Evidence-based medicine and evidence-based health care

The prevailing definition of EBM expresses the need to systematically use knowledge in decision-making:

Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence-based medicine means integrating individual expertise with the best available external evidence from systematic research [31].

Strictly speaking, these phrases are rather an appeal to improve the systematic use of knowledge than a definition because they are too vague and unspecific. For practical usability, a five-step procedure has been developed [31]:

  1. Step 1:

    Converting the need for information about a clinical problem into an answerable question

  2. Step 2:

    Tracking down the best evidence to answer that question

  3. Step 3:

    Critically appraising that evidence

  4. Step 4:

    Integrating the critical appraisal with one’s clinical experience, patients values and specific circumstances

  5. Step 5:

    Evaluating the effectiveness and efficiency in executing steps 1–4 and seeking ways to improve them

This scheme is, at least in principle, the basis for teaching and learning EBM. For the single steps themselves, tools are offered which help the busy clinician to run through this process. For building questions in step 1, the PICO rule is helpful in including the four relevant dimensions into the question that is developed from the clinical scenario: who are the Patients, what is the Intervention, what is the Comparison or Control group, and what are the Outcomes of interest [31]. In step 2, it is necessary to know where to search and how to search. Certain skills may be needed even for simple searches in Medline and the Cochrane Library; going beyond them will require extensive technical knowledge to develop a search strategy which finds the relevant trials without yielding too much unwanted results. A description of relevant sources of evidence follows below. Appraising the quality and the usability of the identified trials in step 3 uses methods from clinical epidemiology. These methods aim at assessing the validity of the conclusions of these trials; that is, the leading principle is to assess how careful the trial results were protected against potential influence of bias and chance (Table 1). Using the hierarchy of study types as described in Table 1 is under ongoing criticism because this classification is mainly based on the design of trials and partly disregards the quality of the conduct of the trials. A more general approach to grade the evidence by including the design and the conduct is currently developed [12].

Table 1 Levels of evidence (a more detailed nomenclature can be found at http://www.cebm.net/levels_of_evidence.asp)
Full size table

While EBM relates to individual patients, EBHC means using evidence for decision-making in groups of patients on a hospital or national level. Although EBHC was initially only one variant of EBM, it is today better established than EBM itself. The reason for this lies in the enthusiastic adoption of EBHC by policy-makers, who needed a scientific model for argumentation. The value of EBHC in cutting the costs of healthcare by rationalizing medicine was also highly appreciated and debated [6].

Evidence-based surgery

The daily practice of evidence-based surgery (EBS) is still limited compared with general medicine [4, 24, 30]. Only 3.4% of all publications in the leading surgical journals are randomized controlled trials (RCTs) [35]. The available evidence, therefore, is often unsatisfactory. In order to follow the principles stated above, the surgeon has to recognize the weak external evidence and carry out his profession according to his knowledge and patients’ perspectives [3]. The correct approach (literature search) will increase surgeon’s awareness of the areas where current evidence is insufficient, and further studies are needed [16]. The systematic acquisition of unbiased evidence is essential to support daily practice, and this is one of the major advantages surgeons learned after EBS was introduced more than a decade ago [25, 33].

Once a high-quality study has been found and shows the superiority of a surgical therapy, the surgeon’s individual expertise is of critical importance. Laparoscopy, for example, requires a certain training and practice to conduct surgical procedure precisely and safely with a reproducible result. If a surgeon has only been trained in open surgery and finds an article which demonstrates a clear advantage of the laparoscopic approach in a certain condition, what shall he or she do? The answer is not straightforward. Would it be correct either to learn the new approach or to send the patient to a colleague who can do the procedure?

In order to cultivate EBS for surgeons of all levels of care, the main goal for the future must be to generate the missing external evidence by conducting well-planned surgical studies within the boundaries of daily clinical routine [14, 28]. Keeping in mind that only 24% of all surgical procedures are based on RCTs [15, 18], surgery should not solve the dilemma by stepping down to lower, more biased levels of evidence. In addition, the special nature of surgery calls for a better reporting of learning curve effects in clinical studies.

Surgeons have no longer time to wait for external evidence and can change their practice from an evidence-seeking to an evidence-generating profession. If they start to treat patients on the basis of clear-cut pragmatic study protocols, they will be also more familiar with reading and evaluating articles in order to practise EBS [28]. How each surgeon is participating in this generating evidence process depends on its setting and background. Several options are possible: for example, (a) writing a systematic review (possibly based on the quality criteria of the Cochrane Collaboration, CC) to check the available evidence and sum it up in a transparent way; (b) creating study protocols in cooperation with colleagues and playing an active role as a principal investigator of a trial; or (c) participating in multicentre trials. The latter approach may be the easiest one but requires some preconditions.

Surgical RCTs are not comparable to pharmaceutical trials. It is essential to be familiar with the techniques applied, and a basic knowledge of trials is mandatory, e.g. the classification of serious adverse events. To find studies that are open for recruitment, two main databases (http://www.controlled-trials.com and http://www.clinicaltrials.gov) can currently be searched, and additional information can be retrieved from experts. Due to the recent announcements of the International Committee of Medical Journal Editors in May 2005 (http://www.icmje.org), it can be assumed that an increasing number of randomized surgical trials will be registered and can be found through the available registers with a core data set which describes the trial. Surgeons should be able to identify the studies that are suitable for them and their patients.

The role of systematic reviews in research synthesis

The lay press and also some medical textbooks give the impression that medicine is driven by spectacular discoveries. Innovation, however, is no longer progressing as quickly as it was in the early days of modern medicine, when new operations revolutionized surgery. The effect of these operations was so clearly visible that there was no need for sophisticated study designs or statistics. The mere feasibility of an operation also indicated its effectiveness. Nowadays, things have changed because progress in medicine happens in an enormous number of relatively small, sometimes tiny, steps forward. Since the effects are often small, current studies usually require larger sample sizes. As many studies simply are not able to recruit sufficiently large number of patients, cumulative procedures to summarize the available evidence have gained importance.

Systematic reviews (with or without meta-analysis) have therefore become commonplace in surgical literature, although their methodology is quite complex and their results often show only the absence of any convincing evidence at all. On the other hand, objectivity of methods and transparency of results is clearly better in systematic than in narrative reviews [5]. Therefore, research synthesis is a central step to make research findings usable (Fig. 1). Systematic reviews play a key role in this process and are therefore an indispensable cornerstone of EBM [27]. Unfortunately, the methodological quality of most surgical systematic reviews is insufficient, and the results, therefore, may be misleading. A critical appraisal is mandatory, and it is likely that a number of published systematic reviews should be redone. In order to prevent the well-known problems of systematic reviews, these should generally be conducted within the CC or at least with the same methods used within the CC. Surgeons who have to face the difficulties in summarizing surgical RCTs or cohort studies especially can have an advantage if they use the offered infrastructure and help of the Cochrane Groups. Because of the methodological challenges of surgical systematic reviews, collaboration with clinical epidemiologist or other experts should be a prerequisite [9].

Fig. 1
figure 1

The “bridge” of clinical research. Evidence-based medicine (EBM) is not shown as one of the arches since walking on the bridge and transferring knowledge mean practising EBM

Full size image

Cochrane reviews of surgical procedures

Our objective for this review is to assess the available finalized systematic reviews within the Cochrane Library (Issue 3, 2004) that compare a surgical procedure to another either surgical, pharmaceutical or conservative (e.g. cast immobilization of fractures) intervention. Two authors with a methodological and surgical background (SS and CMS) independently searched Medline via PubMed in September 2004 with a standardized search strategy (“Cochrane Database Syst Rev”[Journal] AND (“surgery”[Subheading] OR “Surgical Procedures, Operative”[MeSH] OR surg*[TI]) NOT (“Anesthesia”[MeSH] OR “Delivery, Obstetric”[MeSH] OR “Pregnancy”[MeSH])). The results for eight different subspecialties were documented and compared (Table 2). Differences were resolved by discussion. Out of 77 reviews, 42 compared surgical procedures, 30 compared a surgical with a conservative non-drug treatment, and 5 compared a surgical with a pharmacological treatment. The leading subspecialty was trauma/orthopaedics with 27 reviews, and the last (with only three reviews) was paediatric surgery. In Table 3, we give an overview of the different topics covered by the reviews. The topics without any Cochrane review can be added by each subspecialty and are a suggestion rather than a priority list. Clearly, these results show the urgent need for more active surgical research within this field. Recently, similar results were obtained by another group [21]. In consistency with others [29], we recommend first to perform systematic reviews before the planning of RCTs in a certain field, if these have not been carried out already.

Table 2 Number and distribution of Cochrane reviews on surgical topics
Full size table
Table 3 Surgical topics currently with or without coverage in the Cochrane Library (Issue 3, 2004)
Full size table

Access to information: databases and other sources of evidence

There is now an overwhelming abundance of clinical trials worldwide, with a growth rate of more than 10,000 RCTs per year in Medline alone, 41,000 running studies open for recruitment and an estimated number of more than 400,000 controlled trials. These studies can be searched for and filtered out in several databases, of which the best known and most used is Medline (maintained by the US National Library of Medicine), mainly because access is free. A competitor with similar characteristics is Embase (published by Elsevier), which is said to be superior for pharmaceutical trials and slightly quicker in including new trial publications. Its use, however, is seriously inhibited by the costs per search, limiting its use mainly to pharmaceutical companies. Several other databases focus on specific specialties or languages, e.g. PsychLit, CINAHL or LILACS, but are not very relevant for the average surgeon.

The most comprehensive single source of evidence on the effects of healthcare is the Cochrane Library. It is maintained by the CC, a large international not-for-profit network of professionals of health care systems and consumers of health care, with the overall aim to provide up-to-date information about the effects of health care. The Cochrane Library contains six regularly updated evidence-based healthcare databases, with its core piece comprising 2,435 finished systematic reviews and 1,606 protocols of planned reviews. These reviews are supplemented by abstracts of reviews from all over the world, quality assessed and abstracted by the Centre for Reviews and Dissemination in York. Often overlooked but extremely powerful, the third database is a collection of more than 400,000 citations of reports of controlled trials. This database contains all trial reports from Medline and Embase and some of the specialized databases mentioned above. A unique contribution consists of the results of systematic handsearches of medical journals which are not indexed in any of the aforementioned databases. From Germany alone, more than 16,700 trial reports of which 45% were not included in Medline have been added to this Cochrane database [19]. The remaining three databases contain methodological articles, health technology reports and health economical assessments.

Discussion

EBM has taken up the challenge to improve the integration of new knowledge into medical practice by developing new structures and mechanisms to provide results from patient-oriented clinical trials for decision-making in medical care. Improving the rational base for decisions in individual patient care is relevant for physicians and patients but is inseparable from decision-making by institutions in the healthcare system. Moreover, the systematic use of knowledge in health care has a direct impact on health research because evidence gaps are directly identified and can be used for prioritization of further research.

Fashionable catchwords like “translation knowledge into action” or “reducing the know-do gap” seem to have supported the misconception that EBM is equivalent to mechanistically applying trial results as an overall directive in patient care. This has often led to the criticism that EBM is equivalent to double-blind randomized trials, ignoring the interaction of the three dimensions which are the base of EBM: first, the external evidence, e.g. empirical data generated by trials and studies of varying levels of validity; second, internal evidence, e.g. experience from clinical practice; and third, patient values and preferences. That EBM is not a cook book medicine has been emphasized in many articles from the very beginning of the development [23].

In this context, EBM repeatedly has been accused of inhibiting medical progress. This argument seems to stem from the same misconception that interventions are only evidence-based if they are supported by RCTs. If that were the case, EBM could indeed bring medical progress to a complete stop. Taken to the extreme that everything which is not supported by RCTs must be dropped would very seriously affect today’s conventional medicine. Medicine is not and can never be based entirely on RCTs. This is so obvious that it is amazing how often the criticism of RCTs is used as fundamental criticism of the concept of EBM. Therefore, these arguments are not convincing underpinning that EBM is hostile to progress. In contrast, the contention and the discussion under the roof of EBM have led to an enormous increase of awareness of evidence gaps and insight into the need for more research into apparently trivial yet unanswered questions. Instead of being inhibited by the lack of appropriate RCTs, it seems more likely that progress could get a strong boost by exploring new interventions and procedures on a larger scale with far more patients within appropriately designed clinical studies. This would help introduce new therapies on solid ground and provide better protection against avoidable detours which, in the end, prove to be harmful.

Moving towards EBS means introducing the principles of pharmaceutical therapies where efficacy must be shown by good-quality RCTs. However, surgery shares with other non-drug interventions a higher complexity of efficacy investigations. Quality items like blinding or comparing with a placebo intervention are often difficult or impossible to incorporate. This may lead to a reduced protection against bias. Such problems have to be accepted but should not be taken as an invitation not to conduct the necessary trials on the highest achievable quality level. The classic RCT design that is commonly used today does not adequately take into account the surgeon’s expertise. The equivalence of treatments within each trial arm may represent a major problem. In an expertise-based RCT, this is recognized, and therefore, this trial type should be more often applied in surgical research [8].

Better structures and resources for high-quality clinical trials are necessary to reduce the imbalance between basic research and the evaluation of new procedures derived from basic research results. Medical progress is dependent on a strong partnership between innovation from basic research and empirical evaluation of new procedures [20].

A broadening of the evidence base by adding more high-quality studies, however, will not automatically lead to better decision-making in clinical practice. EBM has been struggling for more than a decade to find its way into clinical practice, with much more success on the system level (EBHC) [11, 22]. Certainly, a lot must be done to allow everyday routine use of evidence. Better equipped and easier to access libraries and better electronic tools with user-friendly summaries of the up-to-date evidence are necessary to support clinicians in a world of ever increasing information overload and increasing time pressure due to growing bureaucratic burden. On the individual level, training and education are needed to develop the skills to find and use evidence efficiently [11, 32]. A sufficient generation of evidence can, in a globalized world, be achieved only through sharing the workload internationally, with additional burden for non-English language countries because they have to bridge the language gap between an entirely English language evidence pool and an entirely national language healthcare sphere.

EBM is sometimes discussed as a new paradigm or a new philosophy of how to generate and acquire knowledge. This debate is misleading because it is not only specific to EBM but also to science in general. This focus detracts from what EBM basically is: an empirically based, pragmatic set of concepts to improve the generation and use of knowledge about medical procedures, based on empirical research of the translational process between the research and the healthcare world. From this perspective, the question is not whether to practise EBM, but how. As stated above, surgeons have several options to change the current practice and can play an active part moving from an evidence-seeking to an evidence-generating profession.