Introduction

Evidence-based orthopaedic surgery (EBOS) is a process by which the best available scientific evidence is taken into account along with a surgeon’s experience and the patient’s preferences when making treatment decisions [8, 16]. Applying EBOS principles in practice thus implies that surgeons know the relevant literature. Knowledge and information channels in orthopaedics are growing rapidly, with currently more than 100 orthopaedic journals available—and their number increases every year [11]. Keeping up to date on this information would require a surgeon to spend many hours per week just reading papers. Since nobody involved in daily clinical work is able to read this amount of new publications, several approaches have been developed by readers and by providers of knowledge [9, 18]. Systematic reviews (including meta-analyses), short literature overviews, selections of recommended articles, summaries in web portals, and expert opinions (based on a variety of information sources) are some possibilities to gain knowledge on a given clinical topic in a reasonable timeframe.

The most appropriate approaches depend not only on the local infrastructure (access to the Internet, online libraries etc.) but also on the awareness of these different options, and surgeons should be able to select only the most valid clinical evidence. Some knowledge in research methodology is therefore needed. Surgeons applying the principles of EBOS who are aware of the hierarchy of study designs may primarily look for studies with a high evidence level (e.g., randomized controlled trials) or systematic reviews of these studies rather than simply asking colleagues for information. However, in orthopaedic surgery, as an empiric, experience-based science, the opinions of the opinion leaders, experienced colleagues, and leading organizations still have an impact, and additional information from research organizations, training workshops or advertisements in orthopaedic journals are influential. While the need for education in EBOS principles has been identified and discussed [13], less is known about how surgeons use different channels of knowledge distribution and their behavior regarding searching for information, trusting knowledge sources and theoretical background.

The objective of this study was to document how orthopaedic surgeons of different levels of training and affiliations search for and use evidence-based information and how this practice may be influenced by their knowledge of the principles of evidence-based medicine. The results of this study should make it possible to define strategies for obtaining information.

Materials and methods

Questionnaire

A questionnaire was developed by orthopaedic surgeons and epidemiologists being trained in designing surveys to evaluate strategies for obtaining and using evidence-based knowledge in orthopaedic surgery. Special attention was focused on the surgeons’ educational level, their affiliations, how they use infrastructure and their evidence sources. In addition, we tested respondents on their knowledge of and attitude toward EBOS. The questionnaire contained the following three sections: (1) access to clinical and scientific knowledge, (2) confidence in the sources, and (3) information about the individual’s handling of EBOS. Respondents were asked to identify their access point to electronic literature databases and to describe their main source of scientific knowledge and literature; information regarding the number of scientific papers read and active literature searches was recorded. Respondents indicated their level of confidence in literature recommendations from various providers on a five-point Likert scale. Additionally, they answered a multiple choice question (eight possible answers) to describe their separate approaches to selecting scientific evidence for treatment decision making, literature review, or writing a publication. They defined the term EBOS by choosing one of four proposed definitions. Finally, they described their attitude toward the promotion and practice of EBOS using a five-point Likert scale as well as their practical application of EBOS in their daily work. The questionnaire was prepared in English, as it is the official language of the AO Foundation and the last years have shown that all course participants speak English. We pretested the questionnaire on colleagues within the AO Foundation to assess its face and content validity and made appropriate corrections before its implementation.

Questionnaire administration

The questionnaire was administered to course participants at the 78th and 79th AO courses in Davos, Switzerland, in December 2003. To increase the response rate, the questionnaire was distributed at the AOCID booth close to the entrance to the congress center and at various other sites. The respondents were rewarded with a little gift.

Statistical analysis

Categorical and dichotomous variables were summarized with percentages and continuous variables with means and standard deviations. Cross-tabulations were performed to identify associations between variables. The chi-square test (or Fisher’s exact test for small sample sizes) was utilized to compare proportions. We considered a p value <0.05 as statistically significant. All tests were two tailed. All analyses were conducted using the software Intercooled Stata 8 (Stata Corporation, TX, USA).

Results

Respondent characteristics

Of a total of 1,274 participants at the AO courses, 456 (38%) completed the questionnaire. The mean age of respondents was 41 (range 25–69, standard deviation 8.4) years; the majority (95%) were male. Among the 74 countries represented on the questionnaires, the most respondents (10.5%) came from Switzerland, followed by the United Kingdom and Italy (5.5% each) and the United States and Germany (5.3% each); 55.7% were residing in Europe, 19.6% in Asia, 9.5% in Latin America, 6.1% in North America, 2.6% in Africa, 2.2% in Australia and Oceania, and for 4.3%, the origin was unknown. Of the 452 respondents describing their level of experience, 66.4% were chief or consultant surgeons, 19.9% were senior residents (>3–6 years experience), and 10.6% were junior residents (1–3 years experience). A majority of 78.3% of the 452 respondents were involved in teaching while 70.5% of 438 respondents worked in a teaching hospital, and 63.2% of 452 respondents were involved in clinical research. Of 453 respondents, 60.3% reported themselves to be authors of one to ten publications, 26.9% of more than ten publications, and 12.8% had never published before (including nine chief surgeons). The number of publications was different with respect to the level of experience, as presented in Fig. 1.

Fig. 1
figure 1

Number of publications in relation to professional status, n=435.

Full size image

Knowledge about EBOS

Of 446 respondents, 300 had heard of EBOS. They represented 68% of 148 chief surgeons, 79% of 147 consultants, 60% of 88 senior registrars and 43% of junior registrars. There was no significant difference between those who worked in a teaching hospital and those in a private practice. However, only 45.1% of 288 surgeons who had heard about EBOS gave the correct answer: “EBOS is a process by which the best available scientific evidence is taken into account along with surgeons experience and patient preferences for making treatment decisions.” Of these surgeons, 47.2% answered not quite correctly, i.e., that “EBOS is a process by which only scientific evidence from randomized trials should be taken into account when making treatment decisions.”

Attitude toward EBOS

Of the 300 respondents who had heard of EBOS, 87.6% strongly agreed or agreed that “practicing evidence-based medicine in general improves patient care.” From 297 respondents of the surgeons who had heard about EBOS, 26.6% described their attitude toward the current promotion of EBOS as extremely welcoming and 61.3% as welcoming.

Application of EBOS

Of 295 respondents who had heard of EBOS, 21% stated that research findings were extremely useful in their day-to-day management of patients. For 68.1% it was useful, and 9.2% were not sure about it. Of 286 respondents who had heard about EBOS, 32.2% reported to practice it always in their daily work, 16.4% to practice it daily but not in emergencies, 33.6% to practice it only in difficult or controversial cases, and 10.5% not to practice EBOS at all. These results did not differ between respondents who correctly described EBOS and those who did not.

Access to clinical and scientific knowledge

Nearly two thirds of respondents mentioned that scientific publications were their main source of scientific knowledge, followed by courses and books on orthopaedics (Fig. 2). Four hundred and forty-one surgeons reported to search between 0 and 20 h/week for scientific literature (median 2 h). Of 449 respondents, 18% reported actively searching for scientific literature a few times a year, 61% monthly or weekly and 19.4% two or three times a week or daily. The main sources of scientific literature for the active searchers were electronic databases like Medline or Embase (63.4%). Of the 255 respondents who mentioned electronic databases as their main information source, 62.4% were residing in Europe, 15.7% in Asia, 10.2% in Latin America, 6.3% in North America, and 2.7% each in Australia/Oceania and Africa. Of 445 users of electronic databases, 40.2% had their access at home. Of 455 respondents, 79.3% reported reading between one and five papers per week, 11% between six and ten papers, 4.6% more than ten papers and 5.1% no papers per week. The number of scientific papers read per week differed subject to the level of experience (Table 1).

Fig. 2
figure 2

Main source of scientific evidence, n=453.

Full size image
Table 1 Number of scientific papers read per week.
Full size table

Confidence in the sources of knowledge

Regarding recommendations about useful articles, 87.6% of 411 would trust a chief surgeon, and 20.9% of 391 would trust the industry or the producers (Fig. 3). Neither the sources of scientific knowledge nor the sources of useful recommendations differed for those who correctly defined EBOS and those who did not.

Fig. 3
figure 3

Credibility of sources with respect to recommendations about useful articles.

Full size image

Discussion

Although there is an overwhelming amount of orthopaedic knowledge available, not all orthopaedic surgeons know how to use it efficiently and acquire information with the most valid evidence. Several authors have looked into publication strategies and the quality of various databases [5, 12, 15], but only a few publications exist about strategies to search, find and evaluate orthopaedic publications with respect to EBOS [6, 19]. Although the majority of surgeons claim to know something about EBOS, less than half of the respondents to our questionnaire were actually able to describe the working principle of EBOS. Apparently, the present knowledge of EBOS focuses on an increased awareness of randomized controlled trials (RCTs). So there seems to be a misconception about what EBOS is in reality. Given the paucity of RCTs in orthopaedic surgery, the application of EBOS principles may not be possible if only such studies are to be considered in the medical decision-making process. While this is certainly a desirable objective, we believe the reality is that most surgical interventions in orthopaedics will continue to be assessed by non-randomized studies in the near future, and the “best-available” evidence should be considered. Lower-ranked studies in the hierarchy of evidence, however, should be considered with critical insight [3]. Caution should also be exercised when reading reports of studies labeled as “randomized controlled trials” because reporting and methodology are not always satisfactory [1, 10].

The respondents’ attitudes to and awareness of EBOS principles remains at a high level, but their knowledge about the principles of EBOS did not appear to influence their manner of searching for scientific information or their trust in various sources of recommendations, as assumed by the initiators of the study.

With the development of the Internet, ready access to orthopaedic knowledge should no longer be a problem. Our results disproved the assumption that more difficulties would be encountered in low-income countries [20]. The respondents from such regions preferentially use the Internet to obtain medical knowledge, which is possibly a consequence of the effort of international health care organizations to improve access to medical information in low-income countries [2].

Scientific publications are still the main source of scientific evidence. Despite an ongoing debate about the quality of published articles in orthopaedic journals [7, 14] the fact that nearly 70% of respondents base their opinions on scientific publications rather than on expert opinions (less than 20%) should encourage high-quality research and publications. An expert opinion may be evidence-based if the experts build their opinion on up to date, high-quality, clinical research, but it remains sensitive to personal bias. Recommendations from training courses were more trusted than those obtained from orthopaedic congresses, probably because they are perceived as a more independent and neutral platform for knowledge transfer. Furthermore, they offer more time to teach participants and to advertise relevant medical information. This finding may also support the increased efforts in teaching, with respect to the principles of EBOS.

In line with several critical publications, orthopaedic surgeons have only limited trust in newsletters from the industry [9, 12, 15]. However, it appears that residents can be influenced by “eminence-based medicine” because nearly 90% of respondents reported to trust recommendations from the chief surgeon about useful medical articles. Hence, it seems important to investigate the basis on which such recommendations are formulated. Our results showed that chief surgeons may not necessarily have more clinical research experience than consultants or assistant surgeons, as illustrated by their number of publications, nor may they have a better understanding of EBOS principles. Nevertheless, our findings support their function as opinion leaders and highlight their roles as possible targets for communication efforts from product manufacturers. An example of how companies tried to influence opinion leaders was evident in the tobacco industry after the publication of secret documents from several tobacco companies [17]. In relation to the orthopaedic industry, chief surgeons are probably more susceptible to a conflict of interest than other types of surgeons, and their function as scientific advisors might be misused, resulting in biased recommendations being made to residents. To avoid such pitfalls, it is necessary that, on the one hand, chief surgeons are aware of their role in the opinion-building process and remain independent in their judgment and, on the other hand, that residents critically analyze every article and piece of information, no matter from whom. A profound knowledge about search strategies as well as knowledge of how to assess information is therefore crucial. All surgeons willing to apply the EBOS principles should learn strategies to identify high-quality articles—that means with a high level of evidence—and to find these articles in a reasonable timeframe.

We recognize that orthopaedic surgeons attending a course or workshop might be more interested in clinical research and EBOS than those who are not attending courses. The response rate of 38% may appear low, but despite our effort to distribute and collect the questionnaire, not all participants returned it. It is also not clear how the answers of the nonresponders would have changed the results [4].

For surgeons who have heard or know of EBOS, the majority agreed with the EBOS principles and applied it to their work. Nevertheless, there appears to be a misconception on the part of about half the respondents about what EBOS actually is. Clearly, results from randomized controlled trials are not the only information used to support decision making when applying EBOS principles, but clinical research results should also be systematically included in the decision process and assessed on their own scientific merit.

Surgeons read and trust scientific publications to gain knowledge. Electronic access is no longer a key problem. However, techniques for searching and evaluating relevant literature should be improved and more teaching of EBOS should be offered. Trust in recommendations by chief surgeons emphasizes their role as opinion leaders.