Introduction

It is well known that the use of complementary and alternative medicine (CAM) is increasing in the Western world (Eisenberg et al. 1998). Data on CAM use in Europe show that 20–70% of the population has used CAM at some point during their lives (Ernst 2000). In Denmark the latest study of CAM use shows that 22.5% of the population used some form of CAM within the last year while 42.2% have used CAM at some point in their lives (Ekholm et al. 2006). Moreover, researchers estimate that in Denmark there are twice as many CAM practitioners as general practitioners (Ahlin 2007).

In the ongoing debate between CAM practitioners and researchers there is no consensus as to the right way of assessing the efficacy of CAM (Hufford 2003; Morreim 2003; Barry 2006). In evidence-based medicine (EBM) the gold standard of evaluation is the randomized controlled trial (RCT). A paradigmatic RCT consists of three elements; a randomization process, a control group, and a quantifying process. RCTs, then, are trials that are controlled, quantitative, and comparative. More specifically, RCTs are trials where the participants are split into 2 randomly assembled groups, one being the control group and one being the intervention group. The randomization process is due to the fact that the two groups shall be comparable in every way except the specific intervention that is being tested. Furthermore, it is often the case, especially in the most rigorous trials, that the RCT is blinded or double-blinded, i.e. the participant, the administrator or the statistician or all involved are blinded to the distribution of the intervention. The goal of this process is to limit the possibility that the result will be biased. There are typically two approaches to RCT design, one pragmatic and the other explanatory. Explanatory trials seek to investigate the causal efficacy of a specific component of CAM, whereas pragmatic trials aim to evaluate the clinical effectiveness of a certain clinical practice as a whole (White 2002; Roland and Torgerson 1998; Godwin et al. 2003; Lewith and Little 2007).

Practitioners and researchers disagree as to whether this design is appropriate in investigating CAM (e.g. Borgerson 2005). Some commentators have argued that the reason for the failure convincingly to document the effects of CAM interventions is that CAM cannot be tested by RCTs, and that there are other, more suitable ways of gathering evidence for its effectiveness (Walach 2003; Tonelli and Callahan 2001).

Moreover, the question what, exactly, practitioners would regard as evidence of effect is under explored. Most qualitative studies so far have concentrated on the cultural or social circumstances associated with the use and practice of CAM (Cant and Sharma 1999; Saks 2002; Frank 2002). One study has explored perceptions of evidence-based medicine among traditional acupuncturists in the UK (Jackson and Scambler 2007). However, the aim of that study was to examine the salience and impact of the emphasis on EBM on practitioners of traditional acupuncture; it focused on societal circumstances rather than methodological issues, which is one of the main concerns of the present article.

The principal aim of this article, which focuses on acupuncture, is to clarify and explore the practitioners’ own understanding of evidence, including their conceptions of RCT, in order to obtain detailed understanding of a range of positions towards evidence. The present study will thus provide a platform for further investigation, and a better informed discussion of the subject in general.

Acupuncture was chosen, among the various branches of CAM, for various reasons. First, it is one of the most popular CAM therapies.Footnote 1 Second, it does not necessarily contradict or pose a challenge to conventional medicine. It is possible to explain an effect of acupuncture not only within the framework of traditional Chinese medicine but also, to some extent, within that of conventional Western medicine. Third, acupuncture is integrated into the conventional health system in several European countries. Moreover, it is among the most frequently investigated CAM therapies: over 3,500 trials of acupuncture were listed in the Cochrane Central Register of Controlled Trials in 2008. The overall conclusion in most Cochrane Reviews however, is that there is no clear evidence of effect, and most of the conclusions are uncertain due to poor methodology.

Method

Participants

The study was conducted in Denmark between April and August in 2008. In all, 12 in-depth interviews were conducted. These involved 4 medical doctors practising acupuncture, 4 acupuncturists with a healthcare-related education (other than that of medical doctor), and 4 acupuncturists with no conventional healthcare-related training were interviewed (7 female and 5 male). Interviewees were selected strategically. That is, the respective chairmen of the two acupuncture associations in Denmark–Dansk Medicinsk Selskab for Akupunktur (the Danish Medical Society for AcupunctureFootnote 2) and Praktiserende Akupunktører (Practising AcupuncturistsFootnote 3)—were contacted and asked to participate in an interview and to suggest members of their societies to be interviewed.

Procedure

The interviews were semi-structured. An open-ended interview guide was used with the purpose of inviting the respondents to pursue the themes introduced in whatever direction they chose (Kvale 1996). The interview guide was pilot-tested on a key informant (a medical doctor, experienced practitioner of acupuncture and teacher at an acupuncture school) and a pair of practising acupuncturists, and then adjusted accordingly.

The interviews focused initially on the acupuncturists’ training in, use of, and experience of practising acupuncture. On this basis a more general conversation was pursued about the interviewees’ attitudes to the effects of acupuncture and possible ways of assessing these effects, including issues relating to evidence, other CAM therapies, and the proper place of acupuncture in the official healthcare system. The questions encouraged considerations of both a normative and a descriptive nature. The quantity of interviews increased until saturation had been reached, i.e. the point at which there were a high degree of consensus among all the interviewees.

The interviews took place either at the interviewees’ clinics or at their homes, according to their wishes. They lasted between 45 and 85 min. They were recorded with a digital voice recorder and transcribed verbatim. The transcriptions were made by 2 employees from a company that specializes in transcribing qualitative interviews, both following the same transcription guide. Each transcription was carefully compared to the sound file and, where necessary, adjusted. Interview transcripts were subsequently imported as text files to ATLAS.ti, software used to analyze qualitative data.

Analysis

The interview transcripts were analyzed on the basis of Template Analysis (Crabtree and Miller 1999; King 2004; Coffey and Atkinson 1996) and Grounded Theory (Strauss and Corbin 1998; Strauss and Corbin 1994). Components of the transcriptions, i.e. words, phrases and sentences, were given codes. These were based partly of an a priori set of codes that had emerged from the background literature, and partly on grounded theory, i.e. through the identification of unexpected issues that emerged from the interviews. The a priori codes included “views on effects”, “views on research methodology”, “causal mechanism”, and “views on CAM in public healthcare”. From this an initial template (i.e. code system) was produced in which themes were grouped into a smaller number of higher order codes. This template was then applied to all the transcripts. Whenever a relevant piece of text was identified that did not fit the existing themes, the template was altered accordingly. When all the codes and themes had emerged, the final template was used in the interpretation and writing-up process.

Direct quotations from interview transcripts are included to highlight particular aspects of the themes and to illustrate specific formulations. The quotations have been translated from Danish to English with professional assistance. The interviewees are referred to as medical acupuncturists, healthcare-related acupuncturists, and acupuncturists, and by randomly assigned numbers 1, 2, 3 and 4 in each group. This descriptive apparatus was adopted partly to ensure the anonymity of the interviewees and partly in order to be able to distinguish between the groups interviewed. The same number and group-reference is used throughout for each person cited in the paper.

Results

Through analysis of the interviews two main themes were identified: (i) views on evidence, and (ii) views on RCTs. The evidence section below presents the interviewees’ general reflections on the kind of evidence required to establish knowledge of the effects of acupuncture. The RCT section explores the interviewees’ reflections on the use of RCTs in studying acupuncture.

Theme 1: Views on evidence

The data show that at least two different conceptions, or understandings, of evidence are represented among the acupuncturists: (i) an experience-based conceptionFootnote 4 and (ii) a biomedical conception with origins in EBM which focuses on RCT when effect-evaluation is examined.

Evidence from experience

All interviewees embraced, more or less explicitly, the experience-based conception, either as the ‘only’ valid form of evidence or as a supplement to the biomedical conception. The conception embraces at least 2 distinct understandings, one derived from the practitioner’s own experience with acupuncture, and the other from tradition, i.e. from the fact that acupuncture has been practised for 1,000 of years.Footnote 5

For me … the only thing that matters is what I myself … I actually don’t care about evidence. (…) Because I can hear: yes but is there less headache? Has the migraine changed? Have you had success with fertility? (…) So that’s the thing that works for me. Healthcare-related acupuncturist (1)

And when I think how many other methods of treatment I’ve seen in the course of my long life as a doctor—which have been abandoned after a while…there’s nothing scientific about what I’m saying here, but there can be no doubt that a method that’s kept going for 3–4,000 years can’t just be a con and a swindle. I mean I think you’re allowed to say: well it’s just plain common sense that tells you. Medical acupuncturist (1)

I can’t understand how people can so superciliously and arrogantly reject 1,000 of years of knowledge. We can see that it works! What the hell is the problem? I feel a bit as if…people go on too much about why it works! So long as it works, then for heaven’s sake let it work. No? Honestly! Acupuncturist (1)

The view that experience-based evidence of an effect might be the result of coincidence or some other source of error was expressed only in the MD group.

I do think I’ve personally had good results, or else I’m just good at persuading my patients that they’ll feel good when I’ve been with them. (…) It may well be that we’ve just been lucky, after all I can’t just say that with one patient, can I. Such a long time, I can’t bring it all together and do something … so it’s experience, and you can’t use experience scientifically, can you. Medical acupuncturist (2)

Evidence from EBM

The reasons for embracing, or acknowledging, the biomedical conception differed among the three groups. Almost everyone believed that RCTs might be necessary for pragmatic, political reasons, i.e. in order to be awarded research funds, and, most notably, in order to place acupuncture on the public healthcare menu and obtain official acceptance. Some considered RCTs valuable as instruments, although they were frustrated that RCTs did not demonstrate the same effectiveness that they had experienced in their practices, i.e. they embraced both conceptions of evidence, though these contradicted one another.

Yes, well, I don’t feel too good about that, but at the same time, as I said to you before, I don’t have any problem in so far as my experience (…) shows me that it works. (…) But I would very much like to be able to prove that, and I’d feel just a little better (…) if we could document that what we went and did had an effect, and we can’t by any means always. Medical acupuncturist (1)

None of the interviewees embraced the biomedical concept of evidence without reservation, and some directly rejected it. There were at least two reasons for this rejection. Some rejected biomedical evidence on the grounds that it was futile and inappropriate

God almighty! How much in our ordinary Western scientific development would have happened at all if there’d been some idiot sitting there when Democritus suggested that nature consisted of divisible atoms…then there’d be someone standing right there saying: “yes, but this isn’t evidence-based”. Medical acupuncturist (3)

Some interviewees implicitly expressed a wish to be able to translate their experience into some form of biomedical evidence. This view was represented only in the MD group and the healthcare-related group.

When you’re sitting there as a GP you’re incredibly busy, and you have loads of good ideas like: yes, we should register this and so on. I made my own little statistical system (…) where I tried to follow up in a very primitive way (…) among other things by using various pain scores (…) So there I did try, but I haven’t been able to produce any scientific results myself, because you don’t have time as a GP. Medical acupuncturist (1)

Some of the interviewees claimed that they did not actually care about evidence in the biomedical sense, and again referred to their experience-based conception of evidence.

I think it’s exciting that someone can be bothered [to do it], but I’ve long since stopped bothering with that. For me…the only thing that matters is what I myself … I actually don’t care about evidence. (…) Because I can hear: yes but is there less headache? Has the migraine changed? Have you had success with fertility? (…) So that’s the thing that works for me. Healthcare-related acupuncturist (2)

As the above quote illustrates, some practitioners may express indifference to biomedical evidence by not placing any epistemological weight on RCTs, while at the same time respecting evidence that is based on experience. Others may show their indifference to biomedical evidence by individualizing the notion of evidence, i.e. by taking the view that patients themselves would choose the most effective treatment if they were allowed to.

Well…I think perhaps what should happen is that…that people should be free to choose. And likewise you’d have to reckon with the fact that what people choose is what helps them (…) Instead of being…oh…cleverer than them on their behalf, telling them what’s right, you know? Healthcare related acupuncturist (3)

Others had reservations about the feasibility of conducting RCTs in their field. This will be the subject of theme 2 below.

Theme 2: Views on RCTs

The data show that the acupuncturists were in general sceptical about the use of RCTs in acupuncture research. In particular, they articulated two reasons for thinking that RCTs were problematic in this field. First, they pointed to pragmatic or instrumental problems, including the difficulty of using control groups and of blinding, and especially double-blinding, trials.Footnote 6 Those who referred to these problems included both those interviewees who considered that RCTs carry epistemological weight and those who did not. Second, some gave value-based reasons for their positions, i.e. they expressed the view that the biomedical design, quite simply, would remain incompatible with the proper practice of acupuncture even if it were possible to overcome instrumental obstacles.

Pragmatic reservations

The first set of reasons for scepticism about RCTs is pragmatic and instrumental, in the sense that it relates to the difficulty of conducting RCTs in the field of acupuncture. The particular obstacles the interviewees mentioned concern the difficulty of establishing control groups and the problem of blinding procedures. The problem with establishing a control group here is that placebos cannot be used: plainly, a person receiving acupuncture will know whether she is receiving the actual treatment or just a sham version. Nor, according to some interviewees, could a control group be established by giving acupuncture in non-acu points, since some claim that no matter where you insert a needle you will get a result. Given this, if members of the control group were given acupuncture in non-acu points, they would also display effects of some sort.

As regards the difficulties connected with the blinding procedure, the reasons put forward were either that it is simply impossible to (double-) blind an acupuncture trial; or, again, that it is possible to obtain results no matter where the needle is inserted.

But then the problem is that no matter where on earth you put the needle, something happens, and therefore it’s difficult to do double-blind studies and all that type of thing, you know. Medical acupuncturist (1)

But there already it’s actually completely impossible, because if I were to do a double-blind study on acupuncture, I’d have to be ignorant about whether I was giving correct or incorrect treatment (…) it can never be blind from the practitioner’s side (…) Western research has to be double-blind, in order that as a doctor I can’t go in and influence [the results]. Because if I know that I’m giving a sham treatment, then consciously or unconsciously my attitude could influence the result. And people are so afraid of placebos in Western medicine. Acupuncturist (2)

However, one view was that there was no problem in principle in conducting RCTs; rather, the problem lay in limited resources, both in terms of money and time. On this view, it is not the RCTs as such that are considered problematic.

So either there are studies about treating the classic [acu-]points as opposed to the non-classic points, but then you have to be able to show a difference of between 70 and 50%, right (…) instead of say 70 and 30%. So we have certain methodological problems that shouldn’t of course deter us, but in order to be able to do it you do need to have some sort of funding hoo-hah—you need to have a certain number of patients, and that means having someone who’ll pay for the statistics and so on—(…) In other words, it takes…money, money, money, right—at any rate it’s not something I can do personally. Medical acupuncturist (2)

Value-based reservations

The second group of reasons for being sceptical about the use of RCTs in acupuncture relates to values. The view that RCTs involve a kind of disrespect for the very nature of acupuncture—forcing the treatment into a Western framework by, for example, using trials in which the needle is placed in exactly the same point in every patient.

I mean I do think it’s interesting that although there are people who’ve conducted strictly objective scientific studies (…) once in a while they’ve reached a significant result and that surprises me, because it’s actually against the Chinese system’s own way of thinking. Acupuncturist (3)

However, so long as acupuncture could be incorporated in a trial on its own terms, i.e. with individual treatments, the interviewees were not in general hostile to evaluation by RCT, although trials accommodating individualized treatment did not conform to their conception of biomedical research. More specifically, when the interviewees were asked to consider a pragmatic RCT design, they actually welcomed it.

Most interviewees distinguished very sharply between the proper use of acupuncture as a treatment of the whole person and its use in treating symptoms. Their reservations about RCTs related to proper acupuncture rather than to acupuncture as a means of treating symptoms: thus most of the interviewees agreed that acupuncture as a symptomatic treatment could be investigated by RCTs. However, it should be borne in mind that most of the interviewees first and foremost practise acupuncture proper, and it is this to which they were usually referring to when they spoke of ‘acupuncture’ in the interviews. Though they recognized its usefulness in treating pain, they regarded symptomatic treatment, in general, as less useful than acupuncture proper. By treating the symptoms alone, in their view, you do not deal with the underlying reason why the condition or illness emerged.

Other methods

Some of the interviewees suggested that other kinds of research method, such as those used in the social sciences and the humanities, would be more appropriate in investigating acupuncture.

You can’t measure it in a hard and fast way…you have to use methods that belong to the humanities—for example, qualitative methods and grounded theory, and whatever the hell else we’re using now…narrative theories are really, really, really good in many situations where we want to study the effects of traditional Chinese medicine (…) And I think it’s also a question of rejecting the view that RCT is always—or an observational approach to research is always best. Medical acupuncturist (3)

However, the respondents were not able to explain in more detail how this could be done.

Discussion

One of the most striking findings of the study is that, although many of the interviewees were sceptical about the use of RCTs, they were really opposed only to a certain kind of RCT—namely, the explanatory trial—which seeks to investigate the causal efficacy of a specific component of CAM (White 2002; Roland and Torgerson 1998; Godwin et al. 2003). Most interviewees would actually welcome, or at least not oppose, pragmatic RCTs, although they did not regard these as bona fide RCTs: they assumed that a RCT is necessarily double-blinded and explanatory.

Another interesting point to emerge from the study relates to the predominance of the experience-based conception of evidence in the MD group. A recent study (Tilburt et al. 2009) shows that the majority of internists and rheumatologists, in contrast with, for example, acupuncturists, rate RCTs as ‘very useful’ and patient preference as ‘least important’ in research interpretation. This indicates that the conventional practitioners are more positively disposed to RCTs than the complementary practitioners—a result that is confirmed in the ongoing debate between CAM proponents and opponents (e.g. Borgerson 2005; Tonelli and Callahan 2001; Vickers 2000).

A methodological issue concerning the criteria used for selecting the interviewees therefore arises. Members of the MD group differ from most MDs practising acupuncture in the sense that only one of them (alone, among all the interviewees) completely dismissed the Chinese explanation of the effect of acupuncture and believed that it could be explained in terms of conventional neurophysiology alone. Although a few others also expressed some hesitation about explaining the effects in traditional Chinese terms, the overall framework they used to diagnose, treat and explain acupuncture was traditionally Chinese.

This suggests that the division of the interviewees into groups based on educational background might be less important than, say, a division based on the participants’ orientation towards either traditional Chinese medicine or Western medicine. It also suggests that the study is representative in relation to neither acupuncturists nor CAM practitioners in general.

However, the study reveals a range of positions on the concept of evidence, and although the study is not exhaustive it seems reasonable to suppose that the positions it describes can be commonly found among traditional acupuncturists, at any rate.

The strength of the study lies primarily in the fact that the analysis it offers is based on in-depth interviews with acupuncturists who practise acupuncture on a regular basis, and in the fact that the interviews involved a range of practitioners occupying different positions within and outside the official healthcare system, both public and private. The perspective gained from this study can be used in the preparation of study designs that will facilitate dialogue between the conventional and alternative spheres.

Conclusions

In this article I have made it plausible to assume that acupuncturists operate with two different conceptions of evidence, one biomedical, and one experience-based. The first conception resembles the understanding of evidence found in conventional medicine, while the second is based either on personal experience or on the history of acupuncture. Although most of the interviewees embraced both conceptions, they differed in how much epistemological weight they gave to the different types of evidence. Furthermore, while most of the interviewees were sceptical about the use of RCTs in investigating the effects of acupuncture, their scepticism related only to one specific type of RCT.

My aim has been to explore practitioners’ conceptions of evidence, and thereby to shed light on an area of healthcare provision that is largely unexplored yet at the same time affects many people’s lives. Insight into the field may help clarify the place that CAM, more generally, should have in public healthcare. It may also go some way to explaining how the gap between the conventional and alternative realms can be bridged in the context of proper investigation of the effects of CAM.